Running simulations using terminal on Linux

This article provide details on running your simulation using the Linux command line or terminal.

Using the design environment (CAD/GUI)

• FDTD

fdtd-solutions [options]

• MODE

mode-solutions [options]

• CHARGE, HEAT, DGTD and FEEM

device [options]

• INTERCONNECT

interconnect [options]

Options

filename

• optional, the filename Opens the specified simulation or project file.

-v

• optional, Outputs the product version number.

scriptFile.lsf 

• optional, the script filename Opens the specified script file. 

-safe-mode

• optional, Turn on the safe mode.

-trust-script

• optional, Turn off the safe mode.

-run <scriptfile> 
../[[verpath]]/fdtd-solutions.exe -run <scriptfile> <simulationfile>

• optional, Run the specified script file.
• if a simulation file is required, this is added after the script file. 

-nw  
-hide

• optional, (-nw for FDTD only), -hide for other solvers.
  
• This hides the CAD window from appearing on the Desktop. 

-use-solve
-run script.lsf -use-solve 

• optional, used to run scripts in non-interactive engine mode.
• added after the -run and script file options.
• this will run the script based on the configuration in the CAD.
• see this page for details.

-logall

• optional, Generates a log file for each running process of the simulation job.

-exit

• optional, Exit application after running the script file.

-o

• Change the location that log files are saved to.
• All log files will be saved to the relative or absolute directory passed to -o.
• If the directory ends with .log this will be treated as a file name.
  Useful when running INTERCONNECT with the -logall option.

Examples

Opening FDTD with a specific simulation project

$ /opt/lumerical/[[verpath]]/bin/fdtd-solutions simulationfile.fsp

Running a script with a simulation file while 'hiding' the CAD window and disabling safe mode.

$ /opt/lumerical/[[verpath]]/bin/fdtd-solutions -nw -trust-script -run scriptfile.lsf simulationfile.fsp

Run simulations without MPI 

• FDTD

fdtd-engine [options]
rcwa-engine [options]

• FDE

fd-engine [options]

• EME

eme-engine [options]

• varFDTD

varfdtd-engine [options]

• CHARGE

device-engine [options]

• HEAT

thermal-engine [options]

• DGTD

dgtd-engine [options]

• FEEM

feem-engine [options]

• MQW

mqw-engine [options]

• INTERCONNECT

interconnect [options]

Options

filename

• required, The name of the simulation or project file to run.

-t

• optional, Controls the number of threads used.  If not used or left blank, it will use 1 thread/processor.

-v

• optional, Outputs the product version number.

-fullinfo

• optional, It will print more detailed time benchmarking information to the log file based on walltime and cpu time measurements.

-log-stdout 

• optional, Redirects the log file data to the standard output, rather than saving it to file.
• This option will be ignored when the simulation runs in graphical mode.

-mesh-only

• optional, Mesh the geometry without running the simulation.

-inmaterialfile <file>

• optional, Load simulation mesh data from a <file>.

-outmaterialfile <file>

• optional, Save simulation mesh data to <file> for use on another project.

-logall

• optional, Create a log file for each simulation or sweep.
• Logfiles are named filename_p0.log, filename_p1.log, filename_p2.log
• By default, only filename_p0.log is created.

-mr 

• optional, Print a simple memory usage report for a given simulation file to the standard output.  Output can be piped or saved as a text file.*

-o

• optional, Change the location that log files are saved to.
• All log files will be saved to the relative or absolute directory passed to -o.
• If the directory ends with .log the last section will be treated as a file name.

-resume

• optional, available for FDTD simulations only.
• Resumes the simulation from the last checkpoint. 
• If no checkpoint is found it will start the simulation job from the beginning. 
• Enable the simulation checkpoint feature in the "Advanced Options" of the FDTD Solver object. 

Examples

Running on the local computer with the -resume flag when checkpoint is enabled in FDTD.

/opt/lumerical/[[verpath]]/bin/fdtd-engine -t 8 -resume /path/simulationfile.fsp 

Run with 4 threads and save the log file into a different path.

/opt/lumerical/[[verpath]]/bin/fdtd-engine -t 4 $HOME/temp/filename.fsp -o "~/Documents/logfiles/"

Run an EME simulation using 2 threads.

/opt/lumerical/[[verpath]]/bin/eme-engine  -t 2 -logall $HOME/temp/example.lms

Running simulations with MPI

Notes

• As of 2024 R1, MPICH2 is no longer packaged with the Lumerical installation package on Linux.
• For distributed solve capability, install and run using either Open MPI or Intel MPI.
• See this KB, Running simulations with MPI on Linux for more information. 

Using MPI to run the simulation job with the solver is done for the following used cases:

1. Run 1 simulation file on the local computer (localhost). 
2. Run several simulations at the same time on different machines or nodes. (Concurrent computing)
3. Using several machines to run 1 single simulation to take advantage of their memory (RAM) as required by the simulation. (Distributed computing)
4. Launch and simulate on a local machine to a remote machine or node.

MPI is a complex application with many configuration options and versions.  On Linux OS, Lumerical supports a wide variety of MPI versions.

Stopping your simulation (Quit and Save)

To stop your simulation job, similar to 'Quit and save' on the CAD.

• When on the active terminal window where you simulation job is running,

  # use the keyboard keys
  CTRL + C 

• If the engine process is running in the background, find the process ID <PID> and kill the process,

  # using pgrep to show the list of PID for "fdtd-engine"
  pgrep fdtd-engine 
  
  # from the list kill 1 of the PID
  kill <PID>

Pipe standard output to a text file

• The standard output does not appear in the terminal window. In order to see the report you can simply pipe the output to a text file using the piping command ">".
• For example, to output the engine version number or memory usage report to a file, use the following syntax.

/opt/lumerical/[[verpath]]/bin/device-engine -v > $HOME/temp/version.txt

/opt/lumerical/[[verpath]]/bin/dgtd-engine -mr $HOME/temp/example.ldev > $HOME/temp/example_mem_usage.txt

See also

Running simulations with MPI on Linux

Resource configuration elements and controls 

Compute resource configuration use cases

Getting started with HPC