This page shows the process of running simulations in parallel on the local computer or remotely on another computer. This can be done from the solver's CAD or from the command prompt in Windows or the terminal in Linux
Requisites
- Set up your network or computers for parallel computing and configure passwordless sign in on Linux. See this article for details. The shared folder/network drive should have the same mapping for all PCs that will be running simulations.
- Register your login credentials with Intel MPI on Windows. See this KB page for details.
- A solve license for each computer that is running a simulation up to a maximum of 32 cores/threads.
- License sharing will be used when running concurrent simulation on the local computer (localhost) using up to a total of 32 cores/threads on the localhost per solve license. See this KB page for details.
Run simulations concurrently from the CAD
- Open the Lumerical CAD for your solver.
- Add the simulation file into the job queue using the 'addjob' script command.
- You can add as many simulation files for the current solver to the job queue.
- Configure your resources to run 1 or more simulations by setting the number of processes, threads, and capacity for each "host" in the resource configuration. To run the simulation remotely, simply enter the "hostname or IP address" of the remote machine.
- You can run simulations on your local host and/or remote machines depending on your resource configuration.
- Run the job queue using the 'runjobs' script command.
- To view the results or do your post, open the simulation file individually on the CAD.
Run simulations concurrently from the command line
Windows
- Open a command prompt on your PC and run the simulation on your computer.
"C:\Program Files\Microsoft MPI\Bin\mpiexec.exe" -n 12 "C:\Program Files\Lumerical\[[verpath]]\bin\fdtd-engine-msmpi.exe" -t 1 "\\network_path\filename-1.fsp"
- To run another simulation on a different Windows computer, open another command prompt and run the simulation using 'Intel MPI' to run remote jobs on a different Windows machine.
"C:\Program Files (x86)\IntelSWToolsMPI\mpi\2019.10.321\intel64\bin\mpiexec.exe" -env APPDATA "writable\folder\" -n 32 -host remotehost1 "C:\Program Files\Lumerical\[[verpath]]\bin\fdtd-engine-impi.exe" -t 1 "\\network_path\filename-2.fsp"
- To run more simulations at the same time, open as many command prompts as you want to run simulations and run the job on your local computer or a remote machine using the appropriate commands to run the simulation file remotely.
TIP: Create a script file for this process similar to below. i.e. Run 3 simulations on the local computer and 2 other Windows PCs on your network.
start "Run on local PC" "C:\Program Files\Microsoft MPI\Bin\mpiexec.exe" -n 12 "C:\Program Files\Lumerical\[[verpath]]\bin\fdtd-engine-msmpi.exe" -t 1 "\\network_path\filename-1.fsp"
start "Run on computer 2" "C:\Program Files (x86)\IntelSWToolsMPI\mpi\2019.10.321\intel64\bin\mpiexec.exe" -env APPDATA "writable\folder\" -n 32 -host remotehost2 "C:\Program Files\Lumerical\[[verpath]]\bin\fdtd-engine-impi.exe" -t 1 "\\network_path\filename-2.fsp"
start "Run on computer 3" "C:\Program Files (x86)\IntelSWToolsMPI\mpi\2019.10.321\intel64\bin\mpiexec.exe" -n 16 -env APPDATA "writable\folder\" -host remotehost3 "C:\Program Files\Lumerical\[[verpath]]\bin\fdtd-engine-impi.exe" -t 1 "\\network_path\filename-3.fsp"
Linux
- Open the Terminal and run your simulations.
- Ensure that you append the "&" to the end of your command line.
- Otherwise, open another terminal and run the other simulations from a different terminal.
#Running on the localhost
/opt/lumerical/[[verpath]]/mpich2/nemesis/bin/mpiexec -n 4 /opt/lumerical/[[verpath]]/bin/fdtd-engine-mpich2nem -t 1 /sharedpath/first_simulationfile.fsp &
/opt/lumerical/[[verpath]]/mpich2/nemesis/bin/mpiexec -n 6 /opt/lumerical/[[verpath]]/bin/fdtd-engine-mpich2nem -t 1 /sharedpath/2nd_simulationfile.fsp &
#Running remote to another host
/opt/lumerical/[[verpath]]/mpich2/nemesis/bin/mpiexec -n 12 -hosts remote_node1 /opt/lumerical/[[verpath]]/bin/fdtd-engine-mpich2nem -t 1 /sharedpath/3rd_simulationfile.fsp &
/opt/lumerical/[[verpath]]/mpich2/nemesis/bin/mpiexec -n 32 -hosts remote_node2 /opt/lumerical/[[verpath]]/bin/fdtd-engine-mpich2nem -t 1 /sharedpath/nth_simulationfile.fsp &