Running remote simulation jobs with MPI

This page shows the process of running simulations remotely on another computer from your local computer using Intel MPI with Lumerical 2021 R2 and newer. 

The following errors could occur when the MPI is not configured properly. 

 

• Install Intel MPI in the local computer,
• Open a Windows command prompt,
• Change directory to the location of Intel MPI for your version of Lumerical,

For Lumerical 2021 R1 and older:

cd "C:\Program Files (x86)\IntelSWToolsMPI\compilers_and_libraries_2018.5.274\windows\mpi\intel64\bin"

For Lumerical 2021 R2.2 and newer:

cd "C:\Program Files (x86)\IntelSWToolsMPI\compilers_and_libraries_2020.4.321\windows\mpi\intel64\bin"

• Register "User credentials" with Intel MPI on your local computer before running remote jobs with Intel MPI. These credentials are encrypted and stored in the system registry.
• The 'Username' and 'Password' should be the same as the ones you used to login to Windows.

mpiexec -register

Notes: More information on user authorization and different authentication methods are available here: User Authorization 30. If you do not register your credentials, you will be prompted later when you run Intel MPI for the first time and the simulation job will not run.

Hydra service

Check to ensure that the Hydra service is installed and running on all of the computers.

hydra_service -status

If the service is not running on any of the computers, install and run the service;

hydra_service -install

Validate the installation

mpiexec -validate

• Open Resources. 
• Select "Remote: Intel MPI" as the Job launching preset.
• Specify a "writable location" on the remote resource for the 'APPDATA' variable in the Resource advanced options for Intel MPI.
• Save your settings and run the simulation remotely.

# Example
-env APPDATA "C:\"

Running remote job on Linux

• Open Resources. 
• Select the installed MPI as the Job launching preset.
• Specify a "writable location" on the remote machine for the 'HOME' variable in the Resource advanced options for the MPI.
• Save your settings and run the simulation remotely.

-env HOME "/<writable_path>" 

/installpath/bin/mpiexec -n 32 -host node4 -env HOME ~/.ansys /opt/lumerical/[[verpath]]/bin/fdtd-engine-impi-lcl -t 1 /<path>/simulationfile.fsp

Where: (-n 32) is the number of MPI processes on 1 thread/core (-t 1) each process that is used to run the simulation job on the remote host.   

See also

Running simulations with MPI on Linux

Configuring resources for parallel jobs across several computers on Windows

Distributed computing  

Concurrent Parametric Computing