Checking simulation memory requirements
The simulation and memory requirements utility provides an overview of the size and key properties of your simulation. It breaks down the amount of system memory (RAM) required to run a given simulation, the system memory required to collect all results from different MPI ranks, and the amount of disk space needed to save the results.
Automatic memory check feature in FDTD
When running your simulation job from the Finite Difference IDE (CAD/GUI) on your local computer, by default FDTD will check your simulation to verify if you have enough system memory to run the job. If there is not enough memory you will be prompted with a warning: "Warning: Insufficient Memory" and "There is not enough system memory to run the simulation". From here you can choose to try and run the simulation anyways, or to abort the run.
You can disable the automatic memory check either through the warning window or by opening "Check memory requirements", and unchecking "Auto check simulation memory".
Resolving insufficient memory issue
- Run the simulation job on a machine with a sufficient amount of RAM.
- Run distributed computing using multiple machines/nodes in your network/cluster
(note: MPI Rank 0 will need enough memory to collect the results from all the other nodes/ranks).
Simulation checkpoint
- Enable the simulation checkpoint feature in the "Advanced Options" of the FDTD Solver object.
- You can use the script commands, pause, resume, or resumejobs to stop/pause and resume your simulation job.
Note: When running the simulations from the command line, we provide the option of resuming the simulation job. For details, please refer to "Running simulations from the command line".
Using Intel MPI
Starting with the 2021 R2.1 release you can run your simulation using Intel MPI 2019 version as your "Custom" Job launching preset in the advanced resource configuration on supported RHEL/CentOS. This applies to FDTD and varFDTD using the 'fdtd-engine-impi-lcl' and 'varfdtd-engine-impi-lcl' executable.
- Install Intel MPI using yum on supported RHEL/CentOS:
sudo yum-config-manager --add-repo https://yum.repos.intel.com/mpi/setup/intel-mpi.repo sudo rpm --import https://yum.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB sudo yum -y install intel-mpi-rt-2019.9-304.x86_64
- Copy "libfabric.so.1" and "libmpi.so.12" to the Lumerical "lib" installation folder:
sudo cp /opt/intel/impi/2019.9.304/intel64/libfabric/lib/libfabric.so.1 /opt/lumerical/[[verpath]]/lib/ sudo cp /opt/intel/impi/2019.9.304/intel64/lib/release/libmpi.so.12 /opt/lumerical/[[verpath]]/lib/
- Set your Job launching preset to "Custom".
- Enter the Intel MPI executable path "/opt/intel/impi/2019.9.304/intel64/bin/mpirun" and the FDTD-engine Intel MPI executable "/opt/lumerical/v221/bin/fdtd-engine-impi-lcl" into their corresponding fields.
