There are two approaches to running a simulation job on the available cores in a computer. These are Multi-processes; where several MPI processes are used to run the simulation job, and Multi-threading: a single process is used to run the simulation job using multiple cores/threads on a computer. Click on the Resources button in the “File” tab of the CAD to open the resource configuration window.
Note: Starting with 2023 R2, FDTD supports running on GPU. See the Knowledge Base article Getting started with running FDTD on GPU for more information on how to set the Resource Configuration window to use GPU on FDTD. If GPU is used for the solver, the contents of this article on multi-threading and design environment will only apply to all computation other than the solver itself (e.g. meshing, scripts).
Multi-processes
Either Microsoft MPI and Intel MPI is used on Windows, and MPICH2, Intel MPI, and OpenMPI may be used on Linux.
- In FDTD, varFDTD and EME (2022 R2), the processes field is enabled and set according to the desired number of processes to run the simulation.
- While keeping threads to 1, each MPI process will utilize 1 core/thread on the computer.
- The number of MPI processes should not exceed the maximum number of cores/threads on the computer.
- Each MPI process can be locked to a specific core/thread when running the simulation with the pinning option for the MPI.
Example: On a 32 core computer where Processes=32, Threads=1, Capacity=1. This will run the simulation using 32 MPI processes where each process will utilize 1 core/thread.
Multi-threading
- On the other Lumerical simulation tool, FDE, HEAT, CHARGE, FEEM, DGTD, and when running with ‘Local Computer’ as the ‘Job launching preset’, the processes field is disabled.
- Set 'Threads' to the number of cores/threads you want to use to run the simulation.
- The number of threads should not exceed the maximum number of cores/threads on the computer.
- Since the job is running on a single process, it is not bound to a specific core/thread. Rather the operating system will automatically distribute the job across multiple cores/threads on the computer as per the number of 'threads' in the resource configuration.
- CPU usage typically shows as a percent(%) utilization based on the total number of cores/threads on the computer.
Example: On a 16-core machine where Threads=8, Capacity=1. Run the simulation job using 8 threads on the computer, utilizing 50% of the entire number of logical cores/threads.
Setting Capacity on your resources
- This is used to run parameter/optimization sweep on the local machine or remote computers.
- Setting 'Capacity' to more than 1, allows running sweeps concurrently.
Example: On a 48 core machine, Processes=6, Threads=1, and Capacity=6. This runs the 6 simulations concurrently using 6 MPI processes per sweep on a machine.
Configuration test
The 'Run Test' button is used to confirm if the resources are setup correctly.
Notes
This is supported using Local Computer and the bundled MPI variants, Microsoft MPI, Intel MPI on Windows and MPICH2 on Linux. "Custom" job launching presets, e.g. distributed computing, job scheduler integration or using a different MPI implementation, will not support this option and will "Fail" the test even if you are able to run the simulation job using your "custom" settings.
Advanced resource configuration
The advanced resource configuration for a specific resource is accessed by the 'Edit' button.
- The Job launching preset is changed from the options on the drop-down menu as shown here.
- Additional MPI and solver options can be specified here such as more detailed logs, MPI tuning, as well as other options.
Design environment configuration
- Many calculations in the design environments are multi-threaded (e.g. fft's).
- By default, the software will attempt to use all available cores.
- This can be overridden by disabling "auto detect" and setting the number of threads.
- Occasionally it may be desirable to restrict the number of cores, such as when other applications are running on the computer.
See also
Compute resource configuration use cases
Lumerical solve, accelerator, and Ansys HPC license consumption