The Resource Configuration window is the main window to configure how local and remote resources are used to run simulations in Ansys Lumerical Products. You can open the Resource Configuration window using the “Resources” button in the “File” tab of the CAD.
This article describes the general elements of the Resource Configuration window. For examples on how to set up resources for different use cases, see the Compute resource configuration use cases Knowledge Base article.
Note: Starting with 2023 R2, FDTD supports running on GPU. See the Knowledge Base article Getting started with running FDTD on GPU for more information on how to set the Resource Configuration window to use GPU on FDTD. If GPU is used for the solver, the contents of this article on multi-threading and design environment will only apply to all computation other than the solver (e.g. meshing and scripts).
Setting resources
The resources for each available solver in the product are separate and is controlled independently using the tabs near the top of the Resource Configuration window.
You can add, remove, and duplicate resources in the “Resources” table using the buttons on the side of the Resource Configuration window.
When running the simulation, the resource used depends on the product you are using and the type of simulation being ran:
- When running parametric sweep and optimizations, all active resources are used for all products. In the case of Ansys Lumerical FDTD™, only GPU resources are used for GPU sweeps and optimizations, and only CPU resources are used for CPU sweeps and optimizations.
- For single simulations:
- In Ansys Lumerical FDTD™, the resource selected in the “Run Simulation” group is used to run the simulation.
- In Ansys Lumerical MODE™, Ansys Lumerical Multiphysics™, and Ansys Lumerical INTERCONNECT™, the first active resource in the Resource Configuration window is used.
Running a multi-processes simulation
One method to run a simulation job is by using several MPI processes simultaneously. The “Processes” column controls the number of MPI processes for the set resource.
This is enabled only for the FDTD solver in Ansys Lumerical FDTD™, the EME solver in Ansys Lumerical MODE™, and the varFDTD solver in Ansys Lumerical MODE™. For all other solvers, the processes column is disabled. When an MPI is not utilized, for example, when the resource type is set to “local computer”, the process column is also disabled.
Microsoft MPI and Intel MPI are supported on Windows. MPICH2 (Up until 2024 R1), Intel MPI, and OpenMPI are supported on Linux.
In this case, other columns have the following meaning:
- The threads column sets the amount of core/thread each MPI process uses.
- The “Total Cores” column shows the total number cores used. You can calculate this number using
processes * threads. Ensure this value does not exceed the number of cores on the computer. - Each MPI process can be locked to a specific core/thread when running the simulation with the pinning option for the MPI.
Example: Setting up the resource for the FDTD solver on a 32-core computer where Processes=32, Threads=1, Capacity=1. This will run the simulation using 32 MPI processes where each process will utilize 1 core/thread.
Running a multi-thread simulation
Note: To set up multi-thread simulations in Ansys Lumerical INTERCONNECT, see this Knowledge Base article.
Another method to run a simulation job is by using one process but multiple cores/threads on a computer.
Except for the Ansys Lumerical solvers mentioned in the section above, all other solvers in Ansys Lumerical products can only use one process. Therefore, it is not possible to distribute simulations across several machines, but concurrent simulations for parametric sweeps and optimizations are possible.
In this case, you can set the number of threads you want to use for the simulation using the “Threads” column. Ensure that the number of threads do not exceed the maximum number of cores/threads on the computer. For the RCWA solver, see the RCWA Resource Configuration Knowledge base article for further details of how threads are used for calculation.
Since the job is running on a single process, it is not bound to a specific core/thread. Rather, the operating system will automatically distribute the job across multiple cores/threads on the computer as per the number of threads set in the configuration.
Note: The CPU usage on the computer typically shows as a percentage utilization, based on the total number of cores/threads on that computer.
Example: Setting up the resources for the CHARGE solver on a 16-core machine where Threads=8, Capacity=1. Run the simulation job using 8 threads on the computer, utilizing 50% of the entire number of logical cores/threads.
In addition to setting threads within the Resource Configuration window, you can also set threads from the command line (Windows/Linux), using the argument -t.
Setting capacity for parametric sweeps
For parametric sweeps, the “Capacity” column sets how many concurrent simulations can run on each resource.
Example: Setting up resources for the FDTD solver on a 48-core machine. In this case, Processes=6, Threads=1, and Capacity=6. This runs up to 6 simulations concurrently using 6 MPI processes per simulation.
Note: The total number of cores shown in the table is applies only to single simulations and does not account for the capacity column.
Configuration test
The 'Run Test' button is used to confirm if the resources are setup correctly.
Note: This is supported using Local Computer and the bundled MPI variants, Microsoft MPI, Intel MPI on Windows and MPICH2 (Up until 2024 R1) on Linux. "Custom" job launching presets, e.g. distributed computing, job scheduler integration or using a different MPI implementation, will not support this option and will "Fail" the test even if you are able to run the simulation job using your "custom" settings.
License estimation
The license estimation tab beside the configuration test tab displays the estimated number of license (standard or enterprise) required to run simulations. See the license estimation section of the Knowledge Base article on license consumption for more information.
Resource license settings
The resource license settings section near the bottom of the Resource Configuration window toggles the persist checkout feature. For more information on this setting, see the Persistent license checkout Knowledge Base article.
Resource advanced options
The advanced resource configuration for a specific resource is accessed by the 'Edit' button.
- The Job launching preset is changed from the options on the drop-down menu as shown here.
- Additional MPI and solver options can be specified here such as more detailed logs, MPI tuning, as well as other options.
Design environment configuration
Many calculations run multithreaded in the design environments, such as FFTs. By default, the software will attempt to use all available threads for these calculations. However, you can override the default setting by using the options in this tab and restrict the number of threads, such as when other applications are already running on the computer.
Options in this tab also affect some solvers that run directly in the design environment, such as mode analysis within the FDE solver in Ansys Lumerical MODE, and RCWA when “run within design environment” is selected.
You can set the maximum number of threads in the Lumerical preference file for FDTD. To set it, you can set a design_max_threads variable under the [jobmanager] section.
See also
Compute resource configuration use cases
Lumerical solve, accelerator, and Ansys HPC license consumption