Running CAD jobs on headless Linux systems

When running a job that requires a CAD instance (eg. Lumerical Script, Python API, or LumOpt), you need to have a graphics environment (even if it is hidden).

On clusters, you will have to run scripted workflows on a node with a desktop or graphical environment (GUI interface). Then modify the resources to distribute the workload or submit jobs to the nodes.

The guidelines shown in this KB is for running CAD jobs; Lumerical (.lsf) or Python API (.py) scripts on headless Linux systems. 

• Resource configuration elements and controls
• Running simulations using terminal in Linux 

Notes

• When running a simulation using the script, the resource configuration settings in the 'Lumerical product preference.ini' file will be used to run the simulation called by the script.
• Otherwise, if the 'Product preference.ini' file is not found, it will try to run the simulation using all the avialable cores on the machine. 

Running scripts

Starting with Lumerical 2023 R1, you can run Lumerical scripts (.lsf) from Terminal using 'fdtd-solutions-batch'. This will run the script while "hiding" the CAD/GUI. 

/opt/lumerical/[[verpath]]/bin/fdtd-solutions-batch {script_file.lsf}

When running Python API scripts (.py), set the environment variable "QT_QPA_PLATFORM" to "offscreen". And set the solver to 'hide=true'  in your Python (.py) script. 

#on RHEL/CentOS Linux 
export QT_QPA_PLATFORM=offscreen 
python3 {api_script_file.py} 

#lumapi settings
import lumapi
fdtd = lumapi.FDTD(filename="simulation.fsp",hide=True)

Using virtual display (Xvfb)

For Lumerical 2022 R2 and older releases use the process shown below using Xvfb.

1. Install Xvfb and tmux (consult your operating systems documentation for the install process).
   eg. (installation on RHEL/CentOS)
   

   $ sudo yum install xorg-x11-server-Xvfb tmux

2. Open a "tmux" session and a virtual display by:

   tmux 
   Xvfb :## &
   export DISPLAY=:##

   e.g.
   

   tmux
   Xvfb :99 & 
   export DISPLAY=:99

3. Run your Lumerical script with Xvfb:

   xvfb-run --auto-servernum /opt/lumerical/[[verpath]]/bin/fdtd-solutions -nw -run your_script.lsf
   xvfb-run --auto-servernum python lum_script.py

Notes

• You only need to run the "Xvfb :## &" command once on your "tmux" session. Unless you want to use a different display number.
• You can check the Xvfb display used by:

  ps -ef | grep Xvfb

• You should get the output of the display numbers used for Xvfb.

  username 1639 1 0 12:00 ? 00:00:00 Xvfb :99
  username 1732 1 0 12:03 ? 00:00:00 Xvfb :10
  username 1796 1748 0 12:09 pts/2 00:00:00 grep --color=auto Xvfb

• From the above, we have Xvfb using display :99 and :10. 
• Run the export DISPLAY command every time you enter into a "tmux" session.

  export DISPLAY=:##

See also

Python API overview 

How to run Lumerical API (Python) scripts in Linux

Lumerical scripting >> Running Simulations >> Resource Configuration - Ansys Optics