Lumerical job scheduler integration (Kubernetes)

Overview

This guide covers how to integrate the Lumerical CAD Job Manager with a Kubernetes cluster running the Kubeflow MPI Operator. After following these steps, running a simulation on the cluster will be just as seamless as running it on your local computer. You will be able to click "Run" and the job will automatically be submitted as an MPIJob, the project file will be uploaded to shared storage, the Lumerical Job Manager will periodically update with the simulation's current state and progress, and upon completion the results will be downloaded and loaded in your current CAD session.

See the article Lumerical job scheduler integration (Slurm, Torque, LSF, SGE) for the equivalent guide covering traditional HPC schedulers.

Known Limitations

• The job manager options Quit & Don't Save and Quit & Save are currently not supported. This applies both to single simulations and parameter sweeps that have a Kubernetes resource.
• Concurrent submissions of project files with the same basename will collide. The scheduler derives the MPIJob name from the project file basename, and re-submitting deletes any prior job with the same name.
• A long-lived helper pod named lumerical-file-copier is left running in the target namespace to support concurrent file transfers. It can be removed at any time with python kubernetes_kubectl.py --cleanup.

Requirements

• Ansys Lumerical 2026 R1.3 (or newer)
• A Kubernetes cluster prepared for Lumerical jobs — see Preparing a Kubernetes cluster for Lumerical jobs for the steps your cluster administrator must complete.
• kubectl 1.27 or newer installed on the client computer and on the system PATH
• A kubeconfig file with rights to create, list, exec into, and delete MPIJob, Pod, and use kubectl cp in the target namespace

• The following pieces of information from your cluster administrator:

  Item	Example
  Path to a kubeconfig file	~/.kube/config-lumerical.yaml
  Namespace name	lumerical-jobs
  PVC name	lumerical-simfiles
  Container image reference	registry.example.com/rocky-9-openmpi-ssh:latest
  License server	1055@license.example.com

Verify your kubeconfig works before continuing

kubectl --kubeconfig <path-to-kubeconfig> get nodes
kubectl --kubeconfig <path-to-kubeconfig> get pvc -n <namespace>

Resource Configuration

To configure a Kubernetes cluster resource in the Lumerical Job Manager, follow these steps:

1. Open the resource configuration window and add a new resource.
2. Select Kubernetes as the job scheduler preset. This populates the submission command with python kubernetes_kubectl.py and adjusts the submission script template.
3. Set the number of processes per node to the total MPI rank count you want for the simulation. This value reaches the cluster via the {NUM_PROC} macro, which expands into the mpiexec -n flag in the submission script. The worker pod count is not taken from this dialog — it is set by k8s_num_workers in job_scheduler_input.json and represents a cluster-level setting rather than a per-job choice.
4. Press OK to confirm the resource.

Resource configuration for the cluster is now complete, and you can proceed with simulation.

Notes on advanced settings for Kubernetes

• The Kubernetes preset does not require SSH or SCP. File upload, job submission, and result download are all performed by kubernetes_kubectl.py using kubectl.
• The submission script must reference the project file using the {PROJECT_FILE_PATH} macro. The scheduler script rewrites this path to the in-pod mount location (/home/lumerical/<filename>) before submitting the MPIJob.
• The engine executable embedded in the submission script (e.g. fdtd-engine-ompi-lcl, device-engine-ompi-lcl, interconnect-engine-ompi-lcl) determines which solver runs in the container. The image must contain the corresponding binary on PATH.
• The {NUM_PROC} macro becomes the -n flag passed to mpiexec. Set it to the total MPI rank count across all worker pods.
• Of the macros available in the resource configuration window, only {PROJECT_FILE_PATH} and {NUM_PROC} are consumed by the Kubernetes scheduler. {NUM_NODES} and {NUM_THREADS} are accepted by the UI but ignored — the worker pod count is taken from k8s_num_workers in job_scheduler_input.json, and per-process threading is controlled by the engine's own command-line flags.

Submitting jobs to your job scheduler from your local computer

To submit jobs from your local computer to a Kubernetes cluster, configure Lumerical on your local computer by editing the file job_scheduler_input.json. The configuration with a JSON file was introduced with the Lumerical 2023 R2.2 release.

JSON file template

The template (job_scheduler_input.json) can be found in the Lumerical installation folder:

• Windows: (default install path)

  C:\Program Files\Lumerical\[[verpath]]\scripts\job_schedulers

• Linux: (default install path)

  /opt/lumerical/[[verpath]]/scripts/job_schedulers

Contents of job_scheduler_input.json

{
    "user_name": "",
    "use_ssh": 0,
    "use_scp": 0,
    "cluster_cwd": "",
    "master_node_ip": "",
    "ssh_key": "",
    "path_translation": ["", ""],

    "k8s_namespace": "lumerical-jobs",
    "k8s_num_workers": 2,
    "k8s_image": "registry.example.com/rocky-9-openmpi-ssh:latest",
    "k8s_storage_pvc": "lumerical-simfiles",
    "k8s_kubeconfig": "~/.kube/config",
    "k8s_license_server": "1055@license.example.com",
    "k8s_kubectl_path": "kubectl"
}

Using the JSON file

You can use the JSON file by copying it to your user's "home folder" and editing it. The home directories are shown below. Lumerical products will automatically read from it when launching a cluster job.

Note: Lumerical products look for the exact file name. Do not rename the JSON file after moving it.

Linux: 

~/.config/Lumerical

Windows: 

%APPDATA%\Lumerical

Fields of the JSON file

The table below explains each Kubernetes-specific field. The first block (user_name, use_ssh, use_scp, cluster_cwd, master_node_ip, ssh_key, path_translation) is unused by the Kubernetes scheduler and may be left empty or set to zero.

Important

The scheduler will:

1. Ensure a long-lived helper pod (lumerical-file-copier) exists in the namespace and is mounted to the PVC.
2. Upload your project file to the PVC via kubectl cp.
3. Submit the simulation as an MPIJob via kubectl apply.
4. Poll the job's status and stream the launcher's logs back to the CAD Job Manager.
5. Download all result files matching the project basename back to the original local directory once the job completes.

Note for windows users: kubectl.exe must be on the system PATH, or its full path must be set in k8s_kubectl_path. Windows-style local paths (C:\Users\...) are handled by the scheduler — no path translation is needed.

Results

You can now run any Lumerical simulation (single, sweeps, optimizations, etc.) directly from the CAD's Job Manager. While the simulation is running, you can monitor its status from the cluster side using standard kubectl commands:

# List active jobs
kubectl get mpijob -n lumerical-jobs

# Stream the launcher logs
kubectl logs -n lumerical-jobs -f \
  -l training.kubeflow.org/job-name=<job-name>,training.kubeflow.org/job-role=launcher

# Manually cancel a job
kubectl delete mpijob <job-name> -n lumerical-jobs

See Also

Preparing a Kubernetes cluster for Lumerical jobs, Lumerical job scheduler integration (Slurm, Torque, LSF, SGE), Job scheduler submission scripts (SGE, Slurm, Torque), Resource configuration elements and controls, Ansys optics solve, accelerator, and Ansys HPC license consumption