Guide - How to setup and use the Physical Camera Sensor – Ansys Optics

Physical Camera_Overview.png

Stray light analysis of an optical system and image quality assessment require consideration of the lens geometries and the optomechanical components that constrain them. Due to a high number of ray-object interactions inside the camera system, simulations using all camera geometries take more time or require more rays to achieve the same level of signal compared to simulations using a Speos Camera sensor based on a Reduced Order Model (ROM). The Speos Camera sensor approximates the camera system using a Reduced Order Model, considering only the chief rays (main optical path sequence). In contrast, simulations that include the geometries of the complete camera system allow for the assessment and analysis of stray light from secondary optical paths (higher-order sequences) caused by unintended sources of light reaching the sensor.

Overview

The purpose of the Physical Camera sensor is to accelerate the simulation of the entire camera system within a 3D scene by introducing an aiming area on the first optical surface and accounting for stray light contributions from predetermined ray path sequences based on overall or peak energy. The Physical Camera sensor enables the analysis and separation of these sequences in the result.

Working principle:

Phyiscal Camera Overview.png

The Physical Camera sensor is compatible with direct and inverse simulations.

Direct simulations are recommended for scenarios with individual or "non-ambient" light sources (Ray-file, Luminaire, Surface, Display), to get ghost effects, for instance.
Inverse simulations are recommended for scenarios with ambient light sources (without sun). Since 1 pass of an inverse corresponds to one ray/pixel, per sequence, per wavelength, it is recommended to focus on the main optical path (first sequence) to get the ambient contribution.
Results from direct and inverse simulations can be combined in post-processing.

The Physical Camera sensor can be operated in two modes: either without a sequence file (Mode 1) or with a sequence file included in the sensor (Mode 2), depending on the use-case.

Mode 1 - Without a Sequence File: In this mode, Speos uses a standard Monte-Carlo ray propagation engine to simulate a 3D scene and the optical system. It considers specular and scattering interactions within the optical and optomechanical system. To increase the probability of rays entering the optical system's aperture for a direct simulation, an aiming area (bounding box) is introduced at the first face of the optical system to direct the rays towards it. This approach is also compatible with wide FOV lenses (e.g., "Fish-Eye lenses") since rays entering from wide angles are also efficiently traced. Speos performs importance sampling towards the aiming area on all scattering surfaces. The power of the aimed rays is then rebalanced to ensure unbiased results.

In scenarios where the optical system contains many elements, ray convergence can still take a long time, especially for sequences other than the main optical path.

Setup for Mode 2: During the first simulation with the Physical Camera sensor, an *.OptSequence file is generated in the “Speos output” directory. This file contains a list of ray path sequences sorted by energy. It can be used in a subsequent simulation with the Speos Physical Camera sensor to allow faster result convergence using a predefined number of sequences (Mode 2).

Note: The Physical Camera sensor is compatible with direct and inverse simulations. The content of the *.OptSequence file depends on the simulation type. An *.OptSequence file generated with a direct simulation can only be used as input in a direct simulation and vice versa.

Mode 2 - With a Sequence File: The Physical Camera sensor, including a sequence file, takes advantage of the aiming area and performs optimized ray propagation inside the optical system on the most energetic ray path sequences (included in the sequence file) with a deterministic raytracing algorithm considering only specular interactions (transmission and reflection). To achieve high convergence for the secondary sequences, the same number of rays is launched for each individual sequence. The number of sequences can be specified in the definition panel of the Physical Camera sensor. This enables quick visualization of ghost reflections (lens flare), caused by specular reflections on the optically polished lens faces, even when high-resolution irradiance (image) sensors are used. By changing the active "Layer" in the result, the contribution of each sequence can be displayed and separately evaluated.

Note: Sequences including non-specular interactions are changed to specular interactions. This might lead to incorrect results.

To illustrate the performance gain of using a Physical Camera sensor over a conventional simulation of the complete camera system with an irradiance sensor, a test chart image was simulated, and the image quality of the first three sequences was compared. Speed Comparsion.png

The Physical Camera results demonstrate a significant performance improvement (~x100) compared to a conventional simulation, closely matching the image quality in Sequence 1, with noticeable gains in reduced noise and detail in Sequences 2 and 3 (especially with Mode 2). This highlights the effectiveness of the Physical Camera sensor in reducing simulation time while maintaining high image quality for a predefined number of sequences.
The Physical Camera sensor is suitable for camera applications where considerations regarding lens flare and stray light are crucial, as these factors can negatively impact the optical performance and final image quality. Furthermore, it can drastically reduce simulation time for use cases involving a camera system with a transparent lens cover.

Guide - How to set up a Physical Camera sensor?

In the following example, we are going through the workflow to set up and run simulations, including a Physical Camera sensor.

Prerequisites:

Optional: Import the lens system from Ansys Zemax OpticStudio via .odx file exchange to Speos (Described here in Step 3a).
Definition of the reference axis system for the Light Box and Physical Camera sensor.

Step 1: Setup of the Physical Camera sensor

Generation of a Light Box (containing all geometries related to the camera system, lenses, and, if available, optomechanical parts).
Definition of the Physical Camera sensor.

Step 2: Stray light simulation without a sequence file (Mode 1).

Step 3: Stray light simulation with a sequence file (Mode 2).

Before proceeding, it is highly recommended to read this article: "Stray Light Analysis - Smartphone Camera"

[[NOTES:]] Software Prerequisites

To follow this example, ensure the following tools are installed on your computer:

Ansys Speos 2024 R1 SP2 or Ansys Speos 2024R2.

Data: The dataset can be downloaded here: "How_to_setup_Physical_Camera_Sensor_start_data.zip"

Note: The "Start" dataset can be used to follow the step by step instructions. The folder includes a .html link to download the "End" dataset including all simulation results.

How to set up and use the Physical Camera sensor

Prerequisites (not required for the example dataset):

Optional: Import the lens system from Ansys Zemax OpticStudio via .odx file exchange to Speos (Described here in Step 3a). The optomechanical components can be imported from external CAD tools or designed natively in Speos, thanks to its direct modeling capabilities.
Definition of the reference axis system for the Light Box and Physical Camera sensor:
1. The origin of the reference axis system for the Light Box and Physical Camera sensor needs to be placed with a small offset (in the -z direction) in front of the first lens face. To avoid any tangency issues, an offset of 10x Geometrical Distance Tolerance (GDT) is recommended.
2. The axis system of the Light Box must be defined so the optical axis of the lens system follows the Z axis.

Step 1: Setup of the Physical Camera sensor

Generation of a Light Box:

In the first step, we will prepare the export of the Speos Light Box (containing the lens stack (.odx) and the optomechanics), which will be used as input for the Physical Camera sensor. The Speos Light Box component is a meshed representation of the geometries including their material properties. Running consecutive simulations with a Speos Light Box will save initialization time because the geometry included doesn’t need to be remeshed before the simulation.

Furthermore, the Speos Light Box can be "black-boxed" and shared with camera integrator or OEM.

Note: The mesh settings of the .odx component included in the Light Box must be defined within the options of the .odx component itself. They are accessible by right-clicking > Options.

odx meshing options.png

ODX mesh settings cannot be redefined in the meshing settings of the Light Box component. After applying any changes to the meshing, the Light Box must be recomputed.

Note: ODX is a file container that includes the optical elements' design and associated material information. Therefore, it needs to be selected from the Speos "Simulation panel."

Tip for easier selection: You can hide the "Optical Design Exchange.1" component from the structure panel, then select all geometries in the 3D view and add the "Optical Design Exchange.1" component from the simulation panel while pressing Ctrl on the keyboard.

Please follow these 5 steps to export the Speos Light Box of the complete camera system: LB Export Workflow.png

Start Ansys Speos and open the “Physical Camera Example.scdocx” project (located in “\Speos\”).
From the Light Simulation tab, click Light Box > Export Speos Light Box.
Hide the “Optical Design Exchange.1” component in the structure panel (since these geometries need to be selected from the simulation panel).
In the 3D view, select all optomechanical and lens edge geometries. Then press Ctrl and select the “Optical Design Exchange.1” from the Simulation panel.
Rename the Light Box to "Camera Light Box Export". Select the PhysCam axis system. Verify that all 17 geometries (16x Optomechanical + 1x Optical Design Exchange.1 component) are added to the “Geometries” tab.
Compute the Light Box. 7
After computation, a file "Camera Light Box Export.SPEOSLightBox" will be generated in the "Speos Output" folder.

Note: Files generated from a Speos Light Box Export with the “black box” option enabled are not compatible with the Physical Camera sensor.

Definition of the Physical Camera sensor

Note: To enable the Physical Camera sensor in Speos, you need to "Enable Beta features" in File -> Speos Options -> "Advanced" section.

To create a Physical Camera sensor:

Navigate to "Light Simulation -> System -> Physical Camera sensor".
Define the sensor with parameters as shown below:
Use the same axis system as for the Light Box ("PhysCam").
Include the exported Light Box "Camera Light Box Export" from the Speos output folder and select the irradiance sensor "Imager Layer Seq" predefined in the project.
To perform a simulation without a sequence file and to generate the sequence file that could be used in consecutive simulations, the irradiance sensor of the Physical Camera sensor needs to be set to data-separated Layer: “Sequences” and define sequences per “Faces.”
Change the preview display mode from Bounding Box to Facets. This allows verification of the correct positioning of the Physical Camera.
Compute the "Physical Camera".
The visual representation (yellow transparent facets) should be aligned with the camera geometries in the 3D view.

Step 2: Stray light simulation without a sequence file (Mode 1).

Using the physical camera sensor in Mode 1, "without a sequence file," you benefit from the efficiency of the aiming area and faster result convergence due to a high number of integrated rays in the optical system compared to a conventional simulation without the Physical Camera sensor.

We are going to simulate two different scenarios:

Direct sun source within the camera FOV (with an inclination angle of 20 degrees)
Display source with a test chart image located 1.5 meters in front of the camera

Scenario 1: Direct sun source within the camera FOV (with an inclination angle of 20 degrees)

Define the simulation with parameters as shown below:

Simulation Definition.png

Create a direct simulation and rename it to “SLA_Sun20deg_Mode1.”
In the Simulation Options, set the Geometrical Distance Tolerance (GDT) to 0.001 mm (to respect the recommended 10x GDT distance to the reference axis system).
Add the sun source at 20 degrees (3rd surface source) to the simulation.
Add the Physical Camera sensor. Add one sensor to the simulation.
Set LXP=True and limit the LXP max path to 5e4 rays (to avoid large .lxp files, but still have enough rays to display for a path analysis with all sequences).
Activate LXP for the Physical Camera.1 sensor (in the Sensors tab).
Set the number of rays to 5e8.
Run the simulation on the CPU (simulation time should be less than 4 minutes utilizing 16 cores).
Open the “SLA_Sun20deg_Mode1.Physical Camera.1.lpf” result from the simulation panel and set the scale level to 2000 lx.
Click on the measures icon and maximize the measurement area to cover the complete sensor.
Change the active layer to discover the ray path sequences from sequences 1 to 20. By default, the layer separation in the viewer is set to “All layers,” which means that no sequence filter is applied, and all sequences are displayed. The sequences are sorted by energy integrated by the sensor. As the energy decreases, adapt the scale level for higher sequences.
The results show a high convergence for the first sequence, which represents the main optical path of the optical system. This results in a bright purple spot caused by the sun source focusing on the image plane. The higher-order sequences show lower convergence. This is due to the Monte Carlo ray tracing algorithm, where the main optical path contributes the most to the result. The main optical path contains 93.9% of the overall integrated energy, while all other remaining optical sequences (stray light) contain only 6.1%. (Tip: This information can be retrieved from the Sequence Detection viewer).

Scenario 2: Display source with a test chart located 1.5 m in front of the camera

Copy and paste the direct simulation “SLA_Sun20deg_Mode1” and rename it to “SLA_Testchart_Mode1.”
Replace the “sun” surface source with the “Display_Testchart” source.
Run the simulation on the CPU (simulation time should be less than 10 minutes utilizing 16 cores).
Open the “SLA_Testchart_Mode1.Physical Camera.1.lpf” result from the simulation panel and set the scale level to 2 lx (Tip: Adjust the scale level for sequences with a lower energy level).

Again, the results show a high convergence for the first sequence, which represents the main optical path.

To increase the ray convergence for the higher-order sequences, we will now add the *.OptSequence files from the first simulation to the Physical Camera sensor (Mode 2) and rerun the simulations by considering only the 20 most energetic ray path sequences

Step 3: Stray light simulation with a sequence file (Mode 2)

Scenario 1: Direct Sun Source within the Camera FOV (with an Inclination Angle of 20 Degrees)

Copy and paste the direct simulation “SLA_Sun20deg_Mode1”, and rename it to “SLA_Sun20deg_Mode2”.
Add the “SLA_Sun20deg_Mode1.Physical Camera.1.OptSequence” file located in the "Speos output file" directory to the definition of the Physical Camera sensor.
Run the simulation on CPU (Simulation time should be <6min utilizing 16 cores)
Open the “SLA_Sun20deg_Mode2.Physical Camera.1.lpf” result from the simulation panel and set the scale level to 2000 lx.
Click on the measures icon and maximize the measurement area to cover the complete sensor.
Change the active layer to evaluate the sequences 1 to 20.

You’ll notice that all sequences exhibit higher convergence, which allows for stray light ghost analysis in the higher-order sequences.

Scenario 2: Display Source with a Test Chart Located 1.5m in Front of the Camera

Copy and paste the direct simulation “SLA_Testchart_Mode1” and rename it to “SLA_Testchart_Mode2”.
Add the “SLA_Testchart_Mode1.Physical Camera.1.OptSequence” file located in the "Speos output file" directory to the definition of the Physical camera sensor.
Run the simulation on the CPU (simulation time should be less than 30 minutes utilizing 16 cores).
Open the “SLA_Testchart_Mode2.Physical Camera.1.lpf” result from the simulation panel and set the scale level to 2 lx.
Click on the measures icon and maximize the measurement area to cover the complete sensor.
Change the active layer to evaluate sequences 1 to 20.

Final stray light analysis results using the Physical Camera sensor (for the first 10 sequences):

Known Limitations in Speos 2024R2:

The Physical camera sensor is only compatible with CPU.
Only on-axis refractive optical systems are supported.
Using a sequence file in the Physical Camera sensor will change all non-specular interactions to specular interactions.
Files generated from a Speos Light Box Export with the black box option enabled are not compatible with the Physical Camera sensor.
The Physical Camera sensor acts as a sensor that has no impact on the propagation. It will not generate any shadow. It allows obtaining results from the Physical Camera sensor, but it does not impact the light propagation for the rest of the system.
If the Light Box of the Physical Camera sensor includes an absorbing face located after the irradiance sensor, rays will not be effectively absorbed by this surface. Rays are integrated in the irradiance sensor and ray propagation is stopped.

Additional Resources

Project data:

Project (start data)
- How to setup and use the Physical Camera sensor -w/o results
Project (end data)
- How to setup and use the Physical Camera sensor with results (.html download link is provided in the "start data" folder)

Recommended Speos Camera learning path including content from Ansys Learning Hub (ALH):

More information about smart phone lens design and stray light analysis in Ansys Zemax OpticStudio can be found in:

Guide - How to setup and use the Physical Camera Sensor