This article is a guide to the Speos Camera sensor, a powerful solution for image generation based on the Reduce Order Model of the optical system. The aim of this article is, through a generic example, understand the Ansys Speos camera sensor, where to put the camera and see the light captured by the camera. The step-by-step process described here can be applicable to different use cases.
Overview
Understand the simulation workflow and key results
In today's rapidly evolving technological landscape, the demand for high-quality imaging systems is essential across various industries, from automotive to consumer electronics. Among the key components, Camera sensor, is an important element in modern consumer electronics and imaging devices. Understanding and validating the performance of camera sensor within a 3D scene is crucial for optimizing image quality and optimum field of view coverage. In this article, we introduce camera image generation using Ansys Speos, an advanced non sequential ray tracing optical simulation software by Ansys and explore its camera application in real-world like scenarios.
To simulate the behavior of a camera in Speos, you can create a simulation containing the complete optical system of the camera, or you can create a camera simulation with Reduced-Order Models (ROM) of lens systems. The complete lens system is typically chosen when you need to access and compare the optical system's performance comprehensively, including aspects like lens flare and stray light, but it typically requires longer simulation times due to considering all optical paths. However, simulating the camera with ROMs allows for quick access and comparison of optical system performance. ROMs achieve this efficiency by pre-calculating optical behaviors of the lens system once while focusing on the main sequence of the optical path.
In this article we focus on the camera simulation based on ROM, which facilitates rapid yet accurate camera simulations. By reproducing key real camera lens properties, it enables camera performance validation, generating photometric results in terms of the irradiance captured in front of sensor without compromising lens manufacturer intellectual property.
For further insights into Speos Camera simulation with ROM and generating Electonic map, raw image and final image it is recommended to explore the article of CMOS Sensor Camera - Image Quality Analysis in a 3D Scene.
The main tool needed for this virtual solution:
- Speos 24R2 to generate irradiance map in capture by camera sensor
- [optional] Zemax OpticStudio 24R1 and later versions to export lens model, or the back box model of the lens system as ROM, which can be used by Speos.
Speos Camera Sensor Definition
The Camera Sensor in Speos enables realistic ray integration like an actual camera using ROM of the lens system, based on definition parameters. Speos Camera sensor features two modes:
- Geometric mode, a simplified setup for basic camera sensor definition: In Geometric mode, you can define camera sensors without configuring every parameter, compatible with ROMs extracted from Critical Ray Angle (CRA) data sourced from datasheets (V1), eliminating the need for additional sources or sensors in simulation. You have the flexibility to adjust the following parameters:
-
- Position and orientation of camera
- Reduced-order model of objective lens system
- Sensor size and resolution
Note: No optical properties needed, Speos uses Rendering properties from CAD.
Application Scenario
: These adjustments provide users with great control over the camera setup in Geometric mode, to do quick FOV studies for packaging the camera, run very fast camera simulation while no optical properties of material in the scene are defined or are available.
- Photometric/Colorimetric mode, offering advanced parameter control. The Photometric/Colorimetric mode in Speos offers extensive control over all Camera Sensor parameters, including:
-
- Position and orientation of camera (the origin is positioned differently depending on the ROM version; with V1 ROM, the origin is at the entrance pupil position, while with V2 and Binary ROM, the origin is at the center of the imager)
- Reduced-order model of objective lens system
- Sensor size and resolution
- Acquisition information including integration time and lag time (optional- if timeline feature in simulation is enabled)
- Trajectory of the sensor (optional-if camera is in motion)
- Wavelength range for specifying the spectral propagation range
Notably, the Layer option in camera sensor definition allows for the organization of photometric results into separate layers based on sources and explore the image generated from each source individually.
Application Scenario: Ansys Speos’ photometric camera sensor capability enables accurate modeling of various camera types such as visible, infrared, or thermal cameras, within a 3D scene, replicating realistic illumination conditions and generating quantifiable irradiance results. By enabling timeline simulation and specifying camera integration and lag time, as well as the camera and/or objects’ trajectory in the scene, the illuminance/irradiance map can be converted into an exposure map. The Exposure map represents the power received by the sensor, which is basically multiplication of irradiance data by integration time with the unit of J/mm2 and can be used for motion blur and rolling shutter effects representation.
Optical Reduced-Order Model (ROM) of the Lens System
Generation of the ROM
To integrate ROM of lens designs from Zemax OpticStudio into Speos camera definitions and replicate the effects of a camera lens within a broader model environment, Speos employs files with a *.OPTDistortion extension. These files encapsulate ROM of lens system.
These ROMs can be derived from Critical Ray Angle (CRA) data sourced from datasheets (V1) or, for enhanced accuracy, directly extracted from lens systems designed in Zemax OpticStudio (Binary Version). Zemax OpticStudio offers a dedicated 'Export Reduced Order Model to Speos' tool accessible via the file menu, facilitating adjustments to various parameters as mentioned below:
- Sensor Width and Sensor Height: define the dimensions of the sensor. The default values are used if not specified.
- Skip System Reversal: When checked, this option skips the required reversal of the system during export. Useful for evaluating systems that do not reverse correctly with automated reversal.
- Use Radial Symmetry: For axially symmetric systems, radial symmetry enables computation. Unchecking this option reveals X and Y Sensor Sampling for asymmetric systems (including lens system and imagers).
- X and Y Sensor Sampling: Specifies the number of samples in the x and y directions at the sensor for non-axially symmetric systems.
- R Sensor Sampling: Determines the number of sample points in the radial direction in the image space to generate data. A general recommendation for this quantity varies depending on the distortion level of the lens system. It typically ranges from ~10 for lens systems with minimal distortion to as high as 50 for fisheye lens types, which exhibit significant distortion. The typical value is 30.
- Ray Sampling: Specifies the ray sampling grid used to compute parameters transferred to Speos. This parameter defines the ray sampling grid utilized for computing parameters transferred to Speos. Typically, the default value suffices for most scenarios. However, in cases of lenses with strong aberrations or when dealing with lens systems incorporating offsets and tilts in lenses, switching to 64x64 may prove beneficial. It's advisable to avoid values exceeding 128x128.
- Configuration: Selects a configuration from the dropdown menu for multi-config operands to modify the reversed system, with the selected configuration exported to Speos.
Note: When generating the ROM in Zemax OpticStudio, make sure to set up the wavelength range of interest. From the System Explorer in Zemax OpticStudio, set at least 3 wavelengths, arranged from small to large, for example for visible range set the wavelength range of 400nm to 700nm. The minimum and maximum wavelengths in Zemax OpticStudio determine the maximum wavelength range that can be simulated in Speos. Wavelengths cannot be extrapolated in Speos.
Speos interpolates intermediate wavelength samples based on the results of each wavelength calculation from Zemax OpticStudio. To avoid bias, using more wavelength samples in Zemax OpticStudio can be beneficial.
However, the calculation time for the generation of the *.OptDistortion file in Zemax OpticStudio is proportional to the number of samples. It is recommended to set 5 to 7 wavelength samples to balance accuracy and computation time.
Note: The wavelength weights within Zemax OpticStudio do not affect the export of ROM files.
The latest version of Lens ROM in a *.OPTDistortion file (Binary version) includes information ensures accurate management of FOV, Distortion, Depth of field, Resolution, Entrance pupil position, Vignetting, Dynamic (Rolling Shutter, Motion Blur), Chromatic Aberrations. As a result, simulations conducted using these ROM models show no significant difference in the results from those performed directly on CAD lens models, as long as only the main sequence is considered. It's worth mentioning that the Binary version ROM can be created using Zemax OpticStudio 2024 R1 and later versions.
The goal of using the *.OPTDistortion file into a Speos simulation is to:
- Perform fast simulations of the camera behavior (compared to simulating the complete camera system)
- Reproduce most of the real lens properties
- Perform simulation without exposing the manufacturer intellectual properties
Here is the list of capabilities supported by different version of the ROM:
Validation of the ROM-Binary version
To validate the performance of the Binary ROM, different lens system type including single bi-conic lens, cooke triplet, double gauss, even asphere, mobile phone lens, telescope have been tested. As an example, the ROM, as shown below the even sphere lens system has been validated by comparing ROM to the full lens system simulations both Zemax Opticstudio and in Speos. As shown below, the comparison between Speos ROM and Zemax OpticStudio full lens model MTF50 (Modulation Transfer Function at 50%) analysis which is a measure of sharpness, differs by less than 10%, while the ROM is 10 to 100 times faster.
In another test, using a lens system with a large aperture and low-quality bokeh, comparing ROM simulation to full lens system geometry in Speos, both simulation results showed consistent image quality with chromatic aberrations. In this simulation, set of punctual light sources which represents the different directions in the field of views imaged on an irradiance sensor: the camera simulation with ROM completed in about 6 sec minutes on RTX A5000 GPU cores, while simulation with full geometry took ~30 minutes on a similar GPU, which demonstrate the significant speed improvement.
The above figures show comparison between irradiance maps with the geometries of lens system and with the ROM.
Overall, the Camera Sensor in Speos offers a comprehensive suite of tools for precise camera simulation and analysis, ensuring chromatic aberration and faster image generation compared to the cases where full lens models are used across various applications.
To explore the camera performance in this article, we create a camera sensor based on the ROM previously generated from the Zemax OpticStudio model, which can be found in the "Input for ZOS" folder. In Speos 2024 R2, open the Speos simulation file "Machine_Vision_Camera_.scdocx" from the downloaded folder and follow these steps:
Example of Camera simulation and results in Speos
As mentioned above, the Geometric mode in Speos camera sensor offers a simplified version of Camera Sensor parameters, primarily used for default rendering properties during mesh generation for quick FOV studies such as the one shown in this article Parking assistance camera integration – Ansys Optics. On the other hand, the Photometric/Colorimetric mode provides extensive control over Camera Sensor parameters, including photometric definitions which can be used for image quality analysis in 3D scene. In this example, we will focus more on creating a static photometric camera simulation (without motion) and adjusting the position of the camera system.
Note: ROM binary version is fully spectral, and simulation can be done only with dispersion option enabled in Speos inverse simulation.
Static Camera simulation with Close-up view of the object
Below is the machine vision scene in Speos used in this article.
To proceed let’s define camera sensor parameters in Speos:
-
Define Origin:
- Click on the origin from the Design>Create tab as a standard practice to create camera sensors axis system. Place the origin at BardCode_origin (or anywhere else in the Scene).
- Right click on the origin Created in the Structure tree and rename it to Camera_origin.
- Click on the Camera_orign, select the move tool from the Design>edit menu. From the Option-Move window, select the Enter XYZ coordinate tool highlighted below. This option allows you to enter X, Y, and Z distances to move relative to the World Origin. When you select the option, the Move handle re-orients to be parallel to the World Origin and displays X, Y, and Z input panels. Using your keyboard, assign the following numbers to the X=0 mm, Y=135 & Z=0 mm in the 3D window.
- Drag and drop the Camera_Origin into the CAMERA component in structure tree
-
Define Camera Sensor:
- From the light Simulation > Sensor> system tab, select Camera Sensor.
- Double click on the Camera from simulation tab, a definition window is opened.
- From the Mode drop-down list, select Photometric,
- Rename your camera sensor to Photometric_Camera.
- Set Camera Axis system: Define the orientation of the camera sensor (Origin, Horizontal Direction, and Vertical Direction) by assigning the Camera_origin you created.
- Adjust Optics Parameters: Set values for the following optics parameters, which determine the objective performance. For Distortion browse for the *.optdistortion file in the input folder, which represents the ROM lens stack.
- Define Sensor & Sensitivity Parameters: Browse for sensor sensitivity parameters, defining the overall sensitivity of each channel. This includes the sensor pixel sensitivity and the colored filter array (Bayer filter).
- Adjust the Wavelength: fine-tune the spectral bandwidth and accuracy utilized in the simulation.
Speos enables visualization of camera elements (Camera field, Object field, Aperture) in the 3D window.
- Right click on the Photometric_Camera sensor from simulation tree, select options to adjust preview of the sensor using the Object field visualization radius. This is only for visualization purpose nothing to do with simulation.
- To reframe the 3D view camera and obtaining a sensor's point of view, click on the Automatic Framing icon
in the contextual menu of the sensor and explore what this sensor captures during a simulation.
It is recommended to identify an incorrect framing or test different fields of view before computing a simulation.
The sensor and the source are the minimum physical objects needed to perform a camera simulation. The remaining step is to define the simulation methodology itself. The Camera sensor in Speos works exclusively with Inverse Simulation. Rays are propagated in the reverse direction, starting at the sensor out into the 3D scene. Those that ultimately make it to a light source will contribute to the result.
- From Simulation tree, double-click on the already created inverse simulations “ Camera_Sim1” and “Camera_Sim2” to include the camera sensor. The tool guide icons include items for selecting and verifying Geometries, sources, and sensors in the main GUI window. Select the Sensor icon, select the Photometric_Camera sensor from your simulation tree, validate your selection by the green check mark for both simulations.
You can modify the simulation stop condition such as duration or number of passes.
- To run the simulation, right click the Camera_Sim1 simulation and select Compute for CPU compute or GPU compute. Upon completion of the simulation, the computed files will appear below the simulation tree which includes the following for each camera sensor:
- HTML file
- XMP file (Radiance)
- XMP file (irradiance)
- PNG image file
- HDR file
Note: As previously noted, if you are using a ROM with a binary version in the camera definition, you need to run the inverse simulation considering dispersion. To enable dispersion, user needs to right-click on the inverse simulation (e.g., Camera_sim1) in the simulation tree, then select 'Options' > 'Inverse Simulation' and check 'Dispersion'.
Note: In your camera sensor definition, the following parameters affect the irradiance map:
Other parameters are dedicated to HDR and PNG creation. It is strongly recommended to use the irradiance results, as some assumptions are made specifically for generating the PNG and HDR results, especially if you are performing lens characterization.
If you are interested in generating an electronic map, raw image, or final processed image, it is recommended to use the Ansys Speos Sensor System exporter. Further information can be found here: Getting started with Speos Sensor System Exporter and running your first example – Ansys Optics.
The irradiance.xmp results can be exported as txt from file menu in xmp viewer to be used for additional post-processing of the image.
As mentioned above the output of interest photometric camera simulation with the ROM from the Ansys Zemax OpticStudio design is the Irradiance map, which is visible below the Simulation after simulation computation is completed:
- Double click on the *.Irradiance.xmp results and adjust the level value to 500 using the level icon.
The yellowish color in the irradiance results is due to the selected wavelength of interest in the camera definition. The irradiance map in front of the sensor allows us to assess whether the camera was correctly positioned. In this example, the barcode on the object appears a little blurry, and the lighting is not uniform in this region of interest. Since changes to the design phase, such as altering the camera lens system or the lighting setup, are not desirable in this example, we can adjust the camera's position to find a better placement and capture a more detailed image.
Note: To explore camera Exposure results and considering the trajectory of the camera/objects motion in the scene, integration time, and lag time, user can enable the Timeline feature in the inverse simulation.
Re-Position the Camera Sensor
As it appears from the last step’ simulation results, the rudder positioned within the scene is partially blurred from camera perception, making it hard for the machine vision camera to detect the barcode on the rudder. The solution is to reposition the camera component which includes the light ring as well to make sure the image is more visible. By adjusting its position accordingly, the sensor will have the capability to capture the ROI better.
To do so, you can follow these steps:
- Click on the CAMERA component in the structure tree, that includes the origin you made at Step 2.a (Camera_Origin)
- Click on the Move handle from the Assembly>Edit tab, to move the camera component in the scene 250 mm in vertical direction away from the Conveyor rubber.
With the Move tool, you can translate/rotate objects/axis system in the scene by clicking and dragging on the axis/ curved rotational axis that is aligned with the direction you want to move/rotate the selected /axis system and dragging. As mentioned above, there is also the option that allows you to enter X, Y, and Z distances to move relative to the world Origin.
- Right click on the Camera_Sim2 simulation from simulation tree and select compute (for CPU) or GPU compute. After computation, double click on the irradiance.xmp result to open it and adjust the level value to 100 using the level icon.
By comparing the results of Camera_Sim1 and Camera_Sim2 for the two camera component locations, you can clearly see that the barcode is more readable, more uniformly illuminated, and less distorted in the distance view. If you are interested, you can check the false color (instead of true color) in the xmp results and explore the irradiance values at the pixel level.
In Summary, the Speos Camera Sensor allows users to quickly generate the amount of light captured by camera from the 3D scene considering key camera parameters. Users can evaluate the performance of various sensor types (Visible, SWIR, MWIR, LWIR, Thermal etc.) across different industries. Additionally, users can explore camera dynamics, including roller shutter and motion blur effects when the camera or objects in the scene are moving, as well as the flickering effect caused by light source modulation as perceived by the camera.
Taking the Model Further
Understand the simulation workflow and key results
As mentioned before, in this example, we explored static camera simulation. To consider motion in camera simulation, user can explore dynamic camera simulation and include scene dynamics and camera motion.
To learn more about dynamic inverse simulation which allows a Camera sensor to capture dynamic scenes and include camera integration time and lag time with related effects such as rolling shutter, motion blur, or LED flickering check Dynamic Inverse Simulation (ansys.com).