This article is a guide to the Speos Sensor System Exporter, a powerful solution for post-processing Speos camera (or sensor-based) simulation result. This article presents a collection of practical examples to demonstrate the tool's capability in generating electron maps and raw images from the Reduced Order Model of the sensor, based on EMVA 1288 Standard (release 3.1). It aims to provide a deeper understanding of the Speos Sensor System Exporter's core functionalities and its applications in camera simulations.
The Speos Sensor System Exporter (SSS Exporter) is a powerful tool designed for the post-processing of camera simulation results from Speos. The tool transforms the Spectral Exposure/Irradiance map which represents illuminance/irradiance values reaching the camera sensor, into a raw image, an electron map, and a developed image. The transformation from photometric result to electron map and raw image are based on a Reduced Order Model of the sensor, adhering to the EMVA 1288 standard (release 3.1). EMVA 1288 is an industry-standard specification that characterizes image sensor performance for machine vision and imaging applications. It provides standardized tests for key sensor parameters like responsivity, dynamic range, noise, dark current and quantum efficiency.
To use the SSS Exporter, and generate a raw image and electron map, users need to have a Speos-generated Spectral Exposure map, or Irradiance map, and EMVA 1288 Standard sensor data. The tool then automatically processes this data to create valuable insights about camera sensor acquisition. To generate the electron map, the SSS Exporter utilizes External Quantum Efficiency (EQE) data of the imager, which can be obtained from Lumerical or the imager's datasheet. The article aims to showcase the extensive range of features and possibilities offered by the SSS Exporter tool. Each feature is demonstrated using practical examples, organized into folders and subfolders for easy execution. The tool doesn't have a graphical user interface, so understanding file management is crucial for proper usage. For further insights into SSS Exporter, it is recommended to explore the article of Getting started with Speos Sensor System Exporter and running your first example and CMOS Sensor Camera - Image Quality Analysis in a 3D Scene.
To assist you in finding the desired example quickly, we have provided a list below with direct links to each example within the article:
- Integration time vs System Gain
- Temporal Dark Noise
- PRNU & DSNU
- Thermal Noise
- Statistical Noise
- Lumerical Coupling
Experimenting with each example is effortless - simply navigate to the desired folder and click on the 'Launch Speos Sensor System Exporter.bat' file. This action will run the example, and the results generated will be conveniently stored in the 'Outputs' folder, utilizing data from the corresponding 'Inputs' folder.
Note that normally the batch file should not be modified: it is launching the executable of the SSS Exporter with the correct input file. If needed, you can find the executable inside the installation files of Speos (usually in "C:\Program Files\ANSYS Inc\v232\Optical Products\Viewers\SSSExporter.exe").
- If your local installation is not on the above-mentioned path, you need to adjust it in the batch file.
- If you see the error message "Impossible to connect to the Virtual Photometric Lab", you need to run the corresponding version of Virtual Photometric Lab as administrator once.
In the sensor.yaml file, you can fill out the sensor parameters which are usually located at the beginning of the file.
By default, you can specify:
- The Operating Conditions, which contain information about conditions of usage for the sensor including Gain, Offset, Exposure Time, and Temperature.
- The Properties information about general sensor properties including Resolution, Pixel Size, and Bits depth.
- The EMVA data, which describes the sensor performance in relation to the EMVA 1288 model standard. This includes IR & UV Filter, Quantum Efficiency, Bayer Matrix, System Gain, Temporal Dark Noise, AST, DR, Dark Current and Spatial Non-Uniformity.
Note: Quantum efficiency can be an input from simulation, such as data calculated with Ansys Lumerical that has been exported to a *.json file. In this case, IR & UV Filter, Quantum Efficiency, and Bayer Matrix sub-keys in the EMVA data key are ignored.
Integration time vs System Gain
In various imaging scenarios, achieving a well-balanced exposure is crucial for capturing high-quality images. Whether in low-light conditions, scenes with high contrast, or situations requiring fine details, finding the right balance between sensor Gain and Exposure time is vital. This balance can help to achieve optimal image quality, reduced noise, and accurate representation of the scene's dynamic range.
By simulating different conditions and scenarios, SSS Exporter enables users to analyze the impact of sensor Gain and Exposure Time on image quality, both with and without temporal dark noise with these Keys:
- Operating Conditions
- Exposure Time
- Temporal Dark Noise
Sensor Gain amplifies the captured signal, brightening the image, but it may also amplify noise, degrading image quality. Similarly, longer Exposure Time allows more light to reach the sensor, resulting in brighter images. However, longer exposures can lead to motion blur in dynamic scenes, necessitating a careful balance with the appropriate Gain setting.
Temporal Dark Noise
Temporal Dark noise is characterized by variations in a sensor's output signal when no light is present, often resulting from electronic noise and temperature fluctuations. Understanding the effect of Temporal Dark noise is crucial for evaluating a sensor's performance and suitability for various applications. SSS Exporter enables the user to explore the impact of Temporal Dark noise in sensor outputs under different exposure conditions with the corresponding key:
- Temporal Dark Noise
By experimenting with the sensor Temporal Dark noise key in the sensor YAML file, you can study the noise behavior in the sensor's output. The recorded video showcases developed image results for different temporal dark noise levels. These results have been automatically generated using specific parameters, such as the Repeat option for each set, Add Text option on developed images, and Video generation from images, as specified in the Inputs.yaml file.
Note: The video has been automatically generated using the parameters specified in the Inputs.yaml file.
- Repeat option: The specified parameters have been applied to each set, ensuring repeat of the same frame throughout the video.
- Add Text Option: Developed images are enhanced with text overlays, providing additional context or information to the viewers.
Video Generation: The resulting video combines the images, creating a dynamic visual representation of the data.
With automated video creation, users can process the datasets, saving time and effort while exploring the effect of a variable such as Temporal Dark noise to predict and comprehend how sensors behave, thereby facilitating the testing of imaging systems.
PRNU and DSNU
In the context of image sensor characterization, most of the parameters discussed so far have focused on individual pixels or their average behavior. However, it's essential to recognize that the parameters of an array of pixels can vary from one pixel to another.
For linear sensors, two fundamental nonuniformities play a crucial role in image quality: Photo-Response Non-Uniformity (PRNU) and Dark Signal Non-Uniformity (DSNU). These variations can introduce non-uniformities in pixel sensitivity and dark signal levels, leading to unwanted noise and artifacts in the images. SSS Exporter provides the capability to experiment and calibrate sensor characteristics like PRNU and DSNU. By incorporating the EMVA Data and relevant sub-keys into its simulations, SSS Exporter allows users to explore the effects of these non-uniformities on image quality.
Spatial Non Uniformity:
Photo-Response Non-Uniformity (PRNU): PRNU which refers to the sensitivity variation across pixels of an image sensor can cause slight differences in pixel response to the same incident light intensity. As shown in this example, the calibration of PRNU can have an impact on correcting these sensitivity variations and resulting in more accurate and consistent pixel responses.
Dark Signal Non-Uniformity (DSNU): DSNU accounts for the variation in dark signal levels among individual pixels in an image sensor when no light is incident on the sensor. Calibrating DSNU as shown below has been useful for eliminating unwanted dark signal variations. This calibration can help in ensuring the accuracy and reliability of image data, particularly in challenging lighting scenarios.
SSS Exporter offers the capability to explore Photo-Response Non-Uniformity and Dark Signal Non-Uniformity, enhancing image quality for different imaging applications.
Understanding and quantifying thermal noise is crucial for camera sensor characterization and performance evaluation. This noise arises due to the thermal energy present in the sensor's components, leading to random fluctuations in signal levels. This intrinsic noise phenomenon in sensors, in low-light or high-gain scenarios, can significantly impact image quality, leading to reduced signal-to-noise ratio (SNR) and affecting the overall image clarity. Managing and minimizing thermal noise is essential in applications where high sensitivity and precision are required.
SSS Exporter provides capabilities to experiment with sensor thermal noise as per the EMVA data. By incorporating thermal noise models into the simulation, SSS Exporter allows users to assess and evaluate sensor thermal noise performance under different operating conditions, including varying temperatures in operation condition using corresponding keys:
- Td sub-key corresponds to the temperature interval that doubles the dark current
- Tref sub-key corresponds to the value for Reference Temperature
Temperature key corresponds to the camera temperature when used
- Temperature key corresponds to the camera temperature when used
Note: How to generate a video using the parameters specified in the Inputs.yaml file is explain in Temporal Dark Noise example.
This example was built to highlight the photon noise from the SSS Exporter. It was integrated inside the tool to give more accurate results on homogeneous areas from Speos map, since it’s rare that a real sensor measures the exact same number of photons on multiple pixels. The starting map is an exposure map that only contains homogeneous areas.
The output is the raw photon map in XMP format. We can see that the areas are not homogeneous anymore.
This is the effect of the “statistical noise”. It is proportional to the square root of the number of photons, hence, there is no noise in the top-left area of the sensor, since it contains no photons.
SSS Exporter was originally built to generate sensor images based on the EMVA1288 standard, built to characterize sensor images. However, this standard makes one main assumption about the sensor, which is that the quantum efficiency is wavelength-dependent only and is homogeneous through the sensor area. This might not be true for many sensors. Indeed, the quantum efficiency of the pixels usually depends on the light incidence which is not the same on each pixel of the sensor. Therefore, the real quantum efficiency can be seen as a function of the wavelength and the light incidence (or the position on the pixel matrix).
Such an advanced quantum efficiency can be computed thanks to Ansys Lumerical. . You can use it to design the pixel matrix accurately and export the quantum efficiency in a JSON file. More information about this procedure can be found in this CMOS Sensor Camera- Sensor Characterization article.
The SSS Exporter will read this JSON file as input, and it will be used to replace the EMVA1288 standard inputs.
There are two available modes for the Lumerical files:
Mode 1: QE as a function of incidence
This mode is limited to a lens system with revolution axis. Inside the mode 1 JSON file the QE is defined as a 3D matrix: incidence vs wavelength vs Bayer matrix.
Mode 2: QE as a function of position on the sensor
This mode can be used with any lens system. Inside the mode 2 JSON file the QE is defined as a 3D matrix: pixel position vs wavelength vs Bayer matrix.
In the sensor .yaml file, the Lumerical input is specified under the “Lumerical data” key. You can specify 2 subkeys: “Mode” (V1 or V2) and “Filename” (name of the Lumerical JSON file).
This dataset contains different examples:
Lumerical Mode 1 examples
- 2 monochromatic variations
- 2 basic color variations
Lumerical Mode 2 example
- Random color variation
With these examples we can see that the QE simulation from Lumerical can give more advanced and accurate results.
In summary, the article introduces Speos Sensor System Exporter capability for post-processing the camera simulation results in Speos based on the Camera Sensor feature. This article showcases practical examples that create electron maps and raw images based on the sensor's Reduced Order Model, aligned with EMVA 1288 Standard covering integration time, noise, coupling, etc., with generated results stored in specific folders. If you are looking for more insights on the development step of the tool, you can refer to Speos Sensor System Exporter - Development methods and capabilities.