This article is a guide to the Speos Sensor System Exporter (SSS Exporter) applied to RGB-IR (Red, Green, Blue, Infrared) sensors, an emerging technology for visible and near-infrared imaging.
Overview
The SSS Exporter is a tool in Ansys Speos used for post-processing simulation results from a camera sensor or ODX import. It processes spectral irradiance or exposure map: outputs that accurately represent how a scene will be captured under real-world illumination conditions while considering the optical and sensor properties of the camera. By integrating sensor data according to the EMVA-1288 standard and applying advanced post-processing, the SSS Exporter generates detailed electronic maps, noise maps, raw images, and final/developed images representing realistic full-camera perception.
For further review of how to get started with the SSS Exporter, see the article Getting started with Speos Sensor System Exporter and running your first example. Additionally, for using the SSS Exporter with EMVA-1288 characterized sensors, see the article Speos Sensor System Exporter - Sensor characterization with EMVA-1288 Standard and/or Lumerical. This article will specifically discuss the SSS Exporter applied to RGB-IR sensor technology: how to modify the input and sensor YAML files, run the program, and obtain results. RGB-IR sensors are increasingly used in addition to traditional visible sensors such as RGGB (Red, Green, Green, Blue) sensors due to their ability to capture both visible and infrared light, enhancing image quality in low-light conditions and enabling advanced applications like depth sensing, night vision, and facial recognition. The infrared channel also improves material differentiation and allows for multi-spectral imaging, making RGBIR sensors valuable in agriculture, remote sensing, and medical imaging.
Speos side-by-side simulation with a 1500K IR source (left) and a 6500K visible source (right) with an IR transparent filter.
Below is the unprocessed exposure map (.xmp file) taken from the photometric camera simulation results, considering the camera's integration time. The exposure map includes spectral information from 400-1100nm, and the output shown below is the true-color exposure map.
True color exposure map from the photometric camera sensor
SSS Exporter post processing including sensor parameters of an image taken with a visual range source (left) and an IR source (right) with an IR filter
Index
Bayer Matrix
Standard Bayer RGGB 2x2 matrix (left) and Quad Pixel RGB-IR 4x4 matrix (right) are shown below. A standard Bayer array allows for color imaging by masking pixels on a camera sensor corresponding to red, green, and blue filters. Using this color filter mask allows color imaging by combining these smaller pixels on a macro scale to reproduce and image similar to an LED display.
The visual representation of the Bayer Matrix for RGBIR sensor is such that the cell values follow the ‘xy’ pattern, where x is the row value and y is the column value (see below for the modified YAML file pointing to the specific *.spectrum curves for each of the different pixel assignments). For example, pixel '12' corresponds to row 1 and column 2. The *.spectrum files are based on a sensor’s responsivity to light.
Note: To model the spectral response of the sensor, in the SSS Exporter Sensor YAML file, there are three types of spectral information: the Bayer matrix, quantum efficiency, and IR/UV filter, which are multiplied together. If the user does not have all individual spectral information for those 3 contributions but only the global factor, the recommendation is to set 100% transmission spectrum for both Quantum efficiency (QE) and UV-IR filter keys and set the global contribution for each pixel in the Bayer matrix field.
Development
Generating final images from the raw image can be achieved using the built-in postprocessing tools within SSS Exporter, especially when working with a 2x2 RGGB sensor array. However, external development is required for RGB-IR sensors, necessitating a specific script to cover the image processing aspect.
File management
The required files to run SSS Exporter include the batch script, input YAML file, sensor YAML file, exposure/irradiance map, and additional external files such as spectrum files, External Quantum Efficiency (EQE) specifications, etc.
The batch script allows the program to run directly from a directory on your computer. The input YAML files allow for ease of adjusting system parameters, such as pointing to the correct files to be processed and selecting data to export. The exposure/irradiance map generated from the Speos simulation can be stored in the SSS input folder and be used as an input to the analysis. Lastly, other external files, such as spectrum files or a postprocessing Python file, can be used for sensor information definition, and additional post-process imaging analysis can also be stored in the Input folder.
Example
Modify the Sensor YAML file and Run
As mentioned above, the SSS Exporter folder includes an Input YAML file from the RGB-IR System Parameters folder, which includes information on adjusting system parameters, such as pointing to the correct files to be processed and selecting data to export, as shown below.
This example provides a generic template file for sensors YAML file, which can be found in the “Inputs” folder.
- To proceed, let’s define RGBIR sensor Spectral information, including the IR and UV filter, Quantum efficiency, and the Bayer matrix, in the Sensor YAML file, as shown below.
Note: When applying a 4x4 matrix, the input .xmp map resolution must be a multiple of 4 vertically and horizontally
- Once the correct parameters are adjusted for the sensor used, navigate to the folder directory containing the batch file. Double click the batch file or right click and select “Run as administrator” to begin the SSS Exporter.
- Check the output raw image and developed image found below in the “Outputs” folder:
The output consists of two images: one from the visible spectrum and the other from the IR spectrum. In this example, the development.py file processes the raw image for the RGBIR sensor configuration. It extracts the red, green, blue, and IR channels by averaging specific pixel values, then combines the R, G, and B channels into a color-corrected, balanced image and does the same for the IR channel. Both images are normalized to a [0, 1] range, adjusted for gamma correction and white balance. Finally, the images are combined with a border to create the final output.
The development Python script can be found in the “Inputs” folder named “Development Quad Pixel RGB-IR.”
To go further
In this example, we used a simple development process. To explore additional capabilities, try some additional post-processing features with a unique Python script.
Additional Resources
You can find a general user guide for the SSS Exporter tool here: Speos Sensor System Exporter
To get started with the SSS Exporter tool, understand the parameters, and run an initial batch file, you can refer to Getting started with Speos Sensor System Exporter and running your first example
To gain a deeper understanding of the sensor characterization in the SSS Exporter, refer to Speos Sensor System Exporter—Sensor Characterization with EMVA and Lumerical.
If you are looking for more insights into the tool's development step, refer to Speos Sensor System Exporter—Development methods and capabilities.