This article is a guide to Speos Sensor System Exporter (SSS Exporter), a powerful solution for post-processing a camera (or sensor-based) simulation result. This article aims to provide a deeper understanding of the tool's development process by going through specific examples, emphasizing each development step.
Overview
The SSS Exporter is a powerful standalone tool designed to perform the post-processing of camera simulation results from Speos. You can find a general explanation of the tool behavior in the article Getting Started with Speos Sensor System Exporter and running your first example. For a more detailed understanding of the sensor characterization aspect, refer to Speos Sensor System Exporter - Sensor characterization with EMVA-1288 Standard and/or Lumerical.
One interesting aspect of the tool is the conversion from a RAW image to a developed final one. There are arguably as many ways to do this conversion as there are different sensor models, so the tool is built to stay as flexible as possible. It is also important to note that if you want to process an image without knowing the development method of the sensor manufacturer, it is better to stick to RAW image analysis.
Index
Here is the list of the examples highlighted within the article:
Experimenting with each example is effortless - simply navigate to the desired folder and click on the 'Launch Speos Sensor System Exporter.bat' file. This action will run the example, and the results generated will be conveniently stored in the 'Outputs' folder, utilizing data from the corresponding 'Inputs' folder.
Note that normally the batch file should not be modified: it is launching the executable of the SSS Exporter with the correct input file. If needed, you can find the executable inside the installation files of Speos (usually in "C:\Program Files\ANSYS Inc\v232\Optical Products\Viewers\SSSExporter.exe").
Note:
- If your local installation is not on the above-mentioned path, you need to adjust it in the batch file.
- If you see the error message "Impossible to connect to the Virtual Photometric Lab", you need to run the corresponding version of Virtual Photometric Lab as administrator once.
Development parameters
In the "Sensor.yaml" file, you can fill out the development parameters which are usually located at the end of the file.
By default, you can specify:
- The demosaicing method - You can change the method used to transform the “small resolution” images from each channel of the Bayer Matrix to a higher resolution image in color with RGB triplets. The SSS Exporter comes with 4 available methods for this step (bilinear, Malvar2004, Menon2007, DDFAPD).
- The linearization parameters - If needed, you can correct the non-linearity of the sensor by either specifying a look-up table for the digital numbers correction or polynomial coefficients in array format.
- The orientation of the final image - In case the orientation of the final image differs from the RAW image of the sensor, you can specify the new orientation by choosing between 8 different preset values.
- The color saturation correction - If you select this option an algorithm will run to compensate for pixel saturation that will cause color shifting.
- The colorimetry adjustment - This section enables you to shift from a scene illuminant to a standard color space by colorimetric conversion. You can fill in the shoot illuminant, the targeted color space, and the adaptation method.
You also have the possibility to plug in your own script for developing final images.
You can find more detailed information about how to fill out the sensor file here.
Examples
Demosaicing method
This dataset is used to try out various algorithms for color reconstruction of the images by the camera. Since most sensors use color filter arrays to get color information, we usually get 3 or more under sampled images (one from each color channel), that we want to use to reconstruct a high-resolution image with a color triplet on each pixel. Such algorithms are usually built to avoid color artifacts and not to be too complex in terms of computation.
In practice, images reconstructed by cameras have good results in “color-homogeneous” areas of the image, but they have less resolution and color artifacts on the edges even after the demosaicing. The Speos Sensor System Exporter has 4 different demosaicing algorithms integrated:
Once you have run this example, inside the "Outputs" folder, you get 4 different developed PNG images corresponding to each algorithm.
We can see the impact of each algorithm on the color artifacts on the edges, since these images have all been generated from the same sensor with a regular Bayer Matrix array.
Colorimetry
SSS Exporter enables to control shoot illuminants, target color space and adaptation method, providing imaging professionals with precise control over color reproduction. This is valuable in scenarios where color accuracy is critical, such as product photography. From these three inputs, two intermediate data - Camera Neutral Vector and Color Calibration Matrix – are automatically calculated and added into the sensor file.
- Shoot Illuminant - This key determines the main illuminant used for scene calculation with Speos. Users can select from predefined illuminants like A, B, C, D50, D55, D65, and D75. Alternatively, they can use a Speos spectrum file with a specific black body temperature spectrum. In this example, various shoot illuminant spectra have been tested. As demonstrated below, thanks to the inclusion of the 'Camera Neutral' vector and the 'Color Matrix,' the colorimetry of the developed image remains nearly independent of the shoot illuminant. This capability ensures that the final image's color accuracy remains consistent across different lighting conditions and illuminants, providing reliable and true-to-life color representation regardless of the shoot environment.
- Output Color Space - Users can define the target color space to which the final image will be converted. This helps achieve color accuracy and consistency in the output. Furthermore, SSS Exporter offers the flexibility to explore different development Color Spaces. Users can select from a wide range of supported color spaces, including sRGB, Adobe RGB (1998), Apple RGB, Best RGB, etc. Each color space has its unique characteristics and gamut, allowing users to tailor the final image's color representation to meet specific requirements and preferences.
When choosing a color space, it's essential to consider the intended use of the final image and the capabilities of the devices that will display or reproduce it.
- Adaptation Method - The adaptation method allows users to control how the sensor calculates the linear color transformation from the shoot illuminant to the target illuminant. This ensures that the image maintains its intended color appearance across various viewing environments. More info about the methodology can be found here.
Custom development scripts
The SSS Exporter allows you to plug in any custom development algorithm to convert a RAW image in electrons to a final RGB image. This is convenient for users that don’t use the previously mentioned parameters or techniques to generate their sensor images. To do so, you need to specify the python script file in the External script subkey:
Development:
Type: External Script
External Script: myscipt.py
In the dataset provided there are 3 different examples of custom scripts available:
- Basic LDR image - Custom development script that adjusts the colorimetry of the scene.
- Basic HDR example - Custom development script that generates an HDR image.
- Distance detection - Custom development script that detects objects distances from 2 shifted images (stereo camera).
Thanks to the SSS Exporter, the user can generate accurately any sensor output thanks to these multiple custom development capabilities. It is very useful for applications such as camera integration or detection algorithm training.