CMOS Sensor Camera - Sensor Characterization

In this example, we introduce a CMOS image sensor (CIS) simulation workflow that involves 3D electrical and broadband optical simulations.  We also consider both the azimuthal and polar angles of the incident light, allowing EQE extraction for further simulation in Speos Sensor System (SSS) Exporter as part of the camera system simulation workflow described here . Overall, this example provides a more realistic and versatile demonstration compared to the previous simplified example .

Note: The workflow described here is only compatible with SSS Exporter version 2023 R2 (and later). For the previous version of this workflow please contact support .

download example

Overview

Understand the simulation workflow and key results

The operation of a CIS is based on the photo-generation of charge carriers by light absorption in a silicon substrate, followed by collection of those carriers for signal analysis. Pixels designed to detect light of different colors are arranged in a Bayer matrix that is periodically repeated to cover the entire sensor.

In this example, some key performance metrics of a CIS, such as internal quantum efficiency (IQE) and external quantum efficiency (EQE) are calculated as a function of incident angles, wavelength and pixel type. The EQE describes the performance of the sensor to convert incident photons to collected charge carriers. See this example for the definition of the various efficiencies.

We also show how the EQE can be evaluated at selected sensor positions, using chief ray and marginal ray data from a ray tracing simulation. This data can then be incorporated into the camera system simulation workflow , which includes the macroscopic lens module and the signal processing that leads to intermediate and final images of a realistic 3D scene as captured by the camera.

Step 1: Lens System Design With Zemax OpticStudio – (Not Covered in this Article)

In this step, we design the camera lens system in Zemax OpticStudio and extract the marginal and chief ray angles (CRA and MRA), which define the illumination cone at different positions on the sensor plane. The lens design itself is not covered in this example but we explain how to use a Python automation script to calculate these angles and save the data in a JSON ray file that can be used by the Lumerical workflow described here (see Appendix ).

The camera lens system can also be exported as a reduced order model (ROM) for fast ray tracing camera simulations in Speos as part of the camera system simulation workflow.

Step 2: Electrical Simulation (Weighting Function)

A 3D charge transport simulation of a pixel and its neighbors is used to calculate the weighting function, which is the probability that the charges generated at a given location in the substrate will be collected in the n-well of the pixel. Within reasonable approximations for typical CIS, the weighting function is independent of the incident light, i.e., the wavelength, incident angle, and polarization (see this article for more details). Additionally, we assume that all sensor pixels have the same doping profiles as well as the electrode layouts. Hence, we only calculate the weighting function for the n-well of one pixel in a single simulation and map the result to the other pixels of the Bayer matrix.

Step 3: Optical Simulations (Optical and Quantum Efficiencies)

In this step, we calculate the optical efficiency (OE), IQE and EQE as functions of incident angle (including polar angle \(\theta\) and azimuthal angle \(\phi\)), polarization, and wavelength. The IQE and EQE are also functions of pixel type.

It should be mentioned that the CRA and MRA from step 1 are not uniformly distributed. Therefore, it is not time efficient to sweep over those angles directly. Instead, we first run a sweep over uniformly distributed angles and then interpolate onto the CRA and MRA from step 1 (see EQE at Sampled Sensor Positions (Optional) in Run and Results ). This approach is reasonable if the OE and IQE are smooth functions of the incident angles.

Broadband FDTD simulations with a plane wave source and periodic Bloch boundary conditions provide the OE and the charge carrier generation rate, which are then combined with the weighting function from Step 2 to calculate the quantum efficiencies. This calculation happens within an analysis group in the FDTD project file and the data is saved in individual MATLAB data files for each simulation (for more information see Important model settings ).

Step 4 Average External Quantum Efficiency Over Illumination Cone

In this step the efficiency data from each simulation is consolidated in a single MATLAB data file and averaged over polarization to describe unpolarized light. Users only interested in the angular dependence of the EQE can stop at this point.
For the camera system workflow the EQE is averaged over the illumination cone (CRA and four MRA, see image below) for each sampled sensor position from Step 1. The resulting EQE (function of sensor position, wavelength, and pixel type) is saved in a JSON format that can be consumed by SSS Exporter (see Appendix for more details).

Run and Results

Instructions for running the model and discussion of key results

The entire CIS simulation workflow has been previously run. Step 1 above is described in more detail in the Appendix . In this section we focus on the remaining steps (2-4), which are automated by the scripted solution provided in the download package under “Associated files”.
The scripted solution is executed by the driver script [[CMOS_workflow.lsf]]. There users provide the location and name of a JSON configuration file with all the parameters that control the workflow, including a set of flags to enable or disable the different parts of the process. Information about the key parameters in the JSON configuration file can be found in the Important model settings section. A more complete description of the workflow and the configuration file is provided in the Appendix .
Two examples are provided in the download package: standard Bayer matrix and quad Bayer matrix. For illustration purposes, we focus on the standard Bayer example; for more information on the quad Bayer one, see Taking your model further . Here are the steps to rerun the scripted solution:

This workflow generates several simulations; therefore, it is recommended to check the simulation results already available in the download package first. Here we describe some of the key intermediate results saved in MATLAB files. The results are saved as nested cell arrays of dataset labeled by \(\{p_{y}+1\}\{p_{x}+1\}\) as shown below.

The pixel indices \(p_y=0,1,...\) and \(p_x=0,1,…\) are used in bayer_matrix of [[config_data.json]] to define the matrix arrangement (see Appendix ). To visualize one of the datasets, simply use the script command visualize with the result name and the appropriate indices in the argument.

Electrical Simulation

The weighting function results from the electrical simulations are saved as unstructured datasets in [[W_optimized_fem_data.mat]]. For compatibility with the data from the optical simulations in the next step, we interpolate the unstructured data onto a rectilinear grid and save it in [[W_optimized_rect_data.mat]]. For example, type the following commands in the script prompt to display the weighting function for G1 shown below.

matlabload('W_optimized_rect_data.mat');
visualize(W_rect{1}{1});

This example shows the potential crosstalk effects due to nonzero values of the weighting function in the neighboring pixels of G1. For example, the B pixels on the left and right-hand side of G1 have some probability of charge collection, especially the one on the left, as shown at the right edge of the color plot due to the folding of the weighting function in the Bayer matrix. This can be understood by shifting the B pixel to the left of the G1 pixel using the periodicity of the system, as shown in the diagram below.

The main reason for the larger contribution from the B pixel to the left of the G1 pixel is that the doping distribution is not symmetric around the center of the pixel (the n-well is off-centered). Clearly, this is an important design consideration.

Optical Simulation and Analysis

The results of the optical simulation, OE, IQE and EQE, are consolidated in a single MATLAB file in Step 4 as [[OE&QE_all_angle.mat]].
The OE data is saved as a function of the incident angles (\(\theta\) and \(\phi\)), wavelength, and polarization (1=x-pol, 2=y-pol, 3=unpolarized= (x-pol+y-pol)/2), as shown below.

The IQE and EQE can be visualized with the same method as the weighting function, by specifying the pixel indices. For these quantities, only unpolarized results are saved. Below we show the EQE for the four pixels at their corresponding peak wavelengths.

The higher overall EQE in the blue pixel compared to other pixels is mainly due to the higher absorption coefficient of the silicon at shorter wavelengths. Compared to green and red light, the concentration of charges generated by blue light is higher and more confined close to the silicon surface (where probability of collection is higher).

The term phi_quad in the plots above denotes the \(\phi\) angle for the first quadrant (from 0 to 90 degrees). Assuming four-fold symmetry, we determine the efficiencies for the rest of \(\phi\) angles, which reduces the total number of required simulations. This method is enabled by setting the quad_symmetry flag to true (see Appendix ).

EQE at Sampled Sensor Positions (Optional)

The IQE and EQE at given sensor positions are determined by averaging over the illumination cone specified in the JSON ray file from Zemax OpticStudio. The averaging is done in two steps.

First, the efficiencies at the CRA and MRA for each sensor position in the JSON ray file are calculated by linear interpolation from the angular data in [[OE&QE_all_angle.mat]]. The new efficiency data (function of sensor position, ray, wavelength, and pixel type) is saved in [[OE&QE_all_position.mat]]. The following image shows the EQE results for the four pixel types at their corresponding peak wavelengths, evaluated at the CRA of each sampled sensor position.

Second, a weighted sum of the EQE for the CRA and MRA (from [[OE&QE_all_position.mat]]) is calculated at each sensor position using the product of Weight and Intensity from the JSON ray file as the weighting factor for each ray angle. Since the EQE typically changes slowly over the illumination cone, the CRA and four MRA are enough, but users can specify any number of rays by providing the pupil coordinates. The weights used in this example are based on a geometric estimate by breaking the illumination cone into five regions, but they can also be modified to account for other factors (see Appendix for more information).

Important Model Settings

Description of important objects and settings used in this model

Parameters in the Configuration File

The configuration file [[config_data.json]] defines the parameters that control the CIS simulation workflow. Here we provide an overview of the key parameters. See Appendix for a complete description.

• flags : These parameters determine which part of the workflow will be executed. For example, if the JSON EQE file for SSS Exporter is not required, simply set the export_eqe flag to false.
• bayer_matrix : Here the pixels in the Bayer matrix can be labeled according to their location indices py and px in the FDTD simulation. The user is responsible for providing locations consistent with the setup in the FDTD simulations (see Run and Results for examples). When exporting the EQE data to SSS Exporter, the same pixel indexing based on location is used, as explained in the Appendix .
• optical : This category contains parameters that are applied in optical simulation and analysis. file_names.simulation.reference should match the name of FDTD simulation file. angular_grid consists of polar and azimuthal angles (\(\theta\) and \(\phi\)) where the efficiencies will be sampled. spectral_sampling specifies the range and number of points for wavelength sampling.
• electrical : This category contains parameters that are applied in electrical simulation and analysis. file_names.simulation.reference should match the name of the CHARGE simulation file. nw_config defines the name of the weighting function attribute (associated with the target n-well electrode) and the center position of the simulated pixel in CHARGE coordinates (which can be different from the FDTD ones).

Size of the CHARGE Simulation Region

The electrical simulation can account for the crosstalk between neighboring pixels, which becomes more of an issue as the size of the device becomes smaller. To capture this effect correctly, the CHARGE simulation region should include the pixels next to the pixel of interest. Therefore, the simulation region should ideally include nine pixels to determine the weighting function of the center pixel. However, in most cases, the weighting function decreases quickly within the neighbouring pixels (otherwise crosstalk would be large), and so we included only part of the surrounding pixels in this example to reduce the memory and simulation time. Convergence tests are necessary to determine the optimal size of the simulation region, as it depends on the pixel design.

“CMOS_sensor_optical_analysis” Analysis Group

This analysis group configures the source and monitors in a given FDTD simulation and returns the OE, IQE and EQE as a function of wavelength for the specific angle of incidence (\(\theta\) and \(\phi\)) and polarization specified in the User properties. The IQE and EQE are also functions of the pixel location in the Bayer matrix, according to the indexing in the weighting function, \(W(\mathbf{r})\).
The analysis script first calculates the absorbed power and the generation rate \(G_{norm}\), which is the absorbed power divided by photon energy, ℏω, and normalized to the total generation rate of the Bayer matrix. For details, please visit this whitepaper .
The \(OE(\lambda,\theta,\phi)\) is obtained by integration of absorbed power over the volume of the silicon substrate (normalizing to the input power). Similarly, the IQE and EQE are based on the following equations:

$$ IQE(\lambda,\theta,\phi) = \int_V  G_{norm}(\mathbf{r},\lambda,\theta,\phi)*W(\mathbf{r}) dV  $$

$$ EQE(\lambda,\theta,\phi) = OE(\lambda,\theta,\phi)*IQE(\lambda,\theta,\phi)  $$

Interpolation With Bloch Boundary Condition

There are multiple ways to simulate a broadband planewave source with oblique incidence:

• BFAST
• Single-frequency sweep with Bloch boundaries
• Broadband simulation with Bloch boundaries

For this workflow, we use the last approach because it has a good balance between accuracy and simulation time. For the plane wave source with Bloch boundary conditions, the polar angle of injection varies with wavelength. Therefore, it is necessary to interpolate between broadband results of simulations with multiple values of polar angle to obtain the broadband response for a given polar angle. For more information, please visit this link .

Simulation Bandwidth in FDTD Object

The broadband simulation with Bloch boundaries requires a different source wavelength range for each angle of incidence. To avoid changes in the mesh step and material fitting due to the source wavelength variation, we have enabled the simulation bandwidth option in the FDTD solver object. This option locks the fitting range, as well as mesh step size, in all the simulation files.

Strategies for Speeding Up Simulations

Since the workflow here involves a large number of FDTD simulations, the total simulation time can be very long. This page offers some tips for speeding up the simulations with HPC.

You can also consider using the monitors3D_downsample_x/y/z parameters in the "CMOS_sensor_optical_analysis" group to reduce the size of the recorded 3D data and consequently the size of the FDTD project files after simulations are run.

Updating the Model With Your Parameters

Instructions for updating the model based on your device parameters

Color Filter Materials

We are using the Lorentz material model to create a dispersive material compatible with broadband simulations. See the [[ Lorentz_permittivity_model.lsf]] for details.

Custom Structures

CIS technology keeps progressing with a better performance on resolution, sensitivity, crosstalk, etc. In this example we focused on a simple traditional design using standard Bayer matrix and front-side illumination (FSI). However, more modern sensors now use technologies such as deep-trench isolation (DTI) to reduce the crosstalk pixels, as well as back-side illumination (BSI) and quad Bayer matrix for higher sensitivity. The workflow described in this example is feasible with these more modern sensor designs (e.g., in Taking your model further we briefly explain how to setup a quad Bayer matrix). However, it is suggested to go through the simple standard Bayer example first.

If a new structure is imported, make sure you set the correct file name in the configuration file [[config_data.json]]. Please see "Parameters in the Configuration File" in Important model settings for more information.

Taking the Model Further

Information and tips for users that want to further customize the model

Illumination Cone

Users can change the definition of the rays used to describe the illumination cone and their weight in the averaging of the EQE. This must be changed in the JSON ray file from Zemax (see Appendix ). For example, using more rays increases the accuracy of the EQE when averaged over the illumination cone. The EQE at the ray angles is interpolated from the angular grid defined in the JSON configuration file; therefore, higher accuracy would also require a more finely sampled angular grid.

Pixel Arrangement

Any rectangular arrangement with any number of pixels can be defined in the JSON configuration file. As explained in Run and Results , the download package includes two examples. We only described the results for a standard 2x2 Bayer pattern, but results are also available for a 4x4 quad matrix in the folder [[Uniform_Quad_Bayer]], which also includes the corresponding base FDTD project file and JSON configuration file with the corresponding bayer_matrix definition.

Note that the "CIS_QBC" structure group in [[CMOS_broadband_QBC.fsp]] includes the string-type  parameter bayer_matrix , which supports two options to switch between the 2x2 and 4x4 patterns: "standard" and "QBC", respectively. The CHARGE simulation can be reused but the weighting function needs to be remapped (the resulting [[W_optimized_rect_data_4x4.mat]] is included).

NOTE: Due to file size limitation, the weighting function of the 4x4 patterns sensor is compressed as a zip file. Users need to unzip [[W_optimized_rect_data_4x4.zip]] before running the simulation.

Microlens Shift

As we have shown in the example, OE drops as the incident angle is increased. Moreover, light at oblique incidence heightens the crosstalk between pixels. Shifting the microlens is one way of compensating for these detrimental effects. The microlens shift must vary consistently with the angle of incidence; therefore, this requires spatial variation in the sensor array. The automated workflow described here can only handle uniform pixel arrays. However, the Step 3 of the Run and Results section of this example shows how to determine the optimal shift.

Additional Resources

Additional documentation, examples, and training material

Related publications

1. J. Vaillant, A. Crocherie, F. Hirigoyen, A. Cadien, and J. Pond, "Uniform illumination and rigorous electromagnetic simulations applied to CMOS image sensors," Opt. Express 15, 5494-5503 (2007)
2. A. Crocherie, J. Pond, F. Duque Gomez, K. Channon and F. Fantoni, “Micro to macro scale simulation coupling for stray light analysis,” Opt. Express 29, 37639-37652 (2021)

Webinar

• Cameras Design Workflow, Part 1: Modeling the Impact of Stray Light on the Imaging Performance of a Smartphone Camera System
• Cameras Design Workflow, Part 2: STOP Analysis Using STAR with an Automated Transient Workflow
• Cameras Design Workflow, Part 3: End-to-End Virtual Prototyping of CMOS Image Sensor Camera

See also

• CMOS Sensor Camera - Image Quality Analysis in a 3D Scene
• CMOS image sensor - Angular response 3D
• CMOS - Green's function IQE method
• CMOS - Broadband simulations
• Broadband simulation with Bloch boundary condition
• Image Sensors - List of Examples
• Speos Sensor System Exporter Overview
• Getting started with Speos Sensor System Exporter and running your first example

Related Ansys Innovation Courses

Appendix

In this appendix we describe in more detail the scripted solution to automate the CIS simulation workflow, as well as the different JSON files involved in the workflow: the configuration file, the ray file, and the final EQE file. For the JSON ray format we also explain how it can be generated in Zemax OpticStudio and for the JSON EQE file how it is consumed by SSS Exporter.

Description of the Scripted Solution to Automate the CIS Workflow

As briefly explained in Run and Results , the simulation setup, execution and analysis are driven by an automated scripted solution provided in the download package. The execution and data flow are explained in the flow diagram below.

• The yellow symbols denote inputs defined in the JSON configuration file.
• The script [[main.lsf]] executes the various modules ([[run_electricals.lsf]], etc.) according to the flags in the configuration JSON file.
• The key user inputs are shown in the bottom row of the diagram (CHARGE reference file, etc.).
• The intermediate data (saved mostly in MATLAB files) is denoted by light grey symbols inside the corresponding modules where it is generated.
• The dashed black arrows show the data flow. Note that when a step is skipped in the workflow the required intermediate data must be available.

Description of the JSON Configuration File

At the top-level, the JSON configuration file is a single dictionary with six keys: flags , bayer_matrix , optical , electrical , quantum_efficiency , and speos_data . Each of these keys corresponds to a dictionary with the data structure described below (see examples in download package):

Description of JSON ray file from Zemax OpticStudio

The ray data describing the illumination cone for each sampled position in the sensor is exported from Zemax OpticStudio in a JSON ray file. Note that the format of this ray file is specific to the end-to-end camera workflow.

At the top-level, the JSON ray file is a single dictionary with eight keys: ImageX , ImageY , Pupil , Theta , Phi , Intensity , Weight , and Wavelength . Under each of these keys, data is saved as “matrix” type according to the JSON file format specification described here . The [[rays.json]] file in the download package is an example of such a JSON ray file.

Generation of JSON ray file in Zemax OpticStudio

Follow the next steps in Zemax OpticStudio to generate the JSON ray file described in the previous subsection:

1. Open the project file with the lens system (e.g., [[Cell_Phone_Lens_Example.zar]] in the download package).
2. Use the Design Lock Down tool in the Tolerance tab to ensure that Ray Aiming is turned on and the System Aperture is "Stop".
3. Make sure the Field Type of the system is converted to “Real Image Height”.
4. Click on Programming > Interactive Extension so that OpticStudio is ready to be accessed by the Python script [[Export_AOI_JSON_Weight.py]] provided in the download package.
5. Run the Python script. The JSON ray file [[result.json]] is generated in the same location as the Python file.

Electron Map (XMP) calculation in Speos Sensor System (SSS) Exporter

For each pixel in each Bayer unit cell of the sensor, the exposure map \(E(\lambda)\) is combined with the EQE data to calculate the number of charges \(N_c\) collected at the pixel without saturation according to:

$$ N_c = \int_{\lambda_{min}}^{\lambda_{max}}[\frac{\lambda}{hc}{EQE}_{pix}(\lambda)E(\lambda)A_{pix}]d\lambda $$

In this expression, \(h\) is Planck’s constant, \(c\) is the speed of light in vacuum, \(A_{pix}\) is the area of the pixel and \({EQE}_{pix}(\lambda)\) is the EQE data at a particular wavelength \(\lambda\) and sensor position from the JSON EQE file.

Speos Sensor System (SSS) Exporter calculates the final number of charges collected by the pixel, \(N_{pix}\), truncating \(N_c\) with the absolute threshold sensitivity and dynamic range defined in the EMVA1288 standard (release 3.1) . This final number of charges is saved as an XMP file (see camera simulation workflow ). For more information on the SSS Exporter please visit this documentation page .

Note that SSS Exporter will use the spatial arrangement of pixels in the Bayer matrix specified in the configuration JSON file, as explained next.

Description of EQE JSON data file

The EQE JSON data file is a single dictionary with six keys (see examples in download package):

Using the notation in Lumerical script, an element in the EQE nested list can be denoted by EQE{\(i_y\)}{\(i_x\)}{\(p_y\)}{\(p_x\)}{\(l\)}, where \(i_y \in\) [1, 2, …, length( imagey )], \(i_x \in\) [1, 2, …, length( imagex )], \(p_y \in\) [0,1,…, N_pixely -1], \(p_x \in\) [0,1,…, N_pixelx -1], and \(l \in\) [1, 2, …, length( lambda )]. Note that we use zero-based indices \(p_x\) and \(p_y\) for consistency with the convention in SSS Exporter. The diagram below illustrates the data arrangement for a simple example with length( imagey ) = 2, length( imagex ) = 3, N_pixely = N_pixelx = 2, and length( lambda ) = 5.