This section describes how to calculate the point spread function (PSF) of an image sensor array. The PSF is a measure of spatial cross talk. In other words, if one pixel is fully illuminated, how much light is detected in the neighboring pixels.
The figure on the right shows experimental setup being simulated. One pixel of the image sensor is illuminated. We measure the amount of power detected by the neighboring pixels. Examples for 2D and 3D are also provided to show how to calculate the point spread function (PSF) and Modulation Transfer Function (MTF).
Introduction
Typically the Point Spread Function is defined as the response of the system to a illumination by a point source at a large distance from the camera. In a digital camera, however, the concept of a Point Spread Function must be extended to account for both the spread due to the finite Numerical Aperture (NA) of the lens system, and the spread due to crosstalk in the digital image sensor pixels.
Suppose we take a picture of a flat wall that is painted black, except for one green square. The size of this green square is such that it corresponds to exactly one pixel of the image sensor. (For example, if the wall is 1x1m and the image sensor has 1000x1000 pixels, then the size of the green square should be 1x1mm.) Incoherent, unpolarized light from the green square will propagate away from the wall. Any light within the NA of the lens system will be collected and focused onto the image sensor plane. Ideally, the light will fully illuminate one pixel of the image sensor, without any illumination of neighboring pixels. In practice, the finite NA of the lens system will cause the incoming beam to spread and illuminate more than one pixel, as shown in the following figure. In addition, as the light travels through the image sensor lenses, oxide layers and metallic interconnects some light is scattered into adjacent pixels. Finally, there can be some electrical crosstalk between pixels after the photons have been converted into electron-hole pairs. As a result of all of these effects, an electrical signal will be registered in several pixels rather than a single pixel.
We define the digital Point Spread Function (PSF) as the fraction of power detected in the active region of each pixel as a function of pixel position for the illumination conditions that would ideally result in a response in a single pixel.
Since FDTD is a coherent simulation tool, it is not possible to directly simulate the above system. The problem is that we can't directly reproduce incoherent illumination from the macroscopic lens system. Instead, we recognize that this illumination is equivalent to the sum of a number of incoherent point sources, as shown below.
It is possible to simulate this equivalent system with FDTD. Two simulation are required for each point source position. When the set of simulations are compete, it is possible to reconstruct the desired illumination (incoherent, un-polarized illumination of one pixel).
|
Note: Thin lens source The Thin lens option of the Gaussian beam source is perfectly suited these simulations. Simply specify the NA and focal point of your lens system. |
2D simulation
A screenshot of CMOS_psf2D.fsp is shown above. The simulation is similar to the 2D Angular response simulation, but with the following modifications:
- 10 pixels are included in the simulation region, corresponding to 5 unit cells.
- PML is used on all boundaries. (Periodic BC don't make sense in these simulations)
- A gaussian beam source with a NA of 0.25 has replaced the plane wave.
- The model analysis group is now used to create results for the electric field intensity of the source and the electric field intensity in the entire simulation
- An analysis group called "psf_analysis" is now used to integrate the the fraction of power incident under each of the red and green pixels
Run and results
Once again, the source wavelength is 550nm (green). We expect high transmission through the center green color pixel and lower transmission through the other pixels.
Results from a single simulation are not very useful. Further analysis is required to reconstruct the desired illumination conditions.
Point spread function
The nested parameter sweep object "sweep position" is used to iterate over several positions and both polarizations. This parameter sweep collects a number of results that are average incoherently using the "Mean" operation. After running the parameter sweep, run the script CMOS_psf2D_analysis.lsf to display the results.
This figure show the incident electric field intensity at the source injection plane, and at the surface of the Si layer. The blue line shows the uniform illumination of the central pair of pixels. The green line shows the E field intensity at the Si surface. The red filter has blocked most of the light from entering the red pixel, while the transmission is high into the green pixel.
Next, the script plots |E|^2 through the device.
Finally, the script calculates the point spread function and outputs it to the script prompt. The data has then been plotted below where the Y axis is broken into two sections to better show all of the results on a single figure.
Approximately 30% of the total source power is absorbed in the center green pixel. This corresponds to a 60% efficiency for light incident on the green pixel. Approximately 1% of the total power is absorbed in the center red pixel, while 2% is absorbed in the nearest neighbor green pixel.
Modulation transfer function
The modulation transfer function (MTF) is calculated by Fourier transforming the PSF (of green light only) and taking the absolute value. The MTF is plotted below. We see that the MTF is eventually limited by the 4 micron spacing between the green pixels, meaning that the maximum resolution is 125 cycles per mm (0.5/4 micron). However, even below that resolution the MTF is not ideal because of the light captured in neighboring pixels.
3D simulation
|
Note: 2D vs 3D simulations We recommend starting with the 2D example because the simulations run much faster. It is best to gain some basic understanding of the device from fast 2D simulations before starting the more accurate but slower 3D simulations. This is always a good tip, but it is especially true for PSF calculations because of the large number of simulations that are required. Also, the simulation region must be quite large to include several periods of the device. We can also calculate the modulation transfer function (MTF) by Fourier transforming the PSF. |
A screenshot of CMOS_psf.fsp is shown above. The simulation is similar to the 3D Angular response simulation, but with the following modifications:
- 5x5 pixels are included in the simulation region
- PML is used on all boundaries. (Periodic BC don't make sense in these simulations)
- A gaussian beam source with a NA of 0.25 has replaced the plane wave.
- An analysis group called "surface analysis" is used to calculate the response of the red, green and blue sub-pixels under each of the simulated 3x3 pixels.
Once again, the source wavelength is 550nm (green). The nested analysis object called "sweep x" will sweep the source position of 4x4 positions and 2 polarizations for a total of 32 simulations. The simulation time for the entire sweep is significant because of the number of simulations involved (4x4 positions * 2 polarizations), and because each simulation region is quite large (6x6 pixels). We use conformal variant 1 meshing, which does take a reasonable fraction of the simulation time for this simulation, but allows us to operate at a mesh accuracy of 1 and therefore gives us a net savings for the same simulation accuracy.
The following figure shows the source profile from the first simulation. It also show the other source positions used in this calculations (purple dots). The black lines indicated the pixel edges.
Run and results
Point spread function
When the simulations created by the nested parameter sweep object "sweep x" are complete, run CMOS_psf_analysis.lsf to display the results.
The composite source injection profile (-0.5*real(Pz)). This profile is obtained by incoherently summing the profiles of the individual simulations. The purple dots show the individual source positions used in each simulation. The black lines indicated the pixel edges. Please note that these purple dots and black lines are not created by CMOS_psf_analysis.lsf.
This figure shows the power flow (-0.5*real(Pz)) into the Si layer. As expected, most of the power flows through the green pixels that are illuminated. A smaller amount is absorbed by the neighboring green pixels. This profile is obtained by incoherently summing the profiles of the individual simulations at the Si surface. Again, please note that the black lines are not created by CMOS_psf_analysis.lsf.
Same as the previous figure, but on a log scale.
This figure shows the depletion regions (red) of each pixel. To calculate the power absorbed in each pixel, we integrate Pz over the depletion region area for each pixel. This example uses a square depletion region, with a width of 1.5um. The pixels are 2um wide, which leaves a 0.5um gap (blue) between depletion regions. This figure is not created by the script CMOS_psf_analysis.lsf. To see this figure, run a simulation, edit the surface_analysis object and set make_plots to 1. Run the analysis script and you will see this figure.
The script outputs the PSF in tabular form.
power absorbed in Si layer: 33.5%
power absorbed in all pixels: 30.2%
power to red pixels (%):
0 0.1 0
0 0.4 0.2
0 0.2 0.1
power to green pixels (%):
0.1 0.9 0.2
0.9 24 0.9
0.2 0.9 0.1
power to blue pixels (%):
0.1 0.2 0
0.2 0.4 0
0 0.1 0
Modulation transfer function
The modulation transfer function is also calculated for green light by CMOS_psf_analysis.lsf and is shown below. It is interesting to note that it is not symmetric which is due to the asymmetry introduced by the Bayer pattern. Also, it could be expected that a different type of asymmetry would be found for red and blue light, again due to the Bayer pattern. Finally, we note that the maximum resolution of 125 cycles per mm comes from the pixel dimensions of 4 microns (125/mm=0.5/4micron)
Related publications
- F. Hirigoyen, A. Crocherie, J. M. Vaillant, and Y. Cazaux, “FDTD-based optical simulations methodology for CMOS image sensors pixels architecture and process optimization” Proc. SPIE 6816, 681609 (2008) http://dx.doi.org/10.1117/12.766391
- J. Vaillant, A. Crocherie, F. Hirigoyen, A. Cadien, and J. Pond, "Uniform illumination and rigorous electromagnetic simulations applied to CMOS image sensors," Opt. Express 15, 5494-5503 (2007) . (Produced in collaboration with Axel Crocherie, Flavien Hirigoyen, Jérôme Vaillant and Yvon Cazaux of STMicroelectronics, France.)