Designing Cell phone Camera Lenses Part 1: Optics

Also available in 日本語, 简体中文

This article is part of a four-series articles that will discuss the challenge of smartphone lens modules, from the conception and design to the manufacturing and analysis of structural deformation. This article is part one of the four-part series. It will focus on the design, analyses, and manufacturability of the lens module in OpticStudio.
Designing Cell phone Camera Lenses Part 2: Optomechanical Packaging
Designing Cell phone Camera Lenses Part 3: STOP analysis by using STAR module and ZOS-API

Designing Cell phone Camera Lenses Part 4: Impact Performance Analysis with LS-DYNA – Knowledgebase

Authored By Sandrine Auriol with the help of Chenfeng Gu and Katsumoto Ikeda

Article attachments

Introduction

Smartphones have become an important part of our everyday life and contain a host of high-tech optical systems to meet the demand for great imaging performance. Most smartphones have multiple complex and low-cost camera units fitted in a limited space. This presents a challenge to designers as well as to manufacturers. The injection-molded plastic lenses require an accurate alignment, but every module must work when mounted.

Mobile phone lens specifications

Mobile phone lenses are small form factor cameras, meaning that they are designed to minimize the amount of space they take in the mobile phone. They are lightweight and capture high-quality image at low F#. The usual specifications for mobile phone lenses are a very short system (Total Track (TOTR)<5mm) as mobile phones are getting thinner and thinner, MTF>0.2/0.25 at the Nyquist frequency (this is determined with the size of the detector pixel.), a large field of view and a Fast F#.

We are going to look at an example of a Mobile phone lens from a patent (1):

• Fast F/2.0
• Effective focal length f: @2.4mm
  That gives an Airy Radius = 1.22λf# ≈ 1.22 µm
• Full field of view = 95 degrees
• Pixel size = 2.5µm. The pixel size is close to the Airy disk size.
  By definition, the Nyquist frequency is one cycle in 2 pixels. For a pixel size of 2.5 µm, it is one cycle in 5.0 µm, so a Nyquist frequency of 200 lines/mm. A MTF greater than 20% at the Nyquist frequency is a typical minimum contrast for acceptable image quality.
• Sensor 1280 x 720 pixels. This is 1MP (Mega pixels).
  Although it is not top-of-the-line in terms of resolution for modern smartphone cameras (current smartphone lenses may use 12MP or so), it is still useful for surveillance, and other small-optics applications. Moreover, the concepts and methods presented here are relevant for applications such as modern smartphone optics.
• Semi-diagonal image height: 2-2.7mm
• Short TOTR = 4.8mm

Mobile phone lenses are typically using Extended or Q-type Aspheres to meet these specifications.

Material: Plastic

Injection-molded plastic is often used to produce these lenses, in high volumes and at low cost. The table below summarizes the pros and cons of plastic vs glass lenses. But please note that plastic materials can be further divided into families of plastic and each of these families will exhibit specific characteristics (COC = Cyclic olefin copolymer, COP = Cyclic olefin polymer, PMMA (acrylic), PC=Polycarbonate, PEI = Polyetherimide) (3).

Below is a comparison between a N-BK7 glass from Schott and a PMMA plastic from ALCON. It shows where some of these characteristics can be read in the Material Catalog:

The patent gives the index of refraction and abbe number of the plastic lenses. Let’s swap these with materials that have close characteristics:

Where N_d and V_d are the index of refraction and Abbe number at 587.56 nm.

APL5014C is a material from Mitsui Chemicals, Inc.
EP10000 Wis a material from Mitsubishi Gas Chemical.

Review of the Optical design

In terms of optimization criteria, the points to consider will be the spherical aberration, the coma, the astigmatism, the field curvature, the distortion, the chromatic aberration, the relative illumination and the resolution (or MTF).

For third-order aberration correction, the field curvature is corrected by lowering the Petzval Sum, which can require a large index of refraction difference to correct effectively. Due to the limited choice of index in plastic lenses, designers use highly aspheric lenses in shape, to correct each field.

The optical design here consists of 5 Extended Aspheres. At the front there is a cover glass to protect the optics. At the back we can see an optional infrared filter. It is described in the patent as used to “reduce or eliminate interference of environmental noise on the photosensor”.

In the patent, the STOP surface is located at the edge of the 1^st Extended Asphere lens using a dummy surface. To learn more about the STOP definition, have a look at this post: https://community.zemax.com/got-a-question-7/are-you-choosing-a-physical-stop-3113

Here is the design in OpticStudio:

This file is in the attachments and is called 710_original.zar.
The MTF specifications are not met when we directly enter the lens prescription given in the patent in OpticStudio.

Extended Even and Q-Type Aspheres

This patent is using Extended Asphere polynomials. The most used polynomials for these kinds of systems are Extended and Q-Type Aspheres. Both are available in OpticStudio.

Let’s have a look at the Extended Asphere polynomial. The sag z of an Extended Asphere can be described as:

z = Standard surface + Asphere terms

\(z = \frac{cr^{2}}{1+\sqrt{1-(1+k)c^{2}r^{2}}} + \sum_{i=1}^{n} \alpha_{i} \rho^{2i}\)

Where:

• c is the curvature (the reciprocal of the radius)
• r is the radial coordinate in lens units
• k is the conic constant
• ρ is the normalized radial coordinate
• α_i are the coefficients of the asphere in lens units.

The Extended Asphere polynomial may expand up to the 480th order.

Optimizing for aspheric terms need some care as aspheric terms may fight each other and high order coefficients might lead to non-manufacturable shape. The 2^nd order term can fight with the curvature and is not available with some manufacturing hardware – it is available in OpticStudio for completeness. The 4^th order term can fight with the conic constant. In general, the values of the coefficients can't be compared easily. It is difficult to tell which order has the largest effect based on the value of the coefficients.

This is why sometimes Q-Type Aspheres are preferred over Extended Aspheres. Q-Type Aspheres have orthogonal terms. The Q-Type Asphere is a radially symmetric surface developed by G. Forbes. It has two variations:

• Qbfs (best fit sphere, “Type 0” in OpticStudio) defines a surface characterized by RMS slope departure of the asphere from the best fit sphere. It can be used with mild aspheric variations from a sphere.
• Qcon (conic, “Type 1” in OpticStudio) defines a surface characterized by the sag departure of the asphere from a base conic. It is best used for strong aspheric variations from a sphere.

Q-Type Aspheres are more computationally intensive than the Extended Even Asphere. However, they have several advantages. The magnitude of the coefficients directly relates to the slope or sag departure from the best fit sphere or conic (depending on Type 0 or Type 1). The terms are orthogonal over the normalization radius, so can be controlled directly during optimization to help improve manufacturability. It means that the terms can be optimized together because they do not directly affect each other. The values of the coefficients are also usually larger, so it requires fewer digits of precision

The table below summarizes those pros and cons:

OpticStudio offers a tool to convert between asphere definitions, so let’s swap for a Q-type asphere. Since we have strong aspheric variations from a sphere, we will use the Qcon type (“Type 1”).

If the Number of Terms is set to Automatic, OpticStudio automatically determines the proper order for the new asphere based upon the order of the old asphere.

The conversion to Q-Type Asphere is a 1:1 conversion so the fit is exact. If it is not, increase the number of terms.

Optimization

From the patent, the mobile phone lens module has been modified with real plastic materials and a different polynomial definition. The MTF performances are not met, so let’s rework slightly the design to meet them. Using OpticStudio optimization tools, a merit function can be built to reoptimize slightly the thicknesses. Hopefully this step will be enough to ensure good performances. The thickness of the IR filter is not changed as it is a standard thickness for an IR filter.

The merit function is targeting a small RMS wavefront as well as the MTF for all fields at 200 lines/mm to be superior to 0.2. Another option could be to use the contrast optimization. The merit function can also include operands to control the distortion and the relative illumination. These operands can have a small weight or a weight of zero and be checked after a first round of optimization. Sometimes these values are fixed in software post-processing.

After reoptimizing, the thicknesses are rounded to the 3^rd decimal.

One issue appears after the optimization of thickness. The thickness of surface 14 is now too small and the profile of surfaces 14 and 15 overlaps.

To correct this issue, we can simply increase the air space between surface 14 and surface 15 and remove the extra thickness from surface 16. This will also help to mount the lenses.

In the optimization, the operands FTGT (Full Thickness Greater Than) or DSAG (Can calculate different sag data like the maximum sag) could be used to control this.

The change of thickness for each lens is very small (<0.1mm). The “original” file and “new” file are compared below using the File Comparator:

Here are the new MTF results with this lightly reoptimized design. On axis, the MTF curve is very close the diffraction limit:

The file is called 710_reoptimized_MTF_materials_QType.zar.

Controlling the manufacturability

Aspheres are more challenging to manufacture due to their non-conventional shape. Mobile phone lenses are usually produced through injection plastic molding. Plastic is injected into a mold with the aspheric form. The repeatability of this process is very good, which is why injection molding is a good option for producing large quantities.
There can be some disadvantages in using plastic as these lenses structure tend to be less stable. This will be studied later by using a FEA modelling along the STAR module. 
To able to manufacture the mold, there are a few things to keep an eye on:

• Inflection points: location on the surface where the curvature change sign. Inflection points lead to typical gull wing shapes
• Concave surfaces: depending on the local radius of curvature, this can be limited by manufacturing tool size
• Slope changes: common specification describing how fast the surface irregularity changes over a sub aperture

OpticStudio has tools to control the surface sag, curvature, slope.

All these tools have associated operands that can be added to the merit function to control non-manufacturable solutions. For example, let’s have a look at surface 13. It is a Gullwing surface with a base radius of curvature of 0.777mm:

Below, are plotted the Surface Sag, the Surface Slope and the Surface Curvature

The Sag Table & BFSD (Best Fit Sphere Data) operand define the total volume of material to be removed from the Best-Fit-Sphere (BFS). All of these analyses give important information on the manufacturability and testing of the asphere:

• The sag and sag table gives the surface shape, so the variation of the local thickness z. This with the Best-Fit-Sphere data determines the effort required to make the asphere. The lower the number, the less effort is required. The aspheric sag departure directly affects the fabrication time.
• The curvature shows how the aspheric coefficients modify the local radius of curvature of the surface. The control of the local curvature is vital to allow the cutting tool to function properly.
• The local slope of the aspheric departure determines how quick the surface changes. It is important for manufacturing and for testing. The maximum slope difference between the BFS and the surface directly affects the maximum number of fringes seen in interferometric testing; and the root-mean squared (rms) slope difference affects the total number of fringes across the aperture of the surface. This will determine the size of the sub aperture that can be used to test an optical element using a direct measurement method.

There are merit function operands to compute these values:

The design can be reviewed surface by surface to check the manufacturability and define the best way to test those surfaces.

Analyses (Relative Illumination, Image Simulation, MTF)

Relative Illumination

The Relative Illumination (RI) analysis computes the relative illumination of the optical system as a function of the radial field coordinate for a uniform Lambertian scene. It tends to decrease at the edge of the field because of the cosine fourth law. The cosine fourth law is observed in a thin, slow aberration free system with the STOP at the lens; the image irradiance falls off as the cos⁴ of the field angles.

Image irradiance ≈ \(cos^{4}\)(field angle)

For an angle of 47.5 degrees, it would predict a RI > \(cos^{4}(47.5)\), so roughly 21%.
The Relative Illumination plot finds a higher value. It is plotted for fields along +Y.

Image simulation

Let’s see what the camera sees using the Image Simulation. As the full field of view is 95 degrees, we are going to switch from field angles to field heights. A pixel in degree at an angle of zero degree might not be the same size as the pixel in degree at 47.5 degrees.

For this reason, let’s add a paraxial lens and change the field angles to field objects. The object thickness is set 1000 mm (so the object will be at 1m). Enter a new surface at surface 1 and make it a paraxial surface with a focal length of 1000mm. Set the paraxial surface thickness to be 10 mm. Then convert the field angles to field heights.

We can check the conversion using merit function operands:

So, the maximum field in object height measures 1103mm. With this field, the software is unable to calculate the relative illumination so we will reduce that value down to 1000mm, which corresponds to an angle of 45 degrees.

Let’s how this input image "{Zemax}\IMAFiles\Demo picture -  640 x 480.bmp" can be seen through the camera.

The diagonal of that image is 400 pixels =\(\sqrt{{(\frac{640}{2})^{2}+(\frac{480}{2})^{2}}}\).
So the field height will be 1200 =\(\frac{480*1000}{400}\).
It means that the pixel size is 2.5mm =\(\frac{1200}{480}\).
The magnification is -0.002, so the pixel size at the image will be roughly 5um.
The system is diffracted limited and the airy disk radius is 1.4um. Based on our specification, the pixel size of the detector is 2.5um. So, let’s oversample by 2 to make the “optical” pixel size smaller.

The images below show the results. The PSF grid colors were inversed in an image editor.

The Image Simulation mainly shows us that the image is darker at the edge of the field of view.

FFT through focus MTF

The FFT through focus MTF shows the sensitivity of the design in terms of image plane position. Here is the shift is +/-0.015mm for a frequency of 200cycles/mm.

MTF vs field

The MTF vs field shows the MTF at specified frequencies (here at 50, 100, 150 and 200 cycles/mm)  as a function of the field. It shows how the MTF changes with the field angle.

Conclusion

This article has shown the tools that help designers create a mobile phone lens in OpticStudio. 

Next Article: Designing Cell phone Camera Lenses Part 2: Optomechanical Packaging with OpticsBuilder where we will edit optics with Zemax OpticsBuilder to extend lenses with complex edges, so they can be fitted into a mechanical mounting.

References

1. US patent 2019/0129149 A1 – lens system 710 “Wide FOV 5 Element Lens System”:  https://patents.google.com/patent/US20190129149A1/en
2. Resolution and MTF: https://www.edmundoptics.co.uk/knowledge-center/application-notes/imaging/resolution/
3. Plastic:
   https://optical-tech-group.toyotec.com/en/technicalcolumn/what-materials-are-used-for-plastic-lenses/
   https://www.photonics.com/Articles/Plastic_Optics_Specifying_Injection-Molded/a25487
   https://www.ipt.fraunhofer.de/en/Competencies/Productionmachines/precisiontechnology-plasticreplication/injection-molded-lenses.html
4. [Webinar] Cell Phone Lens: The Fundamentals Behind the Optical System Design [Q&A]
5. Community forum thread: How to control the aspherical surface during the optimization