Endoscope Lighting & Imaging

This example demonstrates a simulation workflow to analyze the image quality of an endoscope camera system under different lighting conditions. The simulation shows how different spatial and spectral lighting conditions affect image quality when imaging the interior of a section of intestine. The example also covers the process of importing a camera model from Ansys Zemax to Ansys Speos to evaluate the camera vision.

download example

Software Prerequisites

To be able to use this Example, the following tools and assets need to be installed on your computer:

• Ansys Speos 2023 R1 or later
• Ansys Zemax OpticStudio 2023 R1 (optional)

Overview

Understand the simulation workflow and key results

An endoscopy is a procedure in which your doctor uses specialized instruments to view and operate on the internal organs and vessels of your body. It allows surgeons to see problems within your body without making large incisions. Like most technologies, endoscopy is constantly advancing. Newer generations of endoscopes use high-definition imaging to create images in incredible detail. Innovative techniques also combine endoscopy with imaging technology for surgical procedures. Through simulation, we can test these advanced imaging techniques and make improvements on the endoscopic system design without the high cost of manufacturing and the time spent building physical prototypes.

Step 1: Lens system design wit h Ansys Zemax OpticStudio (not covered in this article)

In this step, we design and optimize an endoscopic imaging lens system in Zemax OpticStudio and export the reduced order model (ROM) of the lens system. This ROM is referred to as Speos Lens System (SLS). The ROM only needs to be generated once and can be used to perform many system-level camera simulations in Speos, thereafter. After exporting the ROM from Zemax (*.OPTDistortion file), we gather the transmission information for the lens stack to also evaluate the impact of the lens materials and coatings on the image quality.

Step 2: Ansys Speos Simulation

The *.OPTdistortion file is imported into the Speos camera sensor to define the lens performance of the camera system and evaluate sensor perception in a 3D scene with realistic illumination conditions. We run a ray tracing photometric ROM camera simulation, which is about 100x faster than full lens system simulation in Speos, and extract the key imaging metrics, such as from a spectral irradiance map. The irradiance map placed directly in front of the imaging sensor is calculated from the 3D scene, viewed through the camera lens stack. A 3D irradiance map is also used to map the light intensity on the inside walls of the intestine model.

Step 3: Ansys Speos post-processing

The 3D irradiance sensor results are viewed to determine the coverage within the geometry. With this map we can determine the irradiance on the surface of the test geometry at a given distance from the endoscope location. We will study the results of a few different LED spreads. The camera irradiance map enables us to determine the image quality within our test geometry. We will switch between the two LED spectra to see the difference in the test geometry.

Run and Results

Instructions for running the model and discussion of key results

Step 1: Ansys Zemax OpticStudio simulation

The lens system is designed in Ansys Zemax OpticStudio and exported to be used by Ansys Speos as an .OPTdistortion file .

Step 2: Ansys Speos simulation

1. In Speos, open Speos simulation file Intestine.scdocx.
2. All simulations have been pre-run and isolated so the results can be viewed without the simulation time as an issue.
3. Create a camera sensor.
4. Add the *.OPTdistortion file and transmission data gathered from Zemax OpticStudio.
5. Apply a 3D irradiance sensor to the “Facets” geometry. Set “Type” to “Photometric,” and “Integration Type” to “Radial,” and “Layer” to “Source."
6. Add the LED spectrum and flux information to the LED.
7. Run both simulations.
8. Isolate simulations to save the results. (right click simulation in tree, isolate)
9. Change LED to the other spectrum, rename the simulation, and rerun.

Step 3: Ansys Speos post-processing

1. Navigate to SPEOS isolated files\Intestine\Illumination.1.speos
2. Open Illumination.3D Irradiance.1.xm3
3. Adjust number of levels to 30 to provide better definition between maximum and minimum values.
4. Use clipping plane to cut lengthwise along the geometry to visualize the illumination distribution along the intestine . We can see some structures cause a shadow. The camera would need to be moved to a new location in order to image these areas successfully.
5. Navigate to SPEOS isolated files\Intestine\Warm White.1.speos.
6. Open Warm White.Camera.1.Irradiance.xmp
7. Here we can see a simulation of what the camera will see. In this result we can see a yellowing effect due to the light source spectrum
8. Open the measures tab
9. Drag the measures box to the top right corner and select the CRI tab
10. We can see that the light source spectrum is showing a large shift compared to the day light reference. This could be a desired effect based on what color you are searching for in the endoscopy study.
11. Navigate to SPEOS isolated files\Intestine\Cool White.1.speos.
12. Open Cool White.Camera.1.Irradiance.xmp
13. Here we can see a simulation of what the camera will see. In this result we can see a lighting condition closer to daylight.
14. Follow steps 8 and 9 to compare the CRI differences.

We have shown how simulation can be used to quickly evaluate the performance of a complete endoscope optical system when imaging realistic geometry under varying illumination conditions.

Important Model Settings

Description of important objects and settings used in this model

Camera simulation:

In Speos there are 3 main types of camera models to generate irradiance map :

1. Basic camera sensor which is based on pin-hole camera.
2. Reduced Order Model (ROM) camera sensor which uses *.OPTdistortion data from Zemax OpticStudio model also called Speos Lens System.
3. Irradiance sensor using full lens system exported from OpticStudio.'We can now export either CAD parts or use the Zemax Importer Tool.

Selecting between different camera simulations is a balance between accuracy and simulation time. The full lens system would be the most accurate one, but simulation with basic camera model and ROM camera would be about 100 times faster. Usually, the ROM is the best balance between accuracy and simulation time for most of the analysis and is the one we use in this project.

The ROM camera sensor allows easy modification of its setting such as:

• Camera position
• Lighting transfer function exported from Zemax OpticStudio model (*.OPTdistortion file)

Camera Sensor position:

In camera definition, the Camera axis system, x and y direction are defined by an origin system (Camera_origin). You can move the ‘Camera origin’ using ’Move’ tool which is in ‘Design>Edit’ panel , v ary camera position and direction to evaluate the performance of the design in other scenarios. In addition, you can create a new origin using ‘origin’ tool in ‘Design>Create’ panel and set the Camera axis to the new origin you created.  Moving the camera will allow you to determine that full visualization is accomplished. It is also possible to automate this movement and account for a trajectory as well as the effects of exposure time.

Camera Sensor Optical Distortion (Lighting transfer function):

In Camera definition, in ‘Distortion’ section, a file with *.OPTDistortion extension can be loaded which is editable as text file. This file has information regarding the object to image angular relationship and allows Speos to render the camera behavior.

Note: Depending on which version of *.OPTDistortion input file is used for camera optics, different information from lens system could be included. Distortion curve V2 describes an enhanced camera model which considers distortion asymmetry, variable origin (Entrance Pupil Point), vignetting, resolution, and depth of field while Distortion curve V1 is the simplified version and has only the information regarding the chief ray angle curve. File format of an example distortion curve V1 and V2 is shown below.

Further details about Camera sensor settings, and distortion curve can be found in Camera Sensor (ansys.com) , Distortion Curve (ansys.com)

Light source

As mentioned in the results, the angular spread of the light source can be a source of shadowed areas and sections of the test geometry may not be visualized with a centralized path. This can be understood from this type of study. The geometry may be more complex, and the movement of the imaging head may help to illuminate the areas shadowed in the simulation. The key takeaway from the study should be at the edge of our camera field of view and should be illuminated to an appropriate level. Illumination at a distance is less of a concern due to turbidity that will limit visualization.

Optical Properties

In this study we used a simple reflectance model for the outer surface of the test geometry. This material will have transmissive scattering properties, which will affect the light's path and visualization of areas of interest below the surface. This can also be studied within Speos, but care needs to be taken when creating these material models to be sure they represent reality while still being simple enough to simulate in a reasonable time frame.

Simulation Settings

Different types of simulation are provided in Speos such as direct and inverse simulation. With camera models, we should use inverse simulation where light rays propagate from sensors to source.  If you want to adjust the Inverse Simulation advanced settings, see Adjusting Inverse Simulation settings .

Simulation Meshing Settings

Meshing settings are critical for getting the correct simulation results. Meshing settings define the quality of geometries that will be simulated. Rough mesh can lead to poor results, especially at the precise optical component. On the Faceted geometry we are limited to the mesh elements created by the facets. If you import a new geometry, it is important to ensure proper meshing to capture the structures.

Additional Resources

Additional documentation, examples and training material

Relevant Speos Ansys Learning Hub courses.

• Speos Getting Started
  • Speos Workflow & Geometry Management
  • Speos Camera Sensor