Endoscope Lighting & Imaging

This example demonstrates a simulation workflow to analyze the image quality of an endoscope camera system under different lighting conditions. The simulation shows how different spatial and spectral lighting conditions affect image quality when imaging the interior of a section of intestine. The example also covers the process of importing a camera model from Ansys Zemax OpticStudio to Ansys Speos to evaluate the camera vision.

download example 

Software Prerequisites 

To be able to use this example, the following tools and assets need to be installed on your computer:

• Ansys Speos 2023 R1 or later
• Ansys Zemax OpticStudio 2023 R1 or later (optional) 

Overview

Understand the simulation workflow and key results

An endoscopy is a medical procedure in which your doctor uses specialized instruments to view and operate on the internal organs and vessels of your body. It allows surgeons to see problems within your body without making large incisions. Like most technologies, endoscopy is constantly advancing. Newer generations of endoscopes use high-definition imaging to create images in incredible detail. Innovative techniques also combine endoscopy with imaging technology for surgical procedures. Through simulation, we can test these advanced imaging techniques and make improvements on the endoscopic system design without the high cost of manufacturing and the time spent building physical prototypes.

Step 1: Lens system design with Ansys Zemax OpticStudio (not covered in this article) 

In this step, we design and optimize an endoscopic imaging lens system in Zemax OpticStudio and export the reduced order model (ROM) of the lens system. This ROM is referred to as Speos Lens System (SLS). The ROM only needs to be generated once and can be used to perform many system-level camera simulations in Speos, thereafter. After exporting the ROM from Zemax (*.OPTDistortion file), we gather the transmission information for the lens stack to also evaluate the impact of the lens materials and coatings on the image quality.

Step 2: Ansys Speos Simulation

The *.OPTDistortion file is imported into the Speos camera sensor to define the lens performance of the camera system. This then allows you to evaluate the sensor perception in a 3D scene with realistic illumination conditions. We run a ray trace using this ROM, which is about 100x faster than using the full lens system simulation in Speos. From this, one can extract key imaging metrics, such as from a spectral irradiance map. The irradiance map placed directly in front of the imaging sensor is calculated from the 3D scene, viewed through the camera lens stack. A 3D irradiance map is also used to map the light intensity on the inside walls of the intestine model.

Step 3: Ansys Speos post-processing

The 3D irradiance sensor results are viewed to determine the coverage within the geometry. With this map we can determine the irradiance on the surface of the test geometry at a given distance from the endoscope location. We will study the results of a few different LED spreads. The camera irradiance map enables us to determine the image quality within our test geometry. We will switch between the two LED spectra to see the difference in the test geometry.

Run and Results

Instructions for running the model and discussion of key results

Step 1: Ansys Zemax OpticStudio simulation

The lens system is designed in Ansys Zemax OpticStudio and exported to be used by Ansys Speos as an .OPTDistortion file.

Step 2: Ansys Speos simulation

1. In Speos, open the model titled: Intestine.scdocx.
2. Create a camera sensor.
3. Add the *.OPTDistortion file and transmission data gathered from Zemax OpticStudio.
4. Apply a 3D irradiance sensor to the “Facets” geometry. Set “Type” to “Photometric,” and “Integration Type” to “Radial,” and “Layer” to “Source."
5. Add the LED spectrum and flux information to the LED.
6. Run both simulations.
7. Isolate simulations to save the results. (right click simulation in tree, isolate)
8. Change LED to the other spectrum, rename the simulation, and rerun.

Step 3: Ansys Speos post-processing

1. Navigate to SPEOS isolated files\Intestine\Illumination.1.speos
2. Open "Illumination.3D Irradiance.1.xm3"
3. Adjust number of levels to 30 to provide a better gradient between maximum and minimum values.
4. Use a clipping plane to cut lengthwise along the geometry to visualize the illumination distribution along the intestine. We can see some structures cause a shadow. The camera would need to be moved to a new location in order to image these areas successfully.  
5. Navigate to SPEOS isolated files\Intestine\Warm White.1.speos.
6. Open "Warm White.Camera.1.Irradiance.xmp"
7. Here we can see a simulation of what the camera will see. In this result we can see a yellowing effect due to the light source spectrum
8. Open the "Measures" tab
    
9. Drag the measures box to the top right corner and select the CRI tab
10. We can see that the light source spectrum is showing a large shift compared to the day light reference. This could be a desired effect based on what color you are searching for in the endoscopy study.
11. Navigate to SPEOS isolated files\Intestine\Cool White.1.speos.
12. Open Cool White.Camera.1.Irradiance.xmp
13. Here we can see a simulation of what the camera will see. In this result we can see a lighting condition closer to daylight.
14. Follow steps 8 and 9 to compare the CRI differences.

We have shown how simulation can be used to quickly evaluate the performance of a complete endoscope optical system when imaging realistic geometry under varying illumination conditions.

Important Model Settings

Description of important objects and settings used in this model

Camera simulation:

In Speos, there are several ways to simulate a camera sensor. You may model the full lens system or a reduced order model (ROM). In this example, we rely on a ROM, which leverages a V2 camera model. For more info on the different camera model types, you may refer to this guide: Camera Models Description

The perks of using an *.OPTDistortion file to model the camera behavior is to:

• Perform fast simulations of the camera behavior (compared to simulating the complete camera system)
• Reproduce most of the real lens properties
• Perform simulation without exposing the manufacturer intellectual properties

The ROM camera sensor allows easy modification of settings such as:

• Camera sensor position
• Lens system behavior, exported as an *.OPTDistortion file from Zemax OpticStudio

Camera Sensor position:

In the camera definition, the Camera axis system, x and y direction are defined by an origin system (Camera_origin). You can move the ‘Camera origin’ using ’Move’ tool which is in ‘Design>Edit’ panel. Here you can vary camera position and direction to evaluate the performance of the design in other scenarios. In addition, you can create a new origin using ‘origin’ tool in ‘Design>Create’ panel and set the Camera axis to the new origin you created.  Moving the camera will allow you to determine that whether full visibility is accomplished. It is also possible to automate this movement and account for a trajectory as well as the effects of exposure time.

Lens System Behavior:

In the camera definition, in ‘Distortion’ section, the *.OPTDistortion extension can be loaded/edited. This file allows Speos to render the camera behavior. Depending on which version of *.OPTDistortion file is used, different information from the lens system could be included. Distortion curve V2 considers distortion asymmetry, variable origin (Entrance Pupil Point), vignetting, resolution, and depth of field while Distortion curve V1 is simplified, relying only on the relationship between the object angle and image angle. Examples of the file formats of V1 and V2 are shown below.

Light Source Illumination

As mentioned in the results, the angular spread of the light source can be a source of shadowed areas and sections of the test geometry may not be visible with a centralized path. This can be understood from this type of study. The geometry may be more complex, and the movement of the imaging head may help to illuminate the areas shadowed in the simulation. A key takeaway from this study is to validate that the scene is properly illuminated and visible at the edge of the camera field of view. Illumination at a distance is less of a concern due to the turbidity that will limit visualization in the far field.

Optical Properties

In this study we used a simple reflectance model for the outer surface of the test geometry. In reality, this material will have transmissive scattering properties, which will affect the light's path and visualization of areas of interest below the surface. This can also be studied within Speos, but care needs to be taken when creating these material models to strike a balance between accuracy/fidelity and speed.

Simulation Settings 

Different types of simulation are provided in Speos such as direct and inverse simulation. With camera models, we should use inverse simulation where light rays propagate from sensors to source.  If you want to adjust the Inverse Simulation advanced settings, see Adjusting Inverse Simulation settings. 

Meshing

Meshing settings are critical for getting the correct simulation results. Meshing settings define the quality of geometries that will be simulated. Rough mesh can lead to poor results, especially at the precise optical component. On the Faceted geometry we are limited to the mesh elements created by the facets. If importing a new geometry, it is important to ensure proper meshing is defined, in order to capture the required fidelity of the geometry in the simulation result. 

Additional Resources

Additional documentation, examples and training material

Relevant Speos Ansys Learning Hub courses

• Getting Started with Ansys Speos Software
• Ansys Speos Workflow and Geometry Management
• Getting Started with Camera Sensor in Ansys Speos Software