Introduction to Lumerical datasets

Lumerical datasets are structured data objects that collect a set of related matrices into a single convenient object. To introduce this concept, we'll start by providing two examples. Additional information follows.

Ex. 1 – Reflection vs radius and height (matrix dataset)

Suppose you have a parameter sweep that measures the reflection from a particle as a function of particle radius and height. Saving this information generally requires 3 matrix variables. The 2D matrix R contains the reflection value from each simulation, while the 1D vectors radius and height contain the associated position values.

A dataset object can be used to collect all three matrices into a single dataset variable.

The following script code generates some example data, creates a R(radius,height) dataset, and finally creates several plots of the data.

# create example results
radius = 0:10; 
height = 1:0.1:3;
reflection = randmatrix(length(radius),length(height)); 
# create Reflection dataset
R = matrixdataset("R"); # initialize dataset
R.addparameter("radius",radius); # add radius parameter
R.addparameter("height",height); # add height parameter
R.addattribute("R",reflection); # add reflection attribute
# plot data
image(radius,height,reflection); # use original matrices
image(R.radius,R.height,R.R);  # use dataset
# send dataset to visualizer
visualize(R);

Ex. 2 – Electric field data from a monitor (rectilinear dataset)

Field monitors in FDTD and MODE are used to calculate and save spatial electric field data. The raw electric field data within the monitor is distributed between several matrices: Each vector field component is stored in a matrix (Ex, Ey, Ez), and the four associated position vectors (x,y,z,f) are also stored as separate matrices.

A dataset can be used used to collect all of this information into a single dataset variable. The figure to the right shows the three matrices Ex, Ey, Ez that store each component of the electric field. It also shows the associated position vectors x, y, z. All of this information can be stored within a single dataset variable.

Note: The electric field matrices usually have a 4th dimension (time or frequency), but this dimension is not included in the figure due to the difficulty of graphically representing a 4th dimension!

The following script code shows how to get the raw data from a frequency monitor in FDTD (using getdata), and how to manually create a dataset from that data. It also shows how to directly get the electric field dataset from the monitor in a single command (using getresult). The final section shows how to create a few plots of the data

# monitor name
m="monitor";
# get individual data elements with getdata
x=getdata(m,"x");
y=getdata(m,"y");
z=getdata(m,"z");
f=getdata(m,"f");
Ex=getdata(m,"Ex");
Ey=getdata(m,"Ey");
Ez=getdata(m,"Ez");
# manually create the electric field dataset from the raw data
# initialize dataset and provide spatial position vectors
E_manual = rectilineardataset("E_manual",x,y,z);  
# add additional parameter: frequency
E_manual.addparameter("lambda",c/f,"f",f);      
# add vector electric field attribute
E_manual.addattribute("E",Ex,Ey,Ez);        
# all of the above commands can be avoided with a single getresult command
E_fromMonitor = getresult(m,"E");
# plot Ex(x,y) at the first z value and first frequency point
to_plot = pinch(pinch(Ex,4,1),3,1);  # use original matrices
image(x,y,to_plot);          # use original matrices    
to_plot = pinch(pinch(E_manual.Ex,4,1),3,1); # use dataset
image(E_manual.x,E_manual.y,to_plot);     # use dataset
# Plot the entire dataset with the Visualizer
visualize(E_manual);

Attributes and parameters

Attribute: The actual data of a dataset. In the examples above, the reflection R and electric field Ex, Ey, Ez were the Attributes of the dataset.

Parameter: The associated position vectors of a dataset. In the above examples, radius, height, x, y, z, and f were the parameters.

Attribute dimensions

When creating datasets, the size of the attribute matrices must be consistent with the lengths of the associated parameters matrices.

Matrix dataset

Scalar attribute: [\(N_{p1}; N_{p2}; ...; N_{pn}\)]
Vector attribute: [\(N_{p1}; N_{p2}; ...; N_{pn}; 3 \)]

Where \(N_{pi} \) is the length of the \( i^{th} \) parameter.

Rectilinear dataset

Scalar attribute: [\(N_{\mathbf{x}}; N_{\mathbf{y}}; N_{\mathbf{z}}; N_{p1}; N_{p2}; ...; N_{pn}\)]
Vector attribute: [\(N_{\mathbf{x}}; N_{\mathbf{y}}; N_{\mathbf{z}}; N_{p1}; N_{p2}; ...; N_{pn}; 3 \)]
Tensor attribute: [\(N_{\mathbf{x}}; N_{\mathbf{y}}; N_{\mathbf{z}}; N_{p1}; N_{p2}; ...; N_{pn}; 9 \)]

Where \(N_{\mathbf{k}}, \mathbf{k} = \mathbf{x},\mathbf{y},\mathbf{z} \) are the lengths of the coordinate vectors, and \(N_{pi} \) is the length of the \( i^{th} \) parameter. If the dataset is 2D or 1D then you will have singleton dimensions, so that one or multiple \(N_{\mathbf{k}} = 1\). These dimensions can be removed with pinch - Script command.

What is in a dataset? Icons and the '?' operator

The Results view and Script workspace windows provide a list of current monitor results and script variables. Different icons are used for each type of variable (dataset, matrix, string, unstructured data). For datasets, the preview column provides a list of the attributes and parameters in the dataset. In the screenshot below, the 'E' dataset contains one attribute (E) that is a function of 4 parameters (x,y,z, lambda/f).

The '?' operator can be used to output the same information to the script prompt.

?E_field = getresult("monitor","E");
 E vs x, y, z, lambda/f

To output the actual attribute or parameter values, do something like:

?E_field.x;  # output the 'x' position vector
 result: 
 -6.58393e-006 
 -6.5442e-006 
 -6.50447e-006  
 .....

Accessing data in a dataset: the dot '.' operator

Individual matrices stored in the dataset can be accessed with the dot '.' operator. For example, to get the raw x, y, and Ex data from an Electric field dataset, do something like:

x = E_field.x;
y = E_field.y;
Ex = E_field.Ex;

Dataset types

Matrix dataset - Data without spatial parameters (such as example 1) will use matrix datasets.

Rectilinear spatial dataset - Data with spatial information from a rectilinear grid (such as the FDTD mesh) will use rectilinear datasets.

Unstructured spatial dataset - Data with spatial information defined by an arbitrary x/y/z coordinates and a connectivity matrix (such as finite element mesh in Device)

Arbitrary unstructured dataset - Any data type (such as matrix, string, dataset) can be added to struct objects.

Any data type (such as matrix, string, dataset) can be added to struct objects

Operations on datasets

Datasets are primarily intended to be a convenient way to manage and store a collection of related data. It is not possible to apply mathematical operations, such as addition, directly to dataset objects. Instead, the dot operator must be used to get the desired data into a matrix. The operation can then be applied to the matrix. You may choose to create a new dataset to store the result, or you may simply keep the result as a standard matrix.

Scalar and Vector Attributes

It is possible to add both scalar and vector attributes to datasets.

Scalar

The command

R.addattribute("R",reflection); # add reflection attribute

adds a scalar quantity 'R' to a dataset with the same name. To access the 'R' raw data, use:

R.R;

Vector