Centre d'apprentissage

Creating a Multiphysics-Driven Deep Neural Network Surrogate Model

---

To illustrate the process of creating a surrogate model for a multiphysics simulation and parametric CAD model, we will create a surrogate model for the displacement of a thermal actuator. In this course, we will ultimately develop a surrogate model that captures virtually all aspects of the thermal actuator. For now, we will begin with a basic surrogate model that requires minimal data. Despite its simplicity, this initial physics-based example will help you better understand the process of developing this type of surrogate model in COMSOL Multiphysics^®. Additionally, this example shows a technique for generating a response surface using a deep neural network.

Thermal Actuator Model Overview

Start by loading the thermal actuator tutorial model from the Application Libraries in the software. Multiple versions of the model are available under the COMSOL Multiphysics > Multiphysics branch. Select and open the simplified model version.

A screenshot of the COMSOL Multiphysics UI with the Application Libraries window open and the thermal actuator simplified tutorial model selected, which displays the tutorial model name and an image of the model along with several details and properties regarding the mode.

The thermal actuator tutorial model in the Application Libraries.

Note that this model is also featured extensively, in different variations, in the Learning Center course "Defining Multiphysics Models".

The thermal actuator design includes two hot arms made of polysilicon, as shown in the figure below.

The thermal microactuator geometry with the parts labeled. The device is made of polycrystalline silicon.

The actuator is activated through thermal expansion. Joule heating (resistive heating) is used to achieve the temperature increase needed to deform the two hot arms and consequently displace the actuator. The greater expansion of the hot arms, compared to the cold arm, causes the actuator to bend. The material properties of polysilicon vary with temperature, leading to fully coupled physics. As electric current flows through the hot arms, it raises the actuator's temperature, triggering thermal expansion. At the same time, the electrical conductivity changes with temperature and the current flowing through the hot arms changes accordingly, in turn affecting the heating and the thermal expansion. Actuator operation thus involves three coupled physics phenomena:

1. Electric current conduction
2. Heat conduction with heat generation
3. Structural stresses and strains due to thermal expansion

The operation of the device is also detailed extensively in the course on defining multiphysics models.

The Parameters node contains the parameters of the model, including geometric dimensions such as length, width, and the applied voltage, as shown in the figure below. These are the parameters that we can use as a basis for creating surrogate models.

The Model Builder with the Parameters 1 node selected and the corresponding Settings window, which contains a tabular list of parameters used in the model.

The parameters driving the thermal actuator model.

Defining the Surrogate Model

Let's now assume that we are interested in creating a surrogate model for predicting the variation in the maximum displacement of the tip of the actuator. Furthermore, assume that we vary just two of the model parameters: the actuator length and the applied voltage, according to the following table:

Parameter	Min Value	Max Value	Unit	Variable Name
Actuator length	150	400
Applied voltage	0.5	10

As output quantity, we have the maximum displacement at the tip:

Quantity of Interest	Type	Unit	Surrogate Model Function Name
Maximum displacement	Scalar

Abstractly speaking, we are defining one surrogate model function:

where and are the input parameters, representing the length and applied voltage, respectively, and is the maximum displacement at the tip.

The Surrogate Model Training Study

In theory, we could perform a parametric sweep over both input parameters to generate a densely sampled dataset in this simple case. However, in many practical scenarios, this approach is computationally prohibitive. Instead, the parameter space is typically sampled sparsely, requiring surrogate models to accurately approximate unsampled values. This is the reason for using design of experiments (DOE) methods in the data generation process for data-driven surrogate models. DOE methods efficiently explore the parameter space, capturing the essential variations necessary for accurate model approximation. We will begin the process of creating a surrogate model by using a Surrogate Model Training study, which will take care of the DOE sampling for us.

To begin with this study, right-click the Study node and select Surrogate Model Training.

Adding the Surrogate Model Training study.

The Surrogate Model Training study has a number of settings, and the first one we will pay attention to is the Quantities of Interest.

The Surrogate Model Training study Settings window.

We want to provide the maximum displacement as the single quantity of interest. However, we haven't yet created a variable that we can use for this purpose. We will do so through the use of a Point Probe. Right-click the Definitions node, and under Probes, choose Point Probe.

Selecting Point Probe for accessing the maximum displacement at a point.

In the settings for the Point Probe node, under Source Selection, select Tip. This named selection is already defined in the Application Library model that we loaded. It contains a selection of a point at the tip of the actuator.

The Source Selection for the Point Probe feature.

A close-up of the thermal microactuator geometry where the point selected for the Point Probe feature displays a green color with an arrow pointing to it.

The location of the point probe where the maximum displacement is measured.

The default expression that is monitored by the probe is the voltage, V. We will now change this to the magnitude of the displacement. Use the Replace Expression button and menu to locate the expression for Solid Mechanics > Displacement > Displacement magnitude, or enter solid.disp in the Expression field.

The displacement expression used for the Point Probe.

You may notice that the Type setting of the Point Probe is Average. You can also choose Maximum, Minimum, or Integral. However for a Point Probe, they all generate the same result (the average of the value at a point is the value at that point, etc.).

Next, in the Surrogate Model Training node Settings window, click the Add button to add a row to the table for Quantities of interest. Then, type comp1.point1 for the value of the Point Probe belonging to the component comp1.

The Point Probe used as the quantity of interest.

Alternatively, you can use Ctrl+Space in the Expression field to autocomplete entering the expression and browse to the Point Probe, as shown in the figure below.

Browsing to the expression for the Point Probe using code completion.

We are now ready to specify the parameters to vary in the Input Parameters section. Use the Add button in the Input Parameters table to add the parameters L (Actuator length) and DV (Applied voltage). For the parameter L (Actuator length), set 150 for the Lower bound and 400 for the Upper bound, with the unit um (for micrometers). For the parameter DV (Applied voltage), set 0.5 for the Lower bound and 5 for the Upper bound, with the unit V. To set the bounds, you first click on the corresponding row in the Input Parameters table. This will make the fields for Lower bound, Upper bound, and Unit available under the table. Make sure that the length unit of the parameter for the actuator length is set to um, corresponding to micrometers.

The input parameters settings for the Surrogate Model Training study.

Before we start generating the data by running the study, we need to define how many data points we want. The number of data points is another term for the number of parameter tuples generated by the study by solving the full finite element model (or other numerical methods used for the model at hand). It is defined by the Number of input points setting in the Input parameters samplings settings section. Change this setting from the default 20 to 50. This will solve for 50 parameter value tuples for the length and applied voltage with values determined by the underlying DOE method.

The Number of input points setting, which determines the number of times the model is solved.

Now it is time to run the computation, which will solve the model 50 times. Right-click the Study node and select Compute. The computation will take about 5 minutes on a standard workstation.

Information on the data generation is displayed in an Accumulated Probe Table as well as a Design Data table, as shown below.

A screenshot of the Messages/Progress/Log window section of the COMSOL Multiphysics UI with the Data Design table displayed.

The Design Data table showing the generated data points.

Note that during the computation you can switch between different tables by selecting from the Display menu button in the Table window toolbar. When the computation is finished, the Design Data table contains the data we need to train a surrogate model. Note that the Surrogate Model Training study gives you the option to define an empty surrogate model automatically after the computation. However, the default is to define it manually. This is controlled by the Surrogate model setting Design of experiments (No surrogate model). In this example, we will not change this setting but will define the surrogate model manually.

The generated table data only supports the base unit system, which is the SI unit system in this case. We will need to compensate for this later on when we train the surrogate model.

Building the Surrogate Model

To add the surrogate model, right-click the Global Definitions node and under Functions select Deep Neural Network.

Selecting the Deep Neural Network option.

For this surrogate model we can get by with a very small network. Use the Add button under the table to create a network, according to the figure below. For the Data source, select Results table and choose Design Data as the Results table. As described in Part 2, the last layer's Output features setting doesn't need to be changed but is implicitly defined by the number of outputs, or quantities of interest, which in this case is 1, the maximum displacement.

The DNN layer architecture.

Since the Design Data table uses the centralized unit system, in this case SI units (m and V), we will define these units in the Data Column Settings, as shown in the figure below.

Defining SI units for the surrogate model.

In the Training and Validation section, change the Number of epochs to 40,000 and click Train Model at the top of the Deep Neural Network window to start the training. The Number of epochs value is decided by trial-and-error starting from the default value of 1000.

The Training and Validation settings.

The convergence plot for the DNN training showing the loss for the first 6000 epochs.

The training process is quick in this case since the network is very small. After about 38,000 epochs we see a tendency toward overfitting, so anywhere around this point is a good place to stop the training. Recall that we can identify when overfitting sets in by the fact that the training loss keeps decreasing while the validation loss starts increasing.

A line graph containing a blue line and a green line, which both exhibit erratic spikes and dips in the line, with the epoch number on the x-axis and loss value on the y-axis.

The convergence plot showing the loss for the last epochs.

The lowest value of the validation loss is obtained at about 37,000 epochs, and this is the value that will automatically be used by the surrogate model function.

The Information section showing the DNN training results.

We can now plot the surrogate model by selecting Create Plot in the Deep Neural Network window toolbar.

The Model Builder with the Function 1 plot node selected and the corresponding Settings window shown and plot visualization displayed.

A visualization of the DNN function.

Now, you can change the units for the surrogate model function to µm, as shown in the figure below.

The unit setting for plotting the DNN function.

Note, however, that we cannot change the unit for the x-axis due to the fact that the surrogate model is trained on the data in the Design Data table, which is defined in SI units. So, we will keep the unit for the length parameter input argument set to meters.

The DNN function with the height scaled to micrometers.

Comparing the DNN Function with a Linear Interpolation Function

Similar to the example in Part 2, and the fact that we only have two input parameters and one quantity of interest, we can easily visually and quantitatively compare the surrogate model function to an interpolation function.

To perform the comparison, right-click Results and add a 2D Plot Group. The Dataset setting in this plot group now points to a Grid 2D dataset that was automatically added when we selected Create Plot in the Deep Neural Network node Settings window. Note that the Grid 2D dataset only supports length units — meters, in this case — so we need to keep in mind that the axis for x2 is a voltage quantity. Due to this limitation, we will keep all unit settings to their defaults.

Right-click the new 2D Plot Group node and add a Surface plot. In the Expression field, type dnn1(x1,x2) and click Plot.

Now, let's add a linear interpolation function to compare with.

Right-click the Global Definitions node and under Functions select Interpolation. In the Interpolation node Settings window, change the Data source to Result table and make sure the Table from setting is set to Design Data. Change the Number of arguments to 2. This will allow the interpolation function to take the length and voltage as input arguments and output the displacement.

The interpolation function based on the Design Data table.

In the Settings window for the interpolation function, click Create Plot.

"The Model Builder with the 2D Plot Group 11 node selected and the corresponding Settings window and plot displayed in the Graphics window.

A visualization of the table data as a linear interpolation function.

Now, to compare with the DNN function, first navigate to Grid 2D 1. Change the Function setting from Deep Neural Network 1 to All. This will allow the visualization of any of the functions available under Global Definitions.

Next, locate the plot group with the Surface plot of the DNN function. In this case, it is 2D Plot Group 10. Right-click 2D Plot Group and select Surface. In the Expression field, type int1(x1,x2). In the Surface window, click Plot. This will generate two superimposed plots.

In order to separate the plots, we will add a Height Expression node. Right-click Surface 1 and select Height Expression. Change the Scale factor to 1e6.

The Scale factor setting in the Settings window for the Height Expression node.

Next, right-click Surface 2 and select Height Expression. Change the Scale factor to 1e6. This gives the two function plots the same height scaling, which enables a proper comparison. In Surface 2 > Height Expression > Settings window, clear the checkbox for Show height axis. For the Color table setting in the Surface 2 window, change the option to GrayScale. This produces the result shown in the figure below.

The Model Builder with the Height Expression node selected and the corresponding Settings window and plot in the Graphics window. The Graphics window shows a rectangular-shaped surface plot in 3D space that partially displays a rainbow color distribution from dark red to dark blue and partially displays a grayscale color distribution from white to black.

The DNN function plot compared to the corresponding linear interpolation function plot.

We see that the two functions are close in their values. To further analyze their differences, you can evaluate or visualize the expression abs(dnn1(x1,x2)-int1(x1,x2)) using a Surface plot in a new plot group. You can try this on your own. In cases where only two input arguments are involved, an alternative approach would be to use a dense Parametric Sweep method. However, this method becomes increasingly costly for surrogate models with more input arguments, as the computational expense increases with the number of inputs.

In the next part, we will discuss using a Gaussian Process surrogate model, available in the Uncertainty Quantification Module, highlighting the pros and cons of such a surrogate model.