2.1.1. Estimation and reconstruction

The problem of estimating unknown or previously unmeasurable states based on limited information is ubiquitous in fluid mechanics. Denoting the input data as ${\boldsymbol q}_{\textit{in}}$ and the output state as ${\boldsymbol q}_{\textit{out}}$ , the estimation task is framed as finding an accurate machine-learning model $\mathcal F$ that satisfies

(2.1) \begin{align} {\boldsymbol q}_{\textit{out}} = {\mathcal F}({\boldsymbol q}_{\textit{in}}; {\boldsymbol w}), \end{align}

where $\boldsymbol w$ represents the trainable parameters (weights and biases) of the network.

The weight distribution $\boldsymbol w$ needs to be tuned to establish the relationship in (2.1). In response, the mathematical optimisation problem to find the optimal weights ${\boldsymbol w}^*$ is generally solved by minimising a predefined loss function $\mathcal E$ across the available training data in a supervised manner. This optimisation is formulated as

(2.2) \begin{align} {\boldsymbol w}^* =\operatorname *{argmin}_{\boldsymbol w} {\mathcal E}({\boldsymbol w}), \end{align}

with, for example, the $L_{\kern-1.5pt P}$ norm of the residual error,

(2.3) \begin{align} {\mathcal E} = ||{\boldsymbol q}_{\textit{out}}-{\mathcal F}({\boldsymbol q}_{\textit{in}}; {\boldsymbol w})||_P. \end{align}

Once this optimisation is solved, the trained model $\mathcal F$ can be deployed to efficiently estimate or reconstruct the desired fluid states from new, unseen input data.

Typical examples for estimation in fluid mechanics are summarised in table 1. Some of them are inverse problems that aim to reconstruct the entire field from limited sensor data. Alternatively, instead of reconstructing the full field directly, estimation can be combined with compression techniques such as POD, dynamic mode decomposition and autoencoders. In this approach, low-dimensional coefficients are estimated from the sensors and then combined with spatial modes during post-processing to reconstruct the field, which will be covered in the present case study in § 3. From an image processing perspective, state estimation encompasses super-resolution analysis, as well as deriving high-resolution vector fields directly from experimental particle images. Furthermore, other problem settings, such as, estimating an entire three-dimensional volumetric field from limited two-dimensional cross-sections (e.g. measured by PIV), and estimating off-wall flow states at a distance using wall-mounted sensors, also belong to the group of state estimation. To address these diverse problem set-ups, various machine-learning methods have been proposed depending on the complexity and degrees of freedom of the target flow data.

Table 1.

Example combinations of input ${\boldsymbol q}_{\textit{in}}$ and output ${\boldsymbol q}_{\textit{out}}$ for data-driven state estimation in fluid mechanics.

Table 1. Long description

A table with three columns: Input, Output, and Example references. The table has seven rows, each describing different combinations of input and output for state estimation in fluid mechanics. Row 1: Input, Entire flow field; Output, Entire flow field; Example references, Erichson et al. (2020), Fukami et al. (2021c), Maulik et al. (2020), Santos et al. (2023). Row 2: Input, Sensors; Output, Mode coefficients (Field approximation); Example references, Nair & Goza (2020), Dubois et al. (2022), Carter et al. (2021), Mousavi & Eldredge (2025). Row 3: Input, Low-resolution field; Output, High-resolution field; Example references, Fukami et al. (2019a), Kim et al. (2021), Guémes et al. (2022), Page (2025). Row 4: Input, PIV particle images; Output, High-resolution vector field; Example references, Cai et al. (2019), Morimoto et al. (2021a), Majewski et al. (2020), Masker et al. (2025). Row 5: Input, Gappy field; Output, Reconstructed field; Example references, Bui-thanh et al. (2004), Buzzicotti et al. (2021), Li et al. (2023), Zheng et al. (2024). Row 6: Input, 2-D measured fields; Output, 3-D volumetric field; Example references, Huang et al. (2020), Pérez et al. (2022), Matsuo et al. (2024), Xuan & Shen (2023). Row 7: Input, Wall sensors; Output, Off-wall flow states; Example references, Guastoni et al. (2021), Morimoto et al. (2022), Nakamura et al. (2022).

2.1.2. Temporal prediction

Predicting the dynamical evolution of a flow field is essential for tasks such as weather forecasting and flow control. The goal is to predict the future state of a fluid system based on its past and present states. This is formulated as an autoregressive problem where the model $\mathcal F$ predicts the state at the next time step ${\boldsymbol q}(t+n\Delta t)$ given the state at the current time step ${\boldsymbol q}(t)$ such that

(2.4) \begin{align} {\boldsymbol q}(t+n\Delta t) = {\mathcal F}({\boldsymbol q}(t); {\boldsymbol w}), \end{align}

where $n$ denotes the number of time steps. The model can be recursively applied,

(2.5) \begin{align} {\boldsymbol q}({t+k\Delta t}) = \underbrace {{\mathcal F}({\mathcal F}(\ldots {\mathcal F}}_{k \text{ times}}({\boldsymbol q}(t); {\boldsymbol w})\ldots ; {\boldsymbol w}); {\boldsymbol w}), \end{align}

for predicting the temporal evolution of fluid flows over multiple $k$ steps.

Furthermore, the temporal prediction can be alternatively performed in a compressed latent space to reduce computational cost, rather than in the high-dimensional ambient space. With the latent representation ${\boldsymbol \xi }(t)$ of the flow field, the latent dynamics model ${\mathcal F}_{\textit{latent}}$ solves,

(2.6) \begin{align} {\boldsymbol \xi }(t+n\Delta t) = {\mathcal F}_{\textit{latent}}({\boldsymbol \xi }(t); {\boldsymbol w}), \end{align}

so that a flow field at the next step ${\boldsymbol q}(t+n\Delta t)$ is reconstructed from the predicted latent vector ${\boldsymbol \xi }({t+n\Delta t)}$ through a decoder prepared with other data-driven techniques, such as POD and an autoencoder. As introduced with figure 9, machine-learning-based temporal prediction has been examined for a range of flow examples, including bluff body wake (Lee & You Reference Lee and You2019), wall-bounded turbulence (Yousif et al. Reference Yousif, Zhang, Yu, Vinuesa and Lim2023b ), homogeneous isotropic turbulence (Mohan et al. Reference Mohan, Tretiak, Chertkov and Livescu2020), aerodynamic flows (Lui & Wolf Reference Lui and Wolf2019; Liu et al. Reference Liu, Beckers and Eldredge2025b ) and urban models (Jaroslawski et al. Reference Jaroslawski, Patil and McKeon2025).

This type of data-driven temporal prediction model is often discussed as a class of surrogate models. By taking some parameter inputs in addition to the current state vector in (2.5) and (2.6), they can be parameterised in constructing a surrogate model. As they approximate the spatiotemporal behaviour of fluid flows at a reduced cost compared with numerical simulations and wind-tunnel experiments, they are particularly useful when considering a huge parameter space. Even if this does not need to be a complete surrogate, such approximations can guide us to figure out which cases are worth being numerically simulated or experimentally measured.

2.1.3. Compression

Data compression or dimensionality reduction is crucial in fluid mechanics to gain dominant flow characteristics in a compact form. The compressed representation not only facilitates understanding the flow physics but also supports downstream tasks such as reduced-complexity modelling and flow control. The general framework for compression consists of an encoding process that compresses given high-dimensional fluid data ${\boldsymbol q}\in \mathbb{R}^{n_{\boldsymbol q}}$ into a latent space, and a decoding process that reconstructs the data from the compressed representation ${\boldsymbol \xi } \in \mathbb{R}^{n_{\boldsymbol \xi }}$ .

The compressed data dimension, often referred to as the latent dimension, is set to be much smaller than the original data dimension such that $n_{\boldsymbol \xi } \ll n_{\boldsymbol q}$ . This compression and the subsequent reconstruction are expressed as

(2.7) \begin{align} {\widehat {\boldsymbol q}} = {\mathcal F}_d ({\mathcal F}_e({\boldsymbol q})) = {\mathcal F}_d ({\boldsymbol \xi }), \end{align}

where ${\mathcal F}_e$ and ${\mathcal F}_d$ are an encoder and a decoder, respectively.

The primary goal of compression approaches is to retain as much of the original information as possible. This is achieved by minimising the reconstruction error between the original data $\boldsymbol q$ and the reconstructed data $\widehat {\boldsymbol q}$ . Analogous to the estimation/reconstruction problem, the encoding and decoding mappings are parameterised by weights $\boldsymbol w$ that are hence optimised in a manner,

(2.8) \begin{align} {\boldsymbol w}^* = \operatorname *{argmin}_{\boldsymbol w} ||{\boldsymbol q}- {\mathcal F}_d ({\mathcal F}_e({\boldsymbol q}; {\boldsymbol w}_e); {\boldsymbol w}_d)||_2, \end{align}

where ${\boldsymbol w} = \{{\boldsymbol w}_e, {\boldsymbol w}_d\}$ represents the combined set of trainable parameters for the encoder and decoder. When the above optimisation is successful, the latent variable $\boldsymbol \xi$ serves as a low-dimensional latent representation of the original flow data $\boldsymbol q$ . The specific forms of ${\mathcal F}_e$ and ${\mathcal F}_d$ depend entirely on the chosen methodology. For instance, classic modal decomposition techniques widely used in fluid mechanics, such as POD and dynamic mode decomposition, essentially function as linear encoders and decoders.

From a nonlinear machine-learning perspective, an autoencoder based on nonlinear models can be used for fluid flow data compression, often achieving better compression than the linear counterpart (Fukagata Reference Fukagata2023; Fukagata & Fukami Reference Fukagata and Fukami2025). While the encoder ${\mathcal F}_e$ and the decoder ${\mathcal F}_d$ are often structured with neural networks, a combination of various data-driven techniques for the encoder and the decoder can be considered. For example, the isomap-based encoder (Tenenbaum et al. Reference Tenenbaum, de Silva and Langford2000) and the $k$ -nearest-neighbour-based decoder (Fix & Hodges Reference Fix and Hodges1989) are used for a range of controlled scenarios of fluidic pinball flows (Marra et al. Reference Marra, Maceda, Meilán-Vila, Guerrero, Rashwan, Noack, Discetti and Ianiro2024). A vision transformer-based encoder combined with a convolutional decoder is also considered to extract low-dimensional features of fluid flows (Fan et al. Reference Fan, Cheng, de Nazelle and Arcucci2025). By optimising the weights of the network through the process described in (2.8), autoencoders capture a compact representation of fluid flows, which can involve information on multiple linear modes using a smaller number of nonlinear modes (Fukami et al. Reference Fukami, Nakamura and Fukagata2020b ; Murata et al. Reference Murata, Fukami and Fukagata2020), thereby achieving a higher compression ratio.

2.1.4. Generation

While standard regression and compression approaches require pairs of input and output data for training, there exists a different optimisation principle that learns the underlying probability distribution of the data. This enables plausible data generation by sampling points on the learned probability distribution, which is particularly useful in fluid mechanics tasks such as inflow turbulence generation (driver or precursor) for numerical simulations (Kim & Lee Reference Kim and Lee2020; Yousif et al. Reference Yousif, Yu and Lim2022; Liu et al. Reference Liu, Parikh, Fan, Du, Wang, Chen and Wang2025a ) and aerofoil shape generation from given aerodynamic constraints (Li et al. Reference Li, Zhang, Martins and Shu2020; Wang et al. Reference Wang, Shimada and Barati Farimani2023b ).

A prominent example of the algorithm is the generative adversarial network (GAN; Goodfellow et al. Reference Goodfellow, Pouget-Abadie, Mirza, Xu, Warde-Farley, Ozair, Courville and Bengio2020). A GAN consists of two competing networks: a generator ( $G$ ) and a discriminator ( $D$ ). The generator $G$ aims to create realistic ‘fake’ output data $G({\boldsymbol n})$ from random noise $\boldsymbol n$ (or a latent input). In contrast, the discriminator $D$ is trained to distinguish between ‘real’ data $\boldsymbol d$ from the actual data set and the ‘fake’ data produced by the generator, outputting a probability between 0 and 1.

This competition is formulated as a min–max optimisation problem,

(2.9) \begin{align} \underset {G}{\min } \underset {D}{\max } V(D,G) = \mathbb{E}_{{\boldsymbol d} \sim p_{{data}}({\boldsymbol d})}[\log D({\boldsymbol d})] + \mathbb{E}_{{\boldsymbol n} \sim p_{\boldsymbol n}({\boldsymbol n})}[\log (1-D(G(\boldsymbol n)))], \end{align}

where $V(D,G)$ is called the value function. The discriminator maximises this objective function to correctly identify real and fake samples, while the generator minimises it to fool the discriminator. By formulating the problem in terms of statistical likelihood rather than pixel-wise residual minimisation, generative models produce statistically plausible flow structures (Bode et al. Reference Bode, Gauding, Lian, Denker, Davidovic, Kleinheinz, Jitsev and Pitsch2021). Note that the GAN is treated here as a training framework rather than a specific network architecture. This is because GANs can be constructed using a variety of underlying models, including MLPs and CNNs, as detailed in § 2.3. Other generative approaches, such as the diffusion model, will also be covered later in § 2.3. Analogous to temporal prediction models, generative models can also be parameterised by taking some parameters in addition to the input vector.

2.1.5. Action design (reinforcement learning)

In active flow control and aerodynamic shape optimisation, the problem shifts from passive prediction to active decision making. Reinforcement learning provides a framework for this by modelling the interaction between an agent (the controller) and an environment (the flow system) (Vincent et al. Reference Vincent, Peter, Riashat, Marc and Joelle2018). The reinforcement-learning problem is formulated as a Markov decision process. At each time step $t$ , the agent observes the state of the flow ${\boldsymbol s}_t$ (e.g. sensor readings), takes an action ${\boldsymbol a}_t$ (e.g. blowing/suction on the body surface) and receives a scalar reward $r_t$ (e.g. drag reduction) from the environment. The behaviour of the agent is defined by a policy $\pi _{\boldsymbol w}({\boldsymbol a}_t \vert {} {\boldsymbol s}_t)$ . In practice, this policy is typically approximated by a neural network where $\boldsymbol w$ represents the learnable weights and biases. As the state vector ${\boldsymbol s}_t$ propagates through the network, these weights govern the nonlinear transformations that map the flow information to the parameters of the action probability distribution, e.g. the mean and variance for continuous control.

The mathematical objective in action design is to find the optimised policy parameters ${\boldsymbol w}^*$ that maximise the expected cumulative reward over a trajectory of length $T$ ,

(2.10) \begin{align} {\boldsymbol w}^* = \operatorname *{argmax}_{{\boldsymbol w}} \mathbb{E}_{\pi _{{\boldsymbol w}}} \left [ \sum _{t=0}^T \gamma ^t r_t \right ]\!, \end{align}

where $\gamma \in [0, 1)$ is a discount factor that prioritises immediate rewards over future ones. By formulating flow control as a reinforcement-learning problem, we may discover complex, nonlinear control strategies that are challenging to derive using traditional linear control theories (Sonoda et al. Reference Sonoda, Liu, Itoh and Hasegawa2023; Font et al. Reference Font, Alcántara-Ávila, Rabault, Vinuesa and Lehmkuhl2025).

2.2.1. Data characteristics

A wide spectrum of formats, qualities and sizes is considered for fluid flow data. For instance, time-series data measured through surface-mounted sensors typically possess a much lower dimensionality compared with entire flow field data, often enabling the use of relatively simple machine-learning architectures such as MLPs. In contrast, the degree of freedom for flow field data acquired via computational fluid dynamics or experiments such as Schlieren photography and PIV is much larger than sensor readings.

Even under the same category of data source, the data volume and its quality become different. In PIV, the data volume and quality differ significantly depending on whether raw particle images or post-processed velocity fields are considered. For numerical simulations, the temporal axis of the data matrix depends entirely on the simulation method. Steady-state Reynolds-averaged Navier–Stokes simulations yield time-independent data, whereas direct numerical simulations, large-eddy simulations or detached eddy simulations can generate unsteady, time-resolved data sets. The spatiotemporal fidelity of fluid flow data also depends on the simulation approach.

Furthermore, the stored variables, such as velocity, pressure and vorticity, determine the data shape, which also depends on whether the flow is incompressible or compressible. The choice of computational grid, e.g. uniform, non-uniform or unstructured, and how flow fields on them are saved are also important. For certain architectures such as CNNs, it is sometimes advantageous to interpolate or extrapolate unstructured mesh data onto a Cartesian grid, transforming it into an image or video format to simplify subsequent processing. However, such interpolations carry the risk of obscuring the underlying flow physics. To prevent this, one can consider designing the model to directly handle unstructured data, such as point cloud networks or graph neural networks (GNNs). Which of the aforementioned variables and data structures should be considered depends highly on the problem setting and the nature of the fluid flows of interest. In other words, such prior knowledge of fluid mechanics needs to be appropriately involved in the data set preparation.

Figure 12.

Data preparation for machine-learning analysis of fluid flows. $(a)$ Time series of sensor readings or probe data is often handled as a matrix $(n_{{sensor}} \times n_t)$ . A series of flow snapshots can be treated with $(b)$ a matrix ( $n_{{grid}} \times n_t$ ), or $(c)$ a higher-dimensional tensor (e.g. $n_t \times n_x \times n_y \times n_z \times n_{{variable}}$ ), depending on the machine-learning model chosen for the analysis.

2.2.2. Data shape

Let us then consider what data shape we often consider for data-driven analysis in fluid mechanics. When processing time-series data from sensors or probes derived from simulations or experiments, a two-dimensional matrix is typically constructed, as illustrated in figure 12 $(a)$ . In this formulation, the vertical axis denotes the sensor index, corresponding to the ‘features’ in standard machine-learning nomenclature (Goodfellow et al. Reference Goodfellow, Bengio and Courville2016). The horizontal axis represents time, which represents the ‘volume of training data.’ This matrix structure serves as the standard training data format for linear regression or MLP-based analyses of problems with relatively low degrees of freedom.

An example considered in figure 3, estimating the lift coefficient $C_L(t)\in {\mathbb R}$ from the sensor readings ${\boldsymbol s}(t)\in {\mathbb R}^5$ , has indeed used this matrix-based data preparation. Given the relatively low dimensionality of the input–output space, a nonlinear regression via a MLP is an appropriate choice, which will be discussed later. The input matrix ${\boldsymbol Q}_{\textit{in}}$ based on the sensor time trace and the output matrix ${\boldsymbol Q}_{\textit{out}}$ of the aerodynamic coefficient take the form

(2.11) \begin{equation} {\boldsymbol Q}_{\textit{in}} = \begin{pmatrix} s_{1}(t_1) & s_{1}(t_2) & {\cdots} & s_{1}(t_n) \\[5pt] s_{2}(t_1) & s_{2}(t_2) & {\cdots} & s_{2}(t_n) \\[5pt] \vdots & \vdots & \ddots & \vdots \\[5pt] s_{5}(t_1) & s_{5}(t_2) & {\cdots} & s_{5}(t_n) \end{pmatrix} \in \mathbb{R}^{n \times 5}, \quad {\boldsymbol Q}_{\textit{out}} = \begin{pmatrix} C_{L}(t_1) \\[5pt] C_{L}(t_2) \\[5pt] \vdots \\[5pt] C_{L}(t_n) \end{pmatrix} \in \mathbb{R}^{n \times 1}, \end{equation}

where $n$ represents the number of snapshots over time. While this specific example estimates $C_L$ , the model can be easily extended to simultaneously provide the drag coefficient $C_D$ , which simply requires concatenating $C_L$ and $C_D$ to yield an output matrix ${\boldsymbol Q}_{\textit{out}} \in \mathbb{R}^{n \times 2}$ .

For managing two- or three-dimensional data sets of fluid flows, the same strategy as above can be considered, as illustrated in figure 12 $(b)$ . Consider a scenario where snapshots of an unsteady flow field are collected over time. Each of the multidimensional flow field snapshots at each discrete time step is flattened into an elongated one-dimensional column vector, which is then concatenated along the temporal axis, as depicted in figure 12 $(b)$ . The feature variables or different state variables can be populated in the vertical direction. This data matrix construction is used not only for nonlinear machine learning but also for linear modal analyses including POD (Kajishima & Taira Reference Kajishima and Taira2016).

While the aforementioned flattened data matrices are often used, flattening can cause the loss of spatiotemporal information or connectivity of data. Topological features, such as flow structures and their dynamical motion, are critical in fluid mechanics. To preserve such topological characteristics of fluid flows in the data, an alternative approach is to maintain the data in its native multidimensional format, as illustrated in figure 12 $(c)$ . This preparation is used for CNNs, which were originally developed for image and video processing, as introduced later.

Although the example in figure 12 illustrates the operation with vortical snapshots, one can analogously use velocity components ( $u, v, w$ ), treating them similarly to RGB channels in standard video data. For a three-dimensional unsteady flow, the resulting data shape becomes a five-dimensional tensor structured as $n_t$ (time) $\times$ $n_x$ $\times$ $n_y$ $\times$ $n_z$ $\times$ $n_{{variable}}$ (state variables). In parametric steady-state studies, the ‘time’ axis can simply be replaced by a parameter axis (e.g. geometry or Reynolds number) to consider the exact same data-preparation techniques. For problems where the input and output possess significantly different degrees of freedom, the techniques shown in figure 12 can be hybridised to suit the specific model architecture.

2.2.3. Data pre-processing

It is possible to estimate multiple variables simultaneously with a machine-learning model. However, we should be mindful of the potential disparity in their respective orders of magnitude, which can significantly skew estimation accuracy. Since the model is trained to minimise the cost function as discussed in § 2.1, the relative magnitude of the error would be biased toward the one with a larger order of magnitude if an unweighted error metric is applied to the raw data. This can happen in fluid mechanics, as the order of velocity magnitude highly depends on the primary flow direction, e.g. the streamwise velocity and others in a turbulent channel flow. Although the magnitude itself often represents important flow characteristics, it is sometimes necessary to scale the data while being careful not to lose its physical properties.

In response, fluid flow data can be pre-processed via min–max normalisation or standardisation to balance the order and distribution of the variables. They are operated as

(2.12) \begin{equation} q_{\textit{norm}} = \frac {q-q_{\textit{min}}}{q_{\textit{max}}-q_{\textit{min}}}, \quad q_{\textit{std}} = \frac {q-\mu _q}{\sigma _q}, \end{equation}

where $q_{\textit{norm}}$ and $q_{\textit{std}}$ are respectively normalised and standardised variables of $q$ . The min–max normalisation linearly scales the data such that the minimum value $q_{\textit{min}}$ maps to 0 and the maximum $q_{\textit{max}}$ maps to 1. When processing flow variables that possess negative physical values, this range is often adjusted to $[-1, 1]$ . In contrast, the standardisation centres the data around a mean $\mu _q$ of 0 and scales it to a standard deviation $\sigma _q$ of 1. Once the estimation or prediction is performed with the scaled variables, the output of the machine-learning model can be scaled back for downstream assessments.

For data sets heavily contaminated by outliers, other robust transformations, such as logarithmic scaling or the Box–Cox transformation, can be considered (Sakia Reference Sakia1992). Determining the optimal scaling technique that appropriately balances the data magnitudes without losing its fundamental physical characteristics is entirely dependent on the underlying probability distribution. However, we caution that there exist cases where data scaling should be avoided, since the magnitude and probability distribution that the data originally possess are important characteristics of fluid flows. Rather than blindly feeding raw data into machine-learning models, performing statistical analysis is mandatory for any machine-learning pipeline.

2.2.4. Data split

Once the data sets are prepared with special care for the specific problem setting, the flow physics of interest and the chosen model architecture, we then start to consider the training of the model. It is critical to appropriately partition the prepared data to ensure the capability of the trained model. Failing to perform this process often leads to overfitting, resulting in an ineffective model that performs well only on the training data. While various data-splitting techniques exist, this section discusses the $k$ -fold cross-validation while highlighting specific caveats associated with its application to fluid flow data.

Figure 13.

Data split for machine-learning analysis of fluid flows. $(a)$ The prepared data sets are first divided into training, validation and test sets. $(b)$ The relationship between the estimation error and the model complexity, monitored with the training and validation data. (c–f) Interpolation and extrapolation of machine-learning-based analysis for fluid flows. The test data preparation, such as shown in panels  $(c)$ and $(d)$ , needs to be carefully performed, depending on the flows of interest, e.g. $(e)$ transient, and $(f)$ statistically stationary flows.

Nonlinear machine-learning models are typically trained through an iterative process that minimises (or maximises) a cost function using algorithms such as backpropagation (Bishop & Nasrabadi Reference Bishop and Nasrabadi2006). Hence, the selection of data used to evaluate the error during this iterative training process is important. Here, let us consider a data partitioning approach that shuffles the available data set and prepares distinct subsets for specific purposes, as illustrated in figure 13 $(a)$ . The subsets coloured in blue represent the training data, which are directly used to update the model parameters. However, training should not rely only on them since the model will likely overfit, adopting too closely to the training data and losing its ability to generalise to unseen scenarios, as illustrated in figure 13 $(b)$ .

To prevent this, it is necessary to continuously monitor the performance of the model for unused data sets during training. This is achieved with a validation data set, highlighted in orange in figure 13 $(a)$ . During the training process, the validation data are fed into the updated model at the end of each iteration (or epoch) to track the trajectory of both training and validation errors. Excessive training leads to an increase in validation error, indicating a loss of generalisation (overfitting), while insufficient training results in underfitting, as illustrated in figure 13 $(b)$ . The optimised model can achieve a delicate balance between the two. To obtain such a balanced state, early stopping (Prechelt Reference Prechelt1998), which automatically stops the learning process and saves the model parameters when the validation error ceases to improve, is often considered.

Once the model is obtained using the training and validation sets, the test data, a green box in figure 13 $(a)$ , is used to evaluate the model performance on data excluded from the training process. As the model performance for the test data may be sensitive to how the data set was originally partitioned, it is common practice to perform $k$ -fold cross-validation, where the model is trained $k$ separate times with randomly partitioned data sets (Stone Reference Stone1974). This provides the average performance across the $k$ distinct test sets, thereby avoiding producing biased assessments for a particular test set.

When applying the data-splitting strategy to fluid flow data, however, further careful consideration based on the underlying flow physics is needed to appropriately train and assess machine-learning models. To discuss this point, a nuanced understanding of interpolation and extrapolation within the context of data-driven fluid mechanics is required. In general, a model exhibits its expected performance on data that has been ‘seen’ during training; this regime is termed interpolation. For ‘unseen’ data, the model error typically increases, which is termed extrapolation.

Defining these regimes is straightforward for low-dimensional tasks such as simple curve fitting. For example, data falling outside the numerical bounds of the training set is considered extrapolation. However, identifying the clear cut between interpolation and extrapolation for high-dimensional flow field data is challenging. In fact, there are a number of studies reporting that applying a nonlinear machine-learning model to a test data set with a slightly different Reynolds number than the training data can still be regarded as interpolation, i.e. an interpolatory use of the model, when the underlying physics or topological flow structures remain similar (Fukami et al. Reference Fukami, Goto and Taira2024a ). Nonlinear machine-learning models, particularly with some functions based on prior knowledge of flow physics, may capture scale-invariant physics embedded within the data across the spatial length scale, and their performance in these regimes can sometimes far exceed expectations based on traditional linear techniques (Taira et al. Reference Taira, Rigas and Fukami2025). Therefore, carefully defining the criteria by which a model has ‘seen’ or ‘not seen’ data is important when preparing test data sets.

Let us here examine relatively intuitive cases of interpolatory and extrapolatory data sampling with two distinct unsteady flow examples. We consider two scenarios of data preparation, as illustrated in figures 13 $(c)$ and 13 $(d)$ . The grey segments represent the combined training and validation data, while the red counterparts are used for the test data. In example $(c)$ , the training/validation and test data are randomly sampled over time. In contrast, example $(d)$ prepares the test data as a temporal extrapolation, i.e. forecasting into the future.

Consider applying these data-preparation strategies to a transient system, such as a vortex-gust encounter around an aerofoil, presented in figure 13 $(e)$ . In various problem settings, sampling data in a manner of figure 13 $(c)$ enables the model to comprehensively learn the temporal dynamics of the vortex-aerofoil interaction throughout the entire event over time. Thus, the red test data in figure 13 $(e)$ can be considered a physical interpolation. In contrast, the model trained with the data sampling strategy in chronological order, shown in figure 13 $(d)$ , may consider the later stages of the interaction as extrapolation due to the transient nature of the flow dynamics.

It is, however, worth pointing out that such extrapolatory sampling in time can sometimes functionally manifest as physical interpolation. This can happen for data-driven analysis of statistically stationary flow, such as an unsteady turbulent channel flow, shown in figure 13 $(f)$ . Although the instantaneous physical state chaotically evolves over time, its long-term statistics remain constant, possessing ergodicity (Walters Reference Walters2000). For this type of flow, even test data sampled entirely from the future via an approach in figure 13 $(d)$ may still be regarded as ‘seen’ data from the perspective of the underlying statistical distributions. Nonlinear machine-learning analyses have indeed leveraged these characteristics for various problem settings, including turbulence modelling and flow reconstruction (Vinuesa & Brunton Reference Vinuesa and Brunton2022). When preparing test data and evaluating model performance on them, we should be mindful of what results are anticipated from the aspect of data preparation for training and test phases. This becomes particularly important when the expected outcomes are not derived through machine-learning studies, so that the improvements can be considered via data-preparation perspectives based on flow physics.

2.3.1. Multi-layer perceptron

The MLP (Rumelhart et al. Reference Rumelhart, Hinton and Williams1986) is the foundational network architecture, often referred to as a feedforward neural network. The basic concept of MLP is motivated by the structure of biological neural circuits. The model maps an input state vector to an output through a sequence of fully connected hidden layers. The transition of a feature node $i$ from layer $l$ to layer $l+1$ is expressed as

(2.13) \begin{align} h_i^{(l+1)} = \varphi \left ( \sum _{j=1}^{N} W_{ij}^{(l)} h_{\!j}^{(l)} + b_i^{(l)} \right )\!, \end{align}

where $h_{\!j}^{(l)}$ represents the input nodes or units, $W_{ij}^{(l)}$ is the weight matrix connecting every input $j$ to every output $i$ , $b_i^{(l)}$ is the bias and $\varphi$ is an activation function.

Multi-layer perceptrons operate on a pointwise basis while the formulation does not consider spatial topography or temporal sequencing. It is hence highly effective for mapping low-dimensional, local state variables, such as sparse sensor readings and aerodynamic coefficients (Fukami et al. Reference Fukami, Fukagata and Taira2020a ). In addition, MLP-based models have been examined for closure modelling in Reynolds-averaged Navier–Stokes simulations (Duraisamy et al. Reference Duraisamy, Iaccarino and Xiao2019). For turbulence modelling, multiple MLP networks with a specific physics-based constraint, the so-called tensor-basis neural network, are often considered (Ling et al. Reference Ling, Kurzawski and Templeton2016; Tsai et al. Reference Tsai, Wang, Cheng, Hung, Chou, Li, Tseng, Chern and Chou2026). The tensor-basis neural network guarantees the Galilean invariance of the estimated anisotropy tensor by using specialised layers with an invariant tensor basis, which becomes a standard approach in machine-learning-based studies of turbulence modelling (Wu et al. Reference Wu, Xiao and Paterson2018; Zhao et al. Reference Zhao, Akolekar, Weatheritt, Michelassi and Sandberg2020).

Multi-layer perceptrons also serve as the backbone for physics-informed neural networks (PINNs; Raissi et al. Reference Raissi, Perdikaris and Karniadakis2019), where the network takes spatial coordinates and time as input to estimate local flow variables. Physics-informed neural networks, often composed of MLPs, use automatic differentiation (Baydin et al. Reference Baydin, Pearlmutter, Radul and Siskind2018) to penalise the solution by minimising the cost function based on the Navier–Stokes equations, which will be discussed later. In addition to the velocity components, they can provide a local estimate of the pressure variable without the need for pressure Poisson inversions and their associated boundary conditions.

2.3.2. Recurrent neural network/ LSTM

A class of neural networks designed to process sequential data by maintaining a hidden state that carries historical information forward in time is called recurrent neural networks (RNNs). The general formulation of recurrent networks is to predict the hidden vector state $h_t$ based on the current input $x_t$ and the previous hidden state $h_{t-1}$ ,

(2.14) \begin{align} h_t = \varphi (W_{hx} x_t + W_{hh} h_{t-1} + b_h), \end{align}

where $W_{hx}$ denotes the weights between the hidden state and the input, and $W_{hh}$ expresses the weights between the hidden states over time. As hinted by (2.14), this formulation enables the model to account for historical information in predicting a variable.

While standard RNNs could be considered for turbulent flow forecasting, it has also been reported that they often suffer from vanishing gradients in the model learning process, as gradients across long-time sequences need to be propagated inside the model, thereby causing difficulty in stably learning the dynamics (Srinivasan et al. Reference Srinivasan, Guastoni, Azizpour, Schlatter and Vinuesa2019). In response, LSTM (Hochreiter & Schmidhuber Reference Hochreiter and Schmidhuber1997), an RNN with a sophisticated gating mechanism based on the input, forget and output gates, can also be used to regulate long-term memory in the learning process. The use of LSTMs has been widely seen for modelling the temporal evolution of unsteady flows not only in an ambient space but also in a reduced-order space such as the time-varying coefficients of POD and nonlinear autoencoder (Wan & Sapsis Reference Wan and Sapsis2018; Maulik et al. Reference Maulik, Lusch and Balaprakash2021).

Some recurrent networks that embed prior knowledge of flow physics into the model design have been proposed. Shown in figure 9 by Nakamura et al. (Reference Nakamura, Fukami, Hasegawa, Nabae and Fukagata2021) considers a parallel LSTM in predicting the temporal evolution of minimal turbulent channel flow in a reduced-order space identified through a nonlinear autoencoder. This parallel setting includes a number of LSTM models with different numbers of units, enabling the model to learn a range of temporal length scales in turbulence. Aiming to develop robust models against noisy measurements, Ahmed et al. (Reference Ahmed, Pawar, San, Rasheed and Tabib2021) proposed an LSTM nudging that incorporates sensor priors to stabilise the prediction of the two-dimensional vorticity transport problem. They considered combining LSTM-based prediction and data assimilation approaches that have been well established in unsteady flow forecasts (Zaki Reference Zaki2025).

Accurate instantaneous prediction of turbulence with such recurrent models is recognised as challenging due to error accumulation in recursively using the machine-learned model, as expressed in (2.5). However, it is worth pointing out that they are powerful in learning the long-term dynamics of chaotic systems, including turbulence. Some studies have argued that nonlinear machine learning may learn the attractor of the chaotic system, and the prediction can be stable unless an instantaneous prediction is not too far off from the learned attractor. As chaos itself can then automatically dump the offset of the machine-learned prediction, the resulting long-term prediction eventually resides in the vicinity of the original attractor (Linot & Graham Reference Linot and Graham2020).

2.3.3. Convolutional neural network

While MLPs and recurrent networks are suited for problem settings with relatively low-order input and output data, the application to cases with a high degree of freedom, such as flow field reconstruction, becomes computationally prohibitive due to the fully connected nature of the model. In addition, such fully connected models generally require providing data sets in a matrix form, as introduced in figure 12 $(b)$ , implying that the spatial arrangement of flow structures and their relative distance from boundaries may be lost before providing them to the model.

In such a scenario, a CNN (LeCun et al. Reference LeCun, Bottou, Bengio and Haffner1998) can be chosen. The convolutional processes introduced hereafter are inspired by the connectivity pattern of the visual cortex. Convolutional neural networks have been widely used in computer vision for image and video processing, while they have also been actively considered for a variety of fluid mechanics problems (Morimoto et al. Reference Morimoto, Fukami, Zhang, Nair and Fukagata2021b ).

Convolutional neural networks modify the dense matrix multiplication of MLPs by using discrete convolutional filter operations, which enforce sparse connectivity and weight sharing across the input space. While retaining the spatial arrangement of structures in a flow snapshot, the weight sharing enables the substantial reduction of computational costs in extracting the input–output relationship. Equipped with a similar notation to the MLP, but extended to account for $C$ input channels and $K$ output filters, the update for the $\kappa$ th feature map at a spatial node $i$ in a CNN is given by

(2.15) \begin{align} h_{i,\kappa }^{(l+1)} = \varphi \left ( \sum _{c=1}^{C} \sum _{j \in \mathcal{S}(i)} W_{i-j,c,\kappa }^{(l)} h_{j,c}^{(l)} + b_\kappa ^{(l)} \right )\!, \end{align}

where a localised spatial neighbourhood is expressed with $\mathcal{S}(i)$ . Unlike the MLP, whose every node is connected by a unique weight $W_{ij}$ , the CNN node $i$ only aggregates information from a localised spatial neighbourhood $\mathcal{S}(i)$ across all input channels $c$ . Furthermore, the weight tensor $W_{i-j,c,\kappa }^{(l)}$ depends only on the relative spatial offset $i{-}j$ , making the operation translation invariant, while a shared bias $b_\kappa ^{(l)}$ is applied per output filter. This makes CNNs intrinsically suited for structured grids on Eulerian frames of reference (Fukami et al. Reference Fukami, Maulik, Ramachandra, Fukagata and Taira2021c ).

The multi-scale nature of turbulent flows can be embedded into CNNs by employing multiple parallel filter sizes to simultaneously resolve large-scale energy-containing motions and small-scale dissipative eddies (Fukami et al. Reference Fukami, Fukagata and Taira2019a ). The filter size, corresponding to the area coverage of the kernel $\mathcal{S}(i)$ , can also be determined based on characteristic length scales (Zheng et al. Reference Zheng, Li, Ma, Fu and Li2024). Furthermore, specific convolutional kernels and layers can be hard coded and frozen to act as exact discrete differential operators, structurally enforcing divergence-free constraints (Mohan et al. Reference Mohan, Lubbers, Chertkov and Livescu2023). In addition to convolutional filter operations, CNNs can involve pooling operations to adjust the data dimension inside the model, which is often used for both encoder and decoder formulations in figure 1 with various problem settings. Details on each operation from the aspect of their uses for fluid mechanics can be seen in Morimoto et al. (Reference Morimoto, Fukami, Zhang, Nair and Fukagata2021b ).

2.3.4. Graph neural network

As some fluid flow data generated through computational fluid dynamics or sparsely sampled sensors are saved on a non-uniform grid, of interest is whether nonlinear machine-learning-based analysis can be considered for such cases while extracting physical insights that account for the spatial coherence of flow structures, similarly to CNNs. In response, one can consider a GNN that further generalises the concept of local receptive fields from CNNs to a non-Euclidean domain (Scarselli et al. Reference Scarselli, Gori, Tsoi, Hagenbuchner and Monfardini2008).

Given an arbitrary graph structure defined by nodes and edges, the core mechanism of GNNs is the so-called message passing, where a node $i$ updates its feature state by aggregating physical information from its topological neighbourhood $\mathcal{S}(i)$ . This is expressed as

(2.16) \begin{align} h_i^{(l+1)} = \varphi \left ( \sum _{j \in \mathcal{S}(i)} c_{ij} W^{(l)} h_{\!j}^{(l)} + b_i^{(l)} \right )\!, \end{align}

where the shared weight matrix $W^{(l)}$ transforms the features of neighbouring nodes and $c_{ij}$ represents edge-specific coefficients defined by the graph geometry. This formulation is particularly useful in applying machine-learning analyses for fluid flow data on unstructured grids (Li & Farimani Reference Li and Farimani2022). While standard GNNs have often been considered as they are adjustable for arbitrary mesh geometries, enforcing the equivalence of dynamic collective motions of multi-agent systems and the hierarchy of spatial length scales in fluid flows can be considered to incorporate physics-based prior knowledge into the structural design of GNNs (Linot et al. Reference Linot, Hang, Kanso and Taira2025).

2.3.5. Other model candidates

In addition to the model architectures ranging from pointwise mappings to structured and unstructured spatial convolutions, there exist a variety of machine-learning models that could support studies in fluid mechanics. For instance, experimental measurements often yield sparse velocity point clouds rather than organised structured grids. To handle such data directly without relying on mesh interpolation, PointNet (Qi et al. Reference Qi, Su, Mo and Guibas2017) is advantageous. They are specifically designed to process continuous, unordered sets of spatial points, bypassing the need for a grid topology while remaining structurally invariant to the permutation of the input data points. They have been considered for steady flow estimation (Kashefi et al. Reference Kashefi, Rempe and Guibas2021).

Fluid flows often exhibit critical long-range, non-local dependencies. While localised filters in CNNs require a number of layers to communicate information across a wide spatiotemporal domain, transformers (Vaswani et al. Reference Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser and Polosukhin2017) capture such global interactions. With a self-attention mechanism, transformers dynamically weigh the importance of disparate regions across the entire flow field in parallel, effectively extracting non-local features from fluid flow data. In fluid mechanics studies, the use of transformers has been found in dynamical modelling (Solera-Rico et al. Reference Solera-Rico, Vila, Gómez-López, Wang, Almashjary, Dawson and Vinuesa2024) and turbulent flow generation (Yousif et al. Reference Yousif, Zhang, Yu, Vinuesa and Lim2023b ).

Another challenge with standard machine-learning models is their dependence on the specific spatial grid resolution of their training data. While this highly relates to discussions in § 2.2, this challenge is caused by the fact that the majority of data-driven studies in fluid mechanics consider representing the fluid flow system as discrete finite-dimensional data. In other words, a model capable of handling data in a continuous manner may efficiently learn physics from the system without the loss of information (Wang et al. Reference Wang2023a ).

To generalise the formulation in infinite-dimensional function spaces, Fourier neural operators, which perform nonlinear machine-learning operations in the frequency domain, can be considered (Li et al. Reference Li, Kovachki and Azizzadenesheli2021). Such neural operators achieve zero-shot super-resolution of turbulent flows (Li et al. Reference Li, Peng, Yuan and Wang2022; Wang et al. Reference Wang, Li, Yuan, Peng, Liu and Wang2024). This zero-shot capability stems from the fact that Fourier neural operators learn a continuous operator mapping between function spaces, rather than mapping between fixed-size discrete vectors. In contrast to standard convolutional models, whose learned spatial filters are intrinsically tied to the specific grid spacing of the training data, such neural operators parameterise the integral operator in the Fourier domain. By truncating the spectrum to a fixed number of low-frequency modes, the learned network weights become entirely independent of the spatial discretisation. As a result, once the continuous flow function is learned on a coarse mesh, it can be evaluated at any arbitrarily finer, unseen spatial grid points without requiring structural changes to the model or retraining.

For the generation problem discussed in § 2.1.4, it has been reported that the standard GAN algorithm can suffer from training instability while it is data hungry. Diffusion models (Ho et al. Reference Ho, Jain and Abbeel2020) can instead be considered as an alternative for stochastic flow generation. By learning to synthesise data through a gradual reverse-denoising process, they enable the generation of realistic, statistically plausible flow fields from pure noise (Liu & Thuerey Reference Liu and Thuerey2024) or low-order representations of fluid flows including sensor measurements (Shu et al. Reference Shu, Li and Farimani2023).

It is worth noting again that these diverse machine-learning architectures are not mutually exclusive. Rather, they can be synergistically combined to analyse various flow characteristics. In fact, a combination of them, including CNN-MLP models (Zhong et al. Reference Zhong, Fukami, An and Taira2023), CNN-LSTM models (Hasegawa et al. Reference Hasegawa, Fukami, Murata and Fukagata2020), ConvLSTM models (Mohan et al. Reference Mohan, Tretiak, Chertkov and Livescu2020) and graph CNNs (Ogoke et al. Reference Ogoke, Meidani, Hashemi and Farimani2021), has been considered for fluid flow analysis. Such unification of machine-learning architecture may enable the model to be further versatile for a variety of problems in fluid dynamics.

2.3.6. Activation functions

As seen from the mathematical description for each model above, machine-learning models take activation functions to transform the input data of each node into an output signal. Nearly all modern machine-learning frameworks applied to fluid mechanics include internal operations with nonlinear activation functions, illustrated in figure 15. This is key not only to achieving better performance but also to gaining deeper physical insights compared with what has been possible with conventional linear techniques.

Figure 15.

Activation functions.

The fundamental role of these functions is to introduce nonlinearity into the network. In the absence of nonlinear activation, i.e. composed of a purely linear function $\varphi (\beta )=\beta$ , the network entirely reduces to a sole linear transformation. In such a case, the output would remain a mere linear combination of the inputs, regardless of the depth of the architecture. Given that fluid mechanics is inherently nonlinear, nonlinear activation functions assist in capturing complex spatiotemporal dynamics and turbulent multi-scale structures for machine-learning models.

Various activation functions have been proposed in the machine-learning community. Each possesses distinct mathematical properties that influence the training process and convergence. As presented in figure 15, example functions include the following.

1. (i) Linear: $\varphi (\beta )=\beta$ , defining a simple proportional mapping

2. (ii) Sigmoid: $\varphi (\beta )=(1+\text{e}^{-\beta })^{-1}$ , which maps inputs to a range between 0 and 1, frequently used in gating mechanisms.

3. (iii) tanh: $\varphi (\beta )=(\text{e}^{\beta }-\text{e}^{-\beta })/(\text{e}^{\beta }+\text{e}^{-\beta })$ , a zero-centred hyperbolic tangent function bounded between $-1$ and 1, which often aids in stabilising the training process.

4. (iv) ReLU (rectified linear unit): $\varphi (\beta )=\max [0,\beta ]$ , often used due to its computational efficiency and mitigation of the vanishing gradient problem.

5. (v) Leaky ReLU: $\varphi (\beta )=\beta$ (if $\beta \gt 0$ ) and $0.01\beta$ otherwise. This variant prevents ‘dying’ neurones by permitting a small, non-zero gradient for negative inputs.

6. (vi) ELU (exponential linear unit): $\varphi (\beta )=\beta$ (if $\beta \gt 0$ ) and $\text{e}^{\beta }-1$ otherwise, which smooths the output for negative inputs to promote robust learning.

7. (vii) Swish: $\varphi (\beta )=\beta /(1+\text{e}^{-\beta })$ , a smooth, non-monotonic function that has demonstrated superior performance in certain deep-learning applications.

8. (viii) Softsign: $\varphi (\beta )=\beta /(1+|\beta |)$ , an alternative to the tanh function that approaches its horizontal asymptotes more gradually.

The selection of an appropriate activation function is seldom trivial. It depends heavily on the flow physics of interest, the chosen network architecture and empirical parameter tuning to ensure smooth convergence while mitigating issues such as vanishing or exploding gradients. Because the training stability of the model also varies across different activation functions, it is essential to perform cross-validation, as discussed in § 2.2.4.

How nonlinear activation functions assist in studying fluid mechanics can be seen from various problem settings, including flow estimation and prediction. In particular, the distinction between linear and nonlinear functions inside machine-learning models is highlighted for compression problems by comparing a linear dimensional reduction of POD to its nonlinear counterparts. Although POD provides a unique solution without requiring any optimisations, let us express the POD with the reduced basis $\boldsymbol{\varPhi }\in \mathbb{R}^{n_{\boldsymbol{\xi }}\times N}$ as an $L_2$ minimisation problem,

(2.17) \begin{equation} \boldsymbol{\varPhi }^* = \operatorname *{argmin}_{{\boldsymbol \varPhi }} || \boldsymbol{q}^\prime - \boldsymbol{\varPhi }^{{T}} \boldsymbol{\varPhi } \boldsymbol{q}^\prime ||_2 , \end{equation}

where $\boldsymbol{q}^\prime$ denotes the fluctuating components of a variable $\boldsymbol q$ . Equipped with (2.8), the optimisation of a nonlinear autoencoder with respect to the parameters is similarly expressed as

(2.18) \begin{equation} \boldsymbol{\varPhi }^* = \operatorname *{argmin}_{{\boldsymbol \varPhi }} || \boldsymbol{q}^\prime - \varphi _d \boldsymbol{\varPhi }^{{T}} \varphi _e \boldsymbol{\varPhi } \boldsymbol{q}^\prime ||_2 , \end{equation}

where $\boldsymbol{\varPhi }$ here corresponds to the weights inside the network, and $\varphi _e$ and $\varphi _d$ represent the activation functions of the encoder and decoder, respectively. Based on (2.17) and (2.18), a nonlinear autoencoder becomes identical to POD when the activation function is set to be linear, such that $\varphi _e=\varphi _d=1$ (Baldi & Hornik Reference Baldi and Hornik1989; Fukami et al. Reference Fukami, Hasegawa, Nakamura, Morimoto and Fukagata2021b ).

In turn, machine-learning models with nonlinear activation functions could provide better compression by introducing nonlinearities in the form of (2.18). It has been widely reported that the use of nonlinear activation functions reduces reconstruction error at the same number of modes $n_{\boldsymbol{\xi }}$ compared with linear techniques for a variety of flow examples, as nonlinear autoencoders enable projecting the high-dimensional fluid flow data onto a curved, lower-dimensional manifold (Murata et al. Reference Murata, Fukami and Fukagata2020; Linot et al. Reference Linot, Zeng and Graham2023; Fukami & Taira Reference Fukami and Taira2025).

This has often been discussed with the concept of the Kolmogorov barrier or the Kolmogorov $n$ width (Peherstorfer Reference Peherstorfer2022). The Kolmogorov $n$ width expresses the theoretical lower bound of the approximation error achievable by any optimal $n$ -dimensional linear subspace. For advection-dominated flows, this $n$ width decays slowly, indicating that linear methods require a large number of modes to accurately reconstruct the flow. Nonlinear autoencoders are not constrained by this barrier, capturing complex dynamics with more compact representations compared with linear techniques. The above points will also be evident through the present case study in § 3 from a comparison of data compression between POD and nonlinear autoencoders for unsteady aerodynamic flows.

2.3.7. Parameter selection

Once the network architecture is established with nonlinear activation functions, the subsequent challenge lies in tuning the hyperparameters of machine-learning models. In practice, finding a globally ‘optimal’ parameter set is nearly impossible due to several fundamental reasons. First, the loss landscape in nonlinear machine learning is generally non-convex, fraught with local minima and saddle points. Second, modern machine-learning architectures often possess a large number of tunable parameters specifically for fluid flows with a high degree of freedom in space and time, making naive parameter sweeps computationally intractable (Nakamura et al. Reference Nakamura, Fukami and Fukagata2022). Third, the sensitivity of the cost function to different parameters can significantly vary. Along with the second point, this makes finding the optimal parameter set further challenging. Fourth, the optimal model configuration also depends on the input data itself. For example, varying the spatial locations of input sparse sensors for the flow reconstruction problem may change the loss surface, as depicted in figure 16 $(b)$ .

Figure 16.

Loss surface with respect to $(a)$ model parameters, and $(b)$ input data.

Because of such mathematical and computational hurdles, ‘optimised’ or sufficiently robust parameter sets are sought rather than the ‘optimal’ one. One practical remedy is to employ a step-by-step heuristic procedure, often referred to as a manual search (Bergstra & Bengio Reference Bergstra and Bengio2012; Fukami et al. Reference Fukami, Fukagata and Taira2020a ). As an example, let us consider a MLP. The objective is to optimise the number of hidden layers $n_{\textit{layer}}$ and the number of nodes per layer ( $n_{\textit{node}}$ ), by performing an iterative procedure as follows.

1. (i) Fix the number of layers $n_{\textit{layer}}$ and perform a parameter sweep to identify the number of units $n_{\textit{unit}}^*$ that minimises the loss function. To ensure the model generalises well to unseen scenarios, $k$ -fold cross-validation should be performed during this step.

2. (ii) Subsequently, fix the units at the newly found $n_{\textit{unit}}^*$ and vary the number of layers to identify the $n_{\textit{layer}}^*$ that yields the minimum loss, again performing cross-validation.

3. (iii) The resulting parameter pair $(n_{\textit{unit},\textit{opt}}, n_{\textit{layer},\textit{opt}}) = (n_{\textit{unit}}^*, n_{\textit{layer}}^*)$ is then adopted as the optimised configuration for the machine-learning model. One can also iterate over steps (i) and (ii) until convergence is achieved.

This can also be applied to other machine-learning models, e.g. CNNs, in which the number of filters and the kernel size can be optimised. An optimiser used in solving the problem based on the cost function, such as the Adam optimiser (Kingma & Ba Reference Kingma and Ba2014), can be an additional parameter, which may be incorporated into the above routine (Kiyani et al. Reference Kiyani, Shukla, Urbán, Darbon and Karniadakis2025).

One can also consider automated hyperparameter optimisation approaches, such as a grid search (Bishop & Nasrabadi Reference Bishop and Nasrabadi2006), a random search (Bergstra & Bengio Reference Bergstra and Bengio2012) and Bayesian optimisation (Bergstra et al. Reference Bergstra, Komer, Eliasmith, Yamins and Cox2015). Note, however, that these methods perform optimisation of the network parameters, which itself minimises a cost function, within an outer optimisation loop with the original cost function based on the problem setting, implying that this dual loop increases the computational cost. Based on the availability of computational resources and the requirement of model performance, an appropriate approach needs to be selected for the parameter decision of nonlinear machine-learning models.