3.1. Bickley jet

The Bickley jet, an unsteady flow with distinct vortices and a central meandering jet, is defined on the domain $x=[0,20]$ Mm and $y=[-3, 3]$ Mm by the streamfunction

(3.1)\begin{align} \varPsi(x,y,t)&=c_3y-U_0L\tanh(y/L) +A_3U_0L\mathrm{sech}^2(y/L)\cos(k_3x)\nonumber\\ &\quad+A_3U_0L\mathrm{sech}^2(y/L)\cos(k_2x-\sigma_2t). \end{align}

In the following analysis we follow the parameters of Rypina et al. (Reference Rypina, Brown, Beron-Vera, Koçak, Olascoaga and Udovydchenkov2007) and an integration time of $T=10^6$ s. Choosing 10 latent nodes ($N_z=10$), we train the autoencoder architecture for ten distinct runs using values of $\beta$ logarithmically spaced between $\beta =0.001$ and $\beta =1.0$. These values of $\beta$ represent increasing pressures for the network to communicate via robust representations under the influence of random noise, and decreasing relative pressures to reconstruct the trajectories accurately. Example reconstructed trajectories from the Bickley jet are shown in figures 2 and 3. As $\beta$ is increased, the reconstruction accuracy significantly suffers as expected. For trajectories within one of the vortices (figure 2), an increasing value of $\beta$ ultimately confuses the network such that it can no longer even qualitatively resemble the true dynamics. For a sample trajectory in the centre jet (figure 3), a similar degradation occurs with increasing $\beta$, but with some features retained in the reconstructed trajectories. In some sense this hints at a dynamic simplicity associated with the centre jet that is unlike any one of the vortices.

Figure 2.

Example trajectory (in blue) taken from a vortex in the Bickley jet and attempts to reconstruct the trajectory by four different networks for training values of $(a)$ $\beta =0.001$, $(b)$ $\beta =0.01$, $(c)$ $\beta =0.1$ and $(d)$ $\beta =1.0$.

Figure 3.

Same as figure 2 but sample trajectory is taken from the central meandering jet.

As with many dimensional reduction techniques, the reconstructed data are only part of the picture. Because the decoder can only see the representation of the latent layer, viewing these representations directly gives us insight into the essential flow features which are retained by the encoder – it is this process from which Lagrangian structures emerge. Since this autoencoder has $N_z=10$ (a parameter choice whose effect is discussed below), each single trajectory will have some activation on all ten of the latent nodes. To visualize these activations, we apply the autoencoder to trajectories originating on an evenly spaced grid of initial conditions. In figure 4 we then show the mean activations $z_i$ of the latent layer. Here, each pixel of a latent node image represents the activation of a single trajectory starting in that particular initial position.

Figure 4.

All $N_{z} = 10$ latent nodes coloured by activated trajectories. Results are shown for $(a)$ $\beta =0.001$, $(b)$ $\beta =0.01$, $(c)$ $\beta =0.1$ and $(d)$ $\beta =1.0$. Each image corresponds to a single node.

Evident from figure 4, the learned representations of trajectories correspond well with the different dynamic regions of the Bickley jet, structures which have been thoroughly examined in previous studies (Hadjighasem et al. Reference Hadjighasem, Farazmand, Blazevski, Froyland and Haller2017). This is the explicit LCS connection that we sought from our method and a non-locality which departs from PCA. In figure 4 this is represented by the shift from the rightmost ($\beta = 1.0$) to leftmost ($\beta = 0.001$) columns, where the node activations shift from ones which resemble LCS to those which resemble PCA. From the noise perspective, it makes sense to code dynamically similar trajectories (which are usually close to one another in physical space) into the same latent node. To illustrate: consider two trajectories, one belonging to a vortex and one belonging to the central jet (e.g. those in figures 2 and 3). If these two trajectories are coded into the same latent channel, added noise to the latent representation may lead to the confusion of one for the other and a large reconstruction loss. This very event is observed in figure 2(d), where a vortex trajectory is mistaken as one from the jet. So similar trajectories, often from the same dynamic region of the flow, are coded into nearby positions in the latent space using the same latent channel. This way, any noise added to the latent layer will only cause one vortex (jet) trajectory to be mistaken for another vortex (jet) trajectory and, therefore, a smaller reconstruction loss.

As a result of this interpretation, we can also view our method as related to that of Fang, Balasuriya & Ouellette (Reference Fang, Balasuriya and Ouellette2019) and Fang & Ouellette (Reference Fang and Ouellette2021). In these works, the authors define a coherent subset of a flow as a set of trajectories which can be constructed by linear approximations from one another. This definition tends to capture desirable properties of spatiotemporal coherence, and is connected to the operations performed by our autoencoder under the limit of large $\beta$. Rather than defining similarity as the result of linear approximation, the latent layer reflects a richer notion of similarity as stemming from more general nonlinear transformations all related to the same kinds of kinematic motions, like that of a jet or that of a vortex.

We demonstrate this result further in figure 5, where we colour each initial condition with the same colour if they activated the same latent node (therefore, there is a maximum of $N_z$ possible colours). Then, we advect these nodes through the same training time length of the trajectories, since coherence under advection is generally a requirement among LCS techniques (Hadjighasem et al. Reference Hadjighasem, Farazmand, Blazevski, Froyland and Haller2017). Evident from figure 5, the structures detected in the high-noise limit of the Bickely jet are robust to advection at least within the time scale seen during training by the autoencoder. The opposite is true for the low-noise limit, as the critical structures retained by the autoencoder are subject to advection by the flow as would any random scalar field. The only exceptions to this observation are the coherent structures on the very edges of the initial domain. Because the autoencoder transformations are smooth in nature, the network struggles considerably to represent data on a periodic domain. For all training, therefore, trajectories are given to the encoder as continuous, so structures which are continuous on a periodic domain are nonetheless represented as two different structures.

Figure 5.

Initial conditions of trajectories coloured by their corresponding latent node of highest activation. Results are shown for (a) $\beta =0.001$, (b) $\beta =0.01$, (c) $\beta =0.1$ and (d) $\beta =1.0$. Advected final positions of trajectories for corresponding values of $\beta$ are shown in (e–h).

Now, a natural question is how the autoencoder behaves for a different number of latent nodes, $N_z$. The entire point of dimensional reduction is to pass the trajectories through an information bottleneck, so excessively large values of $N_z$ eliminate the purpose of the endeavour altogether. Related to this is the question of what exactly the structures in figure 5 represent. In the present analysis we simply state that, for large $\beta$, these resemble the well-known LCS of the Bickley jet, and possess similar coherence properties under advection. However, they should not be expected to exactly align with LCS found by other techniques because this will be a function of $N_z$.

To pursue this line of inquiry, we repeat the training runs for two additional values of $N_z=5$ and $N_z=15$. As we have noted, it is challenging to determine an optimal number of coherent structures, as a single coherent structure can often be subdivided into two structures with the same level of coherence (this can be observed in figure 5). This a priori specification of the number of structures is a familiar problem with LCS methods in general. However, in the present context there is a limit in which too few latent nodes will not provide a sufficient amount of information to reconstruct flow features accurately. Figure 6 shows reconstructed probability density functions (p.d.f.s) of trajectory velocities for the three different networks trained on the Bickley jet. Evident from the reconstructed p.d.f.s, for all values of $N_z$, major flow features are retained (i.e.  the large velocity magnitude peak which represents the meandering jet), and the accuracy with which these prominent flow features are reconstructed decreases with increasing values of $\beta$.

Figure 6.

Reconstructed velocity p.d.f.s for $(a)$ $N_z=5$, $(b)$ $N_z=10$ and $(c)$ $N_z=15$.

To quantify more precisely how this error scales with the noise in the latent layer, in figure 7 we plot the average root-mean-squared (r.m.s.) error against values of $\beta$. Although different network architectures result in different average error for the same value of $\beta$, the relationship with which they scale remains constant. One interpretation of this behaviour is that the vertical position of this line reports information about the absolute level of accuracy, while the slope of the line indicates something more fundamental about the ease or difficulty of reconstructing trajectories in the flow in question. This interpretation makes the most sense when considering the limiting case: in a completely chaotic system with no spatiotemporal coherence, the autoencoder will not be able to extract any features. Increasing levels of noise will therefore have little effect on the reconstruction accuracy, as any change (regardless of magnitude) will have an equally negative influence on the autoencoder's ability to reconstruct trajectories. So, for a totally random system, we would expect that the reconstruction accuracy is independent of the latent noise $\beta$, or a flat line. The fact that all lines in figure 7 have generally the same slope for all $N_{z}$ reflects an underlying similarity between the network representations, and future efforts could therefore be focused on choosing $N_{z}$ according to the number or character of the coherent structures rather than overall reconstruction accuracy.

Figure 7.

Root-mean-squared trajectory reconstruction error against latent layer noise $\beta$ for the Bickley jet.

3.2. Two-dimensional turbulence

We now seek to demonstrate, briefly, that this method is not only useful for relatively simple problems like the Bickley jet. As done in other studies of LCS identification (Hadjighasem et al. Reference Hadjighasem, Farazmand, Blazevski, Froyland and Haller2017), we analyse DNS of two-dimensional turbulence. To generate the data, we solve the two-dimensional, incompressible Navier–Stokes equations on a periodic domain using a pseudospectral method. The flow is made turbulent by stochastic forcing, the exact set-up of which can be found in Karrasch & Schilling (Reference Karrasch and Schilling2020). We define a Reynolds number as ${Re}=u_{{rms}}L/\nu$, where $u_{{rms}}$ is the r.m.s. velocity, $L$ is the length of the domain and $\nu$ is the kinematic viscosity. Our analysis will focus on a case at a Reynolds number of ${Re}=111$, but we also briefly consider a higher Reynolds number of ${Re}=255$.

As for our previous example, we train ten different networks at the same values of increasing $\beta$. Because it is anticipated that there would be a greater number of underlying degrees of freedom in the flow, or a wider variety of coherent structures that could be identified, we start by training a network for $N_z=15$. Sample reconstructed trajectories for an integration length of $T=2$ s are shown in figure 8. Consistent with the results for the Bickley jet, as the hyperparameter $\beta$ is increased, the reconstruction fidelity decreases.

Figure 8.

Same as figures 2 and 3 but for a sample trajectory from the two-dimensional turbulence field.

As $\beta$ increases, the latent node activations also follow a similar pattern as before. Latent node colourings in the spirit of figure 4 are shown in figure 9. We again observe the trend that as the latent noise increases, node activations transition from local-in-energy structures towards local-in-space structures. By this we mean that for large $\beta$, initial locations which are close in physical space tend to activate together (local-in-space), whereas low values of $\beta$ cause more spatially disparate but energetic features (i.e. PCA) to be activated (local-in-energy). In correspondence with figure 5, figure 10 shows the trajectories coloured at their initial positions by the latent node of highest activation, and again we observe at the highest noise levels structures which strongly resemble LCS. It is worth noting, however, that some structures may be missing from these high-noise representations. Because the reconstruction loss is summed equally across all trajectories, highly coherent structures which contain a relatively small share of the trajectory ensemble are considered less important. As a result, structures like turbulent vortices are often included in some other latent representation, rather than in a latent node by themselves. As we have shown, in flows where vortices consist a large percentage of the trajectories (e.g. the Bickley jet), vortices are often represented in single latent nodes.

Figure 9.

Same node colourings as figure 4 for (a) $\beta =0.001$, (b) $\beta =0.01$, (c) $\beta =0.1$ and (d) $\beta =1.0$.

Figure 10.

Same visualization as figure 5. As before, results are shown for (a) $\beta =0.001$, (b) $\beta =0.01$, (c) $\beta =0.1$, (d) $\beta =1.0$.

To solidify this interpretation, it is useful to compute the finite-time Lyapunov exponents (FTLEs) of the same system, the high-valued ridges of which separate dynamically distinct regions of a flow (see, for example, Shadden, Lekien & Marsden Reference Shadden, Lekien and Marsden2005). We begin by defining a flow map which maps trajectories from their initial positions $x_0$ at time $t_0$ to their final positions $x_1$ at $t_1$,

(3.2)\begin{equation} \varPhi_{t_0}^{t_1}(x_0)=x_0+\int_{t_0}^{t_1}u(x,\tau)\,\textrm{d}\tau, \end{equation}

where $u(x,t)$ is the velocity field of the flow. To understand the behaviour of these trajectories as their initial positions vary, we employ the Cauchy–Green strain tensor

(3.3)\begin{equation} \boldsymbol{\Delta} = [\nabla\varPhi_{t_0}^{t_1}]^{*}[\nabla\varPhi_{t_0}^{t_1}]. \end{equation}

The Cauchy–Green strain tensor offers a notion of how far initially close trajectories will stray over this time frame. This tensor then leads to the definition of the FTLE field as

(3.4)\begin{equation} \mathrm{FTLE}(x_0)=\frac{1}{t_1-t_0}\log(\sqrt{\lambda_{max}(\boldsymbol{\Delta})}), \end{equation}

where $\lambda _{max}(\boldsymbol {\nabla })$ is the largest eigenvalue of $\boldsymbol {\nabla }$ and represents the largest separation that two trajectories advected under (3.2) will experience. Figure 11 shows the FTLE field computed for the same two-dimensional turbulence field. Comparing the ridges of the FTLE field with the boundaries of latent nodes in figure 10, we find strong correspondence for the highest noise levels ($\beta =1.0$). Furthermore, figure 12 shows the final positions of the advected tracers coloured by their latent node of highest activation for the lowest and highest noise levels. Because the ridges of the FTLE field separate regions of the flow where trajectories will diverge, we expect that the highest noise level structures will be most careful about separating trajectories along these lines. Indeed, we confirm that the trajectories in the high-noise limit are still spatially local, or coherent, after advection, whereas those for the low-noise limit are not.

Figure 11.

Finite-time Lyapunov exponent field for the low-Reynolds-number two-dimensional turbulence.

Figure 12.

Trajectories coloured by their latent node of highest activation for the lowest noise level $\beta = 0.001$ (a) and highest noise level $\beta = 1.0$ (b).

Finally, figure 13 shows the reconstructed p.d.f.s from networks with different values of $N_z$. As before, we see consistent results across the different networks, indicating that the primary effect of $N_{z}$ in the present context is to perhaps unnecessarily split coherent structures into multiple segments. Interestingly, as the noise is increased for all networks, there is a bias towards lower velocities, but the error scales similarly for the different networks. Then in figure 14, we again plot the reconstruction error as a function of $\beta$. As in the case of the Bickley jet, the varying size of the latent layer causes the absolute reconstruction error to shift for a given value of $\beta$, but the fundamental rate at which error increases with increasing $\beta$ again remains roughly the same. To add to this, figure 14 also plots this relationship for the same flow but at a higher Reynolds number. At the larger ${Re}$, a jump in the r.m.s. error is seen, but interestingly, the rate at which the error rises with $\beta$ remains nearly constant. As suggested in the case of the Bickley jet, one interpretation of this behaviour is that the slope of this line indicates an underlying complexity of the flow – that is, how fundamentally difficult it is for the autoencoder network to reconstruct trajectories. Under this interpretation, there is a sense in which the two-dimensional turbulence at a higher Reynolds number is not necessarily a more complex flow, even if the errors themselves increase in an absolute sense. We defer this question to future analysis.

Figure 13.

Reconstructed velocity p.d.f.s for (a) $N_z=15$, (b) $N_z=20$ and (c) $N_z=25$.

Figure 14.

Root-mean-squared error against latent layer noise $\beta$ for two-dimensional turbulence at two different Reynolds numbers.