1. Introduction
Navigation and exploration of microswimmers in turbulent and complex flow environments are fundamental tasks in both natural and artificial systems. In nature, for example, phytoplankton and zooplankton live in a turbulent environment, spending significant effort in vertical migration and nutrient foraging (Hays Reference Hays2003; Guasto et al. Reference Guasto, Rusconi and Stocker2012). Artificial microswimmers, by contrast, have shown promise for biomedical applications such as targeted drug delivery and microsurgery (Huang et al. Reference Huang, Uslu, Katsamba, Lauga, Sakar and Nelson2019; Tsang et al. Reference Tsang, Demir, Ding and Pak2020).
An intriguing scenario arises when microswimmers perceive and adapt to environmental cues. This capacity is common among many plankton species, which can sense and respond to environmental cues such as light (phototaxis) (Jékely et al. Reference Jékely, Colombelli, Hausen, Guy, Stelzer, Nédélec and Arendt2008), chemical gradients (chemotaxis) (Stocker & Seymour Reference Stocker and Seymour2012) and hydromechanical signals (Kiørboe & Visser Reference Kiørboe and Visser1999). Beyond modulating motility, some plankton also alter their morphology in response to environmental changes. For instance, Sengupta et al. (Reference Sengupta, Carrara and Stocker2017) showed that certain algae undergo shape transformations under turbulent conditions, thereby reversing their vertical migration by altering their stable orientation. Similarly, some diatoms form long chains to adjust their effective aspect ratio, thereby modifying their dynamics in turbulence and improving vertical migration efficiency (Smayda Reference Smayda2010; Borgnino et al. Reference Borgnino, Boffetta, De Lillo and Cencini2018; Lovecchio et al. Reference Lovecchio, Climent, Stocker and Durham2019).
In synthetic systems, artificial microswimmers must adapt to heterogeneous and time-dependent flow environments to achieve autonomous navigation. Despite substantial progress, this objective has not yet been realised. Advances in biohybrid microrobots have demonstrated adaptive and partially autonomous behaviour (Ricotti et al. Reference Ricotti, Trimmer, Feinberg, Raman, Parker, Bashir, Sitti, Martel, Dario and Menciassi2017; Huang et al. Reference Huang, Uslu, Katsamba, Lauga, Sakar and Nelson2019; Tsang et al. Reference Tsang, Demir, Ding and Pak2020). For example, Huang et al. (Reference Huang, Sakar, Petruska, Pané and Nelson2016) developed a microswimmer with deformable flagella that enable controllable swimming gaits. However, fully autonomous microswimmers that operate without external control remain a major challenge, since adaptive capabilities are typically hard coded during fabrication, which limits flexibility and practical applications.
The study of adaptive navigation strategies is therefore important not only for gaining insight into plankton ecology, but also for guiding the design of artificial microswimmers. In both settings, microswimmers must account for the local flow field to navigate efficiently. In recent years, the motion of microswimmers in turbulent environments has been investigated extensively. Modelling them as Lagrangian spheroidal particles swimming in their instantaneous direction explains phenomena such as small-scale clustering (Durham et al. Reference Durham, Climent, Barry, De Lillo, Boffetta, Cencini and Stocker2013; Gustavsson et al. Reference Gustavsson, Berglund, Jonsson and Mehlig2016) and preferential alignment with flow structures (Zhan et al. Reference Zhan, Sardina, Lushi and Brandt2014; Borgnino et al. Reference Borgnino, Gustavsson, De Lillo, Boffetta, Cencini and Mehlig2019). Particle shape plays a central role in this context: it alters the hydrodynamic torque on the swimmer, modifying preferential orientation (Borgnino et al. Reference Borgnino, Gustavsson, De Lillo, Boffetta, Cencini and Mehlig2019; Ma et al. Reference Ma, Pujara and Thiffeault2022) and location in the flow (Torney & Neufeld Reference Torney and Neufeld2007; Gustavsson et al. Reference Gustavsson, Berglund, Jonsson and Mehlig2016; Borgnino et al. Reference Borgnino, Boffetta, De Lillo and Cencini2018). Because the rotational dynamics directly influences the swimming direction and stability, adaptive morphological changes may provide significant advantages for navigation in complex environments.
(a) Microswimmer navigation maximising distance from the initial position
$\boldsymbol{x}(t_0)$
(red star). Equidistant black dashed circles denote radial distances. Two example trajectories of microswimmers following (2.1) with constant shape factors
$\varLambda =-0.98$
(magenta) and
$\varLambda =0.98$
(black), evolving in a two-dimensional turbulent flow obtained from DNS. Inset: energy spectrum of the two-dimensional turbulent flow. (b) Schematic of a spheroidal microswimmer and its reference frame. (c, d) Microswimmers can adjust their aspect ratio, becoming (c) prolate,
$\varLambda \gt 0$
, or (d) oblate,
$\varLambda \lt 0$
.
Figure 1. Long description
Panel A: A colormap shows microswimmer navigation maximizing distance from the initial position marked by a red star. Equidistant black dashed circles denote radial distances. Two example trajectories of microswimmers are shown in magenta and black, evolving in a two-dimensional turbulent flow obtained from direct numerical simulation. The inset displays the energy spectrum of the two-dimensional turbulent flow with axes labeled E(k) and k. Panel B: A schematic diagram of a spheroidal microswimmer and its reference frame. The diagram includes vectors labeled p, eθ, er, n, and θ, with the initial position marked by a red star labeled x(t0). Panel C: A diagram shows a prolate microswimmer with a shape factor greater than zero, indicated by a black arrow. Panel D: A diagram shows an oblate microswimmer with a shape factor less than zero, indicated by a black arrow.
In this context, reinforcement learning has recently emerged as a powerful tool for discovering optimal strategies in complex environments (Sutton & Barto Reference Sutton and Barto2018; Mehlig Reference Mehlig2021). Since the pioneering work of Colabrese et al. (Reference Colabrese, Gustavsson, Celani and Biferale2017), reinforcement learning has been widely applied to the study of microswimmer navigation strategies (Gustavsson et al. Reference Gustavsson, Biferale, Celani and Colabrese2017; Alageshan et al. Reference Alageshan, Verma, Bec and Pandit2020; Gunnarson et al. Reference Gunnarson, Mandralis, Novati, Koumoutsakos and Dabiri2021; Borra et al. Reference Borra, Biferale, Cencini and Celani2022; Xu et al. Reference Xu, Wu and Xi2022; Zou et al. Reference Zou, Liu, Young, Pak and Tsang2022; Qiu et al. Reference Qiu, Mousavi, Gustavsson, Xu, Mehlig and Zhao2022a , Reference Qiu, Mousavi, Zhao and Gustavssonb ; Mousavi et al. Reference Mousavi, Qiu, Mehlig, Zhao and Gustavsson2024, Reference Mousavi, Qiu, Zhao, Mehlig and Gustavsson2025). However, except for Colabrese et al. (Reference Colabrese, Gustavsson, Celani and Biferale2018), which considered density-changing microswimmers, most existing studies have focused on adaptive motility in response to environmental cues, such as swimming speed, steering direction or locomotory gaits. In contrast, navigation strategies based on active deformation or morphological adaptation remain largely unexplored.
In this study, we use reinforcement learning to investigate the navigation strategy of shape-changing spheroidal microswimmers in two-dimensional stochastic and turbulent flows. We consider the task in which a microswimmer needs to maximise its distance from its initial position over a fixed time horizon (figure 1 a). This serves as a minimal model for ecologically relevant behaviours such as evasion from harmful regions (Incze et al. Reference Incze, Hebert, Wolff, Oakey and Dye2001), and efficient exploration of nutrition-rich environments (Bartumeus et al. Reference Bartumeus, Peters, Pueyo, Marrasé and Catalan2003). The microswimmer is assumed to sense its instantaneous orientation and local flow-velocity gradients, and to adapt its aspect ratio accordingly. While our primary interest is navigation in realistic turbulent flows, we first use a stochastic flow model with adjustable temporal correlations to identify the mechanisms underlying learned strategies, and then assess their robustness and relevance using direct numerical simulations of two-dimensional turbulence.
Our main findings can be summarised as follows: (i) adaptive shape control enhances navigation. The learned strategy consistently outperforms heuristic baselines, with the largest gains in slowly evolving flows. (ii) A minimal analytical model captures the learned behaviour. A reduced interpretable strategy distilled from the reinforcement-learning policy reproduces performance across flow regimes. (iii) The learned adaptation mechanisms are robust and transferable. Policies trained in stochastic flows transfer successfully to turbulent flows and retain high performance under moderate variations of the swimming parameters, indicating that the extracted mechanisms are universal rather than artefacts of a specific flow model.
Taken together, these results show that active shape modulation provides a physically interpretable and efficient control paradigm for microswimmer navigation in complex flows, revealing how morphological adaptation can be leveraged to exploit environmental structure over multiple time scales.
The paper is organised as follows. Section 2 introduces the microswimmer model, stochastic flow model and reinforcement-learning framework. Section 3 defines the benchmark strategies. Section 4 analyses the strategies learned in different flow regimes, compares their performances with the benchmarks, explains the underlying physical mechanisms and examines their transferability to direct numerical simulations (DNS) of turbulence. Finally, § 5 summarises the main findings and outlines future directions.
2. Methods
2.1. Microswimmer model
We model microswimmers as self-propelled spheroidal particles advected by a fluid flow (figure 1
b). Each swimmer is characterised by its position
$\boldsymbol{x}(t)$
and by a unit orientation vector
$\boldsymbol{n}(t)$
aligned with its symmetry axis. The shape is parametrised by the factor
$\varLambda = (\lambda ^2-1)/(\lambda ^2+1)$
, where
$\lambda$
is the spheroid aspect ratio. We assume that microswimmers can actively deform and control
$\varLambda$
, as illustrated in figures 1(c) and 1(d).
Assuming small microswimmers and negligible inertia, their dynamics is governed by the standard model for elongated particles in viscous flows (Pedley & Kessler Reference Pedley and Kessler1987; Durham et al. Reference Durham, Kessler and Stocker2009; Qiu et al. Reference Qiu, Mousavi, Gustavsson, Xu, Mehlig and Zhao2022a ; Mousavi et al. Reference Mousavi, Qiu, Mehlig, Zhao and Gustavsson2024)
\begin{align} \dot {\boldsymbol{x}} = & \boldsymbol{u} + v_{\textit{s}} \boldsymbol{n}, \end{align}
\begin{align} \dot {\boldsymbol{n}} = & \frac {1}{2}\boldsymbol{\omega }\times \boldsymbol{n} + \varLambda \left ( \mathbb{I} - \boldsymbol{n} \boldsymbol{n}\right )\boldsymbol{\cdot }\mathbb{S} \boldsymbol{\cdot }\boldsymbol{n}. \end{align}
Here,
$\boldsymbol{u}(\boldsymbol{x},t)$
denotes the fluid velocity evaluated at the microswimmer position,
$v_{\textit{s}}$
denotes swimming speed,
$\boldsymbol{\omega } = \boldsymbol{\nabla }\times \boldsymbol{u}$
is the vorticity,
$\mathbb{S} = ({1}/{2})(\boldsymbol{\nabla }\boldsymbol{u} + \boldsymbol{\nabla }\boldsymbol{u}^{T})$
is the strain-rate tensor and
$\mathbb{I}$
is the identity matrix.
The translational dynamics in (2.1a
) consists of passive advection by the ambient flow and active propulsion along the swimmer orientation
$\boldsymbol{n}$
. The rotational dynamics in (2.1b
) describes the rotation of a spheroid by velocity gradients in viscous flow (Jeffery Reference Jeffery1922). The contribution of the strain-rate tensor
$\mathbb{S}$
is proportional to the shape factor
$\varLambda$
. Consequently, particle shape strongly impacts microswimmer dispersion and transport (Khurana & Ouellette Reference Khurana and Ouellette2012; Gustavsson et al. Reference Gustavsson, Berglund, Jonsson and Mehlig2016; Piro et al. Reference Piro, Vilfan, Golestanian and Mahault2024). In particular, swimmers with different
$\varLambda$
can exhibit qualitatively distinct trajectories within the same flow field (figure 1
a), indicating that active modulation of
$\varLambda$
may serve as an effective navigation mechanism.
We focus on a two-dimensional flow in which the microswimmer orientation is confined to the flow plane. In this case, the orientation dynamics in (2.1b ) simplifies to
\begin{align} \dot {\theta }=\frac {1}{2}\omega + \varLambda S_{np}, \end{align}
where
$\theta$
is the angle of
$\boldsymbol{n}$
,
$\omega$
is the out-of-plane vorticity,
$S_{np}= \boldsymbol{n}\boldsymbol{\cdot }\mathbb{S} \boldsymbol{\cdot }\boldsymbol{p}$
and
$\boldsymbol{p}$
is the in-plane unit vector perpendicular to
$\boldsymbol{n}$
(figure 1
b).
2.2. Stochastic flow model
To model the turbulent fluctuations, we use a two-dimensional, time-dependent, isotropic stochastic flow model (Bec et al. Reference Bec, Gustavsson and Mehlig2024). The velocity field of the flow is determined by
$\boldsymbol{u}(\boldsymbol{x},t) = (\partial \psi /\partial y, - \partial \psi /\partial x)$
. Following Mousavi et al. (Reference Mousavi, Qiu, Mehlig, Zhao and Gustavsson2024), the streamfunction
$\psi (\boldsymbol{x},t)$
is the superposition of
$M$
time-independent components
$\psi _i(\boldsymbol{x})$
, with time-dependent coefficients
$c_i(t)$
. The components follow the spatial correlation
\begin{align} \langle \psi _i(\boldsymbol{x})\psi _j(\boldsymbol{x}^\prime )\rangle = \frac {1}{2}\delta _{ij}u_{\textit{rms}}^2 \ell _{\textit{f}}^2\exp {\left [-\frac {(\boldsymbol{x}-\boldsymbol{x}^\prime )^2}{2\ell _{\textit{f}}^2}\right ]}, \end{align}
where
$u_{\textit{rms}} = \sqrt {\langle \boldsymbol{u} \boldsymbol{\cdot }\boldsymbol{u} \rangle }$
is the root-mean-square flow velocity, and
$\ell _{\textit{f}}$
is the flow correlation length scale. The time-dependent coefficients
$c_i(t)$
follow an Ornstein–Uhlenbeck process with zero mean and time correlation
\begin{align} \langle c_i(t)c_j(t^\prime )\rangle = \frac {\delta _{ij}}{M} \exp \left ({-\frac {|t-t^\prime |}{\tau _{\textit{f}}}}\right ), \end{align}
where
$\tau _{\textit{f}}$
is the flow correlation time.
The stochastic flow is characterised by a single dimensionless parameter, the Kubo number
$\textit{Ku}=\tau _{\textit{f}} u_{\textit{rms}}/ \ell _{\textit{f}}$
, which compares the flow correlation time
$\tau _{\textit{f}}$
with the advective time scale
$\ell _{\textit{ f}}/u_{\textit{rms}}$
. Large
$\textit{Ku}$
corresponds to slowly evolving flows, approaching a time-independent limit as
$\textit{Ku} \to \infty$
, while
$\textit{Ku}\to 0$
yields temporally uncorrelated flows with rapid fluctuations. This stochastic flow model reproduces essential statistical properties of small-scale velocity-gradient fluctuations in turbulent flows, showing good agreement with DNS for both inertial particles (Bec et al. Reference Bec, Gustavsson and Mehlig2024) and microswimmers (Gustavsson et al. Reference Gustavsson, Berglund, Jonsson and Mehlig2016; Borgnino et al. Reference Borgnino, Gustavsson, De Lillo, Boffetta, Cencini and Mehlig2019; Qiu et al. Reference Qiu, Mousavi, Zhao and Gustavsson2022b
; Mousavi et al. Reference Mousavi, Qiu, Mehlig, Zhao and Gustavsson2024, Reference Mousavi, Qiu, Zhao, Mehlig and Gustavsson2025), especially in the regime of
$\textit{Ku}\sim 10$
.
In this work, we vary the Kubo number by adjusting the flow correlation time
$\tau _{\textit{f}}$
, while keeping the root-mean-square flow velocity
$u_{\textit{rms}}$
and length scales
$\ell _f$
fixed. To match the stochastic flow to turbulence, we take
$u_{\textit{rms}}$
as the characteristic velocity scale and the Kolmogorov time scale,
$\tau _{\textit{K}} = \langle {(\partial u_i}/{\partial x_j}){(\partial u_i}/{\partial x_j)} \rangle ^{-1/2}$
, as the characteristic flow time scale.
2.3. Reinforcement learning
The goal is to maximise the swimmer’s displacement from its initial position over a prescribed time horizon. To identify an effective control strategy, we employ proximal policy optimisation (PPO) (Schulman et al. Reference Schulman, Wolski, Dhariwal, Radford and Klimov2017), a deep reinforcement-learning algorithm designed for continuous state and action spaces. The policy network maps the swimmer state to a continuous control value corresponding to the shape factor
$\varLambda$
. We assume that the microswimmer senses its state at discrete times separated by an update interval
$T_{\textit{u}}$
, and updates its shape
$\varLambda$
accordingly. The state is composed of local signals: swimmer orientation, strain-rate
$\mathbb{S}$
and vorticity
$\omega$
.
Due to the radial symmetry about the initial position, we decompose the orientation
$\boldsymbol{n}$
in the polar basis spanned by the radial unit vector
$\boldsymbol{e}_r = \boldsymbol{x}/|\boldsymbol{x}|$
and the azimuthal unit vector
$\boldsymbol{e}_\theta$
(figure 1
b). The projected components
$n_r = \boldsymbol{n} \boldsymbol{\cdot }\boldsymbol{e}_r$
and
$n_\theta = \boldsymbol{n} \boldsymbol{\cdot }\boldsymbol{e}_\theta$
are included in the state. The former indicates whether the swimmer is heading away from or toward the origin, while the latter provides information on which direction to rotate.
Consistent with previous studies (Kiørboe & Visser Reference Kiørboe and Visser1999; Qiu et al. Reference Qiu, Mousavi, Gustavsson, Xu, Mehlig and Zhao2022a
,
Reference Qiu, Mousavi, Zhao and Gustavssonb
; Borra et al. Reference Borra, Biferale, Cencini and Celani2022), we assume that velocity gradients are measured in the body frame
$\boldsymbol{n},\boldsymbol{p}$
(figure 1
b). In two-dimensional incompressible flow, the strain-rate tensor has two independent components,
$S_{nn} = \boldsymbol{n} \boldsymbol{\cdot }\mathbb{S} \boldsymbol{\cdot }\boldsymbol{n}$
and
$S_{np} = \boldsymbol{n} \boldsymbol{\cdot }\mathbb{S} \boldsymbol{\cdot }\boldsymbol{p}$
, while the vorticity reduces to the scalar
$\omega$
. These quantities are included in the state because they directly govern the rotational dynamics (2.2), encode the local flow topology and have been shown to be central to efficient microswimmer navigation in diverse settings (Jeffery Reference Jeffery1922; Gunnarson et al. Reference Gunnarson, Mandralis, Novati, Koumoutsakos and Dabiri2021; Monthiller et al. Reference Monthiller, Loisy, Koehl, Favier and Eloy2022; Qiu et al. Reference Qiu, Mousavi, Zhao and Gustavsson2022b
; Mousavi et al. Reference Mousavi, Qiu, Mehlig, Zhao and Gustavsson2024, Reference Mousavi, Qiu, Zhao, Mehlig and Gustavsson2025).
To optimise swimmer displacement from the initial location
$\boldsymbol{x}(t_0)$
, we introduce the step-wise reward
\begin{align} r_i = |\boldsymbol{x}(t_i) - \boldsymbol{x}(t_0)| - |\boldsymbol{x}(t_{i-1}) - \boldsymbol{x}(t_0)|, \end{align}
where
$\boldsymbol{x}(t_i)$
denotes the microswimmer position at the discrete update time
$t_i = i T_{\textit{u}}$
. Since rewards are evaluated at fixed time intervals, maximising the cumulative reward is equivalent to maximising the radial velocity away from
$\boldsymbol{x}(t_0)$
. Noting that
$\sum _{i=1}^N r_i = |\boldsymbol{x}(t_N) - \boldsymbol{x}(t_0)|$
, we define the performance index as
\begin{align} \hat {R} = |\boldsymbol{x}(t_N) - \boldsymbol{x}(t_0)|/R^{(0)}_{\textit{max}} \, , \end{align}
where
$R^{(0)}_{\textit{max}}\equiv v_{\textit{s}}T$
is the distance travelled over the duration of an episode
$T = N T_{\textit{u}}$
by a swimmer moving in a straight line from
$\boldsymbol{x}(t_0)$
at speed
$v_{\textit{s}}$
in quiescent flow. In our simulations in § 4 we use
$v_{\textit{s}}=0.707u_{\textit{rms}}$
and
$T=200\tau _{\textit{K}}$
.
Training is performed over a large number of episodes. At the start of each episode, the flow coefficients
$c_i$
and the swimmer’s initial position and orientation are randomly sampled. The swimming dynamics is then numerically integrated over the episode duration
$T$
(see Appendix A) following a policy fixed for one episode. The trajectories collected during each episode are used to update the policy parameters using the PPO algorithm (see Appendix B). Convergence is assumed when the performance index
$\hat {R}$
plateaus as the number of episode increases.
The parameters of numerical simulations and training are summarised in table 1. The swimming speed is set to
$v_{\textit{s}}=0.707u_{\textit{rms}}$
, corresponding to microswimmers with a moderate propulsion speed compared with the fluid flow. The shape factor
$\varLambda$
is constrained between
$-0.98$
and
$0.98$
, corresponding to aspect ratios between
$0.1$
and
$10$
. We consider four Kubo numbers,
$\textit{Ku} = 0.1$
,
$1$
,
$10$
and
$10^7$
, covering regimes from rapidly decorrelating to effectively steady flows. The episode duration is fixed to
$T=200\tau _{\textit{K}}$
, ensuring that initial transients are negligible. This is guaranteed by the fact that the swimmer can travel across a distance much larger than the flow correlation length,
$R^{(0)}_{\textit{max}}/\ell _f$
= 70.7 for all
$\textit{Ku}$
, and that the episode is at least ten times longer than the flow correlation time
$\tau _f$
for time-dependent flows (
$\textit{Ku}\leq 10$
). The control-update interval is set to
$T_u=0.1\tau _K$
, half of the correlation time of the most rapidly decorrelating flow at
$\textit{Ku}=0.1$
, allowing the microswimmer to respond to temporal variations in the local flow. The corresponding travelling distance over
$T_{\textit{u}}$
is approximately
$0.035\ell _f$
.
Summary of the simulation parameters and of the states (signals) available to the swimmer.
Table 1. Long description
A table with three columns and six rows, detailing simulation parameters and states for a swimmer. The columns are labeled Description, Parameter, and Value. The rows are as follows: Row 1: Description, Swimming speed; Parameter, v_s; Value, 0.707u_rms. Row 2: Description, Shape factor; Parameter, Lambda; Value, -0.98 < = Lambda < = 0.98. Row 3: Description, Kubo number; Parameter, Ku; Value, 0.1, 1, 10, 10^7. Row 4: Description, Signals; Value, n_r, n_theta, S_nn, S_np, omega. Row 5: Description, Update interval; Parameter, T_u; Value, 0.1 tau_K. Row 6: Description, Episode duration; Parameter, T; Value, 200 tau_K.
For each value of the Kubo number
$\textit{Ku}$
, the training procedure is repeated three times using different random seeds. In all cases, the performance of the learned smart strategy increases with the number of episodes and saturates after approximately
$5000$
episodes, indicating convergence (figure 8). These three independent runs are sufficient to verify that the learning dynamics is insensitive to the specific random seed: the training curves collapse and the converged performances are quantitatively similar. Consequently, we report results from a single representative training for each
$\textit{Ku}$
. The robustness of the smart strategies to stochastic perturbations to the microswimmer dynamics and to variations in training parameters (such as the update interval
$T_{\textit{u}}$
, the swimming speed
$v_{\textit{s}}$
and the episode duration
$T$
) is documented in Appendix C.
3. Baseline strategies
To assess the quality of the learned policies, we compare their performances with two baseline strategies: one based on optimal control and one based on short-time optimisation.
3.1. Naive strategy
An optimal-control formulation of the navigation problem is obtained by the cost functional
\begin{align} \mathcal{C} = - |\boldsymbol{x}(t)-\boldsymbol{x}(t_0)|^2, \end{align}
so that minimising
$\mathcal{C}$
is equivalent to maximising the distance from the initial position in the shortest time. This choice is consistent with the problem definition and with the reward in (2.5). Applying Pontryagin’s minimum principle (Bryson & Ho Reference Bryson and Ho2018) to the associated Hamiltonian shows that the optimal strategy is to maintain a constant shape factor
$\varLambda = -1$
(see Appendix D). The control problem can, in fact, be mapped to Zermelo’s minimum-time navigation problem (Zermelo Reference Zermelo1931) with an infinitely distant target. In this framework, the angular dynamics for
$\varLambda = -1$
in (2.1b) coincides with that obtained from Zermelo’s formulation (Piro et al. Reference Piro, Vilfan, Golestanian and Mahault2024). Consequently, provided the initial orientation is appropriately chosen, swimmers with
$\varLambda =-1$
reach any admissible target point, including the point farthest from the origin, faster than microswimmers with other values of
$\varLambda$
.
In turbulent and stochastic flows, however, trajectories with
$\varLambda =-1$
are extremely sensitive to initial conditions and most become trapped in high-vorticity regions (Biferale et al. Reference Biferale, Bonaccorso, Buzzicotti, Leoni and Gustavsson2019; Piro et al. Reference Piro, Golestanian and Mahault2022). Realising the formal optimal-control solution therefore requires fine tuning of the initial orientation for each initial position in a given flow realisation, and the resulting performance is not representative of a generic swimmer. For this reason, our naive baseline is not simply the theoretical optimum
$\varLambda =-1$
, but the constant-
$\varLambda$
swimmer that performs best on average over many initial conditions and flow realisations, within a physically reasonable range of shapes. To this end, we restrict the shape factor to
$\varLambda \in [-0.98,0.98]$
, corresponding to aspect ratios between 0.1 and 10, which already cover very elongated oblate and prolate shapes while remaining compatible with experimentally plausible morphologies. We then perform a parameter sweep over
$\varLambda$
in this interval, both in the stochastic flow and in DNS turbulence, and evaluate the mean distance travelled over ensembles of random initial positions and orientations. In all the considered scenarios, we find that the most elongated microswimmers (
$\varLambda = 0.98$
) achieve the largest mean distance and the highest mean performance index
$\hat {R}$
. For this reason, we adopt
$\varLambda =0.98$
as the naive constant-shape baseline in all subsequent analyses.
3.2. Short-time optimisation (STO) strategy
A more refined baseline is based on the idea of STO, which has demonstrated its capacity to solve the optimal steering strategy of a microswimmer performing vertical migration (Monthiller et al. Reference Monthiller, Loisy, Koehl, Favier and Eloy2022) or escaping high-strain regions (Mousavi et al. Reference Mousavi, Qiu, Zhao, Mehlig and Gustavsson2025). In our set-up, we derive the STO by maximising the reward in (2.5) over a short time interval
$\delta t$
. Consider a microswimmer at position
$\boldsymbol{x}_t$
at time
$t$
and choose
$\varLambda$
to maximise the incremental reward until
$t+\delta t$
, i.e. maximise
$|\boldsymbol{x}(t+\delta t)-\boldsymbol{x}(0)|$
. Shifting the coordinate system so that
$\boldsymbol{x}(0)=0$
, we have
\begin{align} |\boldsymbol{x}(t+\delta t)|= |\boldsymbol{x}|+\delta t\,\dot {\boldsymbol{x}}\boldsymbol{\cdot }\boldsymbol{e}_r+\frac {1}{2}\delta t^2\Big [\ddot {\boldsymbol{x}}\boldsymbol{\cdot }\boldsymbol{e}_r+\frac {1}{|\boldsymbol{x}|}\left (\dot {\boldsymbol{x}}^2- (\dot {\boldsymbol{x}}\boldsymbol{\cdot }\boldsymbol{e}_r)^2\right )\Big ]+O(\delta t^3), \end{align}
where the right-hand side is evaluated at time
$t$
. Using the dynamics in (2.1), the only shape-dependent contribution to (3.2) is contained in
$({1}/{2})\delta t^2\ddot {\boldsymbol{x}}\boldsymbol{\cdot }\boldsymbol{e}_r$
\begin{align} |\boldsymbol{x}(t+\delta t)|= \frac {v_{\textit{s}}\varLambda \delta t^2}{2}\boldsymbol{e}_r\boldsymbol{\cdot }(\mathbb{I}-\boldsymbol{n}\boldsymbol{n})\boldsymbol{\cdot }\mathbb{S}(\boldsymbol{x}(t),t)\boldsymbol{\cdot }\boldsymbol{n} +\mbox{$\varLambda $-independent terms}+O(\delta t^3). \end{align}
Here,
$\boldsymbol{e}_r\boldsymbol{\cdot }(\mathbb{I}-\boldsymbol{n}\boldsymbol{n})$
is the projection of
$\boldsymbol{e}_r$
on the subspace perpendicular to
$\boldsymbol{n}$
. In two spatial dimensions, this projection becomes
$(\boldsymbol{e}_r\boldsymbol{\cdot }\boldsymbol{p})\boldsymbol{p}=-n_\theta \boldsymbol{p}$
, implying that
$|\boldsymbol{x}(t+\delta t)|$
in this short-time expansion is maximised by choosing
\begin{align} \varLambda =-\varLambda _{\textit{max}}\textrm {sign}(n_\theta (t))\,\textrm {sign}(S_{np}(\boldsymbol{x}(t),t)). \end{align}
The resulting STO strategy is therefore a bang–bang control: the shape factor
$\varLambda$
always takes one of the two extremal admissible values
$\pm \varLambda _{\textit{max}}$
. This is consistent with Pontryagin’s maximum principle for a control that enters linearly in the cost functional and is bounded, which predicts that optimal controls saturate the bounds whenever possible (Bryson & Ho Reference Bryson and Ho2018); here, we obtain this behaviour directly from the short-time perturbative expansion.
In conclusion, STO is achieved by taking
$|\varLambda |$
constant over
$\delta t$
and as large as allowed (
$\varLambda _{\textit{max}}$
), with the sign set by the local strain and microswimmer orientation. If this strategy is followed and updated continuously, the angular dynamics (2.2) becomes
\begin{align} \dot \theta &=\frac {1}{2}\omega (\boldsymbol{x}(t),t)-\varLambda _{\textit{max}}\textrm {sign}(n_\theta (t))|S_{np}(\boldsymbol{x}(t),t)|. \end{align}
The mechanism can be understood as follows. Neglecting
$\omega (\boldsymbol{x}_t,t)$
, the angular dynamics has fixed points at
$n^*_\theta =0$
, which corresponds to
$\boldsymbol{n}^*=\pm \boldsymbol{e}_r(t)$
. The sign chosen in (3.4) is such that
$\boldsymbol{n}^*=+\boldsymbol{e}_r(t)$
is stable. This deformation causes the strain to rotate the microswimmer away from its initial position, thereby maximising its radial swimming velocity. In simulations, we set
$\varLambda _{\textit{max}}=0.98$
and update
$\varLambda$
using the same update interval
$T_{\textit{u}}$
as in the smart strategy, ensuring a fair comparison of performance.
4. Results
We first compare smart navigation strategies learned via reinforcement learning with naive and STO baselines across flow regimes. We then analyse the smart strategies, identifying the physical mechanisms that govern their performances in rapidly fluctuating (
$\textit{Ku}\to 0$
) and steady (
$\textit{Ku}\to \infty$
) flows, and how these mechanisms interpolate at intermediate Kubo numbers. From this, we distil a minimal analytical model that captures the essential ingredients of efficient navigation. Finally, we evaluate the smart strategy and the minimal model in two-dimensional DNS of turbulence in the direct enstrophy-cascade regime.
4.1. Performance comparison across flow regimes
Comparison across flow regimes of reinforcement-learned smart strategy (orange), naive strategy (
$\varLambda =0.98$
) (black) and the STO strategy (3.4) (green). (a) Normalised performance index
$\hat {R}$
(2.6) against Kubo number
$\textit{Ku}$
. Error bars show the standard deviation of
$\hat {R}$
over 100 flow realisations with
$10^3$
trajectories each. (b–e) Final positions of
$10^3$
microswimmers starting from the origin (red star) for (b)
$\textit{Ku}=0.1$
, (c)
$\textit{Ku}=1$
, (d)
$\textit{Ku}=10$
, (e)
$\textit{Ku}=10^7$
.
Figure 2(a) shows the maximal normalised performance index
$\hat {R}$
versus Kubo number for different control strategies. The STO baseline achieves its highest performance at small
$\textit{Ku}$
values, and
$\hat {R}$
decreases steadily as
$\textit{Ku}$
grows. This behaviour arises because the STO strategy focuses on optimising short-term travel distance (see § 3.2). For small
$\textit{Ku}$
, where flow gradients decorrelate rapidly, the STO strategy already captures the essential physics, and we show in § 4.2 that the smart strategy effectively reduces to STO in this limit. As flow persistence increases, however, STO becomes progressively less effective, and in § 4.3 we demonstrate that the smart strategy in the large-
$\textit{Ku}$
regime relies on qualitatively different mechanisms that exploit long-time correlations in the flow.
The naive (constant shape) baseline shows a non-monotonic dependence on
$\textit{Ku}$
and performs worse than the STO strategy except at very large
$\textit{Ku}$
(
$\textit{Ku}=10^7$
), where the flow is effectively steady, causing the STO strategy to fail, as explained in the previous paragraph. In contrast, the smart strategy obtained via reinforcement leaning based on all signals (
$n_r$
,
$n_\theta$
,
$S_{nn}$
,
$S_{np}$
,
$\omega$
) maintains high performance across all values of
$\textit{Ku}$
. For
$\textit{Ku}\le 1$
, it matches the STO performance, indicating little room for improvement in rapidly fluctuating flows where STO is essentially optimal. At larger
$\textit{Ku}$
, however, it significantly outperforms both baselines: at
$\textit{Ku}=10$
, the displacement is approximately
$30\,\,\%$
larger than STO, and in the quasi-steady limit (
$\textit{Ku}=10^7$
) the gain exceeds
$50\,\%$
relative to both STO and the naive strategies. These improvements reflect not only the degradation of STO as
$\textit{Ku}$
increases, but also the fact that the performance of the smart strategy itself grows markedly with
$\textit{Ku}$
. This trend shows that the smart policy actively exploits long-term flow correlations that become stronger at larger
$\textit{Ku}$
.
Figure 2(b–e) shows strategy performance via the final positions reached at different Kubo numbers. The trends mirror those of
$\hat R$
: for large
$\textit{Ku}$
, smart microswimmers (orange) travel substantially farther than STO and naive swimmers, whereas for small
$\textit{Ku}$
, STO performs comparably to the smart strategy. Representative trajectories are provided in supplementary movies are available at https://doi.org/10.1017/jfm.2026.11923.
Taken together, these results show that the reinforcement-learning strategy enables faster escape from the initial position, particularly in slowly evolving flows. The non-monotonic dependence of the normalised performance index on
$\textit{Ku}$
, with a minimum around
$\textit{Ku}=1$
, reflects that navigation is most challenging in this intermediate regime, where the flow is neither rapidly decorrelating nor quasi-steady and neither STO nor quasi-steady strategies are fully effective. This observation further suggests that distinct mechanisms underlie efficient navigation in rapidly fluctuating and steady flows. In the following sections, we analyse these mechanisms and their dependence on the flow regime.
4.2. Smart strategy in rapidly fluctuating flow
$\textit{Ku}\to 0$
: STO recovery
(a) Trajectories of microswimmers using different strategies at
$\textit{Ku}=0.1$
: smart (orange), naive (
$\varLambda =0.98$
) (black) and STO (3.4) (green). (b) Scatter plot of
$\varLambda$
against
$n_{\theta } S_{np}$
for the smart and STO strategies. Inset:
$\varLambda$
against
$n_\theta$
and
$S_{np}$
for the smart strategy, based on
$10^3$
trajectories.
Figure 3. Long description
Panel A: A plot shows the trajectories of microswimmers using different strategies. The x-axis is labeled x/R_max(0) and the y-axis is labeled y/R_max(0). Three different strategies are represented: smart (orange), naive (black), and STO (green). The trajectories are spread out from the origin, with each strategy showing distinct patterns. Panel B: A scatter plot displays the relationship between n_theta*S_np*tau_K and Lambda for the smart (orange) and STO (green) strategies. The x-axis is labeled n_theta*S_np*tau_K and the y-axis is labeled Lambda. The inset shows a scatter plot of n_theta against S_np*tau_K for the smart strategy, with the x-axis labeled S_np*tau_K and the y-axis labeled n_theta. The color scale ranges from -1 to 1, indicating different values of Lambda.
In the rapidly fluctuating regime (
$\textit{Ku} = 0.1$
), the smart strategy effectively coincides with STO. Figure 2(a) shows identical performance, and the trajectories are qualitatively the same: nearly straight paths directed away from the initial position (figure 3
a). Moreover, the learned control
$\varLambda$
for all signals (
$n_r$
,
$n_\theta$
,
$S_{nn}$
,
$S_{np}$
,
$\omega$
) collapses onto the STO prediction (3.4), when plotted against
$n_\theta S_{np}$
(figure 3
b). Training using only
$n_\theta$
and
$S_{np}$
yields essentially the same behaviour as training with the full state vector, confirming that reinforcement learning recovers STO as the optimal strategy at small
$\textit{Ku}$
(data not shown).
The effectiveness of STO at small
$\textit{Ku}$
follows from the rapid temporal decorrelation of the flow. As velocity gradients fluctuate quickly, long-term planning provides little advantage, and STO approximates the globally optimal strategy. Consequently, the smart strategy does not outperform STO.
In the rapidly fluctuating regime, STO trajectories are nearly straight. The effective STO dynamics in (3.5) breaks symmetry, making swimmers on average align with the radial direction for all values of
$\textit{Ku}$
. However, vorticity causes fluctuations around this direction, and for small
$\textit{Ku}$
it acts as white noise on the otherwise stable orientation. The orientation change over a time interval
$\tau$
due to vorticity is
\begin{align} \Delta \theta = \int _0^{\tau } \omega \textrm {d}t. \end{align}
In the limit
$\tau \gg \tau _{\textit{f}}$
, this yields the diffusive scaling
$(\Delta \theta )^2\approx 2\omega _{\textit{rms}}^2 \tau _{\textit{ f}} \tau$
. A shorter correlation time
$\tau _{\textit{f}}$
therefore reduces orientation perturbations over a fixed time interval, helping the microswimmer remain close to the optimal direction. In this regime, STO is sufficient: it captures the essential physics of efficient microswimmer navigation.
4.3. Smart strategy in steady flow
$\textit{Ku}\to \infty$
: correlation among
$n_r$
,
$S_{np}$
and
$\omega$
Shape factor
$\varLambda$
(colour) for the smart strategy at
$\textit{Ku}=10^7$
as a function of (a)
$n_r$
,
$S_{np}$
and
$\omega$
; (b)
$S_{np}$
and
$\omega$
conditioned on
$n_r\ge 0.8$
; (c)
$S_{np}$
and
$\omega$
with
$n_r\le -0.8$
. Data are based on
$4\times 10^4$
trajectories.
In the limit of large Kubo numbers, the smart strategy differs qualitatively from STO and performs substantially better. To clarify the mechanism, we analysed correlations between the learned control
$\varLambda$
and all three-signal combinations, identifying
$n_r$
,
$S_{np}$
and
$\omega$
as the dominant contributors to the policy (see also § 4.5 for the corresponding ablation study, which provides quantitative evidence in support of this conclusion). Figure 4(a) shows
$\varLambda$
as a function of these variables, revealing non-trivial correlations between
$\varLambda$
and strain/vorticity that depend sensitively on swimmer orientation. For orientations predominantly away from the initial position (
$n_r\gt 0$
), the policy organises clearly in the
$(S_{np},\omega )$
plane (figure 4
b):
$\varLambda \gt 0$
when
$S_{np}$
and
$\omega$
have opposite signs or are both small, and
$\varLambda \lt 0$
when they share the same sign. Thus, the smart strategy selects
$\varLambda$
so that the strain-induced rotation term
$\varLambda \boldsymbol{p} \boldsymbol{\cdot }\mathbb{S} \boldsymbol{\cdot }\boldsymbol{n}$
counteracts the vorticity contribution in the rotational dynamics (2.2), reducing the net angular velocity. The swimmer thereby stabilises its orientation as it moves away from the initial position, thereby weakening the effect of persistent velocity gradients. This mechanism is absent in short-time-optimal strategies and becomes effective only when the flow has long enough memory.
When the microswimmer is oriented towards the initial location (
$n_r\lt 0$
), a different behaviour emerges: the smart policy strongly favours oblate shapes (
$\varLambda \lt 0$
), largely independent of the instantaneous values of
$S_{np}$
and
$\omega$
(figure 4
c). Oblate microswimmers preferentially align against the local flow direction and exhibit self-trapping (Jeffery Reference Jeffery1922; Borgnino et al. Reference Borgnino, Gustavsson, De Lillo, Boffetta, Cencini and Mehlig2019). Here, this mechanism acts as a protective response: by reducing its mobility, the microswimmer suppresses backward motion and avoids being advected back towards the initial position.
In addition to the two dominant mechanisms at large
$\textit{Ku}$
identified in figures 4(b) and 4(c), we identify a subdominant mechanism involving the second strain component,
$S_{nn}$
. In the smart strategy,
$\varLambda$
is weakly negatively correlated with
$S_{nn}$
: extreme positive (negative) values of
$S_{nn}$
are associated with negative (positive)
$\varLambda$
. For example, among the 1 % most negative values of
$S_{nn}$
, the probability of
$\varLambda \gt 0$
is 82 %. Because such events are rare, their contribution to the overall performance is limited. Still, the trend is consistent with prolate and oblate particles known to align with stretching and compressive directions in turbulence (Pumir & Wilkinson Reference Pumir and Wilkinson2011; Parsa et al. Reference Parsa, Calzavarini, Toschi and Voth2012; Gustavsson et al. Reference Gustavsson, Einarsson and Mehlig2014; Zhan et al. Reference Zhan, Sardina, Lushi and Brandt2014; Ni et al. Reference Ni, Ouellette and Voth2014; Zhao & Andersson Reference Zhao and Andersson2016). By actively modulating its shape, the smart swimmer disrupts this passive alignment, helping it avoid strongly stretching or compressive regions that would otherwise destabilise its orientation during navigation.
4.4. Intermediate
$\textit{Ku}$
transition
Shape factor
$\varLambda$
(colour) for the smart strategy as a function of
$S_{np}$
and
$n_\theta$
(top row), and
$S_{np}$
and
$\varOmega$
(bottom row). Panels show: (a, e)
$\textit{Ku}=0.1$
; (b, f)
$\textit{Ku}=1$
; (c, g)
$\textit{Ku}=10$
; (d, h)
$\textit{Ku}=10^7$
. The data are based on
$10^3$
trajectories.
Figure 5. Long description
Panel A: A scatter plot representing the relationship between S np tau K and n theta. The horizontal axis is labeled S np tau K, and the vertical axis is labeled n theta. The data points are color-coded based on the shape factor, ranging from -1 to 1. The plot shows a concentration of data points around the center with a spread towards the edges, indicating some correlation between the variables. Panel B: A scatter plot representing the relationship between S np tau K and n theta. The horizontal axis is labeled S np tau K, and the vertical axis is labeled n theta. The data points are color-coded based on the shape factor, ranging from -1 to 1. The plot shows a more vertical distribution of data points, suggesting a different relationship compared to Panel A. Panel C: A scatter plot representing the relationship between S np tau K and n theta. The horizontal axis is labeled S np tau K, and the vertical axis is labeled n theta. The data points are color-coded based on the shape factor, ranging from -1 to 1. The plot shows a more dispersed distribution of data points, indicating a weaker correlation between the variables. Panel D: A scatter plot representing the relationship between S np tau K and n theta. The horizontal axis is labeled S np tau K, and the vertical axis is labeled n theta. The data points are color-coded based on the shape factor, ranging from -1 to 1. The plot shows a similar dispersed pattern as Panel C, with no clear trend. Panel E: A scatter plot representing the relationship between S np tau K and omega tau K. The horizontal axis is labeled S np tau K, and the vertical axis is labeled omega tau K. The data points are color-coded based on the shape factor, ranging from -1 to 1. The plot shows a concentration of data points around the center with a spread towards the edges, indicating some correlation between the variables. Panel F: A scatter plot representing the relationship between S np tau K and omega tau K. The horizontal axis is labeled S np tau K, and the vertical axis is labeled omega tau K. The data points are color-coded based on the shape factor, ranging from -1 to 1. The plot shows a more vertical distribution of data points, suggesting a different relationship compared to Panel E. Panel G: A scatter plot representing the relationship between S np tau K and omega tau K. The horizontal axis is labeled S np tau K, and the vertical axis is labeled omega tau K. The data points are color-coded based on the shape factor, ranging from -1 to 1. The plot shows a more dispersed distribution of data points, indicating a weaker correlation between the variables. Panel H: A scatter plot representing the relationship between S np tau K and omega tau K. The horizontal axis is labeled S np tau K, and the vertical axis is labeled omega tau K. The data points are color-coded based on the shape factor, ranging from -1 to 1. The plot shows a similar dispersed pattern as Panel G.
At intermediate values of the Kubo number,
$\textit{Ku}=1$
and
$10$
, the smart strategies smoothly interpolate between the small-
$\textit{Ku}$
and large-
$\textit{Ku}$
limits. This gradual transition is evident in figure 5, which shows the smart control action
$\varLambda$
as a function of the relevant state variables for different values of
$\textit{Ku}$
.
At
$\textit{Ku}=0.1$
, the control landscape of
$S_{np}$
and
$n_\theta$
shows the characteristic dependence associated with the STO strategy (3.4) (figures 3
b and 5
a). As
$\textit{Ku}$
increases, this structure progressively weakens and eventually disappears (figure 5
b–d), reflecting that STO becomes less effective in temporally correlated flows. Accordingly, the smart strategy relies less on this mechanism when the flow decorrelates more slowly.
A complementary trend is observed in the stabilisation mechanism of the quasi-steady regime. The
$S_{np}$
–
$\omega$
correlation, which dominates the smart strategy at
$\textit{Ku}=10^7$
(figures 4
b and 5
h), transitions gradually as
$\textit{Ku}$
decreases (figure 5
e–h). While this correlation persists even at small Kubo numbers, its physical origin changes. At
$\textit{Ku}=0.1$
, the correlation emerges because the vorticity acts as a perturbation to the orientation
$n_\theta$
(see § 4.2), which correlates
$\omega$
with
$n_\theta$
. Hence,
$\omega$
acquires a correlation with
$S_{np}$
via the
$n_\theta$
–
$S_{np}$
correlation in the STO strategy (3.4). As
$\textit{Ku}$
increases, long-time orientation stabilisation becomes more important as flow correlation time increases.
Remarkably, because the strategies are trained independently at each value of
$\textit{Ku}$
, the observed smooth evolution of control patterns is not imposed by construction but arises naturally from the interaction between the microswimmer and the flow. This continuity suggests that efficient microswimmer navigation is governed by a small set of underlying physical mechanisms whose relative importance is controlled by the flow regime. The smart microswimmer adapts by continuously reweighting these mechanisms in response to the temporal and spatial structure of the flow, rather than switching abruptly between qualitatively distinct strategies.
4.5. From learned smart strategies to a minimal analytical model of navigation
(a) Relative performance of smart strategies compared with the STO baseline at
$\textit{Ku}=10$
, trained with different sets of input signals. Brackets denote the signals available to each strategy. (b) Performance as a function of
$\textit{Ku}$
for the smart strategy and the minimal model (4.2). (c) Fitted coefficients of the minimal model as functions of
$\textit{Ku}$
.
In §§ 4.2 and 4.3, we identified four distinct physical mechanisms that underpin efficient navigation in stochastic flows: (i) the STO mechanism, which relies on
$S_{np}$
and
$n_\theta$
and is effective at small
$\textit{Ku}$
; (ii) an orientation-stabilisation mechanism, controlled by
$S_{np}$
and
$\omega$
, which dominates at large
$\textit{Ku}$
; (iii) a self-trapping mechanism, associated with negative radial orientation
$n_r\lt 0$
, also relevant at large
$\textit{Ku}$
; and (iv) an extreme strain-avoidance mechanism, governed by
$S_{nn}$
, which contributes primarily in the large-
$\textit{Ku}$
regime.
To quantify the relative importance of these mechanisms in the smart strategy, we conduct a ‘signal-ablation’ experiment at
$\textit{ Ku}=10$
. In this procedure, the microswimmer is trained multiple times using progressively reduced sets of input signals. By selectively removing the signals associated with specific mechanisms, the corresponding behaviours are suppressed, allowing a direct assessment of their relevance to strategy performance.
Figure 6(a) reports the performance of the trained strategies relative to the STO baseline. When all signals are available, the smart strategy outperforms the STO baseline by nearly
$30\,\%$
. Removing
$S_{nn}$
suppresses the extreme strain-avoidance mechanism and leads only to a marginal decrease in performance, consistent with its secondary role (see § 4.3). Further removing
$\omega$
, such that only
$n_r$
,
$n_\theta$
and
$S_{np}$
remain, eliminates the orientation-stabilisation mechanism and causes an abrupt drop in performance to the level of the STO strategy. Excluding
$n_r$
as well results in just a mild decrease, since the remaining signals
$n_\theta$
and
$S_{np}$
are sufficient to recover the STO mechanism. Beyond figure 6(a), any policy trained on three or fewer signals is outperformed by STO (not shown).
To further assess whether each signal is actively used by the smart swimmer, we evaluated, without retraining, the policy trained at
$\textit{Ku}=10$
after randomly permutating the values in selected input signals across times and trajectories, thereby disrupting their correlations with the remaining inputs. The resulting performance degradation follows the same hierarchy as in the ablation study, showing that the policy relies on these correlations during navigation. This confirms that the dominant signals (i.e.
$n_r$
,
$n_\theta$
,
$S_{np}$
and
$\omega$
) are not only important for learning but also for execution.
Amongst all inputs,
$S_{np}$
emerges as the most critical signal. This is expected because it underlies both the STO and orientation-stabilisation mechanisms, and this is confirmed by the red cross in figure 6(a): removing
$S_{np}$
while retaining all other signals leads to a failure of the smart strategy. For completeness, we also trained policies using
$S_{np}$
as the only input signal and found that their performance remains comparable to the naive baseline across Kubo numbers, confirming that none of the mechanisms identified can operate effectively on
$S_{np}$
alone.
Guided by this mechanistic understanding, we propose a minimal analytical model for the smart strategy
\begin{align} \varLambda = \varLambda _{\textit{max}} \tanh \Bigl [ & -c_{\textit{STO}} n_\theta \, \textrm {sign}(S_{np}) - c_{\textit{stab}} H(n_r)\omega \tau _{\textit{K}}\, \textrm {sign}(S_{np}) \nonumber\\ & + c_{\textit{trap}} n_r - c_{\textit{str}} \tanh (S_{nn} \tau _{\textit{K}}) \Bigr ], \end{align}
where
$H(x)$
denotes the Heaviside function and
$\left \{c_{\textit{STO}},c_{\textit{stab}}, c_{\textit{trap}}, c_{\textit{str}}\right \}$
are
$\textit{Ku}$
-dependent coefficients obtained by fitting the model to the learned smart strategies for different
$\textit{Ku}$
using the curve
$\_$
fit function from the SciPy library (Virtanen et al. Reference Virtanen2020). The model combines the four identified mechanisms through a weighted sum, while the outer hyperbolic tangent constrains the control output to the admissible range of
$\varLambda$
.
Each term in (4.2) has a clear physical interpretation. The first term represents the STO strategy. The second term corresponds to the orientation-stabilisation mechanism and is activated only when
$n_r\gt 0$
via the Heaviside function. The third term captures the positive correlation between
$\varLambda$
and
$n_r$
responsible for the self-trapping mechanism (see figure 4), while the final term models the strain-avoidance response associated with
$S_{nn}$
.
The performance of the minimal model is shown in figure 6(b). Its efficiency is nearly indistinguishable from that of the full smart strategy across the range of considered
$\textit{Ku}$
values, demonstrating that the model captures the essential physics underlying efficient navigation.
Beyond reproducing performance, the minimal model provides direct insight into how navigation strategies evolve across flow regimes, as the fitted coefficients quantify the relative weight of each mechanism in the composite strategy. Figure 6(c) shows the coefficients as functions of
$\textit{Ku}$
. Consistent with the STO mechanism being effective only at small
$\textit{Ku}$
, the magnitude of
$c_{\textit{STO}}$
decreases monotonically with increasing
$\textit{Ku}$
. In contrast, the magnitudes of the coefficients associated with large-
$\textit{Ku}$
mechanisms (
$c_{\textit{stab}}$
,
$c_{\textit{trap}}$
and
$c_{\textit{str}}$
) tend to increase with
$\textit{Ku}$
.
4.6. Transfer learning in DNS turbulent flows
(a) Temporal snapshots of the vorticity field of the turbulent flow obtained from DNS. The orange curve is a qualitative illustration of a microswimmer trajectory. The red star indicates its initial position, and the orange spheroids represent its shape and position in the flow field at successive time snapshots. (b) Probability distribution function (PDF) of vorticity
$\omega$
(blue), normal strain
$S_{xx}$
(green) and shear strain
$S_{xy}$
(red) of the turbulent flow, compared with a Gaussian distribution (black). (c) Performance of the strategies relative to the STO baseline (3.4) in two-dimensional turbulence. The orange curve represents the strategy learned in stochastic flows at different
$\textit{Ku}$
. The black dashed curve corresponds to the minimal model (4.2) using the coefficients from figure 6(c). The black dotted line shows the performance of the strategy trained in the turbulent flow. Error bars indicate standard deviations over random initial positions and independent flow realisations. (d, e) Shape factor
$\varLambda$
projected onto (d)
$S_{np}$
and
$n_\theta$
; (e)
$S_{np}$
and
$\omega$
for the DNS-trained strategy.
Figure 7. Long description
Panel A: A series of temporal snapshots of the vorticity field of the turbulent flow obtained from direct numerical simulations (DNS). The orange curve illustrates a microswimmer trajectory, with the red star indicating its initial position and the orange spheroids representing its shape and position in the flow field at successive time snapshots. Panel B: A probability distribution graph comparing vorticity (blue), normal strain (green), and shear strain (red) of the turbulent flow with a Gaussian distribution (black). The x-axis represents the normalized values, and the y-axis represents the probability distribution function (PDF). Panel C: A line graph showing the performance of different strategies relative to the STO baseline in two-dimensional turbulence. The orange curve represents the strategy learned in stochastic flows at different values, the black dashed curve corresponds to the minimal model, and the black dotted line shows the performance of the strategy trained in the turbulent flow. Error bars indicate standard deviations over random initial positions and independent flow realizations. The x-axis represents the normalized value Ku, and the y-axis represents the performance ratio R/R_STO. Panel D: A scatter plot showing the shape factor projected onto nθ and SnpτK for the DNS-trained strategy. The x-axis represents SnpτK, and the y-axis represents nθ. Panel E: A scatter plot showing the shape factor projected onto ωτK and SnpτK for the DNS-trained strategy. The x-axis represents SnpτK, and the y-axis represents ωτK. The color bar indicates the shape factor Λ.
We test the robustness of the smart strategies by transferring them to two-dimensional turbulence generated by DNS. The flow consists of an incompressible velocity field,
$\boldsymbol{\nabla }\boldsymbol{\cdot }\boldsymbol u = 0$
, governed by the Navier–Stokes equations
\begin{align} \partial _t \boldsymbol{u}+\boldsymbol{u}\boldsymbol{\cdot }\boldsymbol{\nabla } \boldsymbol{u}=-\boldsymbol{\nabla }p +\nu \varDelta \boldsymbol{u}-\alpha \boldsymbol{u} +\boldsymbol{F}, \end{align}
where
$p$
is pressure,
$\nu$
the viscosity,
$\boldsymbol{F}$
a large-scale external forcing that drives an enstrophy cascade at smaller scales (Boffetta & Ecke Reference Boffetta and Ecke2012) and the term
$-\alpha \boldsymbol{u}$
represents friction preventing energy accumulation at large scales. Equation (4.3) is integrated using a standard pseudospectral method (see Appendix A). Representative snapshots of the flow vorticity field and the statistical properties of flow-velocity gradients are shown in figures 7(a) and 7(b).
The orange curve in figure 7(c) shows the DNS performance of the smart strategies trained in stochastic flows at different
$\textit{Ku}$
. Performance depends on
$\textit{Ku}$
: the strategy trained at
$\textit{Ku}=10$
performs best, exceeding the STO baseline and nearly matching a strategy trained in DNS turbulence (black dotted line). This indicates that stochastic flows at
$\textit{Ku}=10$
capture key dynamical features of DNS turbulence, consistent with earlier observations that large but finite-
$\textit{Ku}$
models reproduce essential Lagrangian statistics of turbulent transport of passive and active particles (Gustavsson et al. Reference Gustavsson, Berglund, Jonsson and Mehlig2016; Borgnino et al. Reference Borgnino, Gustavsson, De Lillo, Boffetta, Cencini and Mehlig2019; Bec et al. Reference Bec, Gustavsson and Mehlig2024).
Figures 7(d) and 7(e) show that the
$(S_{np}, n_\theta )$
and
$(S_{np}, \omega )$
correlations underlying the STO and orientation-stabilisation mechanisms in stochastic flows, persist in the DNS-trained strategy. Moreover, the close agreement between the minimal model (4.2) (black dashed in figure 7
c) and the transferred strategy at
$\textit{Ku}=10$
further confirms that these mechanisms capture the essential physics of efficient navigation and remain robust in realistic turbulence.
Residual differences between the DNS-trained and transferred strategies likely reflect fine-scale structural discrepancies between stochastic and Navier–Stokes turbulence. Away from
$\textit{Ku}=10$
, these mismatches grow and transfer performance degrades. An exception is
$\textit{Ku}=1$
, where the transferred strategy performs unexpectedly well, outperforming both STO and the minimal model. This is because, at intermediate
$\textit{Ku}$
, the smart strategy learns to combine STO with long-time stabilisation, enhancing robustness to the broader range of scales present in DNS turbulence. By contrast, the minimal model retains only the dominant mechanisms identified in the small- and large-
$\textit{Ku}$
limits, and therefore reproduces the smart strategy very well in those regimes and at
$\textit{Ku}=10$
. Around
$\textit{Ku}=1$
, where navigation is intrinsically more difficult, the smart policy relies on more nuanced combinations of the available signals than those captured by the minimal model. Consequently, the minimal model underestimates the performance of the smart strategy and cannot fully reproduce the high transferability observed in two-dimensional turbulence.
5. Conclusions
We have investigated the navigation of shape-changing spheroidal microswimmers tasked with maximising displacement from their initial position. Using reinforcement learning, we identified smart strategies in which the microswimmer adaptively adjusts its shape factor,
$\varLambda$
, in response to local orientation and fluid-gradient signals.
Across flows with different temporal correlations, characterised by the Kubo number
$\textit{Ku}$
, defined as the ratio of flow correlation to advection time scales, the learned smart strategies consistently outperform both a naive fixed-shape baseline and the STO baseline (3.4). Efficient navigation relies upon multiple complementary mechanisms: the STO mechanism, based on
$n_\theta$
and
$S_{np}$
, maximises short-time displacement; orientation stabilisation, driven by
$S_{np}$
and vorticity
$\omega$
, reduces rotation induced by fluid gradients and stabilises the orientation when pointing away from the initial position (
$n_r\gt 0$
); adopting an oblate shape when pointing toward the initial location (
$n_r\lt 0$
) prevents backward motion; and avoidance of strong normal strain (
$S_{nn}$
) prevents destabilising alignment. The relative contribution of these mechanisms varies with the flow’s correlation time, as quantified by the minimal model (4.2).
The universality of these mechanisms is confirmed by transferring strategies trained in stochastic flows to fully resolved DNS turbulence. Strategies learned at
$\textit{Ku}=10$
perform nearly as well in turbulence as those trained directly in the DNS environment, showing that the stochastic flow model effectively captures the essential Lagrangian dynamics and provides a cost-effective framework for exploring navigation strategies and their underlying physics. Beyond quantitative performance, these results highlight a novel navigation paradigm in which adaptive morphology, rather than direct actuation, enables microswimmers to exploit local mechanical cues.
Furthermore, although we allowed the shape factor
$\varLambda$
to span a wide interval in our main analysis, additional tests show that the smart strategy still outperforms naive and STO baselines when
$\varLambda$
is restricted to more moderate, biologically and experimentally plausible ranges, including the purely prolate case
$\varLambda \in [0,0.98]$
(see Appendix C), indicating that the navigation mechanisms identified here do not rely on access to extreme disc- or rod-like shapes.
Future work could explore more complex tasks, such as point-to-point navigation, incorporate energetic costs of active shape control, which may reveal qualitatively different strategies (Piro et al. Reference Piro, Vilfan, Golestanian and Mahault2024), and investigate hydrodynamic or mechanical interactions among multiple microswimmers, potentially enabling cooperative or adversarial strategies in complex flows (Borra et al. Reference Borra, Biferale, Cencini and Celani2022). Another future direction is to extend the navigation strategy to three-dimensional flows, where the additional degrees of freedom increase the complexity of the state space and dynamics, posing new challenges for control. It would also be interesting to characterise in which other classes of navigation tasks – such as exploration, or pursuit of distant targets – the STO and long-time stabilisation mechanisms identified here remain effective, thereby clarifying the generality and limits of the proposed control paradigm.
Moreover, the present study focuses on reactive strategies in which the microswimmer uses its instantaneous orientation and local velocity gradients without memory of past states. While such memoryless control is natural in the limits of very small and very large Kubo number, where the flow decorrelates rapidly or is effectively frozen, memory-based strategies (El Khiyati et al. Reference El Khiyati, Chesneaux, Giraldi and Bec2023; Sankaewtong et al. Reference Sankaewtong, Molina and Yamamoto2024) are expected to become advantageous at intermediate Kubo numbers, where coherent structures exhibit temporal persistence and could be exploited by, e.g. enriching the swimmer’s state with temporal derivatives of strain and vorticity. A systematic exploration of such memory-augmented strategies is therefore a promising future direction. Furthermore, while this study employs an idealised spheroidal model, future research may explore alternative adaptive morphologies, such as swimmers with flexible (Tarama et al. Reference Tarama, Menzel and Löwen2014) or composite (Zou et al. Reference Zou, Liu, Young, Pak and Tsang2022) body structures.
Supplementary movies
Supplementary movies are available at https://doi.org/10.1017/jfm.2026.11923.
Acknowledgements
The authors thank A. Celani and N. Mousavi for helpful discussions.
Funding
We acknowledge financial support from Vetenskapsrådet, Grants No. 2018-03974, No. 2023-03617 (J.Q. and K.G.), and No. 2021-4452 (B.M.). K.G., B.M. and J.Q. acknowledge support from the Knut and Alice Wallenberg Foundation, Grant No. 2019.0079. This work was also supported by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (Grant Agreement No. 882340).
Declaration of interests
The authors report no conflict of interest.
Appendix A. Numerical simulation
We compute microswimmer trajectories using the Eulerian–Lagrangian scheme of Qiu et al. (Reference Qiu, Mousavi, Gustavsson, Xu, Mehlig and Zhao2022a
,
Reference Qiu, Mousavi, Zhao and Gustavssonb
) and Mousavi et al. (Reference Mousavi, Qiu, Mehlig, Zhao and Gustavsson2024, Reference Mousavi, Qiu, Zhao, Mehlig and Gustavsson2025). In this scheme, (2.1) is integrated with a second-order method with time step
$\delta t$
. At each step, the fluid velocity and its gradients at the swimmer position are obtained by second-order interpolation from the Eulerian flow field.
The two-dimensional stochastic flow is stored on a uniform
$400\times 400$
Eulerian grid with spacing
$0.025\ell _{\textit{f}}$
. As explained in § 2.2, velocity field in (2.3) is generated from
$M=5$
streamfunction components with coefficients evolving via an Ornstein–Uhlenbeck process using the same time step
$\delta t$
. To allow unbounded swimmer motion, the domain is periodically extended in both spatial directions. Our results are insensitive to the domain size, because the flow and its gradients decorrelate before a swimmer traverses the domain. This was confirmed using a ten times larger domain.
Two-dimensional turbulence is obtained by DNS of (4.3) by means of a standard 2/3 de-aliased pseudospectral method on a bi-periodic square domain of size
$2\pi$
using
$512\times 512$
grid points. The flow is forced at large scales by a Gaussian, temporally uncorrelated forcing
$\boldsymbol{F}$
peaked at wavenumber
$k_{\textit{f}}=3$
, in order to induce a direct enstrophy cascade. Friction
$-\alpha \boldsymbol v$
, preventing energy buildup at large scales, is numerically implemented via a hypo-friction term (Boffetta & Ecke Reference Boffetta and Ecke2012). We stored the velocity and gradient fields at approximately every tenth of the Kolmogorov characteristic time. During the simulation of microswimmers, linear interpolation is used to obtain snapshots between successive stored flow fields. The flow velocity and its gradients are then interpolated to the position of microswimmers using the same second-order Lagrangian scheme as in the stochastic flow model. Similar to the stochastic flow, the turbulent flow is also periodically extended, and the domain size is sufficiently large.
Velocity and gradient fields are stored at intervals of approximately one tenth of the Kolmogorov time. During swimmer simulations, intermediate flow snapshots are obtained by linear interpolation in time, and spatial interpolation to swimmer positions uses the same second-order Lagrangian scheme as in the stochastic model. The turbulent flow is also periodically extended, and the domain is sufficiently large to avoid finite-size effects.
In both the stochastic model and DNS turbulence, the microswimmer dynamics is integrated with the time step
$\delta t=0.01\tau _{\textit{ K}}$
. This choice resolves both flow fluctuations and the swimmer dynamics, even for the most rapidly evolving case (
$\textit{Ku}=0.1$
), where the flow correlation time
$\tau _{\textit{f}}$
is still an order of magnitude larger than
$\delta t$
.
Appendix B. Reinforcement-learning algorithm
We use PPO (Schulman et al. Reference Schulman, Wolski, Dhariwal, Radford and Klimov2017) to identify smart deformation strategies for a microswimmer tasked with maximising its displacement from its initial position. The PPO algorithm is particularly well suited to the present problem because both the state variables (orientation and local velocity-gradient signals) and the control variable (shape factor
$\varLambda$
) are continuous, whereas many value-based algorithms such as Q-learning are designed for discrete action spaces. Moreover, PPO is widely recognised as a robust on-policy actor–critic method that performs reliably in noisy environments and can be applied to a range of continuous-control tasks without extensive hyperparameter tuning (Sutton & Barto Reference Sutton and Barto2018). It optimises a stochastic policy
$\pi (\boldsymbol{a}_i|\boldsymbol{s}_i)$
, defined as the probability distribution of taking action
$\boldsymbol{a}_i$
given state
$\boldsymbol{s}_i$
at time
$t=iT_{\textit{u}}$
.
The objective is to approximate the optimal policy that maximises the expected discounted return,
$R=\sum _{i=0}^{N} \gamma ^{i}r_i$
, where the discount factor
$\gamma \in (0,1)$
balances future and instantaneous rewards. The PPO algorithm achieves this by maximising the clipped surrogate objective function
\begin{align} J(\boldsymbol{w}) = \mathbb{E}\left \{ \textrm {min}\left [P_r(\boldsymbol{w}) {A}_i,\textrm {clip}(P_r(\boldsymbol{w}), 1-\epsilon , 1+\epsilon )A_i\right] \right \} \!, \end{align}
where
$\boldsymbol{w}$
denotes the parameters of the neural-network,
$\mathbb{E}$
is the expectation over the sampled trajectories and
$P_r(\boldsymbol{w})=\pi _{\boldsymbol{w}}(\boldsymbol{a}_i|\boldsymbol{s}_i)/ \pi _{\boldsymbol{w}_{old}}(\boldsymbol{a}_i|\boldsymbol{s}_i)$
is the probability ratio between the updated and previous policies. The advantage
$A_i$
measures how much the selected action outperforms the expected action under the current policy, which is computed using generalised advantage estimation following the standard implementation (Schulman et al. Reference Schulman, Wolski, Dhariwal, Radford and Klimov2017). Maximising
$P_r(\boldsymbol{w})A_i$
increases the likelihood of favourable actions, while the clipping and minimising operations restrict excessively large policy updates, thus improving training stability (Schulman et al. Reference Schulman, Wolski, Dhariwal, Radford and Klimov2017). For instance, when an action is favourable (
$A_i\gt 0$
), the optimisation tends to increase
$P_r$
. With the clipping and minimisation, however,
$J(\boldsymbol{w})$
is capped at
$(1+\epsilon )A_i$
, preventing
$P_r$
from growing arbitrarily large. Conversely, when an action is unfavoured (
$A_i\lt 0$
), the optimisation decreases
$P_r$
. If
$P_r\lt 1-\epsilon$
,
$J(\boldsymbol{w})$
is limited to
$(1-\epsilon )A_i$
, preventing an excessive reduction of
$P_r$
.
The PPO algorithm uses two neural networks: an actor network and a critic network. The actor network takes the state
$\boldsymbol{s}_i$
as input and outputs the policy
$\pi (\boldsymbol{a}_i|\boldsymbol{s}_i)$
. We assume a Gaussian action distribution, with the network outputting the mean and the standard deviation represented by a trainable scalar parameter. During training, actions are sampled from this distribution and then passed through a
$\tanh$
function to bound the action values. Since this transformation changes the action distribution, the corresponding probability ratio and entropy terms are modified accordingly in the objective function
$J(\boldsymbol{w})$
. During evaluation, actions are sampled in the same way except that the standard deviation is set to zero, such that the actions are deterministically equal to the mean of the distribution. The critic network also takes
$\boldsymbol{s}_i$
as input and outputs a scalar state-value estimate
$V(\boldsymbol{s}_i)$
, which is used in the calculation of the advantage function
\begin{align} \begin{aligned} &A_i = \delta _i + (\gamma \lambda _{\textit{GAE}})\delta _{i+1}+\cdots +(\gamma \lambda _{\textit{GAE}})^{N-i+1}\delta _{N-1}, \\ &\text{where}\,\,\delta _i=r_i+\gamma V(\boldsymbol{s}_{i+1})-V(\boldsymbol{s}_i), \end{aligned} \end{align}
and
$\lambda _{\textit{GAE}}$
is the generalised advantage estimation parameter (see Schulman et al. (Reference Schulman, Wolski, Dhariwal, Radford and Klimov2017) for details).
Both the actor and critic networks are fully connected feed-forward neural networks with two hidden layers of 64 neurons each, using
$\tanh$
activation functions. For the actor network, a linear output layer is used for the mean of the Gaussian action distribution. The critic network employs a linear output layer for the value estimate. The network parameters are updated via gradient ascent on
$\boldsymbol{\nabla} _{\boldsymbol{w}} J(\boldsymbol{w})$
.
The main hyperparameters used in PPO training are selected based on standard choices in the PPO literature (Schulman et al. Reference Schulman, Wolski, Dhariwal, Radford and Klimov2017) and simple empirical testing to ensure stable, reproducible convergence. They are summarised as follows. The discount rate is set to
$\gamma =0.99$
, the PPO clipping parameter is set to
$\epsilon =0.1$
and the generalised advantage estimation parameter is set to
$\lambda _{\textit{GAE}}=0.99$
. An entropy regularisation term with coefficient 0.005 is included to promote exploration and prevent early convergence of policy. The actor and critic networks are optimised using the Adam optimiser as implemented in PyTorch (Paszke et al. Reference Paszke2019). The learning rates decrease linearly from
$2\times 10^{-4}$
for the actor network and
$4\times 10^{-5}$
for the critic network to zero over 5000 training episodes. Each policy update comprises four epochs, each over 200 minibatches with a batch size of 25.
The reinforcement-learning algorithm is coupled to the numerical simulation of microswimmer dynamics. In each episode, ten microswimmers are initialised with random initial positions and orientations, as described in § 2.3, and simulated in parallel to collect trajectory data efficiently. At each decision time
$t=iT_{\textit{u}}$
, the state variables are computed, and the PPO agent outputs actions that set the shape factors of the microswimmers for the subsequent time interval. The microswimmer trajectories are then advanced in time using the numerical scheme described in Appendix A. This online coupling allows the agent to learn directly from the physically resolved Lagrangian dynamics.
Training curves of the performance index
$\hat R$
against training episode. For each
$\textit{Ku}$
, three independent runs with different random seeds are shown.
Training is performed for 5000 episodes, and each run uses three independent random seeds to ensure statistical robustness. Figure 8 shows the evolution of the navigation performance as a function of training episode. The performance saturates after a sufficient number of episodes, indicating convergence of the learned strategy. The resulting strategies after training are evaluated on flow realisations that differ from the training data to test their generalisability. All results shown in the paper are obtained from this evaluation process.
Appendix C. Robustness against rotational noise and other parameters
Robustness against rotational noise. (a) Performances of smart (orange), naive (black) and STO (green) strategies at different noise levels. The smart strategy is trained at
$\textit{Ku}=10$
,
$D_{\textit{r}}=0$
. (b) Performance of the smart strategy relative to the STO baseline across varying noise levels.
Here, we test the robustness of the smart strategy learned at
$\textit{Ku}=10$
.
First, we consider the robustness against rotational noise. The two-dimensional Brownian noise is implemented as a displacement in
$\boldsymbol{n}$
with a variance of
$\sqrt {2D_{\textit{r}} \delta t}$
, where
$D_{\textit{r}}$
is the rotational diffusion coefficient, and
$\delta t$
is the time step of the integration of the dynamics. Figure 9 shows the performance of the smart, naive, and STO strategies at different noise levels. The performance of the smart strategy begins to decrease when
$D_{\textit{r}}\tau _{\textit{K}} \gt 5\times 10^{-3}$
, yet it still outperforms both baselines even at the largest noise level considered.
Figure 10(a) shows the robustness of the smart strategy against different episode durations
$T$
. The smart strategy trained at
$T=200\tau _{\textit{K}}$
performs well also at other episode durations, yielding an almost constant normalised performance
$\hat {R}$
. This indicates that, with the smart strategy (orange curve), the travel distance increases almost linearly with time
$|\boldsymbol{x}(t_N)-\boldsymbol{x}(t_0)|\propto T$
. Such ballistic-like transport surpasses the diffusive dispersion of naive microswimmers with constant shape whose normalised performance scales as
$\hat {R}\propto T^{-1/2}$
(figure 10(a), black solid curve). Figure 10(b) illustrates the sensitivity of the smart and STO strategies to the update interval
$T_{\textit{u}}$
. The performance of both strategies decreases as
$T_{\textit{u}}$
increases, reflecting the growing delay between state measurement and action execution. Since both strategies rely on signals from the local velocity gradients, they fail when
$T_{\textit{u}}$
becomes comparable to or exceeds the Kolmogorov time scale
$\tau _{\textit{K}}$
, which sets the correlation time of these gradients. Nevertheless, the smart strategy continues to outperform both the STO and naive baselines, whereas the STO strategy approaches the naive one when
$T_{\textit{u}} \gtrsim \tau _{\textit{K}}$
.
Furthermore, figure 10(c) demonstrates the robustness of strategies with respect to the swimming speed
$v_{\textit{s}}$
. The smart strategy outperforms both the STO and naive baselines on all the values of
$v_{\textit{s}}$
. Interestingly, deviating from the specific
$v_{\textit{s}}$
used during training, either by increasing or decreasing the speed, results in increased
$\hat {R}$
. The dependence of
$\hat {R}$
on
$v_{\textit{s}}$
arises from the definition of normalisation
$\hat {R}=R/R_{\textit{max}}^{(0)}$
. In the limit as
$v_{\textit{s}}\rightarrow 0$
,
$\hat {R}$
tends to infinity because the dimensional travel distance
$R$
is bounded by the flow advection (scaled by
$u_{\textit{rms}}$
), while the reference
$R_{\textit{max}}^{(0)}=v_{\textit{ s}}T$
vanishes. In contrast, in the limit of large
$v_{\textit{s}}$
, the influence of flow becomes negligible, and
$\hat {R}$
is expected to converge to unity. Overall, the superior performance of the smart strategy relative to the baselines indicates that its mechanisms are robust to variations in swimming speed.
Performance index
$\hat R$
(2.6) of smart (orange), naive (black) and STO (green) strategies evaluated for different (a) episode durations
$T$
; (b) update intervals
$T_{\textit{u}}$
; (c) swimming speeds
$v_{\textit{s}}$
. The smart strategy was trained at
$\textit{Ku}=10$
,
$T/\tau _{\textit{K}}=200$
,
$T_{\textit{u}}/\tau _{\textit{K}}=0.1$
and
$v_{\textit{ s}}/u_{\textit{rms}} = 0.707$
.
Figure 10. Long description
Three line graphs depict the performance index of smart, naive, and STO strategies under different conditions. Panel A: A line graph shows the performance index as a function of episode durations. The x-axis represents episode durations in units of tau sub K, ranging from 50 to 500. The y-axis represents the performance index, ranging from 0.1 to 1.0. The smart strategy is represented by orange circles, the naive strategy by black squares, and the STO strategy by green diamonds. The smart strategy maintains a higher performance index compared to the naive and STO strategies as episode durations increase. Panel B: A line graph shows the performance index as a function of update intervals. The x-axis represents update intervals in units of tau sub K, ranging from 0.1 to 1.6. The y-axis represents the performance index, ranging from 0.3 to 1.0. The smart strategy shows a decreasing trend, while the naive strategy remains constant, and the STO strategy shows a slight decrease. Panel C: A line graph shows the performance index as a function of swimming speeds. The x-axis represents swimming speeds in units of u sub rms, ranging from 0 to 1.5. The y-axis represents the performance index, ranging from 0.3 to 1.0. The smart strategy shows a U-shaped trend, with the lowest performance index at intermediate swimming speeds, while the naive and STO strategies show a more gradual decrease and then increase.
Robustness of smart strategy performance to restricted shape ranges at
$\textit{Ku}=10$
: normalised performance index
$\hat {R}$
for naive (black), STO (green) and smart (orange) strategies when the shape factor is constrained to
$\varLambda \in [-0.98,0.98]$
,
$[-0.8,0.8]$
,
$[-0.6,0.6]$
,
$[-0.5,0.5]$
and
$[0,0.98]$
.
Finally, to assess the robustness of the navigation mechanisms with respect to realistic constraints on swimmer shape, we evaluated the smart policy trained with
$\varLambda \in [-0.98,0.98 ]$
under progressively restricted shape ranges, without retraining. For each range, the naive baseline uses a constant shape with
$\varLambda =|\varLambda |_{\textit{max}}$
. The STO baseline is computed from (3.4) and then clipped to the admissible interval. In particular, when the action range is restricted to
$ [0,0.98 ]$
, the STO strategy sets
$\varLambda =0$
whenever (3.4) yields a negative value. The smart strategy uses the same policy as in the main text, with its output
$\varLambda$
clipped to the corresponding admissible interval. Figure 11 shows the normalised performance
$\hat {R}$
for naive (black), STO (green) and smart (orange) strategies when
$|\varLambda |_{\textit{max}}$
is constrained to
$= 0.8, 0.6, 0.5$
, corresponding to aspect ratios of approximately 3, 2 and 1.7 for prolate spheroids and 0.33, 0.5 and 0.73 for oblate spheroids, as well as for the purely prolate case
$\varLambda \in [0,0.98 ]$
. In all cases, the smart strategy maintains a clear advantage over both baselines, indicating that the smart navigation mechanisms remain effective even when only moderate shape changes, compatible with biological or artificial microswimmers, are allowed. Moreover, restricting the shape factor to
$ [0, 0,98 ]$
has little effect on the performance of either the smart strategy or the STO strategy. This suggests that allowing the swimmer to approach a spherical shape may already be sufficient to realise the benefits of shape control.
Appendix D. Optimal-control theory
In this appendix, we briefly outline how optimal-control theory leads to the escape strategy discussed in § 3. We consider a spheroidal self–propelled microswimmer whose translational and orientational dynamics in two dimensions are governed by (2.1a ) and (2.2), which we report here for convenience
\begin{align} \dot {\boldsymbol{x}} &= \boldsymbol{u} + v_{\textit{s}} \boldsymbol{n}, \end{align}
\begin{align} \dot {\theta }&=\frac {1}{2}\omega + \varLambda S_{np}, \end{align}
where
$\boldsymbol{n} = (\cos \theta ,\sin \theta )$
and the shape factor for spheroids takes the simple form
$\varLambda =(( {\lambda ^2-1})/({\lambda ^2+1}))$
, with
$\lambda$
the spheroid aspect ratio.
We now aim at finding the optimal protocol for the control
$\lambda$
, or equivalently for the shape factor
$\varLambda$
, that maximises its distance from the initial position
$\boldsymbol{x}(t_0)$
over a fixed episode duration
$T$
. This is formulated by minimising the cost functional (3.1) and identifying the state vector with
$\boldsymbol{q}=(\boldsymbol{x},\theta )$
, where
$\boldsymbol{x}$
denotes the swimmer position and
$\theta$
its orientation.
To apply Pontryagin’s minimum principle, we introduce the costate variables
$\boldsymbol{\pi }_x$
and
$\pi _\theta$
conjugate to
$\boldsymbol{x}$
and
$\theta$
, respectively. The system Hamiltonian reads (Bryson & Ho Reference Bryson and Ho2018)
\begin{align} \mathcal{H} = \boldsymbol{\pi }_x \boldsymbol{\cdot }\left ( \boldsymbol{u} + v_{\textit{s}} \boldsymbol{n} \right ) + \pi _\theta \left ( \frac {1}{2}\omega + \varLambda S_{np}\right ) \, . \end{align}
The necessary condition for optimality is
$\partial _\lambda \mathcal{H}=0$
, which yields
\begin{align} \frac {\partial \mathcal{H}}{\partial \lambda } = \pi _\theta S_{np}\frac {\textrm {d}\varLambda }{\textrm {d}\lambda }= \pi _\theta S_{np} \frac {4\lambda }{(\lambda ^2+1)^2} = 0 \hspace {0.2cm}\Longrightarrow \hspace {0.2cm}\lambda =0 \hspace {0.2cm}\Longleftrightarrow \hspace {0.2cm} \varLambda =-1 \, . \end{align}
The optimal escape strategy, therefore, consists of maintaining a constant disk-like shape. In this limit, the flow-induced torque in Jeffery’s equation has the same torque structure arising in Zermelo’s minimum-time navigation problem (see discussion in § 3 and Piro et al. (Reference Piro, Vilfan, Golestanian and Mahault2024)).
x(t0)
Λ=−0.98
Λ=0.98
Λ>0
Λ<0
Λ=0.98
R^
Ku
R^
103
103
Ku=0.1
Ku=1
Ku=10
Ku=107
Ku=0.1
Λ=0.98
Λ
nθSnp
Λ
nθ
Snp
103
Λ
Ku=107
nr
Snp
ω
Snp
ω
nr≥0.8
Snp
ω
nr≤−0.8
4×104
Λ
Snp
nθ
Snp
Ω
Ku=0.1
Ku=1
Ku=10
Ku=107
103
Ku=10
Ku
Ku
ω
Sxx
Sxy
Ku
Λ
Snp
nθ
Snp
ω
R^
Ku
Ku=10
Dr=0
R^
T
Tu
vs
Ku=10
T/τK=200
Tu/τK=0.1
v s/urms=0.707
Ku=10
R^
Λ∈[−0.98,0.98]
[−0.8,0.8]
[−0.6,0.6]
[−0.5,0.5]
[0,0.98]
