1. Introduction
1.1. Motivation
At present, quadruped robots have achieved the capability to adapt to complex outdoor terrains and accomplish diverse locomotion control tasks, providing strong technical support for their further real-world applications [Reference Margolis and Agrawal1–Reference Liu, Xu, Wang, Tang and Lu4]. Among the various application scenarios of quadruped robots, skill performance represents a commercially valuable direction. However, existing demonstrations are mostly limited to the presentation of a single motion skill or to multiple similar skills, such as different gaits, resulting in performances lacking diversity. In contrast, heterogeneous multi-skill motions require the realization of continuous, diverse, and long-horizon behaviors, involving a broader state space and more complex processes for motion generation and transition. These characteristics pose significant challenges to existing methods.
In addition, expert data containing multiple motion skills and complete motion characteristics are crucial for multi-skill imitation learning (IL) in quadruped robots. Incomplete expert data and insufficient motion details make it difficult for existing approaches to effectively learn and model the transitions between skills. As a result, skill transitions often lack continuity and responsiveness, making it difficult to produce visually appealing multi-skill performances. However, high-quality expert data that contain multiple skills are typically difficult to obtain. Generating long-horizon trajectories with multiple skills through dynamic modeling and trajectory optimization is often tedious, time-consuming, and extremely challenging, significantly increasing the data preparation cost in the early stage. Alternatively, combining trajectories of different skills lacks natural transition processes and requires carefully designed motion transitions. A more practical and cost-effective approach is to collect data by manually controlling robot animation in simulation or real environments. However, the resulting data often lack dynamical information, making them unsuitable for IL. Another approach is to obtain motion data of animals or humans through motion capture or video extraction and then map them to quadruped robots. Nevertheless, this approach suffers from heterogeneous mapping problem between animals and robots and also makes it difficult to design artistic motion skills.
Due to the challenges of long-horizon motion planning or delicate motion transition design, incomplete data, and sparse rewards, multi-skill motion generation remains a highly challenging problem in quadruped robot locomotion control. Methods that combine IL with reinforcement learning (RL) have demonstrated significant advantages, as they allow the policy to imitate reference motions while exploring the limits of control capabilities, thereby achieving more robust motion control. Therefore, this paper proposes a motion skill generation and continuous transition method based on imitation–reinforcement learning (IRL). Multiple independent policies are trained as motion controllers to enable long-horizon multi-skill motion in quadruped robots. Compared to a single unified policy, adopting multiple independent policies with policy switching offers significant advantages: each policy can focus on learning a single skill, thereby improving motion fidelity and stability; it avoids mode collapse and catastrophic forgetting caused by multi-task coupling; it reduces the complexity of policy learning; and it enables more flexible composition among dissimilar skills. Specifically, we learn motion skills by imitating reference motions to avoid complex reward function design; we combine motion trajectories of multiple skills to eliminate the need for long-horizon trajectory planning and optimization; we employ RL to improve the continuity and robustness of skill transitions; and we adopt a two-stage reference state initialization (RSI) curriculum to address the learning difficulties caused by incomplete expert data.
1.2. Contributions
We propose an IRL framework to enable continuous multi-skill motions for a quadruped robot equipped with a 6-DoF manipulator. A variety of skills – including rhythmic trotting, dynamic jumping, double-leg standing, torso swaying, single-leg swinging, double-leg swinging, and leg–arm dancing – are generated and seamlessly transitioned in both simulation and real-world environments, demonstrating robust and coordinated whole-body motion control capabilities. The main contributions of this work are summarized as follows:
-
• We propose three methods for skill trajectory generation based on central pattern generators (CPG), animation design, and motion capture, as well as a trajectory retargeting and optimization method for mapping quadruped animal motions to quadruped robots;
-
• We develop an IRL algorithm for multi-skill motion generation in quadruped robots, which learns diverse motion skills through imitation of reference motions, achieves continuous multi-skill transitions via trajectory composition and RL, and incorporates a self-recovery mechanism during training to enhance robustness;
-
• We design a two-stage curriculum based on RSI to enable continuous skill transitions from arbitrary states and propose an asymmetric actor–critic neural network architecture that allows the robot to generate similar and robust motions without requiring reference motion inputs during deployment.
2. Related work
2.1. Skill control methods based on dynamic models
Traditional control methods, represented by model predictive control and whole-body control, generally follow a modeling–planning–control pipeline, in which motion control is achieved by optimizing and tracking trajectories based on dynamic models [Reference Li, Zhou and Cheng5–Reference Cai, Cai, Xie, Huang and Li8]. For generating motion skills in quadruped robots, the typical approach first specifies an initial state, then performs offline optimization of skill trajectories based on simplified or full-body dynamic models, and finally tracks these trajectories online in real time. This enables the execution of individual motion skills such as jumping, backflips, and handstands [Reference Chen, Rong and Li9–Reference Ding, Pandala, Li, Shin and Park16]. However, such methods are limited by the initial conditions assumed during optimization, making it difficult to achieve continuous multi-skill motions.
To enable continuous long-horizon multi-skill motions, one approach is to concatenate offline trajectories of multiple skills and introduce transition phases and intermediate gaits to ensure smooth switching. By tracking these trajectories online, robots can perform sequences such as stepping up and down stairs, lying down, turning, and crawling under obstacles [Reference Bjelonic, Grandia, Geilinger, Harley, Medeiros, Pajovic, Jelavic, Coros and Hutter17]. However, this approach is generally limited to slow and low-dynamic motions and is not suitable for highly dynamic behaviors such as jumping. To address this limitation, online trajectory optimization methods have been proposed, which generate motion trajectories in real time based on the robot’s current state, enabling continuous execution of dynamic motions such as jumping, backflips, and running jumps [Reference Chignoli and Kim18–Reference Liang, Peng, Zhao and Yan24]. By predefining contact sequences, these methods can also achieve multi-gait locomotion [Reference Youm, Jung, Kim, Hwangbo, Park and Ha25–Reference Xu, Zhu, Zhang and Ho27], and by switching between wheeled and legged modes, multimodal locomotion can be realized [Reference Zhou, Yang, Jiang, Zhang, Chi, Chen, Zhang, Li, Zhang and Wang28]. Nevertheless, such approaches are typically limited to single or similar types of motion skills and cannot handle heterogeneous multi-skill motions.
In summary, traditional control methods rely on accurate motion planning and control to achieve individual motion skills, multi-gait locomotion, and similar multi-skill behaviors. Their performance depends heavily on model accuracy, trajectory quality, and real-time control capability. In real-world scenarios, model mismatches and invalid assumptions often lead to system failures, and these methods generally exhibit limited generalization and robustness when facing changes in tasks or environments. Therefore, multi-skill motion generation remains a highly challenging problem.
2.2. Skill learning methods based on neural networks
In recent years, intelligent learning methods such as RL and IL have been widely applied to generate robust motion skills for quadruped robots, leveraging the strong nonlinear approximation capabilities of neural networks [Reference Vogel, Baines, Church, Lotzer, Werner and Hutter29–Reference Rudin, He, Aurand and Hutter33]. End-to-end neural network policy controllers trained purely with RL do not require trajectory planning, optimization, or tracking and have enabled quadruped robots to perform tasks such as terrain adaptation, obstacle avoidance, and parkour [Reference Rudin, Hoeller, Bjelonic and Hutter34–Reference Zhuang, Fu, Wang, Atkeson, Schwertfeger, Finn and Zhao38]. However, learning complex motion skills with RL often suffers from sparse rewards or requires carefully designed reward functions and extensive parameter tuning. Poorly designed reward functions can prevent the agent from making meaningful learning progress for extended periods. IL offers an alternative approach by learning control policies directly from expert datasets, enabling quadruped robots to acquire motion skills with biomimetic or artistic characteristics [Reference Li, Blaes, Kolev, Vlastelica, Frey and Martius39–Reference Chane-Sane, Roux, Stasse and Mansard41]. Although IL avoids complex reward design, its performance heavily depends on the quality and completeness of expert data, and it generally lacks sufficient exploration and generalization capabilities.
Another line of research combines RL with IL to mitigate the need for complex reward design while improving robustness. For example, reference motion trajectories can be generated using animation tools, and imitation-based reward terms are incorporated into RL frameworks to enable the learning of various skills such as jumping, backflips, acrobatics, and martial arts [Reference Peng, Abbeel, Levine and Panne42–Reference Grandia, Knoop, Hopkins, Wiedebach, Bishop, Pickles, Müller and Bächer44]. In addition, the well-known adversarial motion priors approach imitates motion styles rather than strictly tracking fixed trajectories, resulting in more robust motion skills [Reference Peng, Ma, Abbeel, Levine and Kanazawa45–Reference Li, Vlastelica, Blaes, Frey, Grimminger and Martius47]. However, when expert datasets contain skills with varying levels of complexity, AMP often struggles to learn a unified policy that captures all skills. It may learn only a subset of skills or suffer from catastrophic forgetting during long training, tending to favor simpler or blended behaviors that may not meet practical requirements. To address these limitations, one potential approach is to manually separate different types of skills within the expert dataset and train independent networks for each, although this requires significant manual annotation and computational resources. In addition, some studies combine the strengths of traditional control methods and learning-based approaches to achieve complex motion skills for quadruped robots [Reference Jenelten, He, Farshidian and Hutter48–Reference Shao, Jin, Liu, He, Wang and Yang52].
2.3. Continuous multi-skill motion and transitions
Although existing methods are capable of learning and generating complex motion skills, they often struggle to achieve continuous switching between multiple skills and lack smooth transition capabilities. Tu et al. [ Reference Tu, Wei, Zhang, Hou, Gao, Dong, Zhai and Zhang53,Reference Tu, Zhai, Zhang, Wei, Dong and Zhang54] proposed a progressive adversarial self-imitation skill-switching method, which leverages high-quality historical trajectories to autonomously learn both skills and their transitions, enabling natural switching between walking and crawling, as well as between walking and bipedal locomotion in quadruped robots. Yu et al. [Reference Yu, Acero, Atanassov, Yang, Havoutis, Kanoulas and Li55] employed a high-level policy to generate weights for different gait policies, which are then combined to form multi-skill motions, achieving multiple gaits and smooth transitions on the Unitree A1 quadruped robot. Yang et al. [Reference Yang, Pu, Zou, Wei, Wang and Li56] designed a network architecture that integrates CPG with multilayer perceptrons, and jointly trained the model using gradient-based learning, enabling strong multi-skill capabilities, including fall recovery and various quadrupedal gaits. However, this approach can only switch motions between gaits that are close in the action space. Watanabe et al. [Reference Watanabe, Li and Hutter57] trained a neural network via RL to encode reference motions into structured latent variables, enabling smooth transitions among multiple dance motions. However, their method requires interpolation of the latent variables during deployment to ensure smooth transitions at arbitrary time points. Moreover, strong periodicity assumptions are imposed in both motion representation and policy learning, limiting its applicability to more complex motion skills. Most approaches adopt the idea of adversarial IL, guiding the policy to spontaneously generate transitional motions between different expert datasets. However, due to the use of a single-policy multi-skill learning framework, this often leads to mode collapse and catastrophic forgetting. Consequently, although these works have made considerable progress, achieving long-horizon multi-skill motions for quadruped robots remains a challenging problem.
3. Methods
3.1. Framework overview
We construct an IRL framework, as illustrated in Figure 1, to enable the generation and transition of continuous and diverse motion skills for quadruped robots. The framework consists of three main components: skill planning, skill learning, and policy transfer. In the skill planning stage, a reference trajectory library is built, including rhythmic motions, stylistic motions, and high-dynamic motions. In the skill learning stage, an IRL algorithm is employed to train neural network policies that control the robot to imitate reference motions in a simulation environment. In the policy transfer stage, the learned policies are deployed to a more accurate dynamics simulation environment and real robot hardware, enabling the robot to execute the learned skills and achieve continuous transitions in real-world settings.
Most existing approaches tend to train a single neural network policy to generate multiple motion skills. However, for heterogeneous skills with significantly different state spaces, learning a unified policy is highly challenging. Therefore, we train independent neural network policies for each motion skill and achieve continuous transitions between skills through RL. This approach not only avoids long-horizon motion planning and training processes but also allows more flexible composition of multiple skills to realize diverse long-horizon behaviors. During training, an imitation-based reward mechanism is introduced to address the issue of sparse rewards and to maximize the similarity between generated motions and reference trajectories. In addition, a two-stage curriculum based on RSI is adopted to ensure that the initial state distribution of the agent covers the entire state space of the reference motions, thereby improving learning efficiency.
An IRL framework for multi-skill learning in quadruped robots. The system consists of three key components: skill planning, skill learning, and policy transfer. We employ RL to train neural network policies, where the designed reference motion trajectories are used as imitation rewards and observation inputs to maximize the similarity between the generated motions and the reference motions. After deployment on the real robot, arbitrary and smooth transitions between motion skills can be achieved via policy switching.
Figure 1. Long description
A diagram of the process of skill planning, skill learning, and policy transfer in quadruped robots. Panel 1: Skill Planning. Planning Methods include CPG, Animation design, and Motion capture. These methods produce expressive, rhythmic, and high-dynamic reference trajectories. Panel 2: Skill Learning. The environment interacts with the robot state, which includes robot state, object state, and external state. The actor network and value network process these states to produce actions and value estimates. The PD Controller operates at 200 Hz, and the optimizer updates the networks based on the loss. Panel 3: Policy Transfer. The policy, operating at 50 Hz, is transferred to the PD Controller, which operates at 1000 Hz. The Sim2Sim and Sim2Real processes facilitate the transfer of commands to the robot.
3.2. Multi-skill motion planning method
To enable efficient learning of motion generation by imitating the characteristics of reference trajectories, a large amount of diverse trajectory data is required. We first categorize quadruped robot motions into three types based on trajectory characteristics: rhythmic motions, expressive motions, and high-dynamic motions and propose corresponding methods for acquiring these trajectories:
-
1. Rhythmic motions: These exhibit clear periodicity and consist of simple, repetitive trajectories, such as standing and trotting. These trajectories are generated using CPG.
-
2. Expressive motions: These are designed to be visually expressive and stylistically rich, characterized by complex and diverse trajectories, such as double-leg swinging and leg–arm dancing. These trajectories are generated using animation software.
-
3. High-dynamic motions: These are characterized by explosive and instantaneous movements with highly nonlinear trajectories, such as jumping and flipping. These trajectories are captured from quadrupedal animal motions via motion capture.
Each collected skill trajectory
${x}^{\left ( i \right )}$
contains the following components: the position and linear velocity of the center of mass (CoM), the orientation and angular velocity of the base, the foot positions, the joint angles, and the joint angular velocities:
\begin{equation} x^{(i)}=\left (\,\boldsymbol{p}_t, \boldsymbol{\theta }_t, \boldsymbol{v}_t, \boldsymbol{\omega }_t, \boldsymbol{p}_t^{\mathrm{toe}}, \boldsymbol{q}_t, \dot {\boldsymbol{q}}_t\right )_{t=0}^T \end{equation}
where
$T$
is the trajectory length;
$\boldsymbol{p}_t \in \mathbb{R}^{3 \times 1}$
and
$\boldsymbol{v}_t \in \mathbb{R}^{3 \times 1}$
represent the CoM position and linear velocity;
$\boldsymbol{\theta }_t \in \mathbb{R}^{4 \times 1}$
(quaternion) and
$\boldsymbol{\omega }_t \in \mathbb{R}^{3 \times 1}$
represent the base orientation and angular velocity in the world frame;
$\boldsymbol{p}_t^{\mathrm{toe}} \in \mathbb{R}^{12 \times 1}$
is the foot positions in the body frame; and
$\boldsymbol{q}_t \in \mathbb{R}^{18 \times 1}$
,
$\dot {\boldsymbol{q}}_t \in \mathbb{R}^{18 \times 1}$
are the joint angles and velocities.
Each skill can be expanded into multiple variants with different frequency profiles through sampling and interpolation, thereby augmenting the motion data to form a multi-skill trajectory library containing
$N$
trajectories:
\begin{equation} D= \{x^{(i)}\}_{i=1}^N \end{equation}
3.2.1. CPG-based planning of rhythmic skills
Given the coupled and periodic nature of rhythmic motions – both among limbs and between the limbs and torso – we adopt a CPG model inspired by the mechanisms of rhythmic motor control in animals to construct a flexible trajectory generator for rhythmic motions. CPGs are distributed oscillatory networks composed of multiple neurons that interact to generate self-sustained oscillations. These oscillations exhibit stable phase-locked relationships among neurons. Due to the plasticity of their synaptic connections, CPGs can produce a wide variety of connection patterns and output signals, enabling the generation of diverse rhythmic behaviors [ Reference Ijspeert58,Reference Dzeladini, Ait-Bouziad and Ijspeert59]. CPGs have been widely used in robotics to produce rhythmic gaits and have proven highly effective for designing rhythmic motion trajectories.
To construct a CPG network, it is crucial to select an appropriate oscillator that can generate periodic signals. The Hopf oscillator is characterized by a stable limit cycle in the state space, ensuring that, regardless of the non-zero initial condition, the oscillator converges to a periodic signal of consistent shape [Reference Matos, Santos and Pinto60–Reference Xiao and Wang62]. We adopt the Hopf oscillator to build the CPG network. The mathematical model of the Hopf oscillator is given by:
\begin{equation} \begin{cases} \dot {x}=\alpha _x(\mu ^2-r^2)x-\omega y \\[2pt] \dot {y}=\alpha _y(\mu ^2-r^2)y+\omega x \\[4pt] \omega =\dfrac {\omega _{\mathbf{st}}}{e^{-by}+1}+\dfrac {\omega _{\mathbf{sw}}}{e^{by}+1} \\[4pt] r^2=x^2+y^2 & \end{cases} \end{equation}
here,
$x$
and
$y$
are the output signals of the oscillator.
$\alpha _{x}$
and
$\alpha _{y}$
are the convergence coefficients for
$x$
and
$y$
, respectively, controlling how quickly the oscillator converges to the limit cycle.
$\mu$
denotes the amplitude of the oscillator, and
$r$
is the radius of the limit cycle.
$\omega$
represents the angular frequency of the oscillator, while
$\omega _{st}$
and
$\omega _{sw}$
are the angular frequencies for the stance and swing phases, corresponding to the proportion of stance time and swing time in a complete gait cycle. The constant
$b$
is the time integration factor, which determines the rate at which
$\omega$
transitions between
$\omega _{st}$
and
$\omega _{sw}$
, ensuring a smooth variation of
$\mu$
across different half-planes.
A phase variable
$\phi \in [\!-\!\pi , \pi ]$
is defined to describe the contact state of a leg with the ground: when
$\phi \in [0, \pi ]$
, the foot is in the stance phase; when
$\phi \in [\!-\!\pi , 0]$
, it is in the swing phase, as illustrated in Figure 2(a). The periodic phase
$\phi$
is modeled using the output of the Hopf oscillator:
\begin{equation} \phi =\arctan 2(x,y) \end{equation}
As shown in Figure 2(b), a CPG network is constructed by coupling four Hopf oscillators to model the phase dynamics of the four legs of the quadruped robot:
\begin{equation} \left [\begin{array}{c} \dot {x}_i \\ \dot {y}_i \end{array}\right ]=\left [\begin{array}{cc} \alpha _x(\mu ^2-r^2) & -\omega \\ \omega & \alpha _y(\mu ^2-r^2) \end{array}\right ]\left [\begin{array}{l} x_i \\ y_i \end{array}\right ]+k \sum _{j \neq i} \boldsymbol{R}(\varphi _{i j})\left [\begin{array}{l} x_j \\ y_j \end{array}\right ] \end{equation}
here,
$k$
is the coupling gain, and
$\varphi _i^j=\phi _i-\phi _j$
represents the phase offset of oscillator
$i$
with respect to oscillator
$j$
, where
$i, j \in \{1,2,3,4\}$
. The coupling matrix
$\boldsymbol{R}(\varphi _{i j})$
, which controls the interactions among oscillators and determines the gait characteristics of the quadruped robot, is defined as follows:
\begin{equation} \boldsymbol{R}(\varphi _{i j})=\left [\begin{array}{c@{\quad}c} \cos (\varphi _{i j}) & -\sin (\varphi _{i j}) \\[4pt] \sin (\varphi _{i j}) & \;\;\;\;\;\cos(\varphi _{i j}) \end{array}\right ] \end{equation}
A phase parameter
$\rho$
is defined to represent the normalized phase progression of the foot during the stance or swing phase:
\begin{equation} \rho =\left \{\!\!\begin{aligned} -\frac {\phi }{\pi }, & \;\phi \in [\!-\!\pi , 0) \\ \frac {\phi }{\pi }, & \;\phi \in [0, \pi ) \end{aligned}\right . \end{equation}
Following the polynomial-based foot trajectory generation model proposed in [Reference Shao, Jin, Liu, He, Wang and Yang52], the normalized phase progression parameter
$\rho$
is used as the input variable of a polynomial to generate foot trajectories. The gait period is set to 0.5 s, and the phase relationships among the four oscillators are configured as
$[0, \pi , \pi , 0]$
. Both stance and swing phase frequencies are set to 0.5. Other parameters for the Hopf oscillator are listed in Table I. Under these settings, the phase relationships and foot trajectories corresponding to the trot gait are illustrated in Figure 3. By modifying the coupling phase offsets between the four oscillators and adjusting the stance/swing frequencies, various rhythmic gaits such as walk, pace, and bound can be generated.
Parameter settings of the Hopf oscillator.
The CPG network of a quadruped robot constructed by coupling four oscillators. (
$\mathbf{a}$
) Foot phase. (
$\mathbf{b}$
) Topology of the CPG network. LF, RF, LH, and RH denote the left front, right front, left hind, and right hind legs, respectively, with the oscillators numbered from 1 to 4.
Phase trajectories and foot trajectories in the body frame for the trot gait. Since the trajectories of diagonal legs are identical, only the trajectories of the two front legs are shown.
Figure 3. Long description
Panel A: A line graph shows phase trajectories over time. The x-axis represents time in seconds ranging from 0 to 2, and the y-axis represents phase in radians ranging from -2.5 to 2.5. Two lines, one solid blue labeled LF and one dashed orange labeled RF, depict the phase trajectories of the front legs. The lines show a periodic pattern with the LF and RF phases alternating in a sawtooth wave pattern. Panel B: A line graph shows the x position of the front legs over time. The x-axis represents time in seconds ranging from 0 to 2, and the y-axis represents the x position in meters ranging from -0.1 to 0.1. Two lines, one solid blue labeled LF and one dashed orange labeled RF, depict the x position trajectories of the front legs. The lines show a sinusoidal pattern with the LF and RF positions oscillating out of phase. Panel C: A line graph shows the z position of the front legs over time. The x-axis represents time in seconds ranging from 0 to 2, and the y-axis represents the z position in meters ranging from 0 to 0.1. Two lines, one solid blue labeled LF and one dashed orange labeled RF, depict the z position trajectories of the front legs. The lines show a periodic pattern with the LF and RF positions peaking at different times.
3.2.2. Animation-based planning of expressive skills
For expressive motions that require strong visual appeal, we leverage the intuitive 3D animation software Blender and the physics simulator Webots for trajectory design. Blender offers powerful animation authoring and rendering capabilities, allowing for fine-grained motion design and preview. Webots enables manual specification of robot poses and simulates the planned trajectories in a realistic physical environment. Our workflow for planning expressive motion trajectories is as follows: first, construct the robot’s 3D model and kinematic structure; next, set the key postures of the motion as animation keyframes; then, generate the full motion sequence via linear or Bézier interpolation under kinematic constraints; finally, export the resulting trajectories for direct use in RL algorithm training. The key postures for the double-leg swinging and leg-arm dancing motions are illustrated in Figure 4.
Key pose design for expressive motions. (a) Key poses of the double-leg swinging motion are planned using Blender. From left to right: initial standing pose, intermediate squatting pose, and final upright pose. (b) Key poses of the leg-arm dancing motion are planned using Webots. From left to right: initial pose, intermediate pose, and final pose.
3.2.3. MoCap-based planning of high-dynamic skills
Due to the complexity of high-dynamic motions, it is difficult to obtain corresponding trajectories using analytical trajectory equations or animation design methods. Given the structural similarity between quadruped animals and quadruped robots – and considering that animal motions are often elegant, natural, and fluid – they provide an ideal dataset for IL in quadruped robots. Therefore, we employ motion capture systems to record high-dynamic motion trajectories from quadruped animals and map them onto the robot. Specifically, we attach 28 optical markers to a highly agile Belgian Malinois to track its trajectory during jumping motions, as shown in Figure 5.
Motion trajectory data collection of a Belgian Malinois during jumping. (a) Snapshots of the jumping process. (b) Planar rigid body structure with marker positions. (c) 3D stick figure with optical markers.
Since the morphology of the quadruped robot is not identical to that of the Belgian Malinois, we extract trajectory data from key anatomical landmarks – namely, the mid-spine, shoulders, hips, hands, and feet – to accurately imitate the dog’s motions. Due to differences in body dimensions, the collected data must be mapped onto the robot. This mapping process consists of two steps: (i) Data retargeting, which rescales the motion trajectories to match the robot’s dimensions; (ii) data optimization, which adjusts the trajectories to satisfy the kinematic constraints of the robot.
Data retargeting: Based on the size ratio between the quadruped robot and the dog, we rescale and retarget the trajectories of the CoM, hip positions, and foot positions:
\begin{equation} \left \{\begin{array}{l} \boldsymbol{p}^{\prime }=\xi \boldsymbol{p}_{\text{origin}} \\[3pt] \hat {p}_{\text{hip}, y}=p_{\mathrm{CoM}, y}^{\prime }+0.5 k w_{\text{hip}} \\[3pt] \hat {p}_{\text{foot}, y}=p_{\mathrm{CoM}, y}^{\prime }+0.5 k w_{\text{foot}} \end{array}\right . \end{equation}
where
$\boldsymbol{p}_{\text{origin}}$
is the original data,
$\boldsymbol{p}^{\prime }$
is the rescaled data, and
$\xi =2 / 3$
is the scaling factor. Given that the robot’s torso is wider than the dog’s, we further retarget the hip and foot positions in the lateral (
$y$
) direction using the rescaled CoM position. Specifically, let
$\hat {p}_{\text{hip}, y}$
and
$\hat {p}_{\text{foot }, y}$
denote the y-axis positions of the retargeted hips and feet, respectively. Let
$w_{\text{hip}}$
and
$w_{\text{foot}}$
represent the width between the robot’s left and right hips and between its left and right feet. The leg selection indicator
$k$
is set to
$k=1$
for the left legs and
$k=-1$
for the right legs.
Data optimization: The retargeted data are used as reference trajectories, and we formulate an optimization problem to compute the CoM position, base orientation, and leg joint angles that satisfy the robot’s kinematic constraints.
We first solve for the CoM position and base orientation. The optimization objective is to minimize the discrepancy between the hip positions and the retargeted reference data, while ensuring full-body kinematic feasibility. The optimization problem is formulated as follows:
\begin{equation} \begin{aligned} \underset {\boldsymbol{x}}{\arg \min } \quad & \sum _{i=0}^n \left \|\hat {\boldsymbol{p}}_{\text{hip}, i} - p_{\text{hip}, i}(\boldsymbol{x}) \right \|_{\boldsymbol{Q}_x}^2 \\ \text{s.t.} \quad & p_{\text{hip}, i}(\boldsymbol{x}) = \boldsymbol{p}_{\text{CoM}} + \boldsymbol{R}_{\text{ZYX}} \boldsymbol{T}_{\text{hip}, i} \\ & \underline {\boldsymbol{x}} \leq \boldsymbol{x} \leq \overline {\boldsymbol{x}} \end{aligned} \end{equation}
where the optimization variable
$\boldsymbol{x} \in \mathbb{R}^{6 \times 1}$
includes the CoM position
$\boldsymbol{p}_{\text{CoM}} \in \mathbb{R}^{3 \times 1}$
and base orientation angles
$\boldsymbol{\Theta } \in \mathbb{R}^{3 \times 1}$
;
$\hat {\boldsymbol{p}}_{\text{hip}, i}$
represents the reference position of the
$i$
-th hip;
$p_{\text{hip}}(\boldsymbol{x})$
is the function that computes the hip position from the optimization variable;
$\boldsymbol{R}_{\text{ZYX}}$
denotes the Z-Y-X Euler angle transformation for the base orientation, and
$\boldsymbol{T}_{\text{hip}, i}$
is the translational offset from the CoM to the hip positions.
$n$
is the number of legs,
$\boldsymbol{Q}_x$
is the weighting matrix for the tracking error, and
$\underline {\boldsymbol{x}}, \overline {\boldsymbol{x}}$
are the lower and upper bounds for the optimization variables.
Then, we solve for the leg joint angles. The optimization objective is to minimize the errors between the joint angles and the foot positions with respect to the reference data, while satisfying the leg kinematic constraints and joint limits. An individual optimization problem is formulated for each leg as follows:
\begin{equation} \begin{aligned} \underset {\boldsymbol{q}_i}{\arg \min } \quad & \left \|\hat {\boldsymbol{p}}_{\text{foot}, i}-p_{\text{foot }, i}(\boldsymbol{q}_i)\right \|_{\boldsymbol{Q}_f}^2+\left \|\tilde {\boldsymbol{q}}_i-\boldsymbol{q}_i\right \|_{\boldsymbol{Q}_q}^2 \\ \text{s.t.} \quad & p_{\text{foot}, i}(\boldsymbol{q}_i)=p_{\text{hip}, i}(\boldsymbol{x})+\boldsymbol{R}_{\mathrm{ZYX}} \cdot FK(\boldsymbol{q}_i) \\ & \underline {\boldsymbol{q}}_i \leq \boldsymbol{q}_i \leq \overline {\boldsymbol{q}}_i \end{aligned} \end{equation}
where the optimization variable
$\boldsymbol{q} \in \mathbb{R}^{3 \times 1}$
denotes the three joint angles of the leg.
$\hat {\boldsymbol{p}}_{\text{foot}}$
represents the reference foot position,
$p_{\text{foot}}(\boldsymbol{q})$
is the function that computes the foot position from the optimization variable, and
$FK(\boldsymbol{q}_i)$
represents the forward kinematics function of the leg.
$\boldsymbol{Q}_x$
is the weight matrix for the foot position error.
$\tilde {\boldsymbol{q}}$
denotes the nominal joint angles of the leg when the robot is in a standing posture, and
$\boldsymbol{Q}_q$
is the weight matrix for the joint angle error, which prevents the optimizer from producing unreasonable or singular solutions.
$\underline {\boldsymbol{q}}$
and
$\overline {\boldsymbol{q}}$
define the lower and upper bounds of the feasible joint angles range for the actual robot.
3.3. Multi-skill motion learning method
3.3.1. Policy learning objective
After completing skill planning, we employ the RL algorithm to train neural network policies to imitate the reference motions. The motion imitation problem is formalized as a goal-conditioned RL task, where the policy is trained to mimic the reference motions in dataset
$D$
. The policy is defined as
$\pi _\theta (a \mid s, g)$
, representing a probability distribution over actions
$a \in \boldsymbol{A}$
given the current state
$s \in \boldsymbol{S}$
and task goal
$g \in \boldsymbol{G}$
, with
$\theta$
denoting the parameters of the neural network. We train two rhythmic policies for balancing locomotion, multiple expressive policies for skill performances, and one high-dynamic policy for dynamic jumping to control the quadruped robot. Each policy is used to imitate a single reference motion, so the training of each policy is treated as an independent RL problem.
During the skill learning phase, the neural network parameters
$\theta$
are first randomly initialized and then updated using the collected states, actions, and rewards. At each time step
$t$
, the agent interacts with the environment based on the policy: given the current environment state
$s_{t}$
and task goal
$g$
, an action
$a_{t}\sim \pi _{\theta }(a_{t}|s_{t},g)$
is sampled from the policy. After executing the action, the environment transitions to a new state
$s_{t+1}$
and returns a scalar reward:
\begin{equation} r_t=r(s_t,a_t,s_{t+1},g) \end{equation}
This reward reflects the quality of the state transition and is used to compute the loss for policy optimization. The specific definition of the reward function is provided in Table III. The action
$a_t \in \mathbb{R}^{18 \times 1}$
represents the offset relative to the joint’s equilibrium angle and is fed into a proportional-derivative (PD) controller to compute the joint torques that drive the robot. The agent’s objective is to learn the optimal neural network parameters
$\theta$
to maximize the expected discounted cumulative reward:
\begin{equation} J\left (\pi _\theta \right )=\mathbb{E}_{\tau \sim p(\tau |\pi ,g)}\left [\sum _{t=0}^{T-1}\gamma ^tr_t\right ] \end{equation}
where
$\tau$
denotes a trajectory, consisting of a sequence of states, actions, and rewards:
$\tau =\left \{\left (s_t,a_t,r_t\right )_{t=0}^{T-1},s_T\right \}$
, and
$T$
is the time horizon of the trajectory.
$p\left (\tau |\pi ,g\right )$
denotes the probability of the trajectory under the given policy and goal, where
$p(\tau \mid \pi ,g)=p\left (s_{0}\right )\Pi _{t=0}^{T}p(s_{t+1}\mid s_{t}, a_{t})\pi (a_{t}\mid s_{t},g)$
.
$p\left (s_{0}\right )$
is the initial state distribution, and
$p(s_{t+1}\mid s_{t},a_{t})$
denotes the environment dynamics.
$\gamma \in \begin{bmatrix} 0,1 \end{bmatrix}$
is the discount factor.
3.3.2. Design of asymmetric neural network architecture
Robot motion is represented as temporal trajectories in a high-dimensional state space, consisting of long sequences with multiple time steps and high-dimensional data. If reference motions are provided as inputs to the policy network, it significantly increases network complexity, leading to inefficient learning. Moreover, the policy network tends to “overfit” to the specific shape of the reference trajectory, attempting to imitate every frame of the reference motion. This results in poor generalization, making it difficult for the policy to learn smooth and natural motion transitions. Additionally, any external disturbance that causes deviation from the planned trajectory may cause the policy to fail entirely. Therefore, no reference motion information is used as input to the policy network. Instead, the tracking errors relative to the reference trajectory are fed into the critic network to evaluate how well the policy imitates the reference motion. In addition, robot control in the real world is essentially a partially observable Markov decision process, where certain state information that is easily accessible in simulation is unobservable in the real world. However, this missing information in reality can significantly facilitate policy training in simulation. Although most of the missing yet critical information can be recovered in the real world using state estimation algorithms, the estimation errors can increase the Simulation-to-Real (Sim2Real) transfer gap, making policy deployment on physical robots challenging.
To address this, we design an asymmetric Actor-Critic neural network architecture: the Actor network (policy network) relies only on accurately perceived proprioceptive information, while the Critic network has access to privileged information, such as torso linear velocity and trajectory tracking errors. Both the Actor and Critic networks are implemented as multi-layer perceptrons with ELU activation functions, with the detailed network structure shown in Figure 1. This asymmetric design allows the policy network to generate similar motion skills during deployment without any input of reference motion.
The state vector input to each Actor network is:
\begin{equation} \boldsymbol{s}_t=(\boldsymbol{g}_t, \boldsymbol \omega _t, \Delta \boldsymbol{q}_t, \dot {\boldsymbol{q}}_t, \boldsymbol{a}_{t-1}) \end{equation}
here,
$\boldsymbol{g}_t$
denotes the projected gravity vector onto the torso frame, i.e., the projection of the gravity vector in the world frame onto the torso coordinate frame.
$\Delta \boldsymbol{q}_t$
represents the deviation between the current joint angles and their corresponding equilibrium angles.
$\boldsymbol{a}_{t-1}$
denotes the action output by the policy network at the previous time step.
Inputs and noise levels of the actor network.
Note: The noise follows a uniform distribution
$p \sim \mathcal{U}(\!-\xi , \xi )$
, where
$\xi$
denotes the noise level. This noise is added to the original observation data.
The state vector input to each Critic network is
\begin{equation} \boldsymbol{o}_t= (\boldsymbol{v}_t,\boldsymbol{\theta }_t,\boldsymbol{\omega }_t,\Delta \boldsymbol{q}_t,\dot {\boldsymbol{q}}_t,\boldsymbol{a}_{t-1},\Delta \boldsymbol{x}_t) \end{equation}
where
$\Delta \boldsymbol{x}_t$
represents the deviation between the current robot state and the reference state.
In addition, to bridge the gap between simulation and the real world, realistic random noise is added to the proprioceptive inputs of the Actor network. The observation inputs and noise levels are listed in Table II.
3.3.3. Design of imitation–smoothness reward function
The designed reward function encourages both accurate imitation of the specified reference motion and smoothness of the generated motion. Therefore, the reward function consists of two components: the skill imitation reward and the skill smoothness reward, formulated as follows:
\begin{equation} r_t=r_t^{\text{imitation}}+r_t^{\text{smoothness}} \end{equation}
The detailed definitions of the reward function and weights of rhythmic trotting are listed in Table III. In the formulas, variables with a hat (
$^\wedge$
) denote reference state quantities. The variables are defined as follows:
$\boldsymbol{\tau }$
denotes the vector of all joint torques;
$\boldsymbol{f}_j$
represents the foot contact force vector of the
$j$
-th leg at time
$t$
;
$\,f_{max}$
is the maximum allowable contact force;
$F_{\text{clip}}$
limits the maximum penalty value to prevent the total reward from becoming negative and overwhelming the incentive provided by imitation;
$\bar {q}_i$
and
$\underline {q}_i$
indicate the lower and upper joint limits for the
$i$
-th joint;
$n_c$
counts the number of collisions, including self-collisions and collisions between the calf, thigh, torso, and manipulator with the ground. However, during the training of the double-leg swinging tasks, collisions involving the calf and thigh with the ground are ignored. The weights in Table III are determined through an experience-driven iterative tuning process: initial values are first set, and then each weight is gradually adjusted through multiple rounds of training experiments. For different skills, we adopt a unified reward function structure and share most of the weight parameters, with only a few skill-specific terms being fine-tuned according to their distinct characteristics.
Definition of reward function.
The skill imitation reward is designed to ensure that the policy accurately imitates the reference motion. It is formulated as a measure of discrepancy between the current state and the target state, defined using a Gaussian kernel function:
\begin{equation} r_{t}^{\mathrm{imitation}}=e^{-\lambda \|\hat {\boldsymbol{x}}_{t}-\boldsymbol{x}_{t}\|^{2}} \end{equation}
where
$\lambda$
is a hyperparameter of the Gaussian kernel, and
$\hat {\boldsymbol{x}}_{t}$
and
$\boldsymbol{x}_{t}$
represent the reference and actual motion states at time
$t$
, respectively.
The skill smoothness reward is primarily designed to encourage the policy to generate smooth action trajectories, ensuring continuous and seamless transitions between motions while reducing mechanical jitter and energy consumption. We design smoothness rewards at both the policy output level and the physical manifestation level. At the policy output level, we introduce an action smoothness reward to ensure the smoothness of the action trajectories produced by the policy. The first-order difference term penalizes sudden changes in actions, the second-order difference term penalizes abrupt changes in the rate of action variation, and the
$L_1$
-norm term penalizes excessively large actions. At the physical manifestation level, we design a set of rewards, including joint torque reward, joint torque rate reward, joint acceleration reward, foot contact force reward, joint angle limit reward, and collision reward. Together, these rewards promote compliant and smooth motions in the actual robot execution. The joint torque reward penalizes the use of large torques to reduce energy consumption. The joint torque rate reward penalizes rapid changes in torque to prevent instability or jitter. The joint acceleration reward limits the magnitude of joint accelerations to avoid sudden jolts. The contact force reward encourages compliant foot-ground contact. The joint angle limit reward penalizes movements that approach or exceed the mechanical limits of the joints. The collision reward penalizes both self-collisions among robot parts and collisions with the ground.
To improve the efficiency of policy learning, different reward mechanisms are designed for each skill based on its characteristics. During rhythmic skill learning, the imitation reward only tracks joint angles and torso pose, ensuring that the generated gait achieves the desired posture control while producing rhythmic leg movements. During expressive skill learning, the imitation rewards of different body parts are coupled to encourage synchronized and coordinated limb movements, making the generated motions more visually appealing. For example, when training the double-leg swinging skill, the imitation reward for the forelegs is coupled with the torso pose, encouraging the robot to reach a stable posture before swinging its forelegs. During high-dynamic skill learning, the imitation reward tracks only the robot’s terminal states, without strictly enforcing a specific motion trajectory, which enhances the robustness of the generated motions.
Additionally, to achieve the double-leg standing capability, a specialized reward function is designed:
\begin{equation} r_{\mathrm{st}}=\left (1-C_{\mathrm{front}}\right )\mathrm{exp}\left (-\left (\sum _{i=1}^3|\hat {\theta }_i-\theta _i|\right )\left (\sum _{i=1}^6|\hat {q}_i^{\mathrm{front}}-q_i^{\mathrm{front}}|\right )\right ) \end{equation}
where
$C_{\mathrm{front}}\in \{0,1\}$
indicates whether the forelegs are in contact with the ground,
$\hat {\theta }_i$
and
$\theta _i$
represent the reference and actual torso Euler angles, respectively, and
$\hat {q}_i^{\mathrm{front}}$
and
$q_i^{\mathrm{front}}$
denote the reference and actual foreleg joint angles, respectively.
PPO algorithm hyperparameters.
3.3.4. PPO-based policy training method
All neural network policies are trained using the proximal policy optimization algorithm, with the hyperparameter settings listed in Table IV. The simulation training platform is Isaac Gym, where the policy network outputs actions at 50 Hz, while the dynamics simulation runs at 200 Hz.
Each training episode is terminated at the duration of the reference motion, but will end early if the onboard manipulator or torso comes into contact with the ground. For the four-leg standing task, we expect the robot to learn to recover from arbitrary states to a standing posture; therefore, early termination due to collisions is disabled during its training. In addition, to facilitate Sim2Real transfer, domain randomization is employed during training to improve the generalization capability of the policy. By introducing random perturbations and parameter variations to both the robot and the environment, the agent learns to perform target motions robustly under real-world physical conditions. The randomized variables and their uniform sampling ranges are listed in Table V.
3.3.5. Multi-skill transition method
The initial state distribution determines the state from which the agent starts at the beginning of each training episode. If the agent always starts from a fixed initial state during training (e.g., always from the beginning of the reference motion), the learned policy is typically limited to initiating motion only from that specific state. Moreover, since the multi-skill trajectory library contains trajectories for individual skills but lacks transition data between them, simple IL methods often fail to achieve continuous motion transitions. This issue is particularly pronounced for heterogeneous skills with significantly different state spaces, where directly switching between policies can result in discontinuous actions.
Domain randomization parameters and their uniform sampling ranges.
Two-stage RSI training curriculum.
To enable continuous transitions between different motion skills and allow the robot to start from arbitrary states, we adopt an RSI method and design a two-stage training curriculum, as illustrated in Figure 6. In the first stage, at the beginning of each training episode for a given skill, an initial state is randomly sampled from the corresponding reference trajectory of that skill. In the second stage, the initial state is randomly sampled from all reference trajectories. Through this process, the initial state distribution for each skill is gradually expanded to cover the entire state space of all reference motions. This enables the robot to start from arbitrary states during execution and achieve continuous transitions across multiple motion skills. In addition, the RSI method allows the agent to learn different phases of the reference motions in parallel, thereby improving the efficiency of policy learning [Reference Peng, Abbeel, Levine and Panne42].
3.4. Policy transfer and real-world deployment
The target platform is the quadruped robot Panda7-Arm equipped with a 6-DoF manipulator, as shown in Figure 7. The manipulator is mounted at the center of the robot’s back, resulting in a total of 18 DoFs (12 for the legs and 6 for the manipulator). The prototype parameters are listed in Table VI. The actions output by the policy network represents offsets relative to the joint’s equilibrium angles, which are then passed through a PD controller to generate torque commands for each joint:
\begin{equation} \boldsymbol \tau =\boldsymbol{k}_{\mathrm{p}}(\boldsymbol{a}+\boldsymbol{q}_0-\boldsymbol{q})-\boldsymbol{k}_{\mathrm{d}}\dot {\boldsymbol{q}} \end{equation}
where
$\boldsymbol{q}$
and
$\boldsymbol{q}_0$
represent the current and equilibrium joint angles, respectively, and
$\boldsymbol{k}_{\mathrm{p}}$
and
$\boldsymbol{k}_{\mathrm{d}}$
denote the proportional and derivative gains.
Finally, the trained policy networks are exported in TorchScript format and evaluated using both Simulation-to-Simulation (Sim2Sim) and Sim2Real transfer methods. The policies are first deployed in a higher-fidelity RaiSim [Reference Hwangbo, Lee and Hutter63] physics simulation environment for validation and parameter tuning and then transferred to the real robot prototype. The policies run at 50 Hz on the robot’s onboard AI supercomputer (NVIDIA Jetson AGX Orin), while the low-level joint torque control loop runs at 1,000 Hz. A remote controller sends control commands to the robot via UDP at 200 Hz to enable switching between different motion skills.
The quadruped robot prototype Panda7-Arm equipped with a 6-DoF manipulator.
4. Results
To evaluate the proposed multi-skill learning method, we first verify its effectiveness in generating diverse motion skills, then demonstrate its capability for seamless transitions among multiple skills, and finally assess its performance under external disturbances. The experimental results show that the policies trained in simulation can be successfully transferred to real hardware. The quadruped robot achieves a variety of motion skills, including rhythmic trotting, dynamic jumping, double-leg standing, torso swaying, single-leg swinging, double-leg swinging, and leg–arm dancing. Furthermore, continuous multi-skill behaviors can be realized through seamless skill transitions, and the system exhibits strong robustness against unknown disturbances.
Structural parameters of the Panda7-Arm prototype.
The policy generates multiple motion skills in the Sim2Sim experiments. Subfigures (a–f) correspond to rhythmic trotting, dynamic jumping, torso swaying, single-leg swinging, double-leg swinging, and leg-arm dancing, respectively.
4.1. Single-skill motion generation
Figures 8 and 9 present snapshots of multiple motion skills achieved by the policy in both Sim2Sim and Sim2Real scenarios. The corresponding motion cycle durations are summarized in Table VII. Our method not only enables versatile legged locomotion for quadruped robots but also facilitates more complex coordinated leg-arm motions when equipped with a manipulator. The maximum frequency error between the generated motions and the reference motions is 12.5%, demonstrating good imitation performance.
Figure 10 illustrates the variation ranges of the torso Euler angles during motion. Except for the double-leg swinging motion, the pitch angle remains within the range of
$[\!-\!0.06,0.08]$
rad and the roll angle within
$[\!-\!0.1,0.04]$
rad, indicating stable torso posture control. In the double-leg swinging mode, an upright torso is required, leading to larger motion amplitudes: the pitch angle ranges from
$[\!-\!0.67,0.01]$
rad and the roll angle from
$[\!-\!1.55,-1.46]$
rad. After transitioning from the initial state to steady motion, each skill forms a stable periodic loop in the phase plot, indicating that the Euler angles evolve cyclically within a stable range.
Periods of the generated motion skill in Sim2Sim and Sim2Real experiments.
Note: The error is the maximum error relative to the reference motion.
The policy generates multiple motion skills in the Sim2Real experiments. Subfigures (a–e) correspond to rhythmic trotting, dynamic jumping, torso swaying, single-leg swinging, and double-leg swinging, respectively.
Phase plots of the torso’s pitch and roll angles. Subfigures (a–f) correspond to rhythmic trotting, dynamic jumping, torso swaying, single-leg swinging, double-leg swinging, and leg-arm dancing, respectively.
Figure 10. Long description
Six line graphs depict phase plots of torso pitch and roll angles for different motion skills in quadruped robots. Panel A: Rhythmic Trotting. The line graph shows roll in radians on the vertical axis and pitch in radians on the horizontal axis. The plot displays a series of loops and curves indicating the rhythmic motion. Panel B: Dynamic Jumping. The line graph shows roll in radians on the vertical axis and pitch in radians on the horizontal axis. The plot features more pronounced loops and curves, reflecting the dynamic nature of jumping. Panel C: Torso Swaying. The line graph shows roll in radians on the vertical axis and pitch in radians on the horizontal axis. The plot exhibits smoother, less pronounced curves, indicating swaying motion. Panel D: Single-leg Swinging. The line graph shows roll in radians on the vertical axis and pitch in radians on the horizontal axis. The plot displays distinct loops and curves, representing the swinging of a single leg. Panel E: Double-leg Swinging. The line graph shows roll in radians on the vertical axis and pitch in radians on the horizontal axis. The plot features larger loops and curves, indicating the swinging of both legs. Panel F: Leg-arm Dancing. The line graph shows roll in radians on the vertical axis and pitch in radians on the horizontal axis. The plot exhibits unique patterns, reflecting the coordinated motion of legs and arms.
4.2. Multi-skill motion transitions
To demonstrate the policy’s capability for skill transitions, a long-horizon multi-skill experiment was designed by composing multiple motion skills. The experimental results show that, upon receiving a skill-switching command, the policy can respond rapidly, transitioning to the desired skill within half a motion cycle and maintaining stable execution. During the multi-skill experiments in both Sim2Sim and Sim2Real settings, the trunk Euler angles and the joint angles of the front-right leg are illustrated in Figure 11. Figure 11(a) and (b) presents the Sim2Sim results. The transition sequence is four-leg standing, leg-arm dancing, single-leg swinging, rhythmic trotting, torso swaying, dynamic jumping, double-leg standing, double-leg swinging 1, double-leg swinging 2, and four-leg standing. The robot begins the transition from leg–arm dancing to single-leg swinging at 8 s. The front leg begins to swing rhythmically at 9.5 s. This delay is not due to a slow policy response, but rather because switching to single-leg swinging requires completing a leg-lifting motion beforehand. Similar skill transitions can also be achieved on the real robot, as shown in Figure 11(c) and (d). The transition sequence is four-leg standing, rhythmic trotting, torso swaying, single-leg swinging, double-leg standing, double-leg swinging, four-leg standing, dynamic jumping, four-leg standing, torso swaying, rhythmic trotting, single-leg swinging, torso swaying, and four-leg standing.
These results demonstrate that the policy trained with the reference-state initialization curriculum can initiate motion from arbitrary time steps and achieve seamless transitions between different skills. The policy is able to execute predefined motions and compose multiple skills into continuous behaviors without any reference motion input. In contrast, the policy trained without the reference-state initialization curriculum fails to perform effective skill transitions.
Torso Euler angles and right-front leg joint angles in the multi-skill motion experiments. In the Sim2Sim experiment, motion skills are switched every 4 s, while in the Sim2Real experiment, skill transitions are triggered by remote control commands. The vertical gray dashed lines indicate the start of each transition. hr, hp, and kp denote the hip roll, hip pitch, and knee pitch DoFs, respectively. (a) Torso Euler angles in the Sim2Sim experiment. (b) Right-front leg joint angles in Sim2Sim. (c) Torso Euler angles in the Sim2Real experiment. (d) Right-front leg joint angles in Sim2Real.
Recovery capability test of the robot under push disturbances during double-leg standing. (a) Snapshots of the robot recovering to a double-leg standing posture after being pushed and falling at 8.6 s. (b) Torso Euler angle trajectories under three consecutive push disturbances; red arrows indicate the instants when external forces are applied. The robot transitions to double-leg standing at 2.5 s, and it transitions to four-leg standing at 22.5 s.
Figure 12. Long description
The image contains two elements: a series of photos and a line graph. Panel A: A series of photos showing a robot recovering to a double-leg standing posture after being pushed and falling at 8.6 seconds. The robot is depicted in various stages of recovery, demonstrating its ability to regain balance. Panel B: A line graph titled Torso Euler Angle Trajectories under three consecutive push disturbances. The x-axis represents time in seconds, ranging from 0 to 30 seconds. The y-axis represents the Euler angle in radians, ranging from -1.5 to 0.5 radians. The graph includes three lines representing roll, pitch, and yaw angles, each in different colors: blue for roll, orange for pitch, and yellow for yaw. Red arrows indicate the instants when external forces are applied. The robot transitions to double-leg standing at 2.5 seconds and to four-leg standing at 22.5 seconds.
4.3. Skill robustness experiment
To evaluate the policy’s ability to resist disturbances, the most balance-sensitive skill – double-leg standing – was selected for testing. A sufficiently large push was applied from behind the robot three times consecutively to cause it to fall. Figure 12 shows snapshots of the robot during the test and the variation in trunk Euler angles. After being pushed, the robot pitches forward; the forelegs actively extend forward and push against the ground, eventually restoring the initial double-leg standing state. When subjected to the second disturbance, the first push-off is insufficient for recovery. The trunk begins to pitch forward again, triggering a second foreleg push-off. After two push-offs, the robot regains a stable double-leg standing posture, which results in two oscillations in the pitch angle.
During the recovery process, a stable standing posture may not be achieved in a single attempt. However, the policy can autonomously adjust its outputs, perform multiple corrections, and ultimately complete the task without human intervention. The results demonstrate that the proposed policy can react to unexpected failures at any time, achieve rapid recovery, and re-enter stable motion, exhibiting robust full-body control capability.
5. Conclusion and discussion
To enable continuous multi-skill motion for quadruped robots, we propose an IRL framework that generates and transitions motion skills by imitating reference trajectories. The policy is trained in simulation and successfully transferred to real hardware. The quadruped robot is able to robustly reproduce all reference motion skills and acquire the ability to seamlessly transition among multiple skills. It can not only achieve long-horizon multi-skill behaviors through policy switching but also autonomously recover from external disturbances. These results validate the feasibility of our proposed three types of motion skill trajectory generation methods and demonstrate the effectiveness of the IRL framework. By formulating the multi-skill motion problem as motion imitation and continuous transition, we avoid the challenges associated with long-horizon motion planning and sparse reward design. The proposed reference-state initialization curriculum enables effective learning of continuous skill transitions, allowing the robot to switch from arbitrary states to target motion skills through policy switching. In addition, the designed asymmetric Actor-Critic network architecture enables the policy to generate consistent motion skills without requiring any reference motion information during deployment.
This work provides a practical solution for reference trajectory acquisition, motion skill generation, and skill transition in quadruped robots. It enriches the motion capabilities of quadruped systems and promotes their applications in entertainment and performance industries. In future work, we plan to incorporate music perception into the multi-skill learning framework, where motion generation and transitions are driven by musical rhythm, enabling continuous multi-skill quadruped locomotion synchronized with music.
Author contributions
All first authors contributed equally to this work. C.P. and S.H. led the research, proposed the core concepts, implemented the framework, and wrote the manuscript. X.Z. supervised the research process and provided critical feedback at each stage. W.L. helped with the research, engaged actively in technical discussions, and helped write the manuscript. H.Z., J.W,, and T.Q. undertook the experimental work and data processing. P.X. and L.J. contributed to the study conception and design and provided critical guidance. All authors commented on previous versions and approved the final manuscript.
Data availability
Data available from the corresponding author upon reasonable request.
Financial support
This work was supported by the STI 2030—Major Projects (grant numbers 2021ZD0201403) and the Beijing Natural Science Foundation (grant numbers L243004).
Competing interests
The authors declare no conflicts of interest exist.
Ethical standards
Not applicable.
Use of AI
ChatGPT (version 5.4, OpenAI) was used solely for minor language editing and polishing to improve the readability of the manuscript. All scientific content, analysis, and conclusions were developed by the authors.
a
b
