2.1. AM dynamics

A single-arm AM is studied, as depicted in Fig. 2. Define the aircraft body-fixed frame {B} (forward, right, and downward) and the earth inertial frame {I} (north, east, and downward). ${{\boldsymbol{{p}}}}={ [{x,y,z} ]^{\text{T}}}$ and $\boldsymbol{\varPhi }={[\phi,\theta,\psi ]^{\text{T}}}$ represent AM’s position and attitude states. $l_a$ is the onboard robot arm length, and the arm is connected to the aircraft body center by a hinge, driven by a motor to realize a lateral contact with the environmental surface.

Figure 2.

Frames of the AM system.

The AM system dynamics comprises three parts: UAV position, attitude, and robot arm dynamics. They are

(1) \begin{equation}{\ddot{\boldsymbol{{p}}}} = -{1/ m}{{\boldsymbol{{R}}}_{\varPhi }}{{\boldsymbol{{u}}}_{{1}}} + {{\boldsymbol{{F\!}}}_{{g}}}-{1/ m}{{\boldsymbol{{F\!}}}_{{c}}}({\boldsymbol{\varPhi }},{a_{{p}}}) \end{equation}

(2) \begin{equation}{\ddot{\boldsymbol\varPhi }} = -{{\boldsymbol{{I}}}^{-1}}{\dot{\boldsymbol\varPhi }} \times \boldsymbol{{I}}{{ \dot{\boldsymbol \varPhi }}} + {{\boldsymbol{{I}}}^{-{\rm{1}}}}{\boldsymbol{{B}}\boldsymbol{\tau }} + {{\boldsymbol{\tau }}_m}({\tau _a},{l_a},{m_a}) \end{equation}

(3) \begin{equation}{J_a}{\ddot \theta _a} = {\tau _a} +{\tau _{r}} \end{equation}

where ${\boldsymbol{{u}}_1}={ [{0,{\rm{ }}0,{u_1}} ]^{\text{T}}}$ and $\boldsymbol{\tau }={ [{{u_2},{u_3},{u_4}} ]^{\text{T}}}$ are UAV thrust and attitude torque vectors, and $\boldsymbol{{R}}_\varPhi$ is the transition matrix from {B} to {I}. $\boldsymbol{{F}}_{\!g}={ [{0,{\rm{ }}0,mg} ]^{\text{T}}}$ is system gravity vector, $\boldsymbol{{I}}=\text{diag} \{{{I_x},{I_y},{I_z}} \}$ is inertia matrix, and $\boldsymbol{{B}}=\text{diag} \{{l,l,{\rm{ }}1} \}$ is coefficient matrix. ${m_a},{\theta _a},{\tau _a},{\rm{and}}\,\,{J_a}$ are the robot arm’s mass, joint angle, motor torque, and rotational inertia. $\tau _r$ is the reaction torque generated from UAV attitude control. $\boldsymbol{{F}}_{\!c}$ is the exerted force on the AM during the contact-based landing process, and $\boldsymbol{{\tau }}_{m}$ is the torque effect generated from robot arm joint control.

2.2. Contact feature

The UAV platform has two features: non-self-stabled and under-actuated. Thus, a closed-loop control structure is usually applied to a UAV, making it behave as a spring-mass-damper-like system [Reference Meng, He, Li, Gu, Yang, Yan and Han25]. The following equation expresses the relationship between UAV position response $\boldsymbol{{X}}$ and the exerted external force $\boldsymbol{{F}}^{\text{ext}}$ :

(4) \begin{equation} \begin{array}{l}{\boldsymbol{{X}}}(s) = \cfrac{1}{{{s^2}m + sK_2^i + K_1^i}}{{\boldsymbol{{F}}}^{{\rm{ext}}}}(s), \quad (K_2^i = k_2^im,K_{\rm{1}}^i = k_1^im, \,i = {\rm{{\textit{x,y,z}}}}) \end{array} \end{equation}

where $\boldsymbol{{X}}={ [{x,y,z} ]^{\text{T}}}$ represents the position change. $k^i_{1,2}(i= x,{\rm{ }}y,{\rm{ }}z)$ are position controller gains. The mass-weighted parameters $K_1^i$ can be treated as the system stiffness, and $K_2^i$ are considered damping. The AM’s mass $m$ is the spring-mass-damper system mass.

In addition, owing to UAV’s under-actuation property, when an AM exerts a horizontal contact force on the environment, the AM needs to maintain a tilt angle. The heading $\psi$ is set at 0 for the forward contact-based landing, and the contact force $F_x$ can be derived and expressed in terms of the AM attitude angle $\theta$ and system gravity $mg$ :

(5) \begin{equation}{F_x}=-mg\tan \theta \end{equation}

3.1. Control issue

While an AM conducts the contact-based landing on a mobile platform, it faces several control issues: one controls AM’s steady flight, the second needs to maintain sustained contact with the target, and the last is to realize the robust contact-based following. In this case, the interactive force and torque produced by the relative motion of the mobile platform and the robot arm can be treated as external disturbances for the aircraft and will be eliminated by a robust controller. The steady flight and contact-based following issues can be addressed using a hybrid force and motion control framework [Reference Meng, He and Han26].

3.2. Controller design

An adaptive backstepping controller is designed for the AM system to deal with the unknown bounded disturbance. Firstly, variable substitution and feedback linearization are applied to (1)−(3), and a unified model is obtained. Next, the controller is designed in two steps: the basic controller structure is based on the backstepping method; the disturbance is estimated and eliminated by the sliding mode strategy.

Define the intermediate variable ${\boldsymbol{{h}}}_1={[{{h_{1,x}},{h_{1,y}},{h_{1,z}}} ]^{\text{T}}}$ , and the conversion relationship is:

(6) \begin{equation} \left \{ \begin{array}{l}{h_{ 1,{x}}} = -{{{u_1} \text{cos}\phi \text{sin}\theta }/ m}\\[5pt]{h_{ 1,{y}}} = {{{u_1} \sin \phi }/ m}\\[5pt]{h_{ 1,{z}}} = -{{{u_1} \text{cos}\phi \text{sin}\theta }/ m} + g \end{array} \right. \end{equation}

Thus, (1) can be rewritten as:

(7) \begin{equation}{\ddot{\boldsymbol{{p}}}} = {{\boldsymbol{{h}}}_1}-{1/ m}{{\boldsymbol{{F\!}}}_{{c}}} \end{equation}

Apply feedback linearization to (2), the relationship between the virtual input $\boldsymbol{\tau }^*=[ u_1^*,u_2^*,u_3^*]^{\text{T}}$ and the original input $\boldsymbol{\tau }={ [{{u_2},{u_3},{u_4}} ]^{\text{T}}}$ is

(8) \begin{equation} \left \{ \begin{array}{l}{u_2} = -{{({I_y}-{I_z})\dot \theta \dot \psi }/ l} + u_2^ * \\[5pt]{u_3}= - ({I_z}-{I_x}){{\dot \phi \dot \psi }/ l} + u_3^ * \\[5pt]{u_4}= - ({I_x}-{I_y})\dot \phi \dot \theta + u_4^* \end{array} \right. \end{equation}

Let ${\boldsymbol{{B}}}_{\tau }={\boldsymbol{{I}}^{-1}}\boldsymbol{{B}}$ in (2) and define the new variable ${\boldsymbol{\tau }}_v^*={\boldsymbol{{B}}}_{\tau }{{\boldsymbol{\tau }}^*}$ . Hence, (2) is expressed by the equation:

(9) \begin{equation} \ddot{{{\boldsymbol\varPhi}}} ={\boldsymbol{\tau }}_v^* +{\boldsymbol{\tau }}_{m} \end{equation}

From the above analysis, AM dynamics (1)−(3) can be expressed in a unified form:

(10) \begin{equation} \ddot{\boldsymbol{{x}}}=\boldsymbol{\rho }+\boldsymbol{{d}} \end{equation}

where $\boldsymbol{{x}}$ is the state vector of each sub-system (Making a distinction, the variable $x$ defined in Section 2 represents the position element.), $\boldsymbol{\rho }$ is the system input vector, and $\boldsymbol{{d}}$ is the disturbance vector.

Define state variables

(11) \begin{equation} \begin{array}{l}{{\dot{\boldsymbol{{x}}}}_1} = {{\boldsymbol{{x}}}_2}\\[5pt]{{\dot{\boldsymbol{{x}}}}_2} = {\boldsymbol{\rho }} + \boldsymbol{{d}} \end{array} \end{equation}

The variable errors are

(12) \begin{equation} \begin{array}{l} \Delta{{\boldsymbol{{x}}}_{1}} = {{\boldsymbol{{x}}}_{1}}-{{\boldsymbol{{x}}}_{1{\rm{ref}}}}\\[5pt] \Delta{{\boldsymbol{{x}}}_{2}} = {{\boldsymbol{{x}}}_{2}}-{{\boldsymbol{{x}}}_{2{\rm{ref}}}} \end{array} \end{equation}

Based on (11) and (12), the following relationships can be obtained

(13) \begin{equation} \Delta{{\dot{\boldsymbol{{x}}}}_{1}}={{\dot{\boldsymbol{{x}}}}_{1}}-{{\dot{\boldsymbol{{x}}}}_{1{\rm{ref}}}} ={{\dot{\boldsymbol{{x}}}}_{2}}-{{\dot{\boldsymbol{{x}}}}_{1{\rm{ref}}}} \end{equation}

(14) \begin{equation} \Delta{{\dot{\boldsymbol{{x}}}}_{1}}={{\dot{\boldsymbol{{x}}}}_{2{\rm{ref}}}}+({{\boldsymbol{{x}}}_{2}}-{{\boldsymbol{{x}}}_{2{\rm{ref}}}})-{{\dot{\boldsymbol{{x}}}}_{1{\rm{ref}}}} \end{equation}

Then, according to the backstepping control, the virtual control input is designed:

(15) \begin{equation}{{\boldsymbol{{x}}}_{2{\rm{ref}}}}= -{{\boldsymbol{{K}}}_1}\cdot \Delta{{\boldsymbol{{x}}}_{1}}+{{\dot{\boldsymbol{{x}}}}_{1{\rm{ref}}}} \end{equation}

Thus, based on (14), (15), and (11), the error state space equations are expressed as:

(16) \begin{equation} \begin{array}{l} \Delta{{{\dot{\boldsymbol{{x}}}}}_1}= -{{\boldsymbol{{K}}}_1}\cdot \Delta{{\boldsymbol{{x}}}_{1}}+\Delta{{\boldsymbol{{x}}}_2}\\[5pt] \Delta{{{\dot{\boldsymbol{{x}}}}}_2}={{{\dot{\boldsymbol{{x}}}}}_2}-{{{\dot{\boldsymbol{{x}}}}}_{2{\rm{ref}}}}={\boldsymbol{\rho }}+{\boldsymbol{{d}}}-{{{\dot{\boldsymbol{{x}}}}}_{2{\rm{ref}}}} \end{array} \end{equation}

The system control law is designed:

(17) \begin{equation}{\boldsymbol{\rho }}=-{{\boldsymbol{{K}}}_2}\cdot \Delta{{\boldsymbol{{x}}}_{2}}- \Delta{{\boldsymbol{{x}}}_1} +{{\dot{\boldsymbol{{x}}}}_{2{\rm{ref}}}} \end{equation}

For the unknown bounded disturbance $\boldsymbol{{d}}$ , $ |{ | \boldsymbol{{d}} |} | \le \bar d$ , and $\bar d$ is an unknown constant. Design an adaptive law for $\bar d$ to get the estimated boundary:

(18) \begin{equation} \dot{\hat{{\bar d}}}={k_{d}}\cdot \left \|{\Delta{{\boldsymbol{{x}}}_2}} \right \| \end{equation}

where the initialization and gain conditions are

(19) \begin{equation} \left \{ \begin{array}{l} \hat{{\bar d}}(0) = 0\\[5pt]{k_{d}} \gt 0 \end{array} \right. \end{equation}

A sliding mode term is introduced in control law to address the disturbance impact $\boldsymbol{{d}}$ in (10). Thus, the robust control law is designed:

(20) \begin{equation}{\boldsymbol{\rho }} =-{{\boldsymbol{{K}}}_2}\cdot \Delta{{\boldsymbol{{x}}}_{2}}-\Delta{{\boldsymbol{{x}}}_1} +{{\dot{\boldsymbol{{x}}}}_{2{\rm{ref}}}}-{\frac{{\Delta{{\bf{x}}_2}}}{{\left \|{\Delta{{\bf{x}}_2}} \right \|}}}\cdot{\hat{\bar d}} \end{equation}

From (16) and (20), the closed-loop system is expressed:

(21) \begin{equation} \begin{array}{l} \Delta{{{\dot{\boldsymbol{{x}}}}}_1}=-{{\boldsymbol{{K}}}_1}\cdot \Delta{{\boldsymbol{{x}}}_{1}}+\Delta{{\boldsymbol{{x}}}_2}\\[5pt] \Delta{{{\dot{\boldsymbol{{x}}}}}_2}=-{{\boldsymbol{{K}}}_2}\cdot \Delta{{\boldsymbol{{x}}}_{2}}-\Delta{{\boldsymbol{{x}}}_1}-\dfrac{{\Delta{{\bf{x}}_2}}}{{\left \|{\Delta{{\bf{x}}_2}} \right \|}}\cdot{\hat{\bar d}}+{\boldsymbol{{d}}} \end{array} \end{equation}

where ${\boldsymbol{{K}}}_1$ and ${\boldsymbol{{K}}}_2$ are positive diagonal matrixes.

The Lyapunov function is defined:

(22) \begin{equation} \begin{array}{l} V(\Delta{{\boldsymbol{{x}}}_1},\Delta{{\boldsymbol{{x}}}_2},\hat{\bar{d}} - \bar d)\\[5pt] ={\dfrac{1}{2}}\Delta{\boldsymbol{{x}}}_1^{\rm{T}}\Delta{{\boldsymbol{{x}}}_1}+{{1 \over 2}}\Delta{\boldsymbol{{x}}}_2^{\rm{T}}\Delta{{\boldsymbol{{x}}}_2} +{\dfrac{1}{{2{k_{d}}}}}{(\hat{\bar{d}}- \bar d)^2} \end{array} \end{equation}

Substitute (18) and (21) into (22). The derivative is

(23) \begin{align} \dot V &= \Delta{\boldsymbol{{x}}}_1^{\rm{T}}\Delta{{{\dot{\boldsymbol{{x}}}}}_1} + \Delta{\boldsymbol{{x}}}_2^{\rm{T}}\Delta{{{\dot{\boldsymbol{{x}}}}}_2} + {\dfrac{1}{{{k_{d}}}}}(\hat{\bar{d}}- \bar d)\dot{\hat{\bar{d}}} \nonumber \\[5pt] &= \Delta{\boldsymbol{{x}}}_1^{\rm{T}}( -{{\boldsymbol{{K}}}_1}\Delta{{\boldsymbol{{x}}}_1} + \Delta{{\boldsymbol{{x}}}_2}) + \Delta{\boldsymbol{{x}}}_2^{\rm{T}}( -{{\boldsymbol{{K}}}_2}\Delta{{\boldsymbol{{x}}}_2} - \Delta{{\boldsymbol{{x}}}_1}-{\dfrac{{\Delta{{\bf{x}}_2}} }{{\left \|{\Delta{{\bf{x}}_2}} \right \|}}} \hat{\bar{d}} + {\boldsymbol{{d}}}) + {\dfrac{1}{{{k_{d}}}}}(\hat{\bar{d}} - \bar d){k_{d}}\left \|{\Delta{{\boldsymbol{{x}}}_2}} \right \|\\[5pt] &= -{{\boldsymbol{{K}}}_1}\Delta{\boldsymbol{{x}}}_1^{\rm{T}}\Delta{{\boldsymbol{{x}}}_1} -{{\boldsymbol{{K}}}_2}\Delta{\boldsymbol{{x}}}_2^{\rm{T}}\Delta{{\boldsymbol{{x}}}_2} - \left \|{\Delta{{\boldsymbol{{x}}}_2}} \right \|\hat{\bar{d}} + \Delta{\boldsymbol{{x}}}_2^{\rm{T}}{\boldsymbol{{d}}}{\rm{\,+\,(}}\hat{\bar{d}}- \bar d{\rm{)}}\left \|{\Delta{{\boldsymbol{{x}}}_2}} \right \| \nonumber \end{align}

For the item $\Delta{\boldsymbol{{x}}}_2^{\rm{T}}{\boldsymbol{{d}}}$ and $ \|{\boldsymbol{{d}}} \|\le{\bar d}$ , the relationship $\Delta{\boldsymbol{{x}}}_2^{\rm{T}}{\boldsymbol{{d}}}\le \|{\Delta{{\boldsymbol{{x}}}_2}} \|\bar d$ can be obtained. Thus, (23) can be rewritten:

(24) \begin{align} \dot V & \le -{{\boldsymbol{{K}}}_1}\Delta{\boldsymbol{{x}}}_1^{\rm{T}}\Delta{{\boldsymbol{{x}}}_1} -{{\boldsymbol{{K}}}_2}\Delta{\boldsymbol{{x}}}_2^{\rm{T}}\Delta{{\boldsymbol{{x}}}_2} - \left \|{\Delta{{\boldsymbol{{x}}}_2}} \right \|\hat{\bar{d}} + \left \|{\Delta{{\boldsymbol{{x}}}_2}} \right \|\bar d + (\hat{\bar{d}}- \bar d)\left \|{\Delta{{\boldsymbol{{x}}}_2}} \right \| \nonumber\\[5pt] & = -{{\boldsymbol{{K}}}_1}\Delta{\boldsymbol{{x}}}_1^{\rm{T}}\Delta{{\boldsymbol{{x}}}_1} -{{\boldsymbol{{K}}}_2}\Delta{\boldsymbol{{x}}}_2^{\rm{T}}\Delta{{\boldsymbol{{x}}}_2} - \left \|{\Delta{{\boldsymbol{{x}}}_2}} \right \|(\hat{\bar{d}} - \bar d) + (\hat{\bar{d}}- \bar d)\left \|{\Delta{{\boldsymbol{{x}}}_2}} \right \| \nonumber\\[5pt] & = -{{\boldsymbol{{K}}}_1}\Delta{\boldsymbol{{x}}}_1^{\rm{T}}\Delta{{\boldsymbol{{x}}}_1} -{{\boldsymbol{{K}}}_2}\Delta{\boldsymbol{{x}}}_2^{\rm{T}}\Delta{{\boldsymbol{{x}}}_2} \\[5pt] & \lt 0 \nonumber \end{align}

The system is asymptotically stable. The state $\boldsymbol{{x}}$ will asymptotically converge to the desired value $\boldsymbol{{x}}_{d}$ , and in the process, the adaptive strategy could eliminate the disturbance impact.

3.3. Control framework implementation

The system control framework is depicted in Fig. 3. While an AM performs the contact-based landing task in forward flight, the workspace can be divided into two orthogonally decoupled subspaces—constrained space and free-flight space. The former is the normal direction of the interaction target, and the latter is the descent space along the surface. In the free-flight space, the proposed backstepping controller is used to ensure steady flight and is implemented under the hierarchical inner-outer loop structure. In the constrained space, the contact control can be transformed into position control based on the relationship (4). Meanwhile, an attitude feed-forward strategy is designed based on (5). The above process applies the hybrid force and motion control framework, which assists AMs in maintaining reliable and sustained contact in flight. The onboard robot manipulator is individually controlled based on arm kinematics, maintaining horizontal contact with the target. The trajectory generator serves two purposes: the first generates safe and steady descending trajectory points for the contact-flying AM; the second outputs the desired set-point of horizontal pose for the AM’s end-effector. The end height of the descent motion is set about 10 cm above the touchdown region and can be calculated based on the relative height in the visual recognition process. For the trajectory planning issue, the start and end positions are known. Thus, common methods can be applied, such as the Bezier function method [Reference Faigl and Vana27].

Figure 3.

The control framework for contact-based landing.

4.1. AM system

A hexa-rotor aircraft is designed as the aerial platform for the larger load capacity, and a one-DOF robot arm is installed, forming an AM system, as shown in Fig. 4. A roller-type end-effector is designed for sliding motion along the contact surface, as depicted in Fig. 5, and it is connected to the hinge by a carbon fiber link. A single-axis force sensor measures the force in the contact process. The battery balances AM system’s center of gravity, contributing to the steady flight in take-off and following stages. The AM’s other parameters are presented in Table I.

Figure 4.

AM system structure.

Figure 5.

The roller-type end-effector design drawing.

Table I.

AM parameters.

The roller is made of aluminum and carbon fiber to reduce weight, as shown in Fig. 5. A switch-lock structure, driven by a mini servo, is designed to enable or disable the roller’s spin and stop. The force sensor is installed at the arm link end, on which the roller is fixed.

4.2. Mobile platform

A ground mobile platform is designed, as depicted in Fig. 6. A slide rail is mounted on the support frame. A rectangular deck is fixed on the slider and driven by an end motor, enabling movement along the rail. A vertical board (i.e., the contact board) is installed on the deck platform for the contact-based landing task, providing sufficient support for sustained contact. The mobile platform’s velocity can be set freely according to different motion scenes. Two limit switches are installed on both ends of the slide rail to ensure safe movement. The mobile platform structural parameters are in Table II.

Figure 6.

The ground mobile platform.

4.3. Contact-based landing experiment and analysis

The experiment is conducted in a motion capture environment, and the experimental setting is shown in Table III.

The contact-based landing mission consists of the following steps: approach, contact, descent, and touchdown, as illustrated in Fig. 7 and the attached video (Video #1). The detailed process is as follows: the AM starts moving from the position [−1.0 m, −0.5 m, −1.85 m] $^{\text{T}}$ and contacts the platform’s vertical board in the position [0.0 m, −0.5 m, −1.85 m] $^{\text{T}}$ . The ground platform remains stationary at first and starts to move at a certain speed. Meanwhile, the AM slides down until a safe free-fall height and stops the propellers. Figures 8–12 show AM’s state curves in the contact-based landing process, and it lasts 35 s:

Table II.

Mobile platform parameters.

Table III.

The experimental setting.

$^*$ The value can be set artificially. Others are programmed settings.

Figure 7.

The contact-based landing on a mobile platform.

Figure 8.

The position changes in the laning process.

• Contact stage, ${\quad }{\quad }{\quad }{\quad }{\quad }{\;}{\,}0\sim$ 17.0 s;

• Descent motion, ${\quad }{\quad }{\quad } 17.0\sim$ 25.2 s;

• Complete landing, ${\quad }{\quad }{\;} 25.2\sim$ 35.0 s.

Contact stage $-$ The AM approaches forward and maintains sustained contact with the mobile platform while flying in the air. Descent motion $-$ The AM descends to approach the platform deck until a safe set-point height while maintaining sustained contact. Complete landing $-$ The AM stops propellers and achieves a final free-fall on the mobile platform.

In these figures, subscript $r$ represents the real value, and $d$ is the desired value or the set-point. In Fig. 8, in the beginning, there exists an error in the initial position $x$ . Before the contact stage, the AM is in free-flight mode and approaching the mobile platform in the forward. In the trajectory tracking process, the position error is inevitable. The first stage (the yellow part) indicates the static contact stage. Thus, the three-dimensional position ( $x_r$ , $y_r$ , and $z_r$ ) are constant [0.0 m, $-$ 0.5 m, $-$ 1.85 m] $^{\text{T}}$ . Then, the platform moves forward (the green part in Fig. 8). In the two stages (from 0 to 25.2 s), the AM maintains sustained contact with the mobile platform, showing that the controller can ensure steady contact and tracking flight. In the meanwhile, the AM is commanded to descend from $-$ 1.85 to $-$ 1.45 m (indicated by z-axis position curve in Fig. 8), and it is very close to the platform deck (the relative distance can be found in Fig. 7(b)), which is safe for the touchdown. Therefore, at the time of 25.2 s, the AM stops propellers and successfully reaches the ground platform, completing the contact-based landing mission. The remaining time (the red part, from 25.2 s to the end 35 s) means the AM is on the platform. $x$ trajectory still has a target value in the last stage. The desired trajectory value $x_d$ is generated by AM current position $x_r$ and the position error $\Delta x$ (calculated by (4)).

The position error curves are shown in Fig. 9, and the mean and variance are calculated in Table IV. The range 6−17 s is the constant contact force maintaining stage, and 17−25 s is the descent motion. The mean and variance of $\Delta x$ and $\Delta y$ demonstrate no significant differences. However, the absolute value of $\Delta z$ mean is larger, especially in the range 17−25 s, and the same trend can be observed for the variance. It is because the AM’s height is controlled to realize landing and the tracking error is inevitable.

Figure 9.

The position error curve.

Table IV.

Mean and variance of position.

AM’s attitude angle is recorded in the landing process, and two experimental results are shown in Fig. 10. From 0 to 17 s (the yellow part), the roll and yaw angle errors are in the range $\pm$ 0.03 rad ( $\lt 2^{\circ }$ ), and they are the same precision as in the regular free flight. Due to the aircraft’s under-actuation feature, the AM relies on pitch adjustment to generate a forward contact force. The beginning contact occurs at 3 s, and the desired pitch angle is 0.12 rad. In this case, the pitch error still meets the criteria of $\pm$ 0.03 rad, which means the AM attitude control loop is effective. In the descent stage (the green part), the real roll and pitch angle curves track the desired values and are not obviously affected by the descending motion. The yaw angle fluctuates due to the heading control error in the contact-based landing process.

Figure 10.

AM attitude angles in the landing process.

The attitude error curves are shown in Fig. 11. The larger mean and variance in the two experimental results are calculated in Table V. The attitude variances show no significant differences numerically. The error mean of $\Delta \psi$ in the range 17−25 s is larger than others. It shows that the physical contact affects AM’s heading control more than roll and pitch angles.

Figure 11.

The attitude error curve.

The onboard force sensor measures the real-time contact force data in the flight process and is plotted in Fig. 12. It also presents two experimental results. The desired force for the contact-based landing mission $F_d$ is 3 N (the red dashed line). From 0 to 3 s, the AM is in the transitional period and prepares for the contact motion. From 3 s to 25.2 s, the force $F_r$ is always >0 N, showing that the sustained contact is achieved. The larger mean and variance of the contact force error in the two experimental results are calculated in Table VI. The mean in the range 17−25 s has a greater absolute value than in 6 17 s. The error of $F_r$ may seem larger than regular robots such as industrial manipulators. It is mainly because the AM’s characteristics are different. The AM’s base is an under-actuated aircraft platform, a floating robot. Its position precision can only reach $\pm$ 2 cm in a motion capture environment. However, the ground mobile or industrial manipulators have heavy or fixed bases, and the robot position precision is less than one millimeter. In this work, the AM’s contact force is controlled based on position control, and thus, the force precision will be lower than other manipulators. Nevertheless, the force can make the AM maintain steady contact with the external environment to fulfill complex missions such as contact-based landing. From this point, the proposed method shows the effectiveness in the coordinated control of contact and motion for AMs.

5.1. Cooperative target design

For AM’s landing task, the onboard camera needs to recognize the specific markers to obtain the relative position and attitude information [Reference Zhang, Fang, Zhang, Jiang and Chen28], and an artificial marker (i.e., cooperative target) is usually used. The AM’s height will decrease from several meters (the hovering stage) to a few centimeters (the touchdown stage). Thus, the marker design should be suitable for far-distance recognition, close-range recognition, or even the partially occluded case. The existing markers (such as April Tag [Reference Borowczyk, Nguyen, Phu-Van Nguyen, Nguyen, Saussie and Ny29]) are usually designed very large for far-distance recognition and placed with several small markers around for the close-range case, which will bring larger errors in the relative pose calculation. In this paper, a concentric and nested circle target is designed, as depicted in Fig. 13. The marker consists of 2 $i$ +1 black-and-white circles, and $i$ can be set 2−4 in regular scenarios. Two circles (main $M_i$ and auxiliary $N_i$ ) are distributed in each layer. All $M_i$ circles are concentric, $N_i$ circles are placed in the interval, and their circle centers are collinear. The circle radiuses are proportionally designed for quick calculation. For the landing and recognition task, the marker can provide relative position and heading angle for the landing AM.

5.2. Vision-based recognition and localization

The cooperative target recognition relies on image edge detection. Bilateral filtering algorithm is applied [Reference Tomasi and Manduchi30]. Compared with the regular Gaussian filter, it replaces the intensity of each pixel with a weighted average of intensity values from nearby pixels. In this paper, the range is more concerned than the domain parameter to obtain the clear edge of the cooperative target (as shown in Fig. 13).

A contour extraction method is applied to acquire the nested information [Reference Suzuki31]. The circle detection is determined by the circumference-area ratio:

(25) \begin{equation}{{{L^2}}/ S} = {{{{(2\pi r)}^2}}/{(\pi{r^2})}} = 4\pi \end{equation}

Table V.

Mean and variance of attitude.

Figure 12.

The contact force curve.

Algorithm 1: Target detection and state calculation

In this way, it costs less time than the Hough circle transform method and is effective for all circle detection in the cooperative target. The outermost circle pair is selected as the extracted feature according to the descent height, and the circle center and radius are determined by the minimum circle covering method. The relative height $h$ and heading angle $\psi$ are calculated in the following:

(26) \begin{equation} h = {{({h_1} + {h_2})}/ 2} = {{\left({\frac{{f \cdot{R_1}} }{{{r_1}}}} + {\frac{{f \cdot{R_2}}}{{{r_2}}}}\right)}/ 2} \end{equation}

(27) \begin{equation} \psi = \left \{{\begin{array}{*{20}{c}}{\arctan \left({\dfrac{{{x_2} -{x_1}}}{{{y_2} -{y_1}}}}\right),\quad \quad \;\; \;\;{y_2} \ge{y_1}}\\[14pt]{\pi + \arctan \left({\dfrac{{{x_2} -{x_1}}}{{{y_2} -{y_1}}}}\right),\quad \;{y_2} \lt{y_1}}\\[14pt]{\psi - 2\pi,\quad \quad \quad \qquad \quad \quad \psi \gt \pi } \end{array}} \right. \end{equation}

where $f$ is focal length, $i(1,2)$ means the circle pair. $R_1$ and $R_2$ are the real radiuses of the cooperative target, $r_1$ and $r_2$ are the radiuses on the image plane, and $x_i$ and $y_i$ are pixel positions of the circle center. Algorithm 1 presents the circle’s detection process and the state calculation.

5.3. Method evaluation

The proposed algorithm is tested on the ground platform by using a mini PC-Intel NUC with the CPU as Intel Core i5-6260, 8G RAM, and 128G SSD. The image processing algorithms are running based on OpenCV. The start height, end height, and descent speed simulate AM’s states in the landing descent stage. Therefore, the camera starts to move at 1.85 m and descends to a height of 0.10 m. The experiments are conducted to simulate several scenarios, and the real-time recognition results are presented in Figs. 14−17 and also shown in the attached video (Video #2−#5). The largest red circle is used to calculate the relative height in the following figures. The circle pair (i.e., the same color circles) is used to calculate the heading angle, indicated by a green arrow.

Table VI.

Mean and variance of force.

Figure 13.

The cooperative target design.

Figure 14.

Target detection in Case A.

5.3.2. Case B: Landing on a moving platform

The target is placed on the ground and rotated by a stick, as shown in Fig. 15(a), and in this process, the camera descends. Figure 15(b) is clockwise rotation, Fig. 15(c) is counter-clockwise rotation, the rotation speed is about two revolutions per second, and in this case, the real-time heading angle can still be captured.

Figure 15.

Target detection in Case B.

Figure 16.

Target detection in Case C.

Figure 17.

Target detection in Case D.

Figure 18.

The height information.