2.1. Mechanism design

Figure 2 shows the overall structure and schematic diagram of the NDT inspection process. The proposed UAM is mainly composed of a quadrotor, a 2-DOF manipulator, and an NDT mechanism. It has the following advantages: First, the objects to be detected are mostly large-scale infrastructures, and the shape of the objects is vertically distributed. Detection from the open environment on the side can prevent the UAM from being affected by collision accidents. Second, the two joints of the designed manipulator can compensate for the attitude disturbance of the UAV during the contact inspection and improve the stability of the inspection as well. Third, the designed NDT mechanism can complete the inspection quickly and effectively without damaging the object.

Figure 2.

Schematic diagram of the NDT inspection process.

2.2. Contact inspection mechanism design

The designed manipulator overview is shown in Fig. 3. The manipulator has three joints, two of which are installed at the bottom to control the roll and pitch angles of the manipulator to compensate for the attitude disturbance of the UAV. The third joint is installed behind the manipulator and is used to control the forward and backward movement of the NDT mechanism relative to the manipulator. The robotic arm is made of carbon fiber tubes to reduce weight, and the NDT mechanism is installed at the end of the manipulator to reciprocate with the third joint of the manipulator. It is mainly composed of an elastic compliance mechanism, limit pin, fixed seat, annular stabilizer, and probe. Among them, the elastic compliance mechanism can ensure the safety of the UAM in contact with the detected object, provide active compliance control error, and design a control algorithm for constant contact force.

Figure 3.

UAM manipulator design overview.

Since the modeling in the contact mode is too complicated, the model is modeled on a two-dimensional plane perpendicular to the UAM contact inspection. Ignoring the friction force of the manipulator, define the function:

(1) \begin{align} {{f}_{c}}={{f}_{e}} + {{f}_{k}}-{{f}_{s}} \end{align}

(2) \begin{align} {{f}_{k}}={{\lambda }_{k}}({{x}_{o}}-{{x}_{i}}) \end{align}

(3) \begin{align} {{f}_{s}}={{\lambda }_{s}}({{\theta }_{o}}-{{\theta }_{i}}) \end{align}

where ${f}_{c}$ denotes the contact force between the end effector and object, to simplify the model, we discuss the one-dimensional model of the contact force with respect to the normal direction of the object in this paper. ${f}_{k}$ and ${f}_{s}$ denote the elastic force and pulling force of the steel wire drive motor, respectively. ${f}_{e}$ denotes the contact force perpendicular to the contact surface generated by the annular stabilizing structure, and the direction is perpendicular to the contact surface and inward, ${f}_{e}$ is a constant value during the contact work. ${x}_{o}$ , ${x}_{i}$ denotes the initial length and actual length of the spring, respectively.

${\theta }_{o}$ and ${\theta }_{i}$ are the absolute angles corresponding to the wire spin rudder at the spring length of ${x}_{o}$ and ${x}_{i}$ , respectively, and ${\lambda }_{k}$ , ${\lambda }_{s}$ is the gain factor. Let $r$ be the radius of the wire spin rudder, we have

(4) \begin{align} ({{x}_{i}}-{{x}_{o}})=\frac{\pi r}{180}({{\theta }_{i}}-{{\theta }_{o}}) \end{align}

Using (1, 2, 3, 4), the contact force can be written as:

(5) \begin{equation} \begin{split}{{f}_{c}}&=({{\lambda }_{k}}{{k}_{i}}-{{\lambda }_{s}})\widetilde{\theta }+{{f}_{e}} \\[5pt] &={{J}_{c}}\widetilde{\theta }+{{f}_{e}} \end{split} \end{equation}

where ${{k}_{i}}=\dfrac{\pi r}{180}$ , $\widetilde{\theta } = ({{\theta }_{i}}-{{\theta }_{o}})$ and ${{J}_{c}} = {{\lambda }_{k}}{{k}_{i}}-{{\lambda }_{s}}$ . The force from diagram of UAM is shown in Fig. 4.

Figure 4.

NDT mechanism contacting to target surface with contact force.

2.3. Onboard computer and sensors

The flight control unit (FCU) is an open-source FCU for pixhawk with PX4 firmware, which can stably control the pose of the UAM during a flight mission. The FCU provides an “Offboard” flight mode for position and attitude control by an external onboard computer. These instructions are provided by the Jetson Nano onboard computer. The camera sensor is equipped with an Intel RealSense D435 depth camera, which feeds the captured visual features to the onboard computer for algorithmic processing.

3.1. Inspection process

Autonomous NDT control consists of the following sub-tasks, the switching between each task state is triggered by the corresponding event (such as when the target appears in the camera field of view), and set an error threshold (error per direction in ZYX) during task switching, allowing the UAM to stay within the error threshold. The operator can switch the FCU to the auto-stabilized mode to keep hovering at any time through the button on the RC or activating the emergency safety switch as a safety protection measure. Figure 4 shows the state flow diagram for autonomous NDT control.

1. (1) Move to a location near the target: The position of the target is set to be unknown, so first set an initial state position within 1−1.5 m of the target, and control the UAM to fly to the vicinity of the target surface to ensure that the target object that needs to be non-destructively detected is within the field of view of the camera. In the experiment, UAM’s position and attitude information is defined and provided by the Optitrack motion capture system and can be defined by GPS coordinates in applications. The operator directly controls the sightline to fly to the set position and switches the FCU flight mode to Offboard mode through RC.

2. (2) Pose calibration: When the mode is switched to autonomous control and the target appears within the recognizable field of view of the camera ( $\le2\;\text{m}$ ), the UAM will switch to the pose calibration state after a buffer time (5 s), in which the visual positioning data from the depth camera and the attitude estimation data fusion of the UAV calibrate the position and attitude of the UAM, and the visual tracking algorithm ensures that the front axis of the manipulator (NDT mechanism, that is, the axis of the manipulator) is vertically aligned with the target and controls the UAM to contact the target. The onboard computer has a high refresh rate (70 Hz) to process the attitude estimation data, the data error of the manipulator attitude estimation caused by the time delay is within the allowable threshold. The designed coordinate points will eventually control the NDT mechanism to be perpendicular to the target surface and level with the ground.

3. (3) Return to the safe position after completing the NDT inspection: After the pose calibration is completed, the NDT mechanism is already in front of the target, and the UAM switches to the APPROCHING_TARGEET state. The designed controller generates a control trajectory by calculating the image error between the current visual circle image moment feature and the desired feature, the UAM fuselage moves toward the axis (facing the target) and moves close to the target. When the contact force generated by contact between the end effector and the target exceeds the set threshold, the UAM switches to NDT_MANIPULATION mode. At this time, the UAV keeps the pitch angle forward to continue to provide the contact force, ensuring that the NDT mechanism is in close contact with the target. During this period, the two joints of the manipulator make angle compensation for the attitude deviation of the UAV, keeping the vertical contact between the NDT mechanism and the target and the contact force constant. After the detection is completed, the UAM switches to the WAITING_FOR_COMMAND mode. The UAM falls back to a safe position and remains to hover until the operator switches back to manual mode via the RC.

3.2. System modeling

We use the coordinate system defined in ref. [Reference Lin, Wang, Miao, Zhong, Nie and Fierro14] as the frame in our proposed autonomous control algorithm, the equations of the dynamics with UAM can be depicted as follows:

(6) \begin{align} \dot{\textbf{p}}={\textbf{v}} \end{align}

(7) \begin{align} m\dot{{\textbf{v}}}=m{g{\textbf{e}}_{3}}+{{f}_{c}}-f{\textbf{R}}{{\textbf{e}}}_{3} \end{align}

(8) \begin{align} \dot{{\textbf{R}}}={\textbf{R}}{{\Omega }^{\wedge }} \end{align}

(9) \begin{align} {\textbf{J}}\dot{\boldsymbol{\Omega } }\,{+}\, \boldsymbol{\Omega } \times {\textbf{J}}\boldsymbol{\Omega } \, = \, \boldsymbol{\tau } \end{align}

where $\textbf{p}={{ [ x,y,z ]}^{\text{T}}}$ and ${\textbf{v}}={{ [{{v}_{x}},{{v}_{y}},{{v}_{z}} ]}^{\text{T}}}$ are the position and velocity vectors of the UAM’s center of mass relative to the inertial frame, respectively. ${\textbf{R}}\in SO(3)$ denotes the transformation matrix form body-fixed frame ${\sum }_{b}$ to inertial frame ${\sum }_{i}$ , ${{\textbf{e}}_{3}} = {{ [ 0,0,1 ]}^{T}}$ and the hat map $(\hat{\cdot }):{{\mathbb{R}}^{3}}\to SO(3)$ is the condition that ${\textbf{a}^{\wedge }}{\textbf{b}}={\textbf{a}}\times {\textbf{b}}$ for all ${\textbf{a}},{\textbf{b}}\in{{\mathbb{R}}^{3}}$ . $\boldsymbol{\Omega }$ is the angular velocity in body-fixed frame ${\sum }_{b}$ to inertial frame ${\sum }_{i}$ . $f$ denotes the thrust force of UAM, and $mg$ is the gravity. $\boldsymbol{\tau } = {{[{{\tau }_{1}},{{\tau }_{2}},{{\tau }_{3}}]}^{T}}$ represent the resultant moment of thrust force and contact force in the body-fixed frame.

3.3. Image features and system dynamics

In the contact manipulation task, a camera is installed parallel to the aerial manipulator. For the camera model, we place a black circular pattern next to the detection target to locate the UAM to complete the NDT task. In this paper, we use the classical camera model of a pinhole. For each observed feature point can be described as ${{\textbf{P}}_{c}} = {{ [{{X}_{c}},{{Y}_{c}},{{Z}_{c}} ]}^{T}}$ of the camera frame ${{\Sigma }_{c}}\;:\;{{O}_{c}}-{{x}_{c}}{{y}_{c}}{{z}_{c}}$ can be projected onto the normalized image plane as:

(10) \begin{equation}{\textbf{p}_{i}}={{[{{x}_{i}},{{y}_{i}}]}^{\text{T}}}={{\left [ \frac{{{X}_{i}}}{{{Z}_{i}}},\frac{{{Y}_{i}}}{{{Z}_{i}}} \right ]}^{\text{T}}} \end{equation}

Ref. [Reference Zheng, Wang, Wang, Chen, Chen and Liang29] has proposed a virtual camera approach to deal with the uncontrollable roll and pitch of UAM, the roll and pitch of the quadrotor are decoupled from the aerial manipulator during the contact manipulation; in this paper, we define a virtual camera frame $ \{{{\sum }_{{{c}^{*}}}}\;:\;{{O}_{{{c}^{*}}}}-{{x}_{{{c}^{*}}}}{{y}_{{{c}^{*}}}}{{z}_{{{c}^{*}}}} \}$ and foreign object frame $ \{{{\sum }_{t}}\;:\;{{O}_{t}}-{{x}_{t}}{{y}_{t}}{{z}_{t}} \}$ . The origin of the virtual camera frame ${\Sigma }_{{{c}^{*}}}$ is the same as the real camera frame ${\Sigma }_{c}$ . The coordinate of the observed points in ${\sum }_{{{c}^{*}}}$ are represented by ${{\textbf{P}}_{{{c}^{*}}}}={{ [{{X}_{{{c}^{*}}}},{{Y}_{{{c}^{*}}}},{{Z}_{{{c}^{*}}}} ]}^{\text{T}}}$ and denote ${\textbf{R}}_{c}^{{{c}^{*}}}$ be the rotation matrix of ${\sum }_{{{c}^{*}}}$ with respect to ${\sum }_{c}$ , and then, we have ${{\textbf{P}}_{{{c}^{*}}}}={\textbf{R}}_{c}^{{{c}^{*}}}{{P}_{c}}$ and the corresponding image coordinate ${\textbf{p}_{{{i}^{*}}}}={{[{{x}_{{{i}^{*}}}},{{y}_{i*}}]}^{\text{T}}}$ .

Figure 6.

Block diagram of the designed autonomous control strategy, in which $\frac{1}{M_{s}s^{2}+D_{s}s+K_{s}}$ is the Laplace transformation of the image-based impedance control scheme.

We choose to consider a circle feature attached to the object surface so that the j + k order image moments ${m}_{j,k}$ is defined as:

(11) \begin{equation}{{m}_{j,k}}=\int{\int{x_{{{i}^{*}}}^{j}y_{{{i}^{*}}}^{k}d{{x}_{{{i}^{*}}}}}}d{{y}_{{{i}^{*}}}} \end{equation}

the contour centroid $(\tilde{x},\tilde{y})$ of ${m}_{j,k}$ is given by $\tilde{x} = \dfrac{{{m}_{10}}}{{{m}_{00}}}$ and $h{\tilde{y}} = \dfrac{{{m}_{01}}}{{{m}_{00}}}$ , where ${m}_{00}$ denotes the area of the circular outline; therefore, the image moment features can be defined as

(12) \begin{align} {{\textbf{s}}_{t}}=[{{s}_{x}},{{s}_{y}},{{s}_{z}}] \end{align}

(13) \begin{align} {{s}_{x}}=\tilde{m}\tilde{x},\, \,{{s}_{y}}=\tilde{m}\tilde{y},\, \,{{s}_{z}}=\tilde{m} \end{align}

(14) \begin{align} \tilde{m}=\sqrt{m_{00}^{*}/{{m}_{00}}} \end{align}

According to the selected circular image moment feature, ${s}_{z}$ can be expressed by:

(15) \begin{equation}{{s}_{z}}=\sqrt{m_{00}^{*}/{{m}_{00}}}={{Z}_{c}} \end{equation}

Let ${Z}_{{{v}^{*}}}$ denotes the desired distance between the observed object plane of the camera and the image plane. $m_{00}^{*}$ is the desired value of $m_{00}$ when the desired value ${Z}_{{{v}^{*}}}$ is reached. The relationship between the first derivative of the image feature vector $\dot{\textbf{s}}_{\textbf{t}}$ and the camera velocity $\textbf{v}_{\textbf{c}}^{\textbf{c}}$ relative to the camera frame ${\Sigma }_{c}$ can be expressed as follows:

(16) \begin{equation} {{\dot{\textbf{s}}_{t}}}=-\frac{1}{{{Z}_{{{v}^{*}}}}}\textbf{v}_{\textbf{c}}^{\textbf{c}} \end{equation}

3.4. Visual servo control

The error of the image feature is defined as follows formula:

(17) \begin{equation}{{\textbf{e}}_{t}}={{\textbf{s}}_{t}}-{\textbf{s}}_{t}^{*} \end{equation}

where ${\textbf{s}}_{t}^{*}$ is the desired image feature vector.

The contribution of (16) is that decoupled the UAM’s translation and rotation, in which the model can be simplified and designed with controllers for each transformation.

The purpose of the designed visual servo controller is to stabilize the contact force ${f}_{c}$ in terms of the force error. The next subsection will demonstrate the linear relationship between the UAM’s position and image features, which shows the further influence of image features on contact force ${f}_{c}$ . The computation of desired image feature ${\textbf{s}}_{t}^{*}$ will be elucidated in the next subsection as well. Then, the visual servo control law is designed as:

(18) \begin{equation} {\textbf{v}}_{c}^{c}={\textbf{L}}_{s}^{-1}{{\boldsymbol{\eta } }_{c}}{{\textbf{e}}_{t}} \end{equation}

where ${{\textbf{L}}_{s}}\in{{\mathbb{R}}^{3\times 3}}$ is the first three columns of the interaction matrix illustrated in ref. [Reference Lin, Wang, Miao, Zhong and Fierro18], ${\boldsymbol{\eta } }_{c}$ is the $3\times 3$ positive diagonal matrix.

3.5. IBVS controller design for inner loop

In this part, image-based impedance control is proposed for contact force tracking and control of the force impedance of UAM along the normal direction to the surface of the observed object (the $z$ axis direction of ${f}_{c}$ ). According to the proposed decoupling attribute image feature ${\textbf{s}}_{t}^{*}$ , the image-based impedance control considered only controls the translational DOFs of UAM in this subsection. The impedance model equation is considered as follows:

(19) \begin{equation}{{M}_{s}}({{\dot{v}}_{z}}-\dot{v}_{z}^{*})+{{D}_{s}}({{v}_{z}}-v_{z}^{*})+{{K}_{s}}(z-{{z}^{*}})=\Delta{{f}_{c}} \end{equation}

where ${v}_{z}^{*}$ and ${z}^{*}$ are the desired value of ${v}_{z}$ and $z$ , respectively. ${M}_{s}$ , ${D}_{s}$ , ${K}_{s}$ are the desired inertia, damping, and stiffness coefficients of the system with constant numbers. $\Delta{{f}_{c}}={{f}_{c}}-f_{c}^{*}$ , where $f_{c}^{*}$ is the desired contact force w.r.t $\;{f}_{c}$ . Here we address the compliance problem of contact force (considered as an external disturbance) when UAM performs NDT by designing an impedance controller as a function.

Meanwhile, let ${\textbf{L}}^{e}$ and ${\textbf{L}}^{c}$ denote the relative position between the object with respect to the end effector and camera, respectively, and then, we can get the following relation about the variation of ${\textbf{L}}^{e}$ as:

(20) \begin{equation} \Delta {\textbf{L}}^{e}={\textbf{T}}^{e} \Delta {\textbf{L}}^{c} \end{equation}

where ${\textbf{T}}^{e}$ is the transformation matrix from camera frame ${\sum }_{c}$ to end-effector frame ${\sum }_{e}$ . $\Delta {\textbf{L}}^{e}$ , $\Delta {\textbf{L}}^{c}$ are the variation of ${\textbf{L}}^{e}$ , and ${\textbf{L}}^{c}$ , respectively. It is worth noting that the installation direction of the camera’s optical axis is parallel to the aerial manipulator, and then, we can get $\Delta{{\textbf{L}}^{e}}{{\textbf{e}}_{3}}=\Delta{{\textbf{L}}^{c}{{\textbf{e}}_{3}}}=-\Delta{{s}_{z}}$ , where $\Delta{{s}_{z}}=s_{z}^{*}-s_{z}^{i}$ , where $s_{z}^{i}$ denotes the input image feature with the onboard computer. Substitute it in (19), the visual impedance control equation can be expressed as:

(21) \begin{equation}{{M}_{s}}\Delta{{\ddot{s}}_{z}}+{{D}_{s}}\Delta{{\dot{s}}_{z}}+{{K}_{s}}{{s}_{z}}=\Delta{{f}_{c}} \end{equation}

The block diagram of the designed control strategy shown in Fig. 6. Recall that the image feature vector ${s}_{t}$ is only related to the translation transformation of the UAM; in this part, we control the rotational transformation of the UAM through the proposed geometric controller. Let the tracking errors of position and velocity be given by:

(22) \begin{align} {{\textbf{e}}_{x}}=\textbf{p}-{\textbf{p}_{d}} \end{align}

(23) \begin{align} {{\textbf{e}}_{v}}={\textbf{v}}-{\textbf{v}}_{c}^{c} \end{align}

so the control output ${\textbf{f}}$ can be written as follows:

(24) \begin{equation} {\textbf{f}}=-(-{{k}_{x}}{{\textbf{e}}_{x}}-{{k}_{v}}{{\textbf{e}}_{v}}-mg{{\textbf{e}}_{3}}-{{f}_{c}}+m{\ddot{\textbf{p}_{d}}}) \end{equation}

where ${k}_{x}$ , ${k}_{v}$ are several positive constants.

3.6. Geometric attitude control for outer loop

We consider the desired attitude and angular velocity as ${\textbf{R}}_{d}$ and ${\boldsymbol{\Omega } }_{d}$ , respectively, Let ${{\textbf{R}}_{d}}= [{{\textbf{u}}_{1d}},{{\textbf{u}}_{2d}},{{\textbf{u}}_{3d}} ]$ , in which

(25) \begin{align} \begin{cases}{{\textbf{u}}_{1d}}={{\textbf{u}}_{2d}}\times{{\textbf{u}}_{3d}} \\[7pt]{{\textbf{u}}_{2d}}=\dfrac{{{\textbf{u}}_{3d}}\times \beta }{\left \|{{\textbf{u}}_{3d}}\times \beta \right \|} \\[10pt]{{\textbf{u}}_{3d}}=\dfrac{{\textbf{f}}}{\left \| {\textbf{f}} \right \|} \end{cases} \end{align}

let $\beta ={{ [ \cos{{\psi }^{*}},\cos{{\psi }^{*}},0 ]}^{\text{T}}}$ . Then, we define the error function:

(26) \begin{equation} \mathfrak{S}({\textbf{R}},{{\textbf{R}}_{d}})=\frac{1}{2}tr\left [ I-{\textbf{R}}_{d}^{T}{\textbf{R}} \right ] \end{equation}

Furthermore, we get the angular velocity and attitude error as follows:

(27) \begin{align} {{\textbf{e}}_{\Omega }}=\boldsymbol{\Omega } -\textbf{R}^{T}{{\textbf{R}}_{d}}{\boldsymbol{\Omega }_{d}} \end{align}

(28) \begin{align} {{\textbf{e}}_{R}}=\frac{1}{2}{{({\textbf{R}}_{d}}^{T}{\textbf{R}}-{{\textbf{R}}}^{T}{{\textbf{R}}_{d}})}^{\vee } \end{align}

where the vee map $(\overset{\vee }{\mathop{\cdot }}\,)$ : $SO(3)\to{{\mathbb{R}}^{3}}$ indicates the inverse of the hat map, and define ${k}_{R}$ , ${k}_{\Omega }$ as the positive constant; thus, the control input $\boldsymbol{\tau }$ is obtained as follows:

(29) \begin{equation} \begin{aligned} \boldsymbol{\tau } &=-{{k}_{R}}{{\textbf{e}}_{R}}-{{k}_{\Omega }}{{\textbf{e}}_{\Omega }}+\boldsymbol{\Omega } \times \textbf{J}\boldsymbol{\Omega } \\[5pt] & \quad-{\textbf{J}}\left ( \hat{\boldsymbol{\Omega } }{{\textbf{R}}^{T}}{\textbf{R}}_{d}{{\boldsymbol{\Omega } }_{d}}-{{\textbf{R}}^{T}}{\textbf{R}_{d}}{\dot{\boldsymbol{\Omega } }_{d}} \right ) \end{aligned} \end{equation}

4.1. Experimental setup

The UAM we used was developed based on an existing quadrotor model (P-600) with a wheelbase of 600 mm; the NVIDIA Jetson Nano onboard computer was selected for real-time calculation of contact force and visual feedback, which runs on Ubuntu 18.04 + ROS Melodic operating system. The flight controller used is Pixhawk 4 (firmware version V1.10).

A 1-DOF manipulator is installed on the top of the quadrotor, connected with a 2-DOF rotary joint, and is driven by three servos (Dynamixel XM-430-W350-T). The total length of the manipulator is composed of three carbon fiber tubes with a total length of 90 cm and extends to the outside of the rotor for contact inspection. an NDT device composed of a ring stabilizer and a measuring probe is installed at the end of the manipulator. For the power system of UAM, we choose DJI TB48D (5700 mAh) battery to provide power for the whole system, and considering the balance of the center of gravity, the battery is installed behind the UAM to offset the weight of the manipulator.

The Intel RealSense D435 camera used to obtain visual feedback information is installed on the connecting rod of the manipulator, and the control current of the manipulator can be directly obtained from the servo to measure the actual contact force.

For the experimental scene, a wooden board (25 $\times$ 55 cm) in the room is chosen to be fixed vertically on the horizontal plane as the interaction plane between the UAM and the environment. The use of wooden boards simulates scenarios where non-destructive testing is limited in practical applications (such as non-destructive testing of oil and gas pipeline surfaces), and we can also assume that the experimental scenario is suitable for large-scale infrastructure testing (such as large-scale wind turbine blade flaw detection), etc. A circular marker (10 cm in diameter) used as a visual image feature was installed above the wooden board as a visual guide. We use external position information from the Optitrack motion capture system and attitude information from the Pixhawk 4 IMU for data fusion. The pose of the plank is unknown, and the initial position and pose of the UAM are arbitrarily random so that the circular marker is always in the camera’s field of view.

The total mass of UAM is 3.5 kg, and the parameters of controller is set as: ${{k}_{x}}=25$ , ${{k}_{v}}=4.3$ , ${{k}_{R}}=13$ , ${{k}_{\Omega }}=1.5$ . Additionally, the gain factors of (1)−(3) are ${{\lambda }_{k}}=0.576$ , ${{\lambda }_{s}}=7.48$ . Figure 7 shows the whole contact inspection process. The desired contact force in maintaining contact between the UAM and wooden board is set as $2.5\;\text{N}$ . UAM first take off to a random position and stably detected the circular image feature. Then, the UAM end effector was driven to a stable position perpendicular to the plane of the wooden board under the guidance of visual servo; after that, UAM maintains the force of $2.5\;\text{N}$ . The system remains free-flying from t = 0−14 s and stays in contact after 14 s.

Figure 7.

Snapshots of experiments. The three sub-images on the left (a, b, c) are from a distant top view, and the three sub-images on the right (d, e, f) are from a side view. The three sub-pictures correspond to the initial state of the UAM, the UAM end effector is calibrated in front of the plank, and the UAM is in stable contact with the plank.

4.2. Experimental performance

Figure 8 shows the real-time experimental data of the aerial manipulator contacting the wooden board during the inspection process. Figure 8(a) is the position trajectory of the UAM, (b)−(c) are the velocity trajectories of the UAM, (d)−(e) represent the attitude tracking error of the UAM and the actual and desired contact force between the end effector and the board, respectively. In the approaching stage, the UAM was in free flight with $F_{c}=0$ . After the aerial manipulator touches the wooden board, the UAV flight platform tilts forward to generate contact force. Before t = 14 s, UAM did not contact the board with $F_{c}=0$ . According to the information obtained in the experiment, UAM has no information that contact force can be generated at any time, so the change of $\Delta{{s}_{z}}$ cannot be too fast. According to the image-based impedance control, the UAM took about 5 s to maintain the contact force of the aerial manipulator at around 2.5 N. The designed impedance control law keeps the $F_{c}$ within a certain range, which is still an acceptable error range for UAM in contact inspection. Therefore, the experimental results are acceptable, and the disturbance due to the interaction between the UAM and the environment is still unavoidable for our designed control scheme.

Figure 8.

Experiment result of UAM contact inspection with a desired force of $2.5\;\text{N}$ in outdoor scene, the image feature markers were chosen as circular shape on a bridge pier (25 $\times$ 55 cm). (a) Position trajectory of UAM platform in inspection task. (b) Angular velocity of UAM platform in inspection task. (c) Linear velocity of UAM platform in inspection task. (d) Attitude tracking error of geometric controller, (e) contact force of UAM in inspection task.

Figure 9.

Snapshot of experiment in outdoor contact inspection.

4.3. Outdoor experiment

This experiment validates the effectiveness of the proposed control scheme for the UAM performing contact inspection with force tracking in an outdoor environment. In this experiment, UAM maintains the same contact force as in 4.2 to conduct contact inspection on the piers of outdoor bridges. All the parameters and initial status are the same as the previous experiment. The outdoor experiment scene is shown in Fig. 10. According to Fig. 9, considering the influence of outdoor wind, the fluctuation of the UAM in the whole inspection process is greater than that in the room. It can be seen from Fig. 10(e) that the fluctuation of contact force in the contact process is greater than that in the room, but it is still controlled at around 2.5 N.

Figure 10.

Experiment result of UAM contact inspection with a desired force of $2.5\;\text{N}$ , the image feature markers were chosen as circular shape on a wooden board (25 $\times$ 55 cm). (a) Position trajectory of UAM platform in inspection task. (b) Angular velocity of UAM platform in inspection task. (c) Linear velocity of UAM platform in inspection task. (d) Attitude tracking error of geometric controller, (e) contact force of UAM in inspection task.