2.1. Problem statement

In the planar hole search task, even when the positioning error is relatively large, the hole search task can still be completed by using time-consuming blind search methods. However, such methods are not suitable for thin-walled holes. In this study, the initial positioning error is assumed to be smaller than the sum of the peg radius and the outer radius of the thin-walled hole. Under this assumption, the peg may lose contact with the thin-walled hole during the hole search process, leading to task failure [Reference Ti, Gao, Shi and Zhao17]. The essence of the hole search process lies in guiding the peg into the assembly clearance region formed by the dimensional difference between the peg and the hole, which is typically small. Therefore, the key to a search strategy is to ensure that the search trajectory can cover this region while maintaining contact constraints. As shown in Figure 1(a), when the initial contact position satisfies the condition $L \leq (R + R_{\text{Hole}})/2$ , the spiral search can complete the hole search task, whereas using other trajectories may risk losing contact between the peg and the hole, leading to failure of the hole search task. On the other hand, as shown in Figure 1(b), when the initial contact position satisfies the condition $R + r_{\text{peg}} \gt L \gt (R + R_{\text{Hole}})/2$ , the effective contact area between the peg and the thin-walled hole is small, and the hole will lose contact during the blind search process, resulting in the failure of the hole search.

Figure 1.

Applicability of spiral search in the thin-wall hole search phase. (a) When the initial contact condition is satisfied $L \leq (R + R_{\text{Hole}})/2$ , the spiral trajectory can complete the hole search task. (b) When the condition is satisfied $ R + r_{\text{peg}} \gt L \gt (R + R_{\text{Hole}})/2$ , the hole search task fails.

2.2. Hole search strategy

When humans perform peg-in-hole assembly tasks, they typically rely on visual observation of the relative positional relationship between the peg and the hole for position adjustment, and combine tactile information to complete the assembly. For robots, visual information can be acquired via cameras, and contact forces can be measured using force sensors. However, visual cues are often affected by occlusion, resulting in incomplete feature representation. Moreover, the lack of texture on the peg and the hole surfaces further increases the difficulty of feature extraction.

Inspired by human assembly behavior, this study develops a vision-guided hole search strategy, and the overall framework is illustrated in Figure 2. First, relative position features of the peg-hole contact are extracted using vision. Then, the robot’s movement direction is adjusted based on the visual detection results. The assembly process consists of three phases: approach, search, and insertion; however, this paper primarily focuses on the search phase. To ensure the accurate and orderly completion of tasks at each assembly phase, this paper designs specific indicators to signal the conclusion of each phase, including the robot’s position information and contact forces/torques. When the peg first contacts the hole and enters the search phase, the system records the position information at this time as an important basis for subsequent phase judgments. During the search phase, the robot continuously adjusts the peg position until the insertion depth exceeds the threshold, at which point it enters the insertion phase.

Figure 2.

Hole search strategy flow chart.

Figure 3.

Hybrid force/position control.

2.3. Control frameworks

In assembly tasks, force control technology is required to achieve stable contact between the robot and the environment. Admittance control is the most widely used force control method, in which the robot system is typically modeled as a second-order mass-spring-damper system, as shown in Eq. (1).

(1) \begin{equation} \begin{cases} \boldsymbol{M}_{\boldsymbol{d}} {\ddot{\boldsymbol{x}}_{\boldsymbol{e}}} + \boldsymbol{B}_{\boldsymbol{d}} {\dot {\boldsymbol{x}}_{\boldsymbol{e}}} + \boldsymbol{K}_{\boldsymbol{d}} \boldsymbol{x}_{\boldsymbol{e}} = \boldsymbol{F} - \boldsymbol{F}_{\boldsymbol{d}} \\ \boldsymbol{x}_{\boldsymbol{e}} = \boldsymbol{x} - {\boldsymbol{x}_{\boldsymbol{d}}} \end{cases} \end{equation}

where $M_{{d}}$ , $B_{{d}}$ , $K_{{d}}$ are the positive-definite virtual inertia, damping, and stiffness matrices, respectively; $x$ is the current pose, $x_d$ is the desired pose; $F$ is the current contact force/torque; and $F_{{d}}$ is the desired contact force/torque.

As shown in Figure 3, the control system combines position control and force control to construct a robot hybrid force/position control framework. The control mode is switched through the 6 $\times$ 6 diagonal matrix $S$ , when the diagonal element of the matrix is 1, it means force control, and 0 means position control.

The pose increment under hybrid force/position control can be expressed as:

(2) \begin{equation} \Delta \boldsymbol{p} = (\boldsymbol{I} - \boldsymbol{S}) \cdot \boldsymbol{\Phi }(\boldsymbol{X}_d) + \boldsymbol{S} \cdot {\Gamma }(\Delta \boldsymbol{F}) \end{equation}

Where $\Delta p$ is the position increment of the robot’s end effector, $\Phi$ represents the position control method, $\Gamma$ represents the force control method, and $\Delta F$ is the force/torque error.

3.1. Assembly analysis

Figure 4 shows the hole search model of the proposed strategy. Due to the dimensional difference between the peg and the hole, an assembly clearance region is formed. When the peg enters the assembly clearance region, it will slide into the hole due to the loss of support force. However, the initial positional error between the peg and the hole usually lies outside this region. Therefore, the goal of the search phase is to guide the peg into the assembly clearance region.

Figure 4.

Hole search model.

In circular peg-in-hole assembly, due to pose deviation between the peg and the hole, two-point contact inside the hole is a common contact form. As shown in Figure 5, under the two-point contact state inside the hole, the peg is in the critical geometric condition where it can just enter the hole, corresponding to the maximum allowable tilt angle between the peg and the hole. When the relative tilt angle between the peg and the hole exceeds this critical angle, the peg can no longer enter the hole due to geometric constraints. Based on the geometric analysis of this critical state, under the two-point contact condition inside the hole, the maximum relative inclination angle between the peg and the hole is given by $\alpha _{\mathrm{max}}=\pm \arccos \left (r_{\mathrm{peg}}/R_{\mathrm{Hole}}\right )$ .

Figure 5.

Critical state of two-point contact inside a hole.

3.2. Coordinate system conversion

As shown in Figure 6, there are multiple coordinate systems involved in the assembly task: robot base coordinate system {B}, camera coordinate system {C}, and robot tool coordinate system {T}. The origin of {C} is located at the optical center of the RGB camera, and the origin of {T} is located at the center of the peg end face.

Figure 6.

Coordinate systems in assembly systems.

In the assembly process, both force and motion are defined in the coordinate system {T}, while motion control is performed in the coordinate system {B}. Therefore, the pose and force/torque conversion must be performed between the two coordinate systems to achieve consistency between the perception information and the motion. Formula (3) represents the translation and rotation in the tool coordinate system, and the mapping to the base coordinate system through coordinate transformation. Formula (4) represents the force/torque transformation relationship between different coordinate systems, aiming to convert the force/torque driving the robot’s motion from coordinate system {T} to {B}, in order to achieve motion regulation in admittance control.

(3) \begin{align}&\qquad\quad \begin{cases} \textbf{T} = \begin{bmatrix} \textbf{R} & \textbf{p} \\ 0 & 1 \end{bmatrix} \\ {}^{\text{base}}_{\text{new}}\textbf{T} = {}^{\text{base}}_{\text{ee}}\textbf{T} \cdot {}^{\text{ee}}_{\text{new}}\textbf{T} \end{cases} \end{align}

(4) \begin{align}& \begin{cases} \textbf{D} = \mathcal{S}\left ( {}^B\textbf{p}_O \right ) = \begin{bmatrix} 0 & -p_z & p_y \\ p_z & 0 & -p_x \\ -p_y & p_x & 0 \end{bmatrix} \\ \\[-10pt] \begin{bmatrix} {}^M\textbf{F} \\ {}^M\textbf{M} \end{bmatrix} = \begin{bmatrix} \textbf{R} & \textbf{0} \\ \textbf{D}\textbf{R} & \textbf{R} \end{bmatrix} \begin{bmatrix} {}^N\textbf{F} \\ {}^N\textbf{M} \end{bmatrix} \end{cases} \end{align}

where $p$ is the position vector, $R$ is the rotation matrix, ${}^{\rm ee}_{\rm new}T$ represents the homogeneous transformation matrix that performs pose transformation in the tool coordinate system, ${}^{\text{base}}_{\rm ee}T$ represents the homogeneous transformation matrix of the tool coordinate system relative to the base coordinate system, $D$ denotes the skew-symmetric matrix of the position vector, and ${}^M\textbf{F}, {}^N\textbf{F}, {}^M\textbf{M}, {}^N\textbf{M}$ represent forces and torques in different coordinate systems.

3.3. Elementary movement

Robotic motion control is typically categorized into position control and force control. In practical applications, robots typically employ the position control mode, which offers the advantages of high position-tracking accuracy and a relatively mature implementation framework. Under the force control mode, the robot is driven to adjust its position based on a set desired force. Since the underlying architecture of most commercial robots is based on position control, they typically employ admittance control to convert external force information into position-correction values, thereby indirectly achieving force control functionality. Because the adjustment is performed along the surface of the hole, both the desired force and the position adjustments are defined with respect to the coordinate system {T}.

The robot’s action can be defined as a desired force vector or position vector:

(5) \begin{align} \textbf{a}_f &= [ F_x \quad F_y \quad F_z ] \end{align}

(6) \begin{align} \textbf{a}_p &= [ \Delta x \quad \Delta y \quad \Delta z ] \end{align}

where $F_x$ , $F_y$ , $F_z$ is the desired force along the XYZ direction of the coordinate system {T}, $\Delta x$ , $\Delta y$ , $\Delta z$ are the motion steps along the XYZ direction of the coordinate system {T}. The elementary movement formed by each component of the action vector is shown in Figure 7. The robot’s action can be defined as a combination of one or more elementary movements.

Figure 7.

Elementary movement.

3.4. Contact state model

The core of visual perception is not to directly quantify the positional error between the peg and the hole through the visual system, but to perform contact state classification based on the positional relationships and their appearance in the camera image. Since there is no explicit category definition for the relative positional relationships between the peg and the hole, it is necessary to manually establish classification rules. These rules transform the unordered relative positional relationships into an ordered and discrete set of contact states, facilitates the neural network in learning the visual features associated with different contact states. As shown in Figure 8, when contact occurs between the peg and the hole, under a fixed single-view condition, the contact state can be classified based on the visibility of the hole in the camera’s field of view and the relative positional relationships between the peg and the hole. This observation provides a basis for constructing the subsequent classification rules.

Figure 8.

Contact state classification model under fixed single-view conditions.

Based on the above analysis, we classify the contact states between the peg and the hole into four categories and establish a plane coordinate system{O} with the center of the hole’s upper surface as the origin to describe the specific characteristics of each contact state, as shown in Figure 9. The camera adopts the eye-in-hand configuration, with the assumption that the projection of the RGB camera’s mounting position onto the XOY plane lies on the positive X-axis, while the camera is oriented toward the center of the hole. Under this assumption, the specific classification rules are as follows:

1. 1. When the center point of the peg end face is along the positive direction of the X-axis, and the hole cannot be seen from the camera’s perspective, it is defined as the front class;

2. 2. When the center point of the peg end face is along the negative direction of the X-axis, and both the peg and the hole are symmetrical about the X-axis, it is defined as the back class;

3. 3. In the camera perspective, the peg is located on the left side of the hole, which is defined as the left class;

4. 4. In the camera perspective, the peg is located on the right side of the hole, which is defined as the right class.

The above classification rules are also the foundation for establishing contact datasets in the future, the center of the hole is taken as the origin, and the circle with a radius of $R + \textbf{r}_{\text{peg}}$ is taken as the acquisition area of the contact state images, as shown in Figure 9. When the peg and hole are in contact, the camera acquires as many images as possible to construct the dataset. Both the acquisition of the dataset and the detection of the contact states rely on visual information. However, due to the discrepancy between the actual camera mounting pose and the ideal mounting pose, it is impossible to directly define image labels strictly according to the above classification rules. Therefore, after acquiring the images, the actual display effect in the image needs to be considered when labeling the data.

Figure 9.

Dataset acquisition and contact state division.

3.5. Search trajectory generation and optimization

This paper defines the contact state recognition between the peg and the hole as an image classification problem. Since the region of interest typically occupies only a small portion of the image, conventional full-image classification methods struggle to focus on relevant features effectively. Therefore, the YOLOv8 model is adopted to enable both localization and recognition of the contact state. The model structure adopts a lighter and more efficient C2f module, combined with improvements such as decoupling head, which significantly improves detection accuracy while ensuring real-time performance. With their excellent performance, YOLOv8 and its improved versions have been widely used in various object detection tasks [Reference Liu, Li, Feng and Lian29, Reference Nie, Li, Jiao, Lu, Shi, Song, Shi, Yang, Du and Liu30]. Given the clear target structure and low background complexity in this research task, The YOLOv8n model, as the most lightweight network architecture, is sufficient to meet the requirements.

Figure 10.

YOLOv8n-based contact state detection and position adjustment.

Figure 10. Long description

A diagram of a process for contact state detection and position adjustment using YOLOv8n. The diagram is divided into several sections labeled Backbone, Neck, Head, Output, and Move. The Backbone section includes a series of Conv and C2f layers. The Neck section features Concat and Upsample layers, which connect to the Head section. The Head section contains Detect layers. The Output section lists directions: front, back, left, and right. The Move section shows arrows indicating movement directions: up, down, left, and right. The diagram illustrates the flow of data through these layers and sections, indicating how the system processes information to detect contact states and adjust positions accordingly.

In this study, a camera acquires image sequences in real time and inputs them frame by frame into a pre-trained YOLOv8n model to detect the current contact state between the peg and the hole. Based on the detection results, corresponding position adjustment actions are generated to guide the peg to move in a direction that reduces the positional error between the peg and the hole. A mapping relationship between the contact states and the position adjustment actions is established, as shown in Figure 10.

The contact state detection results depend not only on the relative positional relationship between the peg and the hole but are also influenced by the display conditions from the camera’s viewpoint. Therefore, camera mounting angle deviation can affect visual detection results and further influence the search phase. The adjustment motion of the peg is designed based on the coordinate system {T}, while visual detection is performed in the camera coordinate system {C}. When the projection of the camera coordinate system {C} onto the XOY plane of coordinate system {T} is not aligned with the coordinate axes, the visual detection results will be affected. If there is an angle deviation $\alpha$ between the expected and actual camera mounting positions, some of the left- or right-class objects under the ideal camera pose may be recognized as front class in the actual mounting pose. In conventional circular peg-in-hole assembly tasks, variations in the camera mounting angle can affect the classification results, causing a discrepancy between the expected and actual motion. As shown in Figure 11(a), the expected motion of the peg is $Move_{\mathrm{expect}}$ , while the actual motion is $Move_{\mathrm{actual}}$ . According to the properties of the cosine function, when $\alpha \lt 45^\circ$ , the main direction of peg adjustment remains along the desired adjustment direction, as shown in formula (7). Therefore, the effect of camera mounting angle deviation on conventional circular peg-in-hole assembly is limited.

(7) \begin{equation} \begin{cases} Move_{\mathrm{actual}} \cdot \cos \alpha \lt Move_{\mathrm{expect}} \\ Move_{\mathrm{actual}} \cdot \cos \alpha \gt Move_{\mathrm{actual}} \cdot \sin \alpha , \quad \alpha \lt 45^\circ \end{cases} \end{equation}

Figure 11.

(a) Motion error. (b) search trajectory optimization.

Camera mounting angle deviation has little impact on conventional circular peg-in-hole assembly, and hole search can be successfully performed without any specific compensation. However, in assembly tasks with visual occlusion, relying solely on contact-state-based single-direction position adjustments is often insufficient to ensure a reliable search process. In such cases, displacements in other directions can be superimposed while adjusting the peg position to generate a zigzag or other search trajectory, as shown in Figure 11(b), in order to expand the area scanned by the search path and allow the peg to enter the previously difficult to reach area, effectively solving the problem of visual obstruction in the search process. For a rectangular peg-in-hole assembly task, even if there are no camera mounting angle deviations, the front side of the rectangular peg can still occlude the left and right sides when the peg and hole are in close proximity, thereby affecting the detection of contact states on both sides. In addition, visual accuracy is limited, and it is also necessary to compensate for the lack of visual accuracy by optimizing the search trajectory.

3.6. Proof of convergence of hole search strategy

3.6.1. Problem modeling

Let the positional error between the peg and the hole be defined as a two-dimensional vector $\textbf{d}_k= (x_k, y_k)$ , where $k$ represents the $k$ -th adjustment step. The vision-based classification module divides the current contact state into four classes (e.g. front, back, left, and right). According to the classification result, the robot executes a predefined motion with a fixed step length $h \gt 0$ . The control policy can be expressed as:

(8) \begin{equation} \left \{ \!\begin{aligned} d_{k+1} &= d_k + \Delta d_k \\ \Delta d_k &= \begin{cases} -h \dfrac {d_k}{\| d_k \|} & \text{correct probability } p \\ +hu_s & \text{Incorrect probability } 1 - p \end{cases} \end{aligned} \right . \end{equation}

where $ h \gt 0$ is the fixed step length. When classified correctly, the motion direction is $-\dfrac {\textbf{d}_k}{\|\textbf{d}_k\|}$ , which points toward the hole center. When misclassified, the direction is given by the unit vector $\textbf{u}_s$ , which satisfies $\|\textbf{u}_s\| = 1$ , but does not point toward the hole center.

The classification model satisfies the following assumptions:

1. 1. The classification results at each step are independent and only depend on the current state.

2. 2. There exists a constant $p \gt 0.5$ such that for all $d_k \neq 0$ , the probability of correct classification satisfies

   \begin{equation*}\mathbb{P}(\text{correct classification} \mid d_k) = p \gt 0.5.\end{equation*}

3. 3. Each of the four classes corresponds to a unique control direction, and the correct classification always corresponds to a direction that reduces $\|\textbf{d}_k\|$ .

4. 4. During each assembly, the step size $h$ can be chosen arbitrarily but must be a fixed value. As long as the search trajectory can cover the assembly clearance area around the hole, it can be considered as completing the task.

3.6.2. Convergence analysis in one-dimensional form

For the convenience of mathematical proof, we first consider the one-dimensional error case, where the error is given by

\begin{equation*}d_k \in \{0, h, 2h, 3h, \ldots \}.\end{equation*}

When $d_k \gt 0$ , a correct classification reduces the error to $d_k - h$ , and a misclassification increases the error to $d_k + h$ . Therefore, the system satisfies:

(9) \begin{equation} d_{k+1} = \begin{cases} d_k - h, & \text{with probability } p, \\ d_k + h, & \text{with probability } 1 - p. \end{cases} \end{equation}

The state $d = 0$ is an absorbing state, indicating that the alignment process has been successfully completed.

Define

\begin{equation*}f(i) = \mathbb{P}(\text{starting from } d = ih, \text{ the process eventually reaches } 0).\end{equation*}

It is clear that the boundary condition is

\begin{equation*}f(0) = 1.\end{equation*}

According to the Markov property, $f(i)$ satisfies the recurrence relation:

(10) \begin{equation} f(i) = p\, f(i-1) + (1-p)\, f(i+1), \quad i \ge 1. \end{equation}

The general solution of Eq. (10) is

(11) \begin{equation} f(i) = A + B \left (\frac {p}{1-p}\right )^i. \end{equation}

When $p \gt 0.5$ , we have

\begin{equation*}\frac {p}{1-p} \gt 1,\end{equation*}

and if $B \neq 0$ , $f(i)$ will diverge as $i \to \infty$ , violating the probability constraint $0 \le\, f(i) \le 1$ . Therefore, it must be that $B = 0$ . From $f(0) = A = 1$ , we obtain

(12) \begin{equation} f(i) \equiv 1, \quad \forall i \ge 0. \end{equation}

Equation (12) shows that when the correct classification probability $p \gt 0.5$ , the one-dimensional system will reach the absorbing state $d = 0$ . Even though misclassifications may occur, the overall bias toward the correct direction ensures that the random walk will not drift away indefinitely and will eventually move toward the hole center.

3.6.3. Convergence of two dimensional four classification system

The actual system is a two-dimensional error, which can be represented as:

\begin{equation*}d_k = (x_k, y_k).\end{equation*}

The four classes correspond to the fixed directions of “front,” “back,” “left,” and “right,” with each correct class corresponding to a direction that reduces $\|\textbf{d}_k\|$ . In the actual assembly process, the error along one dimension is first reduced to zero, and then the error along the other dimension is adjusted. During single-dimensional adjustment, the classification results can be simplified into two classes: correct and incorrect. The position adjustment corresponding to the correct class reduces the error in that dimension, whereas the adjustment corresponding to the incorrect class may increase or maintain the error in that dimension.

On this basis, both dimensions follow a one-dimensional biased random walk model with a “probability of moving in the correct direction $\gt$ 0.5,” and thus satisfy, respectively:

\begin{equation*}\mathbb{P}(\lvert x_k \rvert \to 0) = 1, \quad \mathbb{P}(\lvert y_k \rvert \to 0) = 1.\end{equation*}

Since both events occur with probability 1, their intersection still has probability 1, thus:

\begin{equation*}\mathbb{P}(d_k \to 0) = 1.\end{equation*}

That is, the two-dimensional system must inevitably reach the hole center region.

Even if there are recognition errors, as long as the overall bias in a single dimension leans toward the correct direction, the peg can still theoretically reach the center of the hole.

4.1. Experiment environment

In order to verify the effectiveness of the proposed hole search strategy, a robot assembly experimental platform was built as shown in Figure 12. The platform consists of a six-degree-of-freedom collaborative robot (DOBOT CR5), a six-axis force/torque (F/T) sensor (KUNWEI KWR75B), a depth camera (Intel RealSense D455), a peg with a diameter of 8.97 mm, and a hole with a diameter of 9.1 mm. The assembly parts are fabricated from PLA material using 3D printing. For the peg-hole contact direction, admittance control is applied, with the desired contact force set to 10 N. The adjustment actions were implemented in two modes: position control mode and force control mode. In the position control mode, the adjustment step size for the peg-in-hole assembly experiment was set to 0.3 mm, and the step size for the USB-A connector insertion experiment was set to 0.5 mm. In force control mode, the robot relied on external force to drive motion, and the driving force for the peg-in-hole assembly experiment was set to 15 N, and the driving force for the USB-A connector insertion experiment was set to 18 N. In the experiment, the hole search time is defined as the time elapsed from the initial peg-hole contact to when the peg’s insertion depth exceeds 1 mm.

Figure 12.

Assembly experimental platform.

Figure 13.

Training loss curve of the YOLOv8n model.

4.2. Experimental results

To validate the effectiveness of the contact state classification model under fixed single-view conditions, the YOLOv8n model was employed for training and testing on the dataset. A dataset of 360 images was constructed using the aforementioned data collection method and classification rules, with a 17:3 ratio between the training and validation sets. The number of training epochs was set to 300, while all other parameters were kept at their default values. Training was automatically terminated when model performance no longer improved or the specified number of epochs was reached. The YOLOv8n training loss curves are shown in Figure 13. After training began, both train/box_loss and train/cls_loss curves dropped rapidly and gradually stabilized. The decline in the loss functions indicates that the prediction errors continuously decrease and the model performance steadily improves, suggesting that the training process is stable and the model converges well. On the validation set, the val/box_loss curve showed significant fluctuations. This can mainly be attributed to two factors. First, the target region accounts for only a small proportion of the image. Second, the boundary of the target region is not sufficiently clear, so approximate regions were used in the annotation process, which made it difficult for the model to learn precise boundary information. The rapid decrease in val/cls_loss curve indicates that the model can quickly learn effective classification features and has good generalization ability on the validation set. The above results demonstrate that the trained model can accurately identify the contact state in images, indicating good classification performance. Although there are still some deviations in the localization of bounding boxes, this study primarily focuses on contact state recognition rather than precise localization; therefore, the current model is sufficient to meet the requirements of practical applications.

Figure 14 shows the results of contact state detection using the YOLOv8n model. Following data annotation and model training based on the classification criteria proposed in this paper, the model is able to accurately recognize four classes of contact states. Although the val/box_loss curve oscillates on the validation set, indicating that the detection box localization convergence is unstable, the model can still accurately locate the target region in actual use. The training curves and the test results together indicate that the model achieves good performance, which further verifies the feasibility of using image-based contact state classification to guide the search task.

Figure 14.

Object detection results.

Figure 14. Long description

The image contains fifteen separate photos of a metallic surface with holes. Each photo has labeled boxes indicating detected objects with labels such as back, front, left, and right, along with confidence scores. The labels and confidence scores vary across the images, showing different detected positions and orientations. The purpose of combining these images is to demonstrate the object detection results under various conditions.

Figure 15.

Circular peg-in-hole assembly search experiment. (a) Random point distribution. (b) Relationship between Euclidean distance and hole search time. (c) Search process.

While maintaining contact between the peg and the hole, 50 random initial contact positions were generated around the hole center, and their distribution is shown in Figure 15(a). This paper mainly investigates the hole search strategy in the assembly process and therefore focuses on the time consumption during the search phase [Reference Jiang, Yao, Huang, Yu, Wang and Bi31]. For visualization, the random initial contact positions were characterized by the corresponding Euclidean distances between the peg and the hole, and the relationship between Euclidean distance and hole search time is shown in Figure 15(b). The results show that the hole search time generally increases with Euclidean distance, although noticeable fluctuations are observed. When the Euclidean distances are similar, the search time corresponding to some initial positions remains significantly higher than that of neighboring random points. This is because the assembly clearance is small and the visual accuracy is limited, such that the peg sometimes requires more position adjustment steps before it can successfully enter the hole, thereby resulting in increased search time. Under the parameter settings used in this study, both the force control mode and the position control mode achieved a 100% hole search success rate. The average time consumption during the search phase is 4.56 s for the force control mode and 4.68 s for the position control mode. Figure 15 shows the flowchart of the peg-in-hole assembly search phase. After contact between the peg and the hole is established, the peg position is first adjusted in the left–right direction and then in the front–back direction.

When the left or right class is close to the boundary of the back class, the left or right class may be misclassified as the back class. This indicates that the effective region of the back class is larger than the predefined region. When the relative position between the peg and hole falls within this region, it will be classified as the back class, and the corresponding adjustment actions will be performed. However, the adjustment direction for these three contact states is always toward reducing the distance between the peg and the hole.

4.3. Comparative experiments

Blind search methods include various search trajectories, among which the spiral trajectory is the most widely used trajectory for thin-walled hole search tasks. In contrast, random trajectories, Lissajous trajectories, and other trajectories are more likely to cause loss of contact between the peg and the hole, thus leading to the failure of the search task. As shown in Table I, the spiral trajectory can only be applied to the thin-walled hole search phase under specific conditions, whereas the proposed search strategy is not constrained by these conditions. Next, the performance of the spiral search and the proposed search strategy will be compared experimentally under robot position control mode.

Table I.

Comparative experiment of different search trajectories in the thin-walled hole search phase.

Table I. Long description

A diagram of a process for contact state detection and position adjustment using YOLOv8n. The diagram is divided into several sections labeled Backbone, Neck, Head, Output, and Move. The Backbone section includes a series of Conv and C2f layers. The Neck section features Concat and Upsample layers, which connect to the Head section. The Head section contains Detect layers. The Output section lists directions: front, back, left, and right. The Move section shows arrows indicating movement directions: up, down, left, and right. The diagram illustrates the flow of data through these layers and sections, indicating how the system processes information to detect contact states and adjust positions accordingly.

In the spiral search experiments, the initial contact distance between the peg and the hole was increased in steps of 0.5 mm until the conditions for using the spiral search were no longer satisfied. The parameters for the spiral trajectory are an initial radius of 0.1 mm, a maximum radius of 5 mm, a pitch of 0.08 mm, and an angular increment of $5^\circ$ at each step. The search time of the spiral search was compared with that obtained from the above 50 randomly generated initial contact positions under position control mode. As shown in Figure 16(a), under the same Euclidean distance condition, the spiral search takes significantly longer than the proposed strategy and is more sensitive to initial distance errors, indicating that its performance is greatly affected by the initial position. Figure 16(b) shows the comparison of search trajectories between spiral search and the proposed strategy. As the initial distance increases, the traversal trajectory of spiral search significantly increases. Under the same initial position conditions, the search trajectory of the proposed strategy is shorter, so the search time is also shorter.

Figure 16.

(a) Comparison of hole search time. (b) Comparison of search trajectory.

4.4. Rectangular hole search experiment

Due to the differences in visual features across different types of peg-in-hole assemblies, when this method is applied to new assembly objects, it typically requires the reacquisition of image data and the training of a corresponding model. To verify the applicability of the proposed strategy to rectangular peg-in-hole assembly, a USB-A connector was used as the experimental object in this study. Based on the dimensions of the USB-A connector plug and socket, the data acquisition area was defined by establishing a coordinate system with the socket center as the origin and applying offsets of $\pm$ 10 mm along the long axis and $\pm$ 4 mm along the short axis. The collected data were annotated and used for training according to the defined contact-state classification rules. A total of 187 images were collected and divided into training and validation sets at a ratio of 9:1. The training parameters were the same as those used for the circular peg-in-hole task. The training loss and evaluation curves are shown in Figure 17. These curves demonstrate that the YOLOv8n model has successfully learned the relevant visual features.

Figure 17.

Training loss curve of the YOLOv8n model on the USB-A connector contact state dataset.

While maintaining contact between the peg and the hole, 50 random initial contact positions were generated around the hole center, and their distribution is shown in Figure 18(a). To illustrate the relationship between search time and the initial contact position, the 50 random initial contact positions were represented by the Euclidean distances between the peg and the hole. Figure 18(b) shows the relationship between Euclidean distance and hole search time under the two control modes. The results indicate that the hole search time increases with increasing Euclidean distance, indicating that reducing the initial contact position error is beneficial for decreasing the time consumption during the search phase. Some random positions exhibit significantly longer hole search times than nearby positions, mainly because the assembly clearance is small and visual accuracy is limited, making it difficult to directly guide the peg into the assembly clearance region. Therefore, more position adjustment steps are required to complete the hole search process, and the additional adjustment steps increase the hole search time. The hole search success rate was 100% under both control modes. The average time consumption during the search phase is 3.98 s for the force control mode and 4.12 s for the position control mode. Figure 18(c) shows the search process in USB-A connector insertion assembly.

Figure 18.

(a) Random point distribution. (b) Relationship between Euclidean distance and hole search time. (c) Search process.

In rectangular peg-in-hole assembly, significant visual occlusion occurs. The visual detection on the left and right sides is severely affected, whereas the front and rear directions are largely unaffected. To compensate for the information loss caused by visual occlusion, a V-shaped trajectory is designed in the search phase of USB-A connector insertion assembly. When the contact state is identified as the front or back, lateral displacement is superimposed while the peg performs forward or backward position adjustment. When the contact state is identified as the left or right, the peg is adjusted only along the corresponding direction. By enlarging the area swept during the search process, the peg can reach regions that cannot be covered by a single adjustment direction, thereby eliminating the influence of visual occlusion on the search phase.

4.5. Discussion

In practical applications, a small orientation deviation typically exists between the peg and the hole. Based on the dimensions of the circular peg-in-hole assembly, the critical tilt angle at which three-point contact occurs can be calculated. To verify the applicability of the proposed search strategy across the entire range, a boundary-testing approach is adopted. Within this range, the values $-3^\circ$ and $3^\circ$ are selected as the tilt amounts about the two coordinate axes, and each experiment is repeated twice for every test condition. Since the method performs well in extreme cases, it can also be inferred that it has good robustness and adaptability at other angles in this range. The detection results of the contact state between the peg and the hole mainly depend on the distribution characteristics of the clearance under the camera’s perspective. Even if the perspective changes slightly due to camera loading and unloading, the detection results remain stable, indicating that this method has good robustness.

According to the comparison results in Table II between the assembly method based on the proposed search strategy and other assembly methods, the assembly method using this strategy achieves a higher assembly success rate, indicating the good stability of the proposed strategy. The proposed search strategy ensures that the search process converges to the hole center, thereby significantly improving the overall reliability and success rate of the assembly.

Table II.

Comparison experiment of assembly success rate.

Table II. Long description

The image contains fifteen separate photos of a metallic surface with holes. Each photo has labeled boxes indicating detected objects with labels such as back, front, left, and right, along with confidence scores. The labels and confidence scores vary across the images, showing different detected positions and orientations. The purpose of combining these images is to demonstrate the object detection results under various conditions.