2. Related works

Greenhouse crop monitoring has traditionally relied on Internet-of-Things (IoT)-based systems that collect environmental parameters such as temperature, humidity, light intensity, and soil moisture. Numerous studies have demonstrated that such systems can effectively regulate greenhouse microclimates and improve irrigation and resource efficiency [Reference Contreras-Castillo, Guerrero-Ibañez, Santana-Mancilla and Anido-Rifón7–Reference Thilakarathne, Bakar, Abas and Yassin16]. These approaches have been successfully deployed in greenhouse environments for a variety of crops, including strawberries, tomatoes, chili, mustard greens, and hydroponic vegetables [Reference Bharti, Dongargaonkar, Kudkar, Das and Kenia17–Reference Juan Carlos, Estrada, Cesar Augusto, Paola Patricia, Marlon Alberto, G.ález Ramón Enrique, Roberto César, David Alfredo and Carlos Andrés19]. Despite their effectiveness for environmental regulation, sensor-based IoT systems provide only indirect and spatially aggregated information about crop conditions. Environmental measurements cannot capture plant- or fruit-specific attributes such as fruit presence, ripeness stage, spatial distribution, or occlusion. As a result, these systems are inherently limited in supporting harvesting-related decision-making or plant-level yield assessment. This limitation is particularly critical for strawberries, where rapid ripening and short harvest windows demand frequent, fruit-level observations rather than generalized environmental feedback.

To address the lack of plant-specific information, computer vision techniques have been increasingly applied in agricultural monitoring and harvesting. Convolutional neural networks (CNNs), especially YOLO-based object detection models, have demonstrated strong performance in strawberry detection under variable illumination and partial occlusion [Reference Montoya-Cavero, Díaz de León Torres, Gómez-Espinosa and Escobedo Cabello5, Reference Du, Cheng, Ma, Lu, Wang, Meng, Jiang and Hong20–Reference Wang, Yan, Meng, Yao, Han and Zhang23]. High detection accuracies and millimeter-level localization performance have been reported using RGB or RGB-D imagery [Reference Yu, Zhang, Yang and Zhang24]. Similar vision-based approaches have also been successfully applied to apples, tomatoes, citrus fruits, and other crops, confirming the general effectiveness of deep-learning-based perception in agricultural environments [Reference Liu, Nouaze, Mbouembe and Kim25–Reference Yoshida, Onishi, Kawahara and Fukao28]. However, most vision-based perception systems are designed as components of embedded, real-time robotic platforms, where perception, planning, and actuation are tightly coupled. These systems typically rely on high-performance onboard GPUs and execute detection and localization solely to support immediate harvesting actions. This tight integration increases system cost, power consumption, and mechanical complexity, while also limiting scalability. Moreover, because perception is performed only for real-time manipulation, the collected visual data are rarely archived or reused for long-term crop monitoring, historical analysis, or coordination across multiple robotic platforms.

Robotic strawberry harvesting systems have been widely studied, with numerous platforms demonstrating autonomous navigation, fruit detection, localization, and picking in greenhouse environments [Reference Yu, Zhang, Yang and Zhang24, Reference Ge, Xiong, Tenorio and From29, Reference Feng, Chen, Zhang and Wang30]. Related developments in controlled-environment agriculture, such as robotic systems designed for vertical farming, further highlight the growing role of robotics in plant-level operations [Reference Chitre, Dogra and Singla31]. More recent developments include dual-arm harvesting robots, ridge-planted strawberry harvesters, and systems incorporating improved detection and segmentation models [Reference Yoshida, Onishi, Kawahara and Fukao28, Reference Yu, Zhang, Liu, Yang and Zhang32, Reference Zhou, Wang, Au, Kang and Chen33]. These studies highlight the feasibility of autonomous strawberry harvesting and emphasize the importance of accurate perception for successful manipulation. Nevertheless, existing harvesting robots generally adopt a monolithic system architecture, in which data collection, perception, and harvesting are performed sequentially and in real time by a single platform. This architecture results in long harvesting cycles and places strict constraints on perception latency and computational resources. Any failure in perception directly interrupts harvesting, and the system cannot easily decouple monitoring from manipulation. Consequently, these platforms are poorly suited for continuous plant-level monitoring or for scenarios where perception data need to be reused, shared, or acted upon at a later time.

Cloud-connected and edge–cloud robotic systems have been explored in other application domains, including infrastructure inspection and field robotics, where perception data are collected by mobile platforms and processed remotely to reduce onboard computational burden [Reference Pathmudi, Khatri, Kumar, Abdul-Qawy and Vyas4, Reference Park and Kim12]. Recent studies on geometry-aware 3D perception have further demonstrated that accurate spatial understanding can be achieved using remotely processed point cloud data for precise task execution (e.g., cutting-point detection or structural damage recognition) [Reference Hu, Chen, Kang and Tang34, Reference Wang, Zhang, Cao, Hu, Wang, Deng, Gao and Tang35]. However, such cloud-enabled approaches are rarely examined in the context of greenhouse agriculture, and they typically do not address the specific challenges of plant-level geometric repeatability, temporal decoupling between perception and action, and robustness under variable network conditions. In particular, few studies analyze how perception errors propagate through camera–robot–world transformations when harvesting actions are executed after data collection rather than in real time.

In summary, prior research reveals three persistent gaps. First, sensor-based IoT systems lack plant- and fruit-level perception, limiting their ability to support spatially resolved monitoring and decision-making. Second, vision-based robotic systems typically couple perception and action, which constrains scalability, data reuse, and system modularity. Third, existing cloud-connected agricultural systems seldom provide the level of spatial accuracy, repeatability, and robustness required to enable reliable reuse of remotely processed perception data for downstream operations.

The proposed DCR is designed to address these gaps by providing a dedicated mobile platform for plant-specific RGB-D data acquisition, decoupled from real-time operation. By combining repeatable rail-guided positioning, multi-view image capture, and cloud-based detection and localization, the DCR enables accurate and reusable plant-level perception that supports continuous crop monitoring and can be extended to downstream applications such as autonomous harvesting.

3. Data collector robot design overview

The DCR is composed of three main subsystems: an RGB-D sensing unit, a robotic manipulation unit, and a mobile platform. Specifically, the system integrates an Intel RealSense D435i depth camera, a 6-degree-of-freedom (6-DoF) Mycobot 320 M5 robotic arm (Elephant Robotics), and a rail-guided vehicle (RGV) platform. As shown in Figure 1, the D435i camera is mounted on the tool flange of the Mycobot, and the robotic arm is installed on the RGV chassis. The RGV is driven by two stepper motors coupled to the rear wheels, and a proximity sensor is installed on the right side for positional referencing. A Raspberry Pi 4 serves as the main onboard controller, executing Robot Operating System 2 (ROS2)-based coordination of the mobile platform, robotic arm, and data acquisition, while a NodeMCU 32S handles low-level motor control and proximity sensing (see Table I). This architecture intentionally minimizes onboard computational requirements by offloading perception tasks to the cloud, enabling a lightweight and cost-effective system design.

An Intel RealSense D435i depth camera was selected as the sensing unit for plant-level data acquisition. RGB-D cameras have been increasingly adopted in agricultural applications due to their ability to provide both color and depth information in a compact and cost-effective form [Reference Lin, Tang, Zou, Xiong and Li36]. RGB-D cameras can be broadly classified into binocular and active sensing types. Among these, active RGB-D cameras are more commonly used in controlled environments, as they provide synchronized RGB images and depth maps and are equipped with infrared illumination, enabling reliable operation under low-light conditions [Reference Tu, Pang, Liu, Zhuang, Chen, Zheng, Wan and Xue37]. In the proposed system, the RGB images are used for strawberry detection and ripeness analysis, while the depth images provide the geometric information required for localizing individual fruits in 3D space.

Table I.

System components and functional roles.

Table I. Long description

A table with four columns and six rows, detailing the components of a system, their types, primary functions, and responsibilities. The columns are labeled ‘Component’, ‘Type’, ‘Primary function’, and ‘Responsibilities’. Row 1: Component, Raspberry Pi 4; Type, Single-board computer; Primary function, Main onboard controller; Responsibilities, Runs ROS2, coordinates system operation, controls robotic arm, manages communication with cloud server. Row 2: Component, NodeMCU 32S; Type, Microcontroller; Primary function, Low-level control; Responsibilities, Handles proximity sensor input and executes precise motor control for RGV stopping. Row 3: Component, Cloud Server; Type, Remote computing; Primary function, Perception processing; Responsibilities, Performs strawberry detection and localization (YOLO-based), stores results for later use. Row 4: Component, Intel RealSense D435i; Type, RGB-D Camera; Primary function, Perception sensing; Responsibilities, Captures RGB and depth data. Row 5: Component, Mycobot 320 M5; Type, Robotic arm; Primary function, Sensor positioning; Responsibilities, Executes multi-view image acquisition.

Figure 1.

Data collector robot (DCR) design overview. (1) Intel RealSense D435i RGB-D camera, (2) Mycobot 320 M5 robotic arm, (3) Mycobot mounting base, (4) power button, (5) front wheels, (6) proximity sensor, (7) charging port, (8) rear wheels, (9) guide rail, (10) stepper motors, (11) 24-V battery, and (12) onboard computer. The coordinate frames {W}, {R}, and {C} denote the world, robot, and camera frames, respectively.

In greenhouse strawberry cultivation, fruits are often suspended below dense foliage, and leaves may partially occlude the fruit from certain viewpoints. As a result, capturing images from a single fixed perspective is insufficient for comprehensive plant-level observation [Reference Zhou, Wang, Au, Kang and Chen33]. To address this challenge, the DCR employs a 6-DoF Mycobot 320 M5 robotic arm, which enables flexible camera positioning and multi-view image acquisition. The higher degree of freedom allows the camera to capture images from multiple angles around each plant, improving visibility of both fruits and leaves. This capability is particularly important for reliable fruit detection and for collecting plant data that can be reused for downstream analysis.

Beyond enabling multi-view image acquisition, the robotic arm plays a critical role in ensuring repeatable and consistent camera positioning across data collection cycles. This repeatability is essential for maintaining geometric consistency in the camera–robot–world transformation pipeline, which directly affects the accuracy of stored plant-level coordinates. By following predefined motion trajectories, the robotic arm ensures that images are captured from consistent viewpoints, improving both detection reliability and spatial localization accuracy. The use of a manipulator with sub-millimeter positioning resolution (manufacturer-specified) further supports consistent camera placement across repeated measurements. This capability is particularly important for continuous plant-level monitoring, where consistent observation conditions are required to enable reliable comparison of plant state and spatial information across multiple data collection cycles. Within the proposed decoupled perception–action framework, this repeatability ensures that stored perception data remains geometrically consistent and reusable over time. In addition, the use of a robotic arm provides a pathway for future integration with downstream robotic systems, where the same kinematic structure can be extended for manipulation tasks such as harvesting.

A mobile platform is required for the DCR to access individual strawberry plants along the cultivation layout. Two commonly used options in agricultural robotics are automated guided vehicles (AGVs) and rail-guided vehicles (RGVs) [Reference Bac, Van Henten, Hemming and Edan38]. AGVs are typically deployed in open-field or industrial environments where autonomous navigation is necessary; however, they rely on expensive sensing hardware such as light detection and ranging (LiDAR) or simultaneous localization and mapping (SLAM)-based perception systems and require high-performance onboard computing platforms (e.g., NVIDIA Jetson Xavier or Intel Next Unit of Computing (NUC)). These requirements increase system cost and complexity, and AGV positioning repeatability is often limited to the ±3–8 mm range due to odometric drift and mapping uncertainty.

In contrast, RGVs are well suited to controlled greenhouse environments, where tabletop or suspended cultivation systems allow the installation of simple guide rails. The proposed DCR adopts an RGV-based platform that uses a low-cost Raspberry Pi 4 for motion control and does not require map calibration or localization updates. The installation cost of the guide rail is modest (approximately USD 75–100 per 10 m), and rail guidance eliminates navigation drift entirely. This design provides high positional repeatability and enables precise plant-level data collection, making it a practical and economical solution for greenhouse monitoring and future automated harvesting applications.

4. Data collector robot control system

This section describes the control framework of the DCR, which coordinates mobile positioning, robotic arm motion, and image acquisition to enable repeatable plant-level data collection. The control system is designed to ensure consistent alignment between the robot and each strawberry plant while synchronizing sensing and motion for reliable data acquisition.

The DCR control architecture is implemented using ROS2 and is organized into three functional modules: mobile platform control, robotic arm control, and camera control. These modules operate in a coordinated manner to decouple plant-level data acquisition from harvesting actions while maintaining precise spatial repeatability. An overview of the control architecture and module interactions is shown in Figure 2.

To ensure consistent starting conditions, the DCR performs an initial position calibration procedure at system startup. During this process, the mobile platform moves along the guide rail until a proximity sensor detects a fixed position guide plate (PGP). This reference point is used to reset the platform position, ensuring that each data collection cycle begins from a known and repeatable origin, even if the robot has been repositioned manually. During operation, the mobile platform moves sequentially between strawberry plants along the guide rail. At each plant location, the platform stops at a predefined position such that the plant is aligned with the base frame of the robotic arm. This repeatable alignment enables consistent camera viewpoints and simplifies downstream localization.

Figure 2.

Control system architecture of the data collector robot (DCR), illustrating the coordination between the mobile platform, robotic arm, and camera modules for synchronized motion control and plant-level data acquisition.

Once the platform is positioned, the robotic arm executes a predefined motion sequence to capture multiple images of the plant from different viewpoints. A forward-facing view (FFV) is used to capture the fruit region, while additional side views (SV) are acquired to observe plant foliage. Depth information is recorded only for the FFV, which is used for 3D localization of strawberries. This multi-view acquisition strategy improves visibility while limiting unnecessary data transmission. After image acquisition, the collected data are transmitted to an Ali Cloud Elastic Compute Service (ACECS) for processing. Strawberry detection and localization are performed remotely, and the processed results are stored in a database for later access. The mobile platform then advances to the next plant and repeats the same procedure. After completing data collection for all plants, the DCR returns to its initial position, concluding the data collection cycle (see Figure 3).

This control framework enables reliable, repeatable, and scalable plant-level data collection while keeping onboard control and computation lightweight. By separating motion coordination from perception processing, the system supports future integration with autonomous harvesting robots that can operate using the stored plant-level information.

5. Strawberry detection using the YOLOv11s model

Strawberry detection was performed using the YOLOv11s (small) model, a recent addition to the YOLO (You Only Look Once) object detection series, offering improved accuracy and inference speed over previous variants. A total of 3207 manually annotated RGB-D images were used for training, collected across multiple greenhouse sessions to capture natural variation in cultivation conditions. Approximately 35% of the images were acquired under natural sunlight, 45% under LED illumination, and 20% under mixed or partially shaded conditions. These sessions also covered different stages of plant development, resulting in variation in occlusion patterns, leaf density, and fruit clustering. This diversity reflects real-world greenhouse environments and improves the generalization performance of the YOLOv11s model.

The dataset was split into training (80%), validation (10%), and test (10%) subsets while preserving the class distribution. To prevent data leakage, all splits were made at the plant level, ensuring that images of the same plant never appeared in multiple subsets. This avoids artificially inflated performance caused by near-duplicate images being shared across splits.

Controlled data augmentations were applied exclusively to the training subset to enhance robustness under different imaging conditions. Augmentations included horizontal flipping (p = 0.5), brightness adjustment (±15%), and Gaussian blur (σ ≤ 1.2 px, p = 0.3). Because the robot maintained a consistent imaging orientation, large-angle rotations were omitted; only minor angular jitter (≤5°) was introduced to emulate natural mechanical vibration. Image resolution was standardized to 640 × 640 px, and annotations were generated using Roboflow.

Figure 3.

Flowchart of the DCR data collection process, illustrating the sequence of initial position calibration, plant-level positioning, multi-view image acquisition, and cloud-based data transmission.

Ripeness labeling followed the natural anthocyanin progression in strawberries: a fruit was labeled as ripe when at least 80% of its visible surface appeared red. Representative annotated samples under varied lighting conditions are shown in Figure 4. Detection outputs follow the standard YOLO bounding-box format. For an image, the model outputs,

(1a) \begin{equation}\beta =\left[\delta _{1},\ldots ,\delta _{n}\right]\end{equation}

where each bounding box $\delta _{i}$ contains the top-left ( $x_{1},y_{1}$ ) and bottom-right ( $x_{2},y_{2}$ ) pixel coordinates along with a classification label $O_{\textit{class}}$ (ripe or unripe):

(1b) \begin{equation}\delta _{i}=[x_{1},y_{1},x_{2},y_{2},O_{\textit{class}}]\end{equation}

The final trained YOLOv11s model achieved 97.7% mAP@50 (compared to 97.5% for YOLOv8s and 94.2% for YOLOv7s as shown in Table II), with detection accuracies of 97.1% for ripe strawberries and 98.3% for unripe strawberries. Despite having a lower computational complexity than medium and large variants, YOLOv11s demonstrated stable detection across diverse illumination and occlusion conditions, making it well suited for the proposed data collection pipeline.

Table II.

Comparative evaluation of strawberry detection models on the annotated dataset, reporting detection accuracy (mAP@0.5), computational complexity (GFLOPs), and measured per-image inference latency on the ACECS and a Raspberry Pi 4.

Table II. Long description

A table comparing strawberry detection models based on accuracy, computational complexity, and inference latency. The table has five rows and five columns. The columns are labeled mAP@0.5 (%), FLOPS (G), Average per-image latency on ACECS (s), Average per-image latency on Raspberry Pi 4 (s), and Remarks. The rows are labeled with different models: MobileNet-SSD v1, YOLOv7s, YOLOv8s, and YOLOv11s (proposed). Row 1: MobileNet-SSD v1, 90.9, 4.1, 0.1, 0.6, Very lightweight but less accurate. Row 2: YOLOv7s, 94.2, 13.6, 0.3, 1.5, Balanced trade-off. Row 3: YOLOv8s, 97.5, 10.8, 0.7, 3.2, High accuracy, moderate cost. Row 4: YOLOv11s (proposed), 97.7, 10.7, 0.7, 3.2, Highest accuracy and moderate cost.

Figure 4.

Annotated strawberry images captured under varied lighting conditions: (a) LED lighting, (b) natural lighting, and (c) mixed lighting.

5.1. Depth image-based strawberry localization

Once strawberries are detected in the RGB image, their 3D positions are obtained by combining the depth measurement with the camera intrinsics. Let the midpoint of the bounding box be

(2) \begin{equation}M=\left(\frac{x_{1}+x_{2}}{2},\frac{y_{1}+y_{2}}{2}\right)\end{equation}

where ( $x_{1},y_{1}$ ) and ( $x_{2},y_{2}$ ) are the top-left and bottom-right pixel coordinates of the bounding box. The Intel RealSense D435i uses the Brown–Conrady distortion model. To obtain the undistorted pixel coordinates ( $x_{u},y_{u}$ ), the normalized pixel coordinates are first computed as

(3) \begin{equation}x=\frac{M_{x}-p_{x}}{f_{x}}, y=\frac{M_{y}-p_{y}}{f_{y}}\end{equation}

where ( $p_{x},p_{y}$ ) and ( $f_{x},f_{y}$ ) are the principal point and focal lengths. The radial distortion function is

(4a) \begin{equation}f=1+k_{1}r^{2}+ k_{2}r^{4}+k_{5}r^{6}, \quad r^{2}=x^{2}+y^{2}\end{equation}

and the tangential distortion terms are

(4b) \begin{equation}x_{d}=2k_{3}xy+k_{4}(r^{2}+2x^{2}), \quad y_{d}=k_{3}(r^{2}+2y^{2})+2k_{4}xy\end{equation}

The undistorted normalized pixel coordinates become

(4c) \begin{equation}x_{u}=xf+x_{d}, \quad y_{u}=yf+y_{d}\end{equation}

Given the depth value $d$ at pixel M, the 3D point in the camera frame {C} is obtained through the standard pinhole deprojection:

(5) \begin{equation}p_{C}=\left[\begin{array}{c} x_{C}\\[2pt] y_{C}\\[2pt] z_{C}\\[2pt] 1 \end{array}\right]=\left[\begin{array}{c} x_{u}d\\ y_{u}d\\[2pt] d\\[2pt] 1 \end{array}\right]\end{equation}

5.1.1. Transformation from camera to robot base frame

Let ${}^{R}{T_{6}}{}$ denote the forward kinematic transform from the robot base frame {R} to the tool flange {6}, obtained using the standard Denavit–Hartenberg formulation. The transform from the tool flange to the camera frame is the known constant matrix ${}^{6}{T_{C}}{}$ . The 3D point expressed in the robot base frame is then

(6a) \begin{equation}p_{R}={}^{R}{T_{6}}{}{}^{6}{T_{C}}{}p_{C}\end{equation}

The individual DH transforms follow the standard convention

(6b) \begin{equation}{}^{i-1}{T_{i}}{}=\left[\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} cos\theta _{i} & -sin\theta _{i}cos\alpha _{i} & sin\theta _{i}sin\alpha _{i} & a_{i}cos\theta _{i}\\[3pt] sin\theta _{i} & cos\theta _{i}cos\alpha _{i} & -cos\theta _{i}sin\alpha _{i} & a_{i}sin\theta _{i}\\[3pt] 0 & sin\alpha _{i} & cos\alpha _{i} & d_{i}\\[3pt] 0 & 0 & 0 & 1 \end{array}\right]\end{equation}

and the full forward kinematics is

(6c) \begin{equation}{}^{R}{T_{6}}{}={}^{R}{T_{1}}{}{}^{1}{T_{2}}{}{}^{2}{T_{3}}{}{}^{3}{T_{4}}{}{}^{4}{T_{5}}{}{}^{5}{T_{6}}{}\end{equation}

5.1.2. Transformation to world frame

When the DCR moves along the rail, each plant position corresponds to a discrete index n. The world frame { $W$ } is defined at the robot base when the DCR is at its initial position. Since the robot translates 250 mm between adjacent plants, the robot-base-to-world transform is

(7a) \begin{equation}{}^{W}{T_{R}}{}\left(n\right)=\left[\begin{array}{c@{\quad}c@{\quad}c@{\quad}c} 1 & 0 & 0 & 250n\\[3pt] 0 & 1 & 0 & 0\\[3pt] 0 & 0 & 1 & 0\\[3pt] 0 & 0 & 0 & 1 \end{array}\right]\end{equation}

Thus, the fruit position in the world frame is

(7b) \begin{equation}p_{W}={}^{W}{T_{R}}{}(n)p_{R}\end{equation}

If only translation is applied, this simplifies to

(7c) \begin{equation}x_{W}=x_{R}+250n, \quad y_{W}=y_{R}, \quad z_{W}=z_{R}\end{equation}

5.2. Error propagation in the camera–robot–world transformation pipeline

Automated harvesting requires accurate world-frame fruit coordinates generated from the DCR’s perception pipeline. Because the proposed system decouples data collection from harvesting, the fruit coordinates computed during data collection may be used seconds to minutes later by a separate harvesting robot. During this interval, small uncertainties in pixel localization, depth measurement, hand–eye calibration, robot-arm kinematics, or RGV stopping precision may propagate through the camera–robot–world transformation chain. A theoretical error-propagation analysis is therefore useful for identifying the dominant contributors to positional uncertainty and for assessing whether the resulting world-frame estimates satisfy the requirements for downstream harvesting.

The final world-frame fruit position given in (7b) is obtained from the transformation chain $p_{W}={}^{W}{T_{R}}{}(n){}^{R}{T_{6}}{}{}^{6}{T_{C}}{}p_{C}$ , where $p_{C}$ is the 3D point reconstructed in the camera frame, ${}^{6}{T_{C}}{}$ is the hand–eye calibration transform, ${}^{R}{T_{6}}{}$ is the fixed forward kinematic transform to the tool flange during image capture, and ${}^{W}{T_{R}}{}(n)$ is the pure translation from the robot base to the world frame at plant index $n$ . Taking the total first-order variation of $p_{W}$ yields,

(8a) \begin{equation}\delta p_{W}\approx {}^{W}{T_{R}}{}\left(n\right){}^{R}{T_{6}}{}{}^{6}{T_{C}}{}\delta p_{C}+{}^{W}{T_{R}}{}\left(n\right){}^{R}{T_{6}}{}\delta \left({}^{6}{T_{C}}{}\right)p_{C}+{}^{W}{T_{R}}{}\left(n\right)\delta \left({}^{R}{T_{6}}{}\right){}^{6}{T_{C}}{}p_{C}+\delta \left({}^{W}{T_{R}}{}{\left(n\right)}\right)p_{R}\end{equation}

Here, each term corresponds to a distinct source of uncertainty in the pipeline. Using (5), small perturbations in pixel coordinates $(\delta x_{u},\delta y_{u})$ and depth $\delta d$ yield,

(8b) \begin{equation}\delta p_{C}=\left[\begin{array}{c} d\delta x_{u}+x_{u}\delta d\\[3pt] d\delta y_{u}+y_{u}\delta d\\[3pt] \delta d\\[3pt] 0 \end{array}\right]\end{equation}

This propagates to the world frame as

(8c) \begin{equation}\delta p_{W,cam}\approx {}^{W}{T_{R}}{}\left(n\right){}^{R}{T_{6}}{}{}^{6}{T_{C}}{}\delta p_{C}\end{equation}

A small perturbation in the hand–eye transform ${}^{6}{T_{C}}{}$ contributes

(8d) \begin{equation}\delta p_{W,cal}\approx {}^{W}{T_{R}}{}\left(n\right){}^{R}{T_{6}}{}\delta \left({}^{6}{T_{C}}{}\right)p_{C}\end{equation}

This term captures the effect of camera–robot mounting and calibration imperfections. Uncertainty in the tool-flange transform contributes,

(8e) \begin{equation}\delta p_{W,arm}\approx {}^{W}{T_{R}}{}\left(n\right)\delta \left({}^{R}{T_{6}}{}\right){}^{6}{T_{C}}{}p_{C}\end{equation}

This reflects small variations in the manipulator posture during image acquisition. Given the robot-to-world transform in Eq. (7c), if the RGV stopping position varies by $\delta x_{rail}$ , then,

(8f) \begin{equation}\delta p_{W,rail}\approx \delta \left({}^{W}{T_{R}}{}\left(n\right)\right)p_{R}=\left[\begin{array}{c} \delta x_{rail}\\[3pt] 0\\[3pt] 0\\[3pt] 0 \end{array}\right]\end{equation}

This represents the effect of rail-alignment uncertainty on the final world-frame location. Assuming independence between all uncertainty sources, the individual contributions derived in (8c)–(8f) combine additively to produce the overall first-order perturbation in the world-frame estimate. This formulation highlights the pathways through which sensing and mechanical uncertainties influence $p_{W}$ and identifies the components that require tighter control when integrating the DCR’s perception pipeline with downstream automated harvesting systems.

6. Experiments and results

To validate the performance of the proposed DCR and its cloud-assisted processing architecture, four targeted experiments were conducted. These experiments evaluate (a) the repeatability of fruit localization, (b) cloud latency and robustness, (c) the DCR’s multi-plant data collection performance, and (d) a cost–performance comparison of local versus cloud inference. All experiments were carried out using a 3 m tabletop cultivation setup containing nine strawberry plants spaced 250 mm apart, as shown in Figure 5.

6.1. Strawberry localization repeatability experiment

To evaluate the repeatability of the complete perception-to-localization pipeline, a controlled experiment was conducted using a single ripe strawberry positioned at the first plant. Reliable localization requires the DCR to stop consistently at each plant index and acquire depth images under stable geometric conditions. The RGV motion is regulated by a proximity-sensor-based stopping mechanism and a distance-limited velocity profile consisting of an acceleration phase with a limit of 300 mm/s², a symmetric deceleration phase with the same magnitude, and a final low-speed creep phase of 50 mm executed at 35 mm/s. Given an inter-plant spacing of 250 mm, the RGV follows a triangular velocity profile and transitions directly into the creep phase to prioritize positional stability. This motion profile results in a total travel time of approximately 3.1 s between plant positions and ensures a smooth, repeatable stop at the PGPs, providing stable conditions for image capture.

Figure 5.

Fabrication of the data collector robot (DCR) and overview of the experimental setup used for performance evaluation. (1) Tabletop, (2) strawberries, (3) position guide plate, (4) guide rail, (5) Intel RealSense D435i camera, (6) Mycobot robotic arm, and (7) rail-guided vehicle. The axes $x_{W}$ , $y_{W}$ , and $z_{W}$ denote the world coordinate frame { $W$ }.

In each trial, the DCR moved from its initial position to the first plant, captured RGB-D data, transmitted it to the ACECS for processing, and returned back before beginning the next cycle. This procedure was repeated 50 times under consistent indoor lighting to minimize external variability, and the summary of results is presented in Table III. The reconstructed 3D coordinates extracted from the processed RGB-D images exhibited standard deviations of 0.19 mm (X), 0.30 mm (Y), and 0.19 mm (Z), corresponding to an overall localization RMS error of 0.39 mm. The corresponding 95% confidence intervals for the mean were ±0.089 mm (X), ±0.141 mm (Y), and ±0.089 mm (Z).

Table III.

Summary of localization repeatability experiment results over 50 repeated trials.

Table III. Long description

A table summarizing localization repeatability experiment results over 50 trials. The table has four rows and three columns. The columns are labeled Axis, Mean (mm), Std Dev (mm), and 95% CI (mean). The rows are labeled X, Y, and Z. Row 1: Axis, X; Mean (mm), 94.08; Std Dev (mm), 0.19; 95% CI (mean), 0.089. Row 2: Axis, Y; Mean (mm), 291.85; Std Dev (mm), 0.30; 95% CI (mean), 0.141. Row 3: Axis, Z; Mean (mm), 353.53; Std Dev (mm), 0.19; 95% CI (mean), 0.089.

6.2. Cloud latency and robustness evaluation

To assess the robustness of the cloud-connected perception pipeline under varying network conditions, additional experiments were conducted using the Linux NetEm and Token Bucket Filter (TBF) modules to emulate realistic greenhouse communication impairments. The evaluation examined the end-to-end latency of the cloud-based inference pipeline under four classes of induced network conditions: (i) normal operation with full bandwidth (α), (ii) moderate delay and jitter (β), (iii) severe delay and packet loss (γ), and (iv) extreme delay and elevated packet loss (δ). For each condition, RGB-D images were transmitted 100 times from the DCR to the ACECS server, and the following metrics were recorded: total communication delay $\tau _{net}$ , cloud processing time per image $\tau _{proc}$ , total cycle time $\tau _{\textit{total}}=\tau _{net}+\tau _{proc}$ , and incomplete transmission rate $p_{drop}$ (see Table IV).

Table IV.

Summary of measured end-to-end latency under different network conditions, representing increasing greenhouse network stress from ideal ( $\alpha _{0}$ ) to full-stress ( $\delta _{1}$ ).

Table IV. Long description

A table with 8 rows and 10 columns summarizing measured end-to-end latency under different network conditions. The columns are labeled Delay (s), Jitter (s), Loss (%), Rate (Mbps), Clients, τnet (s), τproc (s), τtotal (s), and Pdrop (%). The rows are labeled with different cases: α0, α1, α2, β1, β2, β3, γ1, γ2, γ3, and δ1. Each row provides specific values for each column, representing different network conditions and their impact on latency. Row 1: α0, 0, 0, 0, 5, 1, 0.19, 0.69, 0.88, 0. Row 2: α1, 0, 0, 0, 3, 1, 0.31, 0.68, 0.99, 0. Row 3: α2, 0, 0, 0, 1, 1, 0.89, 0.69, 1.58, 0. Row 4: β1, 0.2, ±0.05, 0, 3, 1, 1.13, 0.69, 1.82, 0. Row 5: β2, 0.2, ±0.05, 0, 3, 3, 1.18, 1.17, 2.35, 0. Row 6: β3, 0.2, ±0.05, 0, 3, 5, 1.55, 1.99, 3.54, 0. Row 7: γ1, 0.5, ±0.05, 1, 3, 1, 2.46, 0.68, 3.14, 0. Row 8: γ2, 0.5, ±0.05, 3, 3, 3, 2.91, 0.70, 3.61, 0. Row 9: γ3, 0.5, ±0.05, 5, 3, 5, 3.75, 0.99, 4.74, 0. Row 10: δ1, 1.8, ±0.08, 5, 2, 5, 11.20, 0.69, 11.89, 0.

Under ideal network conditions, the average end-to-end cycle time was 0.88 s, consisting of 0.19 s of communication delay and 0.69 s of cloud processing. When bandwidth was reduced to 1 Mbps, network delay increased to 0.89 s, yielding a total cycle time of 1.58 s. More severe impairments produced progressively longer delays: with 0.2 s added latency and light jitter, total cycle time ranged from 1.82 to 3.54 s, depending on packet-loss rate; with 0.5 s added latency, ±0.05 s jitter, and 3% packet loss, total cycle times ranged from 3.14 to 4.74 s. The most extreme δ condition introduced 1.8 s latency, ±0.05 s jitter, and 5% packet loss, resulting in a maximum total cycle time of 11.89 s. Cloud inference time itself remained stable across all trials, varying between 0.68 and 1.99 s, and no image uploads or inference results were dropped across all 900 transmissions.

6.3. Plant data collection experiment

A full-cycle evaluation was performed to assess the DCR’s performance during continuous multi-plant operation. Over 100 complete motions along a nine-plant tabletop, the robot paused at each plant, captured an RGB-D image, and transmitted it to the ACECS for processing. Depending on the test case, individual plants contained between two and six strawberries, and detections were recorded over 20 cycles per case. Figure 6 shows an output RGB image result of a plant downloaded from the ACECS database. Across all cases, the DCR consistently detected the exact number of strawberries present, yielding zero false positives and zero false negatives at a 0.75 confidence threshold (see Table V). All 100 cycles executed without transmission failures, dropped data, or missed detections. The average upload payload was approximately 400 KB per plant; the image-capture stage required about 3.0 s (for FFV and SV), and cloud processing required an additional 0.8 s, resulting in stable and repeatable end-to-end operation throughout the test.

Table V.

Summary of plant data collection and cloud-side processing time results.

Table V. Long description

A table summarizing plant data collection and cloud-side processing time results. The table has 6 rows and 9 columns. The columns are labeled as follows: Case, Number of strawberries per plant, Average detections over 20 cycles, False positives over 20 cycles, False negatives over 20 cycles, Data collection success rate over 100 cycles (percent), Average upload payload (KB), Average image capture time (seconds), and Average cloud processing time (seconds). The rows are labeled with cases A to E. Row-wise data: Row 1: Case A, Number of strawberries per plant 2, Average detections over 20 cycles 2, False positives over 20 cycles 0, False negatives over 20 cycles 0, Data collection success rate over 100 cycles 100, Average upload payload 400, Average image capture time 3.0, Average cloud processing time 0.8. Row 2: Case B, Number of strawberries per plant 3, Average detections over 20 cycles 3, False positives over 20 cycles 0, False negatives over 20 cycles 0, Data collection success rate over 100 cycles 100, Average upload payload 400, Average image capture time 3.0, Average cloud processing time 0.8. Row 3: Case C, Number of strawberries per plant 4, Average detections over 20 cycles 4, False positives over 20 cycles 0, False negatives over 20 cycles 0, Data collection success rate over 100 cycles 100, Average upload payload 400, Average image capture time 3.0, Average cloud processing time 0.8. Row 4: Case D, Number of strawberries per plant 5, Average detections over 20 cycles 5, False positives over 20 cycles 0, False negatives over 20 cycles 0, Data collection success rate over 100 cycles 100, Average upload payload 400, Average image capture time 3.0, Average cloud processing time 0.8. Row 5: Case E, Number of strawberries per plant 6, Average detections over 20 cycles 6, False positives over 20 cycles 0, False negatives over 20 cycles 0, Data collection success rate over 100 cycles 100, Average upload payload 400, Average image capture time 3.0, Average cloud processing time 0.8.

¹Each case was tested over 100 trials.

²The ratio of ripe and unripe strawberries was randomized for each plant to mimic a real-world scenario.

Figure 6.

Example output images of a strawberry plant after cloud-based processing: (a) left-side view, (b) right-side view, and (c) forward-facing view.

6.4. Comparison of local and cloud-based processing

To determine the most practical deployment option for the DCR’s perception pipeline, the YOLOv11s model was benchmarked across three independent local-processing platforms, including a Raspberry Pi 4, a Jetson Orin NX, and a laptop computer, and was compared against the ACECS (see Table VI). The evaluation considers inference time, power consumption, and the total operating cost.

Table VI.

Comparative evaluation of computing platforms used for cloud-based and local inference.

Table VI. Long description

A table comparing computing platforms for cloud-based and local inference. The table has four rows and five columns. The columns are labeled Platform, Processor, Power (W), Per image latency (s), and Operating cost (USD). The rows are labeled with different platforms and their specifications. Row 1: Raspberry Pi 4, Quad-core Cortex-A72, 7, 2.94, 109. Row 2: Laptop Computer, Intel Core i5-9300H + GTX 1660Ti, 125, 0.036, 600. Row 3: Jetson Orin NX (16 GB), 8-core Arm Cortex CPU + 1 024-core NVIDIA Ampere architecture GPU, 30, 0.021, 1,400. Row 4: ACECS (2 vCPU, 4 GB RAM), Intel Xeon Cascade Lake CPU, 12, 0.69, 176 (per year). The table provides a comparative evaluation of different computing platforms based on their processor, power consumption, per image latency, and operating cost.

The Raspberry Pi 4 achieved an inference time of 2.94 s, the highest among the alternatives, while consuming approximately 7 W and costing USD 109. Its latency places it below practical real-time requirements despite low power usage and inexpensive hardware. The Jetson Orin NX achieved the fastest performance, with an inference time of 0.021 s at a power draw of 30 W and a hardware cost of USD 1400. The laptop configuration provided similarly strong performance, with an inference time of 0.036 s, a power consumption of 125 W, and a hardware cost of USD 600. Both devices demonstrate the feasibility of real-time local inference, though with markedly different power and cost profiles. The ACECS processed each image in 0.69 s, requiring only around 400 KB of uploaded data per plant. It operates under an annual subscription of USD 176. Although cloud inference is slower than the Jetson and laptop, it avoids high onboard power demands and eliminates the need for costly embedded hardware.