Introduction
Autonomous machinery on the field is going commercial (Ruckelshausen et al., Reference Ruckelshausen, Biber, Dorna, Gremmes, Klose and Linz2009; Bosch, 2016; Raussendorf, 2016) and thus new development tools are needed to adapt the robots to different events and applications. A robust framework (middleware) and a reliable simulator are requirements for developing and optimizing the software part of the robots firmware. Within the project “elWObot” (Linz et al., Reference Linz, Brunner, Fehrman, Grimsel, Herlitzius and Keicher2014a and 2014b) ROS (Quigley et al., Reference Quigley, Conley, Gerkey, Faust, Foote and Leibs2009) and it’s directly supported 3D-Simulator Gazebo (Koenig and Howard, Reference Koenig and Howard2004) is deployed. The framework ROS can be used in parallel on the machine as firmware and on the PC for the simulation. This makes the integration of the software quite simple. There are many other frameworks and simulators on the market, some of them commercial, some of them Open Source, like ROS. The free support of a huge community and the deployment of ROS in big companies like Bosch (Bosch, 2016) or BMW (Aeberhard et al., Reference Aeberhard, Kühbeck, Seidl, Friedl, Thomas and Scheickl2015) led to the use of this combination of middleware and 3D-Simulation environment. Another important point is the ability of Gazebo to simulate different kinds of sensors.
Gil et al. (Reference Gil, Escolà, Rosell, Planas and Val2007) showed that PPP can be reduced by 57%, using ultrasonic sensors to detect the LWA. This can be reached, because the LWA of the trees in an orchard or vineyard row can vary extremely in different heights. Since the use of LIDAR is more simple and inexpensive, and has a much higher spatial resolution than ultrasonic sensors, the laser scanner is the first choice for LWA detection. There are many works (Rosell et al., Reference Rosell, Sanz, Llorens, Arnó, Escolà and Ribes-Dasi2009; Nielsen et al., Reference Nielsen, Slaughter, Gliever and Upadhyaya2012; Rosell and Sanz, Reference Rosell and Sanz2012; Rinaldi et al., Reference Rinaldi, Llorens and Gil2013; Sarri et al., Reference Sarri, Lisci, Rimediotti and Vieri2013; Tagarakis et al., Reference Tagarakis, Liakos, Chatzinikos, Koundouras, Fountas and Gemtos2013; Bargoti et al., Reference Bargoti, Underwood, Nieto and Sukkarieh2015) related to geometrical characterisation of orchard canopy by using laser scanners. Most of them are calculating the sensor data offline, to generate an application map and to get a more precise information of tree biomass, growth estimation, yield prediction, water consumption estimation and health assessment (Rosell and Sanz, Reference Rosell and Sanz2012). There are also online solutions described, deploying commercial sprayer which can adjust the dose rate in real time. (Gil et al., Reference Gil, Escolà, Rosell, Planas and Val2007) utilized ultrasonic for sensing LWA, which is much more inaccurate than LIDAR. (Wellington et al., Reference Wellington, Campoy, Khot and Ehsani2012) utilized LIDAR for sensing LWA, but the “axial airblast sprayer” they used is not able to apply the optimal dose of PPP exactly to the calculated position. With the spray application and the method described in this paper, it is possible to bring out the PPP with the correct flow-rate and air-pressure to the right position (Fig. 1).
Concept of a robotic carrier with the spray application (left). Schematic rear view of the carrier and the spray application in an orchard (right).
A new approach is to use a 3D-Simulation environment for the development and optimization of new algorithm and control processes (e.g. state machines), which can be directly adapted to the hardware with several spray applications in viticulture or horticulture maintenances.
Technologies
Autonomous electrical driven platform “elWObot”
The drive concept of elWObot (Fig. 2 left) is a combination of a diesel engine, a power generator and four independent electrical driven wheels. This enables a very precise operation of the vehicle and allows several steering modes, like front wheel steering, rear wheel steering, four wheels steering, crab mode steering and turning on a point. Each wheel has an installed power of about 7 kW. This is needed to carry an overall load of up to 3.5t (robot+spray application+1000 litre of chemicals). The base control of the platform is integrated in the PLC (Programmable Logical Controller) of the 8 frequency converters (4 x wheels+4 x steering), and can be operated by an industrial radio-control connected to a CAN bus interface. The main navigation concept is based on two sensors, a 2D laser scanner and a 3D laser scanner. The horizontally mounted 2D scanner (Fig. 2 right, a.) with a maximum range of 80 m is responsible for recognizing the row, the end of the row and obstacles in the sight of its scan level. The 3D scanner (Fig. 2 right, b.) has a shorter range (about 9 m), scans horizontal and vertical distances and is installed to detect obstacles in the front of the robot. An additional low cost GPS and an IMU (Inertial Measurement Unit) is fused with the odometry data and generates position information for the U-turn at the end of the row. All sensors are connected to the main navigation unit via an Ethernet connection. Furthermore the navigation unit exchanges data with the PLC of the robot as well as with the spray application using the CAN bus interface.
elWObot - Electrical driven service robot for maintenance tasks in orchards and vineyards equipped with the spray application (left). Sensor fixture: a.) Horizontal laser scanner b.) 3D laser scanner c.) Vertical laser scanner d.) Webcam (right).
Selective electrical driven sprayer
To achieve a high energetic efficiency in combination with the possibility to take into account not only (2D) LWA but also (3D) TRV (Tree Row Volume) (Sutton and Unrath, Reference Sutton and Unrath1984) measurements for precise PPP application, a completely electrically driven sprayer module is developed. Two electric pumps provide arbitrarily adjustable flow rate and pressure combinations. Eight nozzles (four per side) are electrically switchable and one electrical blower per nozzle provides an individually adjustable air stream (Fig. 1 and Fig. 2 left). The concept of using electrical drives, electrically driven ball valves and electrically switchable nozzles in combination with the CAN bus capable PLC, allows sensitive control with respect to five domains:
-
1. General process control: depending on the signal from the navigation module the sprayer can adapt its settings to the current general situation, like activating the induction hopper for mixing PPP with water, driving to the field, moving or turning in headland, spraying, pause or emergency stop and internal cleaning of the whole system.
-
2. While spraying, flow rate and system nozzle pressure can be adapted to the required measure deduced from current LWA.
-
3. Blower air stream and also nozzle pressure can be adapted to be adequate to the current TRV (3rd dimension). So leaves within a very thick canopy may be reached by a stronger air stream, but energy isn’t wasted using a reduced air stream in thinner parts of the canopy. This also helps reducing drift as water drops aren’t shot through beyond the canopy.
-
4. The selectively switchable nozzles allow a very quick reaction to gaps within different height of the canopy. Depending on the canopy structure this can significantly reduce PPP consumption and drift.
-
5. If canopy structure or local plant health state require particularly intensive PPP application the sprayer module controller can send a CAN bus signal to the elWObot carrier vehicle to reduce forward velocity.
Basic technology components of the sprayer are 8 electrical blower with 0.5 kW, 2 electrical pumps with 2 kW (max. 4 MPa) and 8 switchable nozzles. Each blower and each pump is controlled by its own frequency converter.
Methods
3D-Simulation environment
Gazebo is an Open Source 3D-Simulation environment build on the physical engine ODE (Open Dynamic Engine) for simulating rigid body dynamics. The 3D rendering of the scenes is performed by OGRE (Object-Oriented Graphics Rendering engine), which supports the hardware acceleration of most graphic cards. ROS provides an interface to Gazebo and a XML description language (URDF) to describe the models in the simulated world. The URDF model is also used by ROS to transform all points and sensor data to the right position and orientation. Rviz is another important ROS tool. It gives the ability to visualize many kinds of data, like sensor data, position data, maps, markers etc. The source of the data can be provided from a real sensor as well as from a simulated sensor. This altogether forms a practical and helpful development environment for new robotic applications.
Principles of the “Spraycontrol”
The electrical driven sprayer is controlling the dose of PPP by varying the flow-rate and the air-pressure. Thus, there are parameters needed for every spray region individually. In this case there are 8 spray regions roughly defined in a rectangle with “sampleDistance”, defining the distance in the drive direction and “sprayerLevel”, defining the heights of the LWA (Fig. 3 right). The parameter “row_width” is needed, because the scanner is also looking into the next row, and this data should not be included. The markers in Fig. 3 (left) do not define the real application area of PPP, it is only a rough representation used for showing if the sprayer in this region is switched on or off.
Spray markers in Rviz (left). Setup example for LWA regions (right).
To generate data for the 8 spray regions a vertical installed laser scanner in the front of elWObot is deployed (Fig. 2 right, c. and Fig. 4 right, c.). The distance range of the scanner is between 0,02 m and 50 m, the angular range is 270 degrees with a resolution of 0.5 degrees. The sample rate amounts 50 Hz (1/0.02 s) and limits the “sampleDistance”, which depends also on the robots velocity. A minimum of one scan have to be within a “sampleDistance”. For example, if the velocity of elWObot is 1.6 m/s (6 km/h) and the sample distance is defined with 0.15 m, the robot spends 0.094 s, thus the scanner samples 4 times (Fig. 5).
elWObot in the simulation environment Gazebo (left). Sensor fixture (simulation): a.) Horizontal laser scanner b.) 3D laser scanner c.) Vertical laser scanner d.) Webcam (right).
One laser scan in a simulated row of trees (left), Assembled laser scans (point cloud) while driving forward in a simulated row of trees (right).
The position of the scan is different to the position of the sprayer. So, the switch position and the position of the spray levels must be calculated by a transformation, which can directly obtained from the URDF model (Fig. 6 left).
Transformation of the scan data (left) Setup file for “Spraycontrol” algorithm (right).
Furthermore, a setup file gives the ability to adapt parameters for different application scenarios (Fig. 6 right). A simple decision algorithm used for the first tests, checks if there is any hit in the spray region and if the distance of this hit is smaller than the “row_width”. If this is true the sprayer will be switched on, otherwise it will be switched off.
Simulation test scenarios
The world for the first test in the simulation was generated with primitive objects, like a wall and a cube (Fig. 7). This was helpful to verify, if the “Spraycontrol” triggers at the right position.
World in Gazebo with primitive objects (left) Output of “Spraycontrol” algorithm in Rviz (right).
The world in Fig. 8 has been generated to develop the autonomous navigation of elWObot (Linz et al., Reference Linz, Hertzberg, Ruckelshausen and Wunder2014b). There are four rows with some missing trees. This is based on a real scenario, where old or sick trees have been removed. The simulation with elWObot was in a fully autonomous mode. The sprayer stops spraying at the end of the row, and starts again at entering the next row (Fig. 8 right).
World in Gazebo with tree rows and removed trees (left) Output of “Spraycontrol” algorithm in Rviz (right).
Field tests without the spray application
Next step is to verify the software with real sensor data from an orchard. ROS supplies recording data in a so called “rosbag”. This data includes timestamps and can be played back to the system. Fig. 9 shows a test field with young trees on the left side and older trees on the right side, thus the sprayer should bring out less PPP on the left side. By running the software with the collected data, the result is looking promising (Fig. 9 right).
Test field with younger trees on the left side and older trees on the right side (left) Output of “Spraycontrol” algorithm in Rviz (right).
Conclusions
The modelling environment combination of ROS and Gazebo represents a helpful toolchain for developing new robot concepts including the simulation of the most important sensors. Simulation can be done before any hardware is existing. Furthermore real sensor data from the field can be used to verify and optimize the software. The same software can afterwards directly be implemented on the robot. The robot sprayer combination gives some advantages over the classical tractor trailer solution. The sprayer and the robot can interact and control each other. For example, an obstacle stops the robot and the robot turns off the sprayer. The other way round, the sprayer detects no leave wall, stops spraying, and give the information to the robot to increase the velocity.
First field tests with the robot sprayer combination have been conducted on an orchard near Dresden. Water was used for spraying and additionally one bright light for each sprayer indicated if the nozzle was switched on or off. The results showed, that the software implementation from the simulation to the hardware (elWObot) worked as expected. The sprayers switched off at the gaps in the row, at the end of the row and in the U-Turn. Next step will be to verify the quality of the spray result and to compare it with the simulation data. This will help to optimize the model and the method.
Acknowledgements
The project elWObot is supported by funds of the Federal Ministry of Food and Agriculture (BMEL) based on a decision of the Parliament of the Federal Republic of Germany via the Federal Office for Agriculture and Food (BLE) under the innovation support program.
