Hard- and software of the auto-piloted aeroplanes

Our auto-piloted UAV, the aeroplane Mini-Horus®, was specifically designed as a photodrone to carry two cameras during a flight time of typically 45 min. It is powered by an electric motor using high energy density lithium batteries that are charged on the ground with energy from a 60 W solar panel. The Mini-Horus was built from low-cost aeromodelling materials including polyurethane (PU)-foam, balsa and glass fibre-reinforced plastic (Figure 1). Its net (empty) mass was minimized for maximum payload and maximum flight time. With a wing span of 1.74 m, a length of 1.25 m, a maximum payload of 500 g, a flight mass incl. payload of 2070 g, a 200 W electric motor (Hacker A30-14L, Hacker Motor GmbH, Ergolding/Landshut, Germany) and a 11.1 V – 4000 mAh Li polymer battery, the UAV reaches a cruising speed of up to 11 m s⁻¹ (40 km h⁻¹). The Mini-Horus is manually controlled during hand launch and landing using an ordinary RC gear in the 35 MHz band (Robbe/Futaba 35 MHz; robbe Modellsport GmbH & Co. KG, Grebenhain, Germany). Once airborne, control is taken over by the Paparazzi autopilot (Ecole Nationale de l'Aviation Civile (ENAC), Toulouse, France; http://paparazzi.enac.fr/wiki/Main_Page; Figure 2, Hattenberger et al., Reference Hattenberger, Bronz and Gorraz2014; Mayer, Reference Mayer2011). The software is open source and can be adapted by the user for specific tasks.

Figure 1.

The Mini-Horus® photodrone (ceramic toils measure 0.24 × 0.24 m).

Figure 2.

Typical Paparazzi controlled flight path over a 800 × 800 m² area. H = home point with stand-by circle, dots = photo waypoints.

The autopilot hardware consists of a powerful ARM7 processor, a GPS receiver for position control (μ-blox LEA-5H with 19 x 19 mm² patch antenna offering a typical position precision of 3.5–4 m at seven satellites), three sets of infrared sensors for attitude control and a 2.4 GHz telemetry and remote control link to the ground station computer. The flight tasks, written in C, were prepared in advance and loaded to the ARM7 memory prior to each flight. The continuous data link from the ground station allows to intervene at any moment to modify the flight as well as to adapt all parameters of the on-board position and attitude control loops to changing atmospheric conditions.

Basic parameters of a typical flight task to be defined for aerial photography are the outline of the rectangle or trapeze to scan, the distance between two parallel flight legs (e.g. 100 m), the scan direction (north–south or west–east, even tilted), the distance between two photos (e.g. 70 m), the flight altitude (e.g. 300 m) and the flight speed (e.g. 11 m s⁻¹). The distance between two legs and two photos as well as the flight altitude depend on the desired picture resolution, the focal length of the cameras and the overlap between adjacent photos. An overlap of 40 to 50% proved to be an absolute minimum to properly stitch the images together and to compensate for roll and pitch movements of the plane, but 60–70% may be recommendable (see Hardware Requirements).

Camera system

The plane was equipped with two compact digital cameras (Figure 3), one for the visible (Casio EX-S12) and one for the near-infrared spectrum (Fuji F31fd, with the near-IR blocker in front of the image sensor inside the camera removed). These cameras were chosen as a compromise between size, mass and optical quality. Essential criterion for camera choice was a high sensitivity, low noise image sensor to allow short exposure times (e.g. 1/1000 s) at ISO settings up to 400. High pixel count was of less importance. Focal length has proven best at 45 mm (referenced to 35 mm cameras). Both cameras were triggered simultaneously by the autopilot. Spatial resolution was roughly 10 cm from 300 m flight altitude or 5 cm from 150 m.

Figure 3.

Accommodation of two digital cameras in the Mini-Horus: The Casio EX-S12 (left) was used to record image data at visible wavelengths and the modified Fuji F31fd (right) for the near IR spectrum.

For near-infrared photography, a 720 nm near-IR filter was placed in front of the lens. Due to the high IR sensitivity of the Fuji sensor, a spectrum from 720 to approximately 1000 nm could be captured with the same camera speed and aperture settings as for the visible spectrum.

Image acquisition

The system was tested in two environments. Initial pre-testing of the whole system was performed in a hilly, temperate agro-ecosystem in Werleshausen, central Germany (51°19′ N and 9°55′ E) during the winter of 2012. The contiguous area covered in 90 images per scan of each type was 30 ha. Subsequent tests was conducted in a flat, agro-pastoral landscape in SW-Madagascar (24°05′ S and 43°42′ E) during peak vegetation growth in March 2012. There, aerial photographs were taken of the villages Andremba (900 ha), Miarintsoa (1700 ha) and Efoetse (1400 ha). A total of 4000 ha, distributed over the three sites, was covered resulting in 8500 images.

Image stitching

For mosaicking the RGB and near-IR images of each flight covering 1.5 x 1.5 km, we used the software applications developed within the ANDROMEDA project (Application of Drone-Based Aerial Images – Mosaicking, Geocoding and Data Analysis; Böhm et al., Reference Böhm, Böhm, Chmara, Flügel, Krüger, Neumann, Reinhold, Sagischewski, Selsam, Vörsmann and Wilkens2008; Reinhold et al., Reference Reinhold, Selsam, Matejka and Jürgens2008). Part of the project results was a software module GeoLink (to be obtained upon request from GDS GmbH, Talstrasse 84, D-07743 Jena) dedicated to mosaic and stitch a large amount of airborne images to a seamless and geo-rectified product such as an image map. A single flight of a UAV system may produce hundreds or thousands of images, which need to be arranged along the flight path, mosaicked and geo-rectified. As time constraints prevent to do this tedious job manually, a fully automated work flow was developed to minimize internal distortions and projection errors of the resulting image mosaic. To achieve this, we applied a 3D model comprising the position of the aircraft (UAV), the flight path, the orientation of the aircraft (roll, pitch, yaw), the camera model, lens distortion and the control points calculated from the different images. Combining all of this information by ray tracing, the model parameters could be transformed to map co-ordinates and a coarse elevation model (Böhm et al., Reference Böhm, Böhm, Buschmann, Chmara, Flügel, Krüger, Neumann, Püls, Reinhold, Sagischewski, Selsam, Vörsmann and Wilkens2010).

The whole procedure, may be summarized as ‘automated photogrammetry’ whereby the challenge was the optimization procedure. Recorded values of the flight parameters (position, image exposition time, altitude, flight direction, pitch and roll angles) were far beyond the precision needed to calculate the image mosaic with the required geometric accuracy, using the recorded parameters directly. IMU, GYRO's, GPS and others share measurement inaccuracies that sum up to 10 to 20 m error for object positions on the ground while the swath width of the flight path is around 180 m. Consequently, the core objective of the model development was to calculate which combinations of corrected real flight parameters are likely to produce the most accurate image result. Registered flight parameters restrict the possible values of the calculated ones, but only occasionally meet them exactly. Therefore, a combination of Monte Carlo and gradient descent method was used to harmonize the registered flight parameters (Wilkens et al., Reference Wilkens, Krüger, Martin, Scholtz, Vörsmann, Reinhold and Selsam2010). The software for stitching and mosaicking was operated on a standard laptop PC. The typical result of a continuous flight with 228 images was processed in about 1.3 hr.

Image analysis

The final objective of our tests in SW-Madagascar was to obtain a reliable estimate for the plant cover and biomass of the locally widespread samata tree (Euphorbia stenoclada Baill.) and the perennial crop species cassava (Manihot esculenta L. Crantz). For delineation and classification, we used a two-step approach following the process chain of the ILMSimage software. ILMSimage relies on the concept of object-oriented image analysis, that is image elements (pixels) of a data set are not treated separately, but in the context of neighbouring pixels which are spectrally similar. In the first step, image objects were delineated and object specific attributes were assigned. These attributes allow a supervised classification of specified image objects.

Hardware requirements

The above-mentioned software package GeoLink is not restricted for use with a specific UAV, but recording of flight parameters and flight planning have to meet the following requirements to obtain quality results.

Standard flight plans cover an area by parallel flight paths to guarantee sufficient overlap between different images along track and cross track (Figures 2 and 4). Small and lightweight UAV's must be able to control roll and pitch angels actively within a few degrees, otherwise image overlap might become insufficient to create control points. Control points can only be detected if the viewing angels for the same object between adjacent images differ less than ±20 degrees (Lowe, Reference Lowe2004). With standard camera lenses, this limit is reached at the fringe of the images even if they are taken in nadir direction. Every deviation of the aircraft in pitch or roll reduces the area usable to match images.

Figure 4.

Programmed flight tracks of the aeroplane (dotted yellow lines) and projected real image centre (blue lines) for a flight recorded near Andremba in SW-Madagascar (flight no. 24; 26.03.2012) indicating the displacement of flight positions due to wind turbulence. The flight comprised 135 RGB and near-IR images each.

Latitude, longitude, altitude and attitude of the aircraft at the very moment of activation of the cameras were recorded by the autopilot. The delay from activation to the exact image acquisition time was a major challenge during the development of work flow for image mosaicking. Accuracy is reduced by internal delays in the camera, especially the unpredictable time used by the autofocus. Because of camera weight and IR sensitivity, it was not possible to use camera models with adjustable autofocus time. An important selection feature for an appropriate camera model for UAV-based image acquisition should therefore be the possibility to control the camera properties such that they cannot be overridden by internal processes (software patch). Total costs of our equipment comprising UAV, transmitter, motor, battery, flight computer and photo cameras are about 2000 €.

Extraction of data for a Digital Elevation Model (DEM)

To avoid image distortions due to elevation differences, the image projection uses elevation data generated by the optimization procedure. During the procedure of image stitching, the residual errors of the stitching point positions can only be corrected if the distance between camera and surface is introduced in the calculation. This yields a course distance between camera and surface. The difference to the GPS-measured altitude yields the surface elevation. Even if of limited precision, they can be summarized for elevation and slope of each image individually. This information is used for orthogonal projection and in the absence of a high-resolution DEM provides an elevation model related to the density of image capture. Available global data are of limited help for this purpose. While the altitude of the aircraft is known from GPS recordings during the flight, the optimization procedure provides the distance between control points and aircraft. The result is very sensitive to the accuracy of focal length of the camera and control point replacement is transmitted progressively to the results.

For the flight terrain in Werleshausen, 88 elevation points were created from residual errors of stitching point positions (Figure 5). The spatial distance of the derived elevation points was between 60 and 100 m. The points were unequally distributed over the flight terrain due to the curvature of the image centre projection (Figure 4). The elevation points were interpolated using an inverse distance algorithm to create a coarse elevation model that reflects the position and elevation of image objects that were recognized by control points. We compared the result with approximately the same amount of measured points to obtain information about the usability of the model (below).

Figure 5.

Results of the pre-test in Werleshausen, Germany: geo-rectified Red-Green-Blue (A) and Infrared (B) image and the generated Digital Elevation Model (C).

Accuracy assessment

To assess the accuracy of the DEM generated for Werleshausen from the recorded altitude data of the GPS receiver in the Mini-Horus photodrone, we used 50 randomly selected ground control points (GCPs). The altitude measurements were conducted with a handheld Trimble® GeoXT™ from the GeoExplorer® 2008 series using real-time correction with integrated SBAS (Satellite Based Augmentation System) resulting in sub-meter accuracy. For final accuracy assessment, the altitude GPS measurements of the Trimble device were compared with the raster DEM values extracted for each GCP within a GIS.

To validate the image stitching and automated geo-referencing results of the aerial photographs of Madagascar, we used orthorectified Pléiades satellite images, which were acquired two weeks after the aerial photograph mission in April 2012 and covered the entire study region. These image products with a resolution of 0.5 m were corrected from acquisition and terrain off-nadir effects and offered the best location accuracy available. Based on these images, 30 reference points were defined for each study site (Andremba, Miarintsoa and Efoetse) at easily recognizable locations such as buildings, field limits, pathways and trees. To estimate location errors we measured the Euclidean distance (m) of the reference points in the Pléiades image to the corresponding point in the aerial photographs.