Introduction
Turfgrasses are ubiquitous forms of vegetation cover in urban landscapes, including athletic, commercial and residential lawns, golf courses, roadsides, and parks (Balogh and Walker Reference Balogh and Walker1992). Milesi et al. (Reference Milesi, Elvidge, Dietz, Tuttle, Nemani and Running2005) estimated that the total turfgrass surface area in the United States is 163,812 km2, which accounts for approximately 1.9% of the total land of the continental United States. Weeds are a serious issue in turfgrass management, competing for environmental resources such as sunlight, nutrients, and water and reducing turfgrass aesthetics and functionality. The most common approach for controlling weeds is applying herbicides broadcast over the turfgrass, thus including areas without weeds. This is a source of concern, because many herbicides used in turfgrasses are restricted-use pesticides that can potentially pollute the environment and pose a risk to human health (Pimentel Reference Pimentel2005). For example, atrazine, a widely used restricted-use herbicide in warm-season turfgrasses (McCarty Reference McCarty1996; Yu and McCullough Reference Yu and McCullough2016), is one of the most common pesticides found in groundwater (EPA 2018; Pimentel Reference Pimentel2005). Spot spraying is considered an efficient strategy to reduce herbicide input and weed control costs, because weeds generally occur in nonuniform and patchy distributions (Farooq et al. Reference Farooq, Zaman, Boyd and White2019; Zaman et al. Reference Zaman, Esau, Schumann, Percival, Chang, Read and Farooque2011). Nevertheless, manual spot spraying in large turfgrass areas is very time-consuming and expensive.
Accurate weed detection is a prerequisite for automation of weed control with smart sprayers (Fennimore et al. Reference Fennimore, Slaughter, Siemens, Leon and Saber2016; Zaman et al. Reference Zaman, Esau, Schumann, Percival, Chang, Read and Farooque2011). Traditional 2D machine vision technology has been widely studied, but it is unreliable in the case of high weed densities and nonuniform crop stands (Fennimore et al. Reference Fennimore, Slaughter, Siemens, Leon and Saber2016). Using hyperspectral imaging for weed detection has been extensively studied (Fennimore et al. Reference Fennimore, Slaughter, Siemens, Leon and Saber2016; Hamada et al. Reference Hamada, Stow, Coulter, Jafolla and Hendricks2007; Lass et al. Reference Lass, Thill, Shafii and Prather2002), but this technology is costlier compared with using a conventional digital imaging camera (Fennimore et al. Reference Fennimore, Slaughter, Siemens, Leon and Saber2016). Non-imaging techniques for ground-based weed detection include fluorescence, ultrasonic, LiDAR sensors, and spectral reflectance, but their commercial adoption is low (Fennimore et al. Reference Fennimore, Slaughter, Siemens, Leon and Saber2016; Fernández-Quintanilla et al. Reference Fernández-Quintanilla, Peña, Andújar, Dorado, Ribeiro and López-Granados2018; Wang et al. Reference Wang, Zhang and Wei2019). A few smart weed controllers such as AmaSpot® (Amazonen-WerkeH. Dreyer, Hasbergen, Germany), Weed-it® (Rometron, Steenderen, Netherlands), and See & Spray® (Blue River Technology, Sunnyvale, CA, USA) have recently been developed. However, none of these inventions are designed for selective spot-herbicide treatments in turfgrass systems.
Machine learning technologies have made tremendous advances in recent years (Jordan and Mitchell Reference Jordan and Mitchell2015; LeCun et al. Reference LeCun, Bengio and Hinton2015). Among the machine learning techniques, deep learning (DL) has been immensely successful in various scientific applications, including but not limited to object detection (LeCun et al. Reference LeCun, Bengio and Hinton2015), natural language processing (Collobert and Weston Reference Collobert and Weston2008; Collobert et al. Reference Collobert, Weston, Bottou, Karlen, Kavukcuoglu and Kuksa2011), and speech recognition (Graves et al. Reference Graves, Mohamed and Hinton2013; Hinton et al. Reference Hinton, Deng, Yu, Dahl, Mohamed, Jaitly, Senior, Vanhoucke, Nguyen and Sainath2012). DL models are built with multilayer artificial neural networks and can be trained through supervised or unsupervised learning (Schmidhuber Reference Schmidhuber2015). In supervised DL models, the network coefficients are fitted to a data set containing ground-truth target inputs to generate predicted target outputs (LeCun et al. Reference LeCun, Bengio and Hinton2015; Schmidhuber Reference Schmidhuber2015). DL and particularly deep convolutional neural networks (DCNN) demonstrate extraordinary abilities to extract complex features from images (LeCun et al. Reference LeCun, Bengio and Hinton2015). DCNN is currently being used in various machine vision applications such as biological image segmentation (Havaei et al. Reference Havaei, Davy, Warde-Farley, Biard, Courville, Bengio, Pal, Jodoin and Larochelle2017), facial recognition (LeCun et al. Reference LeCun, Bengio and Hinton2015), and traffic sign recognition for self-driving cars (Schmidhuber Reference Schmidhuber2015).
In agriculture, recent research demonstrated the extraordinary ability of DCNN in identifying and phenotyping plant stresses (Ferentinos Reference Ferentinos2018; Ghosal et al. Reference Ghosal, Blystone, Singh, Ganapathysubramanian, Singh and Sarkar2018; Mohanty et al. Reference Mohanty, Hughes and Salathé2016; Singh et al. Reference Singh, Ganapathysubramanian, Sarkar and Singh2018). For example, DCNN was effective at detecting multiple diseases under various image conditions, including complex background and varying illumination (Singh et al. Reference Singh, Ganapathysubramanian, Sarkar and Singh2018). Ghosal et al. (Reference Ghosal, Blystone, Singh, Ganapathysubramanian, Singh and Sarkar2018) reported that DCNN reliably identified several abiotic (chemical injury and nutrient deficiency) and biotic (bacterial and fungal diseases) stresses in soybean [Glycine max (L.) Merr.]. DCNN performed well on many other machine vision tasks such as food grain quality evaluation (Vithu and Moses Reference Vithu and Moses2016), cotton (Gossypium hirsutum L.) bloom detection (Xu et al. Reference Xu, Li, Paterson, Jiang, Sun and Robertson2018), maize (Zea mays L.) tassel counting (Kurtulmuş and Kavdir Reference Kurtulmuş and Kavdir2014), and plant species identification (Dyrmann et al. Reference Dyrmann, Karstoft and Midtiby2016; Larese and Granitto Reference Larese and Granitto2016; Larese et al. Reference Larese, Namías, Craviotto, Arango, Gallo and Granitto2014).
There has been limited research evaluating the feasibility of using DCNN as a method of real-time weed detection in the machine vision subsystem of smart sprayers. Sharpe et al. (Reference Sharpe, Schumann and Boyd2019) reported that DetectNet, an object-detection DCNN, was effective at detecting Carolina geranium (Geranium carolinianum L.) in plastic-mulched strawberry (Fragaria × ananassa D.). Recently, we reported that DCNN achieved excellent performance in the detection of multiple broadleaf weed species in bermudagrass [Cynodon dactylon (L.) Pers.], bahiagrass (Paspalum notatum Flueggé), and perennial ryegrass (Lolium perenne L. ssp. perenne) turfgrasses. Previous studies demonstrated that DCNN can provide a promising avenue for precision control of various broadleaf weeds in actively growing turfgrasses or various weeds in dormant bermudagrass (Yu et al. Reference Yu, Cao, Sharpe, Schumann and Boyd2019a, Reference Yu, Sharpe, Schumann and Boyd2019b, Reference Yu, Sharpe, Schumann and Boyd2019c). However, the feasibility of detection of grassy weeds in actively growing turfgrass using DCNN is unknown.
Doveweed [Murdannia nudiflora (L.) Brenan] is an aggressive summer annual weed found in the southern United States. It is a herbaceous monocot weed species in the Commelinaceae family and has a grass-like appearance with narrow fleshy leaves, parallel veins, and creeping stems (USDA-NRCS 2019). Crabgrass species (Digitaria spp.) are summer annual grass weeds found throughout the tropical, subtropical, and temperate climates of the world, while dallisgrass (Paspalum dilatatum Poir.) and tropical signalgrass [Urochloa distachya (L.) T.Q. Nguyen] are perennial grass weeds found in tropical and subtropical climates (USDA-NRCS 2019). Weed detection with DCNN is based on plant color, pattern, and shape features (Chaki et al. Reference Chaki, Parekh and Bhattacharya2015; Grinblat et al. Reference Grinblat, Uzal, Larese and Granitto2016; Wu et al. Reference Wu, Zhou and Wang2006). Therefore, the detection of grass or grass-like weeds in turfgrasses is a challenging task because of the similarity in plant morphology. The objectives of this research were to evaluate (1) the feasibility of using DCNN for detection of smooth crabgrass [Digitaria ischaemum (Schreb.) Schreb. ex Muhl], M. nudiflora, P. dilatatum, and U. distachya growing in bermudagrass; (2) the detection of multiple grass weed species using a single DCNN model; and (3) the effect of weed density on the performance of DCNN for grassy weed detection.
Materials and Methods
Overview
Previous researchers performed pixel-wise semantic segmentation for separating crops, weeds, and background on the RGB images for improving the performances of neural networks (Milioto et al. Reference Milioto, Lottes and Stachniss2018). However, segmentation did not always improve the performance of DCNN for weed detection (Dyrmann et al. Reference Dyrmann, Karstoft and Midtiby2016). In addition, image segmentation demands more graphics processing unit (GPU) power compared with the simple localization method for weed detection. Furthermore, previous researchers documented the high accuracy of weed detection without implementing image segmentation (Haug et al. Reference Haug, Michaels, Biber and Ostermann2014; Yu et al. Reference Yu, Sharpe, Schumann and Boyd2019b, Reference Yu, Sharpe, Schumann and Boyd2019c). Therefore, we chose not to perform image segmentation. DCNN architectures are designed for either image classification or object detection (Schmidhuber Reference Schmidhuber2015). The training of object-detection DCNN architectures, such as DetectNet (Tao et al. Reference Tao, Barker and Sarathy2016), YOLO (Redmon et al. Reference Redmon, Divvala, Girshick and Farhadi2016), and SSD (Liu et al. Reference Liu, Anguelov, Erhan, Szegedy, Reed, Fu and Berg2016), involves the drawing of bounding boxes within the training images for individual weeds. However, many grass weeds are small and present in large patches in turfgrasses, and thus the drawing of bounding boxes onto individual weeds is overly complex. The present work explored the feasibility of using image classification DCNN architectures, including AlexNet (Krizhevsky et al. Reference Krizhevsky, Sutskever and Hinton2012), GoogLeNet (Szegedy et al. Reference Szegedy, Liu, Jia, Sermanet, Reed, Anguelov, Erhan, Vanhoucke and Rabinovich2015), and VGGNet (Simonyan and Zisserman Reference Simonyan and Zisserman2014) for detection of grass or grass-like weeds in bermudagrass. AlexNet contained a total of eight layers, including five convolutional layers and three fully connected layers (Krizhevsky et al. Reference Krizhevsky, Sutskever and Hinton2012). GoogLeNet consists of 22 convolutional layers in the form of inception modules. GoogLeNet is designed with small convolutions to reduce the computational cost (Szegedy et al. Reference Szegedy, Liu, Jia, Sermanet, Reed, Anguelov, Erhan, Vanhoucke and Rabinovich2015). VGGNet consists of 16 weight layers with a stack of convolutional layers followed by max-poling layers (Simonyan and Zisserman Reference Simonyan and Zisserman2014).
Image Acquisition
Images of D. ischaemum, M. nudiflora, and U. distachya were taken at multiple times from May to November 2018 and images of P. dilatatum were taken in October 2018 using a digital camera (DSC-HX1, SONY® Cyber-Shot Digital Still Camera, SONY, Minato, Tokyo, Japan) at a resolution of 1,920 × 1,080 pixels.
Training images of D. ischaemum, M. nudiflora, and U. distachya were taken at multiple golf courses in Bradenton, FL (27.49°N, 82.47°W), Tampa, FL (27.95°N, 82.45°W), Riverview, FL (27.86°N, 82.32°W), and Sun City, FL (27.71°N, 82.35°W). Images of D. ischaemum, M. nudiflora, and U. distachya taken at multiple institutional lawns and golf courses in Lakeland, FL (28.03°N, 81.94°W) were used in the testing data set (TD) 1. Images of D. ischaemum, M. nudiflora, and U. distachya taken at multiple institutional lawns and golf courses in Homestead, FL (25.46°N, 80.47°W) were used in TD 2. Training images of P. dilatatum were taken at the University of Georgia Griffin Campus in Griffin, GA (33.26°N, 84.28°W). Images of P. dilatatum taken at multiple sport fields, institutional lawns, and golf courses in Peachtree City, GA (33.39°N, 84.59°W) were used in TD 1. Images of P. dilatatum taken at multiple institutional lawns and golf courses in Alpharetta, GA (34.07°N, 84.29°W) and Johns Creek, GA (34.02°N, 84.19°W) were used in TD 2.
Images were taken at a ground sampling distance of 0.05 cm pixel−1. Images were taken from 9:00 AM to 5:00 PM under various illumination conditions, including under tree shadows and on cloudy, partially cloudy, and sunny days. Images were taken with a wide range of bermudagrass surface conditions and mowing heights (Figure 1).
The training and testing images of multiple-species neural networks. (A) Images of Digitaria ischaemum, Murdannia nudiflora, Paspalum dilatatum, and Urochloa distachya were at different weed densities. (B) Images of bermudagrass turfgrasses were taken at different management regimes with varying turfgrass surface qualities.
Training and Testing
DCNN models were trained using a data set containing either a single weed species (single-species neural network) or multiple weed species (multiple-species neural network). Multiple- species neural networks were trained because we were interested in using a single DCNN model to detect multiple grassy weeds growing in bermudagrass. Images containing a single weed species per image were selected and used for training multiple-species neural networks. Images were cropped into subimages with a resolution of 426 × 240 pixels using Irfanview (v. 5.50, Irfan Skijan, Jajce, Bosnia).
For training single-species neural networks, the D. ischaemum training data set contained 6,000 positive images (images with weeds) and 7,000 negative images (images without weeds), the M. nudiflora training data set contained 8,000 positive and 9,000 negative images, the P. dilatatum training data set contained 7,000 positive and 8,000 negative images, and the U. distachya training data set contained 6,000 positive and 7,000 negative images. These training data sets were used to train DCNN models to identify images with a single weed species. The validation data set (VD) and TD of D. ischaemum, M. nudiflora, and P. dilatatum contained 800 positive and 800 negative images. The VD and TD of U. distachya contained 600 positive and 600 negative images.
Multiple-species neural networks were trained using a data set containing 8,000 positive (2,000 images for each weed species) and 9,000 negative images. The performances of multiple-species neural networks were tested using the aforementioned VD and TD. The performances of multiple-species neural networks for detection of low (weeds were ≤20% per image area) and high weed density (weeds were ≥80% per image area) were examined using TD containing 100 positive and 100 negative images for each weed species.
Network training and testing were implemented within the NVIDIA Deep Learning GPU Training System (v. 6.0.0, NVIDIA, Santa Clara, CA, USA) using Convolutional Architecture for Fast Feature Embedding (Jia et al. Reference Jia, Shelhamer, Donahue, Karayev, Long, Girshick, Guadarrama and Darrell2014). Various base learning rates (0.05, 0.1, and 0.3) and training epochs (1, 3, 5, 10, and 15) were examined in the preliminary work (data not shown). The DCNN models trained with the learning rate of 0.3 and training epoch of 3 are presented because of their low validation loss. The other hyperparameters were standardized and are outlined here:
Solver type: AdaDelta
Batch size: 2
Batch accumulation: 5
Learning rate policy: Exponential decay
Gamma: 0.95
The results of validation and testing were arranged in a binary classification confusion matrix including true positive (tp), false negative (fn), false positive (fp), and true negative (tn). We calculated the precision (Equation 1), recall (Equation 2), overall accuracy (Equation 3), and F1 score (Equation 4) according to the results of confusion matrixes.
Precision measures the successful rate of detection for turfgrass area where weeds do not occur and was calculated using the following equation (Hoiem et al. Reference Hoiem, Chodpathumwan, Dai, Fitzgibbon, Lazebnik, Perona, Sato and Schmid2012; Sokolova and Lapalme Reference Sokolova and Lapalme2009).
$${\rm{Precision}} = {\rm}{{{\rm{tp}}} \over {{\rm{tp}} + {\rm{fp}}}}$$
Recall measures successful rate of identification for turfgrass area with target weeds and was calculated using the following equation (Sokolova and Lapalme Reference Sokolova and Lapalme2009).
$${\rm{Recall}} = {{{\rm{tp}}} \over {{\rm{tp}} + {\rm{fn}}}}$$
Overall accuracy is a ratio of the corrected prediction to the total observation and was calculated using the following equation (Sokolova and Lapalme Reference Sokolova and Lapalme2009).
$${\rm{Overall\;accuracy}} = {{{\rm{tp}} + {\rm{tn}}} \over {{\rm{tp}} + {\rm{fp}} + {\rm{fn}} + {\rm{tn\;}}}}$$
The F1 score is defined as the harmonic mean of the precision and recall and was calculated using the following equation (Sokolova and Lapalme Reference Sokolova and Lapalme2009).
$${\rm{F}}1{\rm{\;score}} = {{2{\rm{\,*\,Precision*Recall}}} \over {{\rm{Precision}} + {\rm{Recall}}}}$$
Results
AlexNet exhibited excellent performance, with a high F1 score of 0.99 in the detection of D. ischaemum in the VD (Table 1). AlexNet had reduced F1 scores in TD 1 and TD 2, primarily due to low precision values. The recall values of AlexNet were 1.00 and 0.99 in TD 1 and TD 2, respectively, but the precision values were 0.88 and 0.77 in TD 1 and TD 2, respectively. As a result, the F1 scores were substantially decreased in TD 2. For detection of M. nudiflora, AlexNet, GoogLeNet, and VGGNet exhibited high F1 scores of ≥0.99, with recall values of 1.00 in TD 1 and TD 2. For detection of P. dilatatum, the F1 scores of AlexNet were 0.94 and 0.95, with recall values of 0.93 and 0.93 in TD 1 and TD2, respectively. GoogLeNet exhibited excellent recall in the detection of P. dilatatum in the VD, TD 1, and TD 2, but unacceptable precision (precision ≤0.8889). For detection of U. distachya, GoogLeNet and VGGNet exhibited excellent performance and achieved high F1 scores of ≥0.9975 with recall of 1.00 in the VD, TD 1, and TD 2.
Single-species neural network validation and testing result for detection of weeds while growing in bermudagrass.a,b
a Abbreviations: VD, validation data set; TD 1, testing data set 1; TD 2, testing data set 2.
b The models were trained to detect a single weed species and the training data set contained a single weed species. The Digitaria ischaemum training data set contained 6,000 positive and 7,000 negative images; the M. nudiflora training data set contained 8,000 positive and 9,000 negative images; the P. dilatatum training data set contained 7,000 positive and 8,000 negative images; and the U. distachya training data set contained 6,000 positive and 7,000 negative images. The VD, TD 1, or TD2 of D. ischaemum, M. nudiflora, and P. dilatatum contained 800 positive and 800 negative images. The VD, TD 1, or TD 2 of U. distachya contained 600 positive and 600 negative images.
For detection of D. ischaemum, the multiple-species neural network of AlexNet exhibited higher F1 scores in TD 1 and TD 2 compared with the single-species neural network, primarily due to increased precision values (Table 2). A similar trend was observed for GoogLeNet. GoogLetNet exhibited high precision and recall values in the detection of D. ischaemum in TD 1 (precision = 0.99, recall = 1.00), but these values decreased in TD 2 (precision = 0.97, recall = 0.87). VGGNet performed well in the detection of D. ischaemum, with F1 scores of ≥0.9804 and recall values of 1.00 in the VD, TD 1, and TD 2. For detection of M. nudiflora, all DCNN models achieved high F1 scores of ≥0.9792, with recall values of 1.00 in TD 1 and TD2. For detection of P. dilatatum, all DCNN models achieved excellent performances in the VD, while the accuracy of detection was reduced in TD 1 and TD 2. VGGNet generally performed better than AlexNet and GoogLeNet in the detection of P. dilatatum. The F1 score of VGGNet was 0.99 in the VD but decreased to 0.96 and 0.94 in TD 1 and TD 2, respectively. For detection of U. distachya, all DCNN models exhibited excellent performances, with high F1 scores (F1 score ≥ 0.98) and recall values (recall ≥ 0.99) in the VD, TD 1, and TD 2.
Multiple-species neural network validation and testing results for detection of weeds growing in bermudagrass.a,b
a Abbreviations: VD, validation data set; TD 1, testing data set 1; TD 2, testing data set 2.
b The models were trained to detect multiple weed species and the training data set contained a total of 17,000 images including 8,000 positive (2,000 images for each species) and 9,000 negative images. The VD, TD 1, or TD 2 of D. ischaemum, M. nudiflora, and P. dilatatum contained 800 positive and 800 negative images. The VD, TD 1, or TD 2 of U. distachya contained 600 positive and 600 negative images.
Weed density significantly influenced AlexNet for the detection of all weed species (Table 3). Weed density significantly influenced recall but not precision. In the situation of high weed densities, the F1 scores of AlexNet were 1.00 for all weed species. However, when the individual weeds were widely scattered and weed densities were low, the F1 scores of AlexNet decreased to 0.91, 0.98, 0.97, and 0.98 for D. ischaemum, M. nudiflora, P. dilatatum, and U. distachya, respectively. AlexNet had markedly reduced performances in TD 2 exclusively due to decreased recall values. For detection of D. ischaemum, GoogLeNet showed a similar trend compared to AlexNet. Recall value of GoogLeNet was 0.99 in the case of high weed density but decreased to 0.60 in the case of low weed density. GoogLeNet exhibited excellent performance in the detection of M. nudiflora, P. dilatatum, and U. distachya, while VGGNet exhibited excellent performance in the detection of all weed species, with F1 scores of 1.00 under the conditions of high and low weed densities.
Multiple-species neural network validation and testing results for detection of weed populations with different densities in bermudagrass.a
a The models were trained to detect multiple weed species, and the training data set contained a total of 17,000 images, including 8,000 positive (2,000 images for each species) and 9,000 negative images.
b High weed density is defined as weeds being ≥80% image area, while low weed density is defined as weeds being ≤20% image area. A total of 100 positive and 100 negative images for each weed species were included in the testing data set.
PRE herbicides can prevent summer annual weeds from germinating (Johnson Reference Johnson1996; McCullough et al. Reference McCullough, de Barreda, Sidhu and Yu2014; Shaner Reference Shaner2014). However, PRE herbicides may have limited soil longevity (Shaner Reference Shaner2014), and summer annual weeds typically have several flushes of germination in spring and summer (Fidanza et al. Reference Fidanza, Dernoeden and Zhang1996). Therefore, the most common practice to control summer annual weeds is the application of PRE and POST herbicides (Johnson Reference Johnson1996; Reed et al. Reference Reed, Yu and McCullough2013; Yu and McCullough Reference Yu and McCullough2016). Sequential applications of fenoxaprop-ethyl (Chism and Bingham Reference Chism and Bingham1991; Lycan and Hart Reference Lycan and Hart2005; Reed et al. Reference Reed, Yu and McCullough2013), quinclorac (Johnson Reference Johnson1993, Reference Johnson1996), and herbicide combinations such as dithiopyr + metribuzin or MSMA + metribuzin provided POST control of D. ischaemum in bermudagrass. Atrazine and simazine are effective for control of M. nudiflora (Yu and McCullough Reference Yu and McCullough2016), but the label use rates of atrazine, metribuzin, and MSMA may cause objectionable phytotoxicity on bermudagrass (Busey Reference Busey2004; Johnson Reference Johnson1976; McCarty et al. Reference McCarty, Miller and Colvin1991; Wang et al. Reference Wang, Yu, Zhou, Sapkota, Wei and Wang2018; Yu et al. Reference Yu, McCullough and Czarnota2017). Atrazine, metribuzin, and MSMA also have properties that can potentially contaminate groundwater (EPA 2018; Pimentel Reference Pimentel2005; Welch et al. Reference Welch, Westjohn, Helsel and Wanty2000). The use of MSMA is banned in turfgrass systems in Florida, while in other states only a single broadcast application is permitted for newly constructed golf courses and spot application is permitted for other golf courses (MSMA 6 Plus Label, Drexel, Memphis, TN). The current herbicide program for control of P. dilatatum and U. distachya is neither efficient nor cost-effective (Cross et al. Reference Cross, McCarty and Estes2016; Henry et al. Reference Henry, Yelverton and Burton2007; Pearsaul et al. Reference Pearsaul, Leon, Sellers, Silveira and Odero2018). For example, repeat applications of amicarbazone at 0.25 kg ai ha−1 in combination with sulfonylurea herbicides, including thiencarbazone + foramsulfuron + halosulfuron, provided adequate control of U. distachya (Cross et al. Reference Cross, McCarty and Estes2016; Pearsaul et al. Reference Pearsaul, Leon, Sellers, Silveira and Odero2018). However, amicarbazone is quite expensive, and a single broadcast application at a rate of 0.25 kg ai ha−1 costs approximately US$1500. Digitaria ischaemum, M. nudiflora, P. dilatatum, and U. distachya have prolific growth in summer, and sequential herbicide applications are often required to provide adequate control. Therefore, precision herbicide application may substantially reduce herbicide input and associated control cost, while mitigating the environmental concerns.
Detection of grass or grass-like weeds in turfgrasses is challenging because of the similarity in plant morphology. Nevertheless, DCNN models effectively detected D. ischaemum, M. nudiflora, P. dilatatum, and U. distachya in bermudagrass with different turfgrass surface conditions (Figure 1). However, the DCNN models differed in terms of the accuracy of detection. For single-species neural networks, VGGNet achieved remarkable results at detecting all weed species. For detection of D. ischaemum and P. dilatatum, single-species neural networks of AlexNet and GoogLeNet exhibited excellent recall but unacceptable precision. As a result, these models are more likely to misclassify turfgrass as weeds, leading to herbicide applications on turfgrass areas where weeds are absent. This problem was overcome by training multiple-species neural networks. The multiple-species neural networks of AlexNet and GoogLeNet achieved higher overall accuracy compared with the single-species neural networks for the detection of D. ischaemum and P. dilatatum, respectively. The increased precision in the multiple-species neural networks might be associated with increased number of training images. In previous research, we reported that a single DCNN model, particularly VGGNet, accurately detected dollar weed (Hydrocotyle spp.), old world diamond-flower [Hedyotis cormybosa (L.) Lam.], and Florida pusley (Richardia scabra L.) in bermudagrass, although these weed species differed tremendously in plant morphology (Yu et al. Reference Yu, Sharpe, Schumann and Boyd2019b). DetectNet, an object-detection DCNN, reliably detected a data set containing annual bluegrass (Poa annua L.) and various broadleaf weeds in dormant bermudagrass (Yu et al. Reference Yu, Sharpe, Schumann and Boyd2019b, Reference Yu, Sharpe, Schumann and Boyd2019c). Overall, results in the present research showed a promising avenue for grassy weed detection in turfgrass systems using a machine vision subsystem for smart spraying.
AlexNet exhibited low recall values when individual weeds were widely scattered and did not form patches. The low recall suggests that the target weeds are more likely to be misidentified. This is highly undesirable, because it may result in poor weed control in field applications. GoogLeNet exhibited similar results for detection of D. ischaemum. We hypothesize that increasing the number of training images containing low weed density may increase recall and overall detection accuracy, which warrants further investigation. In comparison, VGGNet reliably detected all weed species in the situation of low or high weed densities, suggesting that VGGNet is the best DCNN model at detecting these weeds in bermudagrass.
The use of DL algorithms generally demonstrated better weed recognition compared with other techniques. For example, 2D image-processing technology, based on a combination of crop and weed detection by plant size and plant detection by color or infrared to red light reflectance ratios, is widely used for sensing in robotic weed controllers (Fennimore et al. Reference Fennimore, Slaughter, Siemens, Leon and Saber2016; Tillett et al. Reference Tillett, Hague and Miles2001, Reference Tillett, Hague, Grundy and Dedousis2008). However, this technology requires a uniform crop stand and a relatively low weed density for the best performance of weed detection (Fennimore et al. Reference Fennimore, Slaughter, Siemens, Leon and Saber2016). Our results demonstrate that DCNN models, particularly VGGNet, overcame this problem and reliably detected all weed species, regardless of weed densities. In other research, Yang et al. (Reference Yang, Prasher, Landry and DiTommaso2000) documented that a back-propagation artificial neural network achieved 100% accuracy for recognition of maize, but weed detection accuracy did not exceed 80%. In comparison, dos Santos Ferreira et al. (Reference dos Santos Ferreira, Freitas, da Silva, Pistori and Folhes2017) reported that DCNN achieved an average accuracy of >99% for detection of various broadleaf weeds in relation to soil and soybean.
For training purposes, Digitaria spp., M. nudiflora, P. dilatatum, and U. distachya constitute a single category of object to be differentiated from bermudagrass turfgrass. This approach is appropriate, because the selectivity of some herbicides targets the specific group of weed species. For example, thiencarbazone + foramsulfuron + halosulfuron provided POST control of Digitaria spp., M. nudiflora, P. dilatatum, and U. distachya in bermudagrass turfgrass (Cross et al. Reference Cross, McCarty and Estes2016; Pearsaul et al. Reference Pearsaul, Leon, Sellers, Silveira and Odero2018). Many herbicides provide effective POST control of both broadleaf and grassy weed species. For example, quinclorac, a synthetic auxin herbicide, provides POST control of Digitaria spp. and various broadleaf weeds in warm-season turfgrasses (Johnson Reference Johnson1993, Reference Johnson1996). Amicarbazone, a photosystem II–inhibiting herbicide, controls not only P. annua and U. distachya, but also various broadleaf weeds (Cross et al. Reference Cross, McCarty and Estes2016; Dayan et al. Reference Dayan, Trindade and Velini2009). Therefore, the feasibility of simultaneous detection of multiple broadleaf and grassy weeds using a single DCNN model needs to be evaluated. The detection of multiple broadleaf and grassy weed species using a single image-classification neural network might be difficult, considering that the training and testing data sets cover a wide range of different species that are dramatically different in plant morphology, and warrants further investigation.
In the present work, the single-species neural network of AlexNet exhibited lower precision in TD 2 than in TD 1 at detecting Digitaria spp., while GoogLeNet exhibited lower precision in TD 1 than in TD 2 at detecting P. dilatatum. The testing images in TD 1 and TD 2 were taken in different turfgrass sites and management zones with varying maintenance levels, which may have influenced the weed detection, resulting in inconsistent precision between TD 1 and TD 2. For this reason, we hypothesize that increasing the number of training images by covering a wide range of turfgrass management zones may be an effective approach to enhance the performance of DCNN and increase the overall accuracy of detection, which needs to be verified in future work.
DCNN recognizes weeds based on plant color, leaf texture, shape, and other morphological features (Chaki et al. Reference Chaki, Parekh and Bhattacharya2015; Ghosal et al. Reference Ghosal, Blystone, Singh, Ganapathysubramanian, Singh and Sarkar2018; Grinblat et al. Reference Grinblat, Uzal, Larese and Granitto2016). However, morphological variations between weed ecotypes are found in different turfgrass management regimes or geographical regions (Busey Reference Busey2004; Dekker Reference Dekker1997; Saidi et al. Reference Saidi, Kadir and Hong2016). For example, Digitaria spp., M. nudiflora, U. distachya, and P. dilatatum are found in different turf habits, ranging from low-maintained highway rights-of-way to golf course putting greens. In Malaysia, morphologically distinct goosegrass [Eleusine indica (L.) Gaertn] ecotypes have been found throughout the country (Saidi et al. Reference Saidi, Kadir and Hong2016). In the United States, dwarf growing E. indica ecotypes have been selected as a result of the long-term low mowing height (Kerr et al. Reference Kerr, Zhebentyayeva, Saski and McCarty2019). In the present research, all training and testing images of D. ischaemum, M. nudiflora, and U. distachya were taken in Florida, while all training and testing images of P. dilatatum were taken in a small geographic region in Georgia. While the DCNN models achieved high classification rates, testing data sets covering weed ecotypes across more diversified turfgrass management regimes or a wider geographic area is desirable and warrants further investigation.
In summary, this research demonstrated the feasibility of using DCNN for detection of grass or grass-like weeds growing in bermudagrass turfgrass. The single-species neural network of VGGNet achieved excellent detection of all weed species with high F1 scores (F1 score ≥ 0.97) and recall (recall = 1.00). The multiple-species neural network of VGGNet achieved excellent detection of D. ischaemum, M. nudiflora, and U. distachya with high F1 scores (≥0.93) and recall (recall = 1.00). The multiple-species neural networks of AlexNet and GoogLeNet improved the performance of detection of D. ischaemum and U. distachya compared with single-species neural networks, respectively. Weed density affected the detection performance of AlexNet and GoogLeNet but not VGGNet. AlexNet (for detection of all weed species) and GoogLeNet (for detection of D. ischaemum) had lower accuracy of detection in the condition of low weed density compared with high weed density. VGGNet reliably detected all weed species under the conditions of low and high weed densities. Overall, VGGNet performed better than AlexNet and GoogLeNet. Further research is needed to evaluate the precision weed control in field conditions using the DCNN models obtained in the present work for in situ video input in smart sprayers.
Acknowledgments
This research received no specific grant from any funding agency or commercial or not-for-profit organizations. The authors declare no conflict of interest.
