Introduction
Almond (Prunus dulcis) is one of the most famous and highly valued nut crops and has a pleasant flavour and a positive effect on health. The global production of almonds as a popular and valuable agriculture product was approximately 2.918 million tonnes in 2013 (FAO, 2013). Regarding its nutritional benefits, almond contains a high concentration of unsaturated fatty acids and dietary fibre, and regular consumption of almonds has been proven to lower low-density lipoprotein cholesterol levels in humans (Berryman et al., Reference Berryman, Preston, Karmally, Deckelbaum and Kris-Etherton2011).
Almond is susceptible to the damage caused by pre- or post-harvest insect attacks. Studies have revealed that insect infestation may be influenced by growth conditions, harvesting techniques and post-harvest storage conditions (Schatzki and Ong, Reference Schatzki and Ong2000). Almonds are usually placed on the ground during post-harvest storage prior to their transport and processing. In this case, insect attacks usually take place and larvae can enter the kernel easily through split hulls. Damage by navel orange worm larvae, peach twig borer and other insects may occur in 0.02–0.10 of all almond products, depending on the year, cultivar, growth and handling conditions (Schatzki and Ong, Reference Schatzki and Ong2000). Infestation causes a reduction in consumer acceptance because of degraded appearance, bitter flavour and lower nutritional value (Nakariyakul and Casasent, Reference Nakariyakul and Casasent2011). Despite insecticide treatment, harvested nuts still have an annual rejection rate of 0.04–0.12 owing to insect damage and associated mould damage (Bentley, Reference Bentley1993). Insect feeding and respiration activities can promote infection-induced aflatoxigenic moulds and aflatoxin residues (Liang et al., Reference Liang, Slaughter, Ortega-Beltran and Michailides2015). Consequently, it has been reported that 0.017 of whole and/or broken natural almonds and 0.097 of the manufacturing stock contain aflatoxin in excess of 1 ng/g (Schatzki, Reference Schatzki1996). To ensure the quality and safety of almonds, it is necessary to develop a rapid and non-destructive method for identifying the hidden insect damage in intact almonds.
Various non-destructive techniques have been developed to evaluate food quality and safety attributes, e.g. near-infrared spectroscopy, hyperspectral imaging systems, nuclear magnetic resonance imaging, X-ray imaging and Raman spectroscopy (Arendse et al., Reference Arendse, Fawole, Magwaza and Opara2018; Khairi et al., Reference Khairi, Ibrahim, Yunus and Faramarzi2018; Kiani et al., Reference Kiani, van Ruth, Minaei and Ghasemi-Varnamkhastid2018; Islam et al., Reference Islam, Nielsen, Stærke, Kjær, Jørgensen and Edelenbos2018a, Reference Islam, Nielsen, Stærke, Kjær, Jørgensen and Edelenbos2018b). Recently, these techniques were applied to the inspection of internal damage (Nakariyakul and Casasent, Reference Nakariyakul and Casasent2011; Nakariyakul, Reference Nakariyakul2014; Rogel-Castillo et al., Reference Rogel-Castillo, Boulton, Opastpongkarn, Huang and Mitchell2016), detection of pinholes and fungal infection (Kim and Schatzki, Reference Kim and Schatzki2001; Liang et al., Reference Liang, Slaughter, Ortega-Beltran and Michailides2015) and quality grading (Teimouri et al., Reference Teimouri, Omid, Mollazade and Rajabipour2015) of almonds. However, it is still difficult to develop an on-line approach for industrial applications because of multi-dimensional data processing, high cost and/or limited morphological information. In particular, existing studies have been conducted on either kernels without a shell or kernels with a shell containing external visible defects, and no effective strategy is available for detecting hidden insect damage of in-shell almonds. Therefore, it is necessary to devise a novel technique for the rapid and non-destructive identification of in-shell insect-infested almonds.
Multispectral imaging (MSI) is a simple imaging technique with narrow bands and is quite suitable for on-line process monitoring and quality control. Recently, MSI has been applied as a powerful analytical tool for the non-destructive inspection of kernel quality characteristics of sunflower seeds (Ma et al., Reference Ma, Wang, Liu, Lu, Chen, Chen, Yang and Zheng2015), determination of transgenic rice seeds and soybean seeds (Liu et al., Reference Liu, Liu, Lu, Chen, Yang and Zheng2014a, Reference Liu, Liu, Lu, Chen, Chen, Yang and Zheng2016a), classification of storage periods of Iron Buddha tea (Xiong et al., Reference Xiong, Liu, Pan, Ma, Xiong, Qi, Chen, Lu, Yang and Zheng2015), detection of adulteration in water-injected beef samples and sucrose adulteration in tomato paste (Liu et al., Reference Liu, Cao, Wang, Pan, Ma, Liu, Chen, Yang and Zheng2016b, Reference Liu, Hao, Su, Chen and Zheng2017) and measurement of quality attributes of agricultural products (Liu et al., Reference Liu, Liu, Lu, Ma, Chen, Yang and Zheng2014b, Reference Liu, Liu, Chen, Yang and Zheng2015; Ma et al., Reference Ma, Yao, Xie, Liu, Chen, Chen and Zheng2014, Reference Ma, Qin, Shi, Zhou, Chen, Hu and Zheng2016; Ebrahimi et al., Reference Ebrahimi, van den Berg, Aunsbjerg, Honoré, Benfeldt, Jensen and Engelsen2015; Pan et al., Reference Pan, Wang, Deng, Li, Chen, Chiang, Yang and Zheng2015; Xiong et al., Reference Xiong, Liu, Liu, Pan, Ma, Chen, Chen, Yang and Zheng2016). Nonetheless, there are no published data regarding the application of MSI in detecting in-shell insect-infested almonds without any visible defects. To our knowledge, this is the first time that the shell/kernel weight ratio has been used to describe the status of insect infestation quantitatively.
Support vector machine (SVM), proposed by Cortes and Vapnik (Reference Cortes and Vapnik1995), is a learning algorithm used for classification and regression tasks. It has been employed effectively to perform multivariate function estimation or non-linear classification. Compared with other analysis methods, SVM does not require a large number of training samples for model development and is not affected by the presence of an outlier. Recently, MSI combined with SVM showed excellent performance when determining quality attributes and the ripeness stage of strawberry fruits (Liu et al., Reference Liu, Liu, Lu, Ma, Chen, Yang and Zheng2014b).
Therefore, the major objectives of the current work were (1) to investigate the feasibility of MSI combined with SVM for the detection of in-shell insect-infested almonds and (2) to optimize the classification approach for future industrial applications.
Materials and methods
Almond samples
In-shell insect-infested almonds were defined as almonds that had similar external features without any visible defects such as insect holes, cracks, black spots and mould; however, kernels had been damaged to different infestation degrees. The in-shell almond samples used in the present work were obtained from Three Squirrels Co., Ltd. (Wuhu, Anhui, China). Initially, the samples were analysed randomly by MSI, followed by shell breaking to test whether the kernels were damaged or not, then the shell/kernel weight ratio was measured for the determination of infestation degrees. Finally, the spectral data were re-assigned to gain 115 intact and 230 insect-infested samples.
The classification analysis was performed by means of a two-class model (intact/infested) and a three-class model (intact/slightly infested/severely infested). For the two-class model, 75 intact and 150 insect-infested almonds were selected randomly for the calibration set to build the model, and the remaining samples were used for the prediction set to test the stability of the two-class model.
For the three-class model, the above-mentioned 230 insect-infested samples were sub-divided into 115 slightly infested and 115 severely infested samples according to the infestation degree to ensure sorting accuracy. Each class of samples was assigned to a calibration set (75 × 3 samples) and prediction set (40 × 3 samples) at a ratio of about 2 : 1.
Spectroscopy
MSI of in-shell samples was carried out by VideometerLab equipment (Videometer A/S, Hørsholm, Denmark). Each multi-spectral image consisted of 19 separate bands ranging from the visible region to the near-infrared region (405, 435, 450, 470, 505, 525, 570, 590, 630, 645, 660, 700, 780, 850, 870, 890, 910, 940 and 970 nm). The acquisition system recorded surface reflections by means of a standard monochrome charge-coupled device chip nested in a Point Grey Scorpion camera; the latter was calibrated using a diffuse white target, a diffuse dark target and a geometric target before acquisition of multi-spectral images to ensure pixel correspondence with all spectral bands (Dissing et al., Reference Dissing, Nielsen, Ersbøll and Frosch2011). After that, the light system could be modified on the basis of the type of objects to be recorded. Each multispectral image was processed in VideometerLab software, version 2.12.23. To analyse a region of interest, the image background was removed by selection of distinguishing colours with implementation of canonical discriminant analysis (Cruz-Castillo et al., Reference Cruz-Castillo, Ganeshanandam, Mackay, Lawes, Lawoko and Woolley1994) and segmentation via a simple threshold.
The shell/kernel weight ratio for determining the degree of insect infestation
After spectral images were acquired, the kernel weight of each sample was measured for determining the degree of infestation. In-shell almonds (W 1) and kernels (W 2) were weighed; these two parameters were used to calculate the proportion of the kernel weight in the whole almond using the following equation:
$$R = \; \displaystyle{{W_2} \over {W_1}}$$This characteristic was employed to reassign the insect-infested almonds into slightly infested and severely infested almonds compared with intact ones. Shell weight was also measured to ensure no significant differences among the samples (Table 1). The mean values of in-shell almond weight and the proportion of kernel are presented in Table 1. In the case of insect consumption, insect-infested kernels had less weight in comparison with intact kernels. For the measurement of kernel weight, almonds were classified according to the following criteria: Grade#1 (intact sample, R was from 0.70 to 0.98), Grade#2 (slightly infested, R was from 0.50 to 0.69) and Grade#3 (severely infested: R from 0.00 to 0.49).
The mean values of weight of in-shell almonds, shells and the proportion of kernels
Grade#1: intact almonds; Grade#2: slightly infested almonds; Grade#3: severely infested almonds; data are presented as mean ± standard deviation of triplicate determinations.
Statistical analysis
Principal component analysis
Principal component analysis (PCA) was conducted to calculate the contribution rate of each principal component (PC) and cumulative scores. In general, the first few PCs could explain most of the variance of data (Fung and LeDrew, Reference Fung and LeDrew1987). This analysis could evaluate the similarity of variables by identifying potential spectral outliers between intact and insect-infested almonds.
Support vector machine
In the classification analysis, the most important task was to establish the model of discrimination between the extracted feature information and the almond infestation degree. The theoretical bases of SVM are the statistical learning theory and Vapnik–Chervonenkis dimension promotion theory. The details of the SVM algorithm can be found in research reported previously (Devos et al., Reference Devos, Ruckebusch, Durand, Duponchel and Huvenne2009). The SVM model could be expressed as:
$$f\,(x) = \mathop \sum \limits_{i = 1}^l \alpha _iK(x_i \times x) + b$$where K(x i × x) indicates the kernel function, x i is the support vector, α i is Lagrange multipliers and b denotes the bias term.
Proper kernel function and optimum kernel parameters were crucial for final classification with the highest accuracy. The radial basis function with a simple Gaussian function was chosen as the proper kernel function in SVM to reduce computational complexity of the training procedure and to increase performance under the general smoothness assumptions.
Two crucial parameters (c, g) were required for SVM. The parameter c is the penalty factor, indicating tolerance of error, and parameter g determines implicitly the distribution of data in new feature space. To find optimum parameters of the model with the best discrimination results, the leave-one-out cross-validation procedure was performed during the calibration step. Leave-one-out cross-validation provided a sensible model selection criterion by finding an almost unbiased estimate of the true generalization ability of the model. Raw data on the samples were recorded in the analysis of insect-infested almonds in different classes, which gave the category labels.
All the data were analysed in Matlab 2010 software (The Mathworks Inc., Natick, MA, USA), Libsvm toolbox (version 1.4.5, Tai Wan) and Origin 8.5.
Results
Spectroscopic analysis
The discriminated classes of intact, slightly infested and severely infested in-shell almonds and their kernels are depicted in Fig. 1. It was challenging to differentiate substantially between infested and intact almonds on the basis of shell appearance because there were no visible defects on the shell: the damage could not be seen without shell breakage.
Images of the intact (a), slightly infested (b) and severely infested (c) in-shell almond and their kernels. Colour online.
The average reflectance spectra collected by MSI are shown in Fig. 2. Apparent variation of reflectance spectra was observed between intact and insect-infested almonds, with reflectance being markedly lower in the visible wavelength region than in the near-infrared region (Fig. 2). Meanwhile, as expected, mean spectral intensity of severely infested almonds was lower than that of slightly infested and intact almonds. Variance of the entire visible spectral range was probably due to the difference in the surface colour of the samples. In addition, reflectance spectra in the near-infrared region may reflect compositional differences, which could be utilized for qualitative classification. Therefore, the multivariate classification models were chosen for further analysis.
Mean spectra from the multispectral images of intact and in-shell insect-infested almonds of the two-class model (a) and the three-class model (b). Vertical bars represent standard deviations from all sample measurements. Colour online.
PCA was performed initially to examine data variations and possible contributing factors among classes by means of all the spectral data. Meanwhile, the main objectives of PCA were to attain qualitative visual differentiation and reduce the data dimensionality. A two-dimensional PCA score plot of 115 intact almonds and 230 infested almonds in terms of the first two PCs revealed good differentiation results for qualitative classification even though there were some overlaps among the different almonds classes (Fig. 3). It was found that the initial two factors accounted for up to 98.4% of the spectral variations (93.3 and 5.2% for the first two PCs, PC1 and PC2, respectively). The loading plot between the first two components and 19 wavelengths is presented in Fig. 4. The best correlations were seen for wavelengths of 700 nm in the red light area for PC1, but 910 nm for PC2. None of the correlations of the 19 wavelengths and the first two principle components was significantly different, meaning that the 19 wavelengths could be employed to create the discrimination model. The results indicated that differentiation between intact and insect-infested almonds was possible, and different spectral attributes of samples were associated with the shell characteristics.
Two-dimensional score plot of the first two PCs for intact (+) and in-shell insect-infested almonds (◇) in the two-class model (a), and intact (+), slightly infested (○) and severely infested almonds (□) in the three-class model (b) with the spectral dataset. The classification was clear between classes, indicating that the spectra between classes had quite different factors. Colour online.
The loading plot between the first two components and the 19 wavelengths. Colour online.
To identify the wavelengths most related to infestation in almonds, the genetic algorithm-partial least squares (GA-PLS) method was applied in the current study. Different numbers of wavelengths from 3 to 19 were selected separately by importance as input features of SVM to simplify the model and reduce data requirements. The model with all features showed better performance than that with selected features only (Fig. 5). This result might be explained as follows: the 19 wavelengths contained more abundant and comprehensive information, whereas the selected wavelengths lost information related to infestation and then led to a decrease in identification accuracy, even though less data could result in a simpler model. Therefore, all wavelengths were used to separate almond classes.
The accuracy of different numbers of significant wavelengths from 3 to 19 selected separately by importance as the input features of SVM in the prediction sets: (a) two-class model and (b) three-class model. Colour online.
Classification analysis
To test the efficiency of insect damage identification in intact almonds by means of the MSI system, PLS and back propagation neural network (BPNN) were also used to analyse the results as comparison methods. The whole dataset of the two models was further categorized into a calibration set and a prediction set. In the two-class model, intact and insect-infested almonds were assigned constant values of 1 and 2, respectively. As in the three-class model, intact, slightly infested and severely infested almonds were assigned the numbers 1, 2 and 3.
For the SVM method, the leave-one-out cross-validation procedure was performed to find the optimum parameters of the model with the best discrimination results during the calibration step. Here, the optimum combination of (c, g) for discrimination of insect-infested almonds was found to be (8, 1.41) (Fig. 6). A plot of actual and predicted results of almond samples using the SVM model in the prediction set was created (Fig. 7). In the two-class model, Fig. 7a clearly indicates that only one infested and two intact almonds were mis-classified, thereby yielding a classification accuracy as high as 97.5% in the prediction set. In the calibration set, two intact almonds were mis-classified into infested almonds and the accuracy was 99.1% (Table 2). The results from the two-class model highlighted the fact that MSI combined with the SVM model afforded enough information to discriminate insect-infested almonds.
The 3D graph result of parameters selection using cross-validation analysis. Colour online.
Plot of actual v. predicted results of intact and in-shell insect-infested almonds using SVM model in the prediction set: (a) two-class model and (b) three-class model. Colour online.
Calibration and prediction results based on the spectra acquired from the two-class and three-class models of in-shell almonds
Grade#1: intact almonds; Grade#2: slightly infested almonds; Grade#3: severely infested almonds; MS: mis-classified samples.
The degree of infestation in almonds in the three-class model was relatively complex and unevenly distributed in each class. To better evaluate almond quality, the three-class model was expected to identify almonds according to different infestation degrees. The quantitative analysis of the three-class model yielded a 4.4% mis-classification rate in the calibration set (Table 2). The discrimination results of the three-class model in the prediction set are illustrated in Fig. 7b. Incorrect classification occurred during sample distribution into different classes, for instance, two intact almonds (Grade#1) and one slightly infested almond (Grade#2) were mis-classified with one another, whereas five severely infested almonds (Grade#3) were mis-classified as slightly infested almonds. The combination of full spectra with the SVM model performed commendable discrimination with total accuracy rates of 95.6 and 93.3% in the calibration set and prediction set, respectively.
For the PLS and BPNN methods, the results can be found in Fig. 8 and Table 3. As for insect infestation, prediction accuracy was 95.8% in the two-class model and 85% in the three-class model by the PLS method. The BPNN method yielded accuracy rates of 95.8 and 90.8% in the two-class and three-class models, respectively. A comparison of the accuracy revealed that SVM is the best method for discrimination of in-shell insect-infested almonds (97.5 and 93.3% in the two-class and three-class models, respectively). The results confirmed that significant differences among intact, slightly infested and severely infested almonds did exist, and the identification of insect damage in intact almonds via a MSI system was feasible. The SVM yielded the best results compared with PLS and BPNN, possibly because of its learning ability with small samples.
Results of PLS model for insect-infested classification in the (a) two-class model and (b) three-class model. Colour online.
Comparison of discrimination performance obtained with PLS, BPNN and SVM methods with the spectral data
MS, mis-classified samples; PLS, partial least squares; BPNN, back propagation neural network; SVM, support vector machine.
To determine which class was most easily confused with others, SVM was performed to assess the classification performance on differentiation of the three classes of almonds (intact, slightly infested and severely infested almonds) with a confusion matrix. The observed data, sensitivity and specificity of each class are described in Table 4. The results suggested that specificity of detection of the three classes was higher than 91%, sensitivity was 95 and 97.8% among intact and slightly infested almonds, respectively. Detection of severely infested almonds had the lowest sensitivity: 87.5%. The major mis-classification problem involved severely infested almonds, with 12.5% of such samples being more easily confused with slightly infested almonds than with intact almonds.
Confusion matrix detailing the multi-class classification results of intact, slightly and severely infested in-shell almonds using spectral dataset for the combined SVM method
Discussion
Excavation injuries of almonds vary and are rarely detected because defects are externally invisible, in contrast to sunflower seeds with a wormhole on the surface (Ma et al., Reference Ma, Wang, Liu, Lu, Chen, Chen, Yang and Zheng2015). For in-shell insect-infested almonds, the prominent involvement of insect metabolic activities can produce heat and moisture, leading to the development of hotspots and an increase of oil oxidation (Pearson, Reference Pearson1999; Neethirajan et al., Reference Neethirajan, Karunakaran, Jayas and White2007). It is well known that spectral differences can reflect the dissimilarity of physical and chemical components of different objects. Therefore, spectral changes may be due to the differences in colour and moisture among different samples.
VideometerLab can carry out MSI at different wavelengths – from the visible region to the shorter wavelengths of the near-infrared region – which includes information on colour, moisture and internal tissues in almonds. In the visible region, the effective wavelengths correspond to the green (489 and 510 nm) and yellow (579 and 596 nm) regions, whereas 726 and 650 nm belong to the red region (Balasundaram et al., Reference Balasundaram, Burks, Bulanon, Schubert and Lee2009; Wang et al., Reference Wang, Nakano and Ohashi2011). Thus, discoloration of almond skin caused by infestation will lead to a spectral change. In the near-infrared region, a relatively high reflectance spectrum at 900 nm with subsequent stability indicates the high absorption potential of this spectral region; this property is considered an effective attribute that creates changes in water or the third hydroxide (O–H) stretching overtone because almonds can be affected by moisture (Schaare and Fraser, Reference Schaare and Fraser2000; Xing and Guyer, Reference Xing and Guyer2008). Therefore, a spectral change in this region was attributed to moisture. Meanwhile, larval mastication can lead to marked disruption of the internal tissues, thereby affecting fruit density, shape and firmness, which should influence the spectral characteristics of samples (Drew, Reference Drew1988). Furthermore, infested almonds yielded lower reflectance in the current study than intact ones because infested tissues absorbed most of the light energy. These results are consistent with other studies on the detection of insect-infested chestnuts and sunflower seeds (Moscetti et al., Reference Moscetti, Haff, Saranwong, Monarca, Cecchini and Massantini2014; Ma et al., Reference Ma, Wang, Liu, Lu, Chen, Chen, Yang and Zheng2015). It was concluded here that both colour and moisture were responsible for the spectral differences. Other studies produced similar results (Xing and Guyer, Reference Xing and Guyer2008; Wang et al., Reference Wang, Nakano, Ohashi, Takizawa and He2010, Reference Wang, Nakano and Ohashi2011).
In the classification analysis, the successful detection and classification of in-shell insect-infested almonds by MSI highlighted the prospect of developing non-destructive sorting equipment based on filters, lasers or diode arrays for the horticultural supply chain. Judging by the current satisfactory classification results, this approach may be regarded as an acceptable method for commercial use.
However, other research should still be conducted to apply this technology in industry. Firstly, to obtain efficient and accurate detection results, wider ranges/larger samples must be tested to establish a database and to set up an adaptable model. Secondly, to obtain on-line equipment used in industrial manufacture, an efficient and accurate image-processing algorithm should be devised. Finally, to popularize the equipment, its cost should be reduced as much as possible.
In the future, MSI together with chemometrics could be tested for the rapid and non-destructive discrimination of other internal problems such as mould, rancidity, decomposition and shrivelling in almonds or other nuts.
Conclusions
In general, the current results indicate that MSI combined with the SVM model achieves excellent classification of intact and insect-infested almonds, with an accuracy of 99.1% in the calibration set and 97.5% in the prediction set for the two-class model. The classification accuracy of the three-class model (intact, slightly infested, and severely infested almonds) is 95.6 and 93.3% in the calibration set and prediction set, respectively. The severely infested almonds are more easily confused with slightly infested almonds than with intact almonds. As a rapid and non-destructive technique, MSI was applied as a powerful analytical processing tool for handling of insect-infested almonds containing invisible defects. It will provide (1) a novel and practical approach to real-time and non-destructive screening of almonds for in-shell insect-infested ones and (2) an essential platform for the development of an MSI instrument for early steps of the post-harvest culling process in the commercial horticultural supply chain.
Financial support
The current work was supported by the National Key RandD Program of China (2017YFC1600603), the Key Science and Technology Specific Projects of Anhui Province (16030701078, 15czz03117) and the Funds for Huangshan Professorship of Hefei University of Technology (407-037019).
Conflict of interest
None.
Ethical standards
Not applicable.
