Indian dairy farming is shifting from small, unorganized and traditional to large, organized, automated and commercial. In the past, management and monitoring were based on individual animal-human interactions but due to increasing herd size, there is a need for an efficient and automatic monitoring system for the well-being of the animals (Banhazi et al., Reference Banhazi, Lehr, Black, Crabtree, Schofield, Tscharke and Berckmans2012). Advancements in the development of sensor technology in recent years have made it possible to monitor animal production, growth and health through vocalization and it can be a promising tool to monitor animal behaviour, well-being and various stress conditions in a non-invasive manner (Mulligan et al., Reference Mulligan, Baker and Murphy2002; Manteuffel et al., Reference Manteuffel, Puppe and Schön2004). Vocalization has been used to study and identify several stressful conditions in ungulates (Tallet et al., Reference Tallet, Linhart, Policht, Hammerschmidt, Šimeček, Kratinova and Špinka2013) as well as to detect pig cough (Exadaktylos et al., Reference Exadaktylos, Silva, Aerts, Taylor and Berckmans2008), weaning stress (Weary and Fraser, Reference Weary and Fraser1997) and social isolation among cattle (Hopster et al., Reference Hopster, O’Connell and Blokhuis1995). Various acoustic features based on energy emissions like call duration, frequency, pitch and amplitude have been used to identify stress-related call types in animals (Moi et al., Reference Moi, Nääs, Caldara, Icla, Garcia, Cordeiro and Seno2015) and animal calls have also been studied in relation to general situations, aversive behaviour, stress hormones and brain activity (Often et al., Reference Often, Kanitz, Tuchscherer and Nürnberg2001; Tuchscherer et al., Reference Tuchscherer, Kanitz, Otten and Tuchscherer2002). Machine learning has been successfully utilized in animal vocalization studies (Valletta et al., Reference Valletta, Torney, Kings, Thornton and Madden2017) as well as in the classification of calls which helps in the decision-making process of a dairy farmer or manager (Acevedo et al., Reference Acevedo, Corrada-Bravo, Corrado-Bravo, Villanueva-Rivera and Aide2009). Voice analysis of individual animals can provide the possibility to recognize their condition through distinct vocalization patterns. With the help of voice signal processing and bio-acoustical analysis techniques, raw data in the form of animal voice can be converted into useful relevant information (Manteuffel et al., Reference Manteuffel, Puppe and Schön2004) for management decision-making. Animals (in this case buffalo) can tell us about their neurophysiological, affective and cognitive processes if we can learn to interpret their vocal behaviour correctly, so studies of buffalo communications are important in discussion of welfare evaluation. Keeping in view the role of vocal cues in animal husbandry automation and welfare programmes and taking into account the paucity of literature in this field, this study was conducted to classify different conditions (oestrus, delayedmilking and isolation) by applying data mining techniques (decision tree, logistic regression and artificial neural network, ANN) on the voice samples of Murrah buffaloes.
Materials and methods
Animal selection and husbandry
The study was conducted at the Livestock Research Centre (LRC) of ICAR-National Dairy Research Institute, Karnal, Haryana, India. The buffaloes were fed a conventional diet comprising of dry fodder, green fodder and compound concentrate feed. The research work protocol was duly approved by the Institutional Animal Ethics Committee (IAEC) of ICAR-National Dairy Research Institute, Karnal. Thirty healthy buffaloes (second to third parity) were selected in their early stage of lactation (0-100 days) for voice recording to identify three different conditions (oestrus, delayed milking and isolation) based on their vocalization pattern and acoustic features. For free movement and better observation of oestrus, the 30 selected buffaloes were kept as part of a larger group (50 total) in a loose housing system (360 m2) with covered and open areas. Delayed milking comprised between 1 and 2 h of delay after the normal milking time. Isolation comprised movement of a single buffalo to a distant shed for collecting individual isolated condition stress calls.
Recording, editing and processing of voice signals
Recording of voice signals of buffaloes in oestrus was done before artificial insemination but only after confirming oestrus by behavioural observation (frequent urination, excitement, swelling of the vulva, mucus discharge, bellowing, movement of the tail, teaser bull parading: Gunasekaran et al., Reference Gunasekaran, Singh and Gupta2007) and per-rectal examination for oestrus by a trained veterinarian. Recording was done only after separating the animal in a nearby individual shed (6 m x 3 m) between 6 and 8 am. Animals were kept loose in the shed at the time of recording and their voice signals were recorded for a sufficient period of time so that we could get at least 20 clips (clip means one complete vocal signal uttered in a single attempt) from each animal. The buffaloes were not disturbed by the voice recording device and environment during the experiment and the recording was done under clear weather during early morning time. Voice recording was performed using a video camera (Sony HDV FX7E, handicam) equipped with a good quality microphone (Sony ECM674, unidirectional). The microphone was placed at the front wall of the fence at a height of 150 cm from the floor, while the video camera was kept on a tripod 150 cm above the floor level at such a place and position to capture each and every signal of an animal. The same equipemnt was used for recording delayed milking (which was done in a milking parlour for individual buffalo with which the buffalo was familiar) and isolation (in the remote shed. We collected a data set comprising 300 records for each state. A second data set of control data was collected from the same animals when in non-oestrus condition in the same adjacent shed, at normal milking time in the same milking parlour and from individual animals kept in the main housing familiar shed rather than the remote isolation shed).
During recording care was taken to avoid disturbances and background noise. The recorded voice signals were then transferred to the computer. Editing was done to detect and manually remove extraneous and superimosed noise. Clips of complete voice signals were prepared from all the recordings for each case of the study using the Adobe Premium Pro-1.5 audio-visual editing software package. Sound signals were resampled at a sampling frequency of 48 KHz and 16 bits. To avoid time-related changes in the audio signal characteristics, the voice signals were broken into short frames of 15msec using a Hamming window for further processing. Mel-frequency cepstrum coefficients were used for voice sample processing and analysis. The following acoustic features were then extracted with the help of the PRAAT 5.1.36 software package developed by Boersma and Weenink (Reference Boersma and Weenink2010):
1. Amplitude (P, minimum and maximum)
2. Total energy (P2s)
3. Pitch (Hz, minimum, range and mean)
4. Intensity (dB, minimum, maximum and mean)
5. Mean power (W/m2)
6. Bandwidth (Hz, B1, B2, B3 and B4)
7. Formants (Hz, F1, F2, F3, F4 and F5)
8. Number of pulses
9. Number of periods
10. Mean period (sec)
11. Unvoiced frame (%)
12. Degree of voice breaks (%)
13. Jitter (%)
14. Shimmer (%)
Date were stored in MS-EXCEL sheet for statistical analysis.
Data analysis
Three data sets were prepared for each physiological condition (oestrus, delayed milking and isolation). Each data set had 600 records consisting of 300 records for the test condition and 300 control records, meaning that a total of 1,800 different recordings were analysed. The input parameters for these data sets comprised all features extracted from acoustic analysis for all the models. The output and target parameters comprised the change in voice due to oestrus, due to delayed milking or due to isolation (all 1Yes/0 No, where the primary event (true condition) was 1). Three algorithms (models) were trained by partitioning the acoustic data into training and validation sets to develop predictive models. Three different ratios were assessed: 60%-40%, 70%-30% and 80%-20% for each condition. Performance evaluation parameters were calculated for accuracy and missclassification rate using these equations:
\begin{equation*}Accuracy = \frac{{TP + TN}}{{TN + FP + FN + TP}} \times 100\end{equation*}
\begin{equation*}Missclassification\,Rate = \frac{{FP + FN}}{{TN + FP + FN + TP}} \times 100\end{equation*}Where:
TN (true negative) is output 0, target 0
FP (false positive) is output 1, target 0
FN (false negative) is output 0, target 1
TP (true positive) is output 1, target 1
Artificial neural network model, ANN
The ANN model is an intelligent data processing technique, which has the ability to predict automatically any event from the given data set after training the network. Such a type of neural network consists of the input layer, hidden layer(s) and output layer and each layer has a specific role in execution of network function. Here, the number of hidden layer was 1 and the numbers of neurons in this were 3, 5, 7, 10, 15, 20. The multilayer network architecture used consisted of multilayer perceptron (MLP) and radial basis function (RBF). The optimization training algorithm consisted of MLP-Levenberg-Marquardt (LM) and conjugate gradient (CG). RBF (default) depends on the number of weights that are applied during execution. The transformation function in the hidden layer used the tangent sigmoid function. Other parameters were based on the default setting of the SAS statistical software.
Decision tree model, DT
Decision tree models have been optimized based on decision and average square error (probability) options to find the best model. This model is a flow chart-like decision support tool based on an algorithm, so a tree-like model in which a sequence of tests are classified/done adaptively and the outcome of the previous test can affect the results of the next test to be performed. In the decision tree, each node represents a “test” on an attribute/trait (e.g. voice feature change or not in oestrus condition), each branch defines the outcome of the test and each leaf represents a class label (1/0). This test gives outcomes in the form of yes or no to competitive alternatives. In this study, the square type node (Kamiński et al., Reference Kamiński, Jakubczyk and Szufel2018) based decision tree was used and the outcome for the change in voice features during the three test conditions was marked as 1 for yes and 0 for no.
Logistic regression, LR
This is a statistical model which predicts the probability of one possible event under study out of two options, by considering the log-odds for the event, which is possibly a linear combination of one or more independent variables or predictors.
Results
Comparative results of the DT, LR and ANN models for the oestrus condition are given in Table 1, for the delayed milking in Table 2 and for the isolation in Table 3. Accuracy represented percentage of correctly detected oestrus, delayed milking or isolation. The ANN model had the highest accuracy of 77.7% for the oestrus condition (sable 1), 79.3% for delayed milking (Table 2) and 85.2% for isolation. The DT (on 80-20 data partition) for the three conditions provided additional information about acoustic features based on a priority basis from most important to least important (Table 4).
Comparative results (accuracy) of decision tree (DT), logistic regression (LR) and artificial neural network (ANN) models for oestrus condition
Comparative results (accuracy) of decision tree (DT), logistic regression (LR) and artificial neural network (ANN) models for delayed milking condition
Comparative results (accuracy) of decision tree (DT), logistic regression (LR) and artificial neural network (ANN) models for isolation condition
Important acoustic features under different physiological conditions based on the decision tree model
The validation results of the ANN model, decision tree and logistic regression analyses are given in the online Supplementary File. Supplementary Tables S1 and S2 present the data for oestrus, Supplementary Tables S3 and S4 for isolation and Supplementary Tables S5 and S6 for delayed milking. The data are based on the minimum misclassification rate for the partition schemes 60:40, 70:30 and 80:20. The misclassification rate (MCR) is the percentage of events misclassified out of total events for a given condition (Zhong, Reference Zhong2007).
Discussion
Several published studies have shwon that vocalization behaviour of animals could be a significant indicator of different physiological and physical states of the animals (Appleby et al., Reference Appleby, Weary, Taylor and Illmann1999; Watts and Stookey, Reference Watts and Stookey1999; Weary and Chua, Reference Weary and Chua2000; Watts et al., Reference Watts, Stookey, Schmutz and Waltz2001; Yin and McCowan, Reference Yin and McCowan2004). This study compared the acoustic features of buffaloes in three different stress-related physiological states through the use of three modelling techniques (artificial neural network, ANN; decision tree, DT and logistic regression, LR). Among the three techniques used, the ANN model was found to be best on the basis of minimum misclassification rates defining the physiological conditions of oestrus, isolation and delayed milking. However, the DT model provided additional information in the form of an acoustic features list in decreasing order (most important to less important) for identification of oestrus condition, namely viz. maximum intensity (dB), minimum amplitude (P), mean period (sec.), maximum amplitude (P), bandwidth 4 (Hz), degree of voice breaks (%), minimum intensity (dB), formant 5 (Hz). Sound amplitude (measured as sound pressure level) and intensity (also a function of amplitude or logarithm of amplitude) are important acoustic parameters indicating the active space of a vocalizing animal (Marten and Marler, Reference Marten and Marler1977) and increased energy expenditure by females during the oestrus phase to attract male animals for successful mating. There are various studies available that provide evidence that calls of females can contain information on the reproductive status. These include changes in amplitude in Barbary macaques (Pfefferle et al., Reference Pfefferle, Heistermann, Pirow, Hodges and Fischer2011) as well as total energy in domestic and African wild cats (Nicastro, Reference Nicastro2004). Modulation of intensity and pitch of the voice signal occurs in human females during the menstrual cycle (Abitbol et al., Reference Abitbol, Abitbol and Abitbol1999; Bryant and Haselton, Reference Bryant and Haselton2009). Fundamental frequency and formant variable analysis (Reby and McComb, Reference Reby and McComb2003) has been used in elephants (Leong et al., Reference Leong, Ortolani, Burks, Mellen and Savage2003), dairy cattle (Yeon et al., Reference Yeon, Jeon, Houpt, Chang, Lee and Lee2006; Chung et al., Reference Chung, Lee, Oh, Park, Chang and Kim2013) and water buffaloes (Devi et al., Reference Devi, Singh, Dudi, Lathwal, Ruhil, Singh, Malhotra, Baithalu and Sinha2019) to provide information about the oestrus phase of the cycle. Here we recorded more or less similar observations in terms of acoustic parameters (amplitude, intensity, bandwidth, fifth formant variable), along with mean period and degree of voice breaks for identification of the oestrus phase in buffalo. We studied four consecutive bandwidths (B1, B2, B3, B4), but B4 was found more relevant to the oestrus phase. Bandwidth represents the frequency domain measure of dampening of voice signals (how rapidly vibrations of folds in the system die away after excitation with an input pulse). The primary reason for changes in acoustic features during an estrous cycle is significant changes in reproductive hormones (Meitzen et al., Reference Meitzen, Moore, Lent, Brenowitz and Perkel2007). Many studies have shown that reproductive hormones influence calling behaviour and call structure, for example in humans (Abitbol et al., Reference Abitbol, Abitbol and Abitbol1999; Bryant and Haselton, Reference Bryant and Haselton2009), nonhuman primates (Pfefferle et al., Reference Pfefferle, Heistermann, Hodges and Fischer2008), and cattle (Schön et al., Reference Schön, Hämel, Puppe, Tuchscherer, Kanitz and Manteuffel2007).
An animal's vocal behaviour may provide a clue of its response to psychological challenges like social isolation as animals are social animals. This study also reported important acoustic features indicating the isolation state of buffaloes like minimum amplitude (P), formant 5 (Hz), maximum amplitude (P), maximum pitch (Hz), mean amplitude (P), formant 3 (Hz), shimmer (%), mean power (W/m2). To differentiate stress due to delayed milking in lactating buffaloes, formant 1 (Hz), minimum amplitude, maximum amplitude, formant 4 (Hz), bandwidth 2 (Hz), and mean amplitude were important features. We could say that amplitude and formants (F1, F3, F4, F5) are the most important acoustic parameters to interpret the distressed state of dairy animals. This study corroborated previous findings in cattle (Boissy and Le Neindre, Reference Boissy and Le Neindre1997), pigs (Weary et al., Reference Weary, Braithwaite and Fraser1998), goats (Carbonaro et al., Reference Carbonaro, Friend, Dellmeier and Nuti1992) and sheep (Porter et al., Reference Porter, Nowak and Orgeur1995). Delayed milking also causes discomfort and distress to cows which is reflected in their behavioural signs. Cows urinated more and remained standing for longer (Stefanowska et al., Reference Stefanowska, Plavsic, Ipema and Mmwb2000). After a four-minute delay in milking, camels showed signs of acute stress, signs of vigilance and attempts to escape from the milking parlour appeared (Atigui et al., Reference Atigui, Marnet, Ayeb, Khorchani and Hammadi2014). In hum,ans, different emotions (fear, anger, sadness, joy) affect vocal acoustic features like an increase in amplitude, fundamental frequency (F0), F0 range, F0 variability, jitter, shimmer and speech rate (Li et al., Reference Li, Tao, Johnson, Soltis, Savage, Leong and Newman2007; Juslin and Scherer, Reference Juslin, Scherer, Harrigan, Rosenthal and Scherer2008). Here, we found important acoustic variables formant 1 (Hz), minimum amplitude (P), maximum amplitude (P), formant 4 (Hz), bandwidth B2 (Hz), and mean amplitude (P) to indicate stress/discomfort due to delayed milking. These vocalizations were acoustically distinctive, even possibly indicating the psycho-physiological state of isolation distress, but this has not yet been clearly demonstrated and requires further exploration in buffalo species. The use of more advanced neural network architectures such as convolutional neural network and recurrent neural network may be able to provide better performance for classification of emotional states of dairy animals.
In conclusion, artificial neural network models were shown to be better than decision tree and logistic regression models for the classification of lactating buffalo calls under different physiological conditions. This may help to improve animal welfare and efficiency of animal husbandry, which is a challenge in intensive and large commercial dairy farms.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0022029925100976.
Acknowledgements
The authors gratefully acknowledge the facilities and financial support from ICAR-National Dairy Research Institute, Karnal, Haryana, India, for carrying out this research work.
