Bovine mastitis, commonly caused by a bacterial infection of the udder, is the most common reason for antibiotic use during lactation in dairy cattle worldwide (Erskine et al., Reference Erskine, Cullor, Schaellibaum, Yancey and Zecconi2004; McDougall et al., Reference McDougall, Hussein and Petrovski2014). The use of antibiotics for the treatment of mastitis in cows creates a number of potential problems, including the risk of antibiotic residues in milk and meat for human consumption, the potential development of antibiotic resistance in pathogens and non-target bacteria, the direct cost of antimicrobial therapy and the loss of milk for human consumption during the period of treatment and the associated milk withholding period. Guidelines have been released describing the prudent use of antibiotics to improve animal health and welfare (e.g. http://www.nzva.org.nz/page/antibioticdairy; Ruegg, Reference Ruegg2014 and, Reference Ruegg2015), and emphasising the need to reduce antibiotic usage. This has led to the demand for effective point-of-care diagnostic tools for human and animal health. On-farm, microbiological culture tests exist for applications such as mastitis, including the use of bi-plates or chromogenic tri-plates to identify the type of bacteria before mastitis treatment (Royster et al., Reference Royster, Godden, Goulart, Dahlke, Rapnicki and Timmerman2014) and DNA-based PCR technologies for bacteria identification to speed the turn-around time of test results (Hiitio et al., Reference Hiitio, Riva, Autio and Pohjanvirta2015). However, PCR systems require skilled professionals to interpret the results, are relatively expensive, and are not yet practical for use on-farm.
The objective of this clinical validation study, using milk samples from cows with clinical mastitis, was to compare the sensitivity and specificity of the Mastatest point-of-care diagnostic for a range of pathogens, including Streptococcus uberis and Staphylococcus aureus, with bacterial culture using the guidelines described by the National Mastitis Council (NMC, 2017). Antibiotic sensitivity testing was also performed to compare results from the point-of-care diagnostic with results determined following the Clinical and Laboratory Standards Institute (CLSI) guidelines (Clinical and Laboratory Standards Institute, 2009).
Materials and methods
Animals and management
Seven spring-calving herds (average herd size approximately 1000 milking cows) were selected from a group of Otago and Canterbury dairy herds in New Zealand. Clinical mastitis was identified by trained herd personnel using agreed criteria (milk from one or more quarters abnormal in colour, viscosity or consistency with or without accompanying heat, pain, swelling, redness of the udder, or generalised illness). A milk sample was collected aseptically from each affected quarter on enrolment. Samples were tested within 1 h after collection using the point-of-care diagnostic system and results were available within 24 h. The remaining milk was frozen within 1 h and stored at −20°C for later microbiological culture.
Bacterial culture and identification and antibiotic susceptibility testing
Bacteriology of the milk samples was performed by the reference laboratory Cognosco, Anexa FVC, Morrinsville, New Zealand (McDougall et al., Reference McDougall, Arthur, Bryan, Vermunt and Weir2007) based on the procedures recommended by the National Mastitis Council (NMC, 2004, 2017). For culture, identification and antibiotic susceptibility testing using the point-of-care diagnostic Mastatest (Mastaplex Ltd, New Zealand), milk samples (~1.3 ml) were loaded into a Mastatest cartridge and the manufacturer's instructions were followed.
For comparative analysis of antibiotic sensitivity testing between the point-of-care diagnostic and conventional culture-based techniques, three antibiotics commonly used in New Zealand to treat mastitis were examined: benzylpenicillin, cloxacillin and a lincomycin/neomycin combination. Six antibiotic concentrations: 4, 2, 1, 0.5, 0.1 and 0.05 mg/l (drug as free form) were selected for both techniques.
Laboratory based antibiotic sensitivity testing on 57 S. uberis isolates was carried out by broth microdilution following the CLSI guidelines (Clinical and Laboratory Standards Institute, 2009). Custom 24 well microplates were created which included the same serial antibiotic dilutions of benzylpenicillin, cloxacillin, and combination lincomycin/neomycin (1 : 1 ratio free form) as used by the point-of-care diagnostic system.
Statistical analysis
As comparing the performance of diagnostic tests is difficult, there being no ‘gold standard’ test available to unequivocally decide the true infection status of samples, latent class analysis (Hui and Zhou, Reference Hui and Zhou1998) was used with the true but unobserved infection status regarded as a latent trait. As the targeted parameters (sensitivities, specificities, and prevalences) are not estimable by the usual frequentist statistical methods (Jones et al., Reference Jones, Johnson, Hansen and Christensen2010), Bayesian analysis was used to give estimates by combining the observed data with assumptions about the model parameters (Dendukuri and Joseph, Reference Dendukuri and Joseph2001).
The posterior distributions were summarised by giving a point estimate (mean) and a measure of uncertainty (standard deviation); these are analogous to the estimate and standard error in a frequentist analysis. The differences Se2-Se1 and Sp2-Sp1 (where 1 denotes standard culture and 2 point-of-care diagnostic) were monitored for evidence of differences in the performances of the two tests: the probability that Se2 ⩾ Se1 (designated zSe) gave a Bayesian equivalent of the P-value: a small value (<0.05) could be regarded as evidence of the inferiority of point-of-care diagnostic with regard to sensitivity. Similarly, zSp summarised the evidence for inferior specificity.
Additional information related to the Materials and Methods section can be found in the Supplementary File.
Results
A total of 256 milk samples (87.7%) were identified as bacteria positive by standard culture-based techniques, and 269 samples were identified as positive by the point-of-care diagnostic (92.1%). No bacteria were detected in 13 samples, by either technique. The most common organism detected in the milk samples was S. uberis, with standard laboratory culturing detecting 195 (66.8%) S. uberis isolates, compared with 190 (65.1%) S. uberis isolates detected by the point-of-care diagnostic. S. aureus were found in nine samples by both tests, another nine samples were positive by standard culture, 17 samples were positive by the point-of-care diagnostic, and 257 samples were negative in both tests. CNS were found in 14 samples with the standard culture technique and in 11 samples with the point-of-care diagnostic. No CNS were detected in 267 samples (91.4%). Coliform bacteria are typically rare in clinical mastitis cases in New Zealand, demonstrated by the low number of positive samples detected by both techniques. Unclassified, but not contaminated, Gram-positive bacteria were detected in 56 samples by standard culture and in 57 samples by the point-of-care diagnostic (data not shown).
Each milk sample was categorised (bacteria present Yes/No) as positive for both tests (+ +), positive for standard culture but negative for the point-of-care diagnostic (+ −), and combinations thereof for S. uberis, S. aureus, coagulase negative staphylococci (CNS), and coliform species (Table 1).
Test results from 292 milk samples: ++ indicates a positive result on both tests; + − a positive for standard culture but negative for the point-of-care diagnostic; − + a negative for standard culture but positive for the point-of-care diagnostic; − − a negative result in both tests
Estimates of the sensitivities, specificities and derived quantities, based on the latent class analysis are shown in Table 2. The sensitivity and specificity for all bacterial targets was 91% (Se1) and 74% (Sp1) with the standard culture technique, and 95% (Se2) and 72% (Sp2) with the point-of-care diagnostic. The sensitivity and specificity for S. uberis for the standard technique was 89% (Se1) and 78% (Sp1) and for the point-of care diagnostic 88% (Se2) and 80% (Sp2) respectively. With regard to S. aureus, standard culture had a sensitivity (Se1) of 84% and specificity (Sp1) of 98% whereas the point-of-care diagnostic had a sensitivity of 88% (Se2) and a specificity of 96% (Sp2). Based on the statistical analysis, there was no evidence of inferiority of Mastatest, with regard to sensitivity or specificity when compared with conventional culture techniques for S. uberis and S. aureus. Indeed, two of the zSe values (which could be used as a measure of significance “P-value <0.05”), 0.968 for all bacteria targets and 0.976 for coliforms, suggested that Mastatest has higher sensitivity than the standard culture test for these targets: 0.946 versus 0.905 (P = 0.032) for all target bacterial species and 0.768 versus 0.536 (P = 0.024) for coliforms. Detailed results showing the initial prior distributions assumed for each parameter and the resulting posterior distributions are available in the Supplementary File.
Summaries of posterior distributions of the model parameters for each target organism: Se1 = sensitivity of standard culture; Se2 = sensitivity of point-of-care diagnostic; Sp1 = specificity of standard culture; Sp2 = specificity of point-of-care diagnostic; pi = prevalence of target organism; rhoD = correlation on true positive samples; zSe = probability that Se2 > Se1; zSp = probability that Sp2 > Sp1
The antimicrobial sensitivities of 57 S. uberis isolates as determined by Mastatest and the CLSI reference method as well as the MIC50 and MIC90 values are shown in Table 3. The distributions of MIC values differed between the antibiotics tested, with the MIC50 and MIC90 being lowest for benzylpenicillin for both testing methods. The MIC90 was highest for cloxacillin. The differences in MIC50 and MIC90 between the two testing methods were, at most, one order of dilution different. Overall, both testing methods show similar trends and are in good agreement.
Percentage distribution of minimum inhibitory concentrations (MIC) for different antibiotics against isolates of Streptococcus uberis (n = 57) from bovine milk samples determined by the point-of-care diagnostic and the reference method (CLSI), with the range of concentrations tested for each antibiotic
Discussion
This study was carried out to determine the sensitivity and specificity of a newly developed point-of-care diagnostic (Mastatest) compared with traditional laboratory bacterial culture. No significant differences were found between the tests in the detection of the mastitis-causing microorganisms: S. uberis, S. aureus, and coliforms. Very few coliforms were detected in the collected milk samples, and the specificity of 99.9% reflects this. To further evaluate the differences between the two techniques' ability to detect coliforms, further investigation is required. Finally, the low number of samples containing CNS, and the potentially low number of bacteria in those milk samples may be the reason why there were no positive results shared between the two testing methods.
In addition, antibiotic susceptibility testing carried out using both the point-of-care diagnostic and the CLSI reference method showed similar trends (maximum one dilution difference in MIC value). The point-of-care diagnostic was able to generate MIC data from bovine milk samples from clinical mastitis cases within 24 h compared with 3–5 d when samples were sent to a pathology lab. The key advantages of this point-of-care technology were that the milk samples did not require delivery to a laboratory, interpretation of results was carried out by qualified experts, and the turn-around time of 24 h enabled timely guidance on treatment decisions.
One limitation of this study was the absence of a gold-standard test to determine which samples were truly positive. It is possible that the assumed prior distribution for the sensitivity, specificity, and prevalence used in the Bayesian analysis were too influential and masked real differences in the performances of the tests. However, the values of zSe for All Organisms and coliforms gave small ‘P-values’ (1–0.968 = 0.032, 1–0.976 = 0.024) indicating the superior sensitivity of the point-of-care diagnostic; this indicates that the analysis allows for differences to manifest.
Relying on data from past studies to derive the prior distributions for parameters in a current analysis can be problematic, as previous studies are usually from different populations. This has been addressed by widening the prior distributions obtained from the Dohoo et al. (Reference Dohoo, Smith, Andersen, Kelton and Godden2011) study of Canadian dairy farms. The New Zealand data on the apparent prevalence of bacterial infections in clinical mastitis cases (McDougall et al., Reference McDougall, Arthur, Bryan, Vermunt and Weir2007) should be more relevant, but the final prevalence estimates for this study differ from those of McDougall et al. (Reference McDougall, Arthur, Bryan, Vermunt and Weir2007). This study used milk samples from approximately half of a lactation period whereas McDougall et al. (Reference McDougall, Arthur, Bryan, Vermunt and Weir2007) collected samples over one entire lactation. Because of New Zealand's seasonal lactation period, it is not surprising that the prevalence differs over the lactation season. Another limitation is that the point-of-care diagnostic analysis was carried out on fresh milk samples whereas the analysis in the lab was conducted on frozen samples. The potential issues are that bacteria classified as pathogens may have died during storage (NMC, 2004), or bacteria potentially trapped in somatic cells are released due to lysis of somatic cells during the freeze thaw cycle, enabling the growth of bacteria (Murdough et al., Reference Murdough, Deitz and Pankey1996). To overcome this issue fresh milk samples should be cultured in a laboratory, but this is logistically difficult under typical commercial farming conditions. Other possible limitations of the point-of-care diagnostic systems could be (i) that the internet is interrupted while running a test; (ii) mastitis milk samples require a dilution step in sterile water to ensure the samples flow into the point-of-care diagnostic.
The MIC50 and MIC90 of S. uberis against benzylpenicillin and cloxacillin found in the current study (Table 4) are similar (plus/minus one dilution) to those reported from a previous New Zealand study (McDougall et al., Reference McDougall, Hussein and Petrovski2014). This supports the validity of using Mastatest as an on-farm or vet-clinic point-of-care diagnostic. As results are sent to farmers and veterinarians, this process enables vets in conjunction with their farmer clients to make antibiotic treatment decisions to manage clinical mastitis cases (FDA, 2017).
Comparison of MIC values of S. uberis from New Zealand mastitis cases published by McDougall et al. (Reference McDougall, Hussein and Petrovski2014) (n = 102) compared with MIC values determined by the point-of-care diagnostic system (n = 57)
A new point-of-care diagnostic (Mastatest) has sensitivity and specificity characteristics similar to those of the standard laboratory-based culture tests. This test also determines an average equivalent MIC values compared to the method following the CLSI guidelines. The use of point-of-care diagnostic for real-time on-farm diagnosis facilitates appropriate early treatment of clinical mastitis, supports the antibiotic selection process for treatment, enables prudent use of antibiotics and offers the opportunity to globally monitor antibiotic sensitivities of commonly used antibiotics in different regions.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0022029919000177
Acknowledgments
The authors would like to acknowledge the help and support of colleagues at Vetlife and the Centre for Dairy Excellence in providing on-farm support and database management for this trial. We would also like to thank the farmers and their staff for hosting this work and for their enthusiasm and commitment.
