Introduction
Transition cow diseases are a set of disease conditions (e.g. mastitis, metritis, ketosis, displaced abomasum, and retained placenta) that occur during the transition period. The transition period for dairy cows is generally defined as approximately 3 weeks before to 3 weeks after calving (Drackley, Reference Drackley1999). During this time, disease susceptibility is greatest due to physiological changes that occur as the cow prepares for calving and transitions into milk production (Sordillo and Raphael, Reference Sordillo and Raphael2013). The enhanced energy demands during this time are accompanied by a diminished dry matter intake (DMI), resulting in a negative energy balance (NEB) (Hutjens and Aalseth, Reference Hutjens and Aalseth2005). NEB results in the mobilization of energy stores from adipose tissue (Ingvartsen, Reference Ingvartsen2006). These are normal physiological processes for cows during the transition period; however, failure to adequately maintain homeostasis may result in metabolic stress. Physiological processes that contribute to metabolic stress include abnormal nutrient utilization, oxidative stress, and unregulated inflammation (Sordillo and Mavangira, Reference Sordillo and Mavangira2014). Biomarkers representing altered nutrient metabolism typically measure lipid mobilization, which is defined as the imbalance between lipogenesis and lipolysis within the adipose tissue (Contreras and Sordillo, Reference Contreras and Sordillo2011). Oxidative stress is defined as an imbalance caused by increased production of reactive oxygen species (ROS) and the biological system's inability to detoxify the intermediates, or repair damage caused by them (Betteridge, Reference Betteridge2000). Lastly, inflammation typically occurs as a result of injury or tissue destruction. However, all dairy cows experience some degree of inflammation, especially postpartum, during the transition period (Bradford et al., Reference Bradford, Yuan, Farney, Mamedova and Carpenter2015). Inflammation manifests as a series of responses including the production of inflammatory mediators (Sordillo and Mavangira, Reference Sordillo and Mavangira2014). Although described separately, these three physiological processes (i.e. nutrient utilization, oxidative stress, and inflammation) are interrelated, often difficult to differentiate, and have the potential to form destructive feedback loops. When homeostasis is interrupted, metabolic stress occurs and is associated with poor uterine health and risk of transition cow diseases (Esposito et al., Reference Esposito, Irons, Webb and Chapwanya2014; Sordillo and Mavangira, Reference Sordillo and Mavangira2014).
Metabolic stress results in greater economic costs to the producer due to increased culling rates, treatment costs, impaired reproduction, and reduced milk yield (Melendez et al., Reference Melendez, Barolome, Archbald and Donovan2003; Petrovski et al., Reference Petrovski, Trajcev and Buneski2006; LeBlanc, Reference LeBlanc2008). Despite advances in research on transition cow disease etiology, prevention, and treatment, transition cow disease incidence overall has remained stable with few exceptions (LeBlanc et al., Reference LeBlanc, Lissemore, Kelton, Duffield and Leslie2006). For instance, the percent of cows reporting clinical mastitis has steadily increased from 13.4% in 1995 to 24.8% in 2013 (NAHMS, 1996, 2018).
Biomarkers that indicate the energy state of the animal are frequently used for health monitoring. These biomarkers include non-esterified fatty acids (NEFA), beta-hydroxybutyrate (BHB), calcium, and body condition score (BCS). Threshold values for NEFA, BHB, and calcium have been established at the individual cow level to indicate high disease risk. To make whole herd decisions, usually a proportion of the herd is sampled. Then, if there is a percentage of the herd above a certain threshold of NEFA or BHB, dietary and environmental interventions are implemented. At the individual cow level, the sensitivity and specificity for serum BHB ≥1.4 mmol l−1 measured in the first and second weeks after calving for detecting subclinical ketosis ranges from 87 to 93% and 93 to 100%, respectively (LeBlanc, Reference LeBlanc2010). However, BHB has only been demonstrated as a valuable biomarker post parturition, when it is too late for proactive interventions to reduce disease incidence in that particular calving cohort. On the other hand, NEFA thresholds have been utilized for both pre-partum and post-partum cows, with threshold levels ranging from approximately 0.30 to 0.40 mmol l−1 (Oetzel, Reference Oetzel2004; Ospina et al., Reference Ospina, McArt, Overton, Stokol and Nydam2013). However, Ospina et al. (Reference Ospina, Nydam, Stokol and Overton2010) reported lower sensitivity and specificity of NEFA measured pre-partum (threshold of ≥0.39 mmol l−1) compared with post-partum (threshold of ≥0.57 mmol l−1) for detection of any of the following diseases: displaced abomasum, clinical ketosis, metritis, or retained placenta (Ospina et al., Reference Ospina, Nydam, Stokol and Overton2010). Pre-partum sensitivity and specificity were 48% (95% CI: 42–54) and 69% (95% CI: 67–72), while post-partum sensitivity and specificity were 75% (95% CI: 66–82) and 61% (95% CI: 58–64) (Ospina et al., Reference Ospina, Nydam, Stokol and Overton2010). Calcium concentrations can be used to identify hypocalcemia, which occurs when calcium levels are below the normal reference range of 8.5–10 mg dl−1 (Goff, Reference Goff2008a). Calcium serum levels below 9.4 mg dl−1 measured 1-week pre-partum had very low sensitivity at 35% for displaced abomasum, while specificity was 71% (Chapinal et al., Reference Chapinal, Carson, Duffield, Capel, Godden, Overton and Santos2011). Herd-level sensitivity is negatively affected by low sensitivity at the individual level, so it is imperative to have high test sensitivity, especially pre-partum when there is still time to intervene (Ospina et al., Reference Ospina, McArt, Overton, Stokol and Nydam2013).
A potential method to increase sensitivity in detecting cattle at risk of transition diseases is to use multiple biomarkers that can capture the physiological complexity of metabolic stress. For instance, metabolic syndrome in humans, an analogous condition to metabolic stress in cows, is diagnosed by measuring a cluster of cardiometabolic biomarkers (IDF, 2006; Sordillo and Mavangira, Reference Sordillo and Mavangira2014). The biomarkers used for metabolic syndrome measure different processes involved, including insulin resistance, visceral adiposity, dyslipidemia, glucose intolerance, hypertension, proinflammatory state, and a prothrombotic state (Huang, Reference Huang2009). A patient does not need to present with all criteria to be diagnosed with metabolic syndrome (IDF, 2006). Metabolic stress in dairy cattle is very similar to metabolic syndrome in dairy cattle, in that both conditions involve interconnected physiological processes, where some or all processes are dysregulated. For metabolic stress in dairy cattle, the three major physiological processes involved are altered nutrient metabolism, oxidative stress, and inflammation. Biomarkers covering these three pillars of metabolic stress need to be implemented in monitoring so that dairy cattle undergoing metabolic stress may be better identified.
Even when multiple biomarkers and other monitoring tools (e.g. milk production, feed intake) are assessed together in multivariable analyses, the models are typically explanatory and do not aim for prediction and application in the field. It is important to understand the differences between predictive and explanatory methods before beginning analysis, because variable selection, model evaluation procedures, and model validation vary greatly between the two methods. Explanatory modeling is used to test causal explanations between the predictors and the outcome, while predictive modeling is used if the goal is to predict future or current events (Sainani, Reference Sainani2014). Since the aim is to identify cattle that will later develop disease as a result of metabolic stress, the focus should be on predictive modeling. Often, methods between explanatory and predictive modeling are conflated, and researchers assume strong explanatory models have high predictive power, which is not always the case (Shmueli, Reference Shmueli2010).
This paper will review predictive modeling methods in the context of transition cow diseases. Herein, we will discuss what predictive modeling is, suggestions for variable selection, study design considerations, and statistical analysis tools.
Predictive versus explanatory modeling
Explanatory modeling
Many studies assessing risk factors for transition diseases use explanatory methods, which are used to investigate or hypothesize a possible causal pathway between the variables (LeBlanc et al., Reference LeBlanc, Leslie and Duffield2005; Huzzey et al., Reference Huzzey, Veira, Weary and von Keyserlingk2007; Dubuc et al., Reference Dubuc, Duffield, Leslie, Walton and LeBlanc2010; Seifi et al., Reference Seifi, LeBlanc, Leslie and Duffield2011; Sainani, Reference Sainani2014; Vanholder et al., Reference Vanholder, Papen, Bemers, Vertenten and Berge2015). For example, Potter et al. (Reference Potter, Guitian, Fishwish, Gordon and Sheldon2010) explored potential risk factors for clinical endometritis. The study reported the relative risk (RR) and population attributable risk fraction (which assumes causality) between significant risk factors (e.g. gender of calf, assisted calving, and parity) and development of endometritis (Potter et al., Reference Potter, Guitian, Fishwish, Gordon and Sheldon2010). The methods were explanatory in that the aim was to explore a mechanism behind the development of disease. In explanatory modeling, potential variables are chosen based on their hypothesized role in the causal pathway. It is important to consider the primary predictors of interest, as well as a priori confounders, and variables that are noted to be confounders in the dataset (Dohoo et al., Reference Dohoo, Martin and Stryhn2003). Automated selection procedures, such as stepwise and best subset selection, are criticized for their use in explanatory models because they optimize model fit with no regard to the role of each variable (Sainani, Reference Sainani2014). Selection of variables is generally driven by P-values and the size of the coefficients (Sainani, Reference Sainani2014). Explanatory models are evaluated based on explanatory power and the strength of the relationship between explanatory variables and the outcome indicated by the model, and then validated based on model fit and internal validity (Shmueli, Reference Shmueli2010).
Predictive modeling
To the best of our knowledge, only a few studies used predictive modeling methods to model disease risk in transition dairy cattle (Vergara et al., Reference Vergara, Dopfer, Cook, Nordlund, McArt, Nydram and Oetzel2014; Wisnieski et al., Reference Wisnieski, Norby, Pierce, Becker, Gandy and Sordillo2019a, Reference Wisnieski, Norby, Pierce, Becker, Gandy and Sordillo2019b, Reference Wisnieski, Norby, Pierce, Becker, Gandy and Sordillo2019c). As opposed to explanatory modeling, predictive modeling is focused on predicting future or current events, rather than causal explanations. In predictive modeling, variables are chosen based on association, data quality, and availability of the predictors before the event of interest (Shmueli, Reference Shmueli2010). The main goal of variable selection is to include the predictors that most improve the predictive value of the model (Sainani, Reference Sainani2014). Automated variable selection procedures are allowed, but it is suggested that Akaike's information criteria (AIC) or Bayesian information criteria (BIC) are used instead of the statistical significance and size of the coefficients of the variables (Shtatland et al., Reference Shtatland, Cain and Barton2001; Shmueli, Reference Shmueli2010; Sainani, Reference Sainani2014). This is because AIC and BIC focus on the strength of evidence for a model rather than hypothesis testing (Mazerolle, Reference Mazerolle2006) and balance model performance against model complexity (Burnham and Anderson, Reference Burnham and Anderson2004). Predictive models are evaluated based on information criteria, discrimination (e.g. receiver operator characteristic [ROC] curve analysis), calibration, and goodness-of-fit (Sainani, Reference Sainani2014), and validated based on internal and external validity (Shmueli, Reference Shmueli2010; Sainani, Reference Sainani2014).
In the end, explanatory and predictive models can produce greatly different results. In addition to their different goals and variable selection strategies, explanatory and predictive models also differ regarding the bias-variance trade-off. Explanatory models aim to minimize bias through obtaining the most accurate representation of the underlying theory, while predictive modeling aims to minimize bias and estimation variance, sometimes sacrificing accuracy for precision (Shmueli, Reference Shmueli2010). It is likely that the variables, explanatory power, and predictive ability differ between explanatory and predictive models (Shmueli, Reference Shmueli2010). In an example from human medicine, Singal et al. (Reference Singal, Mukherjee, Elmunzer, Higgins, Lok, Zhu, Marrero and Walijee2013) compared two models for prediction of hepatocellular carcinoma. One model was a conventional regression model built with explanatory modeling methods and the other was a predictive forest plot built using machine learning techniques. The authors found that both the variables and the predictive ability differed between the two methods, with the forest plot having the higher predictive ability (Singal et al., Reference Singal, Mukherjee, Elmunzer, Higgins, Lok, Zhu, Marrero and Walijee2013). There is a need in transition cow research to consider predictive modeling techniques to determine which variables are most predictive, rather than explanatory.
Candidate variables
Candidate variables that have either demonstrated the ability to predict or are associated with transition cow diseases in prior studies should be considered during study design. Practical considerations, including ease of measurement and analysis, reliability, and affordability, should be incorporated in variable choice. Although not comprehensive, this section will review why some select variables merit attention for modeling transition cow diseases.
Biomarkers
There are generally three major classes of biomarkers: biomarkers of exposure, effect, and susceptibility. Biomarkers of exposure are generally used to assess environmental exposure to a substance, biomarkers of effect measure the success of a treatment, and biomarkers of susceptibility are used to identify who is at risk of a certain disease (Grandjean, Reference Grandjean1995). For predictive disease modeling, biomarkers of susceptibility are of most utility as they can identify cows that have an elevated risk of disease. Specifically, biomarkers that are indicators of the underlying physiological processes that cause transition cow disease. As mentioned earlier, metabolic stress brought on by the physiological transition from dry-off to early lactation is a key contributor to disease development. Effective and predictive biomarkers for metabolic stress are needed to identify cattle that are not transitioning smoothly. There are three components of metabolic stress that should be considered when choosing potential biomarkers: nutrient utilization, oxidative stress, and inflammation (Sordillo and Mavangira, Reference Sordillo and Mavangira2014). As mentioned earlier, typically lipid mobilization markers are used to identify aberrant nutrient mobilization. Although the lipid mobilization biomarkers, including NEFA, BHB, and calcium, have immense value in the identification of cattle undergoing metabolic stress, these biomarkers only capture some of the physiological changes that occur. In an analogous condition in human beings, metabolic syndrome, people typically only need to have three or more cardiometabolic criteria to be diagnosed with metabolic syndrome. Although the underlying physiological processes are interconnected, people with metabolic syndrome may not present with the same risk factors (IDF, 2006). Other groups have recognized that this is also true of metabolic stress in dairy cattle in that multiple biomarkers are needed to capture the variability in how metabolic stress presents (Ingvartsen et al., Reference Ingvartsen, Dewhurst and Friggens2003; Celi, Reference Celi2010; Abuelo et al., Reference Abuelo, Hernandez, Benedito and Castillo2013; Sordillo and Mavangira, Reference Sordillo and Mavangira2014; Wisnieski et al., Reference Wisnieski, Norby, Pierce, Becker, Gandy and Sordillo2019a). There is a need to develop biomarkers that represent other physiological processes involved in metabolic stress, oxidative stress and inflammation, to better identify cows at risk. Although not exhaustive, specific oxidative stress and inflammation biomarkers that deserve consideration will be discussed in the following sections.
Oxidative stress biomarkers
The measurement of both pro- and anti-oxidants is essential in estimating an animal's oxidative status. Abuelo et al. (Reference Abuelo, Hernandez, Benedito and Castillo2013) showed that the use of an oxidative stress index that measured both pro- and anti-oxidants was superior compared with using pro- and anti-oxidants separately. Pro-oxidants, such as ROS, are inherently unstable molecules due to unpaired electrons (Betteridge, Reference Betteridge2000). Although produced in response to normal metabolic activities, they can cause substantial damage in tissues when produced in excess (Sordillo and Aitken, Reference Sordillo and Aitken2009). The action of ROS can be counteracted by antioxidants, which are electron donors. There are a variety of pro- and anti-oxidant biomarkers as described in Celi (Reference Celi2010). This allows for flexibility depending on the researcher's resources and capabilities. For example, the measurement of Trolox equivalent antioxidant capacity is an extremely fast and simple measurement using spectrometry (Celi, Reference Celi2010). However, this method has varying results with sample dilution and varying specificity (Celi, Reference Celi2010).
Many vitamins also have anti-oxidant capabilities and can serve as biomarkers for transition diseases. Specifically, vitamins E and A, and their derivatives, are excellent candidates. Vitamins E and A are important cellular antioxidants and reduce the risk of clinical mastitis when supplemented in feed (LeBlanc et al., Reference LeBlanc, Herdt, Seymour, Duffield and Leslie2004; Spears and Weiss, Reference Spears and Weiss2008; Sordillo and Aitken, Reference Sordillo and Aitken2009; Yang and Li, Reference Yang and Li2015). Not only do vitamins E and A have anti-oxidant roles, but also decrease inflammatory markers and support the immune system when supplemented in feed (Baldi, Reference Baldi2005; McDowell et al., Reference McDowell, Wilkinson, Madison and Felix2007), hence they can also be considered inflammatory biomarkers. Vitamins E and A have been studied in dairy cattle for more than 20 years, which speaks on the strength of evidence for their involvement in dairy cattle health (Weiss, Reference Weiss1998; Baldi, Reference Baldi2005).
Inflammatory biomarkers
Acute phase proteins (APPs) and white blood cells are key players in systemic inflammation. Tissue injury or infections stimulate inflammatory responses that increase APPs, including haptoglobin, serum amyloid A (SAA), and albumin (Yang et al., Reference Yang, Lee, Hu, Pollin, Ryan, Nicklas, Snitker, Horenstein, Hull, Goldberg, Goldberg, Shuldiner, Fried and Gong2006; Ceciliani et al., Reference Ceciliani, Ceron, Eckersall and Sauerwein2012; Pepys, Reference Pepys2012). White blood cells travel to the area of infection or injury and remove foreign cells, dead cells, or microorganisms. They dissipate once the stimulus disappears (Malarkey and McMorrow, Reference Malarkey and McMorrow2011). The use of assays for APP in dairy cattle has grown in popularity for investigating infectious and inflammatory diseases, thereby resulting in increased availability of assays that measure APP concentrations (Ceciliani et al., Reference Ceciliani, Ceron, Eckersall and Sauerwein2012). In addition, epidemiological evidence has demonstrated relationships between APP and disease in dairy cattle. For example, a group in Canada found that haptoglobin and SAA were associated with multiple disease outcomes in transition dairy cattle, including retained placenta, mastitis, and lameness (Dervishi et al., Reference Dervishi, Zhang, Hailemariam, Dunn and Ametaj2015, Reference Dervishi, Zhang, Hailemariam, Dunn and Ametaj2016; Zhang et al., Reference Zhang, Hailemariam, Dervishi, Deng, Goldansaz, Dunn and Ametaj2015). Another group found that high albumin levels pre-calving significantly increased the risk of a disease event (Saun, Reference Saun2004). Wisnieski et al. (Reference Wisnieski, Norby, Pierce, Becker, Gandy and Sordillo2019c) demonstrated that the average concentration of albumin in a cohort at dry-off was a good predictor of cohort-level early lactation disease incidence.
APPs can recruit white blood cells to the area of infection. For instance, SAA recruits monocytes to inflamed tissues (Badolato et al., Reference Badolato, Wang, Murphy, Lloyd, Michiel, Bausserman, Kelvin and Oppenheim1994). In many studies, leukocytes, including neutrophils, lymphocytes, and macrophages, are associated with uterine diseases (Kulberg et al., Reference Kulberg, Storset, Heringstad and Larsen2002; Hammon et al., Reference Hammon, Evjen, Dhiman, Goff and Walters2006; Goff, Reference Goff2008b; Galvão et al., Reference Galvão, Flaminio, Brittin, Sper, Fraga, Caixeta, Ricci, Guard, Butler and Gilbert2010). Milk somatic cell counts, which are comprised mostly of leukocytes, are used to diagnose subclinical mastitis in dairy cattle (Barkema et al., Reference Barkema, Schukken, Lam, Beiboer, Benedictus and Brand1998). The strong support for APP and leukocytes in relation to transition cow disease outcomes lends support for using these biomarkers in disease prediction.
Vitamin D is an optimal candidate biomarker for transition diseases due to the strong connection of vitamin D with inflammatory conditions in human beings, including obesity and metabolic syndrome (Autier et al., Reference Autier, Boniol, Pizot and Mullie2014). Vitamin D is involved in both acute and chronic inflammation, and has potent immunomodulatory properties (Gombart, Reference Gombart2012). Vitamin D regulates the production of cytokines and inhibits the production of proinflammatory cells (Yin and Agrawal, Reference Yin and Agrawal2014). In addition to its role in disease prevention, vitamin D is involved in calcium homeostasis, which is an important lipid mobilization biomarker involved in metabolic stress (Horst et al., Reference Horst, Goff and Reinhardt1994). In transition dairy cattle, a vitamin D deficiency may lead to hypocalcemia, or milk fever (DeGaris and Lean, Reference DeGaris and Lean2008).
Health record
Farm health records include information such as lactation number, DMI, disease history, milk yield, and milk quality. Most of these data are already routinely collected from farms, which makes them ideal candidate variables to consider for predictive modeling. Increasing lactation number, or parity, is associated with greater incidence of disease, including milk fever and mastitis (DeGaris and Lean, Reference DeGaris and Lean2008; Abebe et al., Reference Abebe, Hatiya, Abera, Megersa and Asmare2016). There are also many behavioral differences between heifers (cattle on their first lactation) and cows (cattle on their second or greater lactation), such as time spent feeding and frequency of visits to the feed bunk, which may affect DMI (Neave et al., Reference Neave, Lomb, von Keyserlingk, Behnam-Shabahang and Weary2017). Lower pre-partum DMI is associated with increased risk of metritis and milk fever (Goff and Horst, Reference Goff and Horst1997; Huzzey et al., Reference Huzzey, Veira, Weary and von Keyserlingk2007). Disease history can offer insight into the capability of a cow's immune system. For instance, cows infected with bovine leukemia virus have dramatic shifts in lymphocyte function, which could cause a higher risk for subclinical and clinical mastitis (Sordillo and Erskine, Reference Sordillo and Erskine2010). Milk yield, both from the current lactation and previous lactations, is also associated with disease risk. Cows with higher milk yield in the previous lactation have increased risk of retained placenta, mastitis, milk fever, and metritis (Grohn et al., Reference Grohn, Eicker and Hertl1995; Fleischer et al., Reference Fleischer, Metzner, Beyerbach, Hoedemaker and Klee2001), whereas higher milk yield in the current lactation is associated with a greater risk of milk fever (Fleischer et al., Reference Fleischer, Metzner, Beyerbach, Hoedemaker and Klee2001).
Housing and nutritional management
Similar to information from the health record, information about housing and nutritional management is readily available. A comfortable environment is critical for cow health. Comfortable stalls and walkways, soft mats, or deep bedding lower the risk of lameness (Cook and Nordlund, Reference Cook and Nordlund2009; de Vries et al., Reference de Vries, Bookers, van Reenen, Engel, van Schaik, Dijkstra and de Boer2015). Proper shade and cooling systems prevent heat stress, thereby improving DMI (Collier et al., Reference Collier, Dahl and VanBaale2006). In addition, cow health improves with longer stall length and greater freedom to move (e.g. free stalls instead of tie stalls) (Cook, Reference Cook2003; de Vries et al., Reference DeVries, von Keyserlingk and Weary2004; Zurbrigg et al., Reference Zurbrigg, Kelton, Anderson and Millman2005; Simensen et al., Reference Simensen, Ostera, Boe, Kielland, Rudd and Naess2010). Pen changes and stocking density should also be considered during variable selection. Each pen change results in the establishment of a rank order. Housing multiparous cows separately from heifers, avoiding overstocking, and careful management of maternity and sick pens can reduce social turmoil and disease incidence (Nordlund et al., Reference Nordlund, Cook and Oetzel2006). The nutritional composition of feed can also be incorporated in disease prediction. Proper nutritional management can quell the severity and length of NEB and heat stress during early lactation (Santos et al., Reference Santos, Bisinotto, Ribeiro, Lima and Thatcher2012; Das et al., Reference Das, Sailo, Verma, Bharti, Saikia, Kumar and Kumar2016). Supplements in feed (e.g. zinc, copper, vitamin E, selenium, and beta-carotene) can reduce the incidence of many transition cow diseases, including clinical mastitis, metritis, and retained placenta (Spears and Weiss, Reference Spears and Weiss2008; Heinrichs et al., Reference Heinrichs, Costello and Jones2009). Information from health records and on housing and nutritional management are convenient variables that have consistent associations with disease. It is considered good practice to include a large pool of candidate variables when building predictive modeling, so that any information that can be incorporated from health records, housing, and nutritional management should be incorporated in model selection.
Study design considerations
When designing a study, it is important to consider the goal of the analysis and the characteristics of the target population. Noteworthy considerations for dairy cattle include: (1) the decision to estimate individual versus group-level predictions, (2) the clustering of the cows within farms, (3) the differences between primiparous and multiparous cows, and (4) the presence of critical time-points in the lactation cycle.
Individual versus group-level predictions
Dairy herd managers are often concerned about health at the herd level. Health at the herd/cohort level can be assessed using incidence rates, which are defined as the number of new cases of a disease that occur during a specified time of interest in the population of interest (Dohoo et al., Reference Dohoo, Martin and Stryhn2003). The cohort level, which is defined as cows at the same stage of lactation (e.g. dry-off, close-up, and calving), may be more useful in targeting immediate interventions. This is because it offers information on the most current group of cows undergoing the transition from pregnancy to lactation.
The choice of individual versus group-level prediction will guide which predictors are collected. If the goal is individual-level prediction, using only cohort-level data (e.g. group feed intake) may not capture the heterogeneity of cows within a cohort. This can result in assigning group-level characteristics to individuals that do not possess those characteristics, which is known as the “ecologic fallacy” (Gordis, Reference Gordis2014). However, cohort-level predictors are often already collected and can reflect important management factors that exert contextual effects on individual cows. There are other sources of cohort-level variation in outcomes though, such as uneven composition of cohorts with respect to individual-level predictors (Duncan, Jones, and Moon, Reference Duncan, Jones and Moon1998). Therefore, it is recommended to use both individual-level and cohort-level predictors, to avoid both the ecologic and atomistic fallacies, and to study both contextual and compositional effects (Duncan et al., Reference Duncan, Jones and Moon1998).
Individual-level data can also be aggregated to the group level. In Wisnieski et al. (Reference Wisnieski, Norby, Pierce, Becker, Gandy and Sordillo2019b), we demonstrated the utility in aggregating individual-level biomarkers and covariate data to predict disease incidence at the cohort level (Wisnieski et al., Reference Wisnieski, Norby, Pierce, Becker, Gandy and Sordillo2019a, Reference Wisnieski, Norby, Pierce, Becker, Gandy and Sordillo2019b, Reference Wisnieski, Norby, Pierce, Becker, Gandy and Sordillo2019c). In another previous study, we demonstrated that individual-level disease predictions calculated from individual-level models can be aggregated to predict cohort-level disease incidence (Wisnieski et al., Reference Wisnieski, Norby, Pierce, Becker, Gandy and Sordillo2019a, Reference Wisnieski, Norby, Pierce, Becker, Gandy and Sordillo2019b).
Hierarchical or ‘clustered’ data
A cluster refers to a group of observations that share some common features. For instance, clusters may be formed from individuals that share a common environment or from repeated measures collected from the same individual (Dohoo et al., Reference Dohoo, Martin and Stryhn2003). For dairy cattle, potential cluster variables may include factors such as farm or pen. Careful calculation of sample size is important when using clustered data. For instance, when using farm as a cluster variable, the estimated number of farms and number of animals needed will depend on the variation in the covariates between farms compared with within a farm. This estimate can be calculated using the intraclass correlation coefficient (ICC) and an adjustment to the sample size calculation can be made if needed. Clustered samples typically require a higher sample size than simple random sampling. The ICC ranges from 0 to 1: a high ICC (closer to 1) is indicative that the variation within a group is small relative to between groups, and a low ICC (closer to 0) means that most of the variation is within the groups (Dohoo et al., Reference Dohoo, Martin and Stryhn2003). As ICC increases, the optimal number of clusters increases and the sample size per cluster decreases (van Breukelen and Candel, Reference van Breukelen and Candel2012). For example, if the ICC estimating the variance between farms compared with the variance within farms was high, a greater number of farms would need to be enrolled. This would correspond to a smaller number of cows per farm needed.
Analyses must also be adjusted to account for the hierarchical structure of data. If clustering is not considered when it is present, the standard errors of the estimates may be underestimated. This increases the risk of falsely rejecting null hypotheses (making type-I errors by concluding there is a difference when there is not really one). Different methods to account for clustering include stratifying the model by cluster, adjusting the standard errors, including the cluster as a fixed effect, or including the cluster as a random effect (Dohoo et al., Reference Dohoo, Martin and Stryhn2003). However, predictive models for transition diseases should aim to be generalizable to different populations (e.g. farms). To build predictive models that are generalizable to different farms, farm must be used as a clustering variable and represented as a random effect (Galbraith et al., Reference Galbraith, Daniel and Vissel2010). That is only viable when a sufficient number of farms are available in the sample. There may also be more than one clustering variable in a given study. For instance, cows may be nested in cohorts, which are defined as groups of cows expected to calve around the same time. In this case, three-level models can be used to adjust for both cohort and farm; however, sample size must be sufficient at both levels (Nezlek, Reference Nezlek2008).
Primiparous and multiparous models
There are many physiological differences between primiparous and multiparous cattle. For example, primiparous and multiparous cattle have differing concentrations of many biomarkers that are associated with disease risk (e.g. BHB, leptin, and NEFA) throughout the lactation cycle (Wathes et al., Reference Wathes, Cheng, Bourne, Taylor, Coffey and Brotherstone2007). In addition, as mentioned previously, multiparous cattle have a greater disease risk than primiparous. Including parity as a predictor in models may not be sufficient to adjust for the large physiological differences between primiparous and multiparous cows. It may be advantageous to either build separate predictive models for primiparous and multiparous cattle or allow for interactions between parity and other variables in the same model. For example, Huzzey et al. (Reference Huzzey, Nydam, Grant and Overton2011) reported separate models for primiparous and multiparous cattle for the associations between cortisol and transition disease outcomes. The authors found that higher cortisol levels were protective for disease in primiparous cows but increased the risk for disease in multiparous cows (Huzzey et al., Reference Huzzey, Nydam, Grant and Overton2011). This indicates differences in the physiological control of lipid mobilization in heifers and cows (Wathes et al., Reference Wathes, Cheng, Bourne, Taylor, Coffey and Brotherstone2007). Therefore, the relationships between lipid mobilization biomarkers and disease outcomes may also be different in heifers and cows.
Multiparous cows that have had previous lactations have additional information available in their health record that may improve disease prediction, including prior milk yield, peak milk, and days in milk (Grohn et al., Reference Grohn, Eicker and Hertl1995; Fleischer et al., Reference Fleischer, Metzner, Beyerbach, Hoedemaker and Klee2001). If primiparous and multiparous cattle were combined in the same model, primiparous cows would have empty values for these variables and would be excluded from the analysis. To use all information available for prediction, separate models may be best. However, it is important to note that separate models have implications for sample size. Researchers must ensure ahead of sampling that they will have an adequate number of primiparous and multiparous cows for meaningful statistical power in both models. If sample size permits, researchers should explore separate models for primiparous and multiparous cows, given their many differences.
Critical time-points
There are critical time points during the lactation cycle that are indicative of huge physiological shifts and important management decisions. For instance, dry-off is the time point in the lactation cycle where the farm staff stop milking the cows in preparation for the next calving. At this time, cows receive vaccinations and dry cow treatment regimens that are associated with later risk of diseases (Green et al., Reference Green, Bradley, Medley and Brown2007). In addition, Putman et al. (Reference Putman, Brown, Gandy, Wisnieski and Sordillo2018) found that many metabolic biomarkers associated with metabolic stress in early lactation fluctuated in the immediate days following dry-off. Nutritional management of dry cows is also critical to aid calcium metabolism and to control the release of NEFA from adipose tissue (Overton and Waldron, Reference Overton and Waldron2004). In the period from late pregnancy to early lactation, all dairy cattle undergo physiological stresses, including insulin resistance, reduced immune function, and bacterial contamination of the uterus that increases risk of disease (LeBlanc, Reference LeBlanc2010). Dairy cattle researchers should aim to design studies that capture these critical time-points through a longitudinal study design. Thus, changes between the time-points can be monitored and optimal times for intervention strategies during the lactation cycle may be elucidated.
Statistical analysis
There are several types and variations of statistical analyses that can be used to create predictive disease models. However, it is the researchers' responsibility to choose a type of analysis that will work best for their research question and dataset. A few topics to consider when choosing a statistical model for transition disease prediction are as follows: (1) purpose of the analysis (e.g. farm or individual cow level predictions), (2) sample size availability, (3) scale of independent and dependent variables (e.g. continuous, categorical, and ordinal), and (4) if the data can satisfy the assumptions of the analysis (McCrum-Gardner, Reference McCrum-Gardner2008). The following section reviews statistical analysis options to consider before model building.
Predictive models
Logistic regression
Logistic regression models are well-suited for testing the association between a binary outcome and continuous and/or categorical predictors (Peng et al., Reference Peng, Lee and Ingersoll2002). In the case of modeling transition diseases, our binary response variables would be coded as 0 (i.e. no disease) or 1 (i.e. disease) (Long and Freese, Reference Long and Freese2006). The coefficients in the logistic regression may be converted to odds ratios (ORs) so that researchers can evaluate how each covariate contributes to prediction (Hosmer et al., Reference Hosmer, Lemeshow and Sturdivant2013). The coefficients can also be converted into RRs for prospective data within the generalized linear regression framework. RR estimates are more easily interpreted than ORs (Schmidt and Kohlmann, Reference Schmidt and Kohlmann2008). Logistic regression requires that the probabilities are a logistic function of the independent variables, that the model includes all relevant variables, that the observations are independent and measured without error, and that the variables are not linear combinations of each other (Harrell, Reference Harrell2015). In the literature, there are innumerable examples of studies that used logistic regression to model transition disease outcomes. For example, LeBlanc et al. (Reference LeBlanc, Herdt, Seymour, Duffield and Leslie2004) investigated the associations of vitamin E, retinol, and beta-carotene with retained placenta and clinical mastitis in two separate models. The authors found that pre-partum alpha tocopherol was negatively associated with risk of retained placenta, and retinol levels were negatively associated with risk of clinical mastitis (LeBlanc et al., Reference LeBlanc, Herdt, Seymour, Duffield and Leslie2004). In another study, Huzzey et al. (Reference Huzzey, Veira, Weary and von Keyserlingk2007) used logistic regression analysis to test the associations of pre-partum feeding time and DMI with metritis. The authors found that increased feeding time and DMI were associated with a lower risk of metritis (Huzzey et al., Reference Huzzey, Veira, Weary and von Keyserlingk2007). Wisnieski et al. (Reference Wisnieski, Norby, Pierce, Becker, Gandy and Sordillo2019a) used logistic regression analyses to build individual cow-level models aim to predict an aggregate disease outcome (e.g. mastitis, metritis, retained placenta, and other transition diseases). The best model, which included multiple biomarkers of metabolic stress, had high sensitivity and specificity (both >85%) (Wisnieski et al., Reference Wisnieski, Norby, Pierce, Becker, Gandy and Sordillo2019a).
The use of logistic regression is widespread, probably due to its ease of interpretation and its relationship to analyses of contingency tables (Lemeshow and Hosmer, Reference Lemeshow and Hosmer1982). Logistic regression models can accommodate categorical predictors, which are essential in transition disease prediction where many potential predictors are categorical (e.g. parity, prior disease, and co-morbidities). Another strength of logistic regression analysis for predictive modeling is that interaction terms can be used. Interaction occurs when the effect of two variables together is different than the sum of their individual effects (Dohoo et al., Reference Dohoo, Martin and Stryhn2003). Including interaction terms can improve predictive performance (Kerr and Pepe, Reference Kerr and Pepe2011). Logistic regression models are excellent candidates for transition disease models and should be considered if the researcher has access to statistical packages. However, in the case of small sample sizes, perfect prediction or empty cells of certain covariate patterns are more likely to occur. This leads to unstable coefficients and standard errors, and may lead to the exclusion of important covariates (Zorn, Reference Zorn2005). Exact logistic regression, a variation of logistic regression, can be used to remedy this issue. Exact logistic regression handles small sample sizes and empty cells by constructing a statistical distribution that can be completely enumerated (Hosmer et al., Reference Hosmer, Lemeshow and Sturdivant2013).
Discriminant function analysis
Discriminant function analysis is used to determine which continuous variables discriminate between two or more groups (McLachlan, Reference McLachlan2004). For instance, a researcher may investigate which variables discriminate between cows that develop mastitis and those that do not. Discriminant analysis is a two-step process: testing significance of a set of discriminant functions, which is computationally identical to multivariate analysis of variance (MANOVA), followed by classification. During classification, discriminant analysis automatically determines which set of variables best discriminates between the groups (McLachlan, Reference McLachlan2004).
Discriminant analysis answers the same basic questions as logistic regression, but has additional assumptions. Discriminant analysis has the same assumptions as MANOVA, including equal sample sizes between groups, a normal distribution of the data, homogeneity of variances/covariances, absence of outliers, and independent means and variances. In addition, discriminant analysis requires that the variables used to discriminate between groups are not highly correlated, which can be determined by calculating tolerance values (Klecka, Reference Klecka1980).
To the best of our knowledge, discriminant factor analysis has not yet been used to model transition cow diseases but has been used to investigate multifactorial conditions in human beings, including metabolic syndrome (Leite et al., Reference Leite, Nicolosi, Firmo and Lima-Costa2007; Corica et al., Reference Corica, Corsonella, Apolone, Mannuncci, Lucchetti, Bonfiglio, Melchoinda and Marchesini2008). A benefit of using discriminant analysis is that it can handle smaller sample sizes than logistic regression. However, as mentioned earlier, another option for small sample sizes is exact logistic regression. Exact logistic regression may be a better option than discriminant analysis in transition dairy cattle disease studies, because discriminant analysis assumptions are often difficult to meet. Discriminant analysis requires that the groups (disease versus no disease) are approximately equal in size. It is unlikely that close to half of all cows will present with clinical disease. For instance, clinical mastitis, which is the most common transition disease, only has a median incidence risk of 14.2% (Van Saun and Sniffen, Reference Van Saun and Sniffen2014). This issue can be addressed through retrospective studies (e.g. case-control) that can select equal numbers of diseased and non-diseased cows. In prospective studies (e.g. cohort), this can be addressed by collecting data on a much larger sample, then random subsampling from the healthy group to get balanced sample sizes. However, this option may not be cost-effective. Another consideration for discriminant analysis is that it does not handle categorical variables as well as logistic regression (Press and Wilson, Reference Press and Wilson1978), so researchers may have to exclude important predictors, such as parity.
Factor analysis and structural equation modeling
The intent of factor analysis is to determine the number or nature of factors that account for variation and covariation among a set of variables or indicators (Brown, Reference Brown2015). In other words, the goal is dimension reduction: combining many indicator variables into a smaller number of latent factors that are each broader constructs than the individual indicators. Factor analysis is a measurement model, in that latent factors are ‘measured’ by multiple observed variables. The latent factor is considered the cause of the observed variables that explains why they are correlated (Brown, Reference Brown2015). In general, there are two types of factor analyses: exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). EFA is used to explore the underlying factor structure of a set of variables, whereas CFA is based on prior knowledge or a conceptual foundation (Suhr, Reference Suhr2006). Typically, factor analysis is used in the social sciences for survey development to account for correlations between questions related to the same topic (Brown, Reference Brown2015). This method is useful because it accounts for correlation between two topics. For example, in a mental health survey, all answers to the questions related to depression will be correlated and factor analysis can be used to derive a measure of depression as a latent construct that explains those correlations (Brown, Reference Brown2015). This simplifies studying the relationship of depression to other constructs.
To the best of our knowledge factor analysis has not been used to model transition diseases. However, in theory, factor analysis could apply to grouping transition cow metabolic stress biomarkers, because they relate to the same physiological processes and most likely are correlated. In fact, multiple studies have used CFA to model biomarkers for human metabolic syndrome (Pladevall et al., Reference Pladevall, Singal, Williams, Brotons, Guyer, Sadurni, Falces, Serrano Rios, Gabriel, Shaw, Zimmet and Haffner2006; Li and Ford, Reference Li and Ford2007; Boronat et al., Reference Boronat, Saavedra, Varillas and Novoa2009; Gurka et al., Reference Gurka, Lilly, Oliver and DeBoer2014).
Factor analysis has many advantages, including its high-power data reduction capabilities and the ability to handle measurement errors efficiently (Brown, Reference Brown2015). The output of factor analysis is a factor score, which is a composite variable that relays information about an individual's placement on the factor (DiStefano et al., Reference DiStefano, Zhu and Mindrila2009). For the example of transition cow metabolic stress biomarkers, the factor score would correspond to the degree of metabolic stress an individual cow is undergoing. The factor scores can then be used in further analyses to predict disease risk, as in structural equation modeling. However, factor analysis has some disadvantages. The final factor scores may be difficult to interpret and there's often not a single solution. The lack of a single solution arises from the fact that the parameters are not uniquely defined, and the researcher has choices in how the model is specified (DiStefano et al., Reference DiStefano, Zhu and Mindrila2009). This indeterminacy results in a variable number of solutions that can produce varying factor scores (Grice, Reference Grice2001).
There are different variations of factor analysis that also allow for binary, count, and censored variables (Van der Ark et al., Reference Van der Ark, Croon and Sijtsma2012; Brown, Reference Brown2015). However, the assumptions of factor analysis in its basic form are a normal distribution for each observed variable, normal bivariate distributions for each pair of observed variables, and a normal multivariate distribution (Suhr, Reference Suhr2006). Although transforming variables often solves the problem of non-normality for continuous variables, adjusting for non-linearity of the associations between biomarkers for metabolic stress and disease outcomes may be more difficult. In some situations, very low or very high levels of a biomarker are harmful, whereas intermediate levels are protective. For example, both low and high BCS at calving is associated with increased incidence of ketosis, metritis, retained placenta, and mastitis (Kim and Suh, Reference Kim and Suh2003; Bewley and Schutz, Reference Bewley and Schutz2008; Roche et al., Reference Roche, Friggens, Kay, Fisher, Stafford and Berry2009). This suggests a quadratic relationship between BCS and disease, rather than a linear relationship.
Classification and regression trees
Classification and regression trees (CART) are obtained by partitioning the data in multiple steps to form a simple prediction model. The steps are portrayed as a decision tree (Loh, Reference Loh2011). Classification trees are used when the dependent variable is categorical (i.e. high disease risk or low disease risk), whereas regression trees are used when the dependent variable is continuous (e.g. level of disease risk). The independent variables can be either categorical or continuous (Ma, Reference Ma2018).
Steensels et al. (Reference Steensels, Antler, Bahr, Berckmans, Maltz and Halachmi2016) developed a CART model to detect post-calving lameness and metritis, and reported acceptable sensitivity and specificity at 69 and 87%, respectively. CARTs have also been used to predict difficult calvings (Zaborski et al., Reference Zaborski, Grzesiak and Pilarczyk2016; Fenlon et al., Reference Fenlon, O'Grady, Mee, Butler, Doherty and Dunnion2017). CART models have many advantages, including ease of use, clear rules that are easy to interpret, and the absence of assumptions that statistical models require. This could be of great benefit for practical use in dairy farms. A potential limitation of CART models is that they can only handle one variable at a time, so there is no way to account for interactions between the variables (Loh, Reference Loh2011). In addition, CART diagrams are unstable, as slight changes in the data can result in a vastly different tree (Li and Belford, Reference Li and Belford2002). Fenlon et al. (Reference Fenlon, O'Grady, Mee, Butler, Doherty and Dunnion2017) reported that multinomial regression, which is similar to logistic regression, outperformed the CART diagram for prediction of difficult calvings. CART diagrams are a valid option for predictive modeling and should be considered if more advanced statistical models are unavailable or unfeasible.
Count models (Poisson and negative binomial)
Poisson models are particularly useful for modeling disease incidence, because the Poisson distribution provides more accurate estimates of count data compared with the normal distribution (Long and Freese, Reference Long and Freese2006). To calculate herd/cohort level incidence, an exposure offset can be included in a Poisson model that adjusts for the number of cows in the herd (Dohoo et al., Reference Dohoo, Martin and Stryhn2003).
Poisson models hold a strict assumption that the variance is equal to the mean. However, most distributions do not meet this assumption and have a higher degree of skewness. This higher degree of skewness, which is called over-dispersion, causes a greater difference between the observed and predicted values than expected (Hilbe, Reference Hilbe2014). Models that incorporate a linear dispersion correction (e.g. negative binomial) may be used in place of Poisson regression to account for this discrepancy (Hilbe, Reference Hilbe2014). Zero-inflated and zero-truncated models are versions of Poisson/negative binomial models, which can be used in situations when there are many zeros and few zeros, respectively (Long and Freese, Reference Long and Freese2006). Poisson and negative binomial models have been used to model transition diseases in numerous studies. For example, Richert et al. (Reference Richert, Cicconi, Gamrock, Schukken, Stiglbauer and Ruegg2013) developed a Poisson model to investigate risk factors for mastitis, ketosis, and pneumonia. In Wisnieski et al. (Reference Wisnieski, Norby, Pierce, Becker, Gandy and Sordillo2019b) and Wisnieski et al. (Reference Wisnieski, Norby, Pierce, Becker, Gandy and Sordillo2019c), Poisson models were used to model transition disease incidence at the cohort level. If a researcher is interested in herd level prediction, Poisson or negative binomial is the viable option.
Generalized linear models
The general linear model (GLM) is comprised of both multiple regression and analysis of variance, which allow researchers to investigate the association between one or more independent variables and a continuous dependent variable. The generalized linear model (GLiM) is a flexible rendition of the GLM that can handle binary, count, and ordered and unordered categorical outcomes (Hardin and Hilbe, Reference Hardin and Hilbe2007). Due to this flexibility, the GLiM can adapt to both logistic and Poisson regression, which as mentioned earlier, can be used to predict disease risk. Model building requires specifying the model in two parts: equations linking the response and explanatory variables (e.g. identity, logit, log, and reciprocal), and the probability distribution of the response variable (e.g. normal, binomial, Poisson, and gamma) (Hardin and Hilbe, Reference Hardin and Hilbe2007). There are many advantages of GLiMs, including the ability to handle situations where the variance of the dependent variable is dependent on the mean (Turner, Reference Turner2008). In addition, the GLiM has greater statistical power for non-normal response data compared with a standard linear model (Ngo, Reference Ngo2016).
The assumptions of the GLiM are homoscedasticity of the error terms, uncorrelated errors for individual cases, and the conditional expected value of the errors must be equal to zero. However, errors do not need to be normally distributed (Hardin and Hilbe, Reference Hardin and Hilbe2007). Logistic and Poisson regression are both special cases of GLiMs. They can be used in the GLiM framework as a more flexible approach to consider if researchers have difficulty in meeting certain model assumptions. Many researchers have utilized GLiMs to model transition cow diseases, both using Poisson regression (Lima et al., Reference Lima, Filho, Greco and Santos2012; McArt et al., Reference McArt, Nydam and Oetzel2012, Reference McArt, Nydam and Oetzel2013) and logistic regression (LeBlanc et al., Reference LeBlanc, Duffield, Leslie, Batement, TenHag, Walton and Johnson2002, Reference LeBlanc, Herdt, Seymour, Duffield and Leslie2004, Reference LeBlanc, Leslie and Duffield2005; Benzaquen et al., Reference Benzaquen, Risco, Archbald, Melendez, Thatcher and Thatcher2007; Seifi et al., Reference Seifi, LeBlanc, Leslie and Duffield2011).
The assumption of uncorrelated errors for GLiMs requires that the observations are independent. This may not be the case if cows are clustered within cohorts or farms. As mentioned earlier, cows within farm or cohort may be more alike compared with cows in different farms or cohorts. Data can also be correlated if multiple measurements were taken from the same cow (e.g. multiple time points). Measurements from the same cow are more likely to be alike than measurements from different cows. Researchers can further relax the assumption of uncorrelated errors by moving to generalized linear mixed models (GLMMs). The GLMM can account for this correlation through the addition of a random effect term for each level of clustering (e.g. farm, cohort, and cow) (Rabe-Hesketh and Skrondal, Reference Rabe-Hesketh and Skrondal2008).
Summary
In summary, there are a variety of statistical models to consider, each with their own strengths, weaknesses, and assumptions. It is up to researchers to determine which predictive model is best for their analysis, but there are multiple factors to consider. For instance, if the sample size is limited, exact logistic regression, CART diagrams or discriminant factor analysis are best. For data not conforming to certain assumptions (e.g. normality of response variable), a more flexible analysis using GLiM may be warranted. Factor analysis may be a valid option for researchers having issues with multicollinearity of independent variables but should be wary of non-linear relationships between predictors and the outcome. If health at the cohort or herd level is the purpose of the analysis, count models may be a preferred choice, while many of the other models mentioned are useful for prediction at the individual cow level (e.g. logistic regression). These are not an exhaustive list of considerations, but as mentioned before, there are many factors to consider. Researchers should plan the statistical analysis while designing the study, so that the study design can support the type of statistical analysis that will work best with the resources available.
Additional techniques for predictive models
There are additional techniques specific for predictive models that have seldomly been used to model transition cow diseases. Some examples that deserve attention include model averaging and shrinkage.
Model averaging
Often modeling procedures, such as variable selection and CART, are sensitive to slight changes in the data (Shtatland et al., Reference Shtatland, Kleinman and Cain2004). Model averaging is a technique that reduces model uncertainty by combining or averaging multiple potential models (Ghosh and Yuan, Reference Ghosh and Yuan2009). There are many ways to implement model averaging, but most methods start with performing best subsets selection to identify a set of potential models. Then, regression coefficients or predictions can be averaged over multiple potential models (Shtatland et al., Reference Shtatland, Kleinman and Cain2004). A more advanced technique is Bayesian model averaging, which involves direct model selection, combined estimation, and combined prediction using Bayesian inference (Fragoso et al., Reference Fragoso, Bertoli and Louzada2018). Bayesian methods incorporate the use of probability for quantifying and adjusting for uncertainty in inference (Gelman et al., Reference Gelman, Carlin, Stern, Dunson, Vehtari and Rubin2014).
Shrinkage
Typically, predictive models will not predict as well in future samples as they did in the samples used to build them (Ivanescu et al., Reference Ivanescu, Li, George, Brown, Keith, Raju and Allison2016). Shrinkage is a model validation tool that can account for this discrepancy (Ivanescu et al., Reference Ivanescu, Li, George, Brown, Keith, Raju and Allison2016). Shrinkage pulls the regression coefficients toward zero and moves the prediction toward the average risk and improves predictive value (Steyerberg et al., Reference Steyerberg, Eijkemans and Habbema2001a). This also helps protect against overfitting, which occurs when there are few events compared with predictors. Overfitting causes the model to underestimate the risk in low-risk individuals and overestimate the risk in high-risk individuals (Pavlou et al., Reference Pavlou, Ambler, Seaman, Guttmann, Elliot, King and Omar2015). There are a variety of shrinkage techniques to consider, including shrinking the coefficients by a common factor (e.g. 20%) or by penalized regression (e.g. lasso regression) (Pavlou et al., Reference Pavlou, Ambler, Seaman, Guttmann, Elliot, King and Omar2015). Although not all researchers have the ability to incorporate model averaging or shrinkage, they should be considered as potential options to improve predictive ability.
Model evaluation and validation
As mentioned earlier, predictive models should be evaluated for their discriminant ability, calibration, and goodness-of-fit (Sainani, Reference Sainani2014). Discriminant power is the ability of the model to systematically assign higher predicted probabilities to those that have the outcome (disease) compared with those that do not (no disease) (Sainani, Reference Sainani2014). A common test that measures discriminant power is ROC curve analysis. From the ROC curve, the area under the curve (AUC) can be calculated which gives the probability that a randomly tested disease individual has a greater test value compared with a randomly selected individual with no disease (Dohoo et al., Reference Dohoo, Martin and Stryhn2003). Model calibration is a measure of the accuracy of predictions, which is usually assessed through calibration plots or can be tested through the Hosmer–Lemeshow deciles of risk test (Hosmer et al., Reference Hosmer, Lemeshow and Sturdivant2013; Sainani, Reference Sainani2014). Calibration plots and the Hosmer–Lemeshow deciles of risk test can group the observations based on deciles of predicted probabilities (e.g. 0.10, 0.20, 0.30, and so on) and compare the observed versus expected number of individuals in each decile group (Hosmer et al., Reference Hosmer, Lemeshow and Sturdivant2013). Goodness-of-fit tests depend on the statistical model used. For example, the Hosmer–Lemeshow deciles of risk test also serves as a goodness-of-fit test for logistic regression (Hosmer et al., Reference Hosmer, Lemeshow and Sturdivant2013), and the deviance and Pearson goodness-of-fit tests can be used for Poisson models (Long and Freese, Reference Long and Freese2006). It is the responsibility of the researcher to choose appropriate goodness-of-fit tests for the analysis.
Predictive models should also be validated. External validation involves testing the model in an entirely new data set. However, sometimes this is not feasible due to cost or time (Sainani, Reference Sainani2014). Internal validation is a method that aims to improve the performance of the model in new subjects by partitioning the current dataset and testing how the model performs in this hypothetical set of new subjects (Steyerberg et al., Reference Steyerberg, Harrell, Borsboom, Eijkemans, Vergouwe and Habbema2001b; Pavlou et al., Reference Pavlou, Ambler, Seaman, Guttmann, Elliot, King and Omar2015). Internal validation can be done using a split sample approach, where a proportion of the data is used to build the model and the rest of the data are used to test the model. Cross-validation is another option that is an extension of split sample, where a randomly selected proportion of the data are used to build the model, and the other proportion is used to test the data, and vise-versa. This procedure is performed multiple times and the results are then averaged (Steyerberg et al., Reference Steyerberg, Harrell, Borsboom, Eijkemans, Vergouwe and Habbema2001b). The internal validation method that is most efficient and resistant to bias and variability is called bootstrapping (Steyerberg et al., Reference Steyerberg, Harrell, Borsboom, Eijkemans, Vergouwe and Habbema2001b). The bootstrap procedure involves the random selection of a sample with replacement from the original dataset. The bootstrap procedure selects multiple samples (usually 200 or more), each the same size of the original dataset, and averages the predictive performance over all the random samples (Steyerberg et al., Reference Steyerberg, Harrell, Borsboom, Eijkemans, Vergouwe and Habbema2001b; Pavlou et al., Reference Pavlou, Ambler, Seaman, Guttmann, Elliot, King and Omar2015). If there is no opportunity for external validation in new datasets, researchers should explore internal validation options.
Conclusion
Transition cow disease incidence has remained steady over time, despite improved monitoring and management efforts. Currently, cows are monitored for risk using a multitude of factors, including nutrient mobilization biomarker levels, health records, and milk production. These factors are often assessed separately and not systematically. The biomarkers typically measured for monitoring, NEFA and BHB, are not validated for use in the early dry period and fall short in sensitivity and specificity. A potential solution to improving sensitivity and specificity is to incorporate additional biomarkers that can capture the biological complexity of metabolic stress, including biomarkers for oxidative stress and inflammation. In addition, most researchers use explanatory modeling methods, rather than predictive modeling methods. There is a need for transition cow epidemiologists to explore and evaluate modeling aimed toward prediction, rather than explanation.
