2.1. Prevalence definitions

Prevalence is a dimensionless, unit-free value ranging from zero to one (zero to 100 if expressed as a percentage). Depending on the context, an investigator might be interested in prevalence of infection, infectious animals or disease. We restrict our presentation to infection prevalence.

Two types of prevalence are usually estimated in epidemiologic studies: point and period prevalence. Point prevalence is the proportion of infected individuals in a defined population (or more generally that have a condition of interest) at a given point in time; hence:

Period prevalence is the proportion of infected individuals in a defined population found over a specified time period (e.g. 2 years). For non-resolvable infections, e.g. MAP infection, point prevalence provides valid estimates of the frequency of individuals that have ever been infected and thus we do not further consider period prevalence in this paper.

2.2. Sampling

To justify descriptive inferences and probabilistic interpretations of prevalence estimates based on sample data, random selection of units from the population to be tested is necessary (Greenland, Reference Greenland1990; Greiner and Gardner, Reference Greiner and Gardner2000a). Depending on the goals of the survey, availability of resources and logistics, units may be enrolled using simple random, stratified random, systematic random, single or two-stage cluster sampling schemes (Scheaffer et al., Reference Scheaffer, Mendenhall and Ott1995), among other random sampling schemes.

2.3. Statistical inference

A frequentist approach assumes the prevalence of the infection (P) in the target population to be a fixed, unknown quantity. Frequentist inferences for P are summarized using a point estimate () along with a 100(1−α)% CI that quantifies inferential precision, where typically α=0.05. One interpretation of a 100(1−α)% CI in this context is as follows: if sampling from the source population were repeated to generate all possible samples from the population of a given size without varying the sampling procedure, 100(1−α)% of the CIs so generated would be expected to contain P.

A Bayesian approach assumes a probability distribution for P which characterizes the researcher's uncertainty about the population prevalence independently of the current survey data. Subjective knowledge about P and about test accuracy parameters (i.e. Se and Sp) is elicited from experts familiar with the subject matter or derived from historical or current data obtained from similar populations. This information is formally incorporated into the analysis in the form of probability distributions called prior distributions or, simply, priors. Inferences for P are made by using a likelihood function and the prior distribution to combine information from the data with information independent of the data, respectively, to form the posterior distribution of P. The relationship between the likelihood, prior and posterior is given by:

(1)

where Pr(·) denotes a probability density or mass function, θ denotes parameters in the statistical model and K is a normalizing constant. Summary measures of posterior distributions, such as means or medians, and outer percentiles, form the basis for inference about P. Typically, results are presented in the form of the median and a 100(1−α)% probability interval (PI) (usually α=0.05), which provides a range of values that contains P with probability=1−α, given the data and model. Hence, unlike CIs, PIs have probabilistic interpretations dependent only on currently observed data and a given set of priors. This renders interpretation and application of results more straightforward for both scientists and policy-makers. For further explanation of fundamental concepts of Bayesian inference, we refer the reader to other introductory works (Berry, Reference Berry1996; Bland and Altman, Reference Bland and Altman1998; Dunson, Reference Dunson2001). All Bayesian computations presented in this paper were performed using the freely-available WinBUGS software (downloadable from http://www.mrc-bsu.cam.ac.uk/bugs/winbugs/contents.shtml).

3.1. Apparent prevalence

3.1.1. Frequentist approach

Apparent prevalence (P _A) is the probability that a randomly-chosen unit of observation will test positive (Greiner and Gardner, Reference Greiner and Gardner2000b) and therefore is dependent on the true population within-herd prevalence (P _T) and Se and Sp of the diagnostic test used. A point estimate of P _A based on test outcomes of n randomly sampled individuals from the source population of size N is given by

(2)

where x denotes the number of sampled units that test positive. An approximate 100(1−α)% CI for P _A is given by the Wald CI:

(3)

where z _α/2 is the 1−α/2 percentile of the standard normal distribution. Equation (3) is derived assuming (i) x~bin(n, P _A) and (ii) that this binomial distribution is well approximated by x~N(nP _A, nP _A(1−P _A)). It is usually recommended that both nP _A and n(1−P _A) be sufficiently large e.g. both greater than 5 (Agresti, Reference Agresti1996, Leemis and Trivedi, Reference Leemis and Trivedi1996), for the approximation to be valid. Others have suggested that the normal approximation is appropriate for 95 and 99% CIs if for k=3 and 5, respectively (Enoe et al., Reference Enoe, Georgiadis and Johnson2000). Key limitations of Wald CIs for P _A are that they can provide coverage much less than the nominal level (Agresti and Coull, Reference Agresti and Coull1998), may have lower and upper bounds <0 and >1, respectively, and are inappropriate when no animals test positive. Alternative methods for CI calculation exist and here we highlight three common approaches. First, exact CIs can be calculated either from the binomial distribution (see Appendix A.1 for details) or directly from the F distribution using the following formula (Lui, Reference Lui2004):

(4)

where Q ₁=F _{2n−2x+2,2x,1−α/2} and denotes the 100(1−α/2) percentile of the F distribution with df ₁ and df ₂ degrees of freedom. For x=0 the lower limit of (4) is defined as 0 and for x=n the upper limit is defined as 1. Second, Agresti and Coull (Reference Agresti and Coull1998) have shown that the score CI

(5)

provides coverage closer to the nominal level than both the Wald and exact CIs for small sample sizes (i.e. 5–100). Third, to circumvent Wald and exact CI coverage limitations, as well as the cumbersomeness of (5), Agresti and Coull (Reference Agresti and Coull1998) suggest a modified 95% Wald interval (the ‘add 2 successes and 2 failures’ adjusted Wald interval):

(6)

where . This interval is slightly conservative relative to the score interval but does not suffer from low coverage near 0 and 1 (for sample sizes ranging from 0 to 100) and is still closer to 95% coverage than exact intervals (Agresti and Coull, Reference Agresti and Coull1998).

3.1.1.1. Example

Assume that from a very large herd of dairy cows, n=200 animals are randomly sampled and x=26 animals test positive for MAP infection, yielding an apparent prevalence estimate of _A=0.13. Using (3), a 95% CI for P _A is 0.083–0.177. Exact calculations using either the binomial or F distributions yield a 95% CI from 0.087 to 0.185, while the 95% score CI is from 0.090 to 0.184. The adjusted Wald 95% CI is from 0.090 to 0.185.

If a 100(1−α)% CI is desired when there are no positive test results, an exact CI can readily be calculated using (0, 1−α^1/n) (Louis, Reference Louis1981). When n≥20 and α=0.05, a good approximation to this interval is ‘the rule of three’ 95% CI=(0, 3/n) (Jovanovic and Levy, Reference Jovanovic and Levy1997). If in our last example, 0 of 200 animals test positive, then using (0, 1−α^1/n), a 95% CI for P _A is 0–0.0149 and using the ‘rule of three’ is 0–0.0150. Hence for this scenario, both these methods give practically the same results.

3.2. Diagnostic tests

Diagnostic tests commonly used in prevalence studies of Johne's disease in live animals are based on either the identification of the etiologic agent (MAP) or detection of a serum or milk antibody response to its presence (Manning and Collins, Reference Manning and Collins2001; Collins et al., Reference Collins, Gardner, Garry, Roussel and Wells2006). Researchers typically use fecal culture as a reference MAP-detection test against which to evaluate the performance of serological tests such as the enzyme-linked immunosorbent assay (ELISA) (Manning and Collins, Reference Manning and Collins2001). However, the intermittent fecal shedding of MAP particularly during sub-clinical infection, coupled with late-stage antibody production, makes accurate and precise estimation of test characteristics challenging. Consequently, prevalence estimation methods that rely on these tests should incorporate uncertainty in values of test accuracy parameters such as Se and Sp. Priors for diagnostic tests can be derived based on the results of recent validation studies involving animals with similar characteristics to the ones about to be tested. If during a validation study for test Se, x out of n infected animals test positive for MAP, then a beta(x+1, n−x+1) is an appropriate choice of prior for Se (Vose, Reference Vose2000). This beta distribution is obtained as the posterior of Se when the validation study data are modeled as bin(n, Se) with a uniform (0, 1) prior on Se. For example, if in a recent validation study, 30 of 100 infected cows tested positive for MAP infection, then a beta(31, 71) prior for Se can be used in analysis of the future prevalence survey data. For our examples, we use two commonly-used diagnostic tests, serum (IDEXX) ELISA and fecal culture on Herrold's egg yolk (HEY) medium. Priors for these tests (see Table 1) are based on both the results of validation studies (Sockett et al., Reference Sockett, Carr and Collins1992; Collins et al., Reference Collins, Wells, Petrini, Collins, Schultz and Whitlock2005) and expert opinion.

Table 1.

Priors for sensitivity (Se) and specificity (Sp) of IDEXX serum ELISA and HEY fecal culture based on validation studies and updated with expert opinion

¹ Most likely values of sensitivity and specificity are used as point estimates for frequentist calculations.

3.3. True prevalence

3.3.1. Frequentist approach

True prevalence (P _T) of infection is the probability that a randomly-chosen unit from the source population is infected. Apparent prevalence is related to P _T, Se and Sp through the equation P _A=P _TSe+(1−P _T)(1−Sp). Among the first estimators of P _T was the Rogan–Gladen-estimator (Rogan and Gladen, Reference Rogan and Gladen1978), which is obtained by substitution of estimates of P _A, Se and Sp into this equation to yield

(7)

If Se and Sp are assumed known with certainty, then the standard error of _T is given by: (Greiner and Gardner, Reference Greiner and Gardner2000a)

(8)

where J=Se+Sp−1 denotes the Youden index (Youden, Reference Youden1950), a combined measure of test accuracy. In practice, Se and Sp are not known with certainty (Greiner and Gardner, Reference Greiner and Gardner2000a) in which case

(9)

(Greiner and Gardner, Reference Greiner and Gardner2000a) provides a more valid quantification of uncertainty. Here,

where n ₁ and n ₂ are the numbers of infected and non-infected animals, respectively, in the original test validation study.

3.3.1.1. Example

Assume that a serum ELISA (Se=0.30, Sp=0.96) was used to test the herd described in section 3.1.1. Then _T=0.346. If we assume that Se and Sp are known with certainty, then using (8), a 95% CI is 0.167–0.525. Using (9) with n ₁ and n ₂=415 and 359, respectively (Collins et al., Reference Collins, Wells, Petrini, Collins, Schultz and Whitlock2005), a more realistic 95% CI that captures the uncertainty in Se and Sp is 0.160–0.533. As is evident from this example, the discrepancy between _T (0.346) and _A (0.13) can be substantial. In general, the more accurate the test, i.e. the closer the Youden index is to 1, the smaller the discrepancy. Though most often P _T>P _A for a given herd and diagnostic test, the converse is also possible. Using a test with Se=0.30 and Sp=0.96, we note that _T<_A when _T<0.057. Comparing the CIs obtained using formulas (3), (8) and (9), we also note that precision is greater when P _A is estimated or when P _T is estimated assuming that the diagnostic test's characteristics are known perfectly. In general, while differences between the precision of estimates obtained assuming known and unknown test characteristics are smaller for larger sample sizes (n ₁ and n ₂) of the validation studies originally used to estimate Se and Sp, the difference in bias increases since the bias in _T decreases but not in _A (Rogan and Gladen, Reference Rogan and Gladen1978).

The application of the Rogan–Gladen-estimator is problematic in some situations because _T can be <0 or >1. In the context of MAP infection, only the former is of practical importance and can occur in a small sample with low infection prevalence when the proportion of false-positive results is greater than the apparent prevalence. The limitations of (7) are further exacerbated by the fact that when sample sizes are small and true prevalence low, the normal approximation used for calculation of CIs may not be justifiable. In particular, the condition 0<_T±3s.e.(_T)<1 becomes difficult to satisfy when (9) is used. For these reasons, we advise against using (7) for small samples and when prevalence is believed to be close to 0.

3.4. True versus apparent prevalence

In the context of prevalence estimation, inferences for P _T are preferred to inferences for P _A. Nevertheless, if test Se and Sp are assumed to be constant and we want to ascertain only if P _T is increasing or decreasing over time in a herd, we can make use of the linear relationship resulting from (7); P _T=mP _A+C where m=1/J and C=(Sp−1)/J are both constants. Consequently, if P _A increases (decreases) we infer that P _T increases (decreases). For these conclusions to be valid, the same diagnostic test with constant Se and Sp over time must be used on each occasion and the sampling scheme and cut-off for test interpretation must be unchanged.

4.1. Uses of pooled testing: advantages versus disadvantages

Though pooling has been used for detection of fish pathogens in aquaculture for at least 20 years (Worlund and Taylor, Reference Worlund and Taylor1983), the use of artificially-created pools for diagnostic testing in livestock species is a recent occurrence (Nielsen et al., Reference Nielsen, Thamsborg, Houe and Bitsch2000; Borel et al., Reference Borel, Doherr, Vretou, Psarrou, Thoma and Pospischil2004; Tavornpanich et al., Reference Tavornpanich, Gardner, Anderson, Shin, Whitlock, Fyock, Adaska, Walker and Hietala2004; Letellier et al., Reference Letellier, De Meulemeester, Lomba, Mijten and Kerkhofs2005; Brinkhof et al., Reference Brinkhof, Houwers and van Maanen2006). One pooled sample can be used to determine the pathogen status of a herd (infected or non-infected), while individual-level infection prevalence can be estimated if multiple pools are tested (Cowling et al., Reference Cowling, Gardner and Johnson1999; Christensen and Gardner, Reference Christensen and Gardner2000). In this section, we focus mainly on issues related to individual-level prevalence estimation using data from pooled samples.

When sample collection costs are low relative to test cost and P<0.1 (i.e. the condition is rare) (Sacks et al., Reference Sacks, Bolin and Crowder1989; Tu et al., Reference Tu, Litvak and Pagano1994), pooled testing can be more cost- effective than individual testing and can provide more precise prevalence estimates per number of tests used (Kline et al., Reference Kline, Brothers, Brookmeyer, Zeger and Quinn1989). These advantages are based on the assumption that there is no loss of test accuracy resulting from aggregation of samples and that bias is negligible (implies moderate to small pool sizes and large sample sizes). Disadvantages of pooled testing include the need to store and handle larger volumes and the potential for loss of Se and possibly Sp compared with use of individual samples (Christensen and Gardner, Reference Christensen and Gardner2000).

4.2. Sensitivity and specificity of pooled tests

The pooled sensitivity (PSe) is the probability that a pool will test positive given that it contains at least one sample from an infected animal, while pooled specificity (PSp) is the probability that a pool will test negative given that it does not contain a sample from any infected animal (Munoz-Zanzi et al., Reference Munoz-Zanzi, Thurmond, Hietala and Johnson2006). Although several authors mention the necessity for the ‘no loss of test accuracy assumption’ (PSe=Se and PSp=Sp), especially as it applies to PSe (Kline et al., Reference Kline, Brothers, Brookmeyer, Zeger and Quinn1989; Tu et al., Reference Tu, Litvak and Pagano1995; Cowling et al., Reference Cowling, Gardner and Johnson1999; Christensen and Gardner, Reference Christensen and Gardner2000), there is a dearth of information in the animal health literature on this topic. It is assumed that PSe depends on the prevalence of the agent, its concentration in infected samples, the number of samples per pool, the mechanism by which the test identifies positivity, and the analytic and diagnostic sensitivity of the assay to be used (Munoz-Zanzi et al., Reference Munoz-Zanzi, Thurmond, Hietala and Johnson2006). For instance, an increase in pool size when testing for bovine viral diarrhea using real-time PCR (Munoz-Zanzi et al., Reference Munoz-Zanzi, Thurmond, Hietala and Johnson2006) and when testing for MAP using fecal culture (Wells et al., Reference Wells, Whitlock and Lindeman2002) has been found to decrease PSe. Others found that PSe was greater, the greater the number of infected samples per pool when testing for MAP using fecal culture (Tavornpanich et al., Reference Tavornpanich, Gardner, Anderson, Shin, Whitlock, Fyock, Adaska, Walker and Hietala2004). PSp is a function of the degree of cross contamination of the pool by other related agents. Some authors have suggested that larger pool sizes may increase PSp because of dilution of false-positive-causing analytes (Christensen and Gardner, Reference Christensen and Gardner2000). Others (Munoz-Zanzi et al., Reference Munoz-Zanzi, Thurmond, Hietala and Johnson2006) have found opposite results and have speculated that larger pool sizes increase the probability of more contaminants being included in the pool. The effect of different factors on PSp is likely to be disease- and test-dependent, and would not apply to bacterial or virus isolation, which should be perfectly specific.

Culture of fecal pools has been the testing method of choice for pooled-sample estimation of MAP infection prevalence at both the individual (Kalis et al., Reference Kalis, Hesselink, Barkema and Collins2000; Wells et al., Reference Wells, Whitlock and Lindeman2002; Tavornpanich et al., Reference Tavornpanich, Gardner, Anderson, Shin, Whitlock, Fyock, Adaska, Walker and Hietala2004) and herd level (Wells et al., Reference Wells, Godden, Lindeman and Collins2003; Kalis et al., Reference Kalis, Collins, Barkema and Hesselink2004). Compared with individual animal testing for MAP infection, pools of feces from 5–10 animals lead to substantial economic savings with commensurate minor losses in test accuracy (PSe≈Se and PSp≈Sp). In particular, PSp=Sp when testing for MAP using fecal culture.

4.3. Frequentist estimation

4.3.1. Fixed pool size and no available test information

If test Se and Sp information is not available, a simple approach uses the following formula (Kline et al., Reference Kline, Brothers, Brookmeyer, Zeger and Quinn1989; Sacks et al., Reference Sacks, Bolin and Crowder1989):

(10)

where s _p=pool size, x _p=number of pools testing positive and n _p=total number of pools. An approximate 100(1−α)% CI for P _A is:

(11)

where _p=x _p/n _p (empirical prevalence of positive pools).

4.3.1.1. Example

Assume that we randomly select 120 cows from a large herd. Fecal samples are obtained from all 120 and pooled into groups of 10. Fecal culture is performed on all pools and five pools test positive for MAP. Using (10) and (11) yields _A=0.053 (95% CI: 0.007–0.098). Apart from the implicit assumption of perfect Se and Sp, one disadvantage of this method is that for low numbers of positive pools, negative lower confidence limits may result (Cowling et al., Reference Cowling, Gardner and Johnson1999). If in our example, x _p=2, using (10) and (11), _A=0.018 (95% CI: −0.007 to 0.043). When asymptotic methods fail, one can first calculate an approximate CI for p _p using

(12)

and

(13)

where _p=1−_p, for the lower and upper bounds, respectively (Tu et al., Reference Tu, Litvak and Pagano1994). The lower and upper 100(1−α)% confidence limits for P _A are then obtained by substituting [p _p]_Lower and [p _p]_Upper for _p (=x _p/n _p) in (10). Using (12) and (13) for x _p=2, an approximate 95% CI would range from 0.003 to 0.05. Equations (12) and (13) make use of a continuity correction, bringing continuous normally-distributed variables into closer agreement with the discrete binomial distribution (Fleiss, Reference Fleiss1981). Another alternative is to calculate exact confidence intervals using (4) or from the binomial distribution using x _p as the binomial variate (see Appendix A.1) and then as above, substituting [p _p]_Lower and [p _p]_Upper for _p in (10).

4.3.2. Fixed pool size and imperfect test sensitivity and specificity

If information on Se and Sp is available, the conditions 1−Sp⩽p _p⩽Se and Se+Sp−1>0 must hold in order to estimate P _T (Tu et al., Reference Tu, Litvak and Pagano1994). We assume that PSe=Se and PSp=Sp and if Se and Sp are known, then the following formula can be used to estimate P _T:

(14)

where J _p=PSe+PSp−1. An approximate 100(1−α)% CI for P _T is given by (Tu et al., Reference Tu, Litvak and Pagano1994):

(15)

where A _p=(PSe−_p)/J _p.

If asymptotic assumptions fail, equations (12) and (13) may be used to obtain lower and upper confidence limits for p _p, which can then be substituted into (14) for _p to obtain a CI for P _T (Tu et al., Reference Tu, Litvak and Pagano1994; Cowling et al., Reference Cowling, Gardner and Johnson1999). Alternatively, exact confidence limits may be calculated for p _p and similarly substituted into (14) (Cowling et al., Reference Cowling, Gardner and Johnson1999). The Pooled Prevalence Calculator (PPC) (http://www.ausvet.com.au/pprev/) facilitates easy calculations of both exact and asymptotic CIs for pooled prevalence (see Appendix A.2).

If Se and Sp are not known with certainty, an approximate 100(1−α)% CI for P _T is given by (Cowling et al., Reference Cowling, Gardner and Johnson1999):

(16)

where and n ₁ and n ₂ (as before) refer to the number of infected and non-infected animals, respectively, used in the original test validation study. Using n ₁=182 and n ₂=100 (Sockett et al., Reference Sockett, Carr and Collins1992), _T=0.112 (95% CI: −0.026 to 0.249). The (unrealistic) negative lower confidence limit reflects the increased difficulty in justifying large-sample normal approximations with relatively small sample sizes when test characteristic uncertainty is taken into consideration. Currently, there are no frequentist exact methods that incorporate uncertainty in Se and Sp. Thus, for reasons previously mentioned (see section 3.3.1), we advise against using (16) when prevalence is low.

4.4. Bayesian estimation

As pooled testing is most cost-effective when prevalence is low, approaches that incorporate test Se and Sp and provide realistic results (i.e. estimates and limits within the range 0–1) are particularly desirable for those situations. A Bayesian approach ensures that intervals are bounded below by 0. We assume a binomial distribution (Branscum et al., Reference Branscum, Gardner and Johnson2004) for the number of pools that test positive and thus x _p|(P _T, PSe, PSp)~bin(n, p _p), where . As in the case of prevalence estimation based on individual samples, beta distributions are chosen to model uncertainty in P _T, PSe and PSp, and mixture models can be used to allow for zero prevalence (analogous to the development in section 3.3.2).

5.1. General concepts

5.1.1. Definition of a herd

A herd is defined as any cluster or grouping of animals (Christensen and Gardner, Reference Christensen and Gardner2000), often defined by location (Boelaert et al., Reference Boelaert, Walravens, Biront, Vermeersch, Berkvens and Godfroid2000; Tavornpanich et al., Reference Tavornpanich, Gardner, Anderson, Shin, Whitlock, Fyock, Adaska, Walker and Hietala2004) but may also be defined by size (Baggesen et al., Reference Baggesen, Wegener, Bager, Stege and Christensen1996), intended purpose (Christensen et al., Reference Christensen, Baggesen, Nielsen and Stryhn2002), clinical signs (Wells et al., Reference Wells, Godden, Lindeman and Collins2003), age (Boelaert et al., Reference Boelaert, Walravens, Biront, Vermeersch, Berkvens and Godfroid2000) or combinations of the above. The herd (or herd-level) prevalence (HP) of infection in a population of herds is defined as the proportion of infected herds in the population:

In contrast to individual-level prevalence surveys, the unit of analysis in herd-level surveys is the herd or whole group.

5.2. Frequentist approach

Analogous to individual-level prevalence, true herd-level prevalence is estimated by (Wagner and Salman, Reference Wagner and Salman2004):

(17)

where is the estimated apparent herd prevalence (i.e. number of test-positive herds/total number of herds tested) and J _H=HSe+HSp−1. An approximate 100(1−α)% CI for HP _T is given by

(18)

where n _H is the number of herds tested. The application of (17) and (18) to herd prevalence estimation is problematic as it is necessary to assume common values for HSe and HSp. This also implies a common value for P _A, which would unrealistically imply constant infection prevalence from herd to herd. As a result, we discourage routine use of these formulas in favor of the Bayesian approach to herd prevalence estimation presented in the next section.

5.3. Bayesian approach

In the Bayesian setting with binomial sampling, we model within-herd prevalences as independent and identically distributed according to a common prevalence distribution, and we assume that the test's Se and Sp are invariant from herd to herd. The number of animals that test positive in each herd is modeled as x _i~bin(n _i, P _TiSe+(1−P _Ti)(1−Sp)) with independent beta priors for Se and Sp. As in the single herd setting, each within-herd prevalence is modeled according to a mixture distribution that models zero infection prevalence with probability 1−HP _T where HP _T denotes the herd-level prevalence in the region. For infected herds, the within-herd prevalence distribution is given by P _Ti~beta(μψ, ψ(1−μ)) where μ is the mean true prevalence in the population and ψ is a parameter related to the variance of the prevalence distribution that flexibly allows for disperse (e.g. uniformly distributed) or very similar infection prevalences among herds (Branscum et al., Reference Branscum, Gardner and Johnson2004). Large values of ψ indicate less variability so that within-herd prevalences concentrate around μ, while small values of ψ correspond to more heterogeneity of prevalence values across herds (Donald et al., Reference Donald, Gardner and Wiggins1994; Branscum et al., Reference Branscum, Gardner and Johnson2004).

The mean prevalence (μ) and ψ are typically modeled with beta and gamma distributions, respectively (Hanson et al., Reference Hanson, Johnson and Gardner2003a; Branscum et al., Reference Branscum, Gardner and Johnson2004), but other distributions can be used. A Bayesian approach allows not only for herd-level prevalence estimation but also provides a straightforward means to make predictive inferences about herds not tested during the survey. It is also possible to estimate the intracluster correlation coefficient for true prevalence among the study herds (Branscum et al., Reference Branscum, Gardner, Wagner, McInturff and Salman2005b).

5.3.1. Example

Using a two-stage sampling scheme, 29 dairy herds were randomly selected from dairy practices in central California. Random samples of 60 cows were selected from each herd and tested for MAP infection using serum ELISA. Data from the 29 herds are summarized in Table 2 and we assume a binomial distribution for the numbers of animals testing positive. Based on expert opinion, 60% of herds in this region of California were expected to be infected at the time of sampling with a one-sided 95% limit of 30% (i.e. the expert was 95% sure that the proportion of MAP-infected herds was greater than 0.30). BetaBuster yields a beta(4.8, 3.6) prior for HP _T. The ELISA test Se and Sp are modeled as in previous sections (see Table 1). The mean prevalence (μ) and ψ were modeled using beta(3.283, 17.744) and gamma (4.524, 0.387) priors, respectively (see Hanson et al., Reference Hanson, Johnson and Gardner2003a, for rationale and method of derivation). Finally, a mixture model was used because some herds in the region might be free of MAP-infected animals. The model was fit using WinBUGS (see Appendix B.6 for code).

Table 2.

Results of herd prevalence survey of MAP infection using IDEXX ELISA in 29 central California dairies (60 cows tested per herd)

The estimated posterior median of HP _T was 0.51 with 95% PI from 0.27 to 0.80, while the mean prevalence among infected herds was μ=0.18 (95% PI: 0.08–0.33). The estimated posterior prevalence distribution characterizes the within-herd prevalences in the region. At the time of sampling, 49% of the herds in the region were estimated to be free of MAP infection, 59% of the herds in the region had infection prevalence <0.05 and 82% had infection prevalence <0.2.

6.1. Sequential testing in herds

Investigators often sequentially test herds as a means of evaluating the effectiveness of management practices or interventions aimed at controlling or eradicating MAP infection, or to longitudinally monitor changes in infection prevalence. On other occasions, prevalence might be compared between two herds or two groups of animals within herds to determine whether or not they differ. Depending on the goal of the survey, the samples may or may not be regarded as statistically independent.

6.2. Frequentist comparison of population prevalence

6.2.1. Independent samples

To determine if two apparent prevalences are statistically different based on test results obtained from random samples of animals from distinct herds, a chi-square (χ²) test of homogeneity of proportions may be performed (Agresti, Reference Agresti1996);

(19)

where both i and j=1, 2, O _ij and Ê _ij are the observed and (estimated) expected values, respectively, in the ith row and jth column of a 2×2 contingency table (see Table 3) and . The estimated expected value is Ê _ij=r _ic _j/n, where r _i and c _j are the ith row and jth column totals, respectively. The statistical hypothesis to be tested is: versus with degrees of freedom (df)=(R−1)(C−1)=1, where R and C are the number of rows and columns, respectively (R=C=2 for 2×2 tables). The null hypothesis (H ₀) is rejected if χ²>χ²_df=1,1−α where χ²_df=1,1−α is the value corresponding to the 1−α percentile of the χ₁² distribution and α=type I error probability.

Table 3.

2×2 Contingency table for prevalence data

6.2.1.1. Example

Assume that 1 year after an initial testing, the herd described in section 3.1.1 is again tested for MAP, and 30/200 cows have positive test results, yielding an apparent prevalence (Table 3). To test whether are statistically different, we use (19) and α=0.05. This yields χ²=0.33<χ²_1,0.95=3.84, hence we do not reject H ₀ (i.e., there is no statistical evidence suggesting a change in apparent prevalence). A 100(1−α)% CI for is

(20)

Hence, with 95% CI of −0.05 to 0.09. For statistically independent samples, if any Ê _ij<5, the Fisher exact test is recommended to compare independent proportions (Rosner, Reference Rosner2000).

6.2.2. Dependent samples

In the previous example, if on both occasions the same cows were sampled (matched pair or repeated measures design), the data may be presented as in Table 4 and a McNemar's χ² test of homogeneity of proportions that accounts for correlated data should be performed. Here, (Agresti, Reference Agresti1996)

(21)

where n ₁₂ and n ₂₁ are the numbers of cows with discordant test results (T+T− and T−T+, respectively) at the initial and subsequent sampling. If n ₁₂≠n ₂₁ then and (21) can be used to test versus . Again H ₀ is rejected if χ²>χ²_df=1,1−α. Using α=0.05, χ²=0.333<3.84, hence we do not reject H ₀.

Table 4.

2×2 Contingency table for prevalence data (matched pairs)

n _p=the total number of pairs.

For paired binary data, a 100(1−α)% CI for is (Agresti, Reference Agresti1996)

(22)

where p _ij=n _ij/n _p, p ₁₊=p₁₁+p₁₂, p ₊₁=p₁₁+p₂₁ and n _p refers to the number of cows (each tested twice). Hence, if the same cows were tested on both occasions, the estimated difference between the apparent prevalences is 0.02, with 95% CI of −0.05 to 0.09. Both this CI and the one calculated in the previous section indicate that the data are consistent with being as large as 9% more and as small as 5% less than . If n ₁₂+n ₂₁<20, an exact version of McNemar's test is recommended (Rosner, Reference Rosner2000) (see Appendix A.3).

With data subject to misclassification (i.e. Se and Sp both <1), there is no straightforward frequentist method for testing differences between true prevalences. However an informal test of the difference between two true prevalences may be performed by checking for overlap of their individual CIs with adjustments made for multiple comparisons since a collection of separate 100(1−α)% CIs may have joint confidence less than 100(1−α)%. Overall, caution should be exercised when applying this conservative procedure to Wald CIs. Compared with standard hypothesis testing, the method both rejects the null hypothesis less often when it is true, and fails to reject the null more frequently when it is false (Schenker and Gentleman, Reference Schenker and Gentleman2001).

6.3. Bayesian comparison of population prevalence

Using a Bayesian approach, inferences can be made for differences in true prevalences between two (or more) true population prevalences. Assuming independent binomial sampling and that both herds (section 6.2.1) were tested for MAP infection using the same serum ELISA (Table 1), a Bayesian model for the data is where for j=1, 2 and beta priors are used for , Se and Sp. We require informative priors for Se and Sp to mitigate lack of identifiability (i.e. ensure that prevalences are estimable). Here, based only on the data, x _j/n _j estimates and we require additional input about test accuracy to estimate true prevalence. With the data and expert-elicited information about Se and Sp, inferences for can be made.

6.4. Reporting of results

For presentation of results based on frequentist or Bayesian analysis of prevalence data based on hypothesis tests, we encourage the use of interval estimates (e.g. 95% CIs or PIs) rather than, or in conjunction with, P-values or posterior probabilities. Whereas CIs and PIs help to reveal whether different statistically significant (or insignificant) results have different implications for policy or intervention decisions, P-values, which reduce results into a significant–non-significant dichotomy, obscure quantitative differences between results.

7.1. Relevance of sample size estimation

A critical aspect of the planning phase of a prevalence survey is the estimation of the number of individuals or groups of individuals needed. The sample size determines the precision of our final estimates and ultimately influences whether the survey will be able to address the research questions of interest. Limited time and resources dictate that prior to performing a study, an investigator should be reasonably assured that enough individuals are available so that estimates can be generated with the requisite precision. Consequently, sample size calculations are standard requirements for research grant proposals. Because the components of these calculations must be hypothesized, only estimates can be obtained for any proposed study. This uncertainty becomes more pronounced in prevalence surveys when accuracy of case identification and precision of estimates is also dependent on diagnostic test characteristics. Bayesian approaches to sample size calculations for prevalence surveys account for uncertainty in parameter values (e.g. for Se and Sp) and for data subject to misclassification. As a result, they are computationally intensive and cannot be directly implemented in WinBUGS (see, for example, Johnson et al., Reference Branscum, Gardner and Johnson2004; Branscum et al., Reference Branscum, Gardner and Johnson2006). For this reason, we present only methods for frequentist sample size calculations in this section and point the interested reader to other sources covering the general methodology for Bayesian sample size determination (Adcock, Reference Adcock1997; Rahme et al., Reference Rahme, Joseph and Gyorkos2000).

7.2. Sample size required to estimate true and apparent prevalence with a given precision

Precision of an estimate may be defined as the width of its (100−α)% CI. The sample size necessary for estimating P _A and P _T, respectively, within a given error margin (Δ) is given by (Greiner and Gardner, Reference Greiner and Gardner2000b):

(23)

and (Greiner and Gardner, Reference Greiner and Gardner2000a)

(24)

respectively, where is an a priori estimate of P _A (for example, obtained via pilot surveys or expert opinion) and Δ=half the CI width, which may be either an absolute number or a percentage of (or ). Table 5 provides estimates of the number of cows necessary for testing to estimate P _T for MAP infection when Δ ranges from 0.10 to 0.20, given a priori estimates of P _A for testing with serum ELISA (Se=0.30, Sp=0.96) and fecal culture (Se=0.60, Sp=0.999) and assuming that the test characteristics are known. For fixed (), as Δ increases (decreases), the required sample size decreases (increases), while for fixed Δ, as Se and/or Sp increase (decrease), the required sample size decreases (increases). Thus, the sample size required to estimate P _T (P _A) will be inversely proportional to the desired precision. If Se and Sp are not assumed to be known, then (24) can be generalized to adjust for uncertainty in Se and Sp using (Greiner and Gardner, Reference Greiner and Gardner2000a)

(25)

where B=Se(1−Se)/n ₁, C=Sp(1−Sp)/n ₂, and n ₁ and n ₂ are as previously defined for (9).

Table 5.

Sample sizes to estimate hypothesized true prevalences (_T) with a given precision (Δ) based on a priori apparent prevalence estimates (_A) for serum ELISA and fecal culture assuming known sensitivity (Se) and specificity (Sp)

NA=not applicable.

7.3. Sample size to detect a difference in prevalence

Sample size calculations to detect prevalence differences have four fundamental components: the significance level or the probability of falsely concluding that there is a difference when none exists (α=probability of type I error), the probability of falsely concluding that there is no difference when there is one (β=probability of type II error), the variance of the parameter to be estimated and, most critically, the size of the parameter itself, i.e. the difference to be detected (Wittes, Reference Wittes2002).

7.3.1. Independent random samples

To calculate the sample size (n _s=n ₁+n ₂) necessary to detect a difference from two independent random samples, we use (Rosner, Reference Rosner2000):

(26)

and n ₂ is computed based on k=n ₂/n ₁ where k is usually determined by the investigator. Here, , . A two-tailed hypothesis test is implied by (26), i.e., versus . Common choices of α and 1−β are 0.05 and 0.8, respectively, and total sample size (n _s) is minimized for k=1 (n ₁=n ₂). To estimate sample sizes based on one-tailed hypotheses (to identify either an increase or decrease in P _A), z _α/2 is substituted with z _α in (26). However, if the inherently two-tailed χ² statistic is to be used to test differences in prevalences, then the sample size estimation must be two-tailed to avoid sample size underestimation. One-tailed sample size calculations should be performed only if a one-tailed test is to be used to determine a difference in prevalence during data analysis (Lachin, Reference Lachin1981).

7.3.1.1. Example

Assume that two large herds have prevalences of MAP infection of . We wish to determine if . We therefore calculate the required number of animals to sample in order to have a probability of 0.8 of detecting if such a difference exists. Using α=0.05 and 1−β=0.8 and assuming that equal numbers of animals (k=1, i.e. n ₂=n ₁) are available to randomly sample, (26) yields n ₁=685 and sample size (n _s)=1370.

Sometimes sample sizes need to be calculated to detect a change in test prevalence of MAP infection between two time points in a single herd. Sampling at the first time point may already have occurred when the need for the sample size calculation arises and economic considerations may influence whether the same number of cows can be tested at the second sampling.

7.3.1.2. Example

Assume that we have a very large herd. In an earlier random sample of 600 cows, 60 tested positive, i.e., . New management practices were introduced to reduce the prevalence of MAP infection and, 1 year later, we wanted to evaluate if the apparent prevalence has decreased by 5% . Costs of testing have increased and total costs would be reduced if testing less than 600 animals were possible. First, we need to calculate the ratio of numbers of animals necessary to sample in order to have a probability of 0.8 of detecting this difference (0.05) in prevalence if it exists. Using α=0.05 and 1−β=0.8 and assuming that equal numbers of animals are available to sample (randomly), the implied one-sided hypotheses are (Roussas Reference Roussas2003). We assume for simplicity that the data are statistically independent because of the time between the two sampling dates.

We first calculate the total sample size as if equal sized samples (i.e. n ₂=n ₁) were to be used, denoted n _s(k=1), using z _α=0.05 (one-sided hypothesis) in (26). This yields n_s(k=1)=684. We then make use of the following relationships (Altman, Reference Altman1991):

(27)

(28)

(29)

From (27) and (28), n ₁=n _s(k=1)(1+k)/4k. Since 600 animals have already been sampled, we let n ₁=600 and n_s(k=1)=684 (from above). This yields k≈0.398 and n ₂=239. Hence, we sample 239 cows at the second sampling and 839 cows in total.

7.3.2. Dependent samples

The number of paired observations (n _p) necessary to detect a difference in P _A estimated from testing each cow at two different time points is given by (Lachin, Reference Lachin1992):

(30)

where ₁₂ and ₂₁ are the a priori probabilities that a cow will have test results T+T− and T−T+, respectively, at the two testing times (see Table 1, Appendix A.4). In the context of a group of animals tested at two time points, n _p refers to the number of pairs of tests to be done (i.e. the number of animals to be tested).

7.3.2.1. Example

Assume that we have a large herd known to be infected with MAP. We wish to determine if the prevalence of infection is increasing in the herd and thus plan to select a group of cows and test them longitudinally at two time points. We assume based on previous experience with other herds and with the diagnostic test to be used that ~87% of cows tested will have the same test result at both testings. In addition, we assume that and , respectively (see Appendix A.4). Hence ₁₂+₂₁=0.13 and ₂₁−₁₂=0.05. Using α=0.05, z _α (one-sided hypothesis) and 1−β=0.8, from (30), n _p≈318 pairs of tests (318 cows).

7.4. Sample size based on true prevalence

Sample sizes to detect differences in P _T are done by first determining the difference in P _T that is important to identify. The Rogan–Gladen estimator (7) is used to calculate the equivalent difference on the P _A scale and then equation (26) or (30) is used to find the sample size necessary to detect the difference on the P _A scale. We recommend that sample size estimation be done based on true rather than apparent prevalence estimates (see Table 6), wherever possible.

Table 6.

Sample sizes (numbers in each group)Footnote ¹ required to detect differences in true prevalence (P _T) for independent samples (α=0.05, 1−β=0.80 and k=1(n ₂=n ₁))

¹ Table is symmetrical around the diagonal descending from left to right.

NA=not applicable.

7.5. Sample size required to estimate individual-level prevalence from pooled samples

Formulas for numbers of pools required to estimate individual level true and apparent prevalence can easily be derived given corresponding variance formulas (Tu et al., Reference Tu, Litvak and Pagano1994; Cowling et al., Reference Cowling, Gardner and Johnson1999). For fixed pool sizes, the formula for number of pools necessary to estimate P _A within a given error margin (Δ) is

(31)

If an a priori estimate is given of the prevalence of positive pools (_p), then:

(32)

Given an a priori estimate of true prevalence, when Se and Sp are assumed known

(33)

If Se and Sp are not known,

(34)

where , B _p=PSe(1−PSe)/n ₁, C _p=PSp(1-PSp)/n ₂ and n ₁ and n ₂ (as before) refer to the number of infected and non-infected animals, respectively, used in the original test validation study. As in the case of individual samples, Δ may either be a constant or a percentage of the prevalence to be estimated. Comparing pool sizes of 5 and 10, we note that for a given a priori P _A (or P _T), PSe, PSp and Δ, pool sizes of 10 require overall fewer individual samples (see Table 7).

Table 7.

Sample sizesFootnote ¹ required to estimate hypothesized true prevalences (_T) with a given precision (Δ) based on an a priori apparent prevalence estimate (_A) using pooled fecal culture with known sensitivity (PSe=0.6) and specificity (PSp=0.999)

¹ Numbers of pools in parentheses.

NA=not applicable.

7.6. Sample size required to estimate herd prevalence

Sample size calculations for estimation of herd prevalence have both within-herd and among-herd components. Determination of the sample size to detect infection is an important first step in correctly determining the MAP infection status of each herd. In the context of a herd prevalence survey of M herds (h ₁, h ₂, …, h _M) the number of samples required to detect at least one truly infected animal in the ith herd (h _i) (where i⩽M), assuming binomial sampling, is given by (Christensen and Gardner, Reference Christensen and Gardner2000):

(35)

where C is the level of confidence (e.g. 95%) in detecting at least 1 truly infected animal (i.e. HSe) given an a priori specified true prevalence for that herd and Se _i and Sp _i are the test's sensitivity and specificity in that herd respectively. Because herd-specific values of Se _i and Sp _i are not usually known, sensitivities and specificities are usually assumed to be invariant across herds. Thus the required sample size for each herd is a function of true prevalence, the desired confidence and the diagnostic test's Se and Sp. The total number of animals necessary to sample in order to have a confidence C to detect at least 1 infected animal in each herd is given by:

(36)

Corrections to (35) and (36) can be made for finite population sampling (see, for example, Martin et al., Reference Martin, Shoukri and Thorburn1992).

If within-herd prevalence estimates are not required, after initial sample size calculation using (36) above, risk-based (targeted) sampling may be used to select animals with a greater likelihood of being infected for testing if prior knowledge of risk factors for infection (e.g. previous diagnostic test results, clinical signs, etc.) is available (Christensen and Gardner, Reference Christensen and Gardner2000). For a given sample size, risk-based sampling increases the confidence to detect at least one infected animal in a herd when compared with random sampling. This has been substantiated by results from a number of studies (Nielsen et al., Reference Nielsen, Enevoldsen and Grohn2002a; Tavornpanich et al., Reference Tavornpanich, Gardner, Carpenter, Johnson and Anderson2006). Conversely, risk-based sampling can be used to reduce the sample size necessary to achieve a herd diagnosis for fixed confidence level.

The number of herds required to estimate true herd prevalence (HP _T), within a given error margin (Δ) is given by (Humphry et al., Reference Humphry, Cameron and Gunn2004):

(37)

where n _H=the estimated number of herds, Ω=Hse(H ), Π=(1−HSp)(1−H ), Δ=prescribed error margin, J _H=HSe+HSp−1 and z _α/2 as before, refers to the 1−α/2 percentile of the standard normal distribution. The test's HSe and HSp are both pre-specified by the investigator and therefore assumed known without error. Table 8 provides estimates of the number of herds required to estimate HP _T=0.7 and 0.9, respectively, within an error margin (Δ)=0.1 with 95% confidence.

Table 8.

Number of herds required to estimate hypothesized true herd prevalences of 0.7 (H _T=0.7) and 0.9 (H _T=0.9) with precision (Δ=0.1) and 95% confidence (z _α/2=1.96) for different combinations of herd sensitivity (HSe) and herd specificity (HSp)

7.7. Sample size required to substantiate freedom from infection

Strictly speaking, to unequivocally demonstrate freedom from MAP infection in a herd or in a group of herds from a well-defined geographical region, all animals should be tested using a perfect test. However, a sample may be used to establish with prescribed probability that infection, if present, is below a predetermined level, i.e. a certain minimum expected prevalence (also known as design prevalence). For a herd, the minimum expected prevalence of infection would be the minimum prevalence expected if infection were actually present (Cameron and Baldock, Reference Cameron and Baldock1998). This prevalence estimate should be based on knowledge of the disease and its risk factors, knowledge of the herd itself (including previous results of surveillance and disease-related interventions) and knowledge of the Se and Sp of the diagnostic test to be used. At the regional level, the minimum accepted herd prevalence would be the lowest (non-zero) unacceptable herd prevalence, based on economic or political factors (Cameron and Baldock, Reference Cameron and Baldock1998). For substantiation of freedom from infection, two-stage sampling is usually employed. Sample sizes for both herd-level and within-herd testing to establish that infection is below a specified prevalence in a given region may be estimated using the ‘Freedom from Disease’ drop-down menu within the ‘Survey Toolbox’ or ‘FreeCalc’, which are both available at www.vetschools.co.uk/EpiVetNet/Sampling_software.htm (see Appendix A.5).

7.7.1. Example

Assume that in a region with 1000 dairy herds (herd sizes of 500–1500 cows) where the herd prevalence of MAP infection was initially 90%, substantial eradication efforts are undertaken and it is now claimed that the herd prevalence is below 50%. Veterinary authorities expect that, within any given herd, 2% or more of the animals will be infected if MAP is present. Fecal culture (Se=0.60, Sp=0.999) is to be used for testing. Using the modified hypergeometric exact formula within the Survey Tool Box with HSe=0.95 and HSp=0.98, and 95% confidence and power, we calculate that if 8 herds are chosen randomly and ⩽1 herd tests positive, we can be ~95% confident that HP _T<0.5 (see i–iv, appendix A.5). The Survey Tool Box also calculates the numbers of animals necessary to sample (based on herd sizes) in order to be ~95% certain that infection, if present, is ⩽2%, given type I (1-HSe) and type II (1-HSp) errors of 0.05 and 0.02, respectively (see v–viii, appendix A.5). The survey would then be conducted by randomly sampling 8 herds and then choosing random samples of between 402 and 455 cows from herds of between 500 and 1500 respectively. If >2 animals test positive for MAP in at least 2 of the 8 herds, then the survey has not provided evidence that HP _T<0.5. However, if either 0 or 1 herd has >2 animals testing positive, then we can conclude with more than 95% confidence that the true herd prevalence (HP _T) of MAP infection in the region is <0.5 and where present, the true individual-level prevalence (P _T) of MAP infection is <0.02.

For a given herd, the lower the minimum expected prevalence and the more inaccurate the diagnostic test, the greater the sample size necessary to substantiate freedom from infection (Cameron and Baldock, Reference Cameron and Baldock1998). Given a herd of 1000 cows and that diagnostic testing is to be done using fecal culture (Se=0.60 and Sp=0.999) or ELISA (Se=0.30 and Sp=0.96), Table 9 provides sample size estimates and maximal numbers of animals testing positive in order to establish with 95% confidence (HSe) and 98% power (HSp) that MAP infection, if present, is below a given true prevalence.

Table 9.

Number of cows necessary to sample in a herd of size 1000 to establish with 95% confidence (HSe) and 98% power (HSp) that Mycobacterium avium subspecies paratuberculosis (MAP) infection, if present, is less than a given true prevalence (P _T)

NA=not applicable (herd size too small).

Bayesian approaches to sample size determination for single herd and regional disease freedom surveys have also been developed (see, for example, Johnson et al., Reference Branscum, Gardner and Johnson2004; Branscum et al., Reference Branscum, Johnson and Gardner2006). However, computationally-complex simulation-based approaches are needed for their implementation and thus we do not present them here.

7.8. Power calculations

7.8.2. Power estimation (statistically independent samples)

From (26), the power to detect a difference using a two-sided test is given by (Rosner, Reference Rosner2000):

(38)

where are as defined in section 7.3.1 and Φ(x) denotes the standard normal cumulative distribution function evaluated at x.

7.8.2.1. Example

Assume, as in the second example of section 7.3.1, that having previously sampled 600 cows , we wish to detect a possible decrease in prevalence of MAP infection of 5% at a subsequent sampling i.e., . However, funding dictates that only 160 cows can be tested. Thus, we estimate the probability of detecting this difference in prevalence, if it exists. Using α=0.05 and k=0.267, power=Φ(0.395)=0.65. If 160 cows are sampled, there is a probability of 0.65 of detecting a change of 0.05 in apparent prevalence. If we were to sample 240 animals (k=0.398), power=0.80.

7.8.3. Power calculations (statistically dependent samples)

From (30), with matched pairs or repeated measures data, the power to detect a difference is given by:

(39)

where n _p, p ₁₂, p ₂₁ and z _α/2 are as defined in section 7.3.2.

7.8.3.1. Example

Assume that for the herd described in section 7.3.2, testing costs dictate that only 540 tests can be performed (270 animals). Using the information in section 7.3.2 with n _p≈270, from (39) Φ(Z _β)=0.64 and power=0.74. Hence, there is a 74% chance of detecting an increase in apparent prevalence of 0.05, if it occurred.