Introduction
Understanding the extent to which transmission of an infection has occurred is of critical public- health importance. Cumulative incidence – estimated by proportion of individuals with serological evidence of a prior infection with a pathogen – can inform policy by indicating the number of individuals with acquired humoral immunity as well as how the risk of infection and acquisition of immunity varies over time and among population subgroups. It also allows estimation of other metrics, such as the infection fatality rate (IFR) and infection hospitalization rate (IHR) that are useful measures of disease severity.
However, estimating cumulative infection rates can be challenging. For example, during the COVID-19 pandemic, observed case counts – based on cases reported by clinicians and labs – were the primary method for tracking epidemiology. But these counts were severely biased both by asymptomatic and minimally symptomatic disease leading to a lack of diagnosis and limited testing capacity. Studies suggest that as few as 10% of cases were detected during the first 4 months [Reference Sullivan1–Reference Stringhini3].
One method for estimating cumulative incidence is to use the prevalence of antibodies against the virus SARS-CoV-2. Antibodies specific to this virus reliably increase following infection, allowing prevalence of antibodies, or seroprevalence, to detect SARS-CoV-2 infections missed by testing [Reference Lipsitch4, Reference Levin5]. Seroprevalence surveys of a representative sample of the population can provide useful estimates of the cumulative incidence of prior infection [Reference Brazeau6–Reference Meyer8].
However, for seroprevalence to represent cumulative incidence, several sources of bias must be addressed. First, the assays used to estimate seroprevalence are imperfectly sensitive and specific. Thus, obtaining the true seroprevalence requires adjusting observed seroprevalence to account for the expected amount of both false positives and false negatives. There are several approaches for doing this including the fully Bayesian approach developed by Meyer et al. [Reference Meyer8], which also provides valid uncertainty estimates.
A second problem is that antibody concentration wanes over time, which creates further bias when inferring cumulative incidence from seroprevalence [Reference Müller9, Reference Hay, Routledge and Takahashi10]. Antibody titres can decrease below a particular serological assay’s limit of detection. This seroreversion reduces assay sensitivity with time following infection [Reference Meyer8, Reference Hay, Routledge and Takahashi10–Reference Owusu-Boaitey12], which may cause observed seroprevalence to decrease with time in longitudinal testing of populations [Reference Ward13]. Thus, it is not sufficient to adjust seroprevalence estimates for sensitivity at a single baseline time point, but rather the adjustment must account for how sensitivity declines over time. If this is not done, seroreversion will cause seroprevalence to meaningfully underestimate post-infection immunity and cumulative incidence [Reference Buss7, Reference Takahashi14]. Adjusting seroprevalence using time-varying sensitivity estimates can correct for seroreversion and better approximate cumulative incidence [Reference Meyer8, Reference Hay, Routledge and Takahashi10, Reference Lohse15].
Seroreversion adjustment requires assumptions regarding pandemic severity and immunity that can impact the estimates. Takahashi et al. [Reference Takahashi14], for example, argued seroreversion biased estimates more strongly than suggested by a prior study [Reference Brazeau6]. Buss et al. [Reference Buss7] inferred substantial seroreversion among previously infected residents of Manaus, Brazil, such that post-infection immunity missed by serology might be sufficient for herd immunity and prevention of further waves of infection. Later analyses implied less seroreversion, consistent with a further wave that occurred in Manaus [Reference Kadelka11, Reference Takahashi14, Reference He16, Reference Lalwani17]. Assay-specific differences in time-varying sensitivity could also explain why different assays administered to the same population yield divergent seroprevalence estimates [Reference Lohse15].
Several different approaches to seroreversion correction have been used, though failing to account for seroreversion remains a persistent problem [Reference Haile and Kronthaler18]. Some analyses applied the same correction across assays [Reference Sullivan1, Reference Brazeau6, Reference Pezzullo19, Reference Axfors and Ioannidis20]. COVIDVu seroprevalence studies, for example, applied the anti-spike indirect ELISA adjustments of Shioda et al. [Reference Shioda21] to an anti-nucleocapsid direct ELISA [Reference Sullivan1, Reference Chamberlain22, Reference Lamba23]. Other studies used assay-specific adjustments for a small number of assays [Reference Levin5, Reference Lohse15]. Seroprevalence surveys also detected seroreversion using individuals in their sample with a previously diagnosed SARS-CoV-2 infection, or prior-positives, who then tested seropositive [Reference Neuhauser2, Reference Mutevedzi24, Reference Fantin25]. Levin et al. [Reference Levin5] combined data on prior-positives across multiple study populations to calculate time-varying sensitivity for one assay. Barber et al. [Reference Barber26] instead used prior-positive data from several studies to estimate time-varying sensitivity for eight assays. They extended their model to out-of-sample assays based on the isotype of, and SARS-CoV-2 antigen targeted by, antibodies detected by the assays. Other assay characteristics were not included.
Because some assays exhibit more time-varying sensitivity than others, seroreversion adjustments must take the assay into account. We illustrate the potential for this issue in Figure 1. The unprecedented number of seroprevalence surveys performed during the SARS-CoV-2 pandemic permitted better quantification of the contribution of assay design to time-varying sensitivity. Owusu-Boaitey et al. [Reference Owusu-Boaitey12] used data on prior-positives to calculate assay-specific time-varying sensitivity for fifty assays, while also quantifying the contribution of assay design and antigen target to time-varying sensitivity. This quantification yielded design-specific and antigen-specific predictions consistent with time-varying sensitivity for two out-of-sample assays employed in seroprevalence studies [Reference Bešević27, 28]. The analysis was restricted to before the onset of widespread SARS-CoV-2 vaccination and thus reflects seroprevalence induced by infection instead of population immunity from vaccination.
Example of seroreversion-induced biases. Each panel shows different sensitivity and infection dynamics prior to a serosurvey study. The red line indicates the time-dependent sensitivity of an assay for detecting infections that occurred before the serosurvey. In the left column, the sensitivity curve decays slowly; in the right column, it decays more rapidly. The solid black line represents the true number of infections in an epidemic wave (same number of infections for each panel). In the top row, there is a short delay between the epidemic wave and the serosurvey (infections further to the right); in the bottom row the delay is longer (infections further to the left). Lastly, the dashed black line indicates the number of infections detected by each serosurvey. It is obtained by multiplying the assay’s sensitivity by the true infections at each time point, neglecting false positives for expository simplicity. The proportion of detected infections depends strongly on the interaction between the assay-specific sensitivity decay and the serosurvey timing.
Figure 1. Long description
A four-panel grid of line graphs. Each graph shares a primary x-axis representing Weeks before serosurvey from 40 to 0, a left y-axis for Infections, and a right red y-axis for Sensitivity from 0.0 to 1.0.
Top-Left (Short-delay survey, Slow-decay assay): The true infection wave peaks at 15 weeks. The red sensitivity curve is high (near 0.9) and flat. The dashed line for detected infections almost perfectly overlaps the solid black true infection curve.
Top-Right (Short-delay survey, Fast-decay assay): The true infection wave peaks at 15 weeks. The red sensitivity curve decays more steeply toward the left. The dashed line for detected infections is slightly lower than the true peak, showing a small gap.
Bottom-Left (Long-delay survey, Slow-decay assay): The true infection wave is shifted left, peaking at 30 weeks. The red sensitivity curve remains relatively high (around 0.7). The dashed line for detected infections captures a significant portion of the wave but is visibly lower than the true peak.
Bottom-Right (Long-delay survey, Fast-decay assay): The true infection wave peaks at 30 weeks. The red sensitivity curve has decayed significantly to approximately 0.2 at the time of the wave. Consequently, the dashed line and grey shaded area for detected infections are much smaller than the true infection curve, indicating a large underestimation bias.
A legend at the bottom identifies the red line as Sensitivity, the solid black line as True infections, and the dashed black line as Detected infections.
The present work develops a Bayesian approach for these assay-specific time-varying sensitivity estimates [Reference Owusu-Boaitey12] to infer seroreversion-corrected cumulative incidence from seroprevalence surveys conducted in the United States before the onset of SARS-CoV-2 vaccination. This approach mitigates the impact of time-varying sensitivity differences across assays and accounts for the delay between infection and serosurvey. It also yields more realistic cumulative incidence estimates that incorporate uncertainty from time-varying sensitivity and from the number of samples used to estimate assay specificity [Reference Gelman and Carpenter29]. In order to test the method and see how much difference the phenomenon can make, we applied this model to seroprevalence studies to obtain more accurate estimates of cumulative incidence of SARS-CoV-2 infection in 2020. Our estimates of the contribution of assay design to time-varying sensitivity and cumulative incidence estimates could be applied to assays for other pathogens.
Methods
Data
We constructed a model to estimate COVID-19 cumulative incidence from population-representative seroprevalence surveys using three pieces of information for each population: observed seroprevalence data, information on how the distribution of time elapsed between infection and the seroprevalence surveys (here using the observed pattern of excess mortality), and information on how assays’ sensitivity declines over time after infection.
This study estimates cumulative incidence using previously published seroprevalence surveys that employed probability sampling of the general population. Web Supplementary Appendix S1 includes selection criteria for the seroprevalence surveys and the assessed seroprevalence studies. All included studies were performed in 2020, before the onset of widespread SARS-CoV-2 vaccination. We initially included only US studies to have relative consistency in data quality. However, we subsequently added a series of UK studies as a sensitivity analysis in order to capture more variation in assays’ risk of seroreversion and greater variation in the length of time after infection. For each study we recorded the region sampled, sampling start and end date, median date, serological assay(s) used for testing, number of individuals tested, and reported seroprevalence without adjustments for sensitivity and specificity. Population-weighted seroprevalence was used if provided by the seroprevalence study; otherwise, unweighted seroprevalence was recorded. We also recorded time-invariant assay sensitivity and specificity provided by the assay manufacturer or from another source, consistent with a prior publication [Reference Sullivan1].
Weekly excess mortality data for each region sampled were collected from a publicly available dataset [Reference Neuhauser2, 30]. Excess deaths were tabulated by date of death from the week of March 1, 2020, to three weeks past the median date of serological sampling. Three weeks was selected based on a three-week lag between infection and death, assuming one week lags between infection, symptom onset, seroconversion, and death [Reference Stringhini3–Reference Brazeau6, Reference Yadav31, Reference Zhou32]. While other sources of information about the timing of infections in a population – such as observed hospitalizations – could also be used, we based our estimates on excess mortality because it is expected to be least sensitive to limited testing capabilities or hospital capacity.
Assay-specific sensitivity estimates for each month following infection were taken from Owusu-Boaitey et al. [Reference Owusu-Boaitey12]. These estimates were derived from the proportion of individuals with a previously diagnosed SARS-CoV-2 infection who tested seropositive in seroprevalence studies performed on the general population. This proportion was used as an estimate of assay sensitivity, and data on this proportion were pooled together for studies that used the same assay. Reported median time between infection diagnosis and serological testing was taken from the seroprevalence studies. In cases where studies did not report this median time, it was inferred from reported case curves. From these data Owusu-Boaitey et al. inferred assay-specific monthly estimates of sensitivity up to 14 months following infection. Assays were also pooled by assay design, antibody isotype and antigen target of antibodies to quantify the contribution of these factors to time-varying sensitivity. Based on these estimates [Reference Owusu-Boaitey12], we classified lateral flow assays and anti-nucleocapsid indirect ELISAs as high risk from seroreversion, while other assay types were classified as low risk. Assay-specific monthly estimates of sensitivity were used for each assay, except for the University of Oxford assay for which assay-specific estimates were not provided in Owusu-Boaitey et al. For this assay we instead used anti-spike indirect ELISA monthly estimates from Owusu-Boaitey et al. to match the antigen target and assay design for the University of Oxford assay.
Statistical analysis
The standard correction for sensitivity and specificity (known as the Rogen–Gladen formula [Reference Rogan and Gladen33]) has the form
where
$ C $
denotes the cumulative incidence,
$ P $
denotes the proportion reactive,
$ {S}_p $
denotes the specificity, and
$ {S}_e $
is the sensitivity. Each of these quantities is a proportion, although under the standard correction
$ C $
can be below 0 or above 1. Any model employing this needs to take that into account. An alternative representation re-expresses this quantity as
Let
$ Y=P+{S}_p-1 $
. By this definition,
$ Y $
is no longer bounded below by 0 but by
$ -1 $
and is still bounded above by 1. Further, we assume
$ Y $
is measured with error since it depends on two unknown quantities that both must be estimated. Let
$ C=\mathrm{D}\unicode{x03B2} $
where
$ D $
is an
$ N\times k $
design matrix and
$ \beta $
is a
$ k\times 1 $
vector of cumulative incidences to estimate. The quantity
$ k $
is the number of seroprevalences to be estimated. For
$ P $
and
$ D $
, we use binary data representations. Thus,
$ P $
is a
$ N\times 1 $
vector of 0 s and 1 s with 1 corresponding to positive reactivity and 0 to negative. For a single estimated incidence, the data for
$ D $
can simply be a vector of 1 s with length equal to
$ P $
. If estimating multiple incidences, clustered by region or surveys using the same assay for example,
$ D $
becomes an appropriately structured matrix of indicator variables.
The right-hand side of Equation (1) is then
$ C\left({S}_e+{S}_p-1\right)= D\beta \left({S}_e+{S}_p-1\right) $
. Since
$ \left({S}_e+{S}_p-1\right) $
is a scalar, this term can be rewritten as
$ \left({S}_e+{S}_p-1\right) D\beta $
when estimating a single incidence. Thus, we consider the following linear model to estimate the unknown parameter
$ \beta $
:
where
$ \epsilon $
is a mean zero Gaussian error term with variance
$ {\sigma}_{\epsilon}^2 $
. This model can be simplified by defining
$ X=\left({S}_e+{S}_p-1\right)D $
, giving the linear regression form
When only one seroprevalence is estimated
$ \left(k=1\right) $
, this is just a simple linear model.
The parameter
$ \beta $
only has non-zero support on the unit interval and must be bounded. To enforce this constraint, we employ a fully Bayesian approach and impose a truncated normal prior on the parameter with truncation to the interval
$ \left\{0,1\right\} $
and variance
$ {\sigma}_0^2 $
which we fix at a large value, e.g. 100,000. We place an inverse gamma prior on
$ {\sigma}_{\epsilon}^2 $
with parameters
$ {A}_{\epsilon } $
and
$ {B}_{\epsilon } $
, both set to something small, e.g. 0.25.
Sensitivity and specificity can be fixed at specific values, or their posteriors can be estimated first and used when sampling from the posterior of
$ \beta $
. The latter approach naturally incorporates the variability of the sensitivity and specificity estimates into the estimation of the cumulative incidence. To estimate specificity,
$ {S}_p $
, we select a Beta conjugate prior for
$ {S}_p $
with prior parameters chosen to be small (0.1), thus limiting their influence. We then draw from its posterior distribution where the data for the likelihood is determined by the test’s reported characteristics; specifically the number of false positives (
$ FP $
) and the number of true negatives (
$ TN $
), see [Reference Owusu-Boaitey12]. In context, the prior parameters are equivalent to adding 0.1 true negatives and 0.1 false positives.
We estimate the posterior for
$ {S}_e $
by first generating a posterior distribution for the seroreversion curve [Reference Owusu-Boaitey12], with estimation occurring on a weekly time scale. Drawing on excess mortality data to estimate the average time since infection for respondents to seroprevalence surveys, we allow S
e to vary over time. This approach uses a hierarchical logistic model for estimation with predicted values representing weekly seroreversion curves. We then weight the predicted posterior by the proportion of weekly excess mortality over the desired interval as an indication of the estimated time lag between infection and the survey date. This creates an excess mortality-adjusted seroreversion curve and corresponding distribution. After averaging across time, we obtain a posterior distribution of excess mortality-adjusted seroreversion, which we use for the sensitivity estimate in
$ X $
, i.e.
$ {S}_e $
. The values from the posterior distribution of
$ {S}_e $
and
$ {S}_p $
are fed into the Gibbs sampler for the posterior of
$ \beta $
in a one-to-one fashion. That is, for each pair of samples from the joint posterior of
$ \left\{{S}_p,{S}_e\right\} $
, we obtain one draw from the posterior of
$ \beta $
. The estimation procedure is described in Algorithm 1. Exploratory empirical studies suggest this approach performs well in recovering the ‘true’ cumulative incidence.
To represent the impact of the assay-specific time-varying adjustment, we report the ratio of adjusted seroprevalence estimates to the naive or unadjusted estimate from the original studies, defined as ratio
We calculate this metric in two ways. First, we adjust seroprevalence for test-specific sensitivity and specificity at the time of test, i.e. a time-invariant adjustment. This estimation is done using the method described in Algorithm 1 but setting
$ {S}_e $
equal to the time 0 estimate of seroreversion. Second, we implement Algorithm 1 with a time-varying
$ {S}_e $
calculation as described above. This gives us both time-invariant and time-varying adjusted seroprevalence. We construct the posterior distribution of
$ B $
under both models.
Algorithm 1. Procedure for estimating cumulative incidence, adjusted for excess mortality and seroreversion. The subscript
$ \left\{0,1\right\} $
indicates truncation to the unit interval,
$ {I}_{k\times k} $
is a
$ k\times k $
identity matrix,
$ InvGamma $
denotes the inverse gamma distribution, and
$ n $
is the sample size. The notation
$ ^{(m)} $
indicates the
$ m $
th draw from the posterior while
$ ^{\left(m-1\right)} $
indicates the
$ \left(m-1\right) $
st draw.
-
1. Estimate Sensitivity
-
a. Draw $ M $
total posterior samples from the hierarchical logistic model for seroreversion curves -
b. Obtain posterior of predicted seroreversion on weekly scale
-
c. Weight predicted posterior
-
d. Average over time to obtain posterior of $ {S}_e $
-
-
2. Estimate Specificity
-
a. Draw M total posterior samples from $ {S}_p\sim Beta\left(\mathrm{TN}+0.1,\mathrm{FP}+0.1\right) $
-
-
3. Estimate Cumulative Incidence
-
○ Iterate through the following steps to obtain $ M $
total posterior samples-
i. Construct $ {Y}^{(m)}=P+{S}_p^{\left(m-1\right)}-1 $
and
$ {X}^{\left(m-1\right)}=\left[{S}_e^{\left(m-1\right)}+{S}_p^{\left(m-1\right)}-1\right]D $
-
ii. Draw one sample from $ {\beta}^{\left(m-1\right)}\sim \mathrm{Normal}{\left[{\sigma}_{\beta}^2\left\{\frac{1}{{\sigma_{\epsilon}^2}^{(m)}}{X^{\left(m-1\right)}}^{\prime }{Y}^{\left(m-1\right)}+\frac{1}{\sigma_0^2}{I}_{k\times k}{\mu}_0\right\},{\sigma}_{\beta}^2\;\right]}_{\left\{0,1\right\}} $
, where
$ {\sigma}_{\beta}^2=\left(\frac{1}{{\sigma_{\epsilon}^2}^{(m)}}{X^{\left(m-1\right)}}^{\prime }{X}^{\left(m-1\right)}+{\sigma}_0^2\right) $
-
iii. Draw one sample from $ {\sigma_{\epsilon}^2}^{(m)}\sim \mathrm{InvGamma}\left[\frac{n}{2}+{A}_{\epsilon },{B}_{\epsilon }+\frac{1}{2}\left\{{Y}^{\left(m-1\right)}-{\beta}^{\left(m-1\right)}{X}^{\left(m-1\right)}\right\}^{\prime}\left\{{Y}^{\left(m-1\right)}-{\beta}^{\left(m-1\right)}{X}^{\left(m-1\right)}\right\}\hskip0.24em \right] $
-
-
Results
We have calculated adjustment factors for 15 surveys conducted in 11 jurisdictions in the United States (Table 1). We also report the posterior probability that the adjustment factor is greater than 1; that is, the probability that the adjusted seroprevalence is larger than the reported uncorrected seroprevalence. Estimated adjustment factors for the time-invariant estimates range from 0.938 (95% CrI: 0.428, 1.407) in Puerto Rico to 1.214 in Georgia (95% CrI: 0.954, 1.779). For all surveys except the one conducted in Puerto Rico, it is more likely than not that applying the time-invariant correction results in higher estimated seroprevalence than the reported uncorrected estimate.
Posterior estimates of the adjustment factor, B, comparing reported seroprevalence to adjusted under both time-invariant and time-varying adjustments
Table 1. Long description
The table consists of seven columns: Jurisdiction, Average interval from infection (weeks), Risk from seroreversion, Time-invariant/reported Est. (95% Cr I), Time-invariant P(B > 1), Time-varying/reported Est. (95% Cr I), and Time-varying P(B > 1).
Key data points include:
* United States: 18 weeks, Low risk. Time-invariant Est. 1.207 (0.994, 1.763), P(B > 1) 0.971. Time-varying Est. 1.289 (1.009, 2.217), P(B > 1) 0.980.
* Arkansas Phase 2: 4 weeks, Low risk. Time-invariant Est. 1.051 (0.409, 1.789), P(B > 1) 0.560. Time-varying Est. 1.066 (0.419, 1.805), P(B > 1) 0.576.
* Arkansas Phase 7: 11 weeks, Low risk. Time-invariant Est. 1.156 (0.826, 1.632), P(B > 1) 0.811. Time-varying Est. 1.174 (0.835, 1.706), P(B > 1) 0.831.
* California: 16 weeks, Low risk. Time-invariant Est. 1.208 (0.824, 1.867), P(B > 1) 0.843. Time-varying Est. 1.280 (0.855, 2.154), P(B > 1) 0.895.
* New York City: 18 weeks, Low risk. Time-invariant Est. 1.133 (0.927, 1.359), P(B > 1) 0.896. Time-varying Est. 1.167 (0.956, 1.398), P(B > 1) 0.937.
* Ohio: 9 weeks, High risk. Time-invariant Est. 1.049 (0.163, 1.996), P(B > 1) 0.541. Time-varying Est. 1.225 (0.182, 2.314), P(B > 1) 0.657.
* Puerto Rico: 17 weeks, High risk. Time-invariant Est. 0.938 (0.428, 1.407), P(B > 1) 0.398. Time-varying Est. 1.319 (0.585, 2.272), P(B > 1) 0.802.
* Wisconsin Phase 2: 11 weeks, High risk. Time-invariant Est. 1.102 (0.836, 1.368), P(B > 1) 0.777. Time-varying Est. 1.364 (1.030, 1.712), P(B > 1) 0.983.
Note: Estimates represent posterior medians and are accompanied by 95% credible intervals.
Arkansas data used the Beckman ACCESS assay. California and Georgia used BioRad Platelia. Connecticut used Vitros Ortho, S CoV2 IgG. New York City used DiaSorin. Ohio, Rhode Island, Indiana, and Wisconsin used Abbott Architect. Puerto Rico used Abbott Alinity i.
Adjustment factors that compare time-varying estimates to the reported unadjusted estimates deviate farther from equivalence and for all surveys it is more likely than not that the correction results in higher estimated seroprevalence. The smallest adjustment factor was the 1.066 observed in Arkansas at an average of four weeks post-infection (95% CrI: 0.419, 1.805). The largest was Wisconsin at an average of 11 weeks post-infection: 1.364 (95% CrI: 1.030, 1.712). Figure 2 suggests that the largest differences between time-invariant and time-varying adjustment factors occur for those seroprevalence surveys that were conducted later in the location’s epidemic and thus had a longer average time from infection to survey, as well as for surveys that used assays with a greater susceptibility to seroreversion.
Adjustment factor B when calculated using the time-invariant sensitivity (circles) versus calculated using time-varying sensitivity (triangles). Solid blue indicates low test-specific seroreversion risk while dashed red indicates high seroreversion risk.
Figure 2. Long description
A scatter plot with the x-axis labeled Average interval from infection to testing in weeks, ranging from 0 to 20. The y-axis is labeled Adjustment factor B, ranging from 0.9 to 1.4. A horizontal dashed line is at y equals 1.0.
Legend at top left:
* Red dashed line with triangle: High seroreversion risk.
* Blue solid line with circle: Low seroreversion risk.
Data structure:
* Circles represent unadjusted values using time-invariant sensitivity.
* Triangles represent adjusted values using time-varying sensitivity.
* Vertical lines connect each circle to its corresponding triangle, indicating the magnitude of adjustment.
Trends from left to right:
* At 3 to 4 weeks: Small upward adjustments for both low (blue) and high (red) risk groups, with B values between 1.05 and 1.15.
* At 7 to 12 weeks: Adjustments increase in magnitude. High-risk (red) adjustments are significantly larger, with triangles reaching up to 1.37, while low-risk (blue) adjustments remain below 1.25.
* At 16 to 18 weeks: The largest adjustments occur. One high-risk data point shows a massive shift from an unadjusted value below 0.95 to an adjusted value above 1.3. Low-risk points show consistent upward shifts from approximately 1.2 to 1.3.
When considering only surveys that used an assay with a high susceptibility to seroreversion, the degree of deviation between the time-invariant and time-varying adjustment factors increased monotonically with time. For example, the first Indiana sample (with an average of only 3 weeks passing before the survey) had a time-invariant adjustment factor of 1.069 (95% CrI: 0.414, 1.438) compared to a time-varying adjustment factor of 1.110 (95% CrI: 0.416, 1.493). With these assays, at 11 weeks, the second Wisconsin survey had a time-invariant adjustment factor of 1.102 (95% CrI: 0.836, 1.368) compared to a time-varying adjustment factor of 1.364 (95% CrI: 1.030, 1.712). For those assays that have a lower susceptibility to seroreversion the differences between time-invariant and time-varying adjustment factors tend to be substantially smaller.
We present unadjusted reported seroprevalence as well as the population-level estimate with time-invariant and time-varying adjustments in Table 2. Uncorrected estimates range from 0.9% in the Ohio survey to 20.8% in the New York City survey. The maximum difference we estimated between the uncorrected reported seroprevalence and the estimate with time-varying adjustments was 3.4 percentage points in the New York survey (20.8% versus 24.2%). Even with a long 17-week average period between infection and the survey, 2.7 percentage points of the difference were accounted for by the time-invariant adjustments due to the assay being one with a lower risk from seroreversion. However, the second-largest difference was 2.5 percentage points in the second Wisconsin sample (6.8% versus 9.3%). With shorter average time of 11 weeks post-infection but using an assay with higher risk from seroreversion, the time-invariant adjustments accounted for only 0.7 percentage points, and the time-varying adjustments accounted for the remaining 1.8. In 11 of 15 surveys, the difference was less than two percentage points.
Seroprevalence estimates
Table 2. Long description
The table contains seven columns: Jurisdiction, Average interval from infection (weeks), Risk from seroreversion, Reported, Time-invariant, Time-varying, and Varying/invariant.
Key data points include:
* United States: 18 weeks, Low risk, Reported 0.0524, Time-invariant 0.063, Time-varying 0.068.
* Arkansas Phase 7: 11 weeks, Low risk, Reported 0.1420, Time-invariant 0.164, Time-varying 0.167.
* California: 16 weeks, Low risk, Reported 0.0450, Time-invariant 0.054, Time-varying 0.058.
* New York City: 18 weeks, Low risk, Reported 0.2076, Time-invariant 0.235, Time-varying 0.242.
* Ohio: 9 weeks, High risk, Reported 0.0090, Time-invariant 0.009, Time-varying 0.011.
* Puerto Rico: 17 weeks, High risk, Reported 0.0200, Time-invariant 0.019, Time-varying 0.026.
* Indiana Phase 2: 7 weeks, High risk, Reported 0.0150, Time-invariant 0.016, Time-varying 0.018.
* Wisconsin Phase 2: 11 weeks, High risk, Reported 0.0680, Time-invariant 0.075, Time-varying 0.093.
Adjusted estimates represent posterior medians with 95 percent credible intervals provided in brackets. The Varying/invariant column shows the ratio between the two models, ranging from 1.010 in Arkansas Phase 2 to 1.419 in Puerto Rico.
Note: Adjusted estimates represent posterior medians and are accompanied by 95% credible intervals.
Arkansas data used the Beckman ACCESS assay. California and Georgia used BioRad Platelia. Connecticut used Vitros Ortho, S CoV2 IgG. New York City used DiaSorin. Ohio, Rhode Island, Indiana, and Wisconsin used Abbott Architect. Puerto Rico used Abbott Alinity i.
As a sensitivity analysis, we include studies from London including the Real-time Assessment of Community Transmission (REACT-2) study [34], the UK Office of National Statistics (ONS) COVID-19 infection survey [35], and the Biobank SARS-CoV-2 Serology Study [Reference Bešević27]. Results from this analysis are available in Supplementary Tables S1 and S2 as well as Figure S1 of the Supplementary Material. REACT-2 had a high risk of test-specific seroreversion, and we note that the difference between the time-invariant and time-varying adjustments increases as the average interval from infection increases. Conversely, the UK ONS COVID-19 infection survey and Biobank had a low risk of test-specific seroreversion. While the time-varying adjustment does result in a larger seroprevalence estimate which increases as the average interval from infection increases, it is not as large a difference as we observe in the REACT-2 study.
Discussion
These results demonstrate that the effect of time-varying sensitivity is potentially large and that adjustment can reliably correct for it. Seroreversion decreases time-varying sensitivity more strongly for anti-nucleocapsid versus anti-spike assays, and when serological sampling occurs longer after infection [Reference Owusu-Boaitey12]. Our seroreversion adjustment ratio, for instance, is higher for a Wisconsin study using an anti-nucleocapsid assay compared to an Indiana study using the same assay but with a 2-month shorter delay between infection and serological sampling. At 4 months post-infection time-varying sensitivity adjustment increased cumulative incidence compared to time-invariant adjustment by a factor of 1.03 for a New York City study using an anti-spike assay, but by 1.42 for a study in Puerto Rico using an anti-nucleocapsid assay. Seroreversion can therefore strongly bias cumulative incidence estimates. At its most, seroreversion increased estimated cumulative incidence 1.36-fold at several months after infection compared to the reported seroprevalence estimates. Assay-specific adjustments are needed to detect whether this bias is present. At the same time, the large relative adjustment for Puerto Rico corresponded to a small absolute difference, from 1.9% to 2.6% cumulative incidence. Seroreversion can lead to larger absolute differences later in a pandemic due to longer time since infection and greater cumulative incidence. Getting the adjustment right can be critical when comparing seroprevalence surveys done at different times relative to the peak infection period using different assays.
Our Bayesian seroreversion corrections yield more valid estimates of cumulative incidence as well as valid credible intervals. An advantage of Bayesian over frequentist approaches is that Bayesian models can incorporate uncertainty in test sensitivity and specificity, the rate of seroreversion, and estimates of seroprevalence through the use of priors. Because all of these are usually drawn from relatively small samples, they often have meaningful uncertainty. Prior COVIDVu seroprevalence analyses of California, Georgia, and the USA overall applied the anti-spike indirect ELISA adjustments of Shioda et al. to an anti-nucleocapsid direct ELISA. Time-varying sensitivity was also inferred from decreasing seroprevalence with time in a convenience sample, resulting in a 2-fold increase from reported seroprevalence to cumulative incidence adjusted for time-varying sensitivity [Reference Sullivan1, Reference Shioda21–Reference Lamba23]. But factors other than seroreversion could contribute to decreasing seroprevalence, such as convenience sampling becoming more or less representative of the general population with time [Reference Kadelka11, Reference Takahashi14, Reference He16]. We instead infer time-varying sensitivity from serological testing of prior-positives, yielding greater sensitivity and lower cumulative incidence estimates than reported in COVIDVu. Our time-varying sensitivity-adjusted cumulative incidence estimates are broadly consistent with two prior studies of the USA, including the results of Barber et al. that applied time-varying sensitivity adjustments accounting for only antibody isotype and antigen target for eight assays [Reference Barber26, Reference Irons and Raftery36]. Our cumulative incidence estimates were higher than Barber et al. in Arkansas, likely due to the Arkansas seroprevalence study using a marketing database that overestimates seroprevalence in the general population [Reference Owusu-Boaitey37, Reference Cardenas38].
This analysis improves upon surveys that estimated time-varying sensitivity using study-internal prior-positives [Reference Neuhauser2, Reference Levin5, Reference Mutevedzi24, Reference Fantin25]. These surveys assumed the same amount of seroreversion among prior-positives compared to other study participants, with the participants being representative of the general population. But prior-positives may differ from the general population with respect to attributes that can impact seroreversion, such as COVID-19 severity and age [Reference Takahashi14, Reference Renk39], though age may not strongly impact time-varying sensitivity for some assay designs [Reference Einhauser40, Reference Radon41]. Our model mitigates this selection bias by excluding prior-positive studies that sampled only particular groups, such as nursing home residents. We also assume the same amount of seroreversion among prior-positives compared to reported cases in the same location, instead of compared to the overall population. This approach also draws on prior-positive data from multiple studies, in contrast to more limited data from only one study population [Reference Owusu-Boaitey12], and provides assay-specific time-varying sensitivity estimates for studies lacking data on prior-positives.
Despite these limitations, study-internal prior-positives usefully approximate time-varying sensitivity, especially in an outbreak lacking prior-positives data across multiple study populations. The study-internal proportion of prior-positives who tested seropositive better matches our time-varying sensitivity estimates than time-varying sensitivity estimates applied in three seroprevalence studies [Reference Sullivan1, Reference Chamberlain22, Reference Lamba23]. Our estimates still slightly exceeded this proportion of prior-positives, possibly due to lower sensitivity for the assay in question when compared to other anti-nucleocapsid direct ELISAs [Reference Owusu-Boaitey12]. These results support other seroprevalence studies that inferred time-varying sensitivity-adjusted cumulative incidence using prior-positives within the study sample [Reference Neuhauser2, Reference Fantin25], including studies that relied on those cumulative incidence estimates to infer severity measures [Reference Levin5, Reference Mutevedzi24].
Our time-varying sensitivity adjustments provide more plausible cumulative incidence estimates for inferring severity measures. Seroreversion causes large relative changes in estimated cumulative incidence, inflating estimates of fatality and hospitalization rates from infection (IFR and IHR, respectively) [Reference Brazeau6, Reference Takahashi14, Reference Barber26]. This substantial inflation occurs even with small absolute differences in cumulative incidence estimates. Our analysis suggests IFR was overestimated for Connecticut, Indiana, and Rhode Island due to the absence of adjustments for time-varying sensitivity [Reference Chan42–Reference Levin44]. COVIDVu research applied time-varying sensitivity adjustments in analyses of California, Georgia, and the USA nationally [Reference Sullivan1, Reference Chamberlain22, Reference Lamba23]. Our analysis suggests these adjustments underestimated sensitivity and thereby underestimated IFR. Replacing COVIDVu cumulative incidence estimates with our estimates, for example, increased the national adult IFR estimate from 0.9% to 1.5%, closer to another published estimate [Reference Barber26]. Applying our time-varying sensitivity adjustments would also clarify whether seroreversion strongly [Reference Takahashi14] or mildly [Reference Brazeau6] biases an unusually high IFR estimate for Italy. Our results justify caution regarding IFR analyses that assume the same time-varying sensitivity trend across assay designs and antigen targets [Reference Brazeau6, Reference Pezzullo19, Reference Axfors and Ioannidis20], given the impact of design and target on time-varying sensitivity. Multiple COVID-19 waves and underreporting of COVID-19 deaths [Reference Levin5, Reference Ward13, Reference Barber26, Reference Owusu-Boaitey45] may also bias IFR estimates that use only an initial peak in reported deaths to infer time since infection [Reference Pezzullo19, Reference Axfors and Ioannidis20], in contrast to the excess deaths used to infer time since infection in our approach.
Seroreversion adjustments for SARS-CoV-2 seroprevalence surveys may inform adjustments for surveys for other pathogens. The time-course of infections could be inferred from excess mortality, or by other surveillance methods for pathogens lacking weekly excess mortality estimates [Reference Thompson46]. Design-specific adjustments could apply to assays targeting pathogens other than SARS-CoV-2 [Reference Yao47], though antigen-specific adjustments will likely vary. Future work could generate time-varying sensitivity corrections more applicable across pathogens by quantifying how antigen characteristics impact seroreversion. Seroreversion may differ, for example, for surface antigen epitopes exposed to soluble and membrane-bound immunoglobulins versus for epitopes hidden deeper within the pathogen, as with SARS-CoV-2 spike versus nucleocapsid protein, respectively.
Our approach provides an improved estimate of the cumulative incidence of COVID-19 at the time of survey. In virtually all circumstances, the uncorrected reported seroprevalence values underestimate our cumulative incidence estimates by 1.1-fold and often meaningfully more. Such adjustments are critical for research and surveillance purposes requiring precise estimates of cumulative incidence, such as calculating IFRs or discerning population-wide herd immunity levels. For other purposes, though, adjustments are less important. For example, few routine public health decisions during a pandemic turn on the difference between an unadjusted 4% and adjusted 6% cumulative incidence – absolute differences commonly observed in our data. Thus, simpler, less time-consuming ‘naive’ methods that do not require waiting on the tabulation of mortality data for adjustments may suffice for some public health measures.
Several factors constrain our seroreversion analysis. We focus on serology studies performed before the introduction of SARS-CoV-2 vaccination to avoid detection of vaccine-induced antibodies and anti-spike vaccination reducing sensitivity of anti-nucleocapsid assays [Reference Dhakal48]. This approach, however, may be extended to post-vaccination time periods using seroreversion estimates for vaccine-induced antibodies. We choose to use excess mortality as an indicator of the lag between the average time of infection and the survey date because it was (a) readily available in a weekly time series and (b) less sensitive to temporal biases caused by shortages of tests, testing lags, and other factors that were common in 2020 than counts of cases or deaths. However, this approach implicitly assumes IFR does not significantly vary with time when using excess mortality to infer time since infection, similar to other seroreversion analyses [Reference Takahashi14, Reference Pezzullo19]. This assumption may not strongly bias inferred time when infections are concentrated in a narrow peak of deaths, or during time periods before factors such as vaccination and substantial transmission of variants impacted IFR [Reference De Boer49, Reference Perez-Guzman50]. Future work may test our methods’ robustness to changes in IFR. Further analysis may quantify how much time-varying sensitivity contributes to differences in reported seroprevalence within studies of the same population [Reference Murhekar51]. This would distinguish the impact of time-varying sensitivity from that of factors such as sampling bias [Reference Bartig52].
Adjusting for seroreversion becomes more important with time during outbreaks. Our Bayesian time-varying sensitivity adjustments illustrate a promising method for accounting for antigen and assay characteristics in seroreversion corrections, providing more reliable estimates of time-varying cumulative incidence, post-infection immunity, and severity measures.
Conclusions
Depending on the assay used, the naive use of tests with sensitivity that decays over time can seriously bias estimates of cumulative incidence of SARS-CoV-2 infection. The Bayesian approach to assay-specific time-varying sensitivity adjustment developed in this paper can reliably correct for this bias. In seroprevalence studies conducted in the United States in 2020, adjusting for time-varying sensitivity increased cumulative incidence up to 1.4-fold. This implies that previous seroreversion corrections overestimated cumulative incidence in the United States up to 1.7-fold by not accounting for assay-specific effects. Similarly, estimates of the IFR can be substantially biased, with a corrected estimate in one study 1.7 times higher than the reported rate.
Abbreviation
- CI
-
confidence interval
- COVID-19
-
coronavirus disease 2019
- CrI
-
credible interval
- ELISA
-
enzyme-linked immunosorbent assay
- IFR
-
infection fatality rate
- IHR
-
infection hospitalization rate
- SARS-CoV-2
-
severe acute respiratory syndrome coronavirus 2
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/S0950268826102325.
Data availability statement
The data that support the findings of this study are openly available at https://data.mendeley.com/datasets/yccc76j326/1.
Acknowledgements
The authors thank Dr. Karen A. Alroy for sharing additional seroprevalence data.
Author contribution
Conceptualization: N.O., M.J.M., D.H., L.B., M.L., M.A.S., J.D.K.; Data curation: N.O., M.J.M., D.H., L.B., M.L., S.C.; Investigation: N.O., D.H., L.B., M.L., S.C., M.A.S., J.D.K.; Methodology: N.O., M.J.M., D.H., L.B., M.A.S., J.D.K.; Project administration: N.O., M.A.S., J.D.K.; Supervision: N.O., M.J.M., M.A.S., J.D.K.; Validation: J.D.K.; Visualization: M.J.M., S.C.; Writing – original draft: N.O., M.J.M., M.A.S., J.D.K.; Formal analysis: M.J.M., D.H., L.B., S.C.; Writing – review & editing: D.H., L.B., M.L., S.C.
Funding statement
This research received no specific grant from any funding agency in the commercial or not-for-profit sectors.
Competing interests
All authors declare no conflicts of interest.
Open access
