Child maltreatment is a worldwide health concern involving the physical abuse, neglect, sexual abuse, and psychological abuse of individuals under the age of 18 (World Health Organization, 2014). Research over the past three decades demonstrates that child maltreatment is associated with broad domains of adverse child health and development (Cicchetti & Handley, Reference Cicchetti and Handley2017; Hussey, Chang, & Kotch, Reference Hussey, Chang and Kotch2006; Jonson-Reid, Kohl, & Drake, Reference Jonson-Reid, Kohl and Drake2012; Oh et al., Reference Oh, Jerman, Silverio Marques, Koita, Purewal Boparai, Burke Harris and Bucci2018) while increasing the risk for many of the major causes of morbidity and mortality in adulthood (Felitti et al., Reference Felitti, Anda, Nordenberg, Williamson, Spitz, Edwards and Marks1998; Gilbert et al., Reference Gilbert, Widom, Browne, Fergusson, Webb and Janson2009; Walsh, McLaughlin, Hamilton, & Keyes, Reference Walsh, McLaughlin, Hamilton and Keyes2017). Within this body of research, however, lies considerable cross-study variation in whether child maltreatment increases the risk for specific outcomes as well as in the generation of reproducible effect size estimates quantifying the magnitude of such risk. For example, some studies report a statistically significant risk for teen pregnancy following child maltreatment (Garwood, Gerassi, Jonson-Reid, Plax, & Drake, Reference Garwood, Gerassi, Jonson-Reid, Plax and Drake2015; Noll, Trickett, & Putnam, Reference Noll, Trickett and Putnam2003) while other studies do not (Widom & Kuhns, Reference Widom and Kuhns1996). Even when there is agreement that child maltreatment increases the risk for a specific outcome, such as obesity (Danese & Tan, Reference Danese and Tan2014), the magnitudes of effect size estimates are often highly discrepant across studies (Lissau & Sorensen, Reference Lissau and Sorensen1994; Thomas, Hypponen, & Power, Reference Thomas, Hypponen and Power2008). Such variation weakens conclusions about the potential risk child maltreatment poses for adverse health and development. This has implications for scientists attempting to replicate prior research, child welfare administrators setting policies based on available scientific evidence, as well as providers who must decide whether or not to consider a patient's child maltreatment history when treating specific health complaints.
Contamination, when subjects recruited to a comparison condition or assigned to a control group are exposed to the event or intervention under investigation (Cuzick, Edwards, & Segnan, Reference Cuzick, Edwards and Segnan1997), is a methodological phenomenon that may contribute to variation in the significance and magnitude of effect size estimates reported across child maltreatment studies. One source of contamination in child maltreatment research is the recruitment of comparison subjects who share the same demographic characteristics as subjects in a child maltreatment condition but who vary in terms of child maltreatment status, as this strategy provides an effective counterfactual in nonrandomized, observational research. For example, demographic matching of comparison subjects in observational research has long been regarded as an effective design strategy for controlling extraneous variability due to these observed confounders so that the unique and unbiased effects of child maltreatment can be estimated (Rubin, Reference Rubin1973; Widom, Reference Widom1989). However, while these recruitment and matching strategies can create balance across child maltreatment and comparison conditions on confounders observed at study enrollment, the demographic characteristics most often used to recruit or match comparison subjects, such as age, race, family income, and single-parent households, are themselves well-established risk factors for child maltreatment (Institute of Medicine, 2011). Explicit inclusion of these demographic characteristics therefore also comes at a cost, as it introduces the risk that a certain number of subjects within a comparison condition have already been exposed to child maltreatment or will be exposed during longitudinal follow-up. As the prevalence of such contamination varies across individual child maltreatment studies, so will any bias it has on resulting effect size estimates regardless of the outcomes that are examined in those studies.
Prior research outside the substantive area of child maltreatment has shown through directly observed (Craven, Marsh, Debus, & Jayasinghe, Reference Craven, Marsh, Debus and Jayasinghe2001) and simulated results (Marfo & Okyere, Reference Marfo and Okyere2019) that contamination creates a downward bias on the significance and magnitude of effect size estimates by minimizing between-group differences when they exist. Two well-characterized child maltreatment studies have demonstrated both the prevalence and impact of contamination bias by reporting changes in effect size estimates before and after controlling contamination. The first is a retrospective cohort study that examined the risk for psychiatric disorders in young adulthood for those who were the subject of an official child maltreatment report (Scott, Smith, & Ellis, Reference Scott, Smith and Ellis2010). Despite creating a comparison condition where no one was the subject of a child maltreatment report, 15.4% of these subjects self-reported experiencing maltreatment in childhood. When this contamination was controlled by removing these subjects from statistical modeling, the effect size magnitudes for child maltreatment increased by 22% for any past-year and 32% for any lifetime history of a psychiatric disorder. Of note, the risk for several individual disorders reached statistical significance only after contamination was controlled.
The second is a prospective cohort study that examined global indicators of female adolescent development following exposure to substantiated child maltreatment (Shenk, Noll, Peugh, Griffin, & Bensman, Reference Shenk, Noll, Peugh, Griffin and Bensman2016). Over 44% of the comparison condition in this study either self-reported or had their own history of substantiated child maltreatment. When this contamination was controlled by removing these subjects from statistical modeling, effect size estimates for child maltreatment increased by 24%–130% across obesity, teen births, past-month cigarette use, and clinical levels of major depressive disorder symptoms. Importantly, only when contamination was controlled did the risk for all four outcomes reach statistical significance. Together these two initial studies suggest that contamination exists in child maltreatment research with prevalence estimates varying considerably across studies. Moreover, failing to control contamination resulted in weaker effect size estimates for multiple health and developmental outcomes, making it more difficult to detect a risk attributable to child maltreatment when it existed. Conclusions about the health and developmental risks following child maltreatment can therefore vary within and across studies depending on the degree of contamination present and whether this contamination is controlled.
Fortunately, the same methods used to establish child maltreatment and comparison conditions can be used to detect and control contamination. There are two primary methods for detecting child maltreatment in longitudinal research: (a) self-report, such as screening surveys, interviews, and questionnaires assessing one's subjective history of exposure to child maltreatment, and (b) official case records used to confirm or substantiate child maltreatment, such as those obtained from investigations conducted by government agencies. Self-report methods have the advantage of detecting cases of child maltreatment that go unreported to government agencies. Nationally representative surveys in the US indicate that the past-year incidence of child maltreatment is approximately 152 per 1,000 children (Finkelhor, Turner, Shattuck, & Hamby, Reference Finkelhor, Turner, Shattuck and Hamby2015), an estimate considerably higher than the 31.8 per 1,000 estimate obtained for official case records (U.S. Department of Health and Human Services, 2019). This suggests that using only official case records, and especially substantiated records, to establish and maintain comparison conditions would still contain contamination by including comparison subjects who do not have an official case record but who would self-report child maltreatment if assessed.
Official case records, however, have the advantage of detecting instances of child maltreatment that go unreported during a self-report assessment. Approximately 40%–50% of those with an official history of child maltreatment fail to disclose this history during longitudinal research (Baldwin, Reuben, Newbury, & Danese, Reference Baldwin, Reuben, Newbury and Danese2019; Everson et al., Reference Everson, Smith, Hussey, English, Litrownik, Dubowitz and Runyan2008; Widom & Shepard, Reference Widom and Shepard1996). This suggests comparison conditions established and maintained using only self-report methods would contain contamination by including subjects who do not self-report child maltreatment but who have an official case record of child maltreatment. Thus, a multimethod approach combining both self-report and official case records is much more sensitive to detecting child maltreatment in a comparison condition while yielding a contamination prevalence estimate that is more accurate than any single method alone (Swahn et al., Reference Swahn, Whitaker, Pippen, Leeb, Teplin, Abram and McClelland2006). Moreover, a multimethod approach of detecting and controlling contamination has the potential to identify subgroups within child maltreatment and comparison groups that have differential risks for specific health outcomes based on whether self-report and official case records are concordant or discordant in indicating exposure to child maltreatment (Danese & Widom, Reference Danese and Widom2020; Shaffer, Huston, & Egeland, Reference Shaffer, Huston and Egeland2008).
The current study advances prior efforts examining contamination in child maltreatment research in two important ways so that the detection and control of contamination can be achieved more readily. One, counterfactual models of causal inference (Morgan, Winship, & Vanderweele, Reference Morgan, Winship and Vanderweele2009; Shadish, Cook, & Campbell, Reference Shadish, Cook and Campbell2002) assume that experimental conditions, such as child maltreatment and comparison conditions, are mutually exclusive in order to attribute change in observed outcomes to one of those conditions. Contamination violates this assumption and requires control in order to generate more accurate estimates of between-group differences. While the clinical trials literature has developed approaches that address contamination (Cuzick et al., Reference Cuzick, Edwards and Segnan1997), these methods rely on randomization and are not directly applicable to the nonrandomized design. Other methods that attempt to mimic randomization in nonrandomized research, such as propensity score matching and demographic matching, are applied only at the initial study visit and, to our knowledge, do not offer solutions to: (a) an event that occurs in only one of the experimental conditions established at the initial study visit (e.g., contamination), and (b) unobserved assignment to experimental conditions when it occurs after the initial study visit (e.g., contamination). Thus, the need to develop and test methods capable of detecting and controlling contamination in nonrandomized research, particularly for but not limited to child maltreatment research, is critical for generating more accurate estimates of between-group differences and reducing cross-study variation in the significance and magnitude of those estimates.
Prior child maltreatment research has controlled contamination by omitting subjects from statistical analysis, a method that cannot: (a) demonstrate how omitted subjects contribute to bias in risk estimates when contamination is not controlled, and (b) characterize any unique risk for health outcomes for those comparison subjects omitted from the analysis relative to those who were retained. The current study tests a novel yet straightforward approach to controlling contamination that can accommodate its occurrence at and after the initial study visit while addressing the noted limitations that arise from omitting subjects from the statistical analysis. This approach retains the full sample and controls contamination by separating members of the comparison condition into subgroups who did and did not experience child maltreatment. This allows for direct contrasts among child maltreatment and comparison conditions to identify the effect magnitude for child maltreatment when contamination is controlled, illustrate how contamination contributes bias in risk estimates, and characterize any unique risk for specified health outcomes for those comparison participants who ultimately experience child maltreatment.
Two, this is the first study to examine longitudinal change in trajectories of child behavior problems, specifically internalizing and externalizing behaviors, measured prospectively from early childhood throughout adolescence. Child behavior problems are novel and important outcomes when examining contamination from a developmental psychopathology framework, as they are a critical metric of both normative child development (Achenbach, Reference Achenbach1991; Achenbach, Dumenci, & Rescorla, Reference Achenbach, Dumenci and Rescorla2002) as well as early, global indicators of later psychopathology (Caspi et al., Reference Caspi, Houts, Belsky, Goldman-Mellor, Harrington, Israel and Moffitt2014; Reef, van Meurs, Verhulst, & van der Ende, Reference Reef, van Meurs, Verhulst and van der Ende2010). Moreover, there is a well-established relation between exposure to child maltreatment and child behavior problems (Gilbert et al., Reference Gilbert, Widom, Browne, Fergusson, Webb and Janson2009; Thornberry, Ireland, & Smith, Reference Thornberry, Ireland and Smith2001; Vachon, Krueger, Rogosch, & Cicchetti, Reference Vachon, Krueger, Rogosch and Cicchetti2015), providing an opportunity to prioritize the assessment of contamination bias on effect size estimates for a known risk. This allows for a direct emphasis on the observation of change in effect size magnitude before and after controlling contamination. The primary aim of this study was to examine the risk for internalizing and externalizing behavior trajectories across child development for those with a confirmed history of child maltreatment and demonstrate the degree of change in effect size magnitudes for this risk before and after controlling contamination using a multimethod strategy of both official case records and self-report assessments. Baseline models established the magnitude of effect size estimates before controlling contamination and final models evaluated change in the magnitude of effect size estimates after controlling contamination. Post-hoc models then examined whether concordance or discordance in self-reporting child maltreatment for those with a confirmed history of child maltreatment contributed to a differential risk for each class of child behavior problems after controlling contamination.
Method
Sample
The Longitudinal Studies of Child Abuse and Neglect (LONGSCAN; Runyan et al., Reference Runyan, Hunter, Socolar, Amaya-Jackson, English, Landsverk and Mathew1998) is a multiwave, prospective cohort study of child maltreatment (N = 1,354). Data collection sites were located in diverse geographic regions (Southwest, Northwest, Midwest, South, and East) throughout the US. Subjects were recruited at or before the age of 4 and then assessed every 2 years through age 18. All caregivers provided consent and parental permission for study participation with children providing assent. Each site secured approval from their respective Institutional Review Board as well as the LONGSCAN Data Coordinating Center prior to data collection. Demographic information at the initial age-4 LONGSCAN assessment indicated that the average child age was 4.56 (SD = 0.70), the median family income was between $10,000 and $14,999, and a majority of caregivers reported being unmarried (n = 784; 67.1%). A majority of females were represented in the LONGSCAN cohort (n = 697; 51.5%). Child racial demography included Black (n = 721; 53.2%), White (n = 354; 26.1%), Hispanic (n = 97; 7.2%), Native American (n = 8; 0.6%), Asian (n = 4; 0.3%), Mixed Race (n = 161; 11.9%), and Other (n = 8; 0.6%). Data collected consecutively from the age-4 through age-16 LONGSCAN biennial assessments were included in the present study, where sample attrition was 33.9%. Approximately, 89.6% of all LONGSCAN subjects provided data on three or more measurement occasions during this period of data collection.
Measures
Confirmed child maltreatment
Each LONGSCAN site reviewed official case records obtained from child protective services agencies at least every two years using the Modified Maltreatment Classification System (MMCS; English, Bangdiwala, & Runyan, Reference English, Bangdiwala and Runyan2005). Independent raters using the MMCS reviewed the case record generated by an official investigation into an allegation of child maltreatment and determined whether the information contained in the record met a prespecified definition of child maltreatment. Research has shown the MMCS improves detection and classification of child maltreatment when compared to child protective services agency designations alone (Runyan et al., Reference Runyan, Cox, Dubowitz, Newton, Upadhyaya, Kotch and Knight2005) while allowing for a standardized definition of child maltreatment to be applied across data collection sites given that legal definitions vary by State. MMCS raters met a training criterion threshold of 90% inter-rater reliability for reviewing child maltreatment case records. Raters then entered scores on the MMCS indicating cases of confirmed child maltreatment at each biennial assessment. Any instance of child maltreatment from birth to age 16 that was recorded on the MMCS resulted in assignment to a Confirmed Maltreatment condition (n = 716) and no instance of confirmed child maltreatment during this same age range resulted in assignment to a No Confirmed Maltreatment comparison condition (n = 638).
Self-report of child maltreatment
The LONGSCAN Self-Reports of Physical, Sexual, and Psychological Abuse (SPSPA; Knight et al., Reference Knight, Runyan, Dubowitz, Brandford, Kotch, Litrownik and Hunter2000) were administered at the age-12 and age-16 LONGSCAN assessments to detect exposure to physical abuse, sexual abuse, and psychological abuse. Definitions for discrete, self-reported maltreatment experiences assessed with the SPSPA were based on definitions specified in the original Maltreatment Classification System (Barnett, Manly, & Cicchetti, Reference Barnett, Manly, Cicchetti, Cicchetti and Toth1993) as well as the American Professional Society on the Abuse of Children (Hart, Brassard, & Karlson, Reference Hart, Brassard, Karlson, Briere, Berliner, Bulkley, Jenny and Reid1996). Example item content assesses respondents on whether an adult has ever physically bruised them, put something inside their private parts, or threatened to hurt them badly. The age-12 LONGSCAN assessment measured any prior lifetime exposure to child abuse and the age-16 LONGSCAN assessment measured exposure to child abuse since the age-12 LONGSCAN assessment. The psychometric properties of the SPSPA have been established using the LONGSCAN cohort (Everson et al., Reference Everson, Smith, Hussey, English, Litrownik, Dubowitz and Runyan2008). Indicator variables based on the SPSPA recorded any self-reported exposure to physical abuse, sexual abuse, or psychological abuse at or before age 16 (yes = 1; no = 0).
The About My Parents (AMP; LONGSCAN Investigators, 1998) scale is a 25-item, self-report measure of child neglect. AMP items are phrased positively (“How often did your parents give you enough clothes to stay warm?”) and rated on a 4-point Likert scale (0 = never to 3 = a lot). Based on prior LONGSCAN research, four items measuring educational neglect and two items measuring supervisory neglect were dropped from the present study with the remaining items providing adequate model fit with good measurement invariance and internal consistencies (Dubowitz et al., Reference Dubowitz, Villodas, Litrownik, Pitts, Hussey, Thompson and Runyan2011). Resulting AMP items were reverse scored, where higher ratings indicated more severe exposure to neglect, and administered at the age-12 LONGSCAN assessment to measure child neglect in elementary school and in the past year. AMP items were again administered at the age-14 and age-16 LONGSCAN assessments to measure child neglect in the past year. Endorsement of the most severe rating on any AMP item at any measurement occasion was used to represent self-reported exposure to child neglect (yes = 1; no = 0).
Contamination
A final indicator variable was used to represent the presence of contamination (yes = 1; no = 0), defined as any self-reported instance of child maltreatment in the No Confirmed Maltreatment comparison condition, based on responses from both the SPSPA and AMP. See Table 1 for the prevalence of contamination in this study as well as individual child maltreatment subtypes.
Prevalence of child maltreatment subtypes through age-16 LONGSCAN assessment
Note. LONGSCAN = Longitudinal Studies of Child Abuse and Neglect. Percentages sum to greater than 100% due to children experiencing more than one type or instance of maltreatment.
a = US data on confirmed child maltreatment obtained from U.S. Department of Health and Human Services (2019). Administration for Children and Families, Administration on Children, Youth, and Families, Children's Bureau. Child Maltreatment 2017.
b = US data on self-reported child maltreatment obtained from Finkelhor et al. (Reference Finkelhor, Turner, Shattuck and Hamby2015)
Child behavior problems
The Child Behavior Checklist (CBCL; Achenbach, Reference Achenbach1991) is a well-established instrument assessing child internalizing behaviors, such as somatic complaints, worry, and depression, and externalizing behaviors, such as noncompliance, aggression, and delinquency, in the prior six months via caregiver report. The CBCL was administered every 2 years from the age-4 through age-16 LONGSCAN assessments, where age-standardized T-scores for internalizing and externalizing behaviors were derived and included in subsequent statistical modeling.
Data analytic strategy
Because child behavior problems were measured repeatedly throughout childhood and adolescence, statistical dependencies for both internalizing and externalizing behaviors were assessed using the intraclass correlation coefficient (ICC). Results indicated there were dependencies in the repeated assessments of internalizing (ICC = .46) and externalizing behaviors (ICC = .56). As a result, linear mixed modeling (LMM) via SPSS v26, where repeated assessments of each child behavior problem (Level 1) were nested within each child participant (Level 2), was used to estimate individual and average growth trajectories for internalizing and externalizing behaviors. Internalizing and externalizing behaviors were arrayed according to child chronological age (Range = 3.49–18.00 years), not LONGSCAN study assessment, which was rounded to the nearest whole integer and centered where zero represented the youngest child age at the age-4 LONGSCAN assessment with successive codes representing yearly intervals (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15) that cover the complete age range in which child behavior problems were assessed. This coding and centering of chronological age means that the intercept value in LMMs reported below can be interpreted as average levels of the respective child behavior problem for children in the reference category (e.g. no confirmed maltreatment). This coding of the intercept is meaningful because it allows for an assessment of group differences for the youngest children assessed at the initial study visit. However, the parameter of interest in the current study is the interaction of child maltreatment over time, which can be interpreted as the yearly change in the respective child behavior problem across child development for those exposed to confirmed child maltreatment. Intercepts and slopes were entered as random effects to allow internalizing and externalizing behaviors to vary at study entry and over time for all participants. The covariance matrix for the random effects terms was set to unstructured. Missing data for internalizing and externalizing behavior problems were retained in LMMs under the missing at random assumption and addressed using maximum likelihood estimation. The Satterthwaite approximation was used to estimate degrees of freedom in LMMs.
Fit for linear and nonlinear trends of internalizing and externalizing behavior trajectories was evaluated in unconditional and conditional models with the linear parameter providing the optimal fit for both internalizing and externalizing behaviors. Baseline models of internalizing and externalizing behavior trajectories were conditioned by adding fixed effects terms as Level 2 predictors: a class variable, Child Maltreatment, involving two levels of child maltreatment status (1 = Confirmed Maltreatment, 2 = No Confirmed Maltreatment) where participants with any instance of confirmed child maltreatment from birth to age 16 were assigned to the Confirmed Maltreatment level and where participants with no instances of confirmed child maltreatment during this same age range were assigned to the No Confirmed Maltreatment level; a linear slope parameter, age, representing yearly change in child behavior problems; and their respective interaction, Child Maltreatment×Age, which represents the average yearly change in child behavior problems for any confirmed child maltreatment prior to age 16 relative to no confirmed child maltreatment, that is, when contamination is uncontrolled.
Level 2 covariates measured via caregiver report at the age-4 LONGSCAN assessment, and their respective interactions with age, were also entered as fixed effects to adjust estimates for established risk factors for child maltreatment and child behavior problems: child sex (0 = male; 1 = female), annual family income (1=<$5,000; 2=$5,000–$9,999; 3=$10,000–$14,999; 4=$15,000–$19,999; 5=$20,000–$24,999; 6=$25,000–$29,999; 7=$30,000–$34,999; 8= $35,000–$39,999; 9=$40,000–$44,999; 10=$45,000–$49,999; 11=>$50,000), caregiver marital status (0 = married; 1 = not married), and child race, entered as three dummy-coded variables (Black, Hispanic, Other) with Whites representing the reference category, consistent with prior LONGSCAN research (Dubowitz et al., Reference Dubowitz, Thompson, Arria, English, Metzger and Kotch2016). Final models were executed in the same manner as baseline models with one important exception: the Level 2 class variable, Child Maltreatment, represented three levels of child maltreatment status: 1 = Confirmed Maltreatment, 2 = Contamination, which represents a comparison subgroup with self-reported maltreatment but no confirmed maltreatment prior to age 16; 3 = No Contamination, which represents a comparison subgroup with neither self-reported nor confirmed maltreatment prior to age 16. The reference group for final models was the No Contamination level. The interaction term, Child Maltreatment×Age, in the final models therefore represents the average yearly change in child behavior problems for confirmed child maltreatment prior to age 16 relative to unconfirmed child maltreatment without contamination, that is, when contamination is controlled.
Standardized effect size estimates for LMMs and their corresponding 95% confidence intervals (Feingold, Reference Feingold2015) were obtained for the respective Child Maltreatment×Age parameters generated in baseline and final models using the equation, d = (b * Study duration) / SD pooled, where b = the respective Child Maltreatment×Age term representing the difference in average growth trajectories between child maltreatment conditions; Study duration = the total number of years, measured in one year intervals, in which child behavior problems were assessed minus one (16 − 1 = 15); and SD pooled = the pooled within-group standard deviations for all repeated assessments of internalizing and externalizing behaviors in baseline (10.75, 11.33, respectively) and final models (10.95, 11.30, respectively). Pooled estimates for final models were obtained from within the Confirmed Maltreatment and No Contamination levels. These effect sizes are scale-invariant, standardized measures of the magnitude difference between confirmed child maltreatment prior to age 16 and respective comparison conditions represented as a linear function of time and can be interpreted as small, medium, or large analogous to traditional effect size interpretations.
Results
Baseline models of child behavior problems – contamination uncontrolled
Table 2 reports demographic information, and respective group differences, for the child maltreatment and comparison conditions tested in baseline models. After adjusting for covariates in the internalizing behaviors model, the term, Child Maltreatment, was not significant, b = −.93, p = .099, indicating that children exposed to confirmed maltreatment were not different in internalizing behaviors at the intercept when compared to children in the no confirmed maltreatment condition. There was, however, a significant Child Maltreatment×Age interaction, b = .29, p < .001, 95% CI: 0.16–0.42, indicating that those exposed to confirmed maltreatment demonstrated significantly more internalizing behaviors across child development relative to participants with no confirmed maltreatment. The effect size magnitude for this interaction was d = .40, 95% CI: 0.22–0.59.
Age-4 child demographics across child maltreatment conditions tested in baseline models
Note. a = Statistically significant difference (p < .05) between confirmed maltreatment and unconfirmed maltreatment comparison condition. Analysis of variance, chi-square, and Mann–Whitney U tests were used to test group differences. Post-hoc, Bonferroni-corrected analyses revealed that despite the same median family income between confirmed and unconfirmed maltreatment conditions statistically significant differences in the distribution of family income existed at the < $5,000 rank level. All demographic variables listed above are included as covariates in baseline models.
Baseline models for externalizing behaviors, after adjusting for covariates, indicated that the Child Maltreatment term was significant, b = 1.34, p = .037, demonstrating that externalizing behaviors for the confirmed maltreatment condition were significantly more severe at the intercept when compared to the unconfirmed maltreatment condition. However, the Child Maltreatment × Age interaction was also significant, b = .14, p = .040, 95% CI: 0.01–0.27, indicating that participants exposed to confirmed maltreatment, on average, demonstrated significantly more externalizing behaviors across child development relative to participants in the no confirmed maltreatment comparison condition. The effect size magnitude of this interaction was d = .19, 95% CI: 0.01–0.36.
Final models of child behavior problems – contamination controlled
Table 3 reports demographic information, and respective group differences, for the child maltreatment and comparison conditions tested in final models. The Confirmed Maltreatment versus No Contamination, b = −.78, p = .375, and Contamination versus No Contamination, b = −.02, p = .981, contrasts for the Child Maltreatment term did not reach statistical significance for internalizing behaviors at the intercept after adjusting for covariates. However, the contrast comparing Confirmed Maltreatment and No Contamination levels in the Child Maltreatment×Age interaction term did reach statistical significance, b = .37, p < .001, 95% CI: 0.18–0.56, indicating that those exposed to confirmed child maltreatment, on average, had significantly higher internalizing behavior trajectories when compared to a comparison condition that did not contain contamination. The effect size magnitude for confirmed child maltreatment in the final model, d = .51, 95% CI: 0.25–0.77, increased by 27.5% when contamination was controlled (see Figure 1). The Contamination versus No Contamination contrast did not achieve statistical significance, b = .11, p = .284. Model results also indicated there remains significant variation around the intercept, slope, and their covariance, after estimating all of the parameters in the final model. See Table 4 for full final model results.
Effect size change in trajectories of internalizing behaviors before and after controlling contamination. Effect sizes were determined using the Child Maltreatment×Age interaction term from baseline and final models and represent the differences in average slope trajectories between the respective confirmed child maltreatment and comparison conditions by the end of the study period. The no confirmed maltreatment comparison condition (Top) was separated into two subgroups (bottom): (a) contamination, defined as self-reported but no confirmed maltreatment, and (b) no contamination, defined as no self-reported or confirmed maltreatment.
Age-4 child demographics across child maltreatment conditions tested in final models
Note. Total n = 1,091 due to missing data on self-reports of child maltreatment.
a = Statistically significant difference (p < .05) between confirmed child maltreatment and contamination comparison condition.
b = Statistically significant difference (p < .05) between confirmed child maltreatment and no contamination comparison condition. Group differences were tested across the contamination and no contamination groups but no significant differences were observed. Analysis of variance, chi-square, and Mann–Whitney U tests were used to test group differences. Post-hoc, Bonferroni-corrected analyses revealed that despite the same median family income between confirmed maltreatment and contamination comparison conditions statistically significant differences in the distribution of family income existed at the < $5,000 and $20,000–$24,999 rank levels. All demographic variables listed above are included as covariates in final models.
Final linear mixed model results for internalizing behaviors
a = Values presented in this column under the “Fixed effects” subheading were determined using the t statistic. Values presented in this column under the “Random effects” subheading were determined using the Wald Z statistic.
Final models then assessed change in the magnitude of risk estimates for externalizing behaviors after controlling contamination. After adjusting for covariates, the Child Maltreatment term did not reach statistical significance for the contrast between the Confirmed Maltreatment versus No Contamination levels, b = 1.79, p = .075, or for the contrast comparing the Contamination versus No Contamination levels, b = .20, p = .850. The Child Maltreatment×Age interaction for the contrast between Confirmed Maltreatment and No Contamination levels did reach statistical significance, b = .22, p = .028, 95% CI: 0.02–0.41, illustrating a more severe trajectory of externalizing behaviors throughout child development for those exposed to confirmed child maltreatment. The effect size magnitude for externalizing behaviors in the final model was d = .29, 95% CI: 0.03–0.54, an increase of 52.6% when contamination was controlled (see Figure 2). The contrast for the Contamination and No Contamination levels did not achieve statistical significance, b = .14, p = .189. Model results also indicated there remains significant variation around the intercept, slope, and their covariance, after estimating all of the parameters in the final model. See Table 5 for full final model results.
Effect size change in trajectories of externalizing behaviors before and after controlling contamination. Effect sizes were determined using the Child Maltreatment×Age interaction term from baseline and final models and represent the differences in average slope trajectories between the respective confirmed child maltreatment and comparison conditions by the end of the study period. The no confirmed maltreatment comparison condition (Top) was separated into two subgroups (bottom): (a) contamination, defined as self-reported but no confirmed maltreatment, and (b) no contamination, defined as no self-reported or confirmed maltreatment.
Final linear mixed model results for externalizing behaviors
a = Values presented in this column under the “Fixed effects” subheading were determined using the t statistic. Values presented in this column under the “Random effects” subheading were determined using the Wald Z statistic.
Post-hoc models – concordance of self-report and confirmed child maltreatment
Two post-hoc models then determined whether the risk for internalizing and externalizing behaviors observed for the confirmed child maltreatment condition in final models varied based on whether those participants did or did not also self-report child maltreatment. These post-hoc models were executed in the same manner as the final models except the Child Maltreatment term, and its corresponding interaction with age (Child Maltreatment×Age), was further separated from three to four levels: (a) Confirmed and Self-reported Maltreatment (n = 401), (b) Confirmed but No Self-reported Maltreatment (n = 166), (c) Contamination (n = 341), and (d) No Contamination (n = 183). The reference group for post-hoc models was the No Contamination level. Approximately 70.7% of those with a confirmed history of child maltreatment prior to age 16 also self-reported maltreatment during this same time. After adjusting for covariates, the Child Maltreatment term in the internalizing behavior model did not reveal any statistically significant differences on internalizing behaviors at the intercept when compared to the No Contamination reference group (ps > .23). However, the Confirmed and Self-reported Maltreatment and No Contamination contrast within the Child Maltreatment×Age term was statistically significant, b = .44, p < .001, 95% CI: 0.24–0.64, demonstrating that those with both a confirmed and self-reported history of child maltreatment were at significantly greater risk for internalizing behaviors throughout child development. No other contrasts for the levels within the Child Maltreatment×Age term were statistically significant when assessing the risk for internalizing behaviors (ps > .14).
For externalizing behaviors, the Child Maltreatment term revealed a significant contrast between the Confirmed and Self-reported Maltreatment and No Contamination levels, b = 2.24, p = .033, indicating that those with both a confirmed and self-reported history of child maltreatment had higher levels of externalizing behaviors at the intercept. However, the same contrast for the Confirmed and Self-reported Maltreatment and No Contamination levels within the Child Maltreatment×Age interaction term was also statistically significant, b = .26, p = .011, 95% CI: 0.06–0.46, indicating that those with both a confirmed and self-reported history of child maltreatment were at significant risk for greater externalizing behaviors throughout child development. No other contrast for the levels within the Child Maltreatment×Age term were statistically significant when assessing the risk for externalizing behaviors (ps > .18).
Discussion
The purpose of the current study was to advance research on detecting and controlling contamination that improves upon existing design and statistical methods while examining the magnitude of risk child maltreatment poses for trajectories of child behaviors problems across the entire period of child development. There are four primary implications to this study. First, a multimethod strategy of official case records and self-report assessments detected a contamination prevalence where nearly two-thirds of the original comparison condition (65.1%; see Table 1) reported at least one instance of child maltreatment prior to age 16. This estimate suggests even well-designed prospective cohort studies, the ideal design for conducting child maltreatment research (Widom, Raphael, & DuMont, Reference Widom, Raphael and DuMont2004), are not immune to contamination. When considering the prevalence estimates reported in this study and independent studies (Scott et al., Reference Scott, Smith and Ellis2010; Shenk et al., Reference Shenk, Noll, Peugh, Griffin and Bensman2016), the presence of contamination in child maltreatment research may be the rule, not the exception. To our knowledge, the contamination prevalence estimate reported in this study represents an upper bound estimate based on prior research. The reason for this could very well be the result of recruiting a comparison condition also at risk for child maltreatment as well as the extensive measurement period in which child maltreatment was assessed. These conditions increase the likelihood that comparison subjects will experience child maltreatment at some point during child development. As contamination prevalence varies across child maltreatment studies, so will any impact it has on the significance and magnitude of resulting effect size estimates if left uncontrolled. It is therefore critical to advance ways to detect and control contamination in order to promote effect size estimates that more accurately reflect the true risks of child maltreatment.
Second, the approach to controlling contamination in this study, by retaining the full sample and separating the comparison condition into subgroups who did and did not experience child maltreatment, demonstrated that contamination contributes to bias in effect size estimates by truncating the magnitude of group differences across child maltreatment and comparison conditions (see Figures 1 and 2). This approach also demonstrated that controlling contamination generated effect size estimates for child behavior problems that were 27.5%–52.6% larger in magnitude when compared to models where contamination was uncontrolled. These results are novel and suggest that the risk child maltreatment poses for child behavior problems may be greater than initially anticipated once contamination is controlled. Moreover, the percent increases in effect size magnitude for each type of child behavior problem fall within the range observed for other outcomes examined in prior contamination research (Scott et al., Reference Scott, Smith and Ellis2010; Shenk et al., Reference Shenk, Noll, Peugh, Griffin and Bensman2016), contributing to an expected framework for the degree to which controlling contamination can strengthen risk estimates. The results for child behavior problems also provide a benchmark for measuring reproducibility in child maltreatment research, which places an emphasis on generating effect size estimates that are similar in magnitude and fall within the same confidence interval (Open Science Collaboration, 2015; Tryon, Reference Tryon2016). Future child maltreatment research can now compare effect size estimates for trajectories of child behavior problems, as well as the percent increase in overall effect magnitude across diverse outcomes, once contamination is controlled. Systematic uptake of methods for controlling contamination in future child maltreatment research could make discovery, replication, and reproducibility of effect size estimates more likely, strengthening conclusions about the overall risks for adverse development following child maltreatment.
Third, child maltreatment significantly predicted more severe internalizing and externalizing behavior problems throughout childhood and adolescence in models that did and did not control contamination. This lack of change in the statistical significance of risk estimates across models is not necessarily surprising given LONGSCAN is one of the largest prospective cohort studies of child maltreatment in the United States and prior child maltreatment research establishing a risk for child behavior problems at varying points in development (Gilbert et al., Reference Gilbert, Widom, Browne, Fergusson, Webb and Janson2009; Thornberry et al., Reference Thornberry, Ireland and Smith2001). Moreover, prior research on contamination found statistically significant risks for specific outcomes even before controlling contamination (Scott et al., Reference Scott, Smith and Ellis2010; Shenk et al., Reference Shenk, Noll, Peugh, Griffin and Bensman2016). The concern about variation in statistical significance arises when there is a failure to detect an effect until contamination is controlled, as was also shown in prior contamination research. Replication failures are most often driven by subsequent studies that are statistically underpowered to detect the effect of interest (Maxwell, Lau, & Howard, Reference Maxwell, Lau and Howard2015). Uncontrolled contamination is likely to contribute to such failures, particularly when samples are small and rates of contamination are high, through an attenuation of effect size magnitudes. Controlling contamination may therefore increase the likelihood of detecting a risk attributable to child maltreatment when it exists in both novel and replication research.
Finally, this study contributes to the literature on the degree of concordance across official case records and self-report methods in a child maltreatment condition as well as those subgroups who may and may not be at unique risk for child behavior problems once contamination is controlled. Nearly 71% of those with a confirmed history of child maltreatment between birth and age 16 also self-reported child maltreatment during this same time, a concordance rate that is comparable to other child maltreatment studies (Baldwin et al., Reference Baldwin, Reuben, Newbury and Danese2019). This specific subgroup had the greatest risk for more severe internalizing and externalizing behaviors, a finding that is consistent with prior research (Shaffer et al., Reference Shaffer, Huston and Egeland2008) and offers evidence for how concordance across self-report and official case records may be driving the well-established risk for child behavior problems in the child maltreatment population (Gilbert et al., Reference Gilbert, Widom, Browne, Fergusson, Webb and Janson2009).
However, the current study failed to find significant risks for child behavior problems for those who only had a confirmed history of child maltreatment (no self-reported maltreatment) and those who only had a history of self-reported maltreatment (no confirmed maltreatment). Prior research (Danese & Widom, Reference Danese and Widom2020) has also failed to find a risk for internalizing and externalizing outcomes for those who only had a confirmed history of child maltreatment; however, it did indicate a significant risk for those who only self-reported child maltreatment. The discrepancy across results presented in the current study and that of Danese and Widom (Reference Danese and Widom2020) on whether self-reported maltreatment alone constitutes a unique risk group could be the result of several factors. For example, variation in statistical power, and the ability to reliably detect risks for health outcomes, may occur across studies when original child maltreatment and comparison conditions are divided into smaller subgroups. Yet, the current study, and that of Danese and Widom (Reference Danese and Widom2020), had comparable overall and subgroup sample sizes, and effect size estimates that were similar in magnitude, to identify whether self-reported maltreatment only was a unique risk group. This suggests that the difference between the results of these two studies is more likely due to other factors, such as: (a) the use of a continuously scaled measure of caregiver-reported child behavior problems versus categorical indicators of adult psychiatric disorders based on diagnostic interview, and (b) repeated measurements of outcomes across childhood and adolescence versus a single point of assessment in adulthood. It also suggests the possibility that self-reported maltreatment alone, like confirmed child maltreatment alone, may not constitute a unique risk group. Future research should continue examining variation across concordant and discordant indices of child maltreatment status to shed light on those at risk for child behavior problems after controlling contamination and what challenges there are in detecting such risk. Nevertheless, a common finding in this literature, including results from the current study, is that those with a confirmed child maltreatment history who also self-report maltreatment are at greatest risk for more severe child behavior problems and adult psychiatric disorders (Danese & Widom, Reference Danese and Widom2020; Shaffer et al., Reference Shaffer, Huston and Egeland2008).
While advancing research on how to detect and control contamination, there are also several important limitations to this study. First, this study examined the impact of contamination across confirmed and unconfirmed child maltreatment conditions. This method of establishing child maltreatment and comparison conditions was chosen because it is consistent with federal and state government procedures for detecting child maltreatment in the United States (U.S. Department of Health and Human Services, 2019), yet it may produce results that are at odds with other studies using different methods for establishing child maltreatment conditions. Second, the strategy for identifying contamination in this study used a multimethod approach that ensured no participants in the comparison condition had a confirmed case of child maltreatment and where self-report was used to identify contamination. Self-report methods may inflate rates of contamination by participants remembering aversive parenting experiences as maltreatment that would not otherwise meet accepted definitions of child maltreatment (e.g., false positives). While it is not possible to say whether or not this occurred in the present sample, LONGSCAN investigators employed self-report measures of child maltreatment that have strong content and face validity with empirically derived as well as Federal, State, and professional definitions of child maltreatment (Barnett et al., Reference Barnett, Manly, Cicchetti, Cicchetti and Toth1993; Hart et al., Reference Hart, Brassard, Karlson, Briere, Berliner, Bulkley, Jenny and Reid1996; Knight et al., Reference Knight, Runyan, Dubowitz, Brandford, Kotch, Litrownik and Hunter2000). This means that self-reported endorsements of child maltreatment on these measures are calibrated to widely accepted definitions of what constitutes child maltreatment, as opposed to arbitrary, broad, or investigator-specific definitions that can inflate rates of child maltreatment. Third, the administration of self-report measures of child maltreatment, specifically child neglect, did not provide complete coverage of exposure to child maltreatment during each age of childhood and adolescence under investigation in this study. This raises the possibility that there were instances of child maltreatment during ages or times not assessed with the current measures. Fourth, self-reported exposure to child neglect was determined by an endorsement of the most extreme rating on any individual item, at any measurement occasion, on a dimensional measure of neglect. The rationale for this approach was to generate a binary indicator of exposure to neglect that mirrored the assessment of sexual, physical, and psychological abuse and could be included in the assessment of contamination. While this coding strategy may under or overestimate the prevalence of neglect in comparison conditions, the overall rate obtained in the current study (38.6%) is similar to the rate reported in independent prospective cohort studies assessing the prevalence of child maltreatment in the general pediatric population (41.5%; Hussey et al., Reference Hussey, Chang and Kotch2006). Finally, the current study examined whether any history of confirmed maltreatment between birth and age 16 predicted internalizing and externalizing behavior trajectories across child development to illustrate the change in effect size magnitudes before and after controlling contamination. It did not examine the developmental timing of child maltreatment nor the onset or severity of child maltreatment, factors that can influence the risk for subsequent child behavior problems (Thornberry et al., Reference Thornberry, Ireland and Smith2001).
There are two recommendations to scientists and providers that have the potential to inform the design and analysis of future child maltreatment research as well as the prevention of adverse health outcomes in the child maltreatment population. One, most child maltreatment researchers use a single method to establish child maltreatment and comparison conditions when beginning data collection for a new observational study (e.g., Widom & Kuhns, Reference Widom and Kuhns1996). When this strategy is applied in future studies, researchers are encouraged to use a multimethod screening of contamination that consists of official case records and self-report instruments from the very start of study design, recruitment, and data collection. Doing so will serve as an effective design feature for controlling contamination at the initial study visit that can reduce its overall prevalence in a specific research study. However, it is almost inevitable that members of a comparison condition will be exposed to child maltreatment as a prospective cohort study continues to collect data over extended periods of time. Having such a multimethod screening in place from the start will allow researchers to use the straightforward method for controlling contamination evaluated in this study to exert statistical control during the follow-up phase of data collection. This will allow researchers to report on the prevalence of contamination while generating more accurate and larger estimates of the risk for adverse development following child maltreatment that not only stands to promote replication and reproducibility but will also guide sample size and power calculations for future research. Specifically, larger effect sizes often mean smaller samples are needed to detect between-group differences, which can ease the cost and complexity of future research designs.
Two, even more effort should be devoted to preventing child maltreatment and its public health consequences given that increased effect size estimates following control of contamination suggests the public health concern related to child maltreatment may be greater than currently thought. Specifically, it is imperative that child welfare agencies make well-established interventions, such as parent-training programs (Timmer, Urquiza, Zebell, & McGrath, Reference Timmer, Urquiza, Zebell and McGrath2005), available to children and families exposed to child maltreatment when these families come to the attention of these agencies. This will have the greatest impact on preventing or reducing child behavior problems, an established precursor to adulthood psychiatric functioning (Caspi et al., Reference Caspi, Houts, Belsky, Goldman-Mellor, Harrington, Israel and Moffitt2014; Reef et al., Reference Reef, van Meurs, Verhulst and van der Ende2010), prior to the transition to adulthood. Such efforts will have important public health ramifications by driving down personal and societal burdens associated with healthcare costs and utilization in the child maltreatment population.
Acknowledgments
The data used in this publication were made available by the National Data Archive on Child Abuse and Neglect, Cornell University, Ithaca, NY, and have been used with permission. Data from Longitudinal Studies of Child Abuse and Neglect (LONGSCAN) Assessments 0-12 were originally collected by Desmond K. Runyan, Howard Dubowitz, Diana J. English, Jonathan Kotch, Alan Litrownik, Richard Thompson and Terri Lewis & The LONGSCAN Investigator Group. The collector of the original data, the funder, NDACAN, Cornell University and their agents or employees bear no responsibility for the analyses or interpretations presented here. Funding for LONGSCAN was provided by the Office on Child Abuse and Neglect (OCAN), Children's Bureau, Administration for Children and Families, Dept. of Health and Human Services [The National Center on Child Abuse and Neglect (NCCAN), under the Office of Human Services funded this consortium of studies during the early years of data collection from 04/01/1991 until NCCAN became part of OCAN in 1998.]
Conflicts of Interest
None
