Group comparisons are central to research, theory, and interventions in developmental psychopathology because considering normal and abnormal development together is the essence of the field (Cicchetti, Reference Cicchetti1984; Sroufe, Reference Sroufe1990). Landmark developmental psychopathology research typically employs longitudinal designs that compare, for example, the trajectories of maltreated and nonmaltreated children and youth (Kim-Spoon, Cicchetti, & Rogosch, Reference Kim-Spoon, Cicchetti and Rogosch2013). These contrasts help elucidate the legacy of early experiences, adaptation to environmental demands, and the role of individual and social resources in development (Sroufe, Reference Sroufe1997). Similarly, research comparing Whites and racial/ethnic minorities (henceforth, minorities)Footnote 1 has been crucial in documenting that minorities experience a disproportionate share of many adverse health conditions compared to Whites in the United States (National Center for Health Statistics, 2016).
On the other hand, group comparisons can also have negative implications. For example, comparing minorities to Whites often reinforces deficit models that portray minorities as groups that are inherently maladapted, lacking resources, and struggling (García Coll, Akerman, & Cicchetti, Reference García Coll, Akerman and Cicchetti2000; Medin, Bennis, & Chandler, Reference Medin, Bennis and Chandler2010). In addition, by overemphasizing White–minority differences, important distinctions between- and within-minority groups can be overlooked. In this study, we identify two implicit models that guide most research comparing Whites and minorities. The cultural differences hypothesis is the assertion that there are large differences between Whites and minorities in the United States, and these differences are greater than differences between- (e.g., African Americans and Latinos) and within-(e.g., Latinos: Mexican Americans and Cuban Americans) minority groups. Conversely, the cultural similarities hypothesis argues that there are small differences between Whites and minorities, and these differences are equal or smaller in magnitude than differences between- and within-minorities. Determining which of these hypotheses is supported by evidence has important implications for theory, research, and interventions in developmental psychopathology.
Given the rapid growth of research and meta-analytic studies on White–minority mental health disparities, focusing exclusively on single studies cannot easily settle these questions. For this reason, we employ a second-order meta-analysis to aggregate data from multiple meta-analyses, which in turn can account for numerous studies and large samples. The goals of this study are to (a) test these hypotheses by estimating the absolute average difference between Whites and minorities, as well as between- and within-minority groups, on levels of psychopathology, and (b) determine if general and meta-analytic method moderators account for these differences.
Culture, Development, and Psychopathology
Despite repeated calls for its incorporation into developmental psychopathology (García Coll et al., Reference García Coll, Akerman and Cicchetti2000; Serafica & Vargas, Reference Serafica, Vargas, Cicchetti and Cohen2006), the promise of a field in which culture is considered a fundamental level of analysis has yet to be fulfilled. Instead, culture is often considered more important for minorities than for Whites, both in general psychology (Causadias, Vitriol, & Atkin, Reference Causadias, Vitriol and Atkin2018a) and in developmental sciences (Causadias, Vitriol, & Atkin, Reference Causadias, Vitriol and Atkin2018b). This perspective restricts the importance of culture to nontypical groups and reinforces the notion that it is not a mainstream issue. In reality, culture is much more than that. Culture shapes the development of all human beings through exposure to multiple environmental influences, as well as family socialization and community participation (Causadias, Reference Causadias2013). Culture can be defined as a dynamic web of behaviors, symbols, beliefs, and ideals that are created and maintained by a community, transmitted from one generation to the next, subject to change, and operating simultaneously at the individual and social levels (Kitayama & Uskul, Reference Kitayama and Uskul2011). More than a collection of traits, culture is a shared system of behaviors and cognitions that serves a function within groups that have a shared history (Causadias, Telzer, & Gonzales, Reference Causadias, Telzer, Gonzales, Causadias, Telzer and Gonzales2018; Causadias, Telzer, & Lee, Reference Causadias, Telzer and Lee2017). Culture is closely related to the concepts of ethnicity and race (for a discussion, see Causadias et al., Reference Causadias, Vitriol and Atkin2018a), so much that they are often treated interchangeably (Quintana et al., Reference Quintana, Aboud, Chao, Contreras-Grau, Cross, Hudley and Vietze2006). Because of the close connection between these three concepts, we refer to cultural differences and similarities to represent comparisons between cultural, ethnic, and racial groups. Cultural differences and similarities do not refer specifically to differences on cultural processes or outcomes, but contrasts between groups on any given domain (e.g., psychopathology).
The Cultural Differences Hypothesis in Developmental Psychopathology
Applied to the field of developmental psychopathology, the cultural differences hypothesis states that Whites and minorities in the United States are very different, with all minority groups exhibiting higher levels of mental illness than Whites. White–minority differences are expected to be larger in magnitude than differences- between and within-minority groups. Culture accounts for differences in the meaning and behavioral expression through which distress shapes development (Serafica & Vargas, Reference Serafica, Vargas, Cicchetti and Cohen2006). Moreover, cultural differences are manifested in how symptoms are presented, disorder prevalence, coping mechanisms, caregiving practices (Serafica & Vargas, Reference Serafica, Vargas, Cicchetti and Cohen2006), and cultural promotive, protective, and risk factors that shape health and adaptation (Causadias, Reference Causadias2013). White–minority differences can also be expected to be large because they participate in unique cultural practices and environments that structure the incidence, magnitude, and development of behaviors, symptoms, and syndromes (Chen & Liu, Reference Chen, Liu and Cicchetti2016). The cultural differences hypothesis in developmental psychopathology can be backed with cultural, ecological, and biological arguments.
First, cultural arguments in support of the cultural differences hypothesis come from evidence that Whites and minorities differ in terms of cultural practices, rituals, values, and resources. Some research suggests Southern Whites are more likely to embrace the culture of honor (Cohen & Nisbett, Reference Cohen and Nisbett1994). In addition, minorities have cultural values, practices, and rituals distinct from Whites. For example, Mexican-origin mother–daughter dyads report more shared decision making and open communication after daughters celebrate La Quinceañera, a cultural rite of passage at age 15 (Romo, Mireles-Rios, & Lopez-Tello, Reference Romo, Mireles-Rios and Lopez-Tello2014). Endorsing beliefs that family is an important source of support (i.e., familism) moderates the link between adversity and behavior problems among Mexican American adolescent mothers (Umaña-Taylor, Updegraff, & Gonzales-Backen, Reference Umaña-Taylor, Updegraff and Gonzales-Backen2011). Moreover, evidence suggests African Americans deal with adversity and racial discrimination by using cultural practices that may reduce the likelihood of developing psychopathology, including cultural pride (Gaylord-Harden, Burrow, & Cunningham, Reference Gaylord-Harden, Burrow and Cunningham2012), community support (Cooper, Brown, Metzger, Clinton, & Guthrie, Reference Cooper, Brown, Metzger, Clinton and Guthrie2013), and spiritual and collective-centered coping (Gaylord-Harden & Cunningham, Reference Gaylord-Harden and Cunningham2009).
Second, ecological arguments in support of the cultural differences hypothesis emphasize that, compared to Whites, minorities in the United States are disproportionally exposed to adversity in environments, surroundings, and institutions that increases the likelihood of developing psychopathology. Minority youth typically develop in niches marked by social stratification, exclusion, and marginalization (García Coll et al., Reference García Coll, Crnic, Lamberty, Wasik, Jenkins, Garcia and McAdoo1996; Spencer, Dupree, & Hartmann, Reference Spencer, Dupree and Hartmann1997). Compared to Whites, minorities are often more likely to live in disadvantaged neighborhoods (Small & Newman, Reference Small and Newman2001), be exposed to pollution and health hazards (Gee & Payne-Sturges, Reference Gee and Payne-Sturges2004), witness community violence (Stein, Jaycox, Kataoka, Rhodes, & Vestal, Reference Stein, Jaycox, Kataoka, Rhodes and Vestal2003), have less access to medical and dental health services (Flores & Tomany-Korman, Reference Flores and Tomany-Korman2008), and experience racial discrimination (Schmitt, Branscombe, Postmes, & Garcia, Reference Schmitt, Branscombe, Postmes and Garcia2014).
Third, biological arguments in support of the cultural differences hypothesis come from research on neuroscience, stress neurobiology, and genetics. Research on neuroscience has documented how group differences in multiple psychological processes correspond to different patterns of brain activity (Han & Ma, Reference Han and Ma2014), which are related to psychopathy traits (see Viding & McCrory, Reference Viding and McCrory2012). Research on stress neurobiology has shown that exposure to unique stressors increases drug use vulnerability among minorities compared to Whites (Obasi, Wilborn, Cavanagh, Yan, & Ewane, Reference Obasi, Wilborn, Cavanagh, Yan, Ewane, Causadias, Telzer and Gonzales2018), and has documented differences in diurnal cortisol rhythms between White and Black participants that are related to poorer mental health (Skinner, Shirtcliff, Haggerty, Coe, & Catalano, Reference Skinner, Shirtcliff, Haggerty, Coe and Catalano2011). Furthermore, research on genetics shows that individuals from populations of different geographic ancestry (e.g., European, African, and Asian) show diversity in the presence of rare and common genetic variants, which can affect the prevalence of complex diseases (1000 Genomes Project Consortium, 2012). Some have even suggested that evidence from population-genetic research can identify distinct groups based on genetic differences, and that these differences, amplified by culture, explain racial disparities in economic success between groups (Wade, Reference Wade2014). More recently, the idea of a panhuman nature has been challenged based on the notion that migration and natural selection have resulted in the evolution of different psychological traits between human populations, but serious scientific discussions of these differences are tamed by political and ideological sensitivities (Winegard, Winegard, & Boutwell, Reference Winegard, Winegard and Boutwell2017). In sum, arguments from cultural, ecological, and biological research support substantial White–minority differences in the development of psychopathology.
The cultural differences hypothesis can be quantified using a well-validated effect size metric: Cohen's d (Cohen, Reference Cohen1992), the most common measure of the magnitude of group differences in psychological research (Hyde, Reference Hyde2014). Cohen (Reference Cohen1992) used three values to represent small (0.20), medium (0.50), and large (0.80) group differences. Sawilowsky (Reference Sawilowsky2009) extended these criteria to include groups differences of very small (0.01), very large (1.2), and huge (2.0) magnitude. Following Zell, Krizan, and Teeter (Reference Zell, Krizan and Teeter2015), we integrate these conventions to interpret the magnitude of Cohen's d as very large (d > 1.00), large (d = 0.66–1.00), medium (d = 0.36–0.65), small (d = 0.11–0.35), and very small (0–0.10).
Using this metric, the cultural differences hypothesis translates into medium (0.36–0.65), large (0.66–1.00), or very large (>1.00) differences between Whites and minorities on mean levels of psychopathology. These differences are expected to be larger than the difference between- and within-minority groups on mean levels of psychopathology. For example, support for the cultural differences hypothesis would be consistent with a Cohen's d of 0.60 between Whites and minorities, and with a Cohen's d of 0.20 between- and within-minorities (see Figure 1a). A Cohen's d of 0.60 means that only 76% of Whites and minorities will overlap in levels of psychopathology, while a Cohen's d of 0.20 conveys that 92% of any two minority groups (e.g., African American and Latinos) and any two minority subgroups (e.g., Latinos: Mexican Americans and Puerto Ricans) will overlap in levels of psychopathology (see Magnusson, Reference Magnusson2018).
Graphical example of the (a) cultural differences and (b) cultural similarities hypotheses.
The Cultural Similarities Hypothesis in Developmental Psychopathology
Applied to the field of developmental psychopathology, the cultural similarities hypothesis argues that there are small differences between Whites and minorities on levels of mental illness, and these disparities are equal or smaller in magnitude than differences between- and within-minority groups. In numerical terms and using Cohen's d, the cultural similarities hypothesis can be translated into very small (0–0.10) or small (0.11–0.35) differences between Whites and minorities on mean levels of psychopathology, and these differences are equal or smaller in magnitude than the difference between and within-minority groups (see Figure 1b).
Overemphasizing cultural differences over similarities ignores that there is more within-culture variation than between-group variation in most psychological traits (Adams & Markus, Reference Adams, Markus, Schaller and Crandall2004). For several reasons, portraying cultural groups as bounded, internally homogeneous, and ahistorical entities has been challenged in developmental psychology (Gjerde, Reference Gjerde2004, Reference Gjerde2014; Gjerde & Onishi, Reference Gjerde and Onishi2000). This argument implicitly endorses a view of members of ethnic groups as passive vessels of natural, unchanging, and overwhelming cultural traits, an ideology defined as primordialism (Eriksen, Reference Eriksen2002) or essentialism (Gjerde, Reference Gjerde2004). This approach neglects historical change, intra-ethnic variation, within-group disagreement and conflict, and the role of individual agency in development (Gjerde, Reference Gjerde2014; Gjerde & Onishi, Reference Gjerde and Onishi2000). However, there is no conceptual reason or empirical evidence to suggest that developmental processes operate differently between White and minority individuals (García Coll et al., Reference García Coll, Crnic, Lamberty, Wasik, Jenkins, Garcia and McAdoo1996). The cultural similarities hypothesis can be supported using cultural, ecological, and biological arguments.
First, cultural arguments in support of similarities in developmental psychopathology include evidence of considerable within- and between-minority variation in development. For example, ethnic and racial identity development in adolescence varies in response to differences in a family's immigrant status and parental socialization practices (Umaña-Taylor et al., Reference Umaña-Taylor, Quintana, Lee, Cross, Rivas-Drake, Schwartz and Seaton2014). Heterogeneity in the development of ethnic and racial identity is also amplified by the fact that a growing number of minority youth are of mixed-race or multiracial background, which makes them hard to classify into groups (Phinney, Reference Phinney1990). Between- and within-minority diversity in developmental psychopathology is also exemplified in adaptive cultures that facilitate resilient functioning (García Coll et al., Reference García Coll, Crnic, Lamberty, Wasik, Jenkins, Garcia and McAdoo1996; Gaylord-Harden et al., Reference Gaylord-Harden, Burrow and Cunningham2012), and promote positive outcomes (Causadias, Reference Causadias2013; Fuller & García Coll, Reference Fuller and García Coll2010).
Second, ecological arguments in support of the cultural similarities hypothesis include evidence that both Whites and minorities are exposed to adverse environments that can increase the likelihood of developing psychopathology. For instance, while minorities are disproportionally affected by urban poverty (Small & Newman, Reference Small and Newman2001), poverty and disenfranchisement experienced by rural Whites also has substantial effects on health (Case & Deaton, Reference Case and Deaton2017). Exposure to adverse rural environments partially explains why rural White youth have experienced a heroin epidemic (Cicero, Ellis, Surratt, & Kurtz, Reference Cicero, Ellis, Surratt and Kurtz2014), and have exceeded urban youth in drug abuse (Roberts et al., Reference Roberts, Doogan, Kurti, Redner, Gaalema, Stanton and Higgins2016). In addition, Whites have the highest rate of early death due to drug overdose (Givens, Gennuso, Jovaag, & Van Dijk, Reference Givens, Gennuso, Jovaag and Van Dijk2017). Moreover, Whites can experience maladaptive development in environments traditionally associated with low risk and high well-being. For instance, neighborhoods with mostly highly educated and affluent Whites witness high rates of underage drinking and marijuana use (Reboussin, Preisser, Song, & Wolfson, Reference Reboussin, Preisser, Song and Wolfson2010; Song et al., Reference Song, Reboussin, Foley, Kaltenbach, Wagoner and Wolfson2009).
Third, biological arguments in support of the cultural similarities hypothesis can be drawn from neuroscience, stress neurobiology, and genetics. Research in neuroscience has shown cultural similarities in beliefs, practices, and neural mechanisms of emotion regulation (Qu & Telzer, Reference Qu and Telzer2017). Investigations on stress neurobiology on minorities have found that the effects of adversity are not linear, as the association of racial discrimination and cortisol output is very small (Korous, Causadias, & Casper, Reference Korous, Causadias and Casper2017). In genetics, research shows that polygenic risk scores are linked to greater impulsivity in middle childhood over and above genetic ancestry for both White and Latino participants (Elam et al., Reference Elam, Wang, Bountress, Chassin, Pandika and Lemery-Chalfant2016). Evidence suggests that genetic variation within populations is considerably larger than genetic variation between human populations (Lewontin, Reference Lewontin1972). For instance, genetic stratification in Mexico is so remarkable that some groups are as different as Europeans are from East Asians (Moreno-Estrada et al., Reference Moreno-Estrada, Gignoux, Fernández-López, Zakharia, Sikora, Contreras and Ortiz-Tello2014). Likewise, arguments in favor or distinct or pure genetic groups in the United States are undermined by evidence that over the past 500 years North America has been the site of ongoing mixing between Native Americans, Europeans, and Africans. This resulted in pervasive genetic admixture in all groups (Bryc, Durand, Macpherson, Reich, & Mountain, Reference Bryc, Durand, Macpherson, Reich and Mountain2015).
Testing the Cultural Differences and Similarities Hypotheses
Testing the cultural differences and similarities hypotheses by focusing on individual primary studies can be troublesome. In the last decades, there has been a rapid growth in primary research on cultural differences and similarities in the development of psychopathology in the United States. This swift progression has taken place across various scientific disciplines (e.g., medicine, public health, psychiatry, and clinical psychology), has focused on different mental health outcomes (e.g., depression, alcoholism and drug abuse, and schizophrenia), has employed different research designs (e.g., cross-sectional, case-control/cohort, and longitudinal), and has targeted different samples (e.g., White vs. Black). Unfortunately, this expansion has had the unintended consequence of fostering a fragmentation of knowledge that cannot provide a clear picture of the overall magnitude of differences and similarities between Whites and minorities, and between- and within-minorities, in the development of psychopathology.
Furthermore, primary studies have not clarified the role of sociodemographic processes (e.g., education and socioeconomic status), as well as methodological characteristics of these studies (e.g., longitudinal research design), in accounting for these differences. This vastly different collection of studies, scattered across fields and scientific outlets, poses an unsurmountable challenge for scientists, policymakers, and practitioners, who have to navigate this evidence to inform their work. As a result, they may overemphasize the results of a few highly publicized studies with large and statistically significant findings. Relying on single studies may be misleading because peer-reviewed research is affected by publication bias, or the differential choice to publish studies based on the nature, magnitude, or direction of findings (Higgins & Green, Reference Higgins and Green2011). Evidence shows that published research is more likely to be positive or significant, and report larger effects, than unpublished work (Ioannidis, Munafo, Fusar-Poli, Nosek, & David, Reference Ioannidis, Munafo, Fusar-Poli, Nosek and David2014). Conversely, most studies with null or small effects are never submitted for publication. One of the negative repercussions of this bias is that it leads to overestimation of the magnitude of the effects and failure to replicate findings from published studies (Francis, Reference Francis2013).
One strategy to address these issues is by conducting systematic quantitative reviews or meta-analyses. Meta-analysis is a statistical method that combines the results of many investigations on a given subject (Card, Reference Card2012; Hyde, Reference Hyde2014). Meta-analyses bestow considerable advantages when evaluating differences and similarities between groups because they assess whether multiple studies find the same result, appraise the average magnitude of the differences and not simply if they are statistically significant, explore moderators that may contribute to these differences, and estimate developmental trends in the magnitude of these differences (Hyde, Reference Hyde2005, Reference Hyde2014). Meta-analyses can also correct for publication bias and account for sample size (Card, Reference Card2012). For instance, McCoy and Edens (Reference McCoy and Edens2006) conducted a meta-analysis to test the assertion that individuals of African descent are more likely to be psychopathic (e.g., prone to break rules and be unconsidered to others) than those of European descent. They employed data from 16 studies with adolescents (combined N = 2,199) and reported a small mean difference between groups (d = 0.20). Another meta-analysis tested if White women have greater body dissatisfaction than minority women (Grabe & Hyde, Reference Grabe and Hyde2006). They compiled data from 98 studies (combined N = 42,667) and reported small White–Black mean differences (d = 0.29).
Despite the fact that these meta-analyses offer several advantages, they are also limited in their scope and focus. Meta-analyses tend to center on particular age groups (e.g., adolescents or adults); specific mental health outcomes (e.g., depression or psychosis); and often compare specific groups (e.g., White–Black disparities). Meta-analyses also vary in statistical methods employed to aggregate data, such as the quality of search methods, inclusion criteria, tests of homogeneity and publication bias, and artifact correction (Card, Reference Card2012; Higgins et al., Reference Higgins, Lane, Anagnostelis, Anzures-Cabrera, Baker, Cappelleri and Whitehead2013). In addition, meta-analyses report overall effect sizes in different metrics including mean group differences (i.e., Cohen's d or Hedges's g), correlations (i.e., Pearson's r), and likelihood estimates (e.g., odd ratios). Finally, while some meta-analyses employ fixed-effects models that assume a true effect size across all individual studies, others utilize a random-effects model that assumes the true effect varies across a random sample of individual studies (Borenstein, Hedges, Higgins, & Rothstein, Reference Borenstein, Hedges, Higgins and Rothstein2009).
Using Second-Order Meta-Analyses to Elucidate Cultural Differences and Similarities
Second-order meta-analyses can advance our understanding of differences and similarities between- and within-groups because they aggregate results across multiple meta-analyses, hundreds of studies, and millions of participants (see Johnson, Scott-Sheldon, & Carey, Reference Johnson, Scott-Sheldon and Carey2010; Schmidt & Oh, Reference Schmidt and Oh2013). Second-order meta-analyses consolidate research by incorporating meta-analyses of research across different measures, outcomes, designs, samples, and variables. Second-order meta-analyses can take advantage of heterogeneity across meta-analyses by estimating the role of moderators on mental health disparities between Whites and minorities, and between- and within-minorities. Second-order meta-analyses can also estimate the overall average difference between groups. Finally, second-order meta-analyses can account for publication bias by including published and unpublished meta-analyses, as well as meta-analyses that include unpublished studies. This technique has been employed successfully in the past to investigate the magnitude of group differences, although with different approaches. Hyde (Reference Hyde2005, Reference Hyde2014) formulated the gender similarities hypothesis that argues that males and females are more alike on most psychological processes than they are different, in contrast with the gender differences hypothesis that argues that males and females are vastly different on most psychological processes. Zell et al. (Reference Zell, Krizan and Teeter2015) tested these hypotheses by conducting a study of 106 meta-analyses and reported a relatively small absolute difference between males and females across types (d = 0.21). These results not only delivered compelling support for the gender similarities hypothesis but also highlighted settings under which differences are accentuated. It is important to note that Zell et al. (Reference Zell, Krizan and Teeter2015) labeled their approach as a meta-synthesis. We avoid using this term because it is also employed in reference to research synthesis of qualitative studies (Walsh & Downe, Reference Walsh and Downe2005). We prefer the term second-order meta-analysis for syntheses of meta-analyses (Schmidt & Oh, Reference Schmidt and Oh2013).
Second-order meta-analyses employ different statistical methods. For instance, some use fixed-effects models (Johnson et al., Reference Johnson, Scott-Sheldon and Carey2010; Tamim, Bernard, Borokhovski, Abrami, & Schmid, Reference Tamim, Bernard, Borokhovski, Abrami and Schmid2011), while other use mixed-effects models (Steenbergen-Hu, Makel, & Olszewski-Kubilius, Reference Steenbergen-Hu, Makel and Olszewski-Kubilius2016). We chose a random-effects model typically used for meta-analyses to average across studies because it incorporates variability into the models and allows inferences beyond the sample of studies (Borenstein et al., Reference Borenstein, Hedges, Higgins and Rothstein2009).
The Present Study
To our knowledge, this is the first second-order meta-analysis to aggregate across meta-analyses on the absolute average difference between Whites and minorities, and between- and within-minorities, on mean levels of psychopathology in the United States. The current study utilizes a large number of findings to provide the most exhaustive test of cultural differences and similarities in the development of psychopathology to date. This test can inform the advancement of theory, strategies, policies, training, and interventions that can address the development of maladaptive outcomes. The aims of this study are as follows:
Aim 1. To test the cultural differences and cultural similarities hypothesis in the development of psychopathology by estimating the average difference between Whites and minorities, as well as between- and within-minority groups, on levels of psychopathology. Following the approach of Hyde (Reference Hyde2005, Reference Hyde2014) and Zell et al. (Reference Zell, Krizan and Teeter2015), we will consider support for the cultural differences hypothesis if (a) the absolute average differences (henceforth, differences) between Whites and minorities on mean levels of psychopathology are medium (0.36–0.65), large (0.66–1.00), or very large (>1.00); and (b) the differences between Whites and minorities are larger than the difference between- (e.g., African Americans and Latinos) and within-minority groups (e.g., Latinos: between Mexican Americans, Cuban Americans, and Puerto Ricans) on mean levels of psychopathology. We will consider support for the cultural similarities hypothesis if the differences between Whites and minorities on mean levels of psychopathology are (a) very small (0–0.10) or small (0.11–0.35), and (b) the differences between Whites and minorities are equal or smaller in magnitude than the differences between- and within-minority groups. We will also investigate the distribution of effect sizes to determine which pattern they follow. Our study is informed by an estimation model, rather than null-hypothesis significance testing, because we focus on the magnitude and precision of our estimates (Cumming, Reference Cumming2013).
Aim 2. To determine if general and meta-analytic method moderators account for these differences. First, we will investigate general moderators, including developmental period (e.g., childhood or adolescence), mean age, dimension (i.e., internalizing or externalizing) and type (e.g., depression or anxiety) of psychopathology, sex, socioeconomic status, income, educational attainment (i.e., years of education), decade of publication, measure of psychopathology (i.e., type of measure and number of measures), and research design (i.e., cross-sectional, case-control/cohort, or longitudinal). Second, we will investigate meta-analytic method moderators, including meta-analytic quality (e.g., inclusion criteria and reliability), inclusion of published and unpublished sources, reported effect size metric (e.g., Cohen's d), and statistical method of aggregating data (e.g., fixed vs. random effects). We preregistered this study on Open Science Framework on December 18, 2017, before conducting data analyses, to facilitate replication and transparency about our methods (Nosek et al., Reference Nosek, Alter, Banks, Borsboom, Bowman, Breckler and Contestabile2015). The preregistration is available at https://osf.io/yn58w/. All materials are available at https://osf.io/ujcre/
Method
Literature search
We conducted a systematic search in PsycINFO and Web of Science using the following string: (cultural OR culture OR ethnic OR ethnicity OR race OR racial OR nationality) AND (similarity OR similarities OR sameness OR likeness OR equivalence OR inequality OR inequalities OR discrepancies OR disparity OR disparities OR dissimilarity OR dissimilarities OR disproportionately OR differentiation OR difference OR differences OR prevalence OR incidence OR incompatibility) AND (“meta-analysis”). We restricted the last part of the string to the title. We identified unpublished meta-analyses by searching for dissertations in ProQuest Dissertations and Theses Global using the above string with the restriction that search terms were included in the abstract. We searched for additional meta-analyses by examining specific journals focused on development (e.g., Developmental Psychology and Child Development), psychopathology (e.g., American Journal of Psychiatry), developmental psychopathology (e.g., Development and Psychopathology), and research synthesis (e.g., Psychological Bulletin). We also examined reference lists of the included meta-analyses (backward search), studies that cited the included meta-analyses (forward search), and conference programs (i.e., Association for Psychological Science, Society for Research on Adolescence, Society for Research on Child Development, American Psychiatric Association, and Institute on Psychiatric Services). Our search was finalized on October 31, 2017.
Inclusion and exclusion criteria
We uploaded identified meta-analyses in Rayyan QCRI (Ouzzani, Hammady, Fedoroqicz, & Elmagarmid, Reference Ouzzani, Hammady, Fedorowicz and Elmagarmid2016). All authors independently evaluated the meta-analyses using the inclusion criteria below. Discrepancies were resolved by the first author. A flowchart of our selection and inclusion process is presented in Figure 2.
Flow chart depicting screening and inclusion procedures.
Variables
We included meta-analyses that examined differences on mean levels of psychopathology, or provided sample means or mean-level differences, for two or more cultural, ethnic, and racial groups in the United States. We included outcomes across various dimensions and types of psychopathology including, but not limited to, depression, anxiety, attention-deficit disorders, sleep disorders, body dissatisfaction, psychosis, and substance use. We excluded meta-analyses if they focused on cultural differences and similarities in outcomes related to normal development (e.g., normal sleep), work (e.g., job satisfaction), education (e.g., high school dropout), medical health (e.g., pain or resting heart rate variability), survival (e.g., mortality), genetics and genomics (e.g., moderating role of serotonin gene), health access (e.g., healthcare), response to behavioral interventions (e.g., randomized control trials), response to pharmaceutical treatments (e.g., antidepressant medication), criminal and forensic issues (e.g., arrests or jury bias), psychometric properties of measures without providing means across cultural groups (e.g., reliability of depression inventory), general psychology (e.g., perception), and harmful social experiences (e.g., racial discrimination or maltreatment). We also excluded meta-analyses that focused on the interaction of developmental processes with group variables to predict outcomes, and meta-analyses centered on other types of group differences (e.g., gender or national).
Participants and geographic location
We only included meta-analyses that presented data focused partially or completely on cultural differences and similarities from samples collected in the United States. We included meta-analyses regardless of sample characteristics, including, but not limited to, age, sex, culture, ethnicity, and race.
Language
We only included meta-analyses published in English.
Research design
We included meta-analyses regardless of the statistical model (e.g., adjusted/unadjusted or fixed-/random-effects). We excluded meta-analyses in which sufficient statistics were not provided to compute an effect size. In addition, we excluded meta-analyses that only examined differences by the racial/ethnic composition of a sample (e.g., percent White). We excluded qualitative reviews, descriptive quantitative reviews, primary studies, meta-analyses that only provided regression coefficients as effect sizes or other meta-analytic effect sizes that could not be converted into Cohen's d, including risk and hazard ratios.
Time period
We did not restrict meta-analyses based on date of publication.
Coding
Reliability
We created a manual to guide coding and facilitate replication (available upon request). Two coders (K.M.K. and K.M.C.) independently extracted effect sizes and other data (e.g., moderators and descriptives) from all meta-analyses that met the inclusion criteria. We established reliability by cross-checking each meta-analytic effect and corresponding variables between the two coders. Statistical analyses were also replicated independently by the two coders. The first author (J.M.C.) resolved disagreements in coding and data analyses.
Aim 1. Estimating the overall differences between Whites and minorities, and between- and within-minorities, on levels of psychopathology
We disaggregated means by group and coded them to compute Cohen's d. If means were not provided, we coded standardized mean differences (i.e., Cohen's d and Hedges's g), correlations, or odds ratios.
Estimating absolute versus relative differences
We coded effect sizes as positive, and we computed the inverse of odds ratios with a value less than 1 to reflect the magnitude of the absolute difference between groups. Thus, if an effect size was negative, we reverse coded it to be positive. Positive values indicate that, on average, there are differences between Whites and minorities on mean levels of psychopathology regardless of the direction. While relative differences are informative for reducing health disparities, absolute differences are considered more appropriate in examining overall differences between groups (see Hyde, Reference Hyde2005; Zell et al., Reference Zell, Krizan and Teeter2015). It could be misleading to aggregate across positive and negative values because they might cancel each other out (e.g., –0.3 and 0.3) and suggest there are no differences between groups. This issue is important in our analysis because we are comparing multiple minority groups to a single group (i.e., Whites), which may produce negative and positive values depending on the particular group comparison.
However, because it is also informative to learn if minorities have higher mean levels of psychopathology compared to Whites, we also coded relative differences with positive (+) and negative (–) signs for White–minority comparisons, keeping the direction consistent across meta-analyses, so Whites are the reference group. In this analysis, a negative value would indicate that minorities had, on average, higher levels of psychopathology. This was not feasible for between- and within-minority comparisons due to the limited number of effect sizes we were able to extract to make these comparisons.
Aim 2. Determining if general and meta-analytic method moderators account for these differences
General moderators
First, we coded meta-analyses to determine whether the magnitude of the difference varied across developmental periods. We tested developmental period as a moderator by grouping meta-analyses that include only children and adolescents, only young adults, only adults, or mixed ages (i.e., children/adolescents, young adults, and adults). In addition, we coded the mean age of the meta-analytic sample and combined age ranges to reflect childhood (0–10 years), adolescence (11–18 years), young adulthood (19–25 years), adulthood (26–39 years), and older adulthood (40 and older). In cases that the mean age of the meta-analytical sample was not reported, and the meta-analysis included 30 or fewer primary studies, we extracted this data directly from the primary studies (44% of included meta-analyses). When the studies included in the meta-analyses were not listed, we contacted the corresponding authors to obtain the list.
Second, we coded meta-analyses to examine whether the magnitude of the difference varied across the dimension and type of psychopathology. We combined meta-analyses that assessed mean level differences for dimensions such as internalizing (i.e., depression, anxiety, body dissatisfaction, burden, nonnormative sleep, suicide ideation, and thought disturbances) and externalizing behavior problems (i.e., substance use, irritability, and psychopathy). We also coded meta-analyses for type of psychopathology (e.g., depression).
Third, we coded meta-analyses to test if the magnitude of the difference varied by the sex composition of the meta-analytic sample. We grouped meta-analyses that included only male, only female, or both male and female participants. If a meta-analysis provided effect size estimates for both male and female participants, we coded them separately.
Fourth, we coded meta-analyses to investigate the moderating role of socioeconomic status, annual income, and educational attainment. We extracted data on the average socioeconomic status, annual income, and educational attainment for the participants of the meta-analytic sample. For socioeconomic status, we coded whether the overall sample was low, middle, or high based on how it was reported in the study. We coded annual income as a mean or the midpoint of a reported range. We also grouped meta-analyses according to the average educational attainment of the meta-analytic sample (i.e., number of school years completed) by grouping meta-analyses with participants who have completed less than 12 years of schooling, 12 years (e.g., high school), 14 to 16 years (e.g., some college or bachelor's degree), and 17 or more years (e.g., graduate degree). In cases that socioeconomic status, income, or educational attainment was not reported, we extracted this data from the primary studies included in meta-analyses, but only for meta-analyses with 30 or fewer primary studies (69% of included meta-analyses).
Fifth, we investigated if measure of psychopathology, research design, and publication date of the meta-analysis accounted for differences. We classified meta-analyses by the type of measure of psychopathology (i.e., self-report, questionnaire, or interview) and number of measures (i.e., single measure or multiple). We coded research design by noting if the meta-analysis aggregated cross-sectional, case-control/cohort, or longitudinal designs. In the case that multiple designs were included in one meta-analysis, we coded the most frequent research design among its included studies (greater than 50%). Finally, we coded meta-analyses by the decade of publication (e.g., 1990–1999 or 2000–2009). In case of unpublished meta-analyses (e.g., dissertations), we coded the date of release noted in the document.
Meta-analytic method moderators
We assessed the quality of meta-analyses included by using a quality checklist adapted from Higgins et al. (Reference Higgins, Lane, Anagnostelis, Anzures-Cabrera, Baker, Cappelleri and Whitehead2013). We rated each meta-analysis using six items (up to 2 points each) including the search method (e.g., online databases: two items), inclusion/exclusion criteria (e.g., eligible participants: one item), reliability of coding articles (e.g., interrater reliability: one item), tests of heterogeneity (e.g., Q statistic: one item), and tests for publication bias (e.g., sensitivity analysis: one item). We grouped meta-analyses by their quality; scores between 11 and 12 were considered the highest quality while scores between 4 and 6 were considered the lowest quality. Next, we compared each meta-analysis based on the use of published sources or both published and unpublished sources. In addition, because multiple metrics of effect sizes were reported in the literature, we compared them by combining effect sizes associated with Cohen's d, Hedges's g, Pearson's r, and odd ratios. Finally, we sorted meta-analytical effects by the statistical model employed to aggregate effect sizes, including fixed-, random-, or mixed-effects models; and unweighted estimates.
Study treatment and analyses
To pursue the first aim of this study, we transformed meta-analytic effects into a standard effect size metric of mean-level group differences using common meta-analytic techniques (see Borenstein et al., Reference Borenstein, Hedges, Higgins and Rothstein2009). This standardized mean difference is a sample estimate such that it refers to the difference between two sample means divided by the pooled standard deviation within samples. We aggregated the effects using an unweighted (d) and a weighted (d +) average. To compute the weighted average, we used a random-effects meta-analytic approach (Borenstein et al., Reference Borenstein, Hedges, Higgins and Rothstein2009). Each meta-analysis was weighted using the inverse of their corresponding within-study variance (v) and between-study variability (τ 2). Thus, meta-analytic effects with larger samples sizes and less standard error were given more weight. We estimated the standard error by computing the sum of random-effects weights and taking the square root of its inverse. To examine the precision of our estimates, we used the standard error to compute a 95% confidence interval for the weighted averages. In addition, we report an index of heterogeneity (I 2) and variance (τ 2) of the weighted averages as measures of between-study variability (Borenstein et al., Reference Borenstein, Hedges, Higgins and Rothstein2009).
Next, we computed the difference between Whites and minorities on mean levels of psychopathology across all meta-analyses. Only one overall effect size was included from each meta-analysis. If differences between Whites and minorities were reported for more than one minority sample, we averaged these effect size estimates to compute one estimate of differences between Whites and minorities for each meta-analysis. Likewise, in the case that means were provided for more than one minority sample, we computed the difference between each reported minority sample and Whites, and then averaged across to derive one estimate of differences between Whites and minorities. If meta-analyses reported differences for multiple dimensions or types of psychopathology, we averaged across effect sizes within each meta-analysis to obtain one estimate for the overall analysis. We also examined the distribution of all coded effect sizes across meta-analyses. Because more than one effect size can be extracted from a single meta-analysis, the number of coded effect sizes exceeds the number of included meta-analyses.
Some meta-analyses had overlapping samples because they included some of the same studies, which violates the meta-analytical assumption of independence (Becker, Reference Becker, Tinsley and Brown2000). In past analyses, an overlap less than or equal to 25% was considered acceptable (Zell et al., Reference Zell, Krizan and Teeter2015). Thus, we present one average difference with all meta-analytic effects and another average difference that only includes effects from meta-analyses with 25% or less overlap across studies. For the second set of analyses, we selected the meta-analysis with the most studies and participants or the most recent meta-analysis. To investigate differences between minority groups, for each meta-analysis we computed the difference between minority samples on mean levels of psychopathology. If means for more than two minority samples (e.g., African Americans and Latinos) were provided in one meta-analysis, we computed the difference between each reported minority sample. Unfortunately, we could not test the difference within-minority groups because we did not identify meta-analyses that reported means separately for subgroups, such as Asian Americans (e.g., Chinese American, Filipino American, and Korean American) and Latinos (e.g., Mexican Americans, Cuban Americans, and Puerto Ricans).
To address the second aim of our study, we grouped effect sizes by developmental period, mean age, dimension and type of psychopathology, sex, socioeconomic status, income, educational attainment, measure of psychopathology, research design, and decade of publication. Next, we estimated a random-effects average difference for each subgroup within each moderator. We only included effect sizes from meta-analyses with 25% or less overlap of studies. If a meta-analysis did not report effect sizes for our general moderators, we utilized the statistics provided in those meta-analyses to average across the individual effects (Johnson et al., Reference Johnson, Scott-Sheldon and Carey2010). If multiple effect sizes were reported for a moderator (e.g., all males or all females), each effect size was included in the moderator analysis. We employed both unweighted and weighted averages to compare subgroups within each moderator. Subgroups with fewer than two effect sizes per group were not included in the weighted moderator analysis.
We followed random-effects meta-analytic techniques of categorical comparisons as a conservative test of differences between weighted averages (Borenstein et al., Reference Borenstein, Hedges, Higgins and Rothstein2009). We computed the random effects between subgroup heterogeneity (Q* between) by subtracting the total amount of heterogeneity across effect sizes (Q* total) from the total within-subgroup heterogeneity (Q* within) using a pooled estimate of τ 2. We chose to weight using a pooled estimate of τ 2 across subgroups because most had five or fewer meta-analytic effects, which restricts the accuracy of τ 2 within each subgroup (Borenstein et al., Reference Borenstein, Hedges, Higgins and Rothstein2009). Although the effects were weighted using the pooled estimate, we also report the τ 2 for each subgroup. We used the chi-square distribution to determine the significance level of the between-subgroup heterogeneity. We considered the moderator significant with a p < .05. We employed the same method to investigate the role of meta-analytic method moderators. We sorted effect sizes by the quality of the meta-analysis, whether the meta-analysis included published sources only or both published and unpublished sources, the effect size reported in the meta-analysis, and the statistical method used to aggregate effects. If multiple statistical models were provided (e.g., unweighted and fixed effects), we included each effect size in the moderator analysis.
Results
We identified 16 meta-analyses (13% unpublished) and 132 individual meta-analytic effect sizes (k) from approximately 493 primary studies with 3,036,749 participants (N). Characteristics of the included meta-analyses are presented in Table 1. Approximately 82% of the participants were White, 17% were African American, and 1% were Asian American, Latino, and American Indian. In addition, 13 meta-analyses (81%) included White–Black comparisons, whereas 7 meta-analyses (44%) compared Whites with more than one minority group. In this section, we present unweighted and weighted results for the analyses focused on the overall average difference, but we concentrate on weighted findings for the moderator analyses. Table 2 and 3 include unweighted and weighted results for the general and meta-analytic method moderators, respectively. However, in the discussion we focus on the weighted effect sizes because they account for sample size and provide a more precise estimate of the distribution of differences in the population of studies.
Characteristics and effect size for each study included in second-order meta-analysis
Note: a = White-minority comparison; b = Between-minority comparison; N = approximate number of participants; Dimension/Type = Dimension and Type of psychopathology; Metric = metric of reported meta-analytical effect; % of Overlap = percentage of the number of overlapping studies; SR-Q = self-report or questionnaire; r = Pearson's r; d = Cohen's d; g = Hedge's g; OR = odds ratio. Positive Cohen's d indicates average absolute difference between Whites and minorities on mean-levels of psychopathology. Dashes represent data that was not reported or unavailable.
Summary of general moderators of differences between Whites and minorities
Note: k = number of meta-analytical effects; N = approximate number of participants; d = unweighted average difference across meta-analytical effects; SD = standard deviation of the unweighted average difference; d + = random-effects weighted average difference across meta-analytical effects; 95% CI = 95% confidence interval; I 2 = index of between-study heterogeneity; τ 2 = pooled and within-subgroup estimated variance of the population's distribution of effect sizes; Q*between = random-effects Q-test based analysis of variance; p = p-value. Effect size (i.e., d, d +) represents the absolute average difference between Whites and minorities on mean-levels of psychopathology. Subscript Int. = Internalizing; Subscript Ext. = Externalizing.
Summary of meta-analytical method moderators of differences between Whites and minorities
Note: k = number of meta-analytical effects; N = approximate number of participants; d = unweighted average difference across meta-analytical effects; SD = standard deviation of the unweighted average difference; d + = random-effects weighted average difference across meta-analytical effects; 95% CI = 95% confidence interval; I 2 = index of between-study heterogeneity; τ 2 = pooled and within-subgroup estimated variance of the population's distribution of effect sizes; Q*between = random-effects Q-test based analysis of variance; p = p-value. Effect size (i.e., d, d +) represents the absolute average difference between Whites and minorities on mean-levels of psychopathology. Subscript Int. = Internalizing; Subscript Ext. = Externalizing.
Aim 1. Estimating the overall differences between Whites and minorities, and between- and within-minorities, on levels of psychopathology
Overall difference between Whites and minorities
We found that the unweighted difference on levels of psychopathology between White and minorities was small in magnitude, d = 0.22, SD = 0.122. When only meta-analyses with 25% or less of study overlap were included, the unweighted difference remained small in magnitude, d = 0.23 (SD = 0.131, k = 12, N = 2,931,250). In addition, the weighted difference was equal in magnitude in our random-effects analysis compared to the unweighted difference (d + = 0.23, τ 2 = .008, I 2 = 98.75, 95% confidence interval; CI [0.18, 0.28]). The magnitude of the weighted difference remained small when we removed the smallest effect size of d = 0.09 (d + = 0.25, τ 2 = .008, I 2 = 98.85) or the largest effect of d = 0.45 (d + = 0.21, τ 2 = .007, I 2 = 98.62). These findings show that differences between Whites and minorities on mean levels of psychopathology are small in magnitude (0.11 < d < 0.35), supporting the cultural similarities hypothesis. The distribution of effect sizes representing the differences between Whites and minorities is displayed in Figure 3. We found that, across all meta-analyses, 44.7% of effect sizes were in the close-to-zero range (d ≤ 0.10), 40.2% were in the small range (0.11 < d < 0.35), 12.9% were in the moderate range (0.36 < d < 0.65), and 2.3% were in the large range (d = 0.66–1.00). This distribution provides support to the cultural similarities hypothesis.
Distribution of effect sizes. (a) Absolute meta-analytic effect sizes for White–minority comparisons by range of magnitude. (b) Absolute meta-analytic effect sizes for between-minority comparisons by range of magnitude. Effect sizes are reported as Cohen's d.
We found the same pattern of results averaging across relative rather than absolute differences. The unweighted difference on levels of psychopathology between White and minorities was very small in magnitude, d = –0.08, SD = 0.222. When only meta-analyses with 25% or less of study overlap were included, the unweighted difference remained small, d = –0.13, SD = 0.213. The weighted random-effects difference was also small in magnitude, d + = –0.12, τ 2 = .026, I 2 = 99.65. Compared to Whites, minorities had higher mean levels of psychopathology, although these differences were very small.
Overall difference between-minority groups
We identified five meta-analyses that examined mean-level differences between-minority groups in levels of psychopathology. Across all meta-analyses, including 73 primary studies (k = 25, N = 111,650), the unweighted difference was medium in magnitude, d = 0.50, SD = 0.497. These meta-analyses had less than 25% of study overlap. The weighted random-effects difference was equal in magnitude compared to the unweighted difference (d + = 0.50, τ 2 = .429, I 2 = 99.48, 95% CI [–0.07, 1.08]). We conducted a sensitivity analysis by removing the largest effect size (d = 1.31) and found that the overall weighted difference was smaller in magnitude and the variability was substantially reduced (d + = 0.30, τ 2 = .033, I 2 = 99.06, 95% CI [0.12, 0.48]). The distribution of effect sizes representing the differences between-minorities is displayed in Figure 3. Most (76%) mean-level differences between-minority groups were in the close-to-zero or small range, whereas 24% were in the moderate, large, and very large ranges. These results show that there are small differences between Whites and minorities, and these differences are similar or smaller than differences between-minorities, supporting the cultural similarities hypothesis.
Aim 2. Determining if general and meta-analytic method moderators account for these differences
All the moderator analyses refer to cultural similarities between Whites and minorities in levels of psychopathology. Between-minorities moderation tests could not be conducted because of the lack of available meta-analytic data.
General moderators
Developmental period
We found evidence of cultural similarities across age groups, Q* between (2) = 4.98, p = .083. The weighted difference for meta-analyses with samples composed of children and adolescents only (d + = 0.22), adults only (d + = 0.13), and samples of mixed ages (d + = 0.29) was small in magnitude.
Mean age
We found evidence of cultural similarities across mean ages, Q* between (3) = 5.69, p = .128. The weighted difference for adolescents between 11 and 18 years old (d + = 0.15), young adults (d + = 0.32), adults between 26 and 39 years old (d + = 0.26), and adults 40 years and older (d + = 0.14) was small in magnitude.
Dimension and type of psychopathology
We found evidence of cultural similarities across dimensions of psychopathology, Q* between (1) = 0.17, p = .678. The weighted difference for externalizing (d + = 0.23) and internalizing behavior problems (d + = 0.20) was small. We found evidence of cultural similarities across types of psychopathology, Q* between (2) = 0.04, p = .978. The weighted difference was small in magnitude for anxiety (d + = 0.24), psychopathy (d + = 0.23), and depression (d + = 0.21).
Sex
We found evidence of cultural similarities across sex groups, Q* between (2) = 0.04, p = .980. The weighted difference was small in magnitude for samples composed of all females (d + = 0.25), all males (d + = 0.23), and both males and females (d + = 0.24).
Educational attainment
We found evidence of cultural similarities across levels of educational attainment, Q* between (1) = 1.01, p = .315. The weighted difference that was small in magnitude for samples with an average educational attainment of less than 12 years (d + = 0.26) and with at least 12 completed years of school (d + = 0.20).
Measure of psychopathology
We found evidence of cultural similarities across types of measure of psychopathology, Q* between (1) 3.14, p = .076. The difference for meta-analyses that aggregated across interviews was not different from zero (d + = 0.12) whereas the difference for meta-analyses that aggregated across self-reports or questionnaires was (d + = 0.25). Furthermore, we found evidence of cultural similarities across meta-analyses that aggregated across number of measures, Q* between (1) 0.45, p = .502. Effects were small for single measure (d + = 0.21) and multiple measures (d + = 0.19).
Decade of publication
We found evidence of cultural similarities across decade of publication, Q* between (1) 2.56, p = .110. The weighted difference was small for meta-analyses published from 2000 to 2009 (d + = 0.17) and from 2010 to 2017 (d + = 0.28).
Meta-analytical method moderators
Quality
We found evidence of cultural similarities across quality, Q* between (2) = 2.46, p = .292 (Table 3). Estimates excluding meta-analyses with the lowest quality (d + = 0.19) were consonant to the absolute difference when all meta-analyses were considered (d + = 0.23).
Sources
Most meta-analyses we identified (58%) included published and unpublished sources. We found evidence of cultural similarities across different sources, Q* between (1) = 0.86, p = .353. We found small differences among meta-analyses that included only published sources (d + = 0.29) and both published and unpublished sources (d + = 0.21).
Effect size metric
We found evidence of cultural similarities across effect size metrics, Q* between (2) = 0.63, p = .728. The weighted difference was small among meta-analyses that reported Cohen's d (d + = 0.21), Hedges's g (d + = 0.26), and odds ratio (d + = 0.17).
Statistical model
We found evidence of cultural similarities across statistical models, Q* between (1) = 1.60, p = .205. The weighted differences were small among meta-analyses that employed a fixed-effects (d + = 0.29) and a random-effects model (d + = 0.20).
Discussion
In this study, we conducted the first large-scale quantitative test of the cultural differences and similarities hypotheses in developmental psychopathology with data from 16 meta-analyses on 493 primary studies with over 3 million participants. We employed a second-order meta-analysis to estimate the average absolute difference between Whites and minorities, as well as between- and within-minority groups, on levels of psychopathology. We also examined if moderators attenuated or exacerbated disparities. We found small absolute differences between Whites and minorities (d + = 0.23). As represented in Figure 4, a Cohen's d of 0.23 means that 91% of Whites and minorities will overlap in levels of psychopathology (see Magnusson, Reference Magnusson2018). We also found very small relative differences between Whites and minorities (d + = –0.12), showing that minorities reported higher levels of psychopathology compared to Whites. These findings challenge the expectation articulated by the cultural difference hypothesis that White–minority mental health disparities are substantial and provide support for the cultural similarities hypothesis in developmental psychopathology. However, small effects may represent meaningful group differences (Rosenthal, Reference Rosenthal1990). Thus, these findings also add support to the notion that minorities experience a disproportionate share of adverse health conditions as compared to Whites in the United States.
Graphic depiction of our findings on White–minority and between-minority differences in psychopathology.
Moreover, when we examined disparities between-minorities we found that they were larger (d + = 0.50), or equal in magnitude after excluding an outlier effect size (d + = 0.30), compared to White–minority differences. A Cohen's d of of 0.50 means that 80% of any two minority groups (e.g., African American and Latinos) will overlap, while a Cohen's d of 0.30 conveys that 88% of any two minority groups will overlap in levels of psychopathology. While the Cohen's d of 0.50 with the outlier effect size has a 95% CI that includes zero (i.e., [–0.07, 1.08]), the Cohen's d of 0.30 without the outlier provides a more reliable estimate of group differences because the confidence interval is far from zero (i.e., [0.12, 0.48]). Thus, zero is less likely to be the true value for the parameter of group differences we have estimated (Cumming, Reference Cumming2013). Overall, these findings challenge the assumption of homogeneity between minority groups, a cornerstone of the cultural differences hypothesis, as mental health disparities between any two minority groups are as big or bigger than differences between Whites and any minority group. Taken together, these findings provide support for the cultural similarities hypothesis in developmental psychopathology.
Group comparisons are an enduring problem of cultural research because it is easy to interpret disparities in terms of a deficit model in which the group that differs from the standard group is seen as having failed (Medin et al., Reference Medin, Bennis and Chandler2010). Comparing Whites and minorities in the development of psychopathology is susceptible to the home-field disadvantage, the difficulty intrinsic in research that takes a particular cultural group as the standard for science (Medin et al., Reference Medin, Bennis and Chandler2010). Abnormal development is one important domain in which differences between Whites and minorities may be overestimated, thus reinforcing deficit models. These deficit models are not exclusive to developmental psychopathology, as they are also prevalent in developmental sciences (McLoyd, Reference McLoyd1990, Reference McLoyd2004; Spencer & Markstrom-Adams, Reference Spencer and Markstrom–Adams1990), nor are they exclusive to minorities, as they are also applied to women (Hyde, Reference Hyde2005, Reference Hyde2014). Furthermore, emphasizing cultural differences in the development of psychopathology may reinforce deficit models, bolstering a deficit by difference perspective (Causadias et al., Reference Causadias, Vitriol and Atkin2018b). The other side of deficit models of minorities are surplus models of Whites, which differs from the model minority myth because even if Asian Americans are perceived as superior to African Americans and Latinos, they are often portrayed as foreign compared to Whites (Zou & Cheryan, Reference Zou and Cheryan2017). In contrast, Whites are often perceived as superior and American (Zou & Cheryan, Reference Zou and Cheryan2017). If the underlying assumption of deficit models is that all minorities are struggling and failing, the core message of surplus models is that all Whites are thriving and succeeding. However, Whites also face considerable challenges, and evidence suggest many are not coping adaptively with cultural change and ecological demands. A segment of White youth is struggling with declines in health and increased mortality (Case & Deaton, Reference Case and Deaton2017), opioid addiction (Cicero et al., Reference Cicero, Ellis, Surratt and Kurtz2014), drug abuse (Roberts et al., Reference Roberts, Doogan, Kurti, Redner, Gaalema, Stanton and Higgins2016), and death due to overdose (Givens et al., Reference Givens, Gennuso, Jovaag and Van Dijk2017). Ultimately, both deficit and surplus models are inadequate because they ignore between-group similarities and within-group variation, while overestimating differences between groups that exaggerate otherness. In this study, we show that Whites and minorities vary in the development of psychopathology, but they are more similar than different.
We found evidence in support of the cultural similarities hypothesis across general moderators. White–minority differences in levels of psychopathology remained small across developmental period (e.g., childhood or adolescence), mean age, dimension (i.e., internalizing or externalizing) and type (e.g., depression or anxiety) of psychopathology, sex, years of education, decade of publication, and measure of psychopathology (i.e., type of measure or number of measures). Some of these findings may be explained by our method. For instance, while there is more variability within males compared to within females (Hyde, Reference Hyde2005, Reference Hyde2014), we found that White–minority mental health differences were similar for meta-analyses employing all male samples, all female samples, and both males and females. Due to the limited number of meta-analytic effects in each subgroup, we pooled the variability across the groups to weight their aggregate effect size estimate rather than weighting each with their within-subgroup variability. Thus, it is plausible that we could not capture within-sex variability. Other findings on general moderators can be explained given the availability of data. For example, regarding type of psychopathology, we could only explore Whites–minority differences on anxiety, psychopathy, and depression because we identified three or more meta-analyses on each of these outcomes. In contrast, we only identified one meta-analysis on body dissatisfaction, somatic complains, substance use, and suicide ideation. Likewise, we found evidence in support of the cultural similarities hypothesis across meta-analytic method moderators. White–minority differences in levels of psychopathology remained small regardless of meta-analytic quality (e.g., inclusion criteria or reliability), inclusion of published and unpublished sources, reported effect size metric (e.g., Cohen's d), and statistical method of aggregating data (e.g., fixed vs. random effects). Still, it is important to pay attention to quality in meta-analytic studies of health outcomes (Higgins et al., Reference Higgins, Lane, Anagnostelis, Anzures-Cabrera, Baker, Cappelleri and Whitehead2013), as quality can help address the problem of massive production of ambiguous and conflicted systematic reviews and meta-analyses (Ioannidis, Reference Ioannidis2016). Future studies should examine the role of these moderators by conducting individual person data analysis with data from multiple health studies (Hornburg, Wang, & McNeil, Reference Hornburg, Wang and McNeil2018).
There are several noteworthy limitations to this study. We could not address some of the aims of this study because of lack of meta-analytic research on health disparities in the United States. The participants in the studies included in the meta-analyses were 82% White, 17% African American, and 1% Asian American, Latino, and American Indian. Furthermore, only 44% of meta-analyses compared Whites with more than one minority group and most focused on White–Black comparisons. Given this available data, we could not test within-minority differences in levels of psychopathology. This is problematic because minority groups tend to be heterogeneous:
Membership in an ethnic or racial minority group is not equivalent to a common cultural experience for individuals, given the wide variation that exists within these groups in terms of ethnic identity, social class, regional identification, and, among Latinos, in terms of country of origin, generational history or recency of immigration, acculturation status, language preference, etc. (McLoyd, Reference McLoyd2004, p. 189)
For example, adolescents of Colombian, Guatemalan, Honduran, Mexican, Nicaraguan, Puerto Rican, and Salvadoran ancestry living in the United States are often labeled simply as Latinos, despite evidence that they differ on ethnic identity, self-esteem, emotional autonomy, and familial ethnic socialization (Umaña-Taylor & Fine, Reference Umaña-Taylor and Fine2001). Given this large within-group diversity, some scholars have questioned the validity of panethnic labels such as Asian Americans and Latinos (DiPietro & Bursik, Reference DiPietro and Bursik2012; Umaña-Taylor & Fine, Reference Umaña-Taylor and Fine2001). The same can be said about Whites in the United States, which are often grouped together under one category, but may also vary in terms of ancestry (e.g., Italian American and German American). Therefore, the investigation of between- and within-White differences in mental health is an important future research direction. Another potential limitation is subgroup reporting bias, in which significant effects are selectively reported within-studies that could result in an overestimation of group differences (Hahn, Williamson, Hutton, Garner, & Flynn, Reference Hahn, Williamson, Hutton, Garner and Flynn2000).
In addition, this study was limited because we could not examine the role of several moderators due to lack of available data. For instance, we could not extract data on socioeconomic status or annual income because they were not reported in most of the included meta-analyses. When we searched within the primary studies included in each of these meta-analyses, approximately 14% of them reported data on socioeconomic or annual income. Addressing the role of socioeconomic status is central for testing the cultural differences and similarities hypotheses because economically advantaged and disadvantaged youth are more likely to display elevated levels of internalizing behavior problems like depression and anxiety, as well as externalizing behavior problems like aggression and delinquent behavior (Coley, Sims, Dearing, & Spielvogel, Reference Coley, Sims, Dearing and Spielvogel2017; Duncan, Magnuson, & Votruba-Drzal, Reference Duncan, Magnuson and Votruba-Drzal2017). Future studies should also examine the role of geographic origin (e.g., urban vs. rural) in accounting for White–minority differences, as well as between- and within-minority differences. In addition, we could not examine the moderating role of research design because we did not identify any meta-analyses of case-control/cohort or longitudinal designs. The lack of developmental meta-analyses of White–minority differences is an important weakness of the field. Future studies should examine the degree to which these groups differ in terms of time of onset (early vs. late), intercept and slope in trajectories of psychopathology by group (increasing vs. declining), how these trajectories are impacted by developmental transitions (change vs. continuity), and group variation in how similar experiences lead to different outcomes or different experiences lead to similar outcomes (equifinality vs. multifinality).
Another noteworthy limitation is that this study is restricted to abnormal development, but the essence of developmental psychopathology is considering both normal and abnormal development together (Cicchetti, Reference Cicchetti1984; Sroufe, Reference Sroufe1990). For that reason, support for the cultural similarities hypothesis will remain inconclusive pending research on normal development and well-being, as well as general health outcomes. Instead of relying exclusively on data from meta-analyses, future studies should address these questions by using individual participant data from national representative studies. This approach allows testing mental health outcomes directly in different data sets, for instance, through integrative data analysis (Bainter & Curran, Reference Bainter and Curran2015). In addition, other methods can be useful in accomodating dependencies of effect sizes, including robust variance estimation that provides a more accurate estimate of standard errors (Tanner-Smith & Tipton, Reference Tanner-Smith and Tipton2014). For our moderator analyses, we included multiple effect sizes from meta-analyses, so this dependency can bias standard errors (Hedges, Tipton, & Johnson, Reference Hedges, Tipton and Johnson2010). Future research should assess alternative methods for second-order meta-analyses that account for dependency within and between meta-analyses. Cohen's d also has its limitations because how it is calculated can produce different values and interpretations (Cumming, Reference Cumming2013). Therefore, future research should examine cultural differences and similarities using other tests of group differences, like Mahalanobi's D (Del Giudice, Booth, & Irwing, Reference Del Giudice, Booth and Irwing2012), in addition to Cohen's d.
Although this second-order meta-analysis is the first to reveal small White–minority differences in overall levels of psychopathology, it does not address variation at the cultural, ecological, and biological levels that can explain differences and similarities, or what results from their interplay (Causadias et al., Reference Causadias, Telzer and Lee2017). This is crucial because research on culture is more than the study of group differences (Wang, Reference Wang2016). These processes would explain why, despite minorities’ asymmetric exposure to environmental sources of risk and adversity that increases the odds of developing maladaptive responses compared to Whites, only very small differences between the two emerged. Future research should identify the cultural processes that account for differences and similarities, including cultural values, beliefs, behaviors, rituals, identities, community participation, and family socialization practices (Causadias, Reference Causadias2013). Another challenge to this endeavor is to identify normative cultural processes and practices that are shared by both Whites and minorities, and not bounded to a specific group. Finally, this study focused exclusively on comparing Whites and minorities in the United States in levels of psychopathology. Thus, the degree to which these contrasts apply to groups in the rest of the world remains unknown. Future studies should examine cultural differences and similarities in mental health, as well as other outcomes, among other groups and regions in the globe. This is key given that most psychological research focuses on Americans, providing an incomplete picture of human behavior and cognition (Arnett, Reference Arnett2008).
Despite these challenges, this is the first study to provide evidence in support of the cultural similarities hypothesis in the development of psychopathology. This research validates decades of extensive conceptualizations cautioning against overvaluing cultural differences. As important as cultural and ecological arguments are in anticipating cultural differences, expecting large differences between groups is incompatible with one central genetic argument: the fact that all human beings are part of the same species. Thus, our shared phylogenetic evolution imposes constraints on the possible differences in psychological and developmental functioning (Poortinga, Reference Poortinga2016). After all, “genes hold culture on a leash” (Wilson, Reference Wilson1978, p. 172). We argued in this study that primary studies cannot ascertain the exact magnitude of the difference between Whites and minorities in levels of psychopathology, as they are bounded by restricted samples, idiosyncratic measures, specific outcomes, and diversity in methods. They are also distorted by publication bias and are often likely to report false positives. Meta-analyses address some of these limitations but are also constrained by their focus. To overcome these issues, we employed a second-order meta-analysis that summarizes data across a large number of primary studies and meta-analyses. However, quantitative evidence is unlikely to settle this issue, as behind assertions of differences and similarities between groups lie beliefs about race. To deploy the authority of science cannot mask the reality that debates about race are motivated explicitly or implicitly by ideology, and are often impervious to evidence (Comfort, Reference Comfort2014).
We are presented with, what appears to be, two undesirable options. On the one hand, we can endorse the cultural differences hypothesis that minimizes distinctions between- and within-minorities, and highlights White–minority health disparities. However, they can reinforce deficit models that perpetuate stereotypes of minorities as uniform and pervasively maladjusted. On the other hand, we can support the cultural similarities hypothesis by underscoring variation bewteen-minorities, and commonalities between Whites and minorities, at the cost of neglecting the fact that minorities exhibit higher levels of psychopathology compared to Whites, although these differences were very small. This apparently irresoluble paradox is actually a false dichotomy expressing a Cartesian epistemology of opposites (see Causadias, Updegraff, & Overton, Reference Causadias, Updegraff and Overtonin press). In reality, they are both similar and different, although this study shows that Whites and minorities are more similar than they are different, and various minority groups are more different than they are similar. How do these differences and similarities emerge? What accounts for change and continuity in health disparities? It is time for research on differences and similarities in the development of adaptation between- and within-groups. Do different factors influence initiation and maintenance of pathways to various kinds of problems? Which cultural risk, protective, and promotive factors are shared by some groups, which factors are unique to others, and why? Addressing these questions is central for the future of research on culture, development, and psychopathology (Causadias, Reference Causadias2013).
