In their recent paper, Moffitt et al. (Reference Moffitt, Caspi, Taylor, Kokaua, Milne, Polanczyk and Poulton2009) claim that lifetime prevalence estimates based on retrospective surveys may be half as large as they would be if they were computed as cumulative incidence on the basis of prospective data. They also infer that the persons who are counted as having lifetime disorder in the retrospective surveys over-represent those who have current disorder. The authors base these claims on a string of inferences that they believe allows them to compare lifetime prevalence estimates in the Dunedin cohort to retrospectively ascertained lifetime prevalence estimates from three different large-scale cross-sectional community surveys.
There are several features of the comparison that could raise concerns about the conclusions. For example, the retrospective lifetime estimates are restricted to persons aged 32 and younger and in the Dunedin cohort the cumulative estimates are based on a combination of assessments of 12-month prevalences at only four time points, corresponding to ages 18, 21, 26 and 32. In addition, the Dunedin group used DSM-III-R at two time points and DSM-IV at two others, and unlike the cross-sectional surveys, the Dunedin investigators used trained clinical (rather than lay) interviewers to administer the Diagnostic Interview Schedule (DIS). Moreover, the Dunedin study diagnoses used a rating of impairment and also reported symptoms. Finally, the study populations in two of the comparisons were qualitatively different: Dunedin is a single location in New Zealand and the National Comorbidity Survey (NCS) and the NCS Replication (NCS-R) are representative samples of the USA.
Although these features could undermine the comparisons of lifetime estimates from Dunedin to estimates from the three other studies, they would be expected to have the same effect on 12-month prevalence estimates as on lifetime prevalence estimates. Moffitt and her colleagues argue that their 12-month results are in fact quite close to the other studies for any anxiety disorder, and also for panic, specific phobia, social phobia and generalized anxiety. We accept the argument for panic and specific phobia and for the comparison between Dunedin and NCS-R for any anxiety. In other instances, however, the amount of overlap of the confidence intervals is consistent with reliable differences (Cumming & Finch, Reference Cumming and Finch2005); factors such as the lower refusal rate might have contributed to higher 12-month prevalence estimates in Dunedin.
We do not have to be distracted by that question when we restrict our attention to panic, specific phobia and the NCS-R comparison for any anxiety disorder, as these disorders have a lower 12-month prevalence in Dunedin than in the cross-sectional surveys. Analyses of these disorders provide compelling support for the authors' contention that incomplete recall of past mental disorders, particularly disorders that were transient rather than chronic, will result in substantial underestimation of lifetime prevalence. A very conservative interpretation still leaves us with the conclusion that retrospective estimates of lifetime prevalence are substantially biased. A conservative guess is that they tend to produce values that are two-thirds the size of the actual value. Much larger bias will be observed for disorders that are experienced as acute episodes at one or two times in a lifetime.
Even in this qualified form, the Moffitt et al. results force us to confront some central questions in psychiatric epidemiology that have been raised episodically for more than 50 years but have never been resolved. We can start with the question: Why do we measure lifetime prevalence? Outside of psychiatry, epidemiologists generally give short shrift to this quantity, as it is difficult to interpret precisely and is confounded with the age distribution of the target population (Miettinen, Reference Miettinen1976; Rothman et al. Reference Rothman, Greenland and Lash2008). The quantity is particularly problematic for common disorders, and the Moffitt et al. results suggest that psychiatric disorders may be even more common than the retrospective literature had suggested. Indeed, weaknesses of lifetime prevalence have been noted by psychiatric epidemiologists and also by other epidemiologists (Susser et al. Reference Susser, Schwartz, Morabia and Bromet2006). As early as 1963, Ernest Gruenberg, a renowned psychiatric epidemiologist, stated in his review of the landmark Midtown Manhattan Study of mental disorders in the community (Gruenberg, Reference Gruenberg1963): ‘This particular measure is an example of new gimmicks of mensuration introduced into a field which has enough real troubles without being further burdened by unhelpful tricks. Lifetime prevalence measures are of no visible usefulness’ (Gruenberg, Reference Gruenberg1963, p. 92).
So why estimate lifetime prevalence? One part of the answer, we believe, lies in its intuitive appeal to a general audience. Both clinicians and lay people want to know answers to the questions: How many of us have had a mental disorder? How many of us will get a mental disorder? To many, the lifetime prevalence measure appears to answer these questions. Lifetime prevalence estimates have played an important role in the increasing awareness of the burden of mental disorders in the current era (WHO, 2001), and they can help to reduce stigma by normalizing mental problems such as depression, anxiety and attention deficits. Researchers might also be attracted to lifetime prevalences because they seem to provide more statistical power for risk analyses as more positive cases are available.
Even acknowledging these apparent benefits, the results of Moffitt et al. (Reference Moffitt, Caspi, Taylor, Kokaua, Milne, Polanczyk and Poulton2009) tell us that lifetime prevalence estimates based on retrospective reports will give an inaccurate picture of the overall burden of mental disorders and will inadequately establish how normative common mental disorders can be. Because cross-sectional surveys underestimate the lifetime prevalence, they misinform discussion of real and profound questions about the nature of mental disorders. Prospectively obtained results are likely to find that, under current diagnostic systems in contemporary societies, most, if not all, individuals who survive to old age will have had a diagnosable mental disorder during their lifetime. It does not follow that mental disorders are not ‘real’, that our diagnostic systems are flawed, or that these disorders do not merit prevention or treatment. There are many examples of diseases, such as flu and gastrointestinal infections, that are experienced by almost everyone in a population but that are nonetheless hazardous and/or disabling and are given high priority by public health systems and medical practitioners alike. It does follow, however, that we need to acknowledge this fact when we conceptualize and articulate what we mean by mental health and disorder, and that these issues need to be thoroughly examined in the context of even higher lifetime prevalences than have been previously considered. The debate on whether it is desirable to reduce the prevalences through the imposition of impairment criteria in diagnostic revisions (e.g. Regier et al. Reference Regier, Kaelber, Rae, Farmer, Knauper, Kessler and Norquist1998; Narrow et al. Reference Narrow, Rae, Robins and Regier2002; Kessler et al. Reference Kessler, Merikangas, Berglund, Eaton, Koretz and Walters2003; Horwitz & Wakefield, Reference Horwitz and Wakefield2007; Parker, Reference Parker2007) needs to be informed by the much higher lifetime prevalences suggested by the prospective results.
The analyses of Moffitt and colleagues also tell us that retrospective cross-sectional surveys underestimate the ratio of lifetime to current prevalence, and therefore yield incorrect inferences about the duration of mental disorders. This is serious because recovery from mental disorders is much more common than we can infer from the results of usual cross-sectional surveys. Moreover, because cross-sectional lifetime prevalence assumes homotypy in the manifestations of a disorder, the developmental evolution of mental disorders from birth up to later life cannot be captured by this measure. In light of the growing evidence for heterotypy and co-morbidity of common mental disorders over time, the reliance on lifetime prevalence may encourage us to misrepresent the fundamental nature of these disorders.
By acknowledging these points, we open the way to more effectively tailor our research designs to find the causes of mental disorders. For instance, we may need to consider timing as much as occurrence for the most common mental disorders, similar to studies of Alzheimer's disease in the very old. More generally, there remains an ongoing need to conduct careful longitudinal studies. If we want to study incident cases of disorders such as depression, it is not sufficient to simply include a retrospective report section to determine mental health history. Although the cost of such projects has historically been perceived to be high, the dividends in knowledge relative to cross-sectional retrospective studies may make prospective studies literal bargains. Regrettably, some government agencies have removed epidemiological studies from their list of priorities. The findings of the Moffitt et al. report warn the psychopathology community not to rely too much upon the biased cross-sectional findings of the past. We need to sustain prospective studies, and foster the development of new methodologies to track persons over their lives. Longitudinal work is needed to gain new understanding of development, social contexts, and also coping and prevention strategies related to onset and course of mental disorder.
You have
Access
In their recent paper, Moffitt et al. (Reference Moffitt, Caspi, Taylor, Kokaua, Milne, Polanczyk and Poulton2009) claim that lifetime prevalence estimates based on retrospective surveys may be half as large as they would be if they were computed as cumulative incidence on the basis of prospective data. They also infer that the persons who are counted as having lifetime disorder in the retrospective surveys over-represent those who have current disorder. The authors base these claims on a string of inferences that they believe allows them to compare lifetime prevalence estimates in the Dunedin cohort to retrospectively ascertained lifetime prevalence estimates from three different large-scale cross-sectional community surveys.
There are several features of the comparison that could raise concerns about the conclusions. For example, the retrospective lifetime estimates are restricted to persons aged 32 and younger and in the Dunedin cohort the cumulative estimates are based on a combination of assessments of 12-month prevalences at only four time points, corresponding to ages 18, 21, 26 and 32. In addition, the Dunedin group used DSM-III-R at two time points and DSM-IV at two others, and unlike the cross-sectional surveys, the Dunedin investigators used trained clinical (rather than lay) interviewers to administer the Diagnostic Interview Schedule (DIS). Moreover, the Dunedin study diagnoses used a rating of impairment and also reported symptoms. Finally, the study populations in two of the comparisons were qualitatively different: Dunedin is a single location in New Zealand and the National Comorbidity Survey (NCS) and the NCS Replication (NCS-R) are representative samples of the USA.
Although these features could undermine the comparisons of lifetime estimates from Dunedin to estimates from the three other studies, they would be expected to have the same effect on 12-month prevalence estimates as on lifetime prevalence estimates. Moffitt and her colleagues argue that their 12-month results are in fact quite close to the other studies for any anxiety disorder, and also for panic, specific phobia, social phobia and generalized anxiety. We accept the argument for panic and specific phobia and for the comparison between Dunedin and NCS-R for any anxiety. In other instances, however, the amount of overlap of the confidence intervals is consistent with reliable differences (Cumming & Finch, Reference Cumming and Finch2005); factors such as the lower refusal rate might have contributed to higher 12-month prevalence estimates in Dunedin.
We do not have to be distracted by that question when we restrict our attention to panic, specific phobia and the NCS-R comparison for any anxiety disorder, as these disorders have a lower 12-month prevalence in Dunedin than in the cross-sectional surveys. Analyses of these disorders provide compelling support for the authors' contention that incomplete recall of past mental disorders, particularly disorders that were transient rather than chronic, will result in substantial underestimation of lifetime prevalence. A very conservative interpretation still leaves us with the conclusion that retrospective estimates of lifetime prevalence are substantially biased. A conservative guess is that they tend to produce values that are two-thirds the size of the actual value. Much larger bias will be observed for disorders that are experienced as acute episodes at one or two times in a lifetime.
Even in this qualified form, the Moffitt et al. results force us to confront some central questions in psychiatric epidemiology that have been raised episodically for more than 50 years but have never been resolved. We can start with the question: Why do we measure lifetime prevalence? Outside of psychiatry, epidemiologists generally give short shrift to this quantity, as it is difficult to interpret precisely and is confounded with the age distribution of the target population (Miettinen, Reference Miettinen1976; Rothman et al. Reference Rothman, Greenland and Lash2008). The quantity is particularly problematic for common disorders, and the Moffitt et al. results suggest that psychiatric disorders may be even more common than the retrospective literature had suggested. Indeed, weaknesses of lifetime prevalence have been noted by psychiatric epidemiologists and also by other epidemiologists (Susser et al. Reference Susser, Schwartz, Morabia and Bromet2006). As early as 1963, Ernest Gruenberg, a renowned psychiatric epidemiologist, stated in his review of the landmark Midtown Manhattan Study of mental disorders in the community (Gruenberg, Reference Gruenberg1963): ‘This particular measure is an example of new gimmicks of mensuration introduced into a field which has enough real troubles without being further burdened by unhelpful tricks. Lifetime prevalence measures are of no visible usefulness’ (Gruenberg, Reference Gruenberg1963, p. 92).
So why estimate lifetime prevalence? One part of the answer, we believe, lies in its intuitive appeal to a general audience. Both clinicians and lay people want to know answers to the questions: How many of us have had a mental disorder? How many of us will get a mental disorder? To many, the lifetime prevalence measure appears to answer these questions. Lifetime prevalence estimates have played an important role in the increasing awareness of the burden of mental disorders in the current era (WHO, 2001), and they can help to reduce stigma by normalizing mental problems such as depression, anxiety and attention deficits. Researchers might also be attracted to lifetime prevalences because they seem to provide more statistical power for risk analyses as more positive cases are available.
Even acknowledging these apparent benefits, the results of Moffitt et al. (Reference Moffitt, Caspi, Taylor, Kokaua, Milne, Polanczyk and Poulton2009) tell us that lifetime prevalence estimates based on retrospective reports will give an inaccurate picture of the overall burden of mental disorders and will inadequately establish how normative common mental disorders can be. Because cross-sectional surveys underestimate the lifetime prevalence, they misinform discussion of real and profound questions about the nature of mental disorders. Prospectively obtained results are likely to find that, under current diagnostic systems in contemporary societies, most, if not all, individuals who survive to old age will have had a diagnosable mental disorder during their lifetime. It does not follow that mental disorders are not ‘real’, that our diagnostic systems are flawed, or that these disorders do not merit prevention or treatment. There are many examples of diseases, such as flu and gastrointestinal infections, that are experienced by almost everyone in a population but that are nonetheless hazardous and/or disabling and are given high priority by public health systems and medical practitioners alike. It does follow, however, that we need to acknowledge this fact when we conceptualize and articulate what we mean by mental health and disorder, and that these issues need to be thoroughly examined in the context of even higher lifetime prevalences than have been previously considered. The debate on whether it is desirable to reduce the prevalences through the imposition of impairment criteria in diagnostic revisions (e.g. Regier et al. Reference Regier, Kaelber, Rae, Farmer, Knauper, Kessler and Norquist1998; Narrow et al. Reference Narrow, Rae, Robins and Regier2002; Kessler et al. Reference Kessler, Merikangas, Berglund, Eaton, Koretz and Walters2003; Horwitz & Wakefield, Reference Horwitz and Wakefield2007; Parker, Reference Parker2007) needs to be informed by the much higher lifetime prevalences suggested by the prospective results.
The analyses of Moffitt and colleagues also tell us that retrospective cross-sectional surveys underestimate the ratio of lifetime to current prevalence, and therefore yield incorrect inferences about the duration of mental disorders. This is serious because recovery from mental disorders is much more common than we can infer from the results of usual cross-sectional surveys. Moreover, because cross-sectional lifetime prevalence assumes homotypy in the manifestations of a disorder, the developmental evolution of mental disorders from birth up to later life cannot be captured by this measure. In light of the growing evidence for heterotypy and co-morbidity of common mental disorders over time, the reliance on lifetime prevalence may encourage us to misrepresent the fundamental nature of these disorders.
By acknowledging these points, we open the way to more effectively tailor our research designs to find the causes of mental disorders. For instance, we may need to consider timing as much as occurrence for the most common mental disorders, similar to studies of Alzheimer's disease in the very old. More generally, there remains an ongoing need to conduct careful longitudinal studies. If we want to study incident cases of disorders such as depression, it is not sufficient to simply include a retrospective report section to determine mental health history. Although the cost of such projects has historically been perceived to be high, the dividends in knowledge relative to cross-sectional retrospective studies may make prospective studies literal bargains. Regrettably, some government agencies have removed epidemiological studies from their list of priorities. The findings of the Moffitt et al. report warn the psychopathology community not to rely too much upon the biased cross-sectional findings of the past. We need to sustain prospective studies, and foster the development of new methodologies to track persons over their lives. Longitudinal work is needed to gain new understanding of development, social contexts, and also coping and prevention strategies related to onset and course of mental disorder.
Declaration of Interest
None.