To the Editor:

We welcomed Bjureberg et al.'s (Reference Bjureberg, Dahlin, Carlborg, Edberg, Haglund and Runeson2021) study of the Columbia-Suicide Severity Rating Scale (C-SSRS) Screener's performance under the real-world clinical conditions of psychiatric emergency department (ED) practice (Bjureberg et al., Reference Bjureberg, Dahlin, Carlborg, Edberg, Haglund and Runeson2021). Our group recently published on the performance of the C-SSRS Screener among a sample of 92 643 general medical ED and psychiatric ED patients in a safety net hospital in the United States (Simpson et al., Reference Simpson, Goans, Loh, Ryall, Middleton and Dalton2020). When considering this research alongside our own recent work, it is clear that significant changes in suicide screening, screening research, and emergency psychiatric practice are warranted.

Together, these works provide much-needed data on performance of this screener. The Screener's original publication lacked crucial data related to its development, performance, and validation (Posner et al., Reference Posner, Brown, Stanley, Brent, Yershova, Oquendo and Mann2011); only 10 years after its introduction can it be said that there are performance data with which to judge the Screener's utility in practice. The performance of the Screener at the one month and one year time periods are roughly similar in our two studies; these outcomes are compared in Table 1. In our study, the Screener did not demonstrate superior performance at higher risk strata, so in primary analyses we interpreted the risk as binary. There are some limitations to the current study. Bjureberg et al. note that 51% of their patients with a risk score of 3+ were hospitalized, which undoubtedly introduced a protected period for which they are at reduced risk of suicide and may understate risk associated with a higher C-SSRS scores (i.e. immortality bias). The team handled missing data by presuming that missing scores were missing at random – a likely incorrect presumption. Our own experience suggests that patients missing suicide screening data are at higher risk of suicide than those with negative screens. Regardless of these limitations, the case has been made that suicidal ideation is associated with subsequent risk of suicide after an ED visit.

Table 1.

Performance of the Columbia-suicide severity rating scale screener for predicting suicide after emergency department visit

AUC, area under the receiver operating characteristic curve; CI, confidence interval; ED, emergency department.

Nevertheless, suicidal ideation is a poor way to screen for suicide risk in emergency settings, as the C-SSRS Screener purports to do. Bjureberg et al. (Reference Bjureberg, Dahlin, Carlborg, Edberg, Haglund and Runeson2021) describe the very low predictive values of this instrument given the low prevalence of suicide. Their discussion misses the more significant issue: about half of patients who die by suicide within a month, and fewer than half who die within a year score positive even at the optimal C-SSRS Screener cut off. Higher Screener scores are associated with still lower sensitivity, comparable to our own findings among general ED patients. The authors note that the Screener thus cannot be used to guide intensive psychiatric intervention. Indeed, the Screener should not be used to guide provision of any psychiatric services in emergency settings.

Another intriguing but understated finding of Bjureberg et al.'s (Reference Bjureberg, Dahlin, Carlborg, Edberg, Haglund and Runeson2021) study is that most patients who die after a psychiatric ED visit die of causes other than suicide. Seventy-one percent of patients died of a cause other than by suicide in this study, comparable to the 75% of C-SSRS positive patients who died of causes other than suicide in our own publication. Suicide is a tragedy and an epidemic. Bjureberg's paper likely overestimates the incidence of suicide by including events of undetermined intent; their reported suicide incidence is an extraordinarily high 570 suicides per 100 000 person-years. Yet even so suicide is still not the most common cause of death among even these most severely ill psychiatric patients, who die more than a decade younger than the general population and are at unique risk for cardiovascular disease, substance use disorders, and traumatic death (Goldman-Mellor, Olfson, Lidon-Moyano, & Schoenbaum, Reference Goldman-Mellor, Olfson, Lidon-Moyano and Schoenbaum2019; Plana-Ripoll et al., Reference Plana-Ripoll, Pedersen, Agerbo, Holtz, Erlangsen, Canudas-Romo and Laursen2019). Given this mortality, and the limited evidence underlying treatments to reduce suicide after an ED visit, emergency psychiatry must move to address mortality of other causes among patients with mental illness in the ED. Possible interventions include more assertive addiction treatment or behavioral interventions to enhance medical adherence among patients with severe persistent mental illness.

Finally, these results demand re-assessment of how we design and research suicide screening instruments. The rarity of suicide demands studies with substantial samples. Large, real-world studies using vital statistics registries ought to be the definitive methodology for validating these instruments, particularly when assessing near-term outcomes (Rudd et al., Reference Rudd, Berman, Joiner, Nock, Silverman, Mandrusiak and Witte2006). Smaller trials of screeners do not reflect actual practice and fail to capture a sufficient number of outcomes to inform clinical practice. Assessing observed suicide mortality is feasible and renders surrogate measures moot. (These measures, such as self-reported suicidality, are important in their own right.) The use of receiver operating characteristic (ROC) analyses also appears questionable. ROC analyses naturally overstate the performance when cohorts are of imbalanced size, as will always be the case with suicide epidemiology. Alternative methods including precision-recall analysis, classification and regression tree analysis, or machine learning models may be more suitable to this task (Fu, Yi, & Pan, Reference Fu, Yi and Pan2019). Future tools and studies might consider the utility of these instruments for predicting all-cause mortality. A future instrument might also consider a validation component that could better assess symptom exaggeration or even outright malingering, which is suspected among nearly one-third of emergency psychiatry patients (Rumschik & Appel, Reference Rumschik and Appel2019).

In summary, Bjureberg et al. (Reference Bjureberg, Dahlin, Carlborg, Edberg, Haglund and Runeson2021) provide needed data on the use of this common suicide risk screening instrument. Unfortunately, the benefits of this instrument and ED suicide assessment remain elusive. Our combined works should compel emergency psychiatry to develop new approaches to suicide risk screening and mental health treatment in emergency settings.