Response

Algorithmic decision-making involves two ethically distinct stages: a prediction, and a decision taken (at least partly) based on that prediction. ^{Reference Beigang1} Predictions aim at accuracy whereas decisions made under conditions of scarcity concern the allocation of limited resources according to appropriate distributive principles. A recent feature article by Cockburn and Large ^{Reference Cockburn and Large2} argues that probabilistic suicide risk prediction models – such as OxSATS ^{Reference Fazel, Vazquez-Montes, Molero, Runeson, D’Onofrio and Larsson3} – should not be used in clinical care. However, their critiques conflate prediction with decision-making and are better directed at problems of resource scarcity and unreflective allocation practices than at the predictive function of OxSATS itself.

Conflating predictions and decisions

Cockburn and Large use vignettes to illustrate how the implementation of OxSATS might affect clinical services. One vignette concerns the prioritisation of older male patients over younger female patients, based on the higher correlation between suicide risk and age and male gender. They argue that the use of OxSATS would ‘introduce a systematic gender and age bias’ that ‘would necessitate candid, thoughtful discussions among professionals, patients and healthcare funders’.

However, because the associations between age, male gender and suicide risk are among the most replicated findings in the field, incorporating these variables into a predictive model does not, per se, introduce bias. To suggest otherwise conflates prediction, which is evaluated in terms of accuracy, with allocation, which is evaluated in terms of distributive justice. The relevant ethical question is whether it is permissible to prioritise some individual patients over others based on factors such as their population subgroup, even when their predicted risk is lower.

There may be principled reasons to do so. A relational egalitarian ^{Reference Voigt4} could argue that historical and ongoing failures to adequately address women’s mental health justify additional prioritisation as a corrective measure for inequalities that reflect hierarchies of moral value. A utilitarian ^{Reference Eggleston5} might contend that preventing suicide among younger patients produces greater overall benefit by preserving more future life-years. These arguments are contestable, but they concern allocation decisions rather than the legitimacy of risk prediction. Importantly, such distributive questions would arise regardless of whether OxSATS is used.

Indeed, OxSATS may increase transparency by making explicit the role of age and gender in suicide risk prediction, in contrast to unaided clinical judgement where such factors may influence decisions implicitly and without scrutiny. One advantage of algorithmic tools is that they can expose morally salient considerations that would otherwise remain opaque within individual clinicians’ reasoning, thereby facilitating ethical reflection on allocative decision-making.

Resource scarcity as the underlying problem

Another vignette describes a young patient with a low suicide risk score who may be deprioritised for mental health referral despite significant distress. Cockburn and Large suggest that this outcome would constitute a distributive injustice attributable to the use of OxSATS. However, under conditions of severe resource scarcity, the fact that a patient who could benefit from care does not receive it is not, per se, evidence of injustice.

From a distributive justice perspective, injustice arises when someone with a stronger claim to a scarce resource is denied it in favour of someone with a weaker claim. This situation is analogous to transplant ethics: although all patients on a waiting list would benefit from a transplant, scarcity necessitates prioritisation. ^{Reference Webb6} Ethical evaluation therefore concerns the criteria used to allocate resources, not the mere existence of unmet need. Vignette-based attention to disadvantaged individuals is insufficient to demonstrate that OxSATS would produce unjust outcomes.

Cockburn and Large may nevertheless be correct in saying that the broader context of overstretched mental health services constitutes a systemic injustice, insofar as insufficient societal resources are directed towards mental healthcare. ^{Reference Deahl7} In such non-ideal conditions, allocation decisions must still be made but they are constrained by unjust background circumstances. OxSATS, however, does not create this scarcity. The same allocation challenges would arise whether decisions were made by algorithms, clinicians or a combination of both.

The risk of unreflective allocation

Cockburn and Large further criticise the potential for predictive tools to be treated as determinative of allocation decisions. One vignette concerns a patient with emergent schizophrenia who might be denied appropriate treatment if a low OxSATS score were used as a screening threshold for further assessment. However, this would constitute an inappropriate use of the tool. A judgement that a patient is at low risk of suicide does not entail that they do not require mental health support, particularly given the complex relationship between mental illness and suicide risk. ^{Reference Favril, Yu, Geddes and Fazel8}

The possibility of misuse does not, per se, justify abandoning a predictive tool. Rather, it highlights the importance of clearly specifying appropriate uses and limitations. Cockburn and Large offer inconsistent accounts of OxSATS’s intended role. At one point they ask readers to ‘keep an open mind that good clinicians can and will outperform these tools’, suggesting an adversarial relationship between clinical and algorithmic judgement, whereas OxSATS is intended as a consultative tool for clinicians, exactly because, as they put it, ‘the role of the clinician is much broader than risk prediction’. Of OxSATS, they note that ‘It is designed to be used alongside clinicians as a clinical adjunct, not as a replacement. However, the authors suggest that it could be used to allocate treatment to those most at risk’. Nonetheless, using such a tool to inform allocation decisions is compatible with clinicians retaining ultimate responsibility, and there is evidence from other areas of medicine that predictive tools can augment clinical judgement when properly integrated. ^{Reference Thurtle, Jenkins, Pharoah and Gnanapragasam9}

Cockburn and Large are clearly concerned that algorithmic outputs will be routinely deferred to, leading to automation bias, clinician deskilling or complacency. These risks are genuine and warrant careful examination of how OxSATS should be embedded in clinical workflows, so that its predictions can be used productively in resource allocation decision-making to improve patient outcomes. However, assessing these risks requires empirical investigation of actual practice rather than presuming misuse in advance. The possibility that a tool might be used poorly is not sufficient reason to conclude that it should not be used at all.