Response

Artificial intelligence is unavoidable in the current discourse about the future of psychiatry, with pundits expressing emotions ranging from fear to wonder. For some, artificial intelligence is an existential threat to our profession, envisaging a dystopian future where psychiatrists are replaced wholesale by faceless chatbots. Others are more optimistic, seeing an opportunity to evolve, using artificial intelligence to free us from more mundane tasks and focus on what we do best: building interpersonal relationships and managing complexity. ^{Reference Armitage1}

A core role of the Royal College of Psychiatrists is to set educational standards for the profession. One key mechanism to achieve this is through the college’s membership examinations (MRCPsych). These rigorous tests set a high bar for those aspiring to independent practice in the field, and achieving membership is rightly seen as a mark of quality. The format of the examinations has become increasingly standardised over the years, with the written components currently composed of multiple-choice questions and extended matching items. ^{Reference Ray, Bhardwaj, Malik, Singh and Gupta2}

I read with interest the study of the performance of four leading large language models (LLM) in answering 21 written questions from the MRCPsych examinations, in which Armitage found that these artificial intelligence-powered tools universally obtained high scores. ³ Although only a small sample of questions was tested, if these results were extrapolated to full-length papers, then the LLMs would comfortably pass. Furthermore, Armitage notes that the few incorrect answers were not the result of faulty reasoning, rather of factual errors.

So, what are the implications of such an impressive display by the LLMs? First, we should perhaps consider the narrower consequences for the MRCPsych examination itself. Certainly, this result reinforces the importance of preventing candidates from accessing LLMs during the examination, to maintain the integrity of the assessment as a true reflection of a candidate’s knowledge and reasoning abilities. The fact that artificial intelligence can score so highly does not, however, diminish the credibility or utility of the test itself. The acquisition of a sound knowledge base and the ability to apply it to clinical practice remain fundamental to the practice of psychiatry. The MRCPsych written papers, while imperfect, still offer a largely fair way of determining if candidates meet the required threshold in this regard. ^{Reference Scheeres, Agrawal, Ewen and Hall4}

It is easily conceivable that an artificial intelligence entity, suitably embodied, could also soon pass the Clinical Assessment of Skills and Competencies (CASC) examination, which requires candidates to complete specific tasks within a short timeframe, while demonstrating empathy and excellent communication skills. Although undeniably a step up in complexity from the written examinations, the CASC has been criticised for being unrepresentative of real practice – essentially, a piece of theatre that requires a good performance rather than genuine care or deep critical thinking. ^{Reference Kashyap and Sule5} The CASC’s task-focused stations thus seem a natural fit for AI’s solution-based approach.

A broader question raised by this study is whether LLMs could help psychiatrists in their clinical decision-making and psychiatrists should accept this help. ^{Reference McCradden, Hui and Buchman6} The speed with which the capabilities of LLMs are increasing can make it challenging to foresee what might ultimately be possible in this regard. It is premature to make conclusive inferences about the wider impact of artificial intelligence on clinical decision-making just based on the results of this study. MRCPsych written questions are highly specific, usually focusing more on scientific knowledge than coherent clinical reasoning. Armitage concludes that, due to their unfounded confidence in incorrect answers that such models should ‘not replace, but rather augment and strengthen, clinician decision making’.

Armitage recognises that real-world psychiatric practice is much messier than a neatly packaged exam question. Could the future psychiatrist’s relationship to AI therefore be to ensure that LLMs receive the correct inputs, and then to sensitively interpret and implement the outputs? This model would seem to offer some form of compromise whereby human and machine can operate in partnership, bringing complimentary abilities to bear on solving patients’ problems. LLMs could scour reams of research data and clinical guidelines in an instant, digest this vast library of information and produce personalised summaries to support psychiatrists to make the best decision for each individual patient. Medication choices for example could be informed by detailed clinical history, deep phenotyping, patient preference and the latest evidence, to produce a truly informed and tailored therapy. ^{Reference Johnson, Wei, Weeraratne, Frisse, Misulis and Rhee7}

The rapid development of artificial intelligence raises many further important questions. Is it enough that the physician, real or artificial, chooses the right treatment, or does the therapeutic relationship also need warmth and connection? Can this sense of connection be experienced by patients interacting with an artificial entity? Will we reach a point where AI will experience this sense of empathy itself, not just act as if it does? Transparency will be fundamental if we are to retain the trust of patients as we navigate these uncharted waters. Understandably, people who believe they are interacting with a fellow human but later find this is not the case may experience a sense of betrayal. Many, however, may be happy to interact with artificial intelligence if this is clear from the start. Some may even prefer the anonymity and the opportunity to unburden themselves to an artificial entity. ^{Reference Olawade, Wada, Odetayo, David-Olawade, Asaolu and Eberhardt8}

At the heart of many psychiatrists’ ambivalence towards artificial intelligence is perhaps the ancient guild anxiety that the profession we have pledged ourselves to, and worked so hard to join, may be swept away in the name of progress. The profession has already witnessed many changes to the role of the psychiatrist in recent decades, with increasingly flattened hierarchies and greater distribution of powers to other disciplines. ^{Reference Fiorillo, Volpe, Bhugra, Fiorillo, Volpe and Bhugra9} The speed with which artificial intelligence is evolving makes these questions of professional identity increasingly urgent. While we cannot predict the exact nature of these developments, we must remain alert and adaptable if we are to shape the new order in our best interest and those of our patients. To paraphrase Bob Dylan – the times they are a-changin’, so we’d better start swimmin’!