Introduction
Violence is an important public health issue (Brundtland, Reference Brundtland2002; Krug et al. Reference Krug, Mercy, Dahlberg and Zwi2002). Violence is common in psychiatric inpatient units, and most common in forensic mental health settings, where almost half of inpatients have been violent. According to some estimates, a typical UK inpatient ward will experience more than three violent incidents every week with over 10% leading to serious injury and almost 30% moderate to severe injury (Bowers et al. Reference Bowers, Stewart, Papadopoulos, Dack, Ross, Khanom and Jeffery2011).
There is both a broad clinical and operational consensus that structured professional judgement (SPJ) offers a better predictor of violence risk than unstructured clinical opinion alone (Monahan et al. Reference Monahan, Steadman, Silver, Appelbaum, Clark Robbins, Mulvey, Roth, Grisso and Banks2001; Gardner et al. Reference Gardner, Lidz, Mulvey and Shaw1996; Ægisdóttir et al. Reference Ægisdóttir, White, Spengler, Maugherman, Anderson, Cook, Nichols, Lampropoulos, Walker, Cohen and Rush2006). Best practice guidelines for UK now recommend that every inpatient should be assessed using an evidence-based and validated SPJ instrument and have a risk management plan in place from the point of admission (National Institute for Mental Health in England, 2004).
Nevertheless, the SPJ process has its critics. Some authors have suggested that a large number of SPJ items are effectively redundant (Coid et al. Reference Coid, Yang, Ullrich, Zhang, Sizmur, Farrington and Rogers2011), or are better at identifying low-risk rather than high risk subjects (Fazel et al. Reference Fazel, Singh, Doll and Grann2012). However, it is accepted that many SPJ guides are clinically useful both at identifying risk and also by identifying evidence-based risk related treatment targets and risk management plans. This process now forms part of key national violence risk management guidance in the United Kingdom (National Collaborating Centre for Mental Health, 2009; Department of Health, 2007).
The most widely used SPJ violence risk guide in the UK and internationally is the Historical, Clinical, and Risk Management-20 (HCR-20) (Webster et al. Reference Webster, Douglas, Eaves and Hart1997; Singh, Reference Singh2013). It has established predictive validity (Yang et al. Reference Yang, Wong and Coid2010; Singh et al. Reference Singh, Grann and Fazel2011; Fazel et al. Reference Fazel, Singh, Doll and Grann2012; Campbell et al. Reference Campbell, French and Gendreau2009; O’Shea et al. Reference O’Shea, Mitchell, Picchioni and Dickens2013), and convergent validity with other tools used to predict aggression for mental health inpatients like the Violence Risk Scale (Langton et al. Reference Langton, Hogue, Daffern and Howells2009), the Violence Risk Scale Second Edition (Dolan and Fullam, Reference Dolan and Fullam2007), and the Violence Risk Appraisal Guide (Doyle et al. Reference Doyle, Dolan and McGovern2002). In the UK, the HCR-20 is advanced as the best choice for all secure psychiatric services (Daffern, Reference Daffern2007) and is the most widely used in forensic settings (Khiroya et al. Reference Khiroya, Weaver and Maden2009).
The HCR-20 manual (Webster et al. Reference Webster, Douglas, Eaves and Hart1997) recommends minimum standards for raters’ experience and qualifications as well as the information gathering and rating process. Over time in the UK, a number of commercial HCR-20 training programmes have developed to meet and maintain standards. In spite of this, we were unable to find any published data evaluating the quality of completed HCR-20 ratings and thus the validity of the violence risk management plans that arise from these ratings.
The only published structured guide to assess quality is the Competency Assessment Instrument for Violence (CAI-V). It was developed to evaluate psychiatry and psychology trainees’ competence in violence risk assessment and management (McNiel et al. Reference McNiel, Hung, Cramer, Hall and Binder2011). It was developed from work on violence risk assessment and management (Simon, Reference Simon2003, Reference Simon2008; Appelbaum & Gutheil, Reference Appelbaum and Gutheil2006), and the assessment of medical competencies (Andrews & Burruss, Reference Andrews and Burruss2004). It has high internal consistency, reliability, concurrent validity, inter-rater reliability and user acceptability (McNiel et al. Reference McNiel, Hung, Cramer, Hall and Binder2011). We used the CAI-V as the basis to develop an instrument to audit the quality of HCR-20 assessments within our low secure wards.
Method
This audit was carried out between July 2012 and July 2013 at St. Andrew’s Essex, a low secure service for male and female patients diagnosed with mental illness or personality disorder and who are deemed to pose a risk to themselves or others.
The HCR-20 Version 2 was completed and reviewed six monthly for violence risk assessment for every patient in the service. Each multidisciplinary team (MDT) had members trained in the HCR-20 and each HCR-20 had to be signed off either by a consultant psychiatrist or clinical psychologist.
The CAI-V was adapted to assess the quality of each completed HCR-20. It consisted of 20 items under the headings, interviewing and data collection, case formulation and presentation, and treatment planning (Table 1). An anchored scoring key was developed for each item ranging from 1 to 4 for each of the CAI-V categories: 1 indicated that the task was not done; 2 indicated below level of competence; 3 achieved competence; and 4 exceeded competence. The overall mean score was achieved by the summing all the scores dividing by the number of items scored, and then doubling that score. In the overall mean rating, a score of 1–2 was unacceptable, 3–4 working towards competency, 5–6 in the competent range and 7–8 in the advanced range.
Quality assessment of HCR-20, adapted from McNiel et al. (Reference McNiel, Hung, Cramer, Hall and Binder2011)
The audit proposal was approved by the St. Andrew’s Audit Group.
The auditors were trained in the use of the tool by the lead author (PS). Inter-rater reliability checks were carried out with the lead author. Each patient’s last HCR-20 was rated by two auditors in collaboration (SL and NC). Summary statistics were prepared and areas of service strengths and weaknesses were identified. The audit team developed an action plan to address the areas of weakness. The audit findings were fed back to the clinicians and performance measures and recommendations from the action plan explained.
A re-audit was carried out one year later using the same instrument, by two raters (SL and RRI) following appropriate training and inter-rater reliability checks. The results were fed back to the audit group and to the clinicians.
Results
A total of 51 adult patients in the service were identified in the first round of data collection in July 2012. There were 29 men and 22 women in the sample. All except two were in the 18–35 years age group. Of the men, 18 had a primary diagnosis of schizophrenia or schizo-affective disorder; 9 had a primary diagnosis of personality disorder; the rest had other diagnoses. Of the 22 women, 9 had a primary diagnosis of schizophrenia, schizo-affective or bipolar disorder; 9 had a diagnosis of personality disorder; and the rest had other diagnoses. More than 80% of the men and all the women were on a civil section of the Mental Health Act. They were referred from all parts of the country, but primarily from the east of England and London.
In the first round audit, 47 of 51 HCR-20s were rated overall as ‘Competent’ or ‘Advanced’. The results of the assessment are presented in Figures 1–4.
Mean of the individual raw scores for sources of information reviewed. Competency scores in most areas improved, apart from whether written records were reviewed. Items: 1, Obtains history relevant to risk of violence; 2, Considers findings from MSE pertaining to violence; 3, Considers review of records; 4, Considers obtaining collateral information from other professionals; 5, Considers obtaining collateral information from family/significant others.
Mean of the individual raw scores for information type included in Types of Information. Scores in all areas except ‘considered violent ideation’, improved. Items: 1, Considers current violent ideation, intent and/or plan; 2, Identifies any risk factors for violence; 3, Evaluates past risk factors for violence; 4, Evaluates present risk factors for violence; 5, Evaluates future risk factors for violence.
Mean of the individual raw scores for Case Formulation. At re-audit, scores for items 1, 4 and 5 improved but worsened for ‘explaining the rationale for decision making’ on the summary judgement, and ‘communicating estimate of violence’. Items: 1, Estimates the risk of violence; 2, Explains the rationale for decision making; 3, Communicates the estimate of violence risk clearly; 4, Specifies the time frame; 5, Gives an organised & accurate presentation of findings.
Mean of the individual raw scores for Treatment Planning. Treatment planning improved in all areas except for whether risk factors for violence were addressed in the case management section. However, all areas remained below the level required for competence. Items: 1, Addresses risk factors for violence; 2, Describes an intervention to minimise risk of violence; 3, Considers use of psychoactive medications; 4, Considers increasing the intensity of psychosocial treatment; 5, Considers involving family/significant others.
Areas that were identified for improvement included:
-
1. Collateral information from other professionals was obtained in only five assessments.
-
2. In no assessment was collateral information from family or significant others obtained.
-
3. Risk factors for violence were completed up to competent level in 49 of 51 cases, with the psychopathy item H7 being most commonly omitted.
-
4. The rationale supporting the summary judgement was provided in only 21 assessments.
-
5. Details of regular and PRN medication were provided in only two assessments.
-
6. Psychosocial intervention with information on nature of intervention was provided in 21 of the 51 assessments.
-
7. The treatment planning section on involving family/significant others was not completed in any of the assessments.
-
8. ‘Considers duty to protect’ with mention of possible or probable victim was completed in only 24 of 51 assessments.
The results of the audit and recommendations for service improvement were disseminated to key professionals within each MDT.
The second round of data collection was carried out on 37 patients in July 2013 (Figs 1–4). There were 20 men and 17 women. Of the men, 13 had a primary diagnosis of schizophrenia, schizoaffective disorder or bipolar disorder and 7 of personality disorder. Among the women, 6 had a primary diagnosis of schizophrenia or schizo-affective disorder, and 8 of personality disorder. Except for 5 men, they were all on civil sections of the Mental Health Act, mostly between 18 and 35 years of age. The numbers were lower as inpatient numbers had fallen due to temporary ward closures for refurbishment.
In the second round, 34 of 37 completed assessments were rated as competent or advanced.
Discussion
Within UK forensic psychiatric services, the HCR-20 is one of the most popular violence risk assessment guides, and yet, there are to our knowledge few published guidelines that assess the HCR-20 quality. The only published study describing the effect of training on the quality of HCR-20 assessments within a forensic service described the measure of quality utilised in the study as ‘crude’ (Reynolds & Miles, Reference Reynolds and Miles2009).
This audit represents the first attempt to develop a structured systematic measure of risk assessment quality based on a previously validated teaching assessment tool, the CAI-V. There were, however, some methodological weaknesses. The audit tool was modified from a tool developed for a slightly different purpose, to assess competency of a training programme. However, quality needs to be assessed for a competency assessment, and there are currently no gold standard tools available for HCR-20 quality assessment. Also, the study had different reviewers for the first and second audit, though one reviewer (SL) was common to both. To address this issue, the lead author (PS) provided extensive training in the use of the tool to the reviewers and also carried out an inter-rater reliability assessment, using the same patient for the first reviewing team, as well as the reviewers SL and RRI separately for the second audit. A correlation of 100% was achieved for the overall mean ratings in both audits, though there were some minor variations in the individual item ratings. Other shortcomings include the relatively small sample size and the relatively specialist nature of the study population, though the high number of patients on civil sections suggests that the study group was not exclusively forensic in nature. The last shortcoming would be the raters being part of the staff in the hospital, and thus potentially not completely blind to the histories of the patients.
Despite these shortcomings, this audit highlighted local areas of weaknesses in the HCR-20 assessment process, which led to the following improvements:
-
1. Collateral information collected from other professionals, family members or other sources was often very limited or omitted completely. In the re-audit, ratings improved, but information from family and significant others remained poor. Patients often refused consent for professionals to talk to family members or access other information sources. Other patients are simply disenfranchised from their families and wider societal networks. Nevertheless, this failure to adequately seek out and interrogate third party information sources is unsatisfactory. Clinicians should be aware that they sometimes need to over-ride patient consent on grounds of public safety, especially when they are trying to collect collateral information rather than pass on confidential information.
-
2. Ratings of the individual HCR-20 items were all done up to a competent level, except for the completion of the psychopathy item, which was omitted in a large number of cases. The most likely cause for this was the lack of staff training in the use of the Hare Pyschopathy Checklist (PCL:SV or PCL-R) item H7, which is mandated for this item. One solution was to develop a pool of clinicians trained in the PCL:SV or PCL-R across the hospital site to whom patients could be referred for rating. However, this would be a slow rate-limiting step for all patients. Another solution would be to ensure that every clinical team has at least one trained member of staff. However, version 3 of the HCR-20 includes the assessment of psychopathy within H7, and pays more attention to traits of psychopathic personality as opposed to a diagnosis of psychopathy based on PCL-R or PCL:SV (Douglas et al. Reference Douglas, Hart, Webster and Belfrage2013). This takes away the need for PCL-R or PCL:SV training for all MDTs. It is, of course, entirely possible that there were other reasons for the omission, like clinicians’ own doubts about the validity of psychopathy as a risk-predictive item.
-
3. With regards to case formulation, a key weakness was the failure to document the reasons and information supporting risk assessment and management decisions around the level of risk. In the first audit round, this item was rated close to competency but fell lower in the re-audit. It is possible that changes to key personnel within the Psychology Department between the two audit ratings might have contributed. A solution to this is the dissemination of the audit tool to key members of all teams, particularly psychiatrists and psychologists, so that even with change of staff, there is awareness of the quality requirements for HCR-20. The audit helped to emphasise that there is value in developing a quality assessment tool, as there were shortcomings in the use of the HCR-20 in the real-world setting, even though the manual was available for all to use.
-
4. With regards to treatment and risk management planning, there was evidence of some improvement between the two audits. Risk management strategies included medication type, its route of administration, use of verbal de-escalation, use of family and friends and attention to victim safety considerations. A possible reason for this was the relative lack of involvement of a full MDT when completing the risk assessment tool. This meant that the risk scenario was not thoroughly discussed with details of medical, psychological or social interventions considered thoroughly by the team, which was reflected in the management plan lacking sufficient detail.
Patient involvement in the risk assessment process and in formulating risk management plans was almost universally absent. Ideally, risk formulation and risk management planning should be done in partnership with the patient (Wilson, Reference Wilson2011; Haque & Webster, Reference Haque and Webster2013). This helps to facilitate an effective therapeutic alliance (De Vries Robbé et al. Reference De Vries Robbé, de Vogel and Stam2012), to predict outcome across treatments (De Rubeis et al. Reference DeRubies, Brotman and Gibbons2005), and to ensure that interventions are appropriate and agreed by the patient. This element needs to be assessed in any future audit tool. We have developed an HCR-20 version 3 training programme, which incorporates the action points from this audit for dissemination amongst all charity professionals. Interviewing the patient specifically for the HCR-20 and including the patient’s views is a key part of the HCR-20 version 3 training process. Each service will also use a version of the audit tool developed for this study, modified from the CAI-V, to audit the quality of HCR-20, which is meant to be completed for all patients across the charity, unless there are specific exclusion criteria.
This was the first study that attempted to develop an HCR-20 quality assessment guide based on published literature. It is important for every service not only to complete SPJ risk assessments, but also to determine and measure the quality with which they are being completed, and adapt practice accordingly. Annual audits of violence risk assessment processes should be mandatory for every service. It is important that the tool used for such audits is appropriately adapted to the requirements of the service and ratified by the clinical governance or risk leads within the organisation.
Financial support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Conflict of interest
None.
