Subject enrolment

Admissions were sampled from acute psychiatric in-patient facilities at three sites: Western Psychiatric Institute and Clinic (Pittsburgh, PA); Western Missouri Mental Health Center (Kansas City, MO); Worcester State Hospital and the University of Massachusetts Medical Center (Worcester, MA). Selection criteria for research subjects were: civil admissions; between the ages of 18 and 40 years; English-speaking; White or African American ethnicity (or Hispanic in Worcester only); and a chart diagnosis of schizophrenia, schizophreniform disorder, schizoaffective disorder, depression, dysthymia, mania, brief reactive psychosis, delusional disorder, alcohol or drug abuse or dependence, or a personality disorder. After complete description of the study to the subjects, written informed consent was obtained.

Sample description

We approached a quota sample (to ensure representativeness across sites on gender, race, and age) of 1695 to participate. The refusal rate was 29% (n=492). The final sample given a hospital interview was 1136. Differences between the eligible admissions and the follow-up sample (n=939) are discussed in detail elsewhere (Reference Steadman, Mulvey and MonahanSteadman et al, 1998). Males comprised 57.3% of the sample. Ethnically, 68.7% of the sample was White, 29.1% African American and 2.2% Hispanic. The mean age was 29.9 (s.d.=6.2) years. Depression was the most frequent primary research diagnosis on the DSM-III-R Checklist (Reference Janca and HelzerJanca & Helzer, 1990; 41.9%), followed by alcohol/drug abuse or dependence (21.8%), schizophrenia (17.0%), bipolar disorder (14.1%), personality disorder only (2.1%) and other psychotic disorder (3.1%). The proportion of all cases with a primary research diagnosis of major mental disorder that had a co-occurring diagnosis of substance abuse or dependence was as follows: depression, 49.6%; schizophrenia, 41%; bipolar disorder, 37.7%; and other psychotic disorder, 45%.

Hospital data collection

Hospital data collection was conducted in two parts: an interview by a research interviewer to obtain data on risk factors and violence; and an interview by a research clinician (PhD or MA/MSW in psychology or social work) to confirm the chart diagnosis using the DSM-III-R Checklist and to administer several clinical instruments.

The hospital data set assembled in the MacArthur Violence Risk Assessment Study consisted of 134 risk factors from four conceptual domains: dispositional or personal factors (e.g. age); historical or developmental factors (e.g. child abuse); contextual or situational factors (e.g. social networks); and clinical or symptom factors (e.g. delusions) (Reference Steadman, Monahan, Appelbaum, Monahan and SteadmanSteadman et al, 1994). For the present analysis, we eliminated 28 risk factors that would be the most difficult to obtain in clinical practice, restricting ourselves to the remaining 106. Two criteria were used to eliminate risk factors. The first was to eliminate information generally unavailable to mental health personnel in the context of brief hospitalisation (e.g. information in official arrest records, in distinction to self-report of prior arrests). The second was to eliminate information that required the administration of a lengthy (>12-item) instrument to obtain (e.g. a social network inventory (Reference Estroff, Zimmer, Monahan and SteadmanEstroff & Zimmer, 1994)). A list of all 134 risk factors, with their bivariate correlations with violence and with an indication of which were eliminated from these analyses, is provided in Table 1.

Table 1

Risk factors and bivariate correlations with violence in the first two follow-ups

Domain	Reference	Pearson coefficient R
Personal
Gender: male		0.08*
Age		-0.07*
Race: White		-0.12***
Verbal IQ1		-0.11**
Ever married		0.01
Hare Psychopathy Cheklist: Screening Version >121	Reference Hart, Cox and HareHart et al, 1995	0.26***
Novaco anger: behaviour	Reference Novaco, Monahan and SteadmanNovaco, 1994	0.16***
Novaco anger: cognitive		0.11*
Novaco anger: arousal		0.09**
Novaco anger: intensity		0.08*
Barratt impulsiveness: motor	Reference Barratt, Monahan and SteadmanBarratt, 1994	0.07*
Barratt impulsiveness: non-planning		0.05
Barratt impulsiveness: cognitive		0.05
Historical
Years of education		-0.11***
Socio-economic status1	Reference Hollingshead and RedlichHollingshead & Redlich, 1958	0.05
Employed		-0.05
Age at first hospitalisation		-0.04
No. of prior hospitalisations		-0.03
Involuntary legal status		0.11**
Recent violent behaviour		0.14***
Adult arrest: seriousness		0.25***
Adult arrest: frequency		0.24***
Any arrest: person crime1	Official Report	0.13***
Any arrest: other crime1	Official Report	0.11***
Sexually abused before age 20		-0.03
Seriousness of abuse as child		0.14***
Frequency of abuse as child		0.12***
Father ever used drugs		0.16***
Father ever arrested		0.15***
Father ever excess drinking		0.11**
Father ever admitted to psychiatric hospital		0.02
Lived with father to age 15		-0.09**
Mother ever used drugs		0.05
Mother ever arrested		0.05
Mother ever excess drinking		0.06
Mother ever admitted to psychiatric hospital		-0.02
Lived with mother to age 15		-0.06
Parents ever fought with each other		0.06
Parents ever fought with others		0.03
Any head injury: loss of consciousness		0.10**
Any head injury: no loss of consciousness		0.06
Self-harm thoughts		0.02
Self-harm attempt		-0.03
Attempt to kill self		0.01
Contextual
Living in private residence		-0.05
Homeless		0.05
Living alone		-0.07*
Perceived stress1	Reference Cohen, Kamarck and MermelsteinCohen et al, 1983	0.08*
Social networks	Reference Estroff, Zimmer, Monahan and SteadmanEstroff & Zimmer, 1994
No. of people in social network1		-0.02
% Mental health professionals in social network1		-0.10**
% Family in social network1		0.01
No. of negative persons in social network1		0.07*
No. of positive and material supporters1		-0.07*
Average no. of mentions per negative person1		0.06
Average no. of mentions per positive material person1		-0.03
Frequency of social network contact1		-0.03
Duration of social network contact1		0.02
Clinical
Chart antisocial personality disorder		0.19***
DSM-III-R Checklist	Reference Janca and HelzerJanca & Helzer, 1990
Major disorder, no substance		-0.19***
Major disorder and substance		0.08*
Substance, no major disorder		0.15***
Drug or alcohol		0.18***
Drug		0.17***
Alcohol		0.14***
Schizophrenia		-0.12***
Mania		-0.04
Depression		-0.02
Other psychosis		0.00
Personality disorder only		0.02
Brief Psychiatric Rating Scale	Reference OverallOverall, 1988
Total score1		-0.04
Activation sub-scale1		-0.08*
Hostility sub-scale1		0.08*
Anergia sub-scale1		-0.07*
Thought disturbance sub-scale1		-0.06*
Anxiety/depression sub-scale1		0.01
Global Assessment of Functioning	American Psychiatric Association, 1989	-0.05
Activities of daily living		-0.01
Delusions	Reference Appelbaum, Robbins and RothAppelbaum et al, 1999
Any delusions1		-0.06
Persecutory		-0.07*
Grandiose		-0.01
Body/mind control		-0.09**
Thought broadcasting		-0.05
Religious		-0.08*
Jealousy		-0.02
Guilt		-0.03
Somatic		-0.03
Influence on others		-0.03
Threat/control-override		-0.10**
Other		-0.04
Violent fantasies	Reference Grisso, Davis and VesselinovGrisso et al, 2000
Any		0.13***
Frequent		0.13***
Recent onset		0.07*
Same target		0.03
Focus same person		0.10**
Escalating harm		0.13***
While with target		0.12***
Frequent, not escalating, not with target		-0.01
Frequent, escalating, not with target		0.09**
Frequent, not escalating, with target		0.08*
Frequent, escalating, with target		0.10**
Not frequent, not escalating, not with target		0.13***
Any hallucinations		0.02
Command hallucinations		0.06
Present at time of admission	Record Review
Substance abuse		0.14***
Paranoia		-0.09**
Delusions		-0.09**
Decompensation		-0.09**
Violence		0.09**
Hallucinations		-0.07*
Bizarre behaviour		-0.07*
Medication non-adherence		-0.07*
Aggressive (non-violent)		0.06
Anxiety		-0.05
Suicide attempt		0.05
Mania		-0.04
Personal problems		0.03
Evaluation		0.03
Other		-0.03
Medication change		-0.02
Unable to care for self		0.02
Suicide threat		-0.01
Property damage		-0.01
Court order		-0.01
Depression		-0.003
Drug use
Any drug		0.12***
Cocaine		0.11**
Alcohol		0.10**
Other		0.08*
Marijuana		0.04
Stimulants		0.04
Sedatives		0.03
Opiates		0.04
Mini-Mental State1	Reference Folstein, Folstein and McHughFolstein et al, 1975	0.02
Perceived coercion at admission1	Gardner et al, 1993	0.03

Community data collection

Twenty weeks after hospital discharge was chosen as the time frame for the analysis here because this was the period during which the prevalence of violence by patients in the community was at its highest (Reference Steadman, Mulvey and MonahanSteadman et al, 1998). Research interviewers attempted two follow-up interviews with enrolled patients in the community during this period, approximately 10 weeks apart. A collateral informant who knew of the patient's behaviour in the community during the follow-up period - usually, but not always, a family member - was also interviewed on the same schedule. Arrest and re-hospitalisation records provided the third source of information about the patients' behaviour in the community.

Patients and collaterals independently were asked whether the patient had been involved in any of several categories of violent behaviour in the past 10 weeks (Reference Lidz, Mulvey and GardnerLidz et al, 1993). Only the most serious act for each discrete incident was coded. Violence to others was defined to include the following: acts of battery that resulted in physical injury; sexual assaults; assaultive acts that involved the use of a weapon; or threats made with a weapon in hand. (Battery that did not result in injury was defined as ‘other aggressive act’ (Reference Steadman, Mulvey and MonahanSteadman et al, 1998) and is not considered in the analyses reported here.) Violence reported by any of the three data sources - subject self-report, collateral report, or official records - was reviewed by a team of trained coders. Ethical and legal issues encountered in conducting this research are discussed elsewhere (Reference Monahan, Appelbaum and MulveyMonahan et al, 1994).

Developing the classification tree

To develop the ICT model, we used CHAID (chi-squared automatic interaction detector) software (SPSS, 1993). Specifically, the CHAID algorithm was used to assess the statistical significance of the bivariate association between each of the 106 eligible risk factors and the dichotomous outcome measure - violence in the community - until the most statistically significant value of χ² was identified, with P < 0.05 a necessary condition for risk factor selection. Once a risk factor was selected, the sample was partitioned according to the values of that risk factor. This selection procedure was then repeated for each of the sample partitions, thus further partitioning the sample. The result of the partitioning process was to identify groups of cases that shared the same risk factors and that also shared the same values on the outcome measure of violence.

Iterating the classification tree

We then extended this recursive partitioning approach in an iterative fashion. That is, all subjects not classified into groups designated as either high risk or low risk in the first iteration of CHAID were pooled together and re-analysed in a second iteration of CHAID. This iterative process continued until it was not possible to classify any additional groups of subjects as either high or low risk (with no group allowed to contain fewer than 50 cases).

Choosing two cut-offs

The choice of cut-off scores for high-risk and low-risk categories must be made in the context of legal or policy values external to the methodology chosen for assessing risk. Here, for illustrative purposes, we defined any group of patients with a rate of violence that was less than half the base prevalence rate of the total sample, as in the low-risk category, and any group of patients whose rate of violence was greater than twice the base prevalence rate of the total sample, as in the high-risk category. Because the base prevalence rate of violence during the first 20 weeks after hospital discharge for the total sample was 18.7% (i.e. 18.7% of the patients committed at least one violent act during either the first or second 10-week follow-up period), this meant that the cut-off for the low-risk category was 9% violent and the cut-off for the high-risk category was 37% violent.

The ICT contained three iterations (Fig. 1). In the first iteration, the tree classified 429 of the 939 subjects (45.7%) into either the high- or the low-risk categories. In the second iteration, the tree classified as high- or low-risk 167 (32.7%) of the 510 subjects who were not classified into either high- or low-risk groups at the end of iteration 1. In the third iteration, the tree classified as high- or low-risk 86 of the 343 subjects (25.1%) who were unclassified at the end of iteration 2. At the end of iteration 3, no further groups could be classified as high- or low-risk, given the parameters of the model we had set (e.g. no group with fewer than 50 cases); 257 subjects (27.4% of the total sample) remained unclassified. The final ICT contained 15 contingent risk factors that formed 11 risk groups (four low-risk groups, accounting for 50.9% of the total sample; three high-risk groups, accounting for 21.7% of the total sample; and four unclassified risk groups, accounting for 27.4% of the sample).

Fig. 1

Clinically useful iterative classification tree. – – –, Low violence risk; ——, high violence risk; · · · ·, unclassified.

The risk factors displayed in Fig. 1 are defined as follows. Seriousness of prior arrests was a patient's self-report of the seriousness of arrests since age 15 years. Motor impulsiveness was measured from the motor sub-scale of the Barratt Impulsiveness Scale (Reference Barratt, Monahan and SteadmanBarratt, 1994). Father used drugs was a self-report question on whether the patient's father ever used drugs excessively. Recent violent fantasies was measured by the Schedule of Imagined Violence (Reference Grisso, Davis and VesselinovGrisso et al, 2000). Major disorder without substance abuse refers to a diagnosis of any major mental disorder without any co-occurring substance abuse diagnosis, as reached by research clinicians using the DSM-III-R Checklist. Legal status was the initial status for the baseline hospitalisation, as recorded in hospital admission records. Schizophrenia was the diagnosis reached by research clinicians using the DSM-III-R Checklist. Anger reaction was measured by a short version of the Behavioural Subscale of the Novaco Anger Scale (Reference Novaco, Monahan and SteadmanNovaco, 1994). Employed was a self-report question regarding the patient's paid full- or part-time employment status in the two months prior to hospital admission. Recent violence was a self-report of violence in the two months prior to hospital admission. Loss of consciousness referred to a self-report of any loss of consciousness due to head injury. Parents fought was a self-report by the patient that his or her parents engaged in physical fights with one another when the patient was growing up. (A complete list of the questions comprising these risk factors is available from the first author upon request.)

Receiver operating characteristic

To assess the predictive accuracy of the actuarial model produced by this method and to facilitate further comparisons of our results with other research on violence risk assessment, we used a receiver operating characteristic (ROC) analysis (Reference Gardner, Lidz and MulveyGardner et al, 1996; Reference Quinsey, Harris and RiceQuinsey et al, 1998). The statistic used to summarise the analysis is the area under the ROC curve, which corresponds to the probability that a randomly selected violent patient will have been assessed by the risk assessment tool as higher risk than a randomly selected non-violent patient (Reference SwetsSwets, 1988). The area under the ROC curve for the 11 risk groups presented in Fig. 1 is 0.80 (P<0.001). The distribution of cases that were violent or not violent during the follow-up as a function of the low- and high-risk cut-offs used to generate the ICT is presented in Table 2.

Table 2

Distribution of violent and non-violent cases using two thresholds

Observed	Classification
	Low <9%	Unclassified	High >37%	Total
Not violent	456	195	112	763
Violent	22	62	92	176
Total	478	257	204	939

Bootstrapping

We did not cross-validate the ICT. Cross-validation of a risk assessment model requires estimating the model on a subset of the data and validating the model on the rest. As noted by Gardner et al (Reference Gardner, Lidz and Mulvey1996), however, cross-validation “wastes information that ought to be used estimating the model” (p 43). For this reason, bootstrapping (Reference Mooney and DuvalMooney & Duval, 1993) has become a widely used analytical strategy for estimating the shrinkage to be expected when a model is generalised to a sample other than the one on which it was estimated. In conducting such an analysis, 1000 bootstrapped samples were drawn from the original data set. Table 3 presents the 95% confidence intervals for each of the 11 risk groups in the ICT, in order of decreasing risk. The ranges of these intervals indicate how the ICT is likely to perform on other similar samples.

Table 3

Bootstrapped 95% confidence intervals for the ICT risk groups

Risk group	% violent in risk group	95% confidence interval
B	52.7	41.0-63.8
F	41.8	31.3-52.5
A	39.2	26.2-52.4
G	34.4	22.5-46.1
J	28.2	20.6-35.8
I	18.9	8.2-29.4
K	10.9	3.6-18.2
C	7.4	1.6-13.2
E	6.8	1.3-12.1
H	4.7	0.2-8.8
D	2.7	0.5-4.9

Clinical illustrations

Illustrating the use of the ICT may be helpful. A clinician evaluating a patient's risk of violence using the ICT presented in Fig. 1. would first ask the patient about the seriousness of his or her prior arrests. If the patient stated that he or she had previously been arrested for a violent crime, the clinician would then inquire into whether the patient recently had been fantasising about being violent. If the patient responded affirmatively to this second question, he or she at that point would be placed in the high violence risk category. More specifically, the patient would be placed in risk group B, a group in which approximately 53% of the patients are expected to commit a violent act in the next several months.

If, on the other hand, the patient denied having violent fantasies, the clinician would then indicate whether the patient had a diagnosis of schizophrenia. If the patient did have such a diagnosis, he or she at that point would be placed in the low violence risk category. More specifically, the patient would be placed in risk group E, a group in which approximately 7% of the patients are expected to commit a violent act in the next several months. (For other studies finding rates of violence to be lower among patients with schizophrenia than among patients with other, primarily personality disorder, diagnoses, see: Reference Gardner, Lidz and MulveyGardner et al, 1996; Reference Quinsey, Harris and RiceQuinsey et al, 1998; Reference Wallace, Mullen and BurgessWallace et al, 1998.)

Comparative predictive accuracy

We have demonstrated here that the ICT method may be adapted for clinical use. The method does not require unavailable or costly-to-gather data in order to characterise the risk of violence. Rather, risk factors usually found in patient files, or capable of routine assessment, are all that are required for the ICT to function. The predictive accuracy of the ICT using a reduced set of 106 clinically feasible risk factors from the MacArthur Violence Risk Assessment Study (an area under the ROC curve of 0.80) is comparable to the predictive accuracy that we reported (Reference Steadman, Silver and MonahanSteadman et al, 2000) for risk assessment using the expanded set of 134 risk factors (an area under the ROC curve of 0.82).

Violence risk assessment software

Although the contingent nature of the risk factors identified in Fig. 1 may appear too intricate for use in clinical practice, the utility of the ICT model would be enhanced greatly with the aid of software. Software would facilitate the assessment of an individual patient by guiding the clinician to ask only those questions required to assess risk. We are in the process of developing such software.

CLINICAL IMPLICATIONS

• ▪ The classification tree approach to violence risk assessment proposed here improves on traditional approaches by explicitly acknowledging that violence is an outcome reached by multiple routes.

• ▪ Employing two cut-off scores - one for identifying high-risk cases and one for identifying low-risk cases - is more realistic than attempting to classify all patients as to the risk of violence.

• ▪ A clinically useful actuarial approach for assessing the risk of violence among acute psychiatric patients now exists.

LIMITATIONS

• ▪ This is a clinical study of violence among people hospitalised for mental disorders, not an epidemiological study of violence among people with mental disorders in the general population.

• ▪ The extent to which the accuracy of the actuarial tool developed here generalises to other types of clinical setting (e.g. forensic hospitals) is unknown.

• ▪ The proposed tool is specifically designed for assessing the risk of violence; efforts to manage the risk of violence will require additional data.