Spending on mental health services varies substantially across countries; in Europe ranging from around 2.5 to 14% of total health expenditure (McDaid et al. Reference McDaid, Knapp and Medeiros2007). Psychiatric bed numbers have declined in most European countries in recent years (Knapp et al. Reference Knapp, McDaid, Medeiros, Becker, Johnson, Kilian, Salvador-Carulla, Simon and Tatar2008), but remain an important component of the mental health care system. The move towards community services has had some effects on the characteristics of inpatient patients; they have a higher level of severity of symptoms and behavioural problems, and the percentage of detained patients has increased. Consequently, it has been argued that the inpatient services can sometimes appear more custodial than therapeutic (Bowers, Reference Bowers2005).
Activities on wards have been key in improving the quality and outcomes of inpatient care. Lower quality-of-life has been found to be associated with inactivity (Kelly et al. Reference Kelly, McKenna, Parahoo and Dusoir2001), activity scheduling on inpatient wards has produced reductions in depression compared with psychotherapy (Hopko et al. Reference Hopko, Lejuez, LePage, Hopko and McNeil2003), and activity as a whole may reduce challenging behaviour (Fialko et al. Reference Fialko, Jabati, Gregory and Grey2005). This evidence has not been translated to clinical settings as shown in a review by Sharac et al. (Reference Sharac, McCrone, Sabes-Figuera, Csipke, Wood and Wykes2010). Low activity and social engagement for patients have remained stable in recent years and limited nursing time is spent in direct contact with patients (Sanson-Fisher et al. Reference Sanson-Fisher, Desmond Poole and Thompson1979; Sandford et al. Reference Sandford, Elzinga and Iversen1990) and even less providing therapeutic activities (Ryrie et al. Reference Ryrie, Agunbiade, Brannock and Maris-Shaw1998; Whittington & McLaughlin, Reference Whittington and McLaughlin2000; Bee et al. Reference Bee, Richards, Loftus, Baker, Bailey, Lovell, Woods and Cox2006). Furthermore, it has been shown that higher-grade nurses spend less of their time directly with patients (Cormack, Reference Cormack1976; Higgins et al. Reference Higgins, Hurst and Wistow1999; Bee et al. Reference Bee, Richards, Loftus, Baker, Bailey, Lovell, Woods and Cox2006).
A systematic and comprehensive evaluation of inpatient activities could support greater implementation, and assessments of cost-effectiveness are important. Economic evaluations need to collect information on the use of services associated with different models of care but with inpatient services this use is usually only recorded as the number of days spent in hospital. For evaluations of an inpatient intervention, it would be more helpful to identify all the care inputs received while on a ward. This distinction is important because, even if the length of stay is the same, the use of resources might be different between patients depending on activities attended and the care received – both of which may be associated with patient clinical and demographic characteristics.
The aim of this study was to describe the development and assess the reliability of a tool (the CITRINE) to achieve the objectives of collecting data on the care contacts and therapeutic activities of patients while on a psychiatric ward. This would potentially enable a more precise costing of inpatient psychiatric services.
Methods
Initial draft
In developing the tool, we conducted interviews with inpatient staff to discuss its content and structure. The professionals interviewed included two ward managers, three occupational therapists, one nurse and one ‘therapy and benefits coordinator’. All were working at psychiatric inpatient units in the South London and Maudsley (SLAM) NHS Foundation Trust. The interviews lasted for approximately 30 min and the main objective was to identify the group activities that take place on the wards and which professionals were most likely to have contacts with inpatients. Staff sometimes provided detailed schedules of patient activity on their wards. The first draft of the tool was then circulated for comments and suggestions to a group of experts including psychiatrists, psychologists and nurses.
Feasibility
Twenty-five service users from inpatient psychiatric wards were then interviewed using the questionnaire. The objective was to determine the acceptability of the tool, service users' opinion about difficulties in answering the questions and to evaluate how practical it was to collect data in this way. Two focus groups were conducted in parallel with this feasibility component. The first included eight occupational therapists, with seven nurses from an inpatient ward comprising the second. Participants were asked for their opinions about the tool and for any suggestions for improvements.
The process of consultation with staff resulted in the development of a tool that allows activities specific to an inpatient ward to be measured. The list-of-group activities were amended to include only the activities that had taken place in the previous week. Interviewers list the availability of activities (based on the ward calendar for the relevant week) and therefore each version of the tool is unique to that particular ward and time period. Initial questions on contacts with members of staff were shortened. The staff contacts section was also revised so that service users could indicate the name of professionals they have met even if they did not know or remember their job title. The tool is shown in the Appendix.
Reliability
This assessed the level of congruence between the information provided by service users using the final version of the tool and information that could be obtained from other sources. This took place in two stages. Initially, information on the activities attended by 41 service users over the preceding 7 days was obtained from the occupational therapist of each ward and the number of contacts with nursing staff members was collected from the patient's records for the same period. This information was also collected from the same patients and period using the tool.
Subsequent to this, information on the number and duration of staff contacts and activities attended by 22 other service users was collected over a 14-h (08:00–22:00 h) direct observation period. An adapted 1-day version of the CITRINE questionnaire was used to collect the relevant information from the same service users over this period. (This stage was added later and so the patients are different to the 41 previously included.)
Contacts with staff members were combined with unit costs measured in UK pounds for the financial year 2007/2008 from an established source (Curtis, Reference Curtis2008). In the first phase (n = 41), we assumed an average nurse contact of 15 min. Activity costs were calculated specifically for the study based on session duration, preparation required, staff involved and materials for each activity provided on each ward. These data were combined with unit costs of staff time (Curtis, Reference Curtis2008) and with information on the average number of service users attending each activity to obtain an estimated individual cost for each group activity. Activities were subsequently classified in categories according to their type and cost. (A list of all unit costs is available from the authors.) An average of these activity costs was used in the analyses presented. The significance of differences in reported and observed/recorded mean resource use was assessed using a paired t test and the agreement was further assessed using the concordance correlation coefficient.
Approval for the PERCEIVE study was provided by an appropriate NHS ethics committee and participants were asked to provide written consent.
Results
The CITRINE tool typically took 5–10 min to complete and most of the service users who participated in the feasibility study found it easy or very easy to answer the questions. Table 1 reports 7-day data from the CITRINE and from information covering the same period from occupational therapists and case notes for 41 service users. The mean age was 37 years and 19 of them were women. Most (68%) had a primary diagnostic of psychosis or bipolar disorder and the average length of stay at assessment was 43 days. However, there was great variation in this represented by an s.d. of 66 days.
Mean number and cost (£) of activities and nursing contacts reported by service users and from OTs/case notes
aTests of significance apply to both contacts/duration and cost.
Service users' report attending more activities than is contained in case notes (Table 1). Attaching an average unit cost of activities suggest a cost difference of £10 per person. Case notes also report more one-to-one nursing contacts, resulting in a cost difference of £4 per person. Although neither difference is large nor the discrepancy in the number of activities reported is statistically significant.
The comparison of data obtained from the 1-day observational study and the 1-day version of the tool show good congruence in terms of activities attended and psychiatrist contacts. However, the congruence in terms of contact with nurses and other staff is less good. The differences in these latter services were significant or of borderline significance. The concordance correlation coefficient for total costs from this part of the reliability study was 0.79. The cost implications for the difference in nursing contact is important given that the observational data suggest that this accounts for one-third of the total cost.
Discussion
The process of developing the CITRINE tool has involved input from a wide range of staff involved in providing care to those receiving psychiatric inpatient services (nursing staff, OTs, ward managers and psychologists). Furthermore, key input to the process has been provided by the recipients of this care. The tool collects data directly from service users. From a pragmatic perspective this is appropriate as it is the most practical way of obtaining the breadth of activity data that we require for economic studies. In addition, it emphasizes the service user's perspective in reporting activity. Of course, this could mean that activities that are entirely unmemorable will not be recalled or costed (e.g. conversations with nurses that could be very similar everyday such as ‘you have a telephone call’, ‘do you want to go for a walk?’, etc.). However, this type of very brief contact that is not remembered by service users is not the focus of the tool.
The result is a tool that has its objective in collecting information on the therapeutically relevant activities and staff contacts that take place on psychiatric wards and at the same time is acceptable to service users and relies on them as the main source of the relevant information. There have been other attempts to describe interventions and services provided in inpatient mental health settings, such as the International Classification of Mental Health Care (ICMHC) (de Jong, Reference de Jong2000). However, this instrument is a tool to classify services according to the type of care, rather than one that provides information on the specific services that patients receive.
The main function of the tool will be its use in economic studies, combined with information on relevant unit cost of the services and staff time. Therefore, its level of accuracy in reflecting what is actually taking place within psychiatric wards is paramount. The reliability study carried out offers some reflections on this issue. In particular, the questionnaire: (i) is a good source of information on the number of activities attended by service users and on contacts with psychiatrists, (ii) may provide acceptable information on interactions between service users and non-nursing care staff (OTs, psychologists, social workers, etc.) and (iii) may be more limited in recording time spent in service user–nursing staff interaction.
In relation to nursing staff contacts, there are some aspects that should be considered. First, this type of contact is the one that is common and consequently creates difficulties for accurate recall. Second, service users might report only contacts that they think are significant or meaningful for them. This classification is subjective and can result in some contacts being labelled as ‘non-contacts’ by service users. For example, a member of staff may have spent some time asking how a patient was, and would regard this as a contact, but it may not have been recognized as such by a patient if it was very brief or unwanted. This may have been the key reason for the discrepancy between the observed nurse–patient contacts and the patient-reported contacts. The mental health status of some of the service users may affect their ability to provide accurate information, although data on this are lacking. Although time spent with nursing staff needs to be measured correctly, the difficulty seems only to apply to one-to-one contacts. Contacts as part of organised activities are more readily measured and therefore the disagreement over total cost is limited.
The alternative sources of information on inpatient psychiatric activities and staff contacts are not free of problems. Registers and electronic databases are designed to support clinical care and not to record activities within wards. Furthermore, there might be intra- and inter-ward variability on the level of completion of these and on the accuracy of the information recorded. Observational data are an alternative but require too many resources to be a realistic when studies involve large number of individuals/wards with data collected at several time points.
There are limitations with this work. First, we did not conduct the full range of psychometric tests, although we did assess validity and reliability. It should be stressed that we do not regard this as a clinical instrument but rather a simple recording schedule to ascertain more accurately the use of care on inpatient wards. We anticipate the tool being adapted for different circumstances and extensive tests on one particular version would have limited usefulness. Second, and related, we did not assess inter-rater reliability. The content of the tool though is such that ambiguity of responses should be limited and hence inter-rate reliability is not as important as with a tool where interviewer interpretation is required. Third, the activities related to wards in one particular hospital. It may be that in another setting a more complex array of activities would present more of a challenge. The tool does, however, require the listing of specific activities at the start and so this problem is reduced. Fourth, the relatively poor concordance on contacts with nurses is a course for some concern. This has been discussed above, but the wording of the question may not be clear. Greater definition of what constitutes a contact may be required. Fifth, service users may not always have been clear about the profession of the person they had contact with. This is a problem for all research involving self-report data and one for which data on the extent to which this is a problem is required. However, definitions in the wording of the tool could be refined if necessary. Sixth, to rely on self-report data might be a limitation of the tool although a number of studies have suggested that patient recall of service use is acceptable (Calsyn et al. Reference Calsyn, Allen, Morse, Smith and Templehoff1993; Goldberg et al. Reference Goldberg, Seybolt and Lehman2002; Patel et al. Reference Patel, Rendu, Moran, Leese, Mann and Knapp2005). Asking for the duration of contacts may be difficult but these durations are meant to be approximations.
In conclusion, the CITRINE is a tool that, despite some limitations, provides adequate information on the activities that take place within psychiatric wards. Therefore, its use is recommended, alone or in combination with other sources, in economic analyses of inpatient care.
Declaration of Interest
This study presents independent research commissioned by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research scheme (RP-PG-0606-1050). The views expressed in this publication are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. We also acknowledge the financial support of the NIHR Biomedical Research Centre for Mental Health, South London and Maudsley NHS Foundation Trust/Institute of Psychiatry (King's College London).
Appendix. Survey of inpatient activities and services receipt
Ward: ___________________Patient ID number: ___________________
We are interested in finding out what activities patients have been to in the last 7 days. We would also like to find out what staff members patients have seen. Please use a copy of the ward's activity timetable if it would help remind you about activities you have participated in.
Section 1: Group activities on the ward
Please indicate what activities you have taken part in since last___________ by filling out the table below.
Section 2: Meeting with staff members
Please note any individual meetings you have had with any of the staff members listed below. You do not need to include times where other patients have been involved e.g. in a therapeutic group.
One-on-one time with nurses:
Please indicate if you have spent one-on-one time with a nurse since last_______ by filling out the table below:
Other professionals:
Please indicate if you have met with the following staff members since last______ by filling out the table below:
Please indicate if you have met with any other staff (not nurses) since last__________
e.g. pharmacist, chaplain, psychologist, hairdresser, dentist, physiotherapist, optician, chiropodist, advocate, solicitor
Thank you for taking part in our survey!
