Introduction
Psychiatric discharge summaries provide a vital source of information to general practitioners (GPs) and community mental health professionals regarding the treatment of their patients during a hospital stay and guidance on long-term management plans. There has been a move in recent years to improve communication, and this has led to the introduction of electronic software for the writing of discharge summaries as part of a national movement in the United Kingdom towards electronic records. Electronic records have the potential to improve efficiency and the quality of patient care; however, this has not always been reliably demonstrated in previous systematic reviews (Black et al. Reference Black, Car, Pagliari, Anandan, Cresswell, Bokun and Sheikh2011). There are many barriers to their introduction, particularly technical, social and financial hurdles (Boonstra & Broekhuis, Reference Boonstra and Broekhuis2010).
One potential benefit is a reduction in the delay in information reaching other professionals (O’Leary et al. Reference O’Leary, Liebovitz, Feinglass, Liss, Evans, Kulkarni and Baker2009) and, as such, an improved link between primary and secondary care (Scullard et al. Reference Scullard, Iqbal, White, Olla and Thomson2007).We know that in our trust, following the introduction of electronic discharge (e-discharge) summaries, ~99% of discharge summaries are now sent within 24 hours of discharge. However, there may be other factors influencing the quality of e-discharge summaries, such as the design and the flexibility of the software used (e.g. the amount of text that can be entered into the different sections) or the IT skills of the doctors using them. Consequently, the quality of the information contained in these summaries may suffer at the price of improved efficiency.
We undertook a service evaluation of the discharge summary process before and after the introduction of an e-discharge summary system at the Bradgate Mental Health Unit, Leicester. The aim was to evaluate the quality of these discharge summaries in order to assess whether there had been any improvement.
Methods
The e-discharge system used by the trust was structured by means of a pro forma. It had the following word capped sections: admission details, clinical interventions, progress notes, pharmacy notes, signs of relapse, risks on discharge, aftercare arrangement, GP actions, on/not on Care Programme Approach, diagnosis and further information. The further information section is usually used for background history such as family history, personal history, past psychiatric history, substance abuse history, forensic history, etc. The e-discharge summaries are typed directly by the junior doctors. Before the introduction of e-discharge systems, discharge letters were dictated by junior doctors and typed by medical secretaries. There was no universal template for the dictated letter, but they generally followed the format of a psychiatric history and mental state examination with sub-headings for ‘reason for admission’, ‘past psychiatric history’, ‘family history’, ‘drug history’, etc.
A list of all admissions to Bradgate Mental Health Unit in Leicester in the first 6 months of 2010 (when discharge summaries were dictated and typed) and the first 6 months of 2012 (when e-discharge summaries were used) was obtained from the trust’s electronic system Maracis (version 3.3.2.185, Maracis Solutions Ltd, UK). These admissions were then numbered, and using an online random number generator (www.random.org) 50 discharges from each period were selected. The discharge summaries were printed and evaluated against 15 standards (Table 1). These standards were adapted from Crossan et al. (Reference Crossan, Curtis and Ong2004) audit of psychiatric discharge summaries with the inclusion of local standards from the Leicestershire Partnership Trust’s guidelines for discharge summaries. The standards assumed that all 15 items are present in every discharge summary.
Proportion of discharge summaries meeting each of the 15 standards
*χ 2 test in all indicators apart from initial reason for admission, clinical summary of treatment, diagnosis, medication on discharge, follow-up arrangements and information provided to service user. In these cases, Fisher’s exact test was used.
The bold values are considered statistically significant.
Each discharge summary was examined by two independent reviewers from a group of four (three core trainees and one foundation year one doctor). Each discharge summary was scored separately and then discussed by the two reviewers to reach a consensus score when there were differences. This consensus score for each standard was used for the comparisons, but the original scores were kept to measure inter-rater reliability. Each standard was given a score of 0 (absent) or 1 (present). This scoring method is similar to the one used by Scullard et al. (Reference Scullard, Iqbal, White, Olla and Thomson2007). For each discharge summary, the sum of the scores of all standards was calculated as the total scores. A higher total score means more standards were met. This project was approved by the audit department of the Leicestershire Partnership NHS Trust.
Information on the grade of the author was also collected. Data were then analysed using Statistical Package for the Social Sciences (version 19). Proportions of standards met in the e-discharge summaries and dictated summaries were compared using the χ 2 test and Fisher’s exact test. For comparing the total scores, we used the Mann–Whitney U test as these data were not normally distributed (using Shapiro–Wilk test, the p values in the e-discharge summary group was 0.02 and in the dictated discharge summary group it was 0.01). Inter-rater reliability was measured using Cohen’s κ. The p values were two-tailed, and the significance level was set at 0.05.
Results
The total number of admissions in the first period (dictated discharge summaries) was 509, and in the second period (e-discharge summaries) it was 936.
The proportions of standards met in the two groups are presented in Table 1. There were statistically significant differences in five of the standards. The standards were findings of physical examination (p<0.001), ICD-10 code (p<0.001), forensic history (p<0.001), alcohol history (p<0.001) and drug history (p<0.001). Each of these standards were met in significantly fewer e-discharge summaries than in the dictated summaries. The average total score in the dictated group (mean=9.5, s.d.=2.0) was higher than that in the e-discharge group (mean=6.7, s.d.=1.8), and that difference was statistically significant (p<0.001).
Core trainees wrote a larger proportion of the dictated summaries (n=45, 90%) as compared with the e-discharge summaries (n=34, 68%). However, when this was analysed using the regression analysis, it was found that grade was not a statistically significant predictor of the total score.
As most of the discharge summaries were written by trainees, we repeated the analysis by including only discharge summaries completed by core trainees. This showed very similar results. There were statistically significant differences in the following sections: physical examination (p<0.001), ICD-10 code (p<0.001), forensic history (p=0.003), alcohol history (p=0.01) and drug history (p=0.001). In addition, for this group, the mean total score in the e-discharge summary group was 6.85 (s.d.=1.82) compared with 9.47 (s.d.=2.04) in the dictated discharge summary group. This difference was statistically significant, and the difference in the total scores remained significant using both the t-test (t=−5.89, p<0.001) and the Mann–Whitney U test, which yielded the same result (p<0.001).
The agreements between raters were good (Cohen’s κ=0.90, 95% CI=0.88–0.92, p<0.001).
Discussion
This service evaluation appears to reveal a reduction in the amount of information contained in discharge summaries after the introduction of the e-discharge summary system, compared with the old method of dictated summaries. This result contradicts the findings of other audits, including that by Scullard et al. (Reference Scullard, Iqbal, White, Olla and Thomson2007) and Maslove et al. (Reference Maslove, Leiter, Griesman, Arnott, Mourad, Chow and Bell2009).
The reasons for our findings are not clear, but are possibly related to the design of the e-discharge system used at our trust, the short time period (24 hours) within which the e-discharge summaries are to be completed and the training provided to those using it. The e-discharge system used at the Bradgate Mental Health Unit employs a set pro forma with sub-sections with specific headings, where each section has a word limit. This format is similar to e-discharge summaries used by many healthcare trusts. This word capping, although it helps in keeping letters succinct, may negatively affect the amount of information included, and doctors may neglect to include information that does not have a specific sub-heading. If the e-discharge software had a specific sub-heading for each of the above standards, then adherence may improve, particularly if completing each section was mandatory for the completion of the discharge summary, with an option to write ‘no forensic history’ or ‘no information available’ if necessary. However, these rigid pro formas make it more difficult to tailor letters to individual patients and circumstances.
In general, typing a discharge summary takes more time than dictating one, and this may lead to briefer, less comprehensive letters. Administration time is, however, reduced with the use of e-discharge summaries. The fact that e-discharge summaries have to be completed within a short time period (24 hours) might have affected the amount of information included. There was a significant reduction in the proportion of summaries including the ICD-10 code. A similar trend, although not statistically significant, was found in the case of diagnosis. This is perhaps surprising, as the new e-discharge system included a specific sub-heading for each of these pieces of information; however, it may also be a result of junior doctors having less time to find information from written notes or to check information with their senior colleagues.
There was a significant drop in the proportion of summaries including information on alcohol history, drug history and the results of physical examination. The e-discharge software did not have specific sections for these. It did include a free-text section for ‘further information’, but did not have a drop-down option to select something like ‘no alcohol history’, etc. This might explain these results, although it would not explain the significant difference in relation to items that had specific sections in the e-discharge software, such as ICD-10 code and diagnosis. This drop in the proportion of summaries including information on substance abuse history and physical examination is worrying. Drug and alcohol problems are particularly prevalent in psychiatric inpatients (Conway et al. Reference Conway, Compton, Stinson and Grant2006). Many of the drug treatments used in mental health have adverse effects on physical health, and studies have found that mental health patients have poorer physical health than the general population (Jones et al. Reference Jones, Macias, Barreira, Fisher, Hargreaves and Harding2004). It is, therefore, important that these issues are communicated. Forensic history was covered in a significantly smaller proportion of e-discharge summaries compared with dictated letters. This could have an impact on future risk assessments in the community.
The total number of discharges in the first 6 months of 2012 when e-discharges were used (936) was higher than in 2010 (509), and this increased the workload on junior doctors, which may be a confounding factor on the quality of discharge summaries produced.
A major limitation of this service evaluation is that it only assessed one particular e-discharge system, and therefore cannot reliably be extrapolated to other systems. Specific suggestions are required to be tailored towards individual systems, but may include improvements to the e-discharge system itself with removing word capping and improving the sub-headings used, as well as improved education and training. Another limitation is the scoring system itself. Although we have tried to include a range of standards based on published scoring systems and local guidelines, clearly there may have been information included in the discharge summary, which did not fit into these standards. For example, the e-discharge summaries had a specific section for ‘signs of relapse’, which was not one of our standards. However, we feel that these standards covered a sufficient range of essential information to allow us to effectively compare discharge summaries.
Following the completion of this service evaluation, results were emailed to all the consultants in the trust, highlighting areas for improvement. The results will be presented at a departmental education meeting following which we plan to re-audit to evaluate for any improvement.
This service evaluation has highlighted an important consideration for the users and designers of e-discharge summary systems, which warrants further study. Our findings reveal a decline in the quality of discharge summaries following the introduction of an electronic system. The reasons for this remain unclear and require further evaluation. The move towards the use of more electronic systems in healthcare systems is inevitable and provides a variety of benefits. However, it is important to look closely at the systems used in order to ensure that they maximise the quality of these communications and that training for those using these systems is sufficient.
Acknowledgement
This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Conflicts of Interest
None.
