INTRODUCTION
The assessment of patients' suffering is indispensable to responding to their needs. Cognitive failure is a frequent occurrence in palliative care, and most patients have phases of cognitive impairment during the course of their disease, mainly during the last days of life (Shuster, Reference Shuster1998). Even then, some can communicate their problems in a reasonably clear way, but many of them cannot (Volicer et al., Reference Volicer, Hurley and Blasi2001).
The subjective nature of symptoms makes patient reports the best way to assess them. However, when cognitive impairment precludes patient reporting, it is necessary to employ other means. One alternative may be assessment through a family member or other close associate. It seems that a condition for making this method of assessment reliable is maintenance of close contact with a patient (Cohen-Mansfield, Reference Cohen-Mansfield2002). Another problem with this method is the need for those people to be present when the assessment is done, which also makes the method problematic. The most commonly used alternative is behavioral assessment. Many methods of behavioral assessment have been developed for patients with cognitive impairment and other communication difficulties. Most of them were developed for pain assessment (Wary, Reference Wary1999; Manfredi et al., Reference Manfredi, Breuer and Meier2003; Fuchs-Lacelle & Hadjistavropoulos, Reference Fuchs-Lacelle and Hadjistavropoulos2004; Zwakhalen et al., Reference Zwakhalen, Hamers and Abu-Saad2006; Morrelo et al., Reference Morrelo, Jean and Alix2007; Horgas & Miller, Reference Horgas and Miller2008; Lotan et al., Reference Lotan, Ljunggren and Johnsen2009), though some were developed for assessment of other symptoms (Campbell et al., Reference Campbell, Templim and Walch2010), for quality of life (Logsdon et al., Reference Logsdon, Gibbons and McCurry1999; Martin-Cook et al., Reference Martin-Cook, Hynan and Rice-Koch2005; Ettema et al., Reference Ettema, Dröes and de Lange2005; Kasper et al., Reference Kasper, Black and Shore2009), and nonspecifically for assessment of suffering (Kovach et al., Reference Kovach, Noonan and Griffie2002; Aminoff et al., Reference Aminoff, Purits and Noy2004). Most of these studies were carried out in patients with dementia (Logsdon.et al., Reference Logsdon, Gibbons and McCurry1999; Volicer et al., Reference Volicer, Hurley and Blasi2001; Cohen-Mansfield, Reference Cohen-Mansfield2002; Kovach et al., Reference Kovach, Noonan and Griffie2002; Manfredi et al., Reference Manfredi, Breuer and Meier2003; Fuchs-Lacelle & Hadjistavropoulos, Reference Fuchs-Lacelle and Hadjistavropoulos2004; Aminoff et al., Reference Aminoff, Purits and Noy2004; Martin-Cook et al., Reference Martin-Cook, Hynan and Rice-Koch2005; Ettema et al., Reference Ettema, Dröes and de Lange2005; Morrelo et al., Reference Morrelo, Jean and Alix2007; Horgas & Miller, Reference Horgas and Miller2008; Kasper et al., Reference Kasper, Black and Shore2009; Campbell et al., Reference Campbell, Templim and Walch2010) and in intensive care (Mateo & Krenzischek, Reference Mateo and Krenzischek1992; Puntillo et al., Reference Puntillo, Miaskowski and Kehrle1997; Payen et al., Reference Payen, Bru and Bosson2001; Labus et al., Reference Labus, Keefe and Jensen2003; Odhner et al., Reference Odhner, Wegman and Freeland2003; Puntillo et al., Reference Puntillo, Morris and Thompson2004; Gélinas et al., Reference Gélinas, Fillion and Puntillo2006; Reference Gélinas, Harel and Fillion2009; Arif-Rahu & Grap, Reference Arif-Rahu and Grap2010). In palliative care, tools developed for other situations are often utilized, but, to the best of our knowledge, specific tools do not exist.
These tools are employed to assess symptoms by such things as observing facial expression, mood, social interaction, and verbal behavior. An inherent difficulty in assessing patients who cannot tell you how they are feeling is finding out if the signs of suffering observed are due to pain or have another physical or affective cause, or are even caused by the cognitive failure itself. Possible resources that can be employed to infer the cause of suffering include medical history collected from family, any other past clinical information that may exist, and physical examination. In some situations, it is not possible to educe the cause of observed suffering. The tools that have been developed thus far have significant weaknesses, including faults with their psychometric properties and clinical usefulness (Zwakhalen et al., Reference Zwakhalen, Hamers and Abu-Saad2006; Hølen et al., Reference Hølen, Saltvedt and Fayers2007).
As there is no reliable tool for assessing suffering in patients who cannot report their symptoms, and because this issue has not been studied specifically in the palliative care setting, when patients cannot report what they are feeling in our palliative care unit (PCU), we try to detect discomfort from their overall appearance, that is, if they seem to be comfortable, and recording this in their clinical file (under signs of suffering: “yes” or “no”). However, this method has not yet been submitted to evaluation for validity and clinical utility. It is the results of such evaluations that are presented here.
METHODS
Our study was carried out in a 20-bed PCU in the Porto section of the Portuguese Institute of Oncology. In the usual practice of the unit, all patients have their symptoms assessed by nurses every day in the morning using a 0–4 scale for intensity, independently of other assessments during the day, as needed. If a patient has cognitive failure precluding patient report, the nurse observes the patient and records if there are signs of discomfort. Despite the cognitive impairment of a patient, if they have some verbal capacity to report symptoms, that report prevails over observations. We included in our study patients with some verbal capacity in order to calculate the specificity and sensitivity of the method.
The study was carried out by two doctors and seven nurses working in the PCU, and who therefore know the patients. The included patients had cognitive impairment as assessed with the Consciousness Scale for Palliative Care (Gonçalves et al., Reference Gonçalves, Bento and Alvarenga2008) and a Portuguese version of the Confusion Assessment Method (Inouye et al., Reference Inouye, van Dick and Alessi1990). As this method aims to detect suffering at the moment of observation, some patients were included more than once depending on how long they stayed on the unit, although not on the same day. All nine researchers participated in patient assessment. Any two of them simultaneously assessed the patient and independently recorded if the patient showed signs of discomfort, according to the usual practice of the unit, without trying to stimulate the patient. An intra-rater assessment was not deemed adequate because symptoms change over time, so that assessment at different times may be assessing different situations rather than the method under study. As there is no gold standard for criterion validation, the only way to confirm the results of assessments is patient reports. Patients were thus asked immediately after the first two raters by a third researcher if they had any discomfort, and if they were able to respond at least “yes” or “no” their answer was recorded. If the answer was “yes,” they were questioned about the cause. There were patients who had some verbal capacity despite their severe cognitive impairment.
The statistical methods employed to measure interrater agreement were Cohen's kappa coefficient (Cohen, Reference Cohen1960) and the positive and negative proportions of specific agreement (http://www.john-uebersax.com/stat/raw.html; accessed on January 1, 2011). The sensitivity, specificity, and positive/negative predictive values were also calculated.
Our study was approved by the ethics committee and board of directors of the hospital. Whenever possible, written authorization was requested from a patient's family after an explanation of the aim of our study, as cognitive impairment precluded obtaining informed consent from the patient. However, as this was not possible in many cases and as the study was aimed at assessing normal practice and did not include any invasive procedures or efforts, and, on the other hand, it could be directly and immediately advantageous for patients, many were included without authorization.
RESULTS
The study was carried out between January 24, 2011, and February 22, 2012. Although our unit is not limited to oncological patients, more than 90% had a cancer diagnosis, the median age was 70 (25–93), and 60 (52%) were female (Table 1). Some 200 assessments were performed of 116 patients. They were assessed one to eight times (median = 1), with five or more observations on only four patients (Table 2). In 136 (68%) observations, patients were able to respond at least “yes” or “no” if they were feeling any discomfort. In 99 (73%) of these observations, patients denied any discomfort at the time; of the remaining 37, pain was the most frequent cause of discomfort, occurring in 22 (60%), followed by dyspnea in 3 (8%). There was a combination of pain and dyspnea in one patient.
Demographic and clinical data
Median age = 70 years (25–93). Gender: female 60 (52%), male 56 (48%).
Number of observations per patient (total 116)
For the assessment of interrater agreement, the kappa coefficient of agreement was calculated, and the result was 0.615. The proportions of specific agreement were also calculated, and the results for positive and negative agreement were 0.64 and 0.98, respectively (Table 3).
Agreement between raters
Kappa coefficient = 0.615. Proportions of specific agreement: positive 0.64, negative 0.98.
To assess the clinical usefulness of our method, the sensitivity calculated was 17% (CI 95% = 0.10–0.23), the specificity was 99% (CI 95% = 0.97–1.00), the positive predictive value was 88% (CI 95% = 0.82–0.93), and the negative predictive value was 73% (CI 95% = 0.65–0.80) (Table 4). These data concern rater 1, but those for rater 2 are similar (Table 5).
Sensitivity, specificity, and predictive values (rater 1)
Sensitivity 17%, specificity 99%, positive predictive value 88%, negative predictive value 73%,
Sensitivity, specificity, and predictive values (rater 2)
Sensitivity 14%, specificity 96%, positive predictive value 86%, negative predictive value 71%.
DISCUSSION
Our study describes an evaluation of the method used in daily practice at our unit to detect suffering in patients who are not able to report it. This issue is being studied mainly in dementia and intensive care. No method has been developed specifically for palliative care in general. In a systematic review of the scales for assessment of pain in elderly patients with severe dementia, it was concluded that there is no scale available now that can be convincingly indicated as being more suitable for assessing pain in these patients (Herr et al., Reference Herr, Bjoro and Decker2006). The Doloplus-2 (Wary, Reference Wary1999) and PACSLAC (Fuchs-Lacelle & Hadjistavropoulos, Reference Fuchs-Lacelle and Hadjistavropoulos2004) scales are usually deemed most appropriate. However, the results of one study did not support the criterion validity of the Doloplus-2 as a clinical pain assessment tool in its present form (Hølen et al., Reference Hølen, Saltvedt and Fayers2007). The situation is similar for intensive care (Li et al., Reference Li, Puntillo and Miaskowski2008). A recent review of pain measures for adult patients unable to self-report in an intensive care setting found that existing tools cannot be deemed robust pain measures for those patients (Arif-Rahu & Grap, Reference Arif-Rahu and Grap2010). Therefore, the problem of assessing specific symptoms or suffering in general is very complex. The lack of reliable tools was the reason why we did not adopt any of the existing tools and have continued to use our method.
It could be said that if structured tools are not good enough, unstructured methods would not be expected to do much better. However, clinical observation is sometimes utilized to assess some of the same aspects as structured tools. For example, in a study assessing the utility of PAINAD, Jordan and colleagues stated that some patients scored significantly on PAINAD, “but the behaviors were felt to be caused by a reason other than pain” (specificity), and “one participant assessed to be in pain” did not score as being in pain by PAINAD (sensitivity) (Jordan et al., Reference Jordan, Hughes and Pakresi2011). They thus calculated the sensitivity and specificity of that tool against assessment by unstructured observation, deeming it a gold standard. An analogous setting is the clinician's prediction of survival, where the prediction is carried out based on an impression of the overall state of the patient (Glare et al., Reference Glare, Virik and Jones2003). The surprise question for prognosis, though not a structured method, is based on a clinical impression, but it seems to be most useful (Moroni et al., Reference Moroni, Zocchi and Bolognesi2014).
Using our method, agreement between raters was good according to Cohen's kappa coefficient (Altman, Reference Altman1991). The proportion of negative agreement was very high—almost perfect, in fact. The specificity was very high; accordingly, the positive predictive value was also high, meaning that when an assessment is positive (i.e., when the observer detects signs of discomfort), the probability that the patient is uncomfortable is very high. However, its sensitivity is very low, meaning that only a minority of situations of discomfort will be detected, negatively impacting on the effectiveness of this method for screening suffering in patients with cognitive failure. In the study mentioned above (Jordan et al., Reference Jordan, Hughes and Pakresi2011), the sensitivity of the method was very high, but the specificity was much lower than that of our method. However, the comparison was with observation. Most studies do not report the sensitivity and specificity of its tools. Therefore, these data cannot be compared with most studies, but that information is crucial for assessing their clinical usefulness, since it is not possible to know how useful the tools are without it.
The existing tools for assessment of pain and discomfort usually include items that are very obvious, such as crying or calling out. In those cases, it is evident that the patient is suffering from some cause of discomfort, which may be pain or another problem. However, our study shows that the manifestations of discomfort often go undetected because they are probably more subtle or do not exist at all.
This method is used in the everyday practice on our unit and in this study to detect discomfort and not the specific cause (e.g., pain), because a behavior indicating suffering may be due to different causes. The low specificity observed in the study by Jordan et al. (Reference Jordan, Hughes and Pakresi2011) for detection of pain may be due to this problem. In fact, there are no signs of discomfort specific for pain (Regnard et al., Reference Regnard, Reynolds and Watson2007), so that other causes should be considered (NICE Clinical Guideline 42, 2006). Detection of the specific cause of discomfort requires deeper assessment.
The validation of tools for assessment of symptoms in patients with communication problems often includes behavior in situations that are predictably painful. It was observed that the frequency of pain-related behaviors is low at rest and more frequent during such activities as transfers, standing, or ambulation (Herr et al., Reference Herr, Bjoro and Decker2006). However, detection of discomfort at rest is also a very important objective. Activities can elicit discomfort that may not be present at rest. In fact, assessment during activity or movement introduces a different problem, which is assessment of an acute symptom induced externally whose resultant behavior might be different from a chronic symptom, as mentioned above. A classic statement is that patients with chronic pain may not show typical pain behavior, contrary to what occurs with acute pain. However, detection of discomfort at rest is also a very important aim, which is why our study was carried out on patients at rest without trying to stimulate them.
The results of our study confirm the difficulty of assessing patients with cognitive failure who cannot report what they are feeling. Therefore, one possible reason for the low sensitivity of our method is the ineffectiveness endemic to making decisions based on patient appearance. Another possibility is the influence of cognitive failure on the patient's response reliability, at least in some cases, meaning that patients with cognitive failure might misunderstand the questions asked and/or not give the most adequate answers. In fact, in one study, some patients with behaviors that could indicate pain explicitly denied pain, although their behavior changed after administration of analgesics (Kovach et al., Reference Kovach, Noonan and Griffie2001), confirming that the answers from patients with cognitive impairment might often be unreliable. So which other method would be closer to a gold standard? Behavior is also probably influenced by the intensity of the discomfort. The confirmatory answer we looked for was “yes” or “no.” We did not try to examine the intensity of the pain, which may also have contributed to the low sensitivity of our method. The different behaviors caused by chronic and acute symptoms, consensually accepted for pain in patients without communication problems, constitute an important aspect that will be studied with our method as the next step in our research into this challenging problem.
CONCLUSION
The problem of assessment of discomfort is a difficult one, and there is no widely accepted method that can be recommended. The method used in our study is also far from being adequate for that goal. However, research is an ongoing process where even negative results are important because they generate information leading to improved methods or new avenues of research. Obviously, this important aspect of care must be the object of further inquiry.
ACKNOWLEDGEMENTS
This work was supported in part by the North Section of the Portuguese League Against Cancer.
