Introduction
Accurate survival prediction in advanced cancer patients has paramount importance in palliative care practice. It is absolutely necessary for determining the goals of care among medical professionals, and patients and families need the information to assign priorities and share the precious remaining time at the end of life (Maltoni et al., Reference Maltoni, Caraceni and Brunelli2005; Glare et al., Reference Glare, Sinclair and Downing2008). Generally, the clinicians’ prediction of survival (CPS) is a useful and traditional tool used in clinical practice to estimate survival. However, CPS alone has shown limited accuracy and the tendency to overestimate (Glare et al., Reference Glare, Virik and Jones2003). Thus, multiple prognostic scoring systems have been developed to complement CPS in advanced cancer patients (Maltoni et al., Reference Maltoni, Caraceni and Brunelli2005; Lau et al., Reference Lau, Cloutier-Fisher and Kuziemsky2007; Stone and Lund, Reference Stone and Lund2007).
Objective Prognostic Score (OPS) was developed as an easy-to-use tool that does not require CPS through a multicenter study in Korea (Suh et al., Reference Suh, Choi and Shim2010). OPS consists of anorexia, dyspnea, Eastern Cooperative Oncology Group Performance Status (ECOG PS), leukocyte count, and serum total bilirubin, creatinine, and lactate dehydrogenase (LDH) levels. OPS is optimized to predict 3-week survival (Suh et al., Reference Suh, Choi and Shim2010). It has been validated in Japan (Jho et al., Reference Jho, Suh and Yoon2016) and Korea (Yoon et al., Reference Yoon, Jung and Kim2014, Reference Yoon, Suh and Lee2017) repeatedly. OPS ranged from 0.0 to 8.0 points. Using OPS 3 as a cutoff, the sensitivity, specificity, and overall accuracy to predict 3-week survival was 74.7, 76.5, and 75.5%, respectively (Suh et al., Reference Suh, Choi and Shim2010). A previous Korean study reported that Palliative Prognostic Score (PaP) and OPS showed similar accuracy in predicting the survival of Korean patients with cancer having weeks to live (Yoon et al., Reference Yoon, Jung and Kim2014).
PaP was developed in Italy and is comprised of CPS, Karnofsky Performance Status (KPS), dyspnea, anorexia, leukocyte count, and lymphocyte percentage (Pirovano et al., Reference Pirovano, Maltoni and Nanni1999). PaP aims to predict 30-day survival (Pirovano et al., Reference Pirovano, Maltoni and Nanni1999) and was validated in various clinical settings (Glare and Virik, Reference Glare and Virik2001; Glare et al., Reference Glare, Eychmueller and McMahon2004; Tassinari et al., Reference Tassinari, Montanari and Maltoni2008; Kurashima et al., Reference Kurashima, Latorre and Camargo2010; Naylor et al., Reference Naylor, Cerqueira and Costa-Paiva2010; Numico et al., Reference Numico, Occelli and Russi2011; Tarumi et al., Reference Tarumi, Watanabe and Lau2011; Baba et al., Reference Baba, Maeda and Morita2015). Maximum PaP scores reach a total of 17.5 points. According to the total score, the 30-day survival probability is judged to be over 70% for 0–5.5 points, 30–70% for 5.6–11.0 points, and less than 30% for 11.1–17.5 points (Pirovano et al., Reference Pirovano, Maltoni and Nanni1999). The European Association for Palliative Care also recommended PaP as a useful prognostic tool (Maltoni et al., Reference Maltoni, Caraceni and Brunelli2005). However, inexperienced clinicians may hesitate to approach PaP because it is difficult for them to formulate CPS (Suh et al., Reference Suh, Choi and Shim2010). And CPS could be affected by the patient–physician relationship even in expert clinicians (Christakis, Reference Christakis1999).
However, no prospective study has compared OPS with PaP, which is the most commonly used prognostic model. And no investigation has been performed to compare the two prognostic scores across countries at the same time. Therefore, we aimed to evaluate the accuracy of OPS compared with PaP in inpatients with advanced cancer in three East Asian countries, namely Japan, Korea, and Taiwan.
Methods
Participants
This study was a secondary analysis of an international multicenter cohort study called the East Asian collaborative cross-cultural Study to Elucidate the Dying Process (EASED), which aimed to investigate the dying process and end-of-life care of inpatients with far-advanced cancer in palliative care units (PCUs) nationwide in Japan, Korea, and Taiwan. We consecutively enrolled eligible, newly admitted inpatients in the participating PCUs during the study period. All observations were performed in the course of routine clinical practice. The clinical criterion was a presumed life expectancy of 6 months or less.The inclusion criteria were (1) adults (age ≥ 18 years old in Japan and Korea and ≥20 years old in Taiwan), (2) patients with locally extensive or metastatic cancer, and (3) patients newly admitted to a participating PCU. The exclusion criteria were (1) scheduled discharge within one week and (2) refusal to enroll by the patient or their family.
Data collection
The palliative physicians evaluated the patients’ symptoms by direct observation at admission. We followed up discharged patients for 6 months from admission to PCUs in Japan and Taiwan, and 6 months from discharge from PCUs in Korea. Therefore, we defined survival time as mortality in and outside of hospitals calculated by subtracting the admission date from the death date. From the data collected by the EASED study, we extracted and analyzed the data required for our study to calculate OPS and PaP. We collected data on anorexia, dyspnea, ECOG PS, leukocyte counts, and serum total bilirubin, creatinine, and LDH levels for calculating OPS (Suh et al., Reference Suh, Choi and Shim2010). We collected data on CPS, KPS, dyspnea, anorexia, leukocyte counts, and lymphocyte percentages for calculating PaP (Pirovano et al., Reference Pirovano, Maltoni and Nanni1999).
Data analysis and statistics
All analyses were performed using IBM Statistical Package for Social Science (SPSS) Statistics for Windows, version 21.0 (IBM Corp., Armonk, NY, USA).
We classified the patients into three groups according to a 3-week survival probability using total OPS ranging from 0 to 8 points, with 0–2 points in the longer survival group and 3–8 points in the shorter survival group (Suh et al., Reference Suh, Choi and Shim2010). The patients were categorized into three groups according to a 30-day survival probability using total PaP ranging from 0 to 17.5 points: group A (0–5.5 points), group B (5.6–11 points), and group C (11.1–17.5 points) (Pirovano et al., Reference Pirovano, Maltoni and Nanni1999).
We calculated the median overall survival and 95% confidence intervals (CIs) in each group and constructed survival curves for the risk groups classified by each prognostic score using the Kaplan–Meier method. We compared the sensitivity, specificity, positive predictive value, negative predictive value, and overall accuracy of OPS and PaP. For comparison of overall accuracy, OPS and PaP were dealt as dichotomous groups. OPS was dichotomized groups having scores of <3.0 and scores of ≥3.0. And we categorized PaP as group A and (B + C). Since group A in PaP was validated as having >70% survival in 30-day prediction (Pirovano et al., Reference Pirovano, Maltoni and Nanni1999), thus we regarded group A to have 3-week survival probability to compare to OPS. The discriminatory ability of the risk score was measured using the area under the receiver operating characteristics (ROC) curve. We compared the area under the ROC curve (AUROC) for the accuracy of OPS and PaP using both 3-week and 30-day survival predictions. For AUROC comparison, both OPS and PaP were interpreted as continuous variables.
Ethics
In accordance with the ethical guidelines for human research of Japan's Ministry of Health, Labor, and Welfare, the requirement for informed patient consent was waived in Japan because of the observational nature of the study. In Korea and Taiwan, informed consent was obtained from the patients or their families (if the patient lacked the capacity to make decisions). The study obtained approval from the local Institutional Review Boards of all participating institutions. Also, the Independent Ethics Committee of the Tohoku University School of Medicine (approval no. 2016-1-689) approved this study.
Results
Patient characteristics
We analyzed the data of 2,638 patients across 37 PCUs (22 in Japan, 11 in Korea, and 4 in Taiwan). The patients were enrolled from January 2017 to September 2018. We excluded all 407 Taiwanese patients because 96.1% (391/407) were missing the laboratory data needed to calculate OPS. Among the eligible 2,231 patients, 598 patients (Japan, 531; Korea, 67) were excluded because of missing data needed for calculating OPS or PaP. In addition, five Japanese patients were excluded because of missing data on survival time. Thus, 1,628 patients (Japan, 1360; Korea, 268) were analyzed (Figure 1). In Japanese patients, data were available to calculate the OPS in 80.7% (1,530/1,896) and PaP in 75.0% (1,472/1,896). In Korean patients, OPS was calculated in 80.0% (268/335) and PaP was calculated in 95.5% (320/335). Thus, as for both OPS and PaP, 71.7% (1,360/1,896) of Japanese patients and 80.0% (268/335) of Korean patients were calculated. These patients included 783 men [Japan, 666 (49.0%); Korea, 117 (43.7%)]. The median survival time was 18 days [95% confidence interval (CI): 16.3–19.7] in Japan and 22 days (95% CI: 18.9–25.0) in Korea. Regarding OPS, the longer survival group (scores of <3.0) was comprised of 1,137 patients (Japan, 941; Korea, 196), whereas the shorter survival group (scores of ≥3.0) was comprised of 491 patients (Japan, 419; Korea, 72). As for PaP, group A (scores of 0–5.5) consisted of 356 patients (Japan, 295; Korea, 61), group B (scores of 6.0–11.0) consisted of 761 patients (Japan, 612; Korea, 149), and group C consisted of 511 patients (Japan, 453; Korea, 58). The characteristics of the patients are shown in Table 1, the prevalence of each component of OPS is shown in Table 2, and the prevalence of each component of PaP is shown in Table 3.
Flow diagram of participant selection.
Baseline characteristics of the participants (n = 1,633)
Data are presented as numbers (%).
Abbreviations: SD, standard deviation; CI, confidence interval.
Prevalence of each Objective Prognostic Score category and classification of the participants by two risk groups
Data are presented as numbers (%).
Abbreviations: ECOG, Eastern Cooperative Oncology Group; S, serum; LDH, lactate dehydrogenase.
Prevalence of each Palliative Prognostic Score category and classification of the participants by three risk groups
Data are presented as numbers (%).
Abbreviations: KPS, Karnofsky Performance Status; CPS, Clinicians’ Prediction of Survival; WBC, White Blood Cell.
Median survival time of each risk group according to prognostic score
We calculated the median survival time of each risk group classified by OPS and PaP (Table 4). In OPS groups in Japanese patients, the median survival was 27 days (95% CI: 24.9–29.1) in the longer survival group, whereas it was eight days (95% CI: 6.8–9.2) in the shorter survival group. In terms of OPS in Korean patients, the median survival was 25 days (95% CI: 21.7–28.3) in the longer survival group and 11 days (95% CI: 7.7–14.3) in the shorter survival group. Regarding PaP classifications in Japanese patients, the median survival was 51 days (95% CI: 44.3–57.7) in group A, 22 days (95% CI: 20.1–23.9) in group B, and six days (95% CI: 5.2–6.8) in group C. In regard to PaP classifications in Korean patients, the median survival was 50 days (95% CI: 39.3–60.7) in group A, 23 days (95% CI: 20.6–25.4) in group B, and nine days (95% CI: 6.4–11.6) in group C.
Survival time according to Objective Prognostic Score (OPS) groups and Palliative Prognostic Score (PaP) classifications
Abbreviation: CI, Confidence Interval.
Accuracy
The survival curves of each risk group divided according to OPS and PaP are shown in Figures 2 and 3. Both OPS and PaP showed excellent discrimination in Kaplan–Meier (KM) plots by their cutoff values in Japan (p < 0.01 and p < 0.01, respectively) and Korea (p < 0.01 and p < 0.01, respectively), as well. The performance of OPS and PaP for 3-week survival is shown in Table 5. For OPS, the specificities were higher than the sensitivities in both Japan (87.8%) and Korea (84.4%). On the other hand, PaP showed higher sensitivities than the specificities in both Japan (91.1%) and Korea (88.4%). The AUROC for 3-week survival was 0.74 (95% CI: 0.72–0.77) for OPS in Japan, 0.68 (95% CI: 0.61–0.74) for OPS in Korea, 0.80 (95% CI: 0.78–0.82) for PaP in Japan, and 0.73 (95% CI: 0.67–0.79) for PaP in Korea. The AUROC for 30-day survival was 0.70 (95% CI: 0.68–0.73) for OPS in Japan, 0.71 (95% CI: 0.64–0.77) for OPS in Korea, 0.79 (95% CI: 0.76–0.81) for PaP in Japan, and 0.74 (95% CI: 0.67–0.80) for PaP in Korea (Table 6).
Kaplan–Meier plots according to risk groups defined by Objective Prognostic Score (OPS) risk groups (a) and Palliative Prognostic Score (PaP) classifications (b) in Japan (n = 1,360). The p-values were <0.01 and <0.01 for (a) and (b), respectively. The p-values were derived from log-rank tests.
Kaplan–Meier plots according to risk groups defined by Objective Prognostic Score (OPS) groups (a) and Palliative Prognostic Score (PaP) classifications (b) in Korea (n = 268). The p-values were <0.01 and <0.01 for (a) and (b), respectively. The p-values were derived from log-rank tests.
Performance of Objective Prognostic Score (OPS) groups and Palliative Prognostic Score (PaP) for 3-week survival in Japan and Korea
Data are presented as percent with 95% confidence intervals in parenthesis except prevalence.
Prevalence is defined death events in each time frame per total study population.
Abbreviations: PPV, Positive Predictive Value; NPV, Negative Predictive Value.
Areas under the receiver operating curve (AUROC) and 95% confidence intervals for Objective Prognostic Score (OPS) and Palliative Prognostic Score (PaP) in Japan and Korea
Discussion
The aim of this study was to evaluate the accuracy of OPS compared with PaP in patients with advanced cancer in three East Asian countries, namely Japan, Korea, and Taiwan. This study showed that both OPS and PaP performed well in Japan and Korea. PaP showed better accuracy than OPS in Japan. Unfortunately, OPS cannot be applied in Taiwan because of different clinical practices in the PCUs.
We found that the use of OPS was available and accurate in Japan and Korea but not in Taiwan. In contrast, the use of PaP was available in all three countries. The feasibility of OPS was reported to be 70.0% in Japan (Jho et al., Reference Jho, Suh and Yoon2016) and 89.3% in Korea (Yoon et al., Reference Yoon, Suh and Lee2017), consistent with this study. Our study showed slightly higher availabilities of OPS, at 80.7% in Japan and 80.0% in Korea. The feasibility of PaP in Japan was reported to be 37.5–80.5% (Baba et al., Reference Baba, Maeda and Morita2015), while there has been no previous report on the feasibility of PaP in Korea and Taiwan. And previous studies have reported that the feasibility was 65.7–98% (Glare and Virik, Reference Glare and Virik2001; Glare et al., Reference Glare, Eychmueller and McMahon2004; Tassinari et al., Reference Tassinari, Montanari and Maltoni2008; Kurashima et al., Reference Kurashima, Latorre and Camargo2010; Naylor et al., Reference Naylor, Cerqueira and Costa-Paiva2010; Numico et al., Reference Numico, Occelli and Russi2011; Tarumi et al., Reference Tarumi, Watanabe and Lau2011). Our results showed that the availability of using PaP was 75.0% in Japan, 95.5% in Korea, and 81.6% in Taiwan. Usually, inpatients undergo blood tests when they are admitted to PCUs in Japan and Korea. In Taiwan, the patients in the PCUs were admitted when death was nearer than in the other two countries [median survival time 15 days in Taiwan (data not shown in the Results section) compared with 18 days in Japan and 22 days in Korea] due to the infrastructure of the healthcare system and a majority of these inpatients are transferred from acute care wards in the hospital after shared palliative care (Lin et al., Reference Lin, Chiu and Ho2014). Therefore, Taiwanese palliative care physicians tend not to order blood tests to reduce patients’ suffering in PCUs, while laboratory data before admission were available (Chih et al., Reference Chih, Su and Hu2016). Taiwanese physicians only order blood tests for patients in hospice care when there is an emergent medical situation such as an abrupt change in consciousness or fever. To calculate OPS, the results of liver function tests (LDH and bilirubin) and a renal function test (creatinine) in addition to complete blood counts (WBC and lymphocyte) are needed. In Taiwan, complete blood counts (CBCs) might be obtained at outpatient clinics because CBCs are a basic laboratory test for cancer patients to detect infection, anemia, neutropenia, and other conditions, but liver and renal function tests might not be performed. Hence, the feasibility of calculating OPS was very low while the feasibility of calculating PaP was high in Taiwan.
OPS showed similar accuracy to PaP in predicting both 3-week and 30-day survival in Korea. In contrast, PaP showed better performance than OPS in Japan for 3-week and 30-day survival. It is impressive that simple OPS could show similar accuracy to PaP in Korea. OPS may be an appropriate model for this study population because the median survival time was around three weeks (22 days in Korea). And the availability of being able to calculate OPS in Korea was higher than that in Japan. Because OPS was developed in Korea more than a decade ago, it may suit the medical environment of PCUs in Korea.
In contrast, PaP was a better model than OPS in Japan. We assumed that CPS in PaP enhanced its accuracy in Japan, as previous studies suggested (Maltoni et al., Reference Maltoni, Scarpi and Pittureri2012; Yoon et al., Reference Yoon, Jung and Kim2014, Reference Yoon, Suh and Hui2021; Baba et al., Reference Baba, Maeda and Morita2015). It has been well-documented that the experts have smaller errors in prognostic predictions and could have more accurate prognostic predictions (Maltoni et al., Reference Maltoni, Nanni and Derni1994; Tavares et al., Reference Tavares, Oliveira and Goncalves2018; Christakis and Lamont, Reference Christakis and Lamont2000). In addition, the accuracy of CPS may be higher in patients with shorter predicted survivals (Hui et al., Reference Hui, Ross and Park2019). In the current study, the participating physicians were all palliative physicians, and thus, most of them were experts in survival prediction. The median survival time was also short (18 days in Japan, 22 days in Korea). Therefore, PaP including CPS appeared as a more reliable tool in Japan. Other assumptions were also made. KPS has been proved to be a reliable prognostic parameter mainly in the middle-to-low score range (Yates et al., Reference Yates, Chalmer and McKegney1980; Miller, Reference Miller1991). More than 80% of the analyzed patients had a KPS of less than 20. Therefore, KPS component of PaP may provide more detailed prognostic information than the ECOG PS in OPS in Japan.
Notably, OPS showed high specificities while PaP showed high sensitivities in 3-week survival prediction in this study. Thus, OPS seems to be a better identifying tool for death of patients in final weeks of life. On the contrary, PaP can be a useful screening tool for 3-week survival. Clinicians may use a combination of two scores of high sensitivity (e.g., PaP) and high specificity (e.g., OPS) to make prognostication improved only if there are available laboratory values.
We believe that PaP is an advanced tool for survival prediction. However, not every physician is familiar with CPS. Even experienced physicians sometimes feel less confident in formulating CPS in real-world settings because unexpected events could happen (Amano et al., Reference Amano, Maeda and Shimoyama2015). Therefore, OPS could be an alternative if physicians hesitate to estimate CPS. After calculating OPS, it may be easier for inexperienced physicians to make their own CPS supported by the score profile.
This study had several limitations. First, this study needed laboratory data for calculating the prognostic scores. Nevertheless, the feasibility was sufficiently high in Japan and Korea. Meanwhile, OPS could not be calculated in Taiwan because of missing data on liver function (LDH and bilirubin). Thus, modification of OPS such as needing less laboratory data is warranted. Second, this study population was in PCUs and may not represent the general population of patients with far-advanced cancer. CPS can differ according to the patients’ life expectancies and their places of care. Moreover, CPS might be formulated by “experts” demonstrating excellent prognostic performance. Lastly, our findings may be influenced by the accessibility to palliative care services and PCU admission in each country. Third, there might be an intrinsic mathematical disparity because we compared a tool (OPS) which comprises of two risk groups for 3-week survival with a tool (PaP) which uses three probability risk groups for 30-day survival. Therefore, we calculated AUROC based on continuous outcome to compare two tools although both OPS and PaP have been validated as a categorical outcome. Using this approach, a higher OPS or PaP score was simply regarded as representing a worse prognosis and the prognostic risk groups as described in the original development study might be ignored (Stone et al., Reference Stone, Vickerstaff and Kalpakidou2021). Fourth, calibration analysis for model accuracy was not evaluated. Instead of it, we compared the area under the AUROC for the accuracy of OPS and PaP using both 3-week and 30-day survival predictions. For evaluation of clinical efficacy, decision curve analysis was more appropriate. Future studies can yield more refined results using those advanced analyses.
In conclusion, we demonstrated that OPS was a feasible tool with acceptable accuracy compared with the widely used PaP in Japan and Korea. OPS can predict the final weeks with acceptable accuracy only if there are available laboratory test results performed in routine practice. We suggest that OPS can be more useful for inexperienced physicians who may consider CPS a challenging task. Further studies are necessary to compare modified OPS and PaP in diverse palliative care settings including early palliative care.
Acknowledgments
We are grateful to Harrisco Encorrection (Seoul, South Korea) for proofreading this manuscript for grammar and clarity.
Author contributions
Y.H.: Investigation, Writing – original draft, Writing – review and editing; D.K.: Conceptualization, Methodology, Writing – original draft, Writing – review and editing; S.Y.S.: Conceptualization, Investigation, Methodology, Project implementation, Supervision, Writing – review and editing; S.H.K.: Investigation, Writing – review and editing; S.J.Y.: Investigation, Writing – review and editing; S.J.K.: Investigation, Writing – review and editing S.A.P.: Investigation, Writing – review and editing; J.Y.S.: Investigation, Writing – review and editing; J.H.K.: Investigation, Writing – review and editing; J.P.: Investigation, Writing – review and editing; Y.P.: Investigation, Writing – review and editing; S.W.H.: Investigation, Writing – review and editing; E.S.L.: Investigation, Writing – review and editing; H.C.: Project implementation, Data curation, Formal analysis; H.Y.A.: Data curation, Formal analysis, Supervision; S.Y.C.: Investigation, Writing – review and editing; P.J.C.: Investigation, Writing – review and editing; T.Y.: Investigation, Writing – review and editing; T.M.: Investigation, Writing – review and editing; S.T.: Supervision, Writing – review and editing; M.M.: Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Writing – review and editing; A.I.: Investigation, Supervision, Writing – review and editing.
Funding
This work was supported in part by a Grant-in-Aid from the Japanese Hospice Palliative Care Foundation [Grant Numbers 16H05212 and 16KT0007].
Conflict of interest
The authors declare that there are no conflicts of interest.
