Hostname: page-component-74656f9884-c52kr Total loading time: 0 Render date: 2026-11-07T17:17:37.299Z Has data issue: false hasContentIssue false
  • English
  • Français

Development and validation of the Palliative Care Knowledge Scale (PaCKS)

Published online by Cambridge University Press:  27 December 2016

Elissa Kozlov ,
Brian D. Carpenter  and
Thomas L. Rodebaugh
Elissa Kozlov*
Affiliation:
Department of Psychology, Washington University in St. Louis, St. Louis, Missouri
Brian D. Carpenter
Affiliation:
Department of Psychology, Washington University in St. Louis, St. Louis, Missouri
Thomas L. Rodebaugh
Affiliation:
Department of Psychology, Washington University in St. Louis, St. Louis, Missouri
*
Address correspondence and reprint requests to Elissa Kozlov, Department of Psychology, Washington University in St. Louis, Box 1125, One Brookings Drive, St. Louis, Missouri 63130. E-mail: Elissa.Kozlov@wustl.edu.
Rights & Permissions [Opens in a new window]

Abstract

Objective:

The purpose of this study was to develop a reliable and valid scale that broadly measures knowledge about palliative care among non-healthcare professionals.

Method:

An initial item pool of 38 true/false questions was developed based on extensive qualitative and quantitative pilot research. The preliminary items were tested with a community sample of 614 adults aged 18–89 years as well as 30 palliative care professionals. The factor structure, reliability, stability, internal consistency, and validity of the 13-item Palliative Care Knowledge Scale (PaCKS) were assessed.

Results:

The results of our study indicate that the PaCKS meets or exceeds the standards for psychometric scale development.

Significance of results:

Prior to this study, there were no psychometrically evaluated scales with which to assess knowledge of palliative care. Our study developed the PaCKS, which is valid for assessing knowledge about palliative services in the general population. With the successful development of this instrument, new research exploring how knowledge about palliative care influences access and utilization of the service is possible. Prior research in palliative care access and utilization has not assessed knowledge of palliative care, though many studies have suggested that knowledge deficits contribute to underutilization of these services. Creating a scale that measures knowledge about palliative care is a critical first step toward understanding and combating potential barriers to access and utilization of this life-improving service.

Keywords

Palliative careScale developmentPalliative care knowledge

Information

Type
Original Articles
Information
Palliative & Supportive Care , Volume 15 , Issue 5 , October 2017 , pp. 524 - 534
Copyright
Copyright © Cambridge University Press 2016 

INTRODUCTION

Although the number of palliative care teams is growing in hospitals throughout the United States (Morrison et al., Reference Morrison, Maroney-Galin and Kralovec2005; Veterans Health Administration, 2008), the palliative care services they offer remain underutilized. According to a 2012 survey from the National Hospice and Palliative Care Organization, of all the individuals who died in 2011 in the United States, only 44.6% received hospice care, a subtype of palliative care for patients in the last six months of their lives. No data are available on how many patients received palliative care throughout the course of their illness, but the Center to Advance Palliative Care estimates that at least six million people in the United States could benefit from palliative care they are not receiving (Center to Advance Palliative Care, 2014). In order to expand the use of palliative care services, we need to better understand the barriers to access and utilization. One potential barrier to utilization of palliative care is a lack of knowledge about these services among the general population.

Grossman and Kaestner (Reference Grossman, Kaestner, Behrman and Stacey1997) proposed that patient knowledge of health services drives utilization, and that consumers need to know what a service is and how it is relevant to their situation before they will seek it out or accept it. Previous research, sparse as it is, has confirmed that patients have little knowledge about palliative and hospice services (Born et al., Reference Born, Greiner and Sylvia2004; Koffman et al., Reference Koffman, Burke and Dias2007). For example, one national survey of 800 adults over the age of 18 found that 70% of participants rated themselves as “not at all knowledgeable” about palliative care (Public Opinion Strategies, 2011). If patients and family members are not aware of palliative care, they will not know to ask physicians for a referral. Furthermore, if physicians attempt to refer their patients for palliative care, patients may refuse when they are unfamiliar with the service and/or misunderstand its key aspects. More basic research is certainly needed to elucidate what people know about palliative care.

One challenge to assessing knowledge about palliative care is that there are no psychometrically evaluated assessment instruments currently available for measuring general knowledge about palliative care. One recent study developed an instrument to measure knowledge about care options among curative, palliative, and hospice care (Schulman-Green et al., Reference Schulman-Green, Ercolano and Jeon2012), but the authors acknowledged that this scale was intended to differentiate care options and did not comprehensively evaluate an individual's knowledge of any one type of care. A palliative care knowledge scale would be able to assess and identify what people know about palliative care as well as, more importantly, what information they lack. This scale could be useful for clinicians, program administrators, policy developers, and researchers interested in better understanding how knowledge of palliative care affects utilization of the service. A palliative care knowledge scale could also be utilized to understand knowledge gaps and thus guide important educational interventions with patients and family members. To that end, the main goal of our study was to develop an acceptable, efficient, and valid scale that measures knowledge about palliative care among laypeople.

METHODS

Development of the Scale

Literature Review

To determine the scope of the proposed Palliative Care Knowledge Scale (PaCKS), a literature review of research in the field of palliative care knowledge and education was conducted. Google Scholar and PubMed were searched using the terms “palliative care knowledge,” “palliative care education,” “palliative care assessment,” and “palliative services knowledge test.” The literature revealed minimal research regarding knowledge about palliative care services. The existing palliative care knowledge assessment instruments are intended for nurses, medical students, and medical residents (Fischer et al., Reference Fischer, Gozansky and Kutner2003; Meekin et al., Reference Meekin, Klein and Fleischman2000; Ross et al., Reference Ross, McDonald and McGuinness1996) and are too technical for the general public (e.g., questions about appropriate morphine dosing at the end of life).

Internet Search and Professional Consensus

Next, information available to consumers online was evaluated to determine how palliative care is presented and depicted in information commonly accessed by the general public. Some 22 domains of palliative care knowledge that appeared most frequently throughout the 26 unique online resources were identified. These domains were then rated on a scale of 1 to 5 (“not at all” to “extremely”) by a sample of 20 palliative care professionals on how relevant they were to the overall construct of palliative care knowledge. The professionals were also asked to submit any additional domains that were not represented. All domains were, on average, deemed very to extremely important, and no additional domains were identified. For a complete description of this study, see Kozlov and Carpenter (Reference Kozlov and Carpenter2015).

Preliminary Item Generation

The 22 domains were utilized during the item generation phase of the current study. Items were generated by a team of six researchers from a clinical geropsychology lab, which included two team members with multiple years of experience working specifically in palliative care. All domains were represented by at least one item in order to ensure that the construct's entire conceptual domain was represented. When possible, multiple items were generated to reflect the domains. Items were refined for clarity until all six members of the team were in agreement that the item (1) reflected exactly one domain, (2) reflected only one idea, and (3) utilized simple, straightforward language (Clark & Watson, Reference Clark and Watson1995). A total of 38 true/false items were generated. True/false formatting was chosen, as opposed to multiple choice, because the multiple-choice format is no better than the true/false option at distinguishing false guesses from misinformation (Kline, Reference Kline2005; Stanley & Hopkins, Reference Stanley and Hopkins1972), and the true/false format is more efficient to administer and score. Items were pilot tested on 30 palliative care professionals, who were encouraged to provide feedback on the relevance of each item to the overall construct of palliative care knowledge as well as on the clarity of the item.

Sample

A community-based sample of adults aged 18–89 (n = 614) and a professional sample of palliative care clinicians and researchers (n = 30) were recruited. The community-based sample was selected to reflect likely users of palliative care, and the professionals were selected to provide feedback and establish a professional consensus and construct validity for the scale items. Samples were recruited from community sources that included an academic medical center's volunteer pool, the host university's Department of Psychology Older Adult Subject Pool, and Amazon's Mechanical Turk (mTurk) database. To ensure diverse age representation, recruitment goals for each age decade were set. For a breakdown of participants' demographic characteristics and recruitment source, see Table 1. Once age stratification goals were achieved, the remaining participants were sampled from all sources. The 30 palliative care professionals were recruited through the Hospice and Palliative Care Special Interest Group Listserv of the Gerontological Society of America and through the professional contacts of the researchers.

Table 1.

Demographic information

* Non-mTurk includes the host university's older adult subject pool and the academic medical center's volunteer pool.

Measures

Demographics

To characterize the sample, the following demographic information for each participant was collected: gender, age, race, ethnicity, marital status, and education.

The Experience with Palliative Care Questionnaire

Dow and colleagues (Reference Dow, Matsuyama and Ramakrishnan2010) assessed knowledge of hospice care by simply asking patients “Have you heard of hospice?” This was the only previous research in the literature that attempted to measure knowledge about palliative care, so it was included in our study and expanded upon. In the current study, the item was revised for “palliative care,” and seven other questions were written to address familiarity with palliative care and hospice (see Appendix A). Items had dichotomous yes/no response options or Likert-type responses ranging from 1 to 5 or 1 to 10. Responses to individual items were considered independently in the analyses. The Experience with Palliative Care Questionnaire (EPCQ) was not utilized primarily as a measure to establish validity but rather was included to provide adjunctive information about the PaCKS.

Medical Term Recognition Test (METER; Rawson et al., Reference Rawson, Gunstad and Hughes2010)

The METER is a brief self-administered measure of health literacy. It has excellent internal consistency reliability (α = 0.93) and strong validity, as indicated by its high correlation (r = 0.74) with the Rapid Estimate of Adult Literacy in Medicine, another measure of health literacy (Davis et al., Reference Davis, Crouch and Long1991).

Shipley Vocabulary Test (Shipley, Reference Shipley1940)

The Shipley Vocabulary Test is a 40-item vocabulary test that measures general intellectual functioning. The test has excellent internal consistency (α = 0.87) (see Shipley, Reference Shipley1940).

Palliative Care Knowledge Scale

This new scale was developed to assess general knowledge about palliative care (see Appendix C for the final PaCKS items). In its preliminary form, it contained 38 true/false items (see Appendix B) that addressed a wide range of facts and basic principles about palliative care. Palliative care professionals were asked to rate on a scale from 1 to 10 the relevance and appropriateness of each item to the overall construct of palliative care knowledge.

Procedure

Our study was reviewed and approved by the host university's institutional review board. The study was hosted on the Qualtrics online platform. Participants received a preliminary screening measure that determined eligibility based on age stratification needs. All participants first received the core assessment, which was comprised of demographic questions, the EPCQ, the METER, and the PaCKS. A subsample of 321 adults were also administered the Shipley Vocabulary Test in addition to the core assessment. A subsample of 170 adults were recruited to take the PaCKS two times, near the beginning and at the end of the survey, for test–retest analyses. A sample of 30 palliative care professionals were recruited to take the PaCKS as well.

Data Analysis

Exploratory and Confirmatory Factor Analysis

The full sample was randomly split in half by generating a list of ones and twos using a random number generator (www.random.org) and assigning those sequentially to participants. This yielded a first sample of 299 participants and a second of 345 who were equivalent in terms of all major demographic characteristics. The first subsample was utilized to explore factor structures and scale construction, and the second subsample to confirm the solutions achieved in the first. An exploratory factor analysis (EFA) was performed on the first random half using the Mplus statistical software program. EFA with geomin rotation (the recommended default in Mplus) (Muthén & Muthén, Reference Muthén and Muthén2010) and the weighted least-squares mean and variance-adjusted (WLSMV) estimator were utilized to examine the structure of the 38 items of the PaCKS. The WLSMV estimator was used because all items (having a true/false scale) were treated as categorical. A scree plot of eigenvalues as well as Kaiser's (Reference Kaiser1960) eigenvalue-greater-than-one rule were considered when determining the appropriate number of factors. A principal components analysis was run using SPSS statistical software solely to help determine the appropriate number of factors to be extracted, because Kaiser's rule was based on principal components analysis. After determining the appropriate factor structure, confirmatory factor analysis (CFA) was employed to reduce the number of items and achieve strong indices of fit, still within the first random half of the data. We did this because CFA models often do not fit well, even when EFA models are utilized to inform factor loadings. We thus planned a priori to examine CFA models in the first random half until model fit was at least acceptable; we then planned to test the final factor structure in the second random half of the data.

During CFA procedures, individual items were considered for elimination based on low (under 0.3) (Tabachnick & Fidell, Reference Tabachnick and Fidell2001), nonsignificant, or negative factor loadings, as well as low R 2 values, indicating that the common factors explained little about item responses. Global model fit was evaluated using the following: the Tucker–Lewis Index (TLI; Tucker & Lewis, Reference Tucker and Lewis1973); the comparative fit index (CFI; Bentler, Reference Bentler1990); and the root-mean-square error of approximation (RMSEA; Steiger & Lind, Reference Steiger and Lind1980). The fit indicated by these indices was determined using the recommendations of Hu and Bentler (Reference Hu and Bentler1999), whose guidelines are also considered appropriate for categorical data (Yu & Muthén, 2001, cited in Muthén & Muthén, 2001). Once the factor structure of the PaCKS was established, a CFA was conducted on the second subsample (n = 345) to evaluate the hypothesized factor structure established through EFA and CFA techniques. All subsequent analyses were conducted on the items chosen to remain in the PaCKS.

Reliability of the PaCKS

To assess internal consistency reliability, the Kuder–Richardson Formula 20 (KR-20) value was calculated. To assess test–retest reliability, a two-way random intraclass correlation coefficient (ICC) under the absolute agreement definition (ICC 2,1) between the two administrations of the PaCKS was calculated.

Validity of the PaCKS

To assess construct validity, the qualitative data that the palliative care professionals provided on each item were considered. If multiple professionals raised concerns about an item's relatedness to the overall construct of palliative care knowledge, the item was flagged as a candidate for deletion. Additionally, items where less than 90% of professionals answered correctly were also candidates for deletion.

To assess convergent validity, the relationship between the final version of the PaCKS and total METER score was modeled using structural equation modeling (SEM) in MPlus. SEM was also employed to model the relationship between the PaCKS and total Shipley scores. Correlations within SEM were examined because these correlations attempted to remove the influence of measurement error (Grewal et al., Reference Grewal, Cote and Baumgartner2004). Finally, a Pearson's r correlation was calculated between individual items from the EPCQ and total scores on the PaCKS.

Because it was hypothesized that palliative care professionals would have a high degree of knowledge about palliative care, they were compared to individuals who had a hypothesized low amount of knowledge (i.e., individuals with low health literacy and little experience with palliative care) as a measure of discriminative validity. Thus, PaCKS scores for the 30 individuals with the lowest total METER scores who had not heard of palliative care were compared to the 30 palliative care professionals' scores using an independent-samples t test.

Additional correlational post-hoc analyses were performed to characterize the relationships among the PaCKS, the METER, and the EPCQ.

RESULTS

Demographics

See Table 1 for demographic information on the 614 participants in the professional and community samples. The mean age of the sample was 45.34 (SD = 18.09) and the mean years of education 15.49 (SD = 3.12). Participants were primarily women (61.2%), Caucasian (80.3%), and married (50.6%). The professional sample had a mean age of 50.17 (SD = 10.31). Their mean years of palliative care experience was 9.13 (SD = 5.20). Multiple healthcare backgrounds were represented, including medical doctors, nurses, social workers, psychologists, and chaplains.

Initial Item Screening

PaCKS items 22, 24, 27, 34, and 36 (see Appendix B) were included in the EFA but flagged as candidates for deletion because fewer than 90% of palliative care professionals answered them correctly. Item 26 was also flagged for deletion because of its content similarity to item 27. No items achieved a ceiling (i.e., were answered correctly by more than 95% of respondents) within the community sample, so all were retained at this phase.

Exploratory and Confirmatory Factor Analysis

The principal components analysis revealed 11 factors with eigenvalues greater than 1.0. Given the intended overlap of content among items during the scale development process as well as the lack of theory and utility for a scale with 11 factors and fewer than 38 items (Preacher et al., Reference Preacher, Zhang and Kim2013), this structure was rejected. Instead, the one-, two-, three-, and four-factor solutions were evaluated, with the expectation that some items might need to be eliminated because they were the source of the excessive factors. The two-factor solution suggested that participants approached the PaCKS with response biases, as the two factors appeared to reflect which items had correct true answers versus correct false answers. The three- and four-factor solutions followed this pattern as well. Given the indication of participant response bias in the EFA, a bifactor solution with bias modeled and accounted for was attempted in a CFA with the first half of the full sample. In a bifactor model, each item loads on two factors: a general factor as well as a specific subfactor that can take into account additional dependencies among items (Li et al., Reference Li, Bolt and Fu2006). A bifactor model can be useful in accounting for nuisance dimensionality from response bias or method variance (Cho et al., Reference Cho, Cohen and Kim2014).

In conducting a bifactor CFA for the PaCKS, each item was modeled on a primary factor labeled “knowledge of palliative care” as well as a secondary factor labeled “bias.” Unlike a traditional bifactor model, all items loaded on both the main factor and the secondary factor in this model. This was achieved by fixing factor loadings for the secondary factor of true items at 1 and false items at –1 in order to account for the bias observed in the EFA. This new solution not only conceptually accounted for the bias, but preliminary indices of fit suggested that it also fit the factor structure of the data reasonably well. The bias subfactor in this scale represents people's tendency to answer all items as true or all items as false, independent of their knowledge about palliative care. The bias subfactor did not routinely correlate largely or significantly with hypothesized similar constructs (see below). All subsequent analyses utilized the bifactor model.

Individual items were evaluated and considered for deletion based on a combination of indicators: a high percentage (greater than 10%) of incorrect responses from the professionals, the lowest R 2 values, the lowest factor loadings, and content redundancy. Items were deleted one at a time throughout the data analysis process. A final 13-item one-factor solution with bias modeled was selected based on significant loadings for all items and excellent indices of fit: CFI = 0.97, TLI = 0.96, RMSEA = 0.031, and 90% confidence interval (CI 90%) = (0.00, 0.049). In order to cross-validate this solution, a CFA of the 13 items was conducted in the second subsample (n = 345). Results from the cross-validation CFA also revealed a similar pattern of significant loadings and excellent indices of fit: CFI = 0.99, TLI = 0.98, RMSEA = 0.022, and CI 90% = (0.00, 0.040). See Table 2 for the factor loadings and the value of R 2 for each item on the final 13-item scale for the full sample.

Table 2.

Factor loadings and R2 in the final factor model for final 13-item PaCKS

** p < 0.05.

Reliability of the PaCKS

The KR-20 value for the final 13-item version of the PaCKS was 0.71, which indicated adequate internal consistency among items (Streiner, Reference Streiner2003). No items were deleted based on this analysis. An ICC (2,1) test was conducted on PaCKS total scores at times 1 and 2 for the control participants who had complete PaCKS data (n = 72). The ICC value was found to be 0.70 (CI 95% = [0.56, 0.80]), which indicated good stability across the 10-minute-apart administrations (Terwee et al., Reference Terwee, Bot and de Boer2007, Weir, Reference Weir2005).

Validity of the PaCKS

Table 3 lists all correlations, both the Pearson correlations and structural equation modeling correlations, between PaCKS total score and the bias subfactor and each item of the EPCQ, METER total, and Shipley total in the full sample. The PaCKS was significantly and positively correlated with all the EPCQ items except for the item that asked about experience with hospice care. The main factor of the PaCKS and the METER were correlated at 0.37 (p < 0.001), and the bias factor correlated with the METER at –0.18 (p < 0.001). The main factor of the PaCKS and the Shipley correlated at 0.47 (p < 0.001), and the bias factor correlated at –0.14 (p = 0.06). These results suggest that the main factor of the PaCKS is moderately and positively correlated with health literacy and intelligence. The significant negative correlation of the bias factor with the METER and the nearly significant negative association with the bias factor and the Shipley score suggests that individuals with low health literacy and lower knowledge have a tendency, unrelated to knowledge about palliative care, to answer PaCKS questions as true, regardless of the correctness of such an answer.

Table 3.

Correlations between the 13-item PaCKS and other variables to assess validity

* p < 0.05, ** p < 0.01.

Professionals' total scores on the 13-item PaCKS was compared to the 30 community participants with the lowest METER scores who also reported that they had not heard of palliative care. There was a significant difference in PaCKS scores between professionals (M = 12.73, SD = 0.58) and community members with low METER scores who said they had never heard of palliative care (M = 8.70, SD = 2.74, t(55) = 7.86, p < 0.001, Cohen's d = 0.81), indicating a large effect size. A second post-hoc independent-samples t test was conducted to compare PaCKS scores between the professionals and the full community sample. There was a significant difference in PaCKS scores between professionals (M = 12.73, SD = 0.58) and community members (M = 10.92, SD = 2.2, t(83.87), p < 0.001). In all cases, as would be expected, palliative care professionals knew significantly more than laypeople.

DISCUSSION

The objective of this study was to develop a scale to assess knowledge about palliative care in a community sample. Following several steps to generate a preliminary set of items, subsequent analyses were employed to create a final 13-item self-administered scale—the PaCKS—that meets or exceeds psychometric standards for scale development. The strong indices of fit in the confirmatory bifactor model support the fact that the PaCKS has one general factor measuring knowledge of palliative care. In addition, the PaCKS is reliable and stable. The scores are consistent, at least over a short time period, and scale items have good internal consistency. The scale also demonstrates excellent construct, convergent, and discriminative validity.

The final version of the PaCKS has 13 true/false items that cover a broad range of topics within the construct of palliative care, including goals of palliative care, members of the palliative care team, system-related components of palliative care, timing of palliative care, and the symptoms that palliative care addresses. Given its broad content coverage, the PaCKS would be useful in order to gain a preliminary assessment of individuals' gaps in knowledge about palliative care.

Our study had several limitations that are important to note. First, the data were collected via the internet, and participants could have looked up information when answering the PaCKS. Though the internet, and mTurk in particular, has been used for social sciences research and shown to be a useful way to collect data (e.g., Berinsky et al., Reference Berinsky, Huber and Lenz2012; Buhrmester et al., Reference Buhrmester, Kwang and Gosling2011), it is possible that some participants utilized the internet to look up information about palliative care despite receiving specific instructions not to. Given that the mTurk sample did not achieve a ceiling on the PaCKS, and that the results on the Shipley test were similar to those from previous studies (Edworthy et al., Reference Edworthy, Devins and Watson1995), it is unlikely that the majority of the sample inflated their scores by looking up answers.

As a scale, the PaCKS itself also has some potential limitations. The true/false format, though efficient, forces respondents to guess on items they do not know the answers to or about which they are unsure. Some degree of guessing is expected in all knowledge-dependent tests, but guessing was common enough in our sample that a separate factor was needed to account for it. In practice, the fact that the scale is almost entirely balanced across correct true and false answers means that this bias has a minimal impact on the meaning of scores (as evidenced by the fact that the score and the factor had the same pattern of correlations with other measures). Future research could consider administering the PaCKS with an “I don't know” response option to provide additional information about how often and for which items people are likely to guess on the PaCKS. Notably, however, this option would complicate modeling responses via SEM.

Creating a scale to measure knowledge about palliative care was a critical step to understanding and combating potential barriers to access and utilization of this life-improving service. Prior research in palliative care access and utilization has not assessed knowledge about palliative care, though many studies suggest that knowledge deficits contribute to underutilization of the service (e.g., Brickner et al., Reference Brickner, Scannell and Marquet2004; Ensor & Cooper, Reference Ensor and Cooper2004; Grossman & Kaestner, Reference Grossman, Kaestner, Behrman and Stacey1997; Ogle et al., Reference Ogle, Mavis and Wyatt2002). With the successful development of our scale, new research exploring how knowledge about palliative care influences access and utilization of the service is possible. The PaCKS has the potential to be an integral and useful tool in the clinical, research, and public health contexts in order to identify knowledge deficits in palliative care that may contribute to the systemic underutilization of this life-improving service.

ACKNOWLEDGMENTS

This research received funding from the Department of Psychology at Washington University.

APPENDIX A

Experience and Familiarity with Palliative Care Questionnaire

  1. 1. Have you heard of palliative care?

    Yes    No

  2. 2. How much do you agree with this statement: “I am familiar with palliative care”?

    • 1 – Strongly disagree

    • 2 – Disagree

    • 3 – Neutral

    • 4 – Agree

    • 5 – Strongly agree

  3. 3. How much do you agree with this statement: “I have experience with palliative care”?

    • 1 – Strongly disagree

    • 2 – Disagree

    • 3 – Neutral

    • 4 – Agree

    • 5 – Strongly agree

  4. 4. Select any number between 1 and 10 to indicate how much knowledge you think you have about palliative care.

    • 1 – I know nothing at all

    • 2

    • 3

    • 4

    • 5 – I have some knowledge

    • 6

    • 7

    • 8

    • 9

    • 10 – I am very knowledgeable

  5. 5. Have you heard of hospice care?

    Yes    No

  6. 6. How much do you agree with this statement: “I am familiar with hospice care”?

    • 1 – Strongly disagree

    • 2 – Disagree

    • 3 – Neutral

    • 4 – Agree

    • 5 – Strongly agree

  7. 7. How much do you agree with this statement: “I have experience with hospice care”?

    • 1 – Strongly disagree

    • 2 – Disagree

    • 3 – Neutral

    • 4 – Agree

    • 5 – Strongly agree

  8. 8. Select any number between 1 and 10 to indicate how much knowledge you think you have about hospice care:

    • 1 – I know nothing at all

    • 2

    • 3

    • 4

    • 5 – I have some knowledge

    • 6

    • 7

    • 8

    • 9

    • 10 – I am very knowledgeable

APPENDIX B

Preliminary Items of the Palliative Care Knowledge Scale (PaCKS)

  1. 1. One goal of palliative care is to addresses any psychological issues brought up by serious illness. (T)

  2. 2. Addressing feelings of depression brought up by serious illness is not a part of palliative care. (F)

  3. 3. Stress from serious illness can be addressed by palliative care. (T)

  4. 4. Palliative care focuses exclusively on physical symptoms. (F)

  5. 5. Palliative care can help people manage the side effects of their medical treatments. (T)

  6. 6. One goal of palliative care is to cure serious illness. (F)

  7. 7. When people receive palliative care, they must stop treatments aimed at curing their illness. (F)

  8. 8. One goal of palliative care is to address any spiritual issues associated with serious illness (T)

  9. 9. One goal of palliative care is to address any social issues related to having a serious illness, such as community involvement and relationships. (T)

  10. 10. Palliative care is not designed to address practical issues (e.g., housing, transportation, finances, and insurance) associated with serious illness. (F)

  11. 11. When people receive palliative care, they must give up their other doctors. (F)

  12. 12. Hospice care is another name for palliative care. (F)

  13. 13. People can receive palliative care at any time during a serious illness. (T)

  14. 14. Palliative care is exclusively for people who are in the last six months of life. (F)

  15. 15. Palliative care is specifically for people with cancer. (F)

  16. 16. Palliative care can have the most impact when it starts at the time a serious illness is diagnosed. (T)

  17. 17. People can receive palliative care at home. (T)

  18. 18. People must be in the hospital to receive palliative care. (F)

  19. 19. Palliative care is designed specifically for older adults. (F)

  20. 20. A healthcare provider must be a physician in order to be on a palliative care team. (F)

  21. 21. Palliative care is a team-based approach to care. (T)

  22. 22. There is a specific test that doctors have to pass in order to become a palliative care doctor (T)

  23. 23. One goal of palliative care is to help people better understand their treatment options. (F)

  24. 24. Palliative care helps people decide if and when to stop treatments aimed at curing their illness. (T)

  25. 25. Palliative care encourages people to stop treatments aimed at curing their illness. (F)

  26. 26. Medicare covers some of the costs associated with palliative care. (T)

  27. 27. Medicaid covers some of the costs associated with palliative care. (T)

  28. 28. One goal of palliative care is to improve a person's ability to participate in daily activities. (T)

  29. 29. Improving communication among people, their family, and their healthcare team is a focus of palliative care. (T)

  30. 30. People can learn more about their illness from palliative care professionals (T)

  31. 31. People with serious illness who receive palliative care generally have higher medical costs than people who do not receive palliative care. (F)

  32. 32. Palliative care is not designed to address the needs of family members of people with serious illness. (F)

  33. 33. Palliative care helps the whole family cope with a serious illness. (T)

  34. 34. One goal of palliative care is to prolong life for people with serious illness. (F)

  35. 35. Palliative care is for anyone with a serious illness. (T)

  36. 36. Palliative care ends when a person with serious illness dies. (F)

  37. 37. In order to receive palliative care, people must have been diagnosed with a serious illness for at least two months. (F)

  38. 38. In order to receive palliative care, people must have been hospitalized for their serious illness. (F)

APPENDIX C

Final Palliative Care Knowledge Scale (PaCKS)

  1. 1. One goal of palliative care is to address any psychological issues brought up by serious illness. (T)

  2. 2. Stress from serious illness can be addressed by palliative care (T)

  3. 3. Palliative care can help people manage the side effects of their medical treatments. (T)

  4. 4. When people receive palliative care, they must give up their other doctors. (F)

  5. 5. Palliative care is exclusively for people who are in the last six months of life. (F)

  6. 6. Palliative care is specifically for people with cancer. (F)

  7. 7. People must be in the hospital to receive palliative care. (F)

  8. 8. Palliative care is designed specifically for older adults. (F)

  9. 9. Palliative care is a team-based approach to care. (T)

  10. 10. One goal of palliative care is to help people better understand their treatment options. (T)

  11. 11. Palliative care encourages people to stop treatments aimed at curing their illness. (F)

  12. 12. One goal of palliative care is to improve a person's ability to participate in daily activities. (T)

  13. 13. Palliative care helps the whole family cope with a serious illness. (T)

References

REFERENCES

Berinsky, A.J., Huber, G.A. & Lenz, G.S. (2012). Evaluating online labor markets for experimental research: Amazon.com's Mechanical Turk. Political Analysis, 20(3), 351–368.Google Scholar
Bentler, P.M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246.Google Scholar
Born, W., Greiner, A., Sylvia, E., et al. (2004). Knowledge, attitudes and beliefs about end-of-life care among inner-city African Americans and Latinos. Journal of Palliative Medicine, 7(2), 247–256.Google Scholar
Brickner, L., Scannell, K., Marquet, S., et al. (2004). Barriers to hospice care and referrals: Survey of physicians' knowledge, attitudes, and perceptions in a health maintenance organization. Journal of Palliative Medicine, 7(3), 411–417.Google Scholar
Buhrmester, M., Kwang, T. & Gosling, S. D. (2011). Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6(1), 3–5. Available from http://datacolada.org/wp-content/uploads/2014/04/Burhmester-Kwang-Gosling-2011.pdf.Google Scholar
Center to Advance Palliative Care (2014). Palliative care facts and stats. Available from https://media.capc.org/filer_public/68/bc/68bc93c7-14ad-4741-9830-8691729618d0/capc_press-kit.pdf.Google Scholar
Cho, S.J., Cohen, A.S. & Kim, S.H. (2014). A mixture group bifactor model for binary responses. Structural Equation Modeling: A Multidisciplinary Journal, 21(3), 375–395.Google Scholar
Clark, L.A. & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309–319. Available from http://www.personal.kent.edu/~dfresco/CRM_Readings/Clark_and_Watson_1995.pdf.Google Scholar
Davis, T.C., Crouch, M.A., Long, S.W., et al. (1991). Rapid assessment of literacy levels of adult primary care patients. Family Medicine, 23(6), 433–435.Google Scholar
Dow, L.A., Matsuyama, R.K., Ramakrishnan, V., et al. (2010). Paradoxes in advance care planning: The complex relationship of oncology patients, their physicians, and advance medical directives. Journal of Clinical Oncology, 28(2), 299–304. Epub ahead of print Nov 23, 2009. Available from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2815718/pdf/zlj299.pdf.Google Scholar
Edworthy, S., Devins, G. & Watson, M. (1995). The Arthritis Knowledge Questionnaire: A test for measuring patient knowledge of arthritis and its self management. Arthritis and Rheumatism, 38(5), 590–600.Google Scholar
Ensor, T. & Cooper, S. (2004). Overcoming barriers to health service access: Influencing the demand side. Health Policy and Planning, 19(2), 69–79. Available from http://heapol.oxfordjournals.org/content/19/2/69.long.Google Scholar
Fischer, S., Gozansky, W., Kutner, J., et al. (2003). Palliative care education: An intervention to improve medical residents' knowledge and attitudes. Journal of Palliative Medicine, 6(3), 391–399.Google Scholar
Grewal, R., Cote, J.A. & Baumgartner, H. (2004). Multicollinearity and measurement error in structural equation models: Implications for theory testing. Marketing Science, 23(4), 519–529. Technical appendices and supplementary tables available from http://www.personal.psu.edu/rug2/GCB%20MKS%20Supplementary%20Technical%20Appendices%20Final.pdf.Google Scholar
Grossman, M. & Kaestner, R. (1997). Effects of education on health. In The social benefits of education. Behrman, J.R. & Stacey, N. (eds.), pp. 69–123. Ann Arbor: University of Michigan Press.Google Scholar
Hu, L. & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55.Google Scholar
Kaiser, H.F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141–151.Google Scholar
Kline, T.J.B. (2005). Psychological testing: A practical approach to design and evaluation. Thousand Oaks, CA: Sage Publications.Google Scholar
Koffman, J., Burke, G., Dias, A., et al. (2007). Demographic factors and awareness of palliative care and related services. Palliative Medicine, 21(2), 145–153.Google Scholar
Kozlov, E. & Carpenter, B.D. (2015). “What is palliative care?” Variability in content of palliative care informational web pages. The American Journal of Hospice & Palliative Care, pii: 1049909115615566. Epub ahead of print Nov 5.Google Scholar
Li, Y., Bolt, D.M. & Fu, J. (2006). A comparison of alternative models for testlets. Applied Psychological Measurement, 30(1), 3–21.Google Scholar
Meekin, S., Klein, J., Fleischman, A., et al. (2000). Development of a palliative education assessment tool for medical student education. Academic Medicine, 75(10), 986–992.Google Scholar
Morrison, S., Maroney-Galin, C., Kralovec, P., et al. (2005). The growth of palliative care programs in United States hospitals. Journal of Palliative Medicine, 8(6), 1127–1134.Google Scholar
Muthén, L.K. & Muthén, B.O. (2010). Mplus user's guide: Statistical analysis with latent variables. Users' Guide. Los Angeles: Muthén & Muthén.Google Scholar
National Hospice and Palliative Care Organization (2012). Hospice care in America. Available from http://www.nhpco.org/sites/default/files/public/Statistics_Research/2012_Facts_Figures.pdf.Google Scholar
Ogle, K.S., Mavis, B. & Wyatt, G.K. (2002). Physicians and hospice care: Attitudes, knowledge, and referrals. Journal of Palliative Medicine, 5(1), 85–92.Google Scholar
Preacher, K.J., Zhang, G., Kim, C., et al. (2013). Choosing the optimal number of factors in exploratory factor analysis: A model selection perspective. Multivariate Behavioral Research, 48(1), 28–56. Available from http://quantpsy.org/pubs/preacher_zhang_kim_mels_2013.pdf.Google Scholar
Public Opinion Strategies (2011). Public opinion research on palliative care: A report based on research by Public Opinion Strategies. Available from https://media.capc.org/filer_public/18/ab/18ab708c-f835-4380-921d-fbf729702e36/2011-public-opinion-research-on-palliative-care.pdf.Google Scholar
Rawson, K., Gunstad, J., Hughes, J., et al. (2010). The METER: A brief, self-administered measure of health literacy. Journal of General Internal Medicine, 25(1), 67–71. Available from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2811598/pdf/11606_2009_Article_1158.pdf.Google Scholar
Ross, M., McDonald, B. & McGuinness, J. (1996). The Palliative Care Quiz for Nursing (PCQN): The development of an instrument to measure nurses' knowledge of palliative care. Journal of Advanced Nursing, 23, 126–137. Available from http://prc.coh.org/pdf/pcqn.pdf.Google Scholar
Schulman-Green, D., Ercolano, E., Jeon, S., et al. (2012). Validation of the knowledge of care options instrument to measure knowledge of curative, palliative, and hospice care. Journal of Palliative Medicine, 15(10), 1091–1099. Epub ahead of print Jun 13.Google Scholar
Shipley, W.C. (1940). A self-administering scale for measured intellectual impairment and deterioration. Journal of Psychology: Interdisciplinary and Applied, 9(2), 6–14.Google Scholar
Stanley, JC. & Hopkins, K.D. (1972). Educational and psychological measurement and evaluation. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
Steiger, J.H. & Lind, J.C. (1980). Statistically based tests for the number of common factors. Handout for a talk presented at the Annual Spring Meeting of the Psychometric Society, Iowa City, Iowa, May 30, 1980. Vol. 758, pp. 424–453. Available from http://www.statpower.net/Steiger%20Biblio/Steiger-Lind%201980.pdf.Google Scholar
Streiner, D.L. (2003). Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80(1), 99–103.Google Scholar
Tabachnick, B.G. & Fidell, L.S. (2001). Using mulitvariate statistics, 3rd ed. Needham Heights, MA: Allyn and Bacon.Google Scholar
Terwee, C.B., Bot, S.D., de Boer, M.R., et al. (2007). Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology, 60(1), 34–42. Epub ahead of print Aug 24, 2006.Google Scholar
Tucker, L.R. & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38(1), 1–10.Google Scholar
Veterans Health Administration (2008). Palliative care consultation teams (VHA Directive 2008–066). Available from http://www1.va.gov/vhapublications/ViewPublication.asp?pub_ID=1784.Google Scholar
Weir, J.P. (2005). Quantifying test–retest reliability using the intraclass correlation coefficient and the SEM. The Journal of Strength and Conditioning Research, 19(1), 231–240. Available from http://journals.lww.com/nsca-jscr/Abstract/2005/02000/QUANTIFYING_TEST_RETEST_RELIABILITY_USING_THE.38.aspx.Google Scholar
Figure 0

Table 1. Demographic information

Figure 1

Table 2. Factor loadings and R2 in the final factor model for final 13-item PaCKS

Figure 2

Table 3. Correlations between the 13-item PaCKS and other variables to assess validity