Attempts to shorten long waiting times for scheduled services have had substantial focus on adding resources and improving supply-side management. While this is necessary for improving access, it is insufficient. It becomes ever more important to evaluate and manage the demand for services and to explore the appropriateness of interventions for which waiting time benchmarks have been proposed and care guarantees are possible.
While appropriateness is a commonly used term in health care, there is no standardized operational definition or uniform understanding. At its core is the idea that appropriate care is associated with net benefit, namely maximizing the benefit and minimizing the risk the patient receives from that care. Furthermore, in the natural history of any condition or its treatment, there is a point before which it is inappropriate to intervene or provide a service. Correspondingly, there is a point beyond which service becomes inappropriate, largely because the potential for benefit is lost. Appropriateness, therefore, carries with it direct and indirect implications for the right procedure, for the right person at the right time and setting. At the same time, attention to appropriateness also raises the question, from whose perspective—patient's, physician's, policy makers’. It is reasonable to assume the answer would vary with who is being asked. The investigators of the Western Canada Waiting List project conducted this scoping review and concept analysis of appropriateness to sharpen our understanding of the term, as ground-work for future intervention-specific research. Specifically, the objectives of the study were, in the context of healthcare delivery: (i) To identify definitions, conceptual models, and frameworks currently existing for the concept of appropriateness; (ii) To identify methods for determining appropriateness; and (iii) To identify the range of stakeholder perspectives on appropriateness.
METHODS
Our intent was to broadly scope the literature, not conduct a detailed and conclusive synthesis. Hence, a targeted search of peer-reviewed literature was conducted to identify studies related to the concept of appropriateness in the context of healthcare delivery. The search was conducted using MEDLINE for years 1966 to 2006.
The search was guided by a comprehensive list of search terms including, for example, appropriate, concept, delivery of health care and health services. Citation searches, using Web of Science, were run on all studies tagged for possible inclusion during the abstract screening process to locate any additional studies of interest.
Abstract and Full-Text Screening
A screening tool was developed to aid the review and selection of titles/abstracts. The tool consisted of a set of inclusion and exclusion criteria that were specifically related to the study objectives. The tool was pilot tested on a subset of abstracts (n = 40) by two raters and minor refinements were made as a result. Two independent raters conducted the first screen of titles/abstracts. In the case of disagreement between raters, consensus was reached through discussion and, if necessary, a third reviewer. Titles/abstracts that met at least one of the inclusion criteria were screened in.
A similar approach was used to screen the full-text article for those titles/abstracts that met at least one of the inclusion criteria. A second screening tool was developed that explicitly incorporated the study objectives, as well as the inclusion and exclusion criteria from the first screening tool. In addition, the raters classified the full-text articles into high or low relevancy: articles that addressed at least one objective and that were broadly applicable to the full range of healthcare delivery were deemed highly relevant; articles that addressed at least one objective but in the context of a specific procedure were considered of low relevance. The second screening tool was pilot tested on a subset of full-text articles (n = 5) and minor refinements were made as a result (first and second screening tools available from authors).
The second screen was conducted by two independent raters. Full-text articles deemed to be highly relevant were included for data extraction. Disagreements were discussed by the two raters to reach consensus.
Data Extraction
A data extraction process and form was developed to aid in the synthesis of information from highly relevant articles. This form included fields related to the specific definition of appropriateness, the perspective from which it was considered, concepts and constructs of appropriateness, methods to assess or determine appropriateness, the role of stakeholders and related terms and concepts. Also, the type of health service that was addressed in the article (i.e., surgical, diagnostic) was required to examine potential differences in concepts/constructs across the range of healthcare services and delivery. Two raters conducted the data extraction on a subset of articles (n = 3) to ensure appropriate components of the form and consistency in language and understanding. Data extraction for each article was completed by one of the raters.
RESULTS
A total of 2,829 titles and abstracts were initially reviewed (Figure 1). As a result of the initial screen, 124 titles/abstracts were retained and the full article was retrieved. After this second screen, thirty-seven articles were identified as addressing the specific research objectives and deemed highly relevant for data extraction and interpretation.
Scoping review process.
Definitions of Appropriateness
There were fourteen papers identified that provided a definition of appropriateness. In general, appropriateness was defined in terms of positive effects for patients (Table 1). Woodward and Warren-Boulton defined appropriate care as that which is expected to contribute to patients’ health in a positive manner (Reference Woodward and Warren-Boulton34). The notion of positive effects was more explicitly defined by researchers at the RAND Corporation, with the introduction of risks and benefits. According to RAND, “a procedure was considered to be performed for appropriate reasons to the extent that its expected health benefits exceeded the expected negative consequences by a sufficiently wide margin that the procedure was worth doing.” (Reference Leape, Park and Solomon22, p. 669) This definition applies to an “average patient” presenting to an “average physician” (Reference Hicks15). Conversely, a procedure was deemed inappropriate when the risks exceeded the benefits. Important to note is the fact that at this time, costs were not considered in the definition of appropriateness. This Review indicated that the RAND definition is clearly one of the most commonly used definitions of appropriateness (Reference Brook and Kamberg5;Reference Buetow, Sibbald and Cantrill6; Reference Fraser, Pilpel and Kosecoff12;Reference Hampton14;Reference Hicks15;Reference Kahan, Bernstein and Leape17;Reference McClellan and Brook23;Reference Sharpe29;Reference Shekelle31).
Summary of Results Related to Definitions of Appropriateness
Over time, the notion of appropriateness has become more directly linked to cost. This has likely been due, in part, to the increasing need to address escalating healthcare costs. This shift was described by Sharpe (Reference Sharpe29) in her review of the politics, economics and ethics of appropriateness. She noted that before the 1980s in the United States, the term “appropriate” was relatively uncontroversial. Appropriate care was understood to be care that could serve the best interests of the individual patient. The economic landscape, however, changed from one of perceived abundance to one of perceived scarcity and third-party payers had a greater need to restrict reimbursement for care provided. Accordingly, consideration of cost was increasingly associated with defining appropriateness. Appropriateness was seen by some as occurring within the context of cost-effectiveness and the provision of services within “a nexus of limited resources” (p. 338). Buetow et al. (Reference Buetow, Sibbald and Cantrill6) defined appropriateness as “the outcomes of a process of decision making that maximizes net individual health, within society's available resources” (p. 264).
Others argue that despite efforts to define appropriateness in terms of benefits and risks, the concept remains elusive and difficult to define. After a brief review of evidence on methods to define appropriateness of care and results, Naylor (Reference Naylor24) suggested that defining what is appropriate care “depends on which clinicians are asked, where they live and work, what weight is given to different types of evidence and end-points, whether one considers the preferences of patients and families, the level of resources in a given health system and the prevailing values of both the system and the society in which it operates” (Reference Naylor24, p. 1920).
Conceptual Models and Frameworks for Appropriateness
This scoping review identified two conceptual frameworks which illuminate the various constructs and concepts of appropriate care. Lavis and Anderson (Reference Lavis and Anderson19) proposed a framework identifying appropriateness of service and appropriateness of setting in which health care is provided. Appropriateness of a service is defined as the “effectiveness of that service for a particular type of patient” (p. 322) In this case, appropriateness is determined on the basis of clinical information regarding patients’ symptoms and results of diagnostic tests that lead to an intervention, which is expected to benefit the patient. As a result, the authors adopted a definition similar to that of the RAND, in which appropriateness of services is defined as one that is “expected to do more good than harm for a patient with a given set of indications.” Cost is traditionally not considered. Appropriateness of setting is determined “by whether the patient's clinical characteristics, and the services required for his or her care, match the setting in which the care is provided”. In this case, setting is used as a proxy measure for the amount of resources used to provide care. Accordingly, appropriateness of setting considers cost-effectiveness.
A broader conceptual model of appropriateness for surgical services was introduced in 1996 by Sharpe and Faden (Reference Sharpe and Faden30). The authors identified several key constructs which help illuminate the concept, including the notion that appropriateness should not be seen as a dichotomous construct (i.e., appropriate versus inappropriate) but, rather, may be better represented as a spectrum of care. The authors also urged the need to consider appropriateness within the context of both under- as well as over-utilization. Moreover, there is the notion that there are at least three key perspectives that should be considered: clinical, patient, and societal.
Perspectives on Appropriateness
Clinical perspectives focus primarily on the application of evidence to determine the level of benefit for a patient. Clinical recommendations to proceed with treatment are based on at least four criteria: the patient's clinical profile; the physician's (and team's) skill; the quality of the evidence supporting a procedure; and, the procedure's clinical benefit:harm ratio. Sackett (Reference Sackett28) used a “level of evidence” approach to identify various levels of the harm:benefit ratio. Among procedures thought to be preponderantly beneficial (harmful), interventions considered possibly beneficial (harmful) are those for which evidence is based largely on case reports or uncontrolled clinical impressions or studies—that is, the weakest level of evidence. The lowest level of evidence may also be used to support equivocal interventions—those whose ratio of benefits to harm is roughly equal. Indicated (contraindicated) interventions are those supported by a higher level of evidence, including at least one RCT or a broad meta-analysis of high quality studies. Highly indicated (contraindicated) interventions are those procedures supported by the highest levels of evidence, including a definitive or replicated RCT, and whose expected benefits exceed benefits from other alternative therapies (Reference Sackett28).
Patient perspectives can be incorporated into the determination of appropriateness in two ways. First, empirical information on outcomes of care for the average patient should be considered to determine whether the treatment/service is expected to generally benefit the patient. This evidence is primarily available in effectiveness studies focusing on a broad range of patient outcomes, including clinical outcomes, quality of life and satisfaction. However, the perspectives of individual patients must also be considered. Sharpe and Faden (Reference Sharpe and Faden30) argue that the relevance of these clinical outcomes must be considered by individual patients and that each patient's assessment of these outcomes will be affected by their values, risk preferences, financial and social circumstances. This process is required to determine the second aspect of the patient perspective—the desirability of care. A procedure is considered desirable if it is “freely accepted by the informed patient or valid surrogate” (p. 128).
Societal perspectives of appropriateness have been advocated as a consequence of growing concerns regarding escalating healthcare costs. As a society, there must be some consideration of the type of healthcare goals we are willing to pursue and those we are not. This is fundamentally based on values held, rather than facts that can be provided by scientific evidence. The societal perspective focuses primarily on the cost-worthiness of a treatment, based on considerations of social distribution, cost containment, and rationing.
Methods for Determining Appropriateness
The most widely used method to assess appropriate delivery of healthcare services is that developed by the RAND Corporation (Reference Casparie8;Reference Hicks15;Reference Kosecoff, Fink, Chassin and Brook18;Reference Lazaro and Fitch20;Reference Shekelle31). Most other methods are variations of this. RAND and the University of California, Los Angeles (UCLA) developed the appropriateness method (RAM) in the 1980s, in an effort to determine and identify which types of health care were overused and/or underused. Ideally, health systems seek to reduce inappropriate care and maintain or increase appropriate and necessary care. The RAM is based on the RAND definition of appropriateness, as previously discussed (i.e., net clinical benefit for patients) (Reference Brook, Chassin and Fink4;Reference Park, Fink and Brook25).
In theory, defining and operationalizing the concept of appropriate healthcare services should be easily achieved based on existing evidence about what services ought to be provided. In most cases, however, the existing evidence is limited and focused primarily on efficacy—evidence from highly controlled studies conducted on specific and selected population, the results of which are not always generalizable across the patient population (Reference Kosecoff, Fink, Chassin and Brook18).
In an effort to overcome the limits of available evidence, the RAND Corporation developed a multi-step process that incorporates both the existing evidence and the results of clinical consensus among key experts. The literature review identifies relevant published literature relating to the efficacy, effectiveness, complications, and risks of the procedure being rated. This is followed by a synthesis of the evidence into a summary for each aspect (i.e., efficacy) of the procedure. The next step is to develop a set of very specific clinical indications that encompass all potential uses of the procedure, both appropriate and inappropriate. A panel of [9–12] experts is selected and follows a two-round process to classify the appropriateness of an intervention, using a modified Delphi approach.
If necessity criteria are an objective of the study, a third round of ratings occurs, at which panelists rate the necessity of indications using four criteria, based on the RAM definition: the procedure is appropriate; the physician is obliged to offer the procedure to the patient; it is likely the procedure will benefit the patient; and, the magnitude of that benefit is not trivial.
Related Terms and Concepts
Appropriateness is linked to other terms and concepts, such as small-area variation, medical necessity, critical services and unnecessary surgery. Landmark research conducted by Wennberg et al. (Reference Wennberg, Gittelsohn and Shapiro33) focused attention on large variations in the rates of commonly performed surgeries within small geographic areas. One of the most commonly cited explanations was inappropriate use of services—that is, the high rates of service provision in some areas were the result of inappropriate care delivery (Reference Casparie8;Reference Lazaro and Fitch20;Reference Wennberg32;Reference Wennberg, Gittelsohn and Shapiro33). The evidence to date indicates that there is no clear and consistent relationship between small-area variation and appropriateness. Differences in the rates of service provision may be the result of various factors associated with patients (characteristics, comorbidity), physicians (practice style, method of payment), institutions, and environments (Reference Casparie8;Reference Greer, Goodwin and Freeman13).
Appropriateness has been frequently defined or described in relation to medical necessity. Medical necessity has been commonly used in the United States by third-party payers to define reimbursement criteria and in Canada to define the services that should be provided under the publicly funded system (Reference Bergthold3;Reference Caulfield9–Reference Ford11;Reference Jacobson, Asch, Glassman, Model and Hernandez16). Charles et al. (Reference Charles, Lomas and Giacomini10) provide a common sense definition—medically necessary services are those that “a patient needs to avoid a negative health consequence” (p. 365).
Similarly, appropriateness has been linked to the concept of necessary or crucial services (Reference Kahan, Bernstein and Leape17;Reference Shekelle31). Services are considered necessary or crucial “for a patient presenting with certain indications suggesting that the physician is obliged to recommend that treatment because the physician believes that withholding the treatment would be deleterious to the patient's health” (Reference Kahan, Bernstein and Leape17, p. 358). Services are deemed necessary or crucial if they meet the following criteria: the procedure is appropriate (as defined using the RAND method); it would be improper not to recommend the service; there is reasonable chance that the patient will benefit from the service (i.e., procedures which have a low likelihood of benefit are not considered necessary); and, the benefit to the patient is not small (i.e., procedures providing only a minor benefit are not considered necessary or critical). Clearly, the notion of necessary or crucial services represents an extension of the appropriateness concept. A noted difference between the two is that appropriateness represents an exclusionary criteria (i.e., do not perform the service if it is deemed inappropriate), whereas necessity is an inclusion criteria (i.e., use the procedure if deemed crucial) (Reference Kahan, Bernstein and Leape17).
Finally, appropriateness is indirectly linked to the concept of unnecessary surgery. The terms are similar in that judgment is made based on patient well-being: a necessary operation is one that improves patient well-being; an unnecessary operation is one that makes the patient worse off. According to Pauly (Reference Pauly26), this judgment is made by the “fully informed” patient, requiring as much knowledge and understanding of the possible costs and consequences as the recommending physician. Leape (Reference Leape21) suggests that only the patient can know what is necessary for him/her based on his/her values—how he/she weighs the perceived benefits against the costs (including tolerance of risk, pain or disability, fear of surgery, and preferred lifestyle).
DISCUSSION
The objective of this Review was to identify definitions, conceptual models, frameworks, methods and perspectives on appropriateness in the context of healthcare delivery. While there is no universally accepted definition of appropriateness, the concept is most often defined in terms of net clinical benefit for patients, albeit there are no clear guidelines regarding the magnitude of the benefit. In some cases, appropriateness was described in terms of a continuum as opposed to a dichotomous construct. The review process identified several related terms and concepts, such as small-area variation, medical necessity, crucial or necessary services and unnecessary surgery.
In terms of models and frameworks, appropriateness has been conceptualized and assessed primarily from a clinical perspective. In many cases, these methods fail to include properly the perspectives of key stakeholders. Since the mid- to late-1990s, there have been increasing calls for the involvement of a broader range of input to the determination of appropriate care—most notably from the perspective of patients.
While the notion of clinical benefit is clearly a necessary criterion, patients’ preferences and expectations are also important considerations. Nevertheless, appropriateness criteria are developed for the “average” patient, thus ignoring patient preferences (Reference Barnato and Garber2). The patient perspective is particularly important in cases of elective surgery for conditions that directly affect quality of life. Finally, appropriateness of healthcare services has been primarily considered at the micro or individual patient level. These decisions, however, are occurring within the broader context of healthcare systems that are often constrained in terms of service delivery by limited resources. The perspective of decision makers would add a valuable dimension to our understanding of appropriateness of care. This Review suggests that there is an opportunity, if not necessity, to consider input on appropriateness from patients and decision makers representing a societal view.
The most commonly used method for determining appropriateness is the RAND/UCLA model. While valued and standardized methodologically, this Review points to limitations of the approach. First, the RAND approach is based in part on the opinions of experts which are formed by their own memories, interests and perspectives, which in turn are influenced by a range of factors including medical education, postgraduate training and cultural background (Reference Barnato and Garber2). As a result, the composition of the panel appears to matter. In most cases, panels depend on the opinions of experts and therefore, may not be representative. The determination of whether a procedure is appropriate, or not, will depend in part on the type and mix of clinicians involved in the deliberations. Fraser et al. (Reference Fraser, Pilpel and Kosecoff12) studied the effect of panel composition in the determination of appropriateness criteria for cholecystectomy. Using the RAND method, appropriateness ratings were compared between an all-surgical panel, mixed subspecialist panel and a generalist panel conducted in three countries (Israel, UK, and the United States). The findings indicate that the all-surgical panel were more likely to agree on indications for cholecystectomy and had fewer contraindications than a mixed panel. Mixed panels were more conservative than the all-surgical and generalist panels. There were also differences noted across countries (Reference Ayanian, Landrum, Normand, Guadagnoli and McNeil1).
Second, the RAND method is based on the assessment of risks and benefits, including the magnitude of the effect of the treatment. However, the process does not make explicit the risks and benefits to be considered. Furthermore, information regarding effect size may be based on evidence from the literature or, where limited, will be based on physicians’ estimates of the effect size. As a result, clinicians may be considering different risks and benefits as well as different effect sizes (Reference Hicks15).
Third, critics argue that appropriateness ratings will differ if clinicians are provided with information regarding severity and duration of outcomes of care. Caplan et al. (Reference Caplan, Posner and Cheney7) compared appropriateness ratings derived by anesthesiologists using differing outcomes (i.e., temporary versus permanent) for similar cases. For most cases, there was an inverse relationship between the severity of outcome and appropriateness ratings—clinicians were more likely to consider cases with permanent injuries as inappropriate.
Fourth, the RAND approach focuses on identifying appropriateness criteria for the “average” patient. The resulting appropriateness scores are not, by design, adjusted for differences across individuals regarding their circumstances nor their hopes and fears about the procedures (Reference Hicks15). Such arguments apply to any instrument meant to encompass the “average” case, when then applying that instrument to a specific “individual.”
Fifth, the RAND approach focuses exclusively on the determination of appropriateness when the service has been provided. As a result, there are concerns that the appropriateness evaluation process is not comprehensive because it does not include cases of underuse, that is, failure to perform a procedure when in fact it was appropriate. Finally, and more generally, critics argue that appropriateness criteria developed using the RAND method may lead to biased estimates of the degree of appropriateness, for the reasons previously stated (Reference Barnato and Garber2;Reference Phelps27).
Despite the reality that the RAND approach is well known and in common use, the aforementioned limitations and the opportunity to incorporate patient and societal views prompt reconsideration to include broader inputs and methodologic approaches to capture them.
CONCLUSIONS
This scoping review was conducted to inform next stages of consideration in developing appropriateness criteria or decision-support tools for specific clinical interventions. A broadly focused search and analysis showed that the definitions, concepts and frameworks for understanding and determining appropriateness are varied and, thus far, in large part clinically oriented. This synthesis supports a broader approach and understanding as being both possible and necessary. Logical next steps might include undertaking a comprehensive synthesis of appropriateness focused on specific clinical interventions. Thereafter, a systematic approach is required to collect input from clinical, patient and societal sources to inform a more comprehensive understanding and sound determination of appropriateness, using deliberative approaches and attempts to harmonize the inputs from all relevant sources.
CONTACT INFORMATION
Claudia Sanmartin, PhD (Claudia.Sanmartin@statcan.ca), Research Assistant Professor, Department of Community Health Sciences, University of Calgary, 3330 Hospital Drive NW, Calgary Alberta T2N 4N1, Canada; Senior Analyst, Health Information and Research Division, Statistics Canada, RH Coats Bldg., 24th Floor, Section R, 100 Tunney's Pasture Driveway, Ottawa, Ontario K1A 0T6, Canada
Kellie Murphy, MSc (Kellie.Murphy@statcan.ca), Research Analyst, Health Information and Research Division, Statistics Canada, RH Coats Bldg, 24th Floor, Section L, 100 Tunney's Pasture Driveway, Ottawa, Ontario K1A 0T6, Canada
Nicole Choptain, MSc (nicolewpg@shaw.ca), Research Consultant, Choptain Consulting, 4-360 Island Shore Blvd., Winnipeg, Manitoba R3X 2E2, Canada
Barbara Conner-Spady, PhD (barbara.spady@ualberta.net), Research Assistant Professor, Senior Analyst, Western Canada Waiting List Project, Department of Community Health Sciences, Lindsay McLaren, PhD (lmclaren@ucalgary.ca), Assistant Professor and AHFMR Population Health Investigator, Department of Community Health Sciences, University of Calgary, 3330 Hospital Drive NW, Calgary, Alberta T2N 4N1, Canada
Eric Bohm, MD (ebohm@concordiahospital.mb.ca), Assistant Professor, Department of Surgery, University of Manitoba; Orthopedic Surgeon, University of Manitoba Joint Replacement Group, Concordia Hospital, 1095 Concordia Avenue, Winnipeg, Manitoba R2K 3S9, Canada
Michael J. Dunbar, MD, PhD (michael.dunbar@dal.ca), Associate Professor, Department of Surgery, Dalhousie University, 2500 University Drive NW, Halifax, Nova Scotia, B3H 4H2, Canada; Orthopedic Surgeon, Department of Surgery, QE II Health Sciences Centre, 1796 Summer Street, Halifax, Nova Scotia B3H 3A7, Canada
Suren Sanmugasunderam, MD (suren@telus.net), Clinical Associate Professor, Department of Ophthalmology & Visual Sciences, University of British Columbia, 2550 Willow Street, Vancouver, British Columbia V5Z 3N9, Canada
Carolyn De Coster, PhD (Carolyn.DeCoster@calgaryhealthregion.ca), Research Assistant Professor, Department of Community Health Sciences, University of Calgary, 3330 Hospital Drive NW, Calgary, Alberta T2N 4N1, Canada; Senior Researcher, Department of Health Outcomes, Calgary Health Region, 4520 16th Avenue NW, Calgary, Alberta T3B 0M6, Canada
John McGurran, MSc (jjmcg@telus.net), Lecturer, Department of Public Health Sciences, University of Toronto, 155 College Street, Toronto, Ontario M5T 3M7, Canada
Diane L. Lorenzetti, MLS (dllorenz@ucalgary.ca), Adjunct Assistant Professor, Department of Community Health Sciences, University of Calgary, 3330 Hospital Drive NW, Calgary, Alberta T2N 4N1, Canada; Research Librarian, Institute of Health Economics, #1200, 10405 Jasper Avenue, Edmonton, Alberta T5J 3N4, Canada
Tom Noseworthy, PhD (tnosewor@ucalgary.ca), Professor, Head of Department of Community Health Sciences, Director, Centre for Health and Policy Studies, University of Calgary, 3330 Hospital Drive NW, Calgary, Alberta T2N 4N1, Canada
