In 2000, a landmark publication from the Institute of Medicine estimated that as many as 98,000 people die in any given year from medical errors that occur in hospitals. This report stated that “Health care is not as safe as it should be. A substantial body of evidence points to medical errors as a leading cause of death and injury”.Reference Kohn, Corrigan and Donaldson1 The report has generated significant interest in the evaluation of the quality of medical care. The public, our patients, insurance companies, and the government all demand transparent disclosure of the outcomes of medical treatment.2
In congenital and paediatric cardiac surgery, multiple entities, including governmental agencies, insurance companies, and regional consortiums dedicated to the evaluation and improvement of the quality of medical care, all require documentation of surgical outcomes. The State of Florida exemplifies this ongoing effort to measure the outcomes of congenital and paediatric cardiac surgery on a regional basis. Florida is the fourth largest state in the United States of America, having a population of 17,789,684 in 2005. In 2004, 218,045 live babies were born in Florida, accounting for approximately 1,750 new cases of congenital cardiac disease each year. In Florida, 8 centres provide services for paediatric cardiac surgery, with these programmes all participating in a collaborative initiative to study regional outcomes under the auspices of the State of Florida, using the database of the Society of Thoracic Surgeons to collate their results. The objective of this collaboration is to develop and implement the elements necessary for establishing state-wide oversight of the results of paediatric cardiac surgery through collaboration amongst the multiple centres caring for patients with congenital heart disease in Florida, the ultimate goal being to improve the care of children with cardiac disease.
The Congenital Heart Surgery Database Taskforce of The Society of Thoracic Surgeons has worked with the State of Florida to create a regional report, which has been adopted by the Children’s Medical Services of The State of Florida as the state-wide standard for evaluation of the quality of care for patients requiring surgical treatment of their congenital cardiac malformations. This regional report is the first such effort by the Congenital Heart Surgery Database Taskforce of The Society of Thoracic Surgeons, and hopefully will serve as a model for future regional initiatives. In this manuscript, we describe the initial experience of The Society of Thoracic Surgeons in producing an analysis of regional outcomes of congenital cardiac surgery.
Methods
The Society of Thoracic Surgeons Congenital Heart Surgery Database
In the report of the 2005 Society of Thoracic Surgeons Congenital Heart Surgery Practice and Manpower Survey, undertaken by the Society of Thoracic Surgeons Workforce on Congenital Heart Surgery, it was estimated that 122 centres in the United States of America, and 8 centres in Canada, were performing paediatric and congenital cardiac surgery.Reference Jacobs, Mavroudis, Jacobs, Tchervenkov and Pelletier3 As of January, 2009, The Society of Thoracic Surgeons Congenital Heart Surgery Database contains data from 73 of these 130 centres. It is now the largest database in North America dealing with congenital cardiac malformations.Reference Jacobs, Jacobs and Mavroudis4 It has grown annually since its inception, both in terms of the number of participating centres submitting data, and the number of operations analyzed (Figs. 1 and 2).Reference Jacobs, Jacobs, Mavroudis, Lacour-Gayet and Tchervenkov5 The aggregate report from the Fall of 2009 includes 79,161 operations performed in the 4 year period of July 1, 2004 through June 30, 2008, submitted from 73 centres from North America, 72 from the United States of America and 1 from Canada. One Japanese centre also submits data, albeit that this data is not included in the aggregate report produced by The Society of Thoracic Surgeons. The database uses the identical nomenclature and standards as the congenital database of The European Association for Cardio-Thoracic Surgery. Multiple publications generated from these two databases have standardized the methodologies for reporting outcomes after congenital cardiac surgery, and have reported these outcomes both on a large scale programmatic basis, and also in relation to specific lesions.Reference Jacobs, Jacobs, Mavroudis, Lacour-Gayet and Tchervenkov5–Reference Jacobs, Jacobs and Mavroudis10 These two databases, and multiple associated publications, have standardized the analysis of outcomes after congenital cardiac surgery in the areas of nomenclature, standards for databases, stratification of complexity, verification of data, and collaboration across medical and surgical subspecialties.Reference Jacobs, Jacobs and Mavroudis4, Reference Jacobs, Mavroudis and Jacobs11, Reference Jacobs, Wernovsky and Elliott12
The graph documents the annual growth in the Society of Thoracic Surgeons Congenital Database by number of participating centres submitting data. The aggregate report from Fall 2008 of the Society of Thoracic Surgeons Congenital Database includes data from 73 Congenital Heart Surgery Centres from the United States of America and Canada. One Japanese centre also submits data; however, these Japanese data are not included in the aggregate report produced by the Society of Thoracic Surgeons.
The graph documents the annual growth in The Society of Thoracic Surgeons Congenital Database by the number of operations. The aggregate report from Fall 2008 of The Society of Thoracic Surgeons Congenital Database included 79,161 operations submitted from 73 centres from North America, 72 from the United States of America and 1 from Canada. One Japanese centre also submits data; however, these Japanese data are not included in the aggregate report produced by The Society of Thoracic Surgeons.
Florida Children’s Medical Services and the Children’s Cardiac Programme
Florida Children’s Medical Services is the primary medical programme in Florida that serves children with special needs for health care. This network provides primary and specialty care to eligible children in Florida through a family-centred, comprehensive, and coordinated state-wide managed system of care, linking community-based health care with multidisciplinary, regional, and tertiary paediatric care. Services are provided through public and private providers throughout the state. Services include early intervention, primary care, specialty care, therapeutic care, and long-term care.
Florida Children’s Medical Services celebrated their 75th anniversary in 2004. During these 75 years, Florida Children’s Medical Services has grown from a programme that provided services to a narrowly defined group of children utilizing a traditional medical model into one that, today, provides a broad range of services to a diverse population of children with a wide range of medical needs.
The Florida Children’s Medical Services Cardiac Program was established in 1953. Initially, children received services, particularly surgical procedures, through cooperative agreements with the Minnesota and Maryland Crippled Children’s Programmes. These agreements enabled children to receive care in Rochester, Minnesota, at the Mayo Clinic, at the University of Minnesota in Minneapolis, and the Johns Hopkins Cardiac Center in Baltimore. In the late 1950s, the Florida Crippled Children’s Commission, the former name of Florida Children’s Medical Services, made an arrangement with the newly established University of Florida Medical School in Gainesville for children to receive cardiac care at that institution.
Oversight of the cardiac programme has been provided by a cardiac advisory committee for over 50 years, and this committee currently operates as the Cardiac Sub-committee of the Florida Children’s Medical Services Network Advisory Council. Membership includes physicians representing all the paediatric cardiovascular facilities approved by Florida Children’s Medical Services. The committee meets periodically, operating under promulgated rules of the State of Florida which have the force of law (64C-4.003 F.A.C.).
In 2007, the programme served children with congenital or acquired cardiac conditions, providing services in the private offices of approved physicians, through a network of community based cardiac clinics, and through the 8 centres approved to provide paediatric cardiovascular surgical facilities. The services provided include cardiac evaluation and diagnosis, diagnostic and interventional cardiac catheterization, closed and open-heart surgery, evaluation and treatment of cardiac arrhythmias, cardiac transplants, and screening of patients for cardiac disease utilizing telemedicine echocardiology technology.
Florida Children's Medical Services provides a framework for initiatives involving the 8 paediatric cardiac surgical centres to assess the outcomes and improve the quality of congenital and paediatric cardiac care. The 8 approved centres collaborate so as to improve the quality of the care they provide, voluntarily submitting their results to the database of the Society of Thoracic Surgeons. The Society of Thoracic Surgeons, together with Duke Clinical Research Institute, have now analysed the submitted data to provide detailed regional analysis of the outcomes of congenital cardiac surgical procedures undertaken in Florida, and have therefore prepared The Society of Thoracic Surgeons Congenital Heart Surgery Database Florida Regional Report.
Results
The initial Florida Regional Report was prepared in 2005, and contained data from 4 of the 8 sites in Florida. The report for 2005 included details of 27,819 operations performed during the period 2002 through 2004, with 3,365 (12.17%) of these operations being performed in Florida. The report for 2006 contained data from 5 of the 8 sites in Florida. This report was based on details of 45,635 operations performed during the period 2002 through 2005, with 5,026 (11.1%) of these operations being performed in Florida. The report prepared for 2007 contained data from 7 of the 8 sites in Florida. This was based on 61,014 operations, performed over the period 2003 through 2006, with 6,385 (10.5%) of the operations taking place on Florida. Due to some problems with the entry of data, which have since been corrected, one programme in Florida withdrew their approval to be included in the 2007 report. Data from this centre will be included in the report to be generated for 2008, which will therefore be based on data obtained from all 8 sites.
Each of these reports was based on a huge amount of data, and analyzed both overall outcomes and outcomes for specific lesions. In this manuscript, we have selected a representative sample of this data. In Table 1, we provide data stratified for complexity stratified using both the Aristotle Basic Complexity (ABC) LevelReference Jacobs, Lacour-Gayet and Jacobs7, Reference Lacour-Gayet13–Reference Lacour-Gayet, Clarke and Jacobs15 and the “Risk Adjustment in Congenital Heart Surgery-1” method, known as RACHS-1.Reference Jenkins, Gauvreau, Newburger, Spray, Moller and Iezzoni16, Reference Jenkins17 The techniques of analysis used by the Congenital Heart Surgery Database of The Society of Thoracic Surgeons have been previously published, including a detailed description of criterions for eligibility of an operation for inclusion in the analysis.Reference Jacobs, Mavroudis and Jacobs9, Reference Jacobs, Jacobs, Mavroudis and Lacour-Gayet18 In Table 1, we show that the mortality at discharge after cardiac surgery in Florida, with 95% confidence intervals, is not different from cumulative data from the entire database concerning paediatric cardiac surgical procedures of The Society of Thoracic Surgeons, both for all patients and for patients stratified into all 4 levels in the Aristotle system, and the 5 levels of RACHS-1.
Complexity Stratified Outcome Data – Florida and the Society of Thoracic Surgeons (STS).
In Figure 3, we show overall mortality at discharge from the hospital after cardiac surgery, without stratification for complexity, during the interval from 2003 through 2006, for the 7 participating centres in Florida, comparing the outcomes to those for the entire congenital cardiac surgical database maintained by the Society of Thoracic Surgeons. Overall mortality at discharge from the hospital after cardiac surgery during this period for all 7 sites, with 95% confidence intervals, is not different from the cumulative data from the entire database.
The graph documents overall mortality at discharge from the hospital after cardiac surgery, without complexity stratification, during the 4 year time interval of 2003–2006, for the seven participating programs in Florida, in comparison to the entire Congenital Heart Surgery Database of The Society of Thoracic Surgeons. Figure 3 reveals that the 4-year overall mortality at discharge from the hospital after cardiac surgery in the data from all 7 sites, with 95% confidence intervals, is not different from cumulative data from the entire Congenital Heart Surgery Database of The Society of Thoracic Surgeons.
In Figure 4, we document overall mortality at discharge from the hospital after cardiac surgery, without stratification for complexity, exclusively for 2006, again comparing the 7 centres in Florida to the results from the overall database. Overall mortality at discharge from the hospital after cardiac surgery for this year, again for all 7 sites, when assessed on the basis of 95% confidence intervals, is not different from the cumulative data for the entire database recording results of surgery for congenital cardiac disease. We present the results over periods of 1 and 4 years to demonstrate changes in outcomes over time, as seen when the mortality at discharge from the hospital after cardiac surgery for hospital number 4 is compared in Figures 3 and 4.
The graph documents overall mortality at discharge from the hospital after cardiac surgery, without complexity stratification, during 2006 only, for the seven participating programs in Florida, in comparison to the entire Congenital Heart Surgery Database of The Society of Thoracic Surgeons. Figure 4 reveals that the 1-year overall mortality at discharge from the hospital after cardiac surgery in the data from all 7 sites, with 95% confidence intervals, is not different from cumulative data from the entire Congenital Heart Surgery Database of The Society of Thoracic Surgeons. Results are presented over both a 1-year and 4-year time interval to demonstrate changes in outcomes over time, as demonstrated by comparing the mortality at discharge from the hospital after cardiac surgery for hospital number 4 in Figures 3 and 4.
In Figure 5, we show overall mortality at discharge from the hospital after cardiac surgery, with complexity stratified on the basis of the 4 levels in the Aristotle basic system, for the interval from 2003 through 2006, again comparing the results for Florida with those from the overall database. As before, there is no difference when the data from all 7 sites, with 95% confidence intervals, is compared to the cumulative data from the entire database. In Figure 6, we show the same data stratified according to RACH-1, again with no differences noted for Florida when compared with the overall results, except that Hospital Number 6 has a slightly higher mortality for RACHS-1 Level 3.
The graphs document overall mortality at discharge from the hospital after cardiac surgery, with complexity stratification into the 4 Aristotle Basic Complexity Levels, during the 4 year time interval of 2003–2006, for the seven participating Florida programs, in comparison to the entire Congenital Heart Surgery Database of The Society of Thoracic Surgeons. Figure 5 reveals that the 4-year mortality at discharge from the hospital after cardiac surgery in the data from all 7 sites, with 95% confidence intervals, is not different from cumulative data from the entire Congenital Heart Surgery Database of The Society of Thoracic Surgeons, for patients stratified into all 4 Aristotle Basic Complexity Levels.
These graphs document overall mortality at discharge from the hospital after cardiac surgery, with complexity stratification into the 5 Risk Adjustment in Congenital Heart Surgery-1 Levels, during the 4 year time interval of 2003–2006, for the seven participating Florida programs, in comparison to the entire Congenital Heart Surgery Database of The Society of Thoracic Surgeons. Figure 6 reveals that the 4-year mortality at discharge from the hospital after cardiac surgery in the data from all 7 sites, with 95% confidence intervals, is not different from cumulative data from the entire Congenital Heart Surgery Database of The Society of Thoracic Surgeons, for patients stratified into all 5 Risk Adjustment in Congenital Heart Surgery-1 Levels, except that Hospital Number 6 has a slightly higher mortality for RACHS-1 Level 3.
Despite the fact that these results show relative homogeneity amongst the participating centres, the methodology utilized is sufficient to identify outliers, as demonstrated by Figure 7. This figure documents overall mortality at discharge from the hospital after cardiac surgery for all operations at level 4 of the Aristotle system as included in the results in the report from Fall of 2008 of the Congenital Heart Surgery Database of The Society of Thoracic Surgeons. The methodology identifies several centres that are outliers, using 95% confidence intervals, with rates of mortality at discharge from the hospital after cardiac surgery for the analysed operations different from the overall mortality in the database for this subgroup. In this example, the 95% confidence interval for centres 62, 44, 38, 47, 49, 22, 6, and 26 is above the overall mortality in the database for this subgroup. Meanwhile, the 95% confidence interval for centres 12, 4, and 1 is below the overall mortality in the database for this subgroup.
This graph documents overall mortality at discharge from the hospital after cardiac surgery for all Aristotle Level 4 operations in the Fall 2008 Congenital Heart Surgery Database Report of The Society of Thoracic Surgeons. Figure 7 demonstrates that the methodology utilized in these Reports identifies several centres that are outliers, using 95% confidence intervals, with rates of mortality at discharge from the hospital after cardiac surgery for Aristotle Level 4 operations different from the overall mortality in the database for this subgroup. In this example, the 95% confidence interval for centres 62, 44, 38, 47, 49, 22, 6, and 26 is above the overall mortality in the database for this subgroup. Meanwhile, the 95% confidence interval for centres 12, 4, and 1 is below the overall mortality in the database for this subgroup.
Discussion
In congenital and paediatric cardiac surgery, multiple organisations, including governmental agencies, insurance companies, and regional consortiums dedicated to the evaluation and improvement of the quality of medical care, all require documentation of regional outcomes. The Congenital Heart Surgery Database Taskforce of The Society of Thoracic Surgeons is hopeful that the database will become a template for such regional reporting, and will replace multiple other individualized regional reports, each with their own non-standardized nomenclature, standards, and methodologies for stratification of complexity. The experience thus far in Florida provides valuable lessons into the potential to achieve this goal.
The leaders of the Cardiac Sub-committee of the Florida Children’s Medical Services Network Advisory Council met twice to discuss strategies for state-wide outcome reporting, first in 2004, and then in 2006.Reference Jacobs and Quintessenza19, Reference Jacobs and Quintessenza20 The meeting in 2004 led to the agreement that all 8 centres providing services for paediatric cardiac surgery in Florida would submit their data to the database maintained by the Society of Thoracic Surgeons, and that the State of Florida would collaborate with the Society of Thoracic Surgeons to create the regional report for Florida. The meeting of 2006 led to the decision that, rather than revising the extant and problematic “Congenital Heart Surgery Data Collection Form” of the Florida Children's Medical Services, Florida Children's Medical Services and The State of Florida would use, instead, the individual annual Feedback Reports provided by the Society of Thoracic Surgeons Congenital Heart Surgery Database to each site. It is important to understand the reasons for these decisions.
Florida Children’s Medical Services has participated for many years in the Pediatric Cardiac Care Consortium, and continues to do so. Participation this consortium has provided the State of Florida with substantial valuable data. To date, the consortium provides Florida with the best available multi-institutional data about interventional cardiology. One weakness of the data produced by the consortium, however, is the lag from the clinical event to the release of the data. For example, the report emerging in 2007 only contained data submitted through 2005. Meanwhile, the report of 2007 from the Society of Thoracic Surgeons contains data relating to 2006, and is released in the summer of the year, so that it includes data from within the past 6 months.
The form previously used to submit data to Florida Children’s Medical Services reveals some of the other weaknesses associated with many regionalized efforts to collect data. This form mixed diagnoses and procedures together, used poorly defined diagnostic terms and non-standardized nomenclature, and grouped operations inappropriately. For example, when listing operations on this form, all of the following choices are present:
• “TGV – isolated”,
• “TGV – with PS”,
• “TGV – with VSD”,
• “TGV – with VSD and PS”, and
• “Arterial switch (Jatene) for TGV”.
In this example, “TGV” is “Transposition of the Great Vessels”, “PS” is “Pulmonary Stenosis”, and “VSD” is “Ventricular Septal Defect”. Consequently, several potential options exist for the coding of an “Arterial switch operation (ASO)” or an “Arterial switch operation (ASO) and VSD repair”. Overlapping diagnostic and procedural codes are illogically mixed together. A second example is that the list included both of the following choices: “Pulmonary atresia with intact ventricular septum” and “Fontan completion – either extracardiac or intracardiac”. It is unclear where a patient with pulmonary atresia with intact ventricular septum who has a Fontan procedure should be coded. A third example is that the list contains the term “ventricular inversion”. This historical term is vague, and even in the past had uncertain meaning. It is unclear if this term includes only congenitally corrected transposition, or if it also includes the combination of discordant atrioventricular and concordant ventriculo-arterial connections.Reference Jacobs, Franklin and Wilkinson21 A fourth example is that a “Glenn operation” is listed in the group of closed cases, when most of these procedures performed nowadays utilize cardiopulmonary bypass. Because of these weaknesses in the form, the Cardiac Sub-committee decided not to try and revise this form, but rather to replace it with the individual annual Feedback Reports provided by the Society of Thoracic Surgeons Congenital Heart Surgery Database to each site. These individual annual Feedback Reports made by The Society of Thoracic Surgeons, when combined with Florida Regional Report, present a complete state-wide analysis of outcomes after surgery for congenital cardiac disease in Florida.
The State of Florida plans to use the data from the Florida Regional Report, combined with the individual annual Feedback Reports, both prepared by The Society of Thoracic Surgeons Congenital Heart Surgery Database, as tools to provide the framework for further improvement of quality, and to enhance the current programme of inter-institutional site visits by surgeons and cardiologists. There are multiple additional advantages of utilizing The Society of Thoracic Surgeons Congenital Heart Surgery Database as a source for analysis of regional congenital cardiac surgical outcomes, such as:
• minimizing the burden of entry of data, because one act of entry will meet the needs of multiple organizations, using standardized international nomenclature and database definitions,
• using a dataset that is based upon a common universal nomenclature,
• using a dataset that is based upon standardized rules and definitions,
• incorporating commonly accepted methods of stratification of complexity and risk adjustment, and
• experiencing the advantages of using a dataset that is subject to audit and verification of data.
The adaptation of this methodology by additional governmental agencies, insurance companies, and regional consortiums, will lead to further realization of these advantages and greater minimization of the burden of entry of data.
An initial glance at the data from Florida reveals that none of the centres currently working in Florida are outliers when analyzing the data with 95% confidence intervals. From the data presented in this report, it might be wondered if the database maintained by the Society of Thoracic Surgeons is capable of identifying outliers amongst those performing paediatric cardiac surgical procedures. As is shown in Figure 7, nonetheless, The Society of Thoracic Surgeons Congenital Heart Surgery Database clearly demonstrates the ability to identify outliers, even with 95% confidence intervals.Reference Jacobs, Jacobs, Mavroudis, Lacour-Gayet and Tchervenkov5, Reference Jacobs, Jacobs, Mavroudis and Lacour-Gayet18, Reference Jacobs, Cerfolio and Sade22
With respect to results, the main message is that data about mortality for the centres in Florida, as a group, are not different, within 95% confidence intervals, from that of the aggregate data of all centres participating in the overall Congenital Heart Surgery Database of The Society of Thoracic Surgeons. The same fact is true for the individual centres in Florida, both with respect to cumulative outcomes for all patients, and for patients within specific categories of complexity. Meanwhile, it is evident that the methodology employed has allowed for the identification of outliers on a national level. The ongoing use of the regional report prepared for Florida will allow for continual assurance that congenital cardiac surgery of acceptable quality is being performed at all its centres. In the future, we are confidant that, should outliers appear, they will be identified through the regional report. The identification of such outliers will then allow for the initiation of collaborative initiatives to improve quality, which would address the three domains of structure, process, and outcome. A variety of methods can be used to facilitate such initiatives:
• collaboration between centres,
• bi-directional site visits,
• programmatic mentorship,
• combined patient care conferences, that is Joint Cardiac Surgery and Cardiology Conferences involving members of multiple centres, and
• adaptation of practices employed at “high-performing” centres.
It is important for such activities to occur even without the identification of outliers, in order to facilitate the overall improvement in quality of care, in the domains of structure, process, and outcome, that is delivered to all patients with congenital and paediatric cardiac disease in the State of Florida.
Disclosure
Jeffrey Phillip Jacobs is medical advisor for CardioAccess. Jeffrey Phillip Jacobs and Redmond Paul Burke are shareholders of CardioAccess.
