Paediatric and adult congenital cardiologists manage a diverse population of patients with minor to complex structural and acquired heart disease, as well as a myriad of symptoms which may or may not represent important cardiac disease. Multiple quality improvement collaboratives have arisen to address the needs in specific domains, including the care of single-ventricle patients, cardiac intensive care management, cardiac neurodevelopmental outcomes, and others.Reference Clauss, Anderson and Lannon 1 – Reference Jenkins, Castañeda and Cherian 5 In 2015, the Adult Congenital and Pediatric Cardiology branch of the American College of Cardiology formed the Quality Network, a national-level quality improvement collaborative. The working groups of the Quality Network developed metrics spanning a broad range of care, from single congenital lesions to adverse events with sedation.Reference Martin, Mitchell and Beekman 6 , Reference Chowdhury, Gurvitz and Marelli 7 Diverse organisations, including large academic institutions, small private hospitals, and individual practices, were invited to engage in data collection on vetted metrics, with the initial purpose of establishing benchmarks since none existed. Metrics demonstrating wide practice variation and potential for enhanced care were chosen for national quality improvement efforts. Individual participating centres determined strategies for change most likely to succeed in their environment, with sharing of best practices within the collaborative.
In this paper, we describe the experience of our institution, Boston Children’s Hospital, engaging with the Quality Network. We assess change in our performance over time in comparison to benchmarks derived from national performance. In addition, we describe our internal change strategies for two metrics selected nationally for improvement: counselling for elevated body mass index and offering 22q11.2 deletion genetic testing to patients with tetralogy of Fallot.
Methods
During the initial 2 years of collaboration, the Quality Network provided 24 metrics to participating programmes for the purpose of establishing a national average or benchmark on various care measures. Data collection occurred on a quarterly basis from October 2015 to December 2017. A minimum of six institutions are required to submit data to determine a national benchmark. Either the entire quarterly patient populations could be included or a random sampling of 20 patients. For metrics pertaining to small populations of patients, a minimum of five patients are required to report on the metric. Appropriate patients for each metric at our institution were identified through International Classification of Disease 9 and 10 codes and internal procedural codes. Automated queries to provide data for the metrics were developed and verified through chart auditing for accuracy during the first year of study. Data that could not be obtained through this process were gathered via chart review. The team consisted of a quality improvement project manager, with Masters of Public Health (10% effort) and cardiologist who performed data collection and verification, as well as a Kawasaki disease nurse practitioner for specific measures. Numerators and denominators were provided to the Quality Network without exchange of personalised health information. The Boston Children’s Hospital Institutional Review Board determined that participation in the Quality Network did not meet the regulatory criteria for human subjects’ research and that ongoing oversight was not required.
Following a period of national data collection and benchmarking, two metrics were selected for improvement through the collaborative based on high variability in centre-to-centre performance and opportunity to impact care. The metrics chosen by the Quality Network involved recognition of and appropriate counselling for elevated body mass index in patients and offering 22q11.2 deletion testing to patients with tetralogy of Fallot. An aim of 100% performance by the collaborative by year-end 2017 was set for both metrics. At our centre, we used principles from the Institute for Healthcare Improvement to develop change strategies.Reference Reinertsen, Gosfield, Rupp and Whittington 8 We formed multi-disciplinary working groups, appointed physician and nurse champions to lead the planning of change strategies, collaborated with quality improvement specialists, dieticians, parent focus groups, experts in the fields of preventive cardiology and cardiac genetics, and information technologists. Educational materials were provided to families on healthy diet, for example Healthy Eating Plate©, Harvard T. H. Chan School of Public Health (https://www.hsph.harvard.edu/nutritionsource/healthy-eating-plate/) and physical activity for body mass index reduction through mobile applications and regional programmes on fitness. Educational forms were distributed in clinic and linked to the electronic medical record for future parent reference. Talking points for cardiologists were developed by colleagues in cardiac genetics regarding the importance of 22q11.2 testing in tetralogy of Fallot. Materials included summary of comorbidities, such as neurodevelopment, feeding difficulties, immunodeficiency, and others; navigating insurance, such as payment coverage/deductible/co-pay, insurance coverage denial for genetic disorder; and available counselling and clinical services. Information technologists modified the medical record to facilitate data gathering and enhance education. Data review was used to drive iterative change and inform intervention strategies. Physician engagement in these quality improvement projects was encouraged by offering Maintenance of Certification, 25 points of part IV credit per initiative, provided and negotiated through the American College of Cardiology.
The comprehensive list of metrics from the Quality Network is published at the following online site: https://cvquality.acc.org/initiatives/acpc-quality-network/quality-metrics.Reference Kuczmarski, Ogden and Guo 9 Definitions within the metrics, as established by the Quality Network, for family history pertaining to chest pain visits includes acquisition of premature coronary artery disease, cardiomyopathy, and sudden death in first- or second-degree relatives; echocardiogram for the presenting complaint of exertional chest pain must include assessment of coronary artery anatomy; for Kawasaki disease, echocardiograms must include measurements of the left anterior descending and right coronary arteries at a minimum; elevated body mass index is defined as ⩾85th percentile by age and gender;Reference McCrindle, Rowley and Newburger 10 tetralogy of Fallot patients for whom 22q11.2 testing is recommended excludes those with other genetic abnormalities – trisomy 13, 18, 21, Alagille syndrome – or associated atrioventricular canal, pulmonary atresia with multiple aortopulmonary collaterals, and absent pulmonary valve; adverse events from sedated echocardiogram defined as: minor – desaturation, apnoea, allergic reaction not requiring treatment, vomiting, prolonged sedation or recovery, and inadequate sedation; moderate – oxygenation or ventilation compromise requiring non-invasive ventilation, intubation or reversal agents, aspiration, haemodynamic compromise, unplanned overnight observation, allergic reaction requiring treatment, agitation or delirium, intravenous-related complications, emergent sedation consultation, hypoglycaemia requiring treatment, hypothermia, stridor, wheezing, and laryngospasm; and severe – cardiopulmonary arrest, permanent injury or disability, and death.
Statistical analysis
Results of all metrics are graphically represented to demonstrate quarterly centre performance compared with national mean; the latter provided through the American College of Cardiology. Internal run charts were generated to track variability in our performance over time. Our centre means were generated for all metrics, although they were not additive for metrics with consistently high performance; they are shown graphically only for the two metrics chosen for quality improvement. In addition, our performance for the same two quality improvement metrics has been analysed in control charts. Historic data points were insufficient to characterise baseline performance before change efforts, and therefore the control charts are not shown in this article. The data were collected quarterly and in a relatively small sample size. An interrupted time series analysis is not feasible at this time and will require a longer period of study.
Results
Among the 24 metrics offered through the Quality Network to the collaborative, 13 metrics (Table 1) were consistently adopted by sufficient participating centres for assessment of national benchmarks. The remainder of this manuscript describes outcomes of the 13 metrics, as well as details of the two quality improvement projects. Entire clinic populations were analysed for the metrics pertaining to Kawasaki disease, adverse events with sedated echocardiography, and influenza vaccination; for all other metrics, a random sampling of 20 patients was used. Results of the chest pain metrics (Metrics #1–3, Table 1) are shown in Figure 1, with our centre’s quarterly performance compared with the national average. Figure 1a and b shows acquisition of family history and electrocardiogram in all patients presenting to clinic for chest pain complaints, respectively. The rate of echocardiography in the assessment of exertional chest pain is shown in Figure 1c. Figure 2 demonstrates performance on metrics pertaining to patients diagnosed with Kawasaki disease (Metrics #4–7, Table 1). Adherence to Kawasaki disease guidelinesReference Newburger, Takahashi and Gerber 11 , Reference Friedman, Kane and Rathod 12 including echocardiographic assessment of coronaries, prescribing aspirin therapy at discharge, and post-discharge follow-up assessment with repeat echocardiogram are shown in Figure 2a–c, respectively. Metric #7, which assesses documentation for a statement of no activity restrictions at the 6-week follow-up visit and beyond, if no coronary artery aneurysms are present, shows the most variability in performance, both at our centre and nationally (Fig 2d).
Quality network metrics.
BMI=body mass index; KD=Kawasaki disease; TOF=tetralogy of Fallot
Chest pain metrics. ( a ) Family history acquisition; ( b ) Electrocardiogram acquisition for any complaint of chest pain; ( c ) Echocardiogram performance for exertional chest pain.
Kawasaki disease metrics. ( a ) Echocardiogram documentation of coronary artery measurements; ( b ) Prescribing rate of daily low-dose aspirin for ⩾6 weeks upon hospital discharge; ( c ) Follow-up visit rate with echocardiogram within 3 weeks of hospital discharge; ( d ) Proportion of clinic notes recommending return to normal physical activity after 6 weeks from Kawasaki disease diagnosis without coronary aneurysms.
Figure 3 illustrates complication rates from sedated echocardiography (Metric #8, Table 1) for patients <3 years of age. In the 2 years of our study, we report a single adverse event, which was moderate in severity. A 2-year-old child with past medical history of asthma, admitted for acute Kawasaki disease, received chloral hydrate and developed airway obstruction, desaturation, and hypotension, requiring supportive care in the paediatric ICU. Figure 4 demonstrates the performance of axial imaging in adults with repaired coarctation (Metric #9, Table 1), including angiography, cardiac MRI, and/or CT. Compliance with influenza vaccination in cardiac healthcare teams (Metric #10, Table 1) is demonstrated in Figure 5. Documentation of measured body mass index and percentile calculation (Metric #11, Table 1) in all children presenting to outpatient cardiology clinics is shown in Figure 6.
Adverse events related to sedated echocardiogram.
Complete aortic evaluation in adults presenting to clinic with coarctation of the aorta.
Influenza vaccination compliance in healthcare personnel.
Documentation of body mass index and percentile calculation in cardiology clinic patients.
Quality improvement with the collaborative
The final two metrics, counselling for elevated body mass index and 22q11.2 testing in tetralogy of Fallot (Metric #12 and #13, Table 1), were selected by the Quality Network for improvement with the collaborative, beginning January 2017. Regarding counselling for elevated body mass index (Metric #12, Table 1), quality improvement efforts (Fig 7) began with staff education about the metric. A patient education sheet about elevated body mass index was made available for distribution to patients and parents in clinic. Nurses and clinical assistants were recruited to improve clinician recognition of patients with elevated body mass index and to aid in distribution of educational materials. Information technologists assisted in making body mass index percentile an accessible variable for download and electronically linking the patient education sheet to the medical record for ease of access and future referencing. Counselling for elevated body mass index at our centre was highly variable (25–75%), before and following quality improvement efforts, without a clear upward trend related to change strategies.
Counselling for elevated body mass index ⩾85th percentile. QI=quality improvement
Performance of testing or the offering of testing for 22q11.2 deletion in children with tetralogy of Fallot (Metric #13, Table 1) is shown in Figure 8. It was recognised early in the effort that the ascertainment of historic genetic labs, located in numerous domains within our electronic medical record, was particularly challenging and prohibiting accurate reporting. With the help of information technologists, historic genetic lab results were made more easily visible within the active patient chart. Along with staff education about the metric, “talking points” were distributed to providers to aid in the discussion of 22q11.2 deletion testing with patients. Lastly, a simultaneous initiative was underway among our colleagues in general genetics to summarise and interpret relevant genetic findings into a “genetics narrative”. The genetics narrative, as documented in the geneticist’s letter, was developed to improve patient and provider understanding of genetic lab information. The genetics narrative was made available by electronic link to the cardiology documentation system. Testing for 22q11.2 ranged from 60 to 85% over the course of data collection.
Proportion for patients with tetralogy of Fallot for which 22q11.2 testing was performed or offered. QI=quality improvement
Discussion
In 2015, the American College of Cardiology launched the Quality Network, a national quality improvement collaborative designed to benchmark performance in paediatric and adult congenital cardiology care. The Quality Network has successfully engaged diverse healthcare organisations, providing a platform for self-evaluation and centre-to-centre comparison. In addition to analysis of more than a dozen broad scope metrics for benchmarking, the Quality Network chose two metrics for cross-centre collaboration and internal change efforts: (1) ensuring patients with elevated body mass index received appropriate counselling and (2) offering 22q11.2 testing for at-risk tetralogy of Fallot patients. We describe our centre’s experience engaging with the Quality Network, including performance on metrics over the initial 2 years and internal quality improvement efforts.
Several Quality Network metrics involve large populations of clinic patients. Over the past 3 years, we evaluated ~500 patients with chest pain annually in our paediatric cardiology clinics; <1% had a cardiac aetiology to their complaint.Reference Kane, Friedman, Fulton, Geggel and Saleeb 13 – Reference Kane, Fulton, Saleeb, Zhou, Lock and Geggel 15 The magnitude of patient volume and associated high expense of care prompted an earlier quality improvement effort at our institution to standardise care and target more judicious testing to higher yield complaints through the chest pain Standardised Clinical Assessment and Management Plan®.Reference Ogden, Carroll, Kit and Flegal 16 Our management strategies included obtaining detailed family history and electrocardiograms in all children complaining of chest pain, in alignment with principles in the Quality Network metrics. Similarly, we consider echocardiogram to be the test of choice for exertional complaints for detection of myopathic processes and coronary lesions, consistent with the Quality Network position. Similar principles between the chest pain Standardised Clinical Assessment and Management Plan® and the Quality Network metrics likely explain our high performance on the chest pain metrics.
Kawasaki disease management was the focus of multiple metrics shared by the Quality Network. A multi-disciplinary Kawasaki disease team at Boston Children’s Hospital, comprising a small number of physicians, a nurse practitioner, and a nurse, allows for consistency and uniformity in care. Multiple safeguards through the inpatient and discharge process reduce risk of deviation from established care plans and loss of patient follow-up. We directly engaged an integral member of the Kawasaki disease team in our Quality Network workflow. Our performance was high on metrics measuring adherence to Kawasaki disease guidelines,Reference Friedman, Kane and Rathod 12 , Reference Kane, Friedman, Fulton, Geggel and Saleeb 13 including echocardiographic assessment of coronaries, prescribing aspirin, and planned follow-up. Documentation regarding lack of activity restriction following Kawasaki disease diagnosis without coronary involvement was somewhat variable and has become a new focus of internal quality improvement efforts.
Obesity is a global epidemic, affecting one-third of adults and nearly one-fifth of American children.Reference Mandl and Kohane 17 Attention to obesity counselling through the Quality Network is appropriate and timely. Similar to other centres, our performance on counselling for elevated body mass index was highly variable with continued opportunity for betterment. Data collected for this metric required in-depth chart review for each case, lending most centres to report results of a 20 patient audit. In our experience at a large academic centre, the random audit would at times favour clinicians highly likely to counsel, such as the preventive cardiology group, and at other times would land on those highly disinclined to counsel, such as electrophysiology physicians planning procedures. The small sample size results in a high degree of variability, as we have experienced. Despite lower than desired progress on this metric, our ongoing audits note an uptick in counselling by some clinicians for which this behaviour was not customary, suggesting that the message has been received and is being implemented to a degree.
Among change strategies we used, handouts developed by our multi-disciplinary preventive cardiology group, and vetted by patient families, were well received and shared with families by our diverse faculty. Recruitment of nursing and clinical assistants into the project enhanced physician recognition of elevated body mass index and distribution of materials to families. Co-development of our cardiac rehabilitation programme presented an opportunity for longitudinal interaction with our CHD patients, particularly for those sceptical of the effectiveness of in-clinic counselling alone. We continue to strive for improved physician awareness and engagement via sharing of cross-institutional experience through Quality Network forums.
High prevalence of 22q11.2 deletion in tetralogy of Fallot patients and numerous associated non-cardiac comorbidities facilitated reception of the metric by our broad clinician base. Although early quarters of data collection would suggest our institutional performance was close to the national mean, testing was being performed at higher levels before quality improvement effort (~80%) and was initially underestimated as results of historic genetic labs were difficult to uncover. Early quality improvement efforts focussed on improving visibility of historic genetic testing in the electronic medical record, making results more easily apparent in the active patient chart. Similar to our experience with counselling for elevated body mass index, although our reported performance of 22q11.2 testing in tetralogy of Fallot patients did not appear to improve significantly with continued change efforts, our random audit noted both increased physician documentation of testing status in recurring portions of notes, as well as discussion of the importance of testing with patients, including risk of inheritability of the deletion. Such behaviour would suggest the intent of the metric was heard and applied in an effort to improve quality of care. Resistance to testing remains for teenage patients in whom the genetic abnormality is not suspected by the cardiologist, accounting in large part for the plateau in our performance around 85%.
Among our quality improvement strategies, “talking points” developed by our cardiac genetics colleagues to assist in discussion of testing of 22q11.2 deletion with families were well received, particularly around clarification of insurance questions for our patients. Modifications of the medical record with respect to visualising genetic lab results assisted not only this project, but care of other patients with genetic anomalies. Ongoing efforts by the working group through data sharing and education will hopefully allow us to improve performance, ensuring all patients are appropriately tested and counselled, particularly before their reproductive period.
Our initial experience with the Quality Network as a large academic institution has been rewarding. We were able to identify our areas of strength for maintenance, and those requiring ongoing dedication to improve patient care. Data collection was minimally burdensome for the majority of metrics, with random sampling assisting in metrics requiring more time-consuming chart review. Data sharing with the Quality Network was facilitated through a personalised web link, requiring only numerators and denominators to be loaded. As is often the case in quality improvement effort, engagement with the Quality Network allowed us to partner more effectively with our own quality improvement colleagues, which further bolstered other work throughout the department.
Despite the benefits, there are limitations to participation in the Quality Network. For some metrics, small clinic populations of patients are targeted, limiting the reach of the quality improvement effort. The metrics shared thus far are mostly process-based; a focus on patient outcomes would be ideal, particularly as medical care models are shifting to be patient-centric.Reference Barry and Edgman-Levitan 18 , Reference Kilo 19 Data collection on a quarterly basis limits opportunities for frequent incremental test of change cycles. Participation in the Quality Network requires institutional prioritisation and monetary commitment, including membership fees, physician time away from other activities, support personnel, and travel for meetings and presentations. At our institution, team members engaged in other quality improvement work were directed towards this effort, however at other centres, teams may need to be built and funded from the ground-up. Quality improvement collaboratives have proliferated as a potentially powerful tool to improve patient care, although evidence for impactful change, enduring improvement, and cost-effectiveness is limited.Reference Wells, Tamir, Gray, Naidoo, Bekhit and Goldmann 20 – 23 Definitive improvement on the Quality Network projects for change has not been realised yet, as evidenced by our performance as well as the performance of the aggregate during quality improvement efforts.
In conclusion, we feel the platform created by the Quality Network shows promise in engaging providers at the local and national level in domains of paediatric and adult congenital cardiac care that have not been previously measured effectively. Assessment of sustainability, cost-effectiveness, applicability across care models, and ability to meaningfully impact care and promote culture change towards quality improvement will ultimately determine the success of the Quality Network.
Acknowledgements
We would like to acknowledge the contributions of Patrick Gould and Zachary Pitkowsky in data collection.
Financial Support.
This research received no specific grant from any funding agency, or from commercial or not-for-profit sectors.
Conflicts of Interest.
None.
