Introduction
Surgical site infections (SSI) are severe healthcare-associated infections that can be prevented with appropriate preincision antimicrobial prophylaxis. Reference Bwire, Magati and Ntissi1 Antimicrobials administered after skin closure for the purposes of reducing infections are both ineffective and harmful, increasing postoperative adverse events and contributing to long-term antimicrobial resistance. Reference Berenguer, Ochsner, Lord and Senkowski2–Reference Carey, Stefos, Zhao, Borzecki and Rosen4
To increase the appropriate use of antimicrobial prophylaxis, the Surgical Care Improvement Project (SCIP) was developed by the Joint Commission and Centers for Medicare and Medicaid Services and broadly implemented in 2005, including in the Veterans Health Administration (VHA). 5–12 A major component of SCIP was active surveillance with public reporting of facility-level compliance to encourage uptake of evidence-based practices. Implementation of the surveillance program included manual review for identification of eligible surgeries with complex inclusion and exclusion criteria followed by identification of compliance with several evidence-based infection prevention metrics that assessed appropriate timing (INF-1), antimicrobial choice (INF-2), and discontinuation (INF-3), defined as within 24 hours after surgery for non-cardiac cases and within 48 hours after surgery for major cardiac procedures. A significant concern was exclusion of surgeries with appropriate use of antimicrobials (eg, pre- or postoperative infection). Reference Schonberger, Barash and Lagasse10
VHA compliance with SCIP INF metrics increased rapidly over time and achieved >95% facility compliance by 2009. Reference Hawn, Vick and Richman13 In the setting of sustained improvements, SCIP INF metrics were retired in 2015, as the surveillance and public reporting process was felt to be costly with limited additional expected benefit. Reference Hawn, Vick and Richman13,Reference Itani14 SSI rates are still publicly reported in the VHA which presents an opportunity for an electronic surveillance approach to measure compliance with appropriate discontinuation of antimicrobial prophylaxis for SSI prevention. 15 The objective of this study was to develop and test such a tool using the large corpus of SCIP manually reviewed compliance data from the VHA as the gold standard and the extensive electronic health record (EHR) data to program algorithm components. To demonstrate the validity of this approach, we focused our efforts on measuring compliance with SCIP INF-3, appropriate discontinuation of antimicrobial prophylaxis in major cardiac surgeries. Reference Branch-Elliman, Elwy and Lamkin16
Methods
Data sources
This retrospective cohort study used VHA EHR data from the Corporate Data Warehouse (CDW) and SCIP manually reviewed cardiac surgery compliance/non-compliance data from the External Peer Review Program (EPRP). Reference Veterans Health Administration17 EPRP data were obtained with a data use agreement with the VHA Office of Analytics and Performance Integration. Sample demographics and comorbidities were obtained from the CDW. Reference Elixhauser, Steiner, Harris and Coffey18 Manual EHR review was performed by the study team using the VA Joint Legacy Viewer. Reference Williams19 CDW data were accessed in SQL Server, and data analysis was performed in SAS Enterprise Guide 8.3. This study was approved with a waiver of informed consent by the local Research and Development Committee.
Study population
We examined SCIP-qualifying cardiac surgeries performed in the VHA from Oct 1, 2005– Sept 30, 2015 [fiscal years 2006–2015] when the EPRP manual surveillance program was active (n = 52,310 cardiac surgeries from 43 VHA facilities). We excluded surgeries from FY2005, the year the program was initiated, as facilities had significant data missingness.
Qualifying cardiac surgeries were identified from the CDW Surgery Tables by Current Procedural Terminology (CPT) codes for the two cardiac surgery types targeted by SCIP: coronary artery bypass surgery (CABG) and valve replacement procedures. CDW procedures were merged with surgeries from EPRP that included INF-3 data, n = 33,061 (62.6% of all CDW cardiac surgeries) by patient identifier, surgery date, and surgery type (Figure 1, the Development Sample).
Development and validation samples for INF-3 cardiac surgery electronic algorithm.
Note: CDW = Corporate Data Warehouse; CPT = Current Procedure Terminology; EPRP = External Peer Review Program; ID = identifier.
Figure 1. Long description
The flowchart begins with two data sources: VA CDW SCIP Qualifying CPT-Coded Surgeries and EPRP Chart Review ICD9/10 Raw Data. Each data source starts with a large number of cases, which are then filtered to remove cases with missing identifiers or duplicate rows, keeping only cardiac surgery cases. The VA CDW data is further filtered to exclude non-cardiac surgeries, resulting in 42,643 eligible cases. The EPRP Chart Review data is filtered to keep cardiac surgery and INF-3 cases, resulting in 29,431 eligible cases. The eligible cases from both data sources are matched by patient ID, date, and surgery type. The matched data is then split into Non-INF-3 Reviewed Cardiac Surgeries and reviewed by EPRP for other INF measures. The Non-INF-3 Reviewed Cardiac Surgeries are further split into a validation sample and a sample reviewed by EPRP for other INF measures, resulting in 10,131 and 3,081 eligible cases, respectively after applying exclusion criteria.
CDW cases with SCIP-qualifying procedures and without a matching EPRP-reviewed surgery were used for algorithm validation (n = 12,703, 24.3%); discussion with the VHA Office of Analytics and Performance Integration determined cases may not have been reviewed by EPRP due to facility-level staffing gaps. An additional 6,546 (12.5%) CDW cardiac surgeries were in the EPRP sample and did not have INF-3 results (Figure 1)
Algorithm development
Our algorithm development process involved two general steps: (1) iteratively honing exclusion criteria to classify cases as either eligible or ineligible for SCIP Infection Measure Designator INF-3, Prophylactic Antibiotics Discontinued Within 48 H after Cardiac Surgery End-Time review, eg, measurement of the appropriate denominator, and (2) classifying eligible cases as receiving either guideline compliant or guideline non-compliant surgical prophylaxis, eg, measurement of the numerator.
Exclusion criteria in electronic algorithm: determining the correct denominator
The SCIP guidelines had several exclusion criteria that were manually determined, 5 and the EPRP data set did not include data on how case exclusion was applied. To address this gap, we began applying the SCIP exclusion criteria by excluding surgeries in patients under 18, patients with a length of stay > 120 days, patients with additional surgeries under general anesthesia within four days, and patients who died within two days following surgery using the CDW Surgery, Inpatient, and Death Ascertainment Tables.
In addition to data errors and evidence of additional surgical care meriting prophylactic treatment within four days of surgery, the driving goal of the SCIP INF-3 exclusion list was to remove cases where antimicrobial administration >48 hours postoperatively was appropriate due to the presence of an active infection in whom prolonged antimicrobial use is clinically necessary. Cases with active infections included patients with preoperative infectious diseases or infections, patients who were already taking antibiotics prior to surgery for certain conditions (eg, chronic trimethoprim/sulfamethoxazole prophylaxis), and patients who were started on antimicrobial drugs following surgery for a postoperative infection (eg, early postoperative sepsis or pneumonia). To address the aim of the SCIP denominator exclusion process, the standard for active infection adopted by the study team was whether the EHR contained clinical documentation of, for example, pneumonia, urinary tract infection, SSI, or sepsis. Cases where clinicians administered antibiotics due to concern for an infection (eg, postoperative fever leading to concern for sepsis that was later ruled out) were also excluded as appropriate use was appropriate while the work-up was ongoing. Clinician concern about prolonged antimicrobial use for preventing infection, rather than treatment of an infection, was considered inappropriate prophylaxis and not a cause for exclusion. We developed multiple approaches to replicate the SCIP exclusion criteria (Table 1) and confirmed reliability of these approaches with manual chart review. The lead clinician on the study team trained chart reviewers in applying exclusion criteria, particularly identification of infection, and identifying medication timestamps in the EHR. The clinician performed routine case re-review to ensure reliability and handled any questions about case ascertainment.
Comparison of SCIP Exclusion Criteria Used by EPRP and Application to Electronic Data in the INF-3 Algorithm Development Sample (n = 33,061)
Table 1. Long description
A table with 15 rows and 3 columns. The columns are labeled ‘SCIP exclusion criteria’, ‘Application to electronic data’, and ‘N (%)’. The table compares various SCIP exclusion criteria and their application to electronic data, including the number and percentage of cases excluded for each criterion from the development sample. Notable criteria include patient age under 18 years, length of stay over 120 days, and patients with pre or post-operative infections.
Note: Cases may be excluded for multiple reasons. These exclusion data are among the development sample of CDW cardiac surgeries matched to EPRP reviewed surgeries from 2006–2015 (n = 33,061).
Several strategies were evaluated to electronically identify potential infection cases that should be appropriately excluded from the SCIP denominator to determine the appropriate cohort for ascertaining compliance. These included assessing diagnosis codes, laboratory tests, vital signs, and imaging orders for infection signals. Reference Foster, Branch-Elliman and Mull20 We also developed a list of therapeutic antimicrobial drug names that may be used to treat ongoing infections as an option to detect pre-and postoperative infections. The list was developed with chart review and clinical input from surgeon and infectious disease physicians on the study team.
One addition was made to the electronic algorithm to narrow the eligible sample of cardiac surgeries. Patients on dialysis were expected to have more complicated medication adjustments that would lead to inconsistent measures of daily exposure and the potential for gaps in antimicrobial pharmacy administration resulting in inability to classify dosing regimens as continuous or new administrations following surgery. Drug administration data in this sub-population is unreliable and there was a high potential for misclassification. Reference Snyder, Patel, Kallen, Strom, Tucker and DAgata21 Therefore, patients with an average serum creatinine >3.0 mg/dL during the six months prior to the surgical procedure were assumed to be on dialysis and/or with significantly impaired renal function impacting measurement of antibiotic dosing intervals and excluded. Reference Golenbock, Basnet and Mull22
Compliance measure: ascertaining the numerator
Following application of the exclusions to determine the correct denominator, the electronic algorithm was developed to determine which cardiac cases were compliant with appropriate antimicrobial discontinuation, and which cases were non-compliant with INF-3 and had inappropriate prolonged prophylaxis medication administration after 48 hours of skin closure. Using the CDW Surgery Tables, we determined time of skin closure to be operation end time. The algorithm compliance/non-compliance measurement criteria were adjusted against gold standard EPRP findings and the algorithm was considered complete when the a priori goal of sensitivity >80% and specificity >95% was reached. This threshold is appropriate in developing process measure tools to minimize false negatives. Reference Bujang23
With clinical input, we categorized pharmacy administration data into a list of perioperative prophylactic antimicrobials for cardiac surgery. We included medications that are known to be administered for postprocedural prophylaxis Reference Mull, Stolzmann and Kalver24,Reference Bratzler, Dellinger and Olsen25 ; many of these are oral medications that are provided at the time of discharge (eg, cephalexin, trimethoprim/sulfamethoxazole, doxycycline). 5
The iterations to develop the INF-3 compliance measure included adjusting drug names in the prophylaxis and treatment categories, drug administration route (oral or intravenous), and timing, including adjusting the starting point for “skin closure to better align with the EPRP findings, in consideration of possible data entry delays in the CDW. Samples of discordant cases with EPRP underwent manual review to determine whether the algorithm correctly identified the case as non-compliant.
Algorithm validation
To validate our electronic algorithm, we first applied our SCIP-derived exclusion criteria to determine the appropriate denominator (eg, to exclude cases with preexisting infections) and then determined case compliance (numerator) in the validation sample of CDW-cardiac surgeries that were never reviewed by EPRP. A trained reviewer manually assessed a random sample of noncompliant surgeries (n = 210) and randomly sampled compliant surgeries (n = 90) using an electronic chart review tool in REDCap. This process confirmed the accuracy of the electronic algorithm in assessing compliance and excluding cases. Questionable cases were adjudicated by a clinician on the study team. Validation results were used to further adjust the algorithm to ensure adherence to SCIP metrics.
Comparison of facility INF-3 compliance by EPRP and algorithm
When the final validated algorithm reached sensitivity and specificity goals, the algorithm was applied again to the development sample, and to all CDW surgeries to compare the overall compliance rates from the eligible CDW population to that of the EPRP reviewed population.
Results
Of the 52,310 VHA cardiac surgeries from FY2006-2015, patients were older age (65.8 years, ± 8.7 years), white race (80.9%), and male sex (98.7%, Table 2).
Characteristics of VHA cardiac surgeries from FY2006–2015 (n = 52,310, 41 VHA hospitals)
Table 2. Long description
A table summarizing characteristics of VHA cardiac surgeries in the development and validation samples from FY2006-2015. The table presents descriptive data on various characteristics such as age, sex, race, and surgical outcomes in the overall population and the development and validation samples. Notable trends include the majority of patients being older, white, and male. Specific values and detailed trends are visible in the table.
Note: Descriptive table of cardiac surgeries from the CDW and the EPRP data set. Columns 2 + 3 + 4 + 5 are from the CDW data set (column 1, n = 52,310); Columns 3 and 5 are from the EPRP data set with INF-3 data. Values are means and standard deviation or median and 25th and 75th quartile for continuous variables; percentages or totals or both for categorical variables.
Values of polytomous variables may not sum to 100% due to rounding.
**Comorbidities calculated with Elixhauser software. Reference Elixhauser, Steiner, Harris and Coffey18
Denominator ascertainment
In the development sample of 33,061 matched CDW-EPRP qualifying cardiac surgeries with INF-3 data, we applied the electronic exclusions for age, length of stay, mortality, additional surgical encounters, dirty wound, and serum creatine with EHR structured data.
The potential electronic algorithm exclusion components for preoperative infection (diagnosis codes) and postoperative infection (eg, diagnosis codes, vital signs, microbiology and imaging orders) all overcounted cases with a true infection and excluded too many cases from the denominator assessment. Therefore, we adjusted the algorithm to exclude cardiac cases with a preoperative infection/infectious condition or a postoperative infection within 72 hours (0–3 days) of surgery using antimicrobial administration data. The list of therapeutic antimicrobials indicative of a pre-or postoperative infection underwent multiple revisions with chart review confirmation to improve precision in sorting appropriate therapeutic/prophylactic antimicrobials and timing of administration (Supplemental Digital Content 1).
We expanded the SCIP INF-3 exclusion for additional surgeries before or after the index case to include cardiac device placement procedures. We also added an exclusion for same day emergency procedures.
Numerator ascertainment
There were 13 VHA facilities with low sensitivity compared to EPRP after these adjustments. A subsequent manual review of 271 randomly selected discordant cases in these facilities found discrepancies between EHR data and EPRP results in 39/116 (33.6%) of cases. These results explained facility-level discordance and the algorithm was compared to EPRP to establish criterion validity. At this development stage, the algorithm excluded 7,579/33,061 (22.9%) of the matched cases and the algorithm agreement rate was 95.6% compared to 96.0% in EPRP. Sensitivity and specificity were 83.7% (95%CI: 81.6%, 85.8%) and 98.9% (95%CI: 98.8%, 99%), respectively.
Algorithm validation
In applying the algorithm to validation data and sampling 300 cases for review, 18 flagged noncompliant cases met exclusion criteria for pre/postoperative infection. Further, 16 flagged noncompliant cases had appropriate discontinuation of prophylaxis and then a new antimicrobial prescribed after the 48-hour window in manual review. Lastly, 20 cases had incomplete or missing medication administration documentation in the EHR.
Based on the validation findings, we adjusted the electronic algorithm’s pre/postoperative infection exclusion to account for the timing of prophylactic and therapeutic drug administration. We found many instances of both therapeutic and prophylactic antimicrobial pharmacy administration at the time of surgery with no evidence of infection in chart review. We determined that the algorithm was excluding cases that should be in the INF-3 denominator and the exclusion rate was artificially high. Therefore, we updated the algorithm to (1) exclude from the denominator cases with a pre-or postoperative infection based on ≥ 2 doses consecutive therapeutic antibiotic doses within 24 hours 1–7 days before surgery or up to 72 hours after, or (2) flag cases as SCIP INF-3 non-compliant if ≥ 2 doses consecutive prophylactic antibiotic doses were administered within 24 hours up to 72 hours after surgery (Figure 2). A gap greater than 24 hours between doses was treated as separate antimicrobial administration events based on our chart review findings and did not impact numerator compliance determination. Following the validation changes, we applied the final algorithm to the EPRP reviewed development data set and excluded 3,630/33,061 (10.7%) of cases from the denominator of SCIP eligible surgeries (Table 1).
Final cardiac surgery electronic algorithm definitions for INF-3 compliance, non-compliance, and exclusion criteria.
Note: This Figure represents specific scenarios of drug administration timing and drug type that distinguishes prophylactic administration from therapeutic administration consistent with a possible infection.
These changes appropriately excluded SCIP-ineligible cases from the denominator and improved the sensitivity and specificity of the algorithm in the validation data set: 92.1% (95%CI: 88%, 96.2%) and 97% (95%CI: 93.6%, 100%), respectively.
The final algorithm applied to the EPRP development sample had sensitivity = 75.8% (95%CI: 73.4%, 78.2%) and specificity = 99.3% (95%CI: 99.2%, 99.4%) using EPRP as the gold standard; however manual review of discordant cases found inaccuracy in EPRP results. The patient and surgery characteristics were similar in the development and validation samples (Table 2). Excluded cases had longer duration and were more often in American Society of Anesthesiologists Class 4–5 or valve replacements, which is as expected finding as valve replacement surgeries are more likely to be performed in the setting of an active infection (eg, for management of endocarditis) than CABG surgeries. Slightly over half (3,465/6,546, 52.9%) of the EPRP-reviewed cardiac surgeries missing INF-3 data were excluded from the denominator by the electronic algorithm, suggesting these cases with missing INF-3 results were due to EPRP reviewer’s undocumented determination of SCIP denominator ineligibility.
The final INF-3 algorithm excluded 9,667 (18.5%) of the 52,310 cardiac surgeries in the CDW from the denominator (Supplemental Digital Content 2). The primary driver of exclusion from the denominator was any antimicrobial pharmacy administration indicative of pre/postoperative infection (7,360/52,310 14.1%), typically due to receipt of broad-spectrum antibiotics during the immediate perioperative period. We observed very high exclusion rates for infection based on use of broad spectrum drugs during the early postoperative period in three facilities and investigation with local clinicians determined therapeutic drugs were used prophylactically due to a facility-level response to antimicrobial resistant outbreaks (specifically, use of cefepime rather than cefazolin due to an increase in Pseudomonas SSI).
Comparison of facility INF-3 compliance by EPRP and algorithm
Overall INF-3 compliance by year, as measured by the electronic algorithm, was very similar to the EPRP determined rate (Figure 3). Of the 42,643/52,310 (81.5%) CDW cardiac surgeries eligible for the INF-3 algorithm denominator, 40,576 (95.2%) were compliant with timely discontinuation of antimicrobial prophylaxis compared to 31,865/33,061 (95.9%) of cases in the EPRP cardiac surgery sample with INF-3 results.
INF-3 compliance in cardiac surgeries: EPRP reviewed results vs Electronic algorithm.
Note: Sample for the EPRP compliance rate were EPRP reviewed cases matched to CDW cardiac surgery cases (the development sample before exclusions, n = 33,061). Sample for the algorithm rate were all CDW cardiac surgery cases with exclusion criteria applied (n = 42,643).
Discussion
SCIP was highly successful at improving antimicrobial stewardship practices in the United States, although the program was ultimately canceled due to the high cost of ongoing implementation. Reference Hawn, Gray and Vick26,Reference Hawn, Richman and Vick27 This study used VHA EHR data to develop and validate an electronic algorithm to replicate the SCIP surveillance process for measuring compliance with timely discontinuation of antimicrobial prophylaxis in major cardiac surgery; this electronic tool can be broadly applied to EHR data to support ongoing evaluation of compliance with peri-operative prophylaxis recommendations. Furthermore, the methodological approach could be adapted to evaluate adoption of more recent guidance, which recommend immediate postoperative discontinuation. Reference Donnelly, Janssen, Vinod, Stone, Harnett and Shaw28–Reference Gliklich, Leavy and Dreyer31 The overall exclusion rate for contiguous therapeutic antibiotic use within 72 hours postoperatively was comparable to rates of postoperative infectious complications, including pneumonia and urinary tract infections, in cardiac surgery research. Reference Gelijns, Moskowitz and Acker32,Reference Zukowska and Zukowski33 The accuracy of this SCIP eligibility exclusion criteria provided an unexpected benefit: a mechanism for identifying clusters of multidrug-resistant organism SSIs based on changes in local practice patterns that fit the algorithm’s postoperative infection exclusion flag. Thus, while the algorithm was developed for the purpose of measuring compliance with appropriate stewardship measures, it could also be used to flag potential SSI clusters by infection prevention programs.
The manual review process deployed during SCIP may be more likely to identify cases with a cause for SCIP denominator ineligibility (eg, clinical trial participation) that the EHR data will not show; however, the complex SCIP denominator eligibility criteria are also prone to human error. Reference Allison, Wall and Spettell34 We found that the EPRP manual review documented INF-3 compliance for surgeries that met electronic exclusion criteria in more than 10% of cases. Manual review is also subject to inconsistencies regarding how and to what extent exclusion criteria are documented, and whether data are available in the EHR. Reference Jones, Haroldsen and Madaras-Kelly35 Compared to all cardiac surgeries that qualified for SCIP INF-3 review, EPRP reviewed only 77%; facility-level data missingness over time suggested staffing issues, which were confirmed by EPRP. An electronic algorithm can be applied broadly and overcome the limitations of manual record review in accuracy, cost, and staff availability to support ongoing perioperative stewardship initiatives.
The exclusion and inclusion development process for this electronic algorithm to measure compliance with timely discontinuation of antimicrobial prophylaxis demonstrates a paradigm for how complex policies that require assessments of both denominators and numerators can be translated into electronic tools to support ongoing surveillance. The algorithm development and validation approach can be expanded to additional surgical specialties, including those that were not initially included under the umbrella of the SCIP program, for surveillance of appropriate antimicrobial practice. We established a robust method to iteratively refine an algorithm that can be applied across a large multi-site health system. SCIP implementation data, in this case from the VHA’s EPRP, can be a valid gold standard for any healthcare system that participated in the program.
This study is not without limitations. As with all studies using VHA data, the results may not generalize to commercial healthcare settings. The patient population in the VHA is disproportionally older, white, and male relative to the civilian population. 36 The electronic algorithm depended on accurately coded data in the EHR and the pharmacy administration records. Therefore, missing or mis-specified data may compromise the validity of the algorithm. This was a particular concern for intermittent dosing of typical prophylaxis drugs (cefazolin, vancomycin) and antibiotic administration documentation with respect to dialysis use. Reference Snyder, Patel, Kallen, Strom, Tucker and DAgata21 Determining whether use greater than 48 hours was intended for prevention or treatment is not possible without manual chart review and potentially discussion with physicians about the rationale for their antimicrobial use. As a result, we excluded patients with renal dysfunction from the SCIP INF-3 algorithm. We used both external EPRP review data and manual review by our team to determine accuracy and found the algorithm was slightly more likely to exclude cases that should be in the denominator and flagged for non-compliance; however, sensitivity and specificity were high for a compliance metric. Reference Bujang23 Additionally, trust in measurement is an important element for obtaining buy-in from clinicians for quality improvement efforts and therefore we believe a bias in favor of higher rates of compliance is preferable to a tool that is overly punitive.
An additional limitation is the passage of time, and the advancements in clinical care that have been achieved since the SCIP program was discontinued. Reference Bwire, Magati and Ntissi1,Reference Calderwood, Anderson and Bratzler37 Alth we made minor adjustments to the denominator defined by SCIP, our objective was a faithful replication of the program’s approach to measure compliance with prophylaxis guidelines from 2006 to 2015. In future work, our methods to develop an electronic algorithm for timely discontinuation of surgical prophylaxis can be adjusted to comply with newer guidelines, which focus on shorter duration of antimicrobial prophylaxis following skin closure. Reference Calderwood, Anderson and Bratzler37 Furthermore, we can apply the algorithm to a wider selection of qualifying surgeries and procedures, such as transcatheter aortic valve replacements, and examine compliance rates and spread to other surgical types and specialties beyond the 2015 SCIP end date.
Conclusions
Our novel electronic algorithm can extend measurement of appropriate antimicrobial use in cardiac surgery through EHR data and support perioperative stewardship initiatives. The tool is efficient and relies on the extensive data fields available in most robust EHR systems. Adopting electronic measurement tools replaces burdensome and non-standardized manual approaches and enables ongoing surveillance of surgical practice.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/ice.2026.10552
Acknowledgements
We acknowledge our steering committee members: Drs. Melissa Perkal, Makoto Jones, Matthew Goetz, Allison Kelly, Joe Francis, Roman Schumann, Danny Chu, Ann Baylor, Shipra Arya, and Elizabeth Wick. Data were obtained from the VHA Office of Performance Measurement (17API2), within the Office of Analytics and Performance Integration, under the Office of Quality and Patient Safety and VA Informatics and Computing Infrastructure, VA ORD 22-D4V, U.S. Department of Veterans Affairs.
Financial support
This study was funded by VA HSR IIR 20-101 (PI Mull & Branch-Elliman). The views expressed in this manuscript are those of the authors. The contents do not represent the views of the U.S. Department of Veterans Affairs or the United States Government.
Competing interests
The funders had no role in the design of the study or preparation of the manuscript. It represents original work that is not under consideration for publication elsewhere. All authors have seen the manuscript, agreed to submit for publication, and declare no conflicts of interest.
