Hostname: page-component-6898cfcb55-kwdml Total loading time: 0 Render date: 2026-11-11T10:45:28.488Z Has data issue: false hasContentIssue false
  • English
  • Français

Assessing concordance and data quality of linked electronic health record and medicare data for amyotrophic lateral sclerosis patients in a multistate research network

Published online by Cambridge University Press:  09 November 2026

Yahia MohamedOpen the ORCID record for Yahia Mohamed[Opens in a new window],
Xing Song,
Tamara McMahon,
Suman Sahil,
Lemuel R. Waitman,
Sudeepa Bhattacharyya and
Greater Plains Collaborative
Yahia Mohamed*
Affiliation:
Biomedical and health informatics, University of Missouri Kansas City, Kansas, MO, USA
Xing Song
Affiliation:
Health Management and Informatics, University of Missouri, Columbia, MO, USA
Tamara McMahon
Affiliation:
Biomedical and health informatics, University of Missouri Kansas City, Kansas, MO, USA
Suman Sahil
Affiliation:
Biomedical and health informatics, University of Missouri Kansas City, Kansas, MO, USA
Lemuel R. Waitman
Affiliation:
Biomedical and health informatics, University of Missouri Kansas City, Kansas, MO, USAHealth Management and Informatics, University of Missouri, Columbia, MO, USA
Sudeepa Bhattacharyya
Affiliation:
Biomedical and health informatics, University of Missouri Kansas City, Kansas, MO, USA
Greater Plains Collaborative
Affiliation:
GPC Collaborators
*
Corresponding author: Y. Mohamed; Email: ym3xb@umkc.edu
Rights & Permissions [Opens in a new window]

Abstract

Background:

Linked electronic health records (EHR) and Medicare claims data are excellent sources for amyotrophic lateral sclerosis (ALS) research. However, quality assessment of data in these sources is critical to produce accurate real-world evidence.

Methods:

In this study we evaluated the consistency and concordance of ALS-linked EHR and Medicare data across the Greater Plain Collaborative (GPC) clinical research network. We used rule-based methods to assess the consistency of demographic, ALS diagnosis, riluzole prescription, and percutaneous endoscopic gastrostomy (PEG) procedure data in both data sources. We used Cohen’s Kappa statistics (k) to assess the concordance between the two data sources and multiple logistic regression models to identify potential driving factors of discordance.

Results:

Our findings revealed that demographic data was mostly consistent across most sites, with the highest discrepancies observed in race and ethnicity. For ALS-specific diagnosis, most sites demonstrate moderate (k = 0.41–0.60) to substantial (k = 0.61–0.80) concordance for ALS diagnosis, riluzole prescription, and PEG procedure. Sensitivity analysis showed that the data in both data sources were concordant across many sites (Sensitivity > 70%). However, results also revealed discordant data at some sites. Age distribution, sex, and the type of insurance used in the index encounter were key factors contributing to discordance.

Conclusion:

These results provided insight into the level of ALS data concordance between EHR and Medicare, as well as illustrated quality issues requiring assessment before use in clinical research and in providing recommendations based solely on available secondary data.

Keywords

Electronic health recordsMedicarePCORnetCommon Data ModelALS

Information

Type
Research Article
Information
Journal of Clinical and Translational Science , Volume 10 , Issue 1 , 2026, e197
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2026. Published by Cambridge University Press on behalf of Association for Clinical and Translational Science

Introduction

Amyotrophic lateral sclerosis (ALS) is a rare, progressive neurodegenerative disorder affecting both upper and lower motor neurons. The disease is uniformly fatal, with most patients dying within three to five years of symptoms onset [Reference Foster and Salajegheh1]. Despite decades of research, ALS remains poorly understood, with limited insight into its underlying pathophysiology, no disease-modifying therapies, and persistent delays in diagnosis [Reference Richards, Morren and Pioro2,Reference Ludolph, Grandjean and Reviers3]. The emphasis in treatment of ALS disease is managing symptoms to improve quality of life [Reference Gupta, Bhandari and Grover4]; however, current gaps that delay early diagnosis affect quality of life and increase financial burden, underscoring an urgent need to leverage high-quality observational data to generate real-world evidence that can improve ALS diagnosis, prognosis, and care delivery.

Electronic health records (EHR) capture clinical findings and interactions between providers and patients including billable and unbillable services delivered within healthcare systems [Reference Rowan, Flory and Gerhard5]. However, EHR do not capture services received outside of the originating system. In contrast, claims data derived from insurance records capture billable healthcare services across multiple systems, making them a rich source of longitudinal information for researchers. Claims data includes information on healthcare utilization and facilitate longitudinal tracking of patient care but lacks clinical findings (e.g. serum potassium level, weight) [Reference Rowan, Flory and Gerhard5,Reference Heintzman, Bailey and Hoopes6]. A key concern in using EHR data for research is the reliability of the data due to inherent data quality issues such as record incompleteness, anomalies of data values or units, and inconsistency of temporal relationships [Reference Weiskopf and Weng7–Reference Kahn, Callahan and Barnard9]. Additionally, claims data have quality concerns regarding diagnoses or procedures due to miscoding, lag and timeline incongruities, inconsistent trends over time, and incomplete dispensing data, especially for beneficiaries who did not enroll in Part D coverage or used self-payment methods [Reference Leonard, Brensinger and Nam10,11]. Researchers have developed approaches to assess and improve the quality of real-world data (RWD) aiming to detect and address data errors to generate reliable real-world evidence [Reference Weiskopf and Weng7,Reference Weiskopf, Hripcsak, Swaminathan and Weng12]. However, there is no consensus on a comprehensive approach to evaluate the data quality of healthcare data.

Interest continues to grow in linking EHR and claims data to study patients with rare diseases who interact with multiple healthcare systems, thereby providing a more complete view of their healthcare journey. In the United States, individuals diagnosed with ALS become immediately eligible for Medicare coverage, creating a uniquely comprehensive longitudinal record of healthcare utilization. The Patient-Centered Clinical Research Network (PCORnet) is one of the national distributed research networks that harmonize and standardize EHR data from eight clinical data research networks (CDRNs) into a Common Data Model (CDM) [Reference Forrest, McTigue and Hernandez13,14]. The Greater Plains Collaborative (GPC) network is one of the PCORnet CDRNs that include 13 leading health systems with data for more than 30 million patients across over eight states in the central United States [15,Reference Waitman, Aaronson, Nadkarni, Connolly and Campbell16]. In addition, the GPC EHR data are integrated with Medicare (2011–2020) and Medicaid (2011–2012) claims data from the Center for Medicare and Medicaid Services (CMS) within the GPC Reusable Observable Unified Study Environment (GROUSE). This integration offers expanded opportunities to support clinical research and to evaluate the quality of EHR data relative to claims [Reference Waitman, Song and Walpitage17]. This makes the GROUSE data environment an especially valuable real-world data source for ALS research, as it captures a near-complete continuum of care through Medicare enrollments and claims while providing enriched clinical context via linked EHR data. However, realizing this potential requires rigorous evaluation of data quality beyond syntactic checks, such as standard PCORnet Empirical Data Characterization reports [Reference Qualls, Phillips and Hammill18–Reference Mohamed, Song and McMahon20]. Specifically, assessing completeness, consistency, and contextual fit-for use across data sources is critical to producing accurate, reliable, and longitudinally integrated datasets capable of supporting high-quality real-world evidence generation in ALS research.

In this study, we assess the data quality of GROUSE through a fit-for use evaluation that systematically compares EHR data and Medicare claims data for their suitability in real-world ALS research. Specifically, we aim to (1) describe discrepancies in demographic and clinical information, (2) analyze data errors in selected data elements, and (3) measure the statistical concordance of the selected, disease-specific data elements across both data sources.

Materials and methods

Ethical approval

CMS approved the reuse of Medicare data (Data Use Agreement # 59442), and the University of Missouri-Kansas City Institutional Review Board (IRB; 396985) approved the reuse of data in GROUSE to conduct this study.

Data source and study population

This study leveraged linked EHR and Medicare data available through the GROUSE environment from January 2011 to December 2020. GROUSE provides de-identified data that follows the PCORnet Common Data Model (CDM v6.1) and is stored in the Snowflake cloud platform on the University of Missouri’s AWS data enclave [Reference Waitman, Song and Walpitage17,Reference McDowell, O’Rorke and Schroeder19]. Data from 13 clinical sites within the GROUSE environment were included in these analyses. To preserve anonymity, sites were labeled A through M. The eligible population consisted of ALS patients identified within the GROUSE database during the study period. ALS patients were identified using specific ICD9-CM (335.20) and ICD10 (G12.21) codes. Patients were eligible if they had one or more ALS ICD-9 or ICD-10 diagnosis code recorded in the EHR data during the study period. These codes were used to extract a crosswalk cohort of patients present in both EHR and Medicare data across GPC sites (Figure 1). This cohort was then used to assess the consistency and completeness of ALS-related information between the two data sources. As this study focuses on data quality, ALS diagnosis was defined based on the presence of ICD codes in patient records, rather than using established algorithm for ascertain clinical ALS diagnosis.

Figure 1.

Flowchart for unique ALS diagnosis across GPC sites.

A Venn diagram showing the distribution of ALS patients across different sites in EHR and Medicare datasets.
Figure 1. Long description

A Venn diagram comparing the distribution of ALS patients across various sites in EHR and Medicare datasets. The diagram consists of three main sections: ALS Only in EHR, ALS in Both Sources, and ALS Only in Medicare. Each section lists the number of ALS patients at different sites. The Total ALS Patients Across Sites is 16664. The Total ALS in EHR is 12420, and the Total ALS in Medicare is 10026. The ALS Only in EHR section has 6637 patients distributed across 13 sites. The ALS in Both Sources section has 5782 patients distributed across the same 13 sites. The ALS Only in Medicare section has 4244 patients distributed across the same 13 sites. Each site’s patient count is detailed within these sections.

Implementation of data quality check

Thorough data quality checks were implemented using SQL in the Snowflake platform [21]. We developed rules to assess the consistency and completeness of demographic data, ALS diagnosis codes, riluzole prescriptions, and PEG procedures. We used unique patient counts to quantify the discrepancies between the two data sources, and percentages were calculated to represent the data inconsistencies or incompleteness. We stratified analyses by site level to account for variation in clinical workflows, data entry, and coding practices across GROUSE sites. This approach allows us to identify data quality issues and provide resolutions at the site.

Assessment of demographic data

Demographic data in the PCORnet CDM demographic table includes birth date, birth time, sex, sexual orientation, gender identity, ethnicity, and race [22]. Medicare demographic data in the Master Beneficiary Summary File (MBSF) includes birth date, death date, sex, age, and race [23,Reference Jarrín, Nyandege, Grafova, Dong and Lin24]. The linked Medicare data available within GROUSE are transformed into the PCORnet Common Data Model, which contains separate race and ethnicity variables. Our analyses used these harmonized PCORnet variables rather than the raw Medicare source files. The “Other” race category reflects the standardized PCORnet CDM race classification and may include multiple racial groups that are not separately represented in the analytic dataset. We assessed the consistency of sex, birth date, death date, race, and ethnicity across the EHR and Medicare datasets using all available linked records for each patient during the study period. We queried the demographic and death tables using rules that extracted and matched the cohort of ALS patients across both data sources. Inconsistencies were detected and evaluated at the patient level.

Assessment of diagnosis codes

We extracted ALS diagnoses from 01-01-2011 to 12-30- 2020 from both data sources and compared the number of unique patients identified in each. To explore potential sources of discordance between data sources, we focused on patients with ALS diagnosis documented in the EHR but not observed in Medicare claims. We conducted a series of hypothesis-driven analyses to identify patient and coverage level factors associated with missing Medicare capture. Specifically, we stratified the cohort: (a) by age at diagnosis, to assess whether younger individuals who may not yet be age-eligible for Medicare or may experience delays in enrollment; (b) by death status and timing of death relative to diagnosis, to evaluate whether early mortality or limited post-diagnosis survival reduced opportunities for ALS-related Medicare claims to be generated; and (c) by insurance type and coverage pathway, to examine whether primary coverage through non-Medicare payers (e.g., private insurance or Medicaid) or delayed transition to Medicare was associated with incomplete claims capture. To conduct these analyses, we used the index encounter for diagnosis, defined as the first encounter with an ALS ICD code. We excluded sites I and M from insurance analysis due to the unavailability of their EHR insurance data.

Assessment of medication prescription and dispensing

We evaluated the concordance of medication data by comparing riluzole prescribing information recorded in the EHR with corresponding dispensing records found in the Medicare claims. Riluzole was selected for this assessment because, during the study period, it was one of the few FDA-approved disease-modifying therapies for ALS and the standard first-line pharmacologic treatment [Reference Ludolph, Grandjean and Reviers3,Reference Thakore, Lapin and Mitsumoto25,Reference Andrews, Jackson, Heiman-Patterson, Bettica, Brooks and Pioro26]. ALS patients with riluzole prescriptions documented in the EHR prescribing tables were identified and linked to Medicare claims to evaluate concordance, completeness, and temporal alignment between prescribing and dispensing data. Riluzole prescription in the EHR were identified using RxNorm codes and medication names, while dispensing events in Medicare claims were identified using National Drug Codes (NDCs). To further investigate discordance between EHR prescribing and claims-based dispensing records, we examined patients’ insurance coverage at the time of the documented riluzole prescriptions, defined as the first encounter with RxNorm codes or names of riluzole, with particular attention to non-Medicare primary coverage that may explain incomplete Medicare capture. We excluded site I and M from this analysis due to the unavailability of EHR insurance data.

Assessment of procedures

We used a similar approach to assess the concordance of procedural recorded in the EHR and Medicare by focusing on percutaneous endoscopic gastrostomy (PEG) tube placements, which are commonly required in ALS patients with bulbar-onset symptoms [Reference Thomas, Schrager, Neel and Brook27,Reference Son, Lee, Ryu, Park and Kim28]. PEG placement serves as a well-defined, high-salience procedural marker in ALS care and is therefore well suited for assessing across-source agreement in procedural ascertainment. PEG procedures were identified in both data sources using a comprehensive set of Current Procedural Terminology (CPT) codes, including 49440, 43246, 43750, 43653, 43830, 43832, 44372, 44372, and 74350. We compared the presence and timing of PEG placement events across the EHR and Medicare claims to assess concordance, completeness, and potential temporal discrepancies between sources. For patients with PEG placements documented exclusively in the EHR but absent from Medicare claims, we conducted additional analyses stratified by insurance coverage at the time of the index encounter of the procedure, defined as the first encounter with CPT codes for the procedure performed after the index date of ALS diagnosis. This analysis aimed to identify whether non-Medicare primary insurance, delayed Medicare eligibility, or alternative payment pathways contributed to incomplete claims capture and procedural discordance. Site I and M were excluded because of the unavailability of EHR insurance data. We also excluded site J because all patients with PEG procedures have missing EHR insurance data.

Statistical analysis

We used Cohen’s Kappa statistic (k) to calculate the degree of agreement between EHR data and Medicare claims. Our interpretation of Kappa results will follow Cohen suggestion as no agreement for values of ≤0, non to slight for values ranging from 0.01 to 0.2, fair for values ranging from 0.21 to 0.40, moderate for values ranging from 0.41 to 0.60, substantial for values ranging from 0.61 to 0.80, and almost perfect for values ranging from 0.81 to 1.00. [Reference McHugh29] In a secondary analysis to assess agreement between EHR and Medicare data, we calculated positive agreement, negative agreement, sensitivity, and specificity for ALS diagnosis, riluzole prescription, and PEG procedure. We treated EHR data as the reference source because it contains richer clinical detail and is closer to real-world clinical practice. A true positive was defined as documentation of a patient record in both the EHR and Medicare data. In contrast, a false positive was defined as documentation present in EHR but not in Medicare. A false negative was defined as documentation present in Medicare but not in EHR, and a true negative as absence in both sources. Variable presence was defined as the presence of at least one recorded code or documentation within the study period, and all comparisons were conducted at the patient level. These measures reflect relative differences in data capture and ascertainment.

We used multivariable logistic regression to evaluate driving factors leading to data discordance of diagnosis, medication, and procedure. Outcomes were the indicator of presence of a certain entity of interest (ALS diagnosis, riluzole prescription, PEG procedure). covariates included age, sex, race, and type of insurance. For demographic data, we considered EHR data as the standard of comparison; therefore, if a data element had missing values in one source, it was classified as discordant (supplementary Table 1). A summary of missing data for demographic and insurance EHR data are illustrated in the supplementary Tables 2, 3. Concordance analyses were conducted among patients successfully linked between the EHR and Medicare data sources, which defines the study denominator. For each variable, we classified observations as all ALS data in the EHR database, the crosswalk cohort, only in the EHR, or only in the Medicare database (supplementary Table 4,6,8). Patients with missing data in both sources were excluded from the concordance calculation as they were non-informative for this analysis. We calculated odds ratios and 95% confidence intervals (CI) using two-tailed statistical significance for comparisons. We used R software (version 4.5.2; R Foundation for Statistical Computing) to conduct analyses and create visualizations.

Results

Demographic data consistency

Across the 13 participating sites, demographic information was largely consistent between EHR and Medicare data for sex and birth date, with concordance exceeding 95% at all sites. In contrast, race, ethnicity, and death date showed substantial gaps. Race data were inconsistent for up to 40.9% of patients at several sites, with sites J, I, K, and I having the most inconsistencies. Ethnicity was almost absent in Medicare, resulting in >65% discordance across most sites. Death dates were generally concordant; however, one site displayed systematic errors with all patients showing mismatched death dates. A summary of demographic inconsistences is presented in (Supplementary Table 1).

Diagnosis concordance

ALS diagnoses were frequently present in EHR data but absent in Medicare claims. At several sites (J, H, F, B, and A) more than half of patients had diagnoses documented in both sources (Figure 2) (Supplementary Table 4). Among patients whose diagnoses appeared only in EHR, more than 40% were younger than 65 years across GPC sites. Additionally, more than 20% of patients with EHR-only diagnoses died before December 2020 (Supplementary Figure 1). Insurance data further explained these discrepancies; many patients with EHR-only diagnoses used private insurance, Medicaid, and other types of medical insurance at the time of their first encounter. However, at sites G, L, E, and K over 20% of patients used Medicare as their primary payer during the index visit, yet their codes did not appear in the Medicare. Insurance data were incomplete at sites C and D, limiting further interpretation (Figure 3) (Supplementary Table 5).

Figure 2.

Proportion of ALS patients with matched EHR-medicare data across GPC sites.

Bar graph showing proportions of ALS patients with matched EHR-medicare data across different sites.

Figure 3.

Type of insurance used for the first encounter in EHR data.

Three bar graphs show insurance types across clinical sites for ALS diagnosis, Riluzole prescription, and PEG procedure.

Medication (riluzole) concordance

Results showed more than 40% of ALS patients in sites A, B, D, F, H, and J have riluzole prescriptions in both sources, while at other sites like, E and L fewer than 20% of prescriptions captured in both sources (Figure 2) (Supplementary Table 6). Variability was influenced by insurance type. At several sites, over one-third of the patients used private or Medicaid insurance for their first riluzole prescription. EHR insurance data of riluzole prescriptions for patients in sites C, D, and F had over 50% missing (Figure 3) (Supplementary Table 7).

Procedure (PEG) concordance

Our results showed that in sites H, L, F, G, and A, respectively more than 20% of patients who had an ALS diagnosis in EHR data had the PEG procedure recorded in EHR data. Most sites revealed that more than 45% of procedure data were captured by both data sources (Figure 2) (Supplementary Table 8). Insurance coverage contributed significantly to discordance; 40% of patients at sites G, L, K, and F used Medicare while fewer than 40% of patients at sites H, E, D, C, B, and A used Medicare insurance. More than 25% of patients at most sites used private insurance during the initial PEG encounter. Overall, insurance use for PEG is illustrated in Figure 3, with site-level data reported in supplementary Table 9. There is significant incompleteness in EHR insurance data for PGE procedure at sites C, D, and E.

Overall concordance and agreement

ALS diagnosis reached substantial concordance at two sites, F and H (k ≥ 0.61–0.80), moderate levels for the majority (K, B, J, A, M, C, L, G, D, and E (k ≥ 0.41–0.60), and fair concordance at one (I (k ≥ 0.21–0.40)). Riluzole prescriptions were captured with moderate concordance for several sites (F, J, H, A, M, and B, respectively (k ≥ 0.41–0.60)) and with fair concordance at others (K, C, I, E, and L respectively (k ≥ 0.21–0.40)). PEG procedures showed high concordance at sites H and F (k ≥ 0.61–0.80), moderate at sites G, D, A, and K (k ≥ 0.41–0.60), but only fair to slight for sites M, L, E, and C (k ≥ 0.21–0.40), and sites I, B, and J respectively (k ≥ 0.01–0.40) (Figure 4). Sensitivity analyses, using EHR data as the reference, indicated that >70% of ALS diagnoses were concordantly captured in Medicare data at three sites; D, F, and L. Similarly, >70 % of Riluzole prescriptions were concordantly captured in Medicare data at sites A, D, F, H, J, K, and L. Finally, >70% of PEG procedures for ALS beneficiaries were concordantly captured in Medicare data for sites G, H, and L. (Supplementary Table 10,11,12).

Figure 4.

Kappa statistics and 95% confidence intervals.

Three scatter plots depict kappa statistics and 95% confidence intervals for ALS diagnosis, Riluzole prescription, and PEG procedure across different clinical sites.
Figure 4. Long description

Panel A: The scatter plot titled ALS Diagnosis shows kappa statistics on the vertical axis ranging from 0.0 to 0.8 and clinical sites labeled A to M on the horizontal axis. Each site has a data point with error bars indicating the 95% confidence intervals. Panel B: The scatter plot titled Riluzole Prescription displays kappa statistics on the vertical axis ranging from 0.0 to 0.8 and clinical sites labeled A to M on the horizontal axis. Each site has a data point with error bars indicating the 95% confidence intervals. Panel C: The scatter plot titled PEG Procedure shows kappa statistics on the vertical axis ranging from 0.0 to 0.8 and clinical sites labeled A to M on the horizontal axis. Each site has a data point with error bars indicating the 95% confidence intervals.

Factors associated with data discordance

Multivariable regression identified drivers of discordance between data sources. Patients under 65 years of age were significantly less likely to have concordant ALS diagnosis compared to the age group > 65 years for site D ( OR 0.27, 95% CI 0.21–0.34, p = < 0.001), F(OR 0.82, 95% CI 0.71–0.95, p = 0.01), G (OR 0.32,95% CI 0.20–0.47, p = 0.001), and H (OR 0.71,95% CI 0.54–0.92, p = 0.01). Whereas ALS diagnosis concordance in this age group was higher in site E (OR 2.06, 95% CI, 1.32–2.26, p = 0.001) and J (OR 3.55, 95% CI 2.11–6.13, p < 0.001). ALS diagnosis concordance was higher among females compared to males in sites C (OR 2.23, 95% CI 1.31–3.91, p = 0.003), F(OR 1.22, 95% CI 1.10–1.35, p < 0.001), and site G (OR 1.38, 95% CI 1.06–1.80, p = 0.02). While it was lower in site E (OR 0.2, 95% CI 0.13–0.31, p = 0.001) and site K (OR 0.39, 95% CI 0.32–0.49, p < 0.001) (Figure 5).

Figure 5.

Forest plot for logistic regression output of ALS diagnosis data concordance.

Forest plots display logistic regression outputs for ALS diagnosis data concordance across multiple sites.

Riluzole concordance was higher for patients less than 65 years only in site E (OR 1.59, 95% CI 1.15–2.23, p = 0.01). While it was lower in this age group for majority of the other sites like A (OR 0.39, 95% CI 0.34–0.44, p < 0.001), B (OR 0.58, 95% CI 0.46–0.74, p < 0.001), D (OR 0.76, 95% CI 0.65–0.88, p < 0.001), F(OR 0.25, 95% CI 0.19–0.32, p < 0.001), G (OR 0.25, 95% CI 0.12–0.51, p < 0.001), H (OR 0.50, 95% CI 0.28–0.90, p = 0.02), K (OR 0.54, 95% CI 0.46–0.65, p < 0.001) and L (OR 0.51, 95% CI 0.28–0.89, p = 0.02). Female sex was associated with higher Riluzole prescription concordance in the majority of the sites like A (OR 3.80, 95% CI 3.33–4.33, p < 0.001), D (OR 1.22, 95% CI 1.08–1.37, p < 0.001), E (OR 2.25, 95% CI 1.69–3.03, p < 0.001), G (OR 2.27,95% CI 1.15–4.50, p = 0.02 ), H (OR 1.92, 95% CI 1.13–3.28, p = 0.02), and J (OR 1.44, 95% CI 1.37–1.53, p < 0.001). However, the odds of concordance were lower in site C (OR 0.33,95% CI 0.27–0.41, p < 0.001) and site K (OR 0.76, 95% CI 0.64–0.90, p = 0.001) (Figure 6).

Figure 6.

Forest plot for logistic regression output of riluzole prescription data concordance.

Multiple forest plots display odds ratios for various demographic and insurance factors.
Figure 6. Long description

Panel A: A forest plot shows odds ratios for different demographic and insurance factors. The x-axis represents the odds ratio on a log scale, and the y-axis lists categories such as age, sex, race, and insurance type. Each point represents an odds ratio with error bars indicating confidence intervals. Panel B: Another forest plot with similar structure and axes, showing different odds ratios for the same factors. Panel C: A forest plot with odds ratios for demographic and insurance factors, with the x-axis and y-axis labeled similarly. Panel D: A forest plot displaying odds ratios with the same axes and categories. Panel E: A forest plot with odds ratios for the listed factors, following the same structure. Panel F: Another forest plot with similar axes and categories, showing different odds ratios. Panel G: A forest plot with odds ratios for demographic and insurance factors, maintaining the same structure. Panel H: A forest plot displaying odds ratios with the same axes and categories. Panel J: A forest plot with odds ratios for the listed factors, following the same structure. Panel K: Another forest plot with similar axes and categories, showing different odds ratios. Panel L: A forest plot with odds ratios for demographic and insurance factors, maintaining the same structure.

Concordance of PEG procedures were lower for patients <65 years of age in sites B (OR 0.02, 95% CI 0.0–0.66, p = 0.03), C (OR 0.03, 95% CI 0.00–0.32, p =0.002), D (OR 0.44, 95% CI 0.25–0.77, p = 0.003), J (OR 0.05, 95% CI 0.00–0.30, p < 0.001) and site K (OR 0.61, 95% CI 0.40–0.92, p = 0.02). Female patients had lower concordance for PEG procedures only in site L (OR 0.52, 95% CI 0.28–0.96, p = 0.04) (Figure 7).

Figure 7.

Forest plot for logistic regression output of PEG procedure data concordance.

Forest plots display logistic regression outputs for PEG procedure data concordance across multiple panels.

The data was insufficient to provide reliable results for race and insurance. Small sample sizes for African American, other, or unknown races compared to the whites, and different types of insurance compared with Medicare and missing EHR insurance data at several sites resulted in unreliable results. (supplementary Tables 13,14,15).

Discussion

The GROUSE database links heterogenous real-world data sources (e.g., EHR and CMS administrative claims) together, enabling a more comprehensive and longitudinal characterization of patient care, utilization, and outcomes than either source alone. The EHR data provides detailed clinical information, including diagnoses, vital signs, laboratory results, and prescribing intent, while CMS claims offer near-complete capture of reimbursed healthcare encounters, procedures, and medication dispensing across care settings. Successful linkage requires attention to information alignment between different data provenances. By systematically evaluating concordance and discordance across key clinical, medication, and procedural domains, this analysis provides empirical insight into which data elements derive the greatest incremental value from multi-source integration and informs best practices for leveraging linked EHR-claims data in real-world ALS research.

Core demographic variables, including sex and date of birth, demonstrated high concordance (95%) across EHR sites, while race and ethnicity exhibited substantially lower consistency across most GPC sites. Race information was missing or discordant in up to 40% of linked records, while ethnicity data were largely absent from Medicare claims, reflecting well-documented limitations in federal administrative data, particularly among underrepresented populations [Reference Jarrín, Nyandege, Grafova, Dong and Lin24]. One possible reason for ethnicity missingness is that CMS encodes Hispanic in the race field, and therefore, there is no separate Hispanic field as exists in EHR data, which creates inconsistency during data harmonization. These findings corroborate prior reports that highlight incomplete demographic documentation as a major barrier for evaluating health equity using secondary data sources [Reference Magaña López, Bevans, Wehrlen, Yang and Wallen30]. Additionally, inconsistencies in death rates were also noted for some sites. Site G, had systematic error (all dead ALS patients have the same year of death in EHR which was inconsistent with year of death recorded in claims data) likely arising from data ingestion and warrant local validation. Other sites had a few inconsistencies due to differences in death dates between the two data sources that varied by a few days or months. These inconsistences could be due to time lag or might be related to the different sources from where mortality data was sourced [Reference Jamal-Allial, Sponholtz and Vojjala31]. Together, these results emphasize that even in harmonized networks, demographic data uniformity cannot be assumed, and site-level audits are essential for ensuring reliable use in population-level or disparity-focused analyses.

ALS diagnosis codes showed moderate to substantial concordance across most sites. But patterns of discordance were evident in many sites. This discrepancy was primarily driven by age and insurance status. Over one-third of patients with EHR-only diagnosis were younger than 65, probably being ineligible for full Medicare coverage at the time of diagnosis. Even though ALS qualifies patients for immediate Medicare eligibility [32], practical enrollment lags or claims generation is delayed due to concurrent private or Medicaid coverage. A 2022 survey of ALS patients about their experiences with health insurance and payment for ALS found that 80% of participants reported using Medicare [33]. Therefore, sites in GROUSE with >20% of patients who used Medicare at the first ALS encounter but did not appear in claims data suggest coding or data capture issues, a gap in Medicare eligibility, or false-positive ALS.

These findings highlight incompleteness in claims data does not necessarily indicate EHR error but rather reflect real-world administrative and payer dynamics. Nelson et al. found that the missing diagnosis in the national ALS registry was likely due to patients who were younger than 65 using private insurance or other insurance provided by employers [Reference Nelson, Topol and Kaye34]. The average time elapsed from index diagnosis to death for patients with only EHR data and died before December 2020 was 1.2 years, which indicates a limitation for the likelihood of generating Medicare claims, which is seen as discordance (supplementary Table 16). An additional reason may be the reliance on diagnosis codes rather than the validated ALS case-ascertainment algorithm [Reference Mehta, Raymond and Han35]. The National ALS Registry algorithm for identifying ALS patients using ICD diagnosis codes, neurology visits, whether the cause of death was due to ALS, and riluzole prescriptions in patient records. Further classifies the diagnosis as definite ALS, possible ALS, or not ALS. According to Mehta et al., structured codes alone can misclassify “rule-out” or provisional ALS diagnoses, leading to false positives in EHR not reflected in claims data [Reference Mehta, Raymond and Han35].

Riluzole prescribing rates derived from EHR data varied substantially across GROUSE sites, ranging from 12% to 56%. This variability likely reflects differences in clinical context, such as care delivered in ALS specialty centers versus primary care settings, as well as heterogeneity in medication documentation practices across sites. Overall, approximately 30% of riluzole prescriptions recorded in EHR could be matched to a corresponding dispensing event in Medicare claims. Notably, across most sites, up to 40% of patients used private insurance at the time of their initial riluzole prescription, which plausibly accounts for a substantial proportion of the observed discordance between EHR prescribing and claims-based dispensing records. Additionally, the difference between prescribing intent and dispensing intent reflected in EHR and claims data, respectively, may reduce apparent concordance because not all prescriptions are filled or captured through Medicare Part D [Reference FitzHenry, Peterson, Arrieta, Waitman, Schildcrout and Miller36,Reference Lin, Daley and Boone-Heinonen37]. Concordance analyses using Cohen’s Kappa indicated fair to moderate agreement for riluzole use, consistent with findings that evaluated agreement between EHR and Medicare Part D data for oral anticancer medications [Reference Sharma, Johnson, Zhao, Giordano and Holmes38].

For PEG procedures our results revealed fewer than 10% of ALS patients across EHR sites had a PEG procedure recorded, which is lower than 13% to 40% previously reported rates r [Reference Katzberg and Benatar39]. Across most sites, approximately 45% of PEG procedures were captured in both the EHR and Medicare claims databases, indicating moderate concordance between data sources. Examination of insurance coverage at the time of the initial PEG procedure documented in the EHR revealed that more than 25% of patients at most sites had private or other non-Medicare insurance as their primary payer. This coverage pattern plausibly explains why fewer than half of PEG procedures were observable in Medicare claims. Collectively, these findings underscore how variation in insurance coverage and site-level procedural documentation practices can substantially influence procedural ascertainment in linked EHR-claims analyses.

Overall, the sensitivity analysis showed that high-data-quality EHR sites had more than 70% of ALS-related data elements were concordantly captured in both sources. Multivariate regression analysis revealed demographic factors identified as drivers of discordance. Younger age (<65) and non-Medicare insurance types were strongly associated with missing or inconsistent data. Female patients tended to show higher concordance in several sites, possibility reflecting greater healthcare utilization or more complete documentation. Although race and insurance showed varying site-specific significance, small sizes and missing data limited interpretation. Together, these findings suggest that demographic, care setting, and insurance type are non-random systemic factors causing much of the variation in data completeness across sites.

The strength of our study lies in its use of a large, multisite database to evaluate two critical aspects of data integrity, data quality and concordance between linked EHR and Medicare records for ALS patients. To our knowledge, this is the first study of this scale that examines and quantifies the variability in data consistency and completeness across 13 clinical sites, providing a robust comparison of EHR data with Medicare claims. The findings contribute valuable insight into the reliability of real-world data and increase our understanding of how healthcare information is captured, harmonized, and potentially biased across data systems. Importantly, this analysis offers insight that are generalizable and may have significant implications for ALS research by highlighting differences in data capture between two key secondary data sources. These insights can help researchers evaluate the validity and reliability of linked datasets, reduce analytic bias, and strengthen the generation of real-world evidence. Moreover, by quantifying Riluzole use and PEG procedures among ALS patients, our results may inform both clinical decision-making and design of future studies aimed at improving care quality and treatment outcomes. Although focused on ALS, our approach for evaluating concordance and inconsistency in data capture between EHR and claims data can be applied to other clinical conditions and data sources, including cancer and heart failure registries and patient-reported data. This framework can be used to systematically assess data quality issues such as redundancy and consistency across heterogeneous healthcare data systems.

This study has several limitations. Even though we identified the number of unique patients who appeared only in EHR or Medicare data, we could not account for care received outside the GPC network, which may contribute to incomplete data capture. We used ICD-9 and ICD-10 codes to identify the ALS patients, which has relatively lower positive predicted value in accurately identifying the true ALS patients. Furthermore, our analyses were limited to structured data elements, which restricts generalizability to unstructured sources such as clinical narratives. In addition, the significant number of missing EHR insurance information in several sites limited our ability to fully determine the causes of data inconsistencies. Finally, the observed discordance may reflect differences in data collection practices, delayed Medicare enrollment, or a change in Medicare enrollment status over time rather than true disagreement alone.

Future direction

Future work will apply the validated algorithm for ALS ascertainment to improve diagnostic specificity and enable more accurate tracking of the patient journeys across the GPC network. We plan to quantify the ALS diagnostic delay from symptom onset to confirmed diagnosis. Additionally, assessing whether ALS patients were using private, government, or both payers will provide a better understanding of insurance coverage for rare fatal diseases. We will test concordance for encounter type (inpatient, outpatient, emergency), provider specialty, and facility type to provide a full picture of the concordance between EHR and Medicare data.

Conclusion

This multi-site study demonstrates that linking EHR and Medicare data provides a framework for capturing the longitudinal care of patients with rare diseases such as ALS. However, our findings reveal meaningful inconsistencies in demographic, diagnostic, medication, and procedural data across sites, driven largely by variation in insurance coverage, age distribution, and documentation practices. Concordance was substantial to moderate for ALS diagnosis and moderate to fair for Riluzole prescriptions and PEG procedures, underscoring both the promise and the limitations of current linkage processes. Improving demographic completeness, particularly for race and ethnicity, and standardizing coding and data submission practices are important next steps to enhance data reliability across distributed research networks such as GPC. Establishing automated quality-assurance tools and incorporating validated disease-ascertainment algorithms will further strengthen the use of linked real-world data for clinical research and health system evaluation. These findings highlight the importance of ongoing data quality assessment as an integral part of real-world evidence generation in biomedical informatics.

Supplementary material

The supplementary material for this article can be found at https://doi.org/10.1017/cts.2026.10834.

Acknowledgements

This work includes data that were originally funded through a Patient-Centered Outcomes Research Institute® (PCORI®) program award (#RI-MISSOURI-01-PS8).

Author contributions

Yahia Mohamed: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Visualization, Writing–original draft, Writing–review & editing; Xing Song: Funding acquisition, Methodology, Resources, Software, Supervision, Validation, Writing–review & editing; Tamara McMahon: Conceptualization, Methodology, Project administration, Validation, Writing–review & editing; Suman Sahil: Data curation, Formal analysis, Methodology, Writing–review & editing; Lemuel R. Waitman: Conceptualization, Funding acquisition, Investigation, Methodology, Resources, Software, Supervision, Writing–review & editing; Sudeepa Bhattacharyya: Conceptualization, Formal analysis, Investigation, Methodology, Supervision, Validation, Writing–review & editing.

Funding statement

The dataset(s) used for the analyses described were obtained from the Greater Plains Collaborative (GPC), which is supported by the Patient Centered Outcomes Research Institute (RI-MISSOURI-01-PS1) and institutional funding from its member organizations. Dr Song was supported through the Center of Disease and Control (CDC) award No. R01TS000336.

Competing interests

The authors have no conflict of interest.

References

Foster, LA, Salajegheh, MK. Motor neuron disease: pathophysiology, diagnosis, and management. Am J Med. 2019;132:32–37. doi: 10.1016/j.amjmed.2018.07.012.CrossRefGoogle ScholarPubMed
Richards, D, Morren, JA, Pioro, EP. Time to diagnosis and factors affecting diagnostic delay in amyotrophic lateral sclerosis. J Neurol Sci. 2020;417:117054. doi: 10.1016/j.jns.2020.117054.CrossRefGoogle ScholarPubMed
Ludolph, AC, Grandjean, H, Reviers, E, et al. The preferences of people with amyotrophic lateral sclerosis on riluzole treatment in Europe. Sci Rep-UK. 2023;13:22497. doi: 10.1038/s41598-023-49424-3.CrossRefGoogle ScholarPubMed
Gupta, R, Bhandari, M, Grover, A, et al. Predictive modeling of ALS progression: an XGBoost approach using clinical features. Biodata Min. 2024;17:54. doi: 10.1186/s13040-024-00399-5.CrossRefGoogle ScholarPubMed
Rowan, CG, Flory, J, Gerhard, T, et al. Agreement and validity of electronic health record prescribing data relative to pharmacy claims data: a validation study from a US electronic health record database. Pharmacoepidem Dr S. 2017;26:963–972. doi: 10.1002/pds.4234.CrossRefGoogle Scholar
Heintzman, J, Bailey, SR, Hoopes, MJ, et al. Agreement of Medicaid claims and electronic health records for assessing preventive care quality among adults. JAMIA. 2014;21:720–724. doi: 10.1136/amiajnl-2013-002333.Google ScholarPubMed
Weiskopf, NG, Weng, C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. J Am Med Inform Assn. 2013;20:144–151. doi: 10.1136/amiajnl-2011-000681.CrossRefGoogle ScholarPubMed
Kahn, MG, Raebel, MA, Glanz, JM, Riedlinger, K, Steiner, JF. A pragmatic framework for single-site and multisite data quality assessment in electronic health record-based clinical research. Med Care. 2012;50:10.1097/MLR.0b013e318257dd67. doi: 10.1097/MLR.0b013e318257dd67.CrossRefGoogle ScholarPubMed
Kahn, MG, Callahan, TJ, Barnard, J, et al. A harmonized data quality assessment terminology and framework for the secondary use of electronic health record data. eGEMs. 2016;4:1244. doi: 10.13063/2327-9214.1244.CrossRefGoogle ScholarPubMed
Leonard, CE, Brensinger, CM, Nam, YH, et al. The quality of Medicaid and Medicare data obtained from CMS and its contractors: implications for pharmacoepidemiology. BMC Health Services Research. 2017;17:304. doi: 10.1186/s12913-017-2247-7.CrossRefGoogle ScholarPubMed
Strengths and Limitations of CMS Administrative Data in Research | ResDAC. (https://resdac.org/articles/strengths-and-limitations-cms-administrative-data-research). Accessed March 29, 2026.Google Scholar
Weiskopf, NG, Hripcsak, G, Swaminathan, S, Weng, C. Defining and measuring completeness of electronic health records for secondary use. J Biomed Inform. 2013;46:830–836. doi: 10.1016/j.jbi.2013.06.010.CrossRefGoogle ScholarPubMed
Forrest, CB, McTigue, KM, Hernandez, AF, et al. PCORnet® 2020: current state, accomplishments, and future directions. J Clin Epidemiol. 2021;129:60–67. doi: 10.1016/j.jclinepi.2020.09.036.CrossRefGoogle ScholarPubMed
The National Patient-Centered Clinical Research Network. The national patient-centered clinical research network, (https://pcornet.org/) Accessed July 6, 2023.Google Scholar
Greater Plains Collaborative (GPC). (https://gpcnetwork.org/) Accessed January 17, 2023.Google Scholar
Waitman, LR, Aaronson, LS, Nadkarni, PM, Connolly, DW, Campbell, JR. The greater plains collaborative: a PCORnet clinical research data network. JAMIA. 2014;21:637–641. doi: 10.1136/amiajnl-2014-002756.Google ScholarPubMed
Waitman, LR, Song, X, Walpitage, DL, et al. Enhancing PCORnet Clinical Research Network data completeness by integrating multistate insurance claims with electronic health records in a cloud environment aligned with CMS security and privacy requirements. JAMIA. 2022;29;660–670. doi: 10.1093/jamia/ocab269.Google Scholar
Qualls, LG, Phillips, TA, Hammill, BG, et al. Evaluating foundational data quality in the National Patient-Centered Clinical Research Network (PCORnet®). eGEMs. 2018;6:3. doi: 10.5334/egems.199.CrossRefGoogle ScholarPubMed
McDowell, BD, O’Rorke, MA, Schroeder, MC, et al. Implementing cancer registry data with the PCORnet common data model: the greater plains collaborative experience. JCO Clinical Cancer Informatics. 2024;8:e2400196. doi: 10.1200/CCI-24-00196.CrossRefGoogle Scholar
Mohamed, Y, Song, X, McMahon, TM, et al. Electronic health record data quality variability across a multistate clinical research network. J Clin Transl Sci. 2023;7:e130. doi: 10.1017/cts.2023.548.CrossRefGoogle ScholarPubMed
The Snowflake AI Data Cloud - Mobilize Data, Apps, and AI. (https://www.snowflake.com/content/snowflake-site/global/en) Accessed April 25, 2025.Google Scholar
PCORnet Common Data Model (CDM) Specification, Version 7.0.Google Scholar
Medicare Eligibility and Enrollment Files, RIF Versions | ResDAC. (https://resdac.org/articles/medicare-eligibility-and-enrollment-files-rif-versions) Accessed March 30, 2026.Google Scholar
Jarrín, OF, Nyandege, AN, Grafova, IB, Dong, X, Lin, H. Validity of race and ethnicity codes in Medicare administrative data compared to gold-standard self-reported race collected during routine home health care visits. Med Care. 2020;58:e1–e8. doi: 10.1097/MLR.0000000000001216.CrossRefGoogle ScholarPubMed
Thakore, NJ, Lapin, BR, Mitsumoto, H. Pooled resource open-access ALS clinical trials consortium. Early initiation of riluzole may improve absolute survival in amyotrophic lateral sclerosis. Muscle Nerve. 2022;66:702–708. doi: 10.1002/mus.27724.CrossRefGoogle ScholarPubMed
Andrews, JA, Jackson, CE, Heiman-Patterson, TD, Bettica, P, Brooks, BR, Pioro, EP. Real-world evidence of riluzole effectiveness in treating amyotrophic lateral sclerosis. Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration, Taylor & Francis. 2020;21:509–518. doi: 10.1080/21678421.2020.1771734.Google ScholarPubMed
Thomas, K, Schrager, J, Neel, R, Brook, L. Percutaneous endoscopic gastrostomy tube placement in amyotrophic lateral sclerosis: a case series with a multidisciplinary, team-based approach. Annals of Gastroenterology. 2020;33:480–484. doi: 10.20524/aog.2020.0517.Google ScholarPubMed
Son, B, Lee, J, Ryu, S, Park, Y, Kim, SH. Timing and impact of percutaneous endoscopic gastrostomy insertion in patients with amyotrophic lateral sclerosis: a comprehensive analysis. Sci Rep. 2024;14:7103. doi: 10.1038/s41598-024-56752-5.CrossRefGoogle ScholarPubMed
McHugh, ML. Interrater reliability: the kappa statistic. Biochem Medica. 2012;22:276–282.10.11613/BM.2012.031CrossRefGoogle ScholarPubMed
Magaña López, M, Bevans, M, Wehrlen, L, Yang, L, Wallen, GR. Discrepancies in race and ethnicity documentation: a potential barrier in identifying racial and ethnic disparities. J Racial Ethn Health Disparities. 2017;4:812–818. doi: 10.1007/s40615-016-0283-3.CrossRefGoogle Scholar
Jamal-Allial, A, Sponholtz, T, Vojjala, SK, et al. Validation of mortality data sources compared to the National Death Index in the healthcare integrated research database. Pragmatic and Observational Research. 2025;16:19–25. doi: 10.2147/POR.S498221.CrossRefGoogle Scholar
5 things you need to know about signing up for Medicare | CMS. (https://www.cms.gov/training-education/find-provider-type/employers-unions/top-five-medicare-enrollment) Accessed April 20, 2025.Google Scholar
ALS Focus Results: Insurance and Payment for ALS Care | The ALS Association. (https://www.als.org/research/als-focus/survey-results/survey-6-results) Accessed April 2, 2026.Google Scholar
Nelson, LM, Topol, B, Kaye, W, et al. Evaluation of the completeness of ALS case ascertainment in the US National ALS Registry: application of the capture-recapture method. Neuroepidemiology. 2022;56:104–114. doi: 10.1159/000521591.CrossRefGoogle ScholarPubMed
Mehta, P, Raymond, J, Han, M, et al. A revision to the United States national ALS registry’s algorithm to improve Case-Ascertainment. Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration. Amyotroph Lateral Scler Front Degener. 2023;24:230–236. doi: 10.1080/21678421.2022.2121168.CrossRefGoogle Scholar
FitzHenry, F, Peterson, JF, Arrieta, M, Waitman, LR, Schildcrout, JS, Miller, RA. Medication administration discrepancies persist despite electronic ordering. Journal of the American Medical Informatics Association: JAMIA. 2007;14:756–764. doi: 10.1197/jamia.M2359.CrossRefGoogle ScholarPubMed
Lin, P-ID, Daley, MF, Boone-Heinonen, J, et al. Comparing prescribing and dispensing data of the PCORnet common data model within PCORnet antibiotics and childhood growth study. eGEMs. 2019;7:11. doi: 10.5334/egems.274.CrossRefGoogle ScholarPubMed
Sharma, M, Johnson, ML, Zhao, H, Giordano, SH, Holmes, HM. Concordance between electronic health record data and medicare part D claims data for oral anticancer drug use. JAMA Netw Open. 2020;3:e203821. doi: 10.1001/jamanetworkopen.2020.3821.CrossRefGoogle ScholarPubMed
Katzberg, HD, Benatar, M. Enteral tube feeding for amyotrophic lateral sclerosis/motor neuron disease. Cochrane Database Syst Rev. 2011;2011:CD004030. doi: 10.1002/14651858.CD004030.pub3.Google ScholarPubMed
Figure 0

Figure 1.Figure 1 long description.Flowchart for unique ALS diagnosis across GPC sites.

Figure 1

Figure 2.Proportion of ALS patients with matched EHR-medicare data across GPC sites.

Figure 2

Figure 3.Type of insurance used for the first encounter in EHR data.

Figure 3

Figure 4.Figure 4 long description.Kappa statistics and 95% confidence intervals.

Figure 4

Figure 5.Forest plot for logistic regression output of ALS diagnosis data concordance.

Figure 5

Figure 6.Figure 6 long description.Forest plot for logistic regression output of riluzole prescription data concordance.

Figure 6

Figure 7.Forest plot for logistic regression output of PEG procedure data concordance.

Supplementary material: File

Mohamed et al. supplementary material

Mohamed et al. supplementary material
Download Mohamed et al. supplementary material(File)
File 291.4 KB