Introduction
Outbreaks and pandemics have frequently occurred throughout history with the most recent coronavirus disease 2019 (COVID-19) pandemic, from December 2019 to May 2023 [1], accounting for more than 769 million confirmed cases worldwide, almost 7 million deaths as of 8 August 2023 [2], and major negative economic and social impact [Reference Pereznieto and Oehler3, Reference Nicola4].
Adequate pandemic preparedness is essential to face the many challenges a pandemic poses to the healthcare system and the entire society. Experiences from the COVID-19 pandemic and other outbreaks [5–Reference Hadjisotiriou7] are key in achieving worldwide protection and strengthening pandemic preparedness, as laid down in the Pandemic Agreement adopted by the World Health Assembly in 2025 [8].
A key component of pandemic preparedness is a well-functioning early warning system (EWS) for infectious diseases. An EWS should be able to alert policymakers, epidemiologists, and physicians at an early stage of the rise of an infectious disease, allowing earlier action, prevention of further spread in the population, and subsequently lower morbidity and mortality [Reference de Boer9].
Internationally, multiple EWSs are used with varying effectiveness [Reference Meckawy10]. EWSs based on pre-diagnosis data were found to be more timely in outbreak detection [Reference Meckawy10]. General practice data contain such pre-diagnosis routine data as they reflect the patient’s first point of contact with symptoms in the healthcare system. Using general practice data to yield early warnings has thus advantages, such as the real-time potential, the availability of contextual information about the patient and of the general practice catchment area, and the minimal additional work burden if these data are being recorded structurally and systematically [Reference Meckawy10–Reference Randrianasolo14]. Currently, in the Netherlands, infectious disease surveillance for early warning is mainly based on routine notifications of a limited number of notifiable diseases from general practitioners (GPs), hospital specialists, and laboratories to public health services (PHS). Furthermore, sentinel surveillance based on general practice data of the Dutch Sentinel practices of Nivel (Netherlands Institute for Health Services Research Primary Care Database) is conducted weekly by monitoring single diagnostic codes and combined clinical codes of 400 practices [15, Reference Baliatsas16].
In the Netherlands, the Family Medicine Network (FaMe-Net) is a practice-based research network that systematically collects structured general practice morbidity data. It contains extensive, uniform, and detailed information from routine registrations, including information on the first spontaneous statement of a patient about the reason for visiting the GP, as well as the GP’s preliminary and final diagnoses. This statement by the patient and the preliminary diagnosis may be registered earlier by the GP than the final diagnosis. We explored structured general practice data, using FaMe-Net, to find out whether this type of data can be used within a surveillance system and consequently if alert signals for increases in infectious diseases could be generated.
Our overall aim was to perform a proof-of-concept study to assess the potential of structured routine general practice registrations for an EWS of selected infectious disease syndromes. Specifically, we evaluated the routine general practice registrations descriptively with criteria used for surveillance systems. In addition, we investigated the frequency, seasonality, and variability of alert signals across several infectious disease syndromes: respiratory infection, gastroenteritis, rash, infectious conjunctivitis, and fever.
Methods
Study setting
In the Netherlands, each inhabitant needs to be registered at a general practice. The GP has a gatekeeping role, which means patients have to be referred to medical specialists and hospitals by their GP [Reference Faber, Burgers and Westert17]. There are 14102 GPs working in 4837 practices [18]. The average number of enrolled patients per GP who works full time is 2095 [19].
The general practice data registry used in this study, FaMe-Net, registers morbidity of all patients who visit the GP in electronic health records. It is the world’s oldest practice-based research network that is still functioning [20]. As of 2022, it consisted of 6 general practices and 35 GPs in the Nijmegen/Amsterdam region in the Netherlands, including >40000 patients. There is uniformity in registration and coding. Data collection occurs in a continuous and longitudinal manner, with data extraction once a year [20]. As this is a study with FaMe-Net data, it is exempt from ethical review by the Medical Ethics Review Committee in Nijmegen in the Netherlands (CMO number: 2020–6871). Patients are informed that their medical data are used in FaMe-Net and have the possibility to opt out.
Descriptive evaluation with surveillance attributes
FaMe-Net was evaluated descriptively based on criteria developed by the European Centre for Disease Prevention and Control (ECDC) that are used for surveillance systems [21]: internal completeness, internal validity, timeliness, representativeness, simplicity, flexibility, acceptability, stability, and reliability. Most criteria were modified for the purpose of this study mainly because FaMe-Net is currently not used as a surveillance system. For internal validity, we assessed the structures put in place to ensure uniform registration and improve data quality. We were not able to assess other reported surveillance attributes, such as external completeness, external validity, sensitivity, specificity, positive predictive value, negative predictive value, reliability, adequacy, and case ascertainment with the currently available information. This evaluation consisted of input provided by a board member and the coordinator of FaMe-Net.
Data for alert signals
For the generation of alert signals, we included data from electronic health records of patients registered at four out of the six current FaMe-Net general practices, for which complete data were available over the period 2014 until 2024. The number of patient years per year ranged from 23209 to 30708.
Types of data
Three types of registered health data from FaMe-Net were included: reason for encounter (RFE), encounter diagnosis (Ed), and the final diagnosis within the episode of care (EoC). The RFE is the first spontaneous statement of a patient about the reason for visiting the GP. This can be a symptom or a disease described by the patient (subjective), such as a cough or the flu. If a patient presents with multiple RFEs, all are registered by the GP (i.e. ‘Doctor I have a cough, a fever and I think it is a flu’ means RFE = cough, fever, and flu) (Figure 1). Ed is the diagnosis registered by the GP during an encounter (objective). An EoC comprises the initial encounter of a patient with the GP regarding one specific health problem until the last encounter about this health problem. Thus, the EoC contains the final diagnosis of the GP (objective), such as upper respiratory infection acute. The Ed or final diagnosis in the EoC recorded by the GP can either be a symptom or a disease (Figure 1).
Example of a patient visiting the general practitioner (GP) and what the data types reason for encounter (RFE), encounter diagnosis (Ed), and episode of care (EoC) represent. R03 (wheezing), R05 (cough), R07 (sneezing/nasal congestion), R21 (throat symptom/complaint), and R74 (upper respiratory infection acute) are diagnostic International Classification of Primary Care – 2nd (ICPC-2) codes.
Figure 1. Long description
The diagram is organized along a horizontal timeline at the top labeled Episode of Care.
1. First Encounter: A patient is shown alone. A blue speech bubble reads ‘I am wheezing.’ Below this, the RFE is coded as R0 3. An arrow points upward to a red box for Encounter diagnosis: R0 3: Wheezing. A dashed line connects this to a yellow circle on the main timeline labeled preliminary diagnosis: R0 3.
2. Second Encounter: The patient is seated with a G P. A blue speech bubble reads ‘I am also coughing a lot.’ The RFE is coded as R0 5. An arrow points to a red box for Encounter diagnosis: R0 5: Cough. A dashed line connects this to a second yellow circle on the timeline labeled preliminary diagnosis: R0 5.
3. Third Encounter: The patient is seated with a GP. A blue speech bubble reads ‘Now I am also sneezing and my throat hurts.’ The RFE is coded as R0 7 and R2 1. An arrow points to a red box for Encounter diagnosis: R7 4: Upper respiratory infection acute. A dashed line connects this to a final yellow circle at the end of the timeline labeled final diagnosis: R7 4.
Black horizontal arrows between the encounter blocks indicate the progression of time from the first to the third visit.
Encounter types consist of office consultations, home visits, telephone consultations, e-consultations, repeated prescriptions, and out-of-hours encounters. We excluded administrative or letter encounters because within these encounter types the patient does not have contact with the GP.
The registered RFEs (reason of patient for visiting the GP), Eds (encounter diagnosis), and EoCs (final diagnosis), consisting of symptoms or diseases, were coded using the International Classification of Primary Care – 2nd (ICPC-2) [22]. We included all RFEs, Eds, and EoCs that contained an ICPC-2 code related to a respiratory infection, gastroenteritis, rash, infectious conjunctivitis, or fever (Supplementary Tables 4–8).
Infectious disease syndromes
We focused on several infectious disease syndromes with outbreak potential: respiratory infection, gastroenteritis, rash, and infectious conjunctivitis. Fever was analysed separately as a stand-alone syndrome definition, given its frequent association with infections. Existing, modified through expert opinion, and newly developed, based on the FaMe-Net dashboard [20], infectious disease syndrome definitions were used [Reference Groeneveld12, 20, Reference Van den Wijngaard23, 24]. We made a distinction between narrow and broad infectious disease syndrome definitions [20]. The narrow infectious disease syndrome definition consisted of a limited list of disease diagnoses, and the broad infectious disease syndrome definition consisted of an extensive list of both symptoms and disease diagnoses. These infectious disease syndrome definitions focused on diseases, a combination of symptoms (e.g. cough) and diseases (e.g. bronchitis) and fever separately, in order to assess whether these variations in infectious disease syndrome definitions had an effect on the number and timing of generated alert signals. In total, we had nine overarching definitions for infectious disease syndromes: two infectious disease syndrome definitions for respiratory infection, two for gastroenteritis, two for rash, two for infectious conjunctivitis, and one for fever as a distinct category (Supplementary Tables 4–8). The limited list of diseases was assessed to investigate whether these diseases provided enough information to generate alert signals. The extensive list of symptoms and diagnoses was selected to improve the sensitivity of signal detection (Supplementary Tables 4–8). The nine infectious disease syndrome definitions were applied to RFE, eight were applied to Ed, and seven were applied to EoC data, which led to a total of 24 sets of combinations used in this study (Supplementary Tables 4–8).
Statistical analysis
Statistical analysis was performed in RStudio version 2024.04.2 [25] separately for respiratory infection, gastroenteritis, rash, infectious conjunctivitis, and fever syndromes. First, we calculated the weekly number of counts for each infectious disease syndrome within each data type (i.e. RFE, Ed, and EoC data), correcting for patient years (Figure 2). These numbers of counts reflect the total number of registered diagnostic codes rather than the number of unique individual patients, as a single patient may have multiple diagnostic codes registered per patient. Consequently, individual patients could be counted more than once. This increases sensitivity, mostly for RFE and Ed. It means that the more severe patients, who present with multiple symptoms or visit the GP more often, will contribute more counts. Fluctuations in the number of counts, the presence of peaks and troughs, and seasonality were visually assessed (Figure 2). The respiratory season started at week 40 of a given calendar year up to and including week 20 of the following year [26]. For gastroenteritis, rash, infectious conjunctivitis, and fever, no seasons were specified, as seasonality was not expected for these syndromes. We constructed an alert threshold with the Farrington flexible method [Reference Noufaily27], with baseline counts (b) of the previous 5 years and a window half-width (w) of 3 weeks, which were the number of weeks before and after a given week for each year. In Supplementary Material 1.3, details of the model and parameters can be found. The baseline period was 2014–2018, and an alert threshold was constructed for the period 2019–2024 (Figure 2). The Farrington flexible method accounted for trend, seasonality, and down-weighted past outbreaks. The outcome was an alert signal, defined as a week in which the observed count exceeded the alert threshold. For the evaluation of this outcome, the period from week 40 of 2020 up to and including week 39 of 2023 during the COVID-19 pandemic was excluded as trends were distorted due to the COVID-19 pandemic (excluded period). The remaining weeks from 2019 to 2024 were included (included period: a total of 154 weeks). We assessed the total number of alert signals, the number of signals within and outside a season (if applicable), and large alert signals (exceeding the threshold by 10%) to differentiate between a slight and more pronounced threshold exceedance. Moreover, we determined the first alert signals of the season, because this could indicate an increase in the period when respiratory infections occur more often. Finally, we differentiated between single and consecutive alerts. A single alert signal was defined as an alert signal that was not preceded or followed by another alert signal. We counted alert signals of consecutive weeks, without interruption, as one consecutive alert signal, as this could indicate an outbreak that lasts longer than 1 week.
Weekly counts for a) narrow respiratory infection syndrome and b) broad respiratory infection syndrome in RFE data for the period 2014–2024 (black bars), including the alert threshold (pink line) and alert signals, visualized with red triangles; W stands for week.
Figure 2. Long description
The figure consists of two vertically stacked bar graphs, labeled a and b. Both share an x-axis representing Year Week from 2015 W 01 to 2024 W 01 and a y-axis for Counts. Light blue vertical bands indicate the Respiratory Season from W 40 to W 20 each year.
Panel a, narrow respiratory infection syndrome. The y-axis ranges from 0 to 40. Grey bars show observed counts, which exhibit seasonal peaks within the blue bands. Starting in 2019, a dashed pink line represents the alert threshold, which fluctuates between 15 and 50 counts. Red triangles along the x-axis indicate alert signals where observed counts exceed the threshold, occurring most frequently in late 2022 and early 2023.
Panel b, broad respiratory infection syndrome. The y-axis ranges from 0 to 600. The grey bars follow a similar seasonal pattern to panel a but at a higher magnitude, with peaks often reaching between 200 and 400. A significant spike occurs in early 2020, reaching nearly 600. The dashed pink alert threshold begins in 2019, ranging from approximately 150 to 380. Red triangles marking alert signals are clustered in early 2020, late 2021, and throughout 2022 into 2023.
The EWS potential was defined based on the generation of alert signals. It had potential if alert signals for the defined infectious disease syndromes were generated within RFE, Ed, and EoC data. We also compared the infectious disease syndrome definitions within the data types and assessed whether there were similarities or variations in the number of alert signals. Validation of alert signals for the detection of infectious disease outbreaks was beyond the scope of this study.
Results
Evaluated surveillance attributes
The surveillance attributes have been evaluated based on the operations of FaMe-Net up to and including 2024. Internal completeness was high for RFE (<0.01% missing), Ed (~5.5% missing), and EoC (<1% missing). Furthermore, for uniform registration and data quality, FaMe-Net has agreements and checks in place for the general practices (Table 1). As for timeliness, data extraction is currently performed twice a year, and it is available for research purposes within approximately 1 month (Table 1). We found it was representative in regard to sex but not completely for age, as FaMe-Net has relatively more young patients (0–50 years) and less elderly (50 years and older), compared to the Dutch population [28] (Supplementary Tables 1 and 2). FaMe-Net covered the Nijmegen/Amsterdam region [20]. A team managed and cleaned the data, of which data extraction was largely automatized. It was possible to add new data fields, which occurred more rapidly during the COVID-19 pandemic. FaMe-Net has been operational since 2013, complying with legal requirements, and general practices have been highly willing to continue to be part of it (Table 1).
Evaluation of FaMe-Net with surveillance attributes a
Table 1. Long description
The table consists of three columns: Surveillance attributes, Modified definition based on ECDC criteria, and FaMe-Net.
* Internal completeness: Defined as the percentage of missing ICPC-2 codes. FaMe-Net results show RFE is less than 0.01 percent missing, Ed is approximately 5.5 percent missing, and EoC is less than 1 percent missing.
* Structure for internal validity: Focuses on uniform registration. FaMe-Net utilizes agreements for ICPC coding of RFE, Ed and Eo C, with clinical coding instructions sent every 2 months and feedback provided twice a year.
* Timeliness: Speed of extraction and availability. FaMe-Net data are extracted twice a year and available for research within 1 month.
* Representativeness: Accuracy regarding Dutch population age, sex, and geography. FaMe-Net is representative for sex but has more young patients 0 to 50 years and fewer elderly 50 plus years than the general population. It covers 6 practices with over 40,000 patients and 35 GPs, primarily in Nijmegen and Amstelveen.
* Simplicity: Ease of operation. FaMe-Net extraction is largely automated and managed by a dedicated team.
* Flexibility: Ability to add new fields. New fields can be added in 2 to 3 months; COVID-19 codes were rapidly implemented and checked retrospectively.
* Acceptability: Willingness to participate. High willingness among practices due to acknowledged data relevance.
* Stability and reliability: Operational ability without failure. FaMe-Net has been operational since 2013 following a merger and complies with all legal requirements.
a FaMe-Net: Family Medicine Network; ICPC-2: International Classification of Primary Care – 2nd [22]; RFE: reason for encounter; Ed: encounter diagnosis; EoC: episode of care; GP: general practitioner.
b Based on all general practices and all patients that visited the GP within the period 2014–2022.
c Representativeness was calculated in regard to age and sex, based on 1 January 2024, for five FaMe-Net general practices (Supplementary Material: Tables 1 and 2).
Alert signals of infectious syndromes
The study population consisted of 35468 patients who had at least one diagnostic code related to a respiratory infection, gastroenteritis, rash, infectious conjunctivitis, or fever diagnostic code in the period 2014 to 2024. This corresponded to 197045 RFEs, 274134 Eds, and 163529 EoCs.
Weekly counts for respiratory syndromes, gastroenteritis, rash, infectious conjunctivitis, and fever definitions were calculated in RFE, Ed, and EoC data. Weekly counts for respiratory syndromes based on RFE are shown in Figure 2, where the red triangles represent the alert signals, corresponding to the counts that exceeded the threshold, with the respiratory season (week 40–week 20) visualized in blue (Figure 2). The respiratory heatmap (Figure 3a) showed the alert signals per week, in red, for each respiratory syndrome definition and data type (RFE, Ed, EoC). If an alert signal was generated, this was shown as a red bar in that specific week and the blue background corresponded to the respiratory season (week 40–week 20) (Figure 3a). For visualization of the alert signals of the gastroenteritis, rash, infectious conjunctivitis, and fever syndrome definitions, a heatmap was also used, shown in Figure 3b–e.
Weekly alert signals of (a) respiratory infection, (b) gastroenteritis, (c) rash, (d) infectious conjunctivitis, and (e) fever syndrome definition within data types for the period 2019–2024; the x-axis represents the years and weeks and the y-axis the definitions within data types, where the red bar shows the presence of an alert signal in a week and light blue represents the season. The included period was week 1 2019 to week 39 2020 and week 40 2023 to week 52 2024 (green), the excluded period was week 40 2020 to week 39 2023 (red). For each syndrome definition within a data type, alert signals were generated for each week. A single alert signal was defined as an alert signal that was not preceded or followed by another alert signal. Alert signals of consecutive weeks, without interruption, were defined as one consecutive alert signal. W: week; RFE: reason for encounter; Ed: encounter diagnosis; EoC: episode of care.
Figure 3. Long description
A multi-panel figure containing five heatmaps labeled a through e. Each heatmap shares a common x-axis representing time from 2019 to 2024, divided by Year and Week (W). A horizontal bar at the top of each plot indicates the Included period in green and the Excluded period (late 2020 to late 2023) in red. The y-axis lists syndrome definitions across data types: RFE (Reason for Encounter), Ed (Encounter Diagnosis), and EoC (Episode of Care).
* Panel a: Respiratory alert signals. Shows light blue vertical bands representing the respiratory season (W 40 to W 20). Red vertical bars indicate alert signals, which are concentrated in late 2021 and throughout 2022 across all data types.
* Panel b: Gastroenteritis alert signals. Red alert bars are sparse in 2019 and 2020, with a higher density of alerts appearing in early 2022 and early 2024.
* Panel c: Rash alert signals. Displays a high density of red alert signals starting in late 2021 and continuing through 2023, particularly in the Ed and RFE data types.
* Panel d: Infectious conjunctivitis alert signals. Shows scattered alerts in 2019, with a significant cluster of red bars appearing in late 2022 and throughout 2023.
* Panel e: Fever alert signals. Features distinct blocks of red alerts in early 2020, mid-2021, and throughout 2022, with fewer signals in 2023 and 2024.
In all panels, grey areas represent weeks with no alert signal.
All infectious syndrome definitions in RFE, Ed, and EoC data generated alert signals. In 154 weeks, the total number of alert signals ranged from 3 to 20, depending on the syndrome definition used for respiratory infection, gastroenteritis, rash, infectious conjunctivitis, fever, and the data type (RFE, Ed, EoC) (Tables 2 and 3).
Alert signals for the respiratory infection syndrome definitions a
Table 2. Long description
The table categorizes alert signals by Data type and Respiratory infection syndrome definition.
* RFE (Reason for Encounter):
- Narrow definition: 9 total alerts, 6 within season, 3 outside season. Signal types include 6 large, 7 single, and 1 consecutive.
- Broad definition: 20 total alerts, 19 within season, 1 outside season. Signal types include 7 large, 5 single, and 4 consecutive.
* Ed (Encounter Diagnosis):
- Narrow definition: 12 total alerts, all 12 within season, 0 outside. Signal types include 9 large, 1 single, and 4 consecutive.
- Broad definition: 15 total alerts, all 15 within season, 0 outside. Signal types include 5 large, 3 single, and 3 consecutive.
* EoC (Episode of Care):
- Narrow definition: 11 total alerts, 10 within season, 1 outside season. Signal types include 4 large, 2 single, and 3 consecutive.
- Broad definition: 7 total alerts, all 7 within season, 0 outside. Signal types include 2 large, 1 single, and 2 consecutive.
* Respiratory total (percentage):
- Total: 74 (100 percent).
- Within season: 69 (93 percent).
- Outside season: 5 (7 percent).
- Large signals: 33 (45 percent).
- Single signals: 19 (26 percent).
- Consecutive signals: 17 (23 percent).
a RFE: reason for encounter; Ed: encounter diagnosis; EoC: episode of care.
b The respiratory season consisted of 97 weeks.
c Outside the respiratory season, there were 57 weeks.
Alert signals for gastroenteritis, rash, infectious conjunctivitis, and fever syndrome definitions a
Table 3. Long description
The table is divided into four main sections based on syndrome definitions. Each section includes columns for Data Type, Syndrome Definition, and Alert Signals (Total, Large, Single, and Consecutive).
1. Gastroenteritis Syndrome:
- RFE Narrow: Total 5, Large 4, Single 3, Consecutive 1.
- RFE Broad: Total 11, Large 5, Single 6, Consecutive 2.
- Ed Narrow: Total 3, Large 2, Single 3, Consecutive 0.
- Ed Broad: Total 8, Large 1, Single 6, Consecutive 1.
- EoC Narrow: Total 3, Large 1, Single 1, Consecutive 1.
- EoC Broad: Total 4, Large 1, Single 2, Consecutive 1.
- Gastroenteritis Total: 34 (100 percent), Large 14 (42 percent), Single 21 (62 percent), Consecutive 6 (18 percent).
2. Rash Syndrome:
- RFE Narrow: Total 8, Large 5, Single 4, Consecutive 2.
- RFE Broad: Total 6, Large 2, Single 6, Consecutive 0.
- Ed Narrow: Total 11, Large 3, Single 5, Consecutive 2.
- Ed Broad: Total 10, Large 3, Single 7, Consecutive 1.
- EoC Narrow: Total 3, Large 0, Single 3, Consecutive 0.
- Rash Total: 38 (100 percent), Large 13 (34 percent), Single 25 (66 percent), Consecutive 5 (13 percent).
3. Infectious Conjunctivitis Syndrome:
- RFE Narrow: Total 5, Large 5, Single 5, Consecutive 0.
- RFE Broad: Total 8, Large 5, Single 4, Consecutive 2.
- Ed Narrow: Total 5, Large 1, Single 5, Consecutive 0.
- EoC Narrow: Total 4, Large 2, Single 2, Consecutive 1.
- Infectious Conjunctivitis Total: 22 (100 percent), Large 13 (59 percent), Single 16 (73 percent), Consecutive 3 (14 percent).
4. Fever Syndrome:
- RFE Fever: Total 11, Large 5, Single 3, Consecutive 3.
- Ed Fever: Total 12, Large 9, Single 6, Consecutive 2.
- EoC Fever: Total 8, Large 6, Single 6, Consecutive 1.
- Fever Total: 31 (100 percent), Large 20 (65 percent), Single 15 (48 percent), Consecutive 6 (19 percent).
Initialisms used: R F E (Reason for Encounter), Ed (Encounter Diagnosis), E o C (Episode of Care).
a RFE: reason for encounter; Ed: encounter diagnosis; EoC: episode of care.
For each infectious disease syndrome (respiratory infection, gastroenteritis, rash, infectious conjunctivitis, fever), there were between 34 and 65% large alert signals (Tables 2 and 3). The total number of single (n = 19, 26%) and consecutive (n = 17, 23%) alert signals was similar for the respiratory infection syndrome (Table 2). This was different for the gastroenteritis, rash, infectious conjunctivitis, and fever groups, where there were more single alert signals (48–73%) than consecutive alert signals (13–19%) (Tables 2 and 3).
For the respiratory infection syndromes in RFE, Ed, and EoC data, the number of respiratory alert signals ranged from 6 to 19 in 97 weeks within the respiratory season and from 0 to 3 in 57 weeks outside the respiratory season (Table 2). Most respiratory alert signals were shown within the season (n = 69, 93%), compared to a low number of respiratory alert signals outside the season (n = 5, 7%) (Table 2).
The first alert signal in the respiratory season 2019–2020 was shown by the broad respiratory infection syndrome definition in RFE data, while in 2023–2024 this was shown by the narrow respiratory infection syndrome in RFE and Ed data (Figure 3a).
The weeks when alert signals were generated (per infectious disease or symptom group) varied between most infectious disease syndrome definitions within the data types (RFE, Ed, and EoC), as shown by the different patterns in the heatmaps. However, there were also weeks when multiple syndrome definitions within the data types (RFE, Ed, and EoC) generated alert signals simultaneously (Figure 3a–e).
There were infectious disease syndrome definitions in a specific data type that had an equal number of total, within- or outside-season, large, single, or consecutive alert signals, of which the timing of alert signal generation was sometimes the same. For example, for the narrow gastroenteritis syndrome in Ed and EoC data, the number of total alert signals was equal. Moreover, two out of three alert signals were generated at the same time, but the number of large, single, and consecutive alert signals differed (Table 3, Figure 3b). Similarities in the number of alert signals were also found between the narrow infectious conjunctivitis syndromes in RFE and Ed data (Table 3).
Discussion
In this proof-of-concept study, we evaluated a structured general practice data registry and found that it harboured the potential to be used for surveillance and as an EWS of infectious diseases. It performed adequately on most surveillance attributes: internal completeness, structure for internal validity, simplicity, flexibility, acceptability, and stability and reliability, but suboptimal on representativeness and poorly on timeliness, based on how FaMe-Net currently operates. Weekly counts of the nine infectious disease syndrome definitions related to respiratory infection, gastroenteritis, rash, infectious conjunctivitis, and fever within structured general practice data showed potential for an EWS because for all syndrome definitions in all data types alert signals were generated, but not for every week. Furthermore, while there were similarities, we mostly observed variation between infectious disease syndrome definitions in the data types in terms of the number of total, first of the season, within- and outside-season, large, single, and consecutive alert signals and their timing.
Alert signal generation is the first step in outbreak detection in an automated (data-driven) manner. Our findings demonstrate that routinely collected general practice data have weekly counts of infectious disease syndromes that exceed thresholds, accounting for trend, seasonality, and past outbreaks. For an EWS, balancing the sensitivity and specificity of alert signal generation is pivotal. For instance, an alert signal generation should not yield a constant or high number of alerts because the EWS would thus be too sensitive, requiring too much effort from public health professionals to verify each alert. At the same time, if no alert signals are generated, it would not be useful as an EWS, because no exceedance would be detected, including the ones that might lead to outbreaks. The suitable composition of the syndrome definitions in the data types (i.e., RFE, Ed, and EoC) that perform optimally in generating alerts requires careful calibration and validation in practice.
There were similarities and variations in the number and timing of alert signals between infectious disease syndrome definitions per infectious disease or symptom group. However, no syndrome definitions in RFE, Ed, and EoC data were completely the same in terms of all of the following: number of total, within- and outside-season, large, single, and consecutive alert signals and the weeks when alert signals were generated in the included period. For that reason, no infectious disease syndrome definition, in a specific data type, could be excluded prior to validation based on alert signals. Variation means that in the next step, the validation, there could be a difference in the detection of an outbreak, between the infectious disease syndrome definitions within the data types.
The data registry FaMe-Net was found to be an acceptable, stable, and reliable source, with high internal completeness of data and structures put in place to ensure uniform registration and high quality of data. It is somewhat flexible regarding the addition of new data fields, and for the most part, it is simple, in terms of structure and ease of operation. The data have a high level of granularity, but they are suboptimal in regard to representativeness compared to the Dutch population [28]. Thus, a general practice data registry can comply with most surveillance attributes required for a surveillance system and EWS. Some surveillance systems with general practice data have been previously evaluated with surveillance attributes. The Australian Sentinel Practice Research Network syndromic surveillance system was found to be flexible but also not timely enough and not representative of Australia [Reference Clothier, Fielding and Kelly29]. Furthermore, the GP house calls syndromic surveillance system in France, consisting of home medical visits requested by patients when the GP is not available, also performed well on many surveillance attributes such as simplicity, acceptability, flexibility, data quality, reliability, and timeliness because it is an automated and real-time system, but representativeness could not be evaluated [Reference Flamand11]. In the Netherlands, the Nivel syndromic surveillance system, based on a subset of approximately 400 general practices, is representative, but the inclusion of more general practices would improve representativeness [Reference Baliatsas16]. In regard to FaMe-Net, it will be in the near future technically and logistically possible to improve timeliness, in order for it to operate in near real time, with daily data extraction and fast data cleaning procedures. Thus, improving the timeliness of GP-based surveillance is feasible and must move towards near real-time data availability, as this attribute is essential for early signal detection to detect outbreaks timely.
In this study, alert signals for infectious disease syndromes were generated within structured general practice data, which shows there is potential for an EWS. This is in agreement with other studies that also found potential for the use of structured general practice data for syndromic public health surveillance. The feasibility of primary care consults of the Singapore Armed Forces for syndrome surveillance was assessed, and counts were generated above the threshold for infectious disease syndromes, and some of these peaks above the threshold were linked to specific clusters or outbreaks in that period, but not all peaks could be verified [Reference Ang30]. A case study of 2 years in the Netherlands with the Integrated Crisis Alert and Response System (ICARES), a real-time automated daily detection tool for early detection of clusters of infectious diseases, showed the potential of general practice and also hospital data, as alert signals were generated above a threshold for infectious syndromes. These were also validated by the local infectious disease control unit and research team through further investigation. They gathered raw data and geographical information and asked the treating physicians for information about the diagnosis and potential cluster, simultaneously being informed by ICARES about the potential cluster. They found that out of four alerts based on general practice data and one alert based on general practice and hospital data, there appeared to be one true cluster [Reference Groeneveld12]. Thus, these studies also showed the potential of structured general practice data because increases and alert signals of infectious diseases above the threshold could be identified and they linked the increases above the threshold to clusters or outbreaks. However, in the study of Ang [Reference Ang30], not all peaks could be linked to clusters or outbreaks and the ICARES tool showed false positives and they noted that this tool could also lead to false negatives. Nevertheless, it should also be considered that if the number of false positives is low, it would require little time to verify them by public health professionals. This shows that it is essential to validate alert signals as the next step. In the Netherlands, Nivel does weekly surveillance using structured general practice data and it notifies public health professionals when there are increasing trends [Reference Baliatsas16], and this also shows the potential of structured general practice data. The added value of our approach, compared to monitoring trends, is that it generated alert signals when (unusual) observed increases exceeded a threshold. These alert signals were based not only on Ed and EoC data but also on RFE data. These data consisted of high-quality coded symptoms and diagnoses.
A strength of this study is that we used data of the FaMe-Net data registry, which contains elaborate and very accurate epidemiological data with a uniform and standardized coding system [20]. We investigated alert signal generation from a broad perspective, as we used multiple syndrome infectious disease syndromes in different data types registered by the GP. The specific added value of these data is the presence of well-registered and standardized information on RFE and Ed. The syndrome definitions in RFE and Ed data were able to generate alert signals. In the next step of validation, it should be investigated whether RFE, which includes the earliest wording of a symptom or disease by the patient at presentation (without professional judgement by the GP) or Ed (early preliminary diagnosis by the GP) or both, has more potential for a real-time EWS than EoC data, containing information about the final diagnosis, which is registered later and is a result of further investigations by the GP. Formal external validation against independent surveillance data is a planned next step. In the validation study, we will investigate whether the alarm signals are associated with real regional outbreaks and the COVID-19 pandemic, making a comparison with mandatory notifications and possibly laboratory data as a reference.
Another strength is that we used the Farrington flexible method, which has been found to perform well for outbreak detection, including correction for trend, seasonality, and past outbreaks [Reference Noufaily27]. However, there are also limitations. First of all, we did not evaluate timeliness throughout all steps of a surveillance system (e.g. patient or doctor’s delay), as we focused on the speed of data extraction and data availability. Secondly, the location of the general practices used in this study was not randomly selected, limiting generalizability. The study was performed on the available general practices to investigate the potential. Thirdly, we had to exclude the alert signals during a large part of the COVID-19 pandemic period. Due to changes in policy and measures taken by the government, the behaviour of people in society changed during the COVID-19 pandemic. This meant there was less transmission of non-respiratory infectious diseases [31]. Lower counts distorted the Farrington flexible model and its output during and after the COVID-19 pandemic, as it takes trends into account for its alert threshold calculation. Nevertheless, we included the data after the COVID-19 pandemic. Besides, as validation was not part of this study, future studies should report whether the generated alert signals are valid in yielding signals for outbreaks against regional and national outbreak data. This needs to be investigated to understand the full EWS potential of these data.
We provided a method to improve the understanding of the potential of structured general practice data for the application in an EWS. We found that structured general practice data showed potential for surveillance and an EWS for many infectious disease syndromes, because it performed well on most surveillance attributes and weekly counts of the infectious disease syndromes led to the generation of alert signals, which must be subjected to validation. Leveraging the potential of structured general practice data for surveillance and early warning is key to preparedness for outbreaks and pandemic threats. Ultimately, it could be used as a complementary tool for infectious disease control, with interpretation in multidisciplinary settings (GPs, public health consultants, epidemiologists), within the existing public health surveillance landscape. This approach with structured general practice data is likely to enhance pandemic preparedness due to earlier detection of potential outbreaks and threats.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/S0950268826101800.
Data availability statement
Data are available upon request.
Acknowledgements
The authors acknowledge the research infrastructure provided by the Dutch Collaborative Academic Practice for Public Health Infectious Diseases (CAPI).
We would also like to thank the general practices of FaMe-Net for their efforts in regard to the routine registrations in the electronic health records and the RTC-Health data team, including Hans Peters and José Donkers, for providing us the data and information about FaMe-Net.
Artificial intelligence (AI) was used to search for commands and identify causes of errors in RStudio, and the following AI programmes were used: OpenAI ChatGPT, Perplexity, and Copilot.
Author contribution
J.M.A.D. conceptualized the study, curated the data, performed the formal analysis, investigated the data, designed the methodology, managed the study, visualized the data, wrote the original draft, and wrote, reviewed, and edited the manuscript.
C.H.M.J. conceptualized the study, designed the methodology, involved in formal analysis, supervised the study, and wrote, reviewed, and edited the manuscript.
A.K. curated the data, performed the formal analysis, designed the methodology, visualized the data, wrote the details of the statistical method in the Supplement, and wrote, reviewed, and edited the manuscript.
A.U. validated the data, investigated the data, designed the methodology, and wrote, reviewed, and edited the manuscript.
F.G. involved in formal analysis, designed the methodology, and wrote, reviewed, and edited the manuscript.
T.C.O.H. wrote, reviewed, and edited the manuscript.
S.F.H.R. conceptualized the study, designed the methodology, involved in formal analysis, supervised the study, and wrote, reviewed, and edited the manuscript.
A.T. conceptualized the study, supervised the study, wrote, reviewed, and edited the manuscript, and acquired funding.
Funding statement
This project was funded by the Ministry of Health, Welfare and Sport of the Dutch Government within the programme ‘Strengthening infectious disease control and pandemic preparedness of the regional Public Health Services’. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Open access funding provided by Radboud University Medical Center.
Competing interests
The authors declare none.
Open access
