Natural hazards are prevalent events worldwide. They are an integral part of nature. In recent years, however, climate change and other factors have increased the global incidence and intensity of natural hazards.Reference Smith, Wasiak and Sen 1 - Reference Revati Phalkey, Dash and Mukhopadhyay 3 When natural hazards occur in vulnerable communities, they lead to disasters. The usual outcomes of disasters are the destruction of infrastructure and health facilities or disruption of function,Reference Ardalan, Mowafi and Khoshsabeghe 4 mortality, injuries, environmental degradation, increasing vector feeding areas, mass displacement of the population, poor personal hygiene, and lack of access to safe drinking water. In addition to physical damage, disasters disrupt health programs,Reference Djalali, Hosseinijenab and Hasani 5 such as regular vaccinations and vector control. Consequently, the disaster-stricken areas are prone to communicable disease (CD) outbreaks.Reference Myint, Kaewkungwal and Singhasivanon 6 - Reference Date, Vicari and Hyde 8 Although the media may exaggerate the epidemics that follow a disaster,Reference Yan and Mei 9 any outbreak of CD after a disaster is always a major concern for public and health officials.
Therefore, one of the major tasks of disaster health response organizations is communicable disease management (CDM). The most important part of CDM in disasters is to set up a surveillance system (SS).Reference Sabatinelli, Kakar and Rahim Khan 10 A communicable disease surveillance system (CDSS) aims at systematic and continuous collection, analysis, interpretation, and publication of CD-related data for the management of disease in disaster-stricken areas. The system also helps in the development of prevention programs to reduce the incidence of morbidity and mortality and in health protection programs. 11 CDSSs were established after Hurricane Katrina,Reference Topran, Ratard and Bourgeois 12 , Reference Williams, Guariso and Guillot 13 the earthquake in Haiti,Reference Polonsky, Luquero and Francois 14 the flood in Pakistan,Reference Sabatinelli, Kakar and Rahim Khan 10 the earthquake and tsunami in Japan,Reference Tohma, Suzuki and Otani 15 and the Sichuan earthquake in China.Reference Yan and Mei 9
Like any program, the performance of a SS should be evaluated. Strengths and weaknesses should be identified to correct any weaknesses before the next disaster. Such assessments were done in previous disasters.Reference Choudhary, Zane and Beasley 16 , Reference Noha, Farag and Rey 17 Although different performance assessment criteria and indicators exist, the updated guidelines of the US Centers for Disease Control and Prevention (CDC) are a well-known tool for evaluating public health SSs. 11
On August 11, 2012, two earthquakes measuring 6.4 and 6.3 magnitude on the Richter scale, respectively, shook the towns of Ahar, Heris, and Varzaqan in East Azerbaijan. 18 These earthquakes killed 228 and injured 3037 people. More than 272 villages with 20,715 households and 85,466 residents were affected. In total, 13,837 residential units were destroyed. 19 In Varzaqan, 5 health centers, 25 health houses, and a medical laboratory were damaged. Two health centers and 49 health houses were damaged in Ahar. Three health centers and 14 health houses needed reconstruction in Heris. The earthquake-stricken areas were divided into 10 zones (5 zones in Varzeqan, 1 zone in Ahar, and 4 zones in Heris) to provide health services. On the basis of the emphasis and order of the Minister of Health for settling primary health care workers (called behvarz in the Iranian health system) in villages with over 20 households, 100 behvarz were recalled from other cities of the province to settle in the affected villages.
Like all health systems, the Iranian health system set up a CDSS after the twin earthquakes of East Azerbaijan in 2012. The SS was designed for 21 diseases or syndromes, including acute watery diarrhea, dysentery, water and food-borne outbreaks, animal bites, snake and scorpion bites, maculopapular rash and fever, rash, severe respiratory infections, flu-like syndrome, whooping cough, diphtheria, meningitis, cutaneous leishmaniasis, malaria, tuberculosis, acute flaccid paralysis (below 15 years), sexually transmitted diseases, botulism and acute jaundice syndrome, typhoid, pediculosis, and fever of unknown origin. The list of syndromes and diseases was decreased to 19 by September 20 (typhoid and pediculosis were excluded). The present study aimed to evaluate the performance of the CDSS.
The present study differed from previous studies in 2 respects. First, the CDC guideline 11 has never been used following earthquakes, the characteristics of which differ from other hazards. Second, this study was done for the first time in Iran. Iran has a special health system that is considerably different from other health systems in the world. In particular, the system is efficient in providing primary health care in rural areas. 20 , Reference Bagheri Lankarani, Alavian and Peyman 21 Since the earthquakes discussed here occurred in rural areas, this study is of particular importance.
Methods
In this study, a mixed-methods (qualitative-quantitative) approach was used for performance assessment of a CDSS in response to the twin earthquakes of August 11, 2012, in the East Azerbaijan province of Iran. The updated CDC guideline was used as a tool for performance assessment. The guideline was created in 1988 to assess the public health SS and was updated in 2003. The revised guideline includes assessing the usefulness of an SS and 9 attributes: simplicity, flexibility, data quality, acceptability, sensitivity, positive predictive value (PPV), representativeness, timeliness, and stability. 11
Description of the Attributes Used
The following attributes from the CDC guideline for evaluating a public health SS were used: 11
-
∙ Usefulness: An SS is useful when its output can be used in the prevention, control of adverse effects and health events, and performance assessment of other programs. To evaluate usefulness, the results of the SS should be compared with the setup objectives.
-
∙ Simplicity: This attribute has 2 dimensions: structure and implementation. The SS flowchart can demonstrate simplicity. Other components of simplicity are the amount and type of required demographic and behavioral data, the number of organizations involved in taking reports, the degree of integration with other systems, data collection method, data management techniques, the number of follow-ups to keep the data updated, methods of analysis and publication of data, resources needed for training staff, and time spent on system maintenance.
-
∙ Flexibility: Flexibility refers to the ability of the SS to adapt to changes in information needs, operating conditions, time and staffing constraints, and resource allocation.
-
∙ Data quality: Data quality reflects the completeness and validity of the collected data. Blank items and unknown responses are outnumbered in high-quality data. Data quality is characterized by the performance of the diagnostic units of diseases, transparency of the forms used, training quality of the individual completing the forms, and accuracy used for data management.
-
∙ Acceptability: Acceptability represents the interest of individuals and organizations for participation in the SS. The following quantitative methods can be used to assess acceptability: rate of individual and organizational participation; rate of questionnaire completion; refusal to answer questions in interviews; reporting level of physicians in the private sector, laboratories, and hospitals; and timeliness of reports. Acceptability can also be assessed by use of qualitative variables.
-
∙ Sensitivity: The sensitivity of an SS can be calculated for the reported cases (sensitivity to case detection) and the system’s ability in diagnosing overflows.
-
∙ Positive predictive value: The PPV is the proportion of reported cases that are actually related to the health events under surveillance.
-
∙ Timeliness: This variable shows the speed of an SS in different stages of setup; in other words, it shows the time required to identify trends, outbreaks, or the effects of preventive measures.
-
∙ Stability: This implies the reliability (the ability to collect, manage, and provide the data, correctly without error) and availability (capable of being operational at the time of need) of a system.
Assessment Method for the CDC Guideline Attributes
There are some methods for evaluating the attributes of the updated CDC guidelines. In the current study, usefulness, flexibility, and simplicity were qualitatively studied. Timeliness, data quality, representativeness, and stability were quantitatively assessed. Acceptability was assessed both quantitatively and qualitatively.
For the qualitative part of the study, in addition to direct observation of the performance of the SS by one of the researchers, some interviews were conducted. Interviewees were purposefully selected from among the SS participants. Only those who directly participated in the SS (data collection, summarization, and analysis) and expressed their interest to participate were selected for the interviews. We visited their workplaces with prior appointments. Their verbal consent was obtained, and then they were asked to give their opinions on the SS. We did not use a questionnaire for the interviews. We started our semi-structured interviews with a basic question and then according to the interviewee’s response, we asked some probing questions for gathering detailed data. The interviews were continued until data saturation.
Different methods were used in the quantitative part of study to assess the SS attributes. To assess the acceptability of the SS, the number of health teams, health relief posts, private and public hospitals, clinical laboratories, and volunteers who participated in the SS were divided by the total number of each and multiplied by 100. In addition, the proportion of disease cases that were reported by private hospitals, laboratories, and the public, and the indicators showing system acceptability, were calculated. For timeliness of the system, activation time and the time required for reporting diseases or outbreaks to the authorities of the health center from the onset (mean interval between onset of disease or outbreak to its reporting to the provincial health center) were assessed. To assess the quality of the data, 10% of the daily reporting forms and the completed forms for the samples and disease cases were randomly extracted, and the number of items completed was divided by the total number of items.Reference Choudhary, Zane and Beasley 16 To examine the representativeness of the SS, the total population and area covered by the SS was divided by the total number of the affected population and damaged areas. To study stability, the number of days or weeks that data were reported by the SS was divided by the number of days or weeks that the SS was active and multiplied by 100. To assess the sensitivity and PPV on the basis of existing literature, a gold standard is required. Since no such system existed for the Azerbaijan earthquake, PPV was not calculated.
Because an SS can be used even with low sensitivity in the monitoring process when sensitivity is constant, 11 the trend for diseases and syndromes was compared with the expected national and provincial rates. The number of cases was divided by the expected number of cases (based on previous estimates in the instructions; when the expected number was not mentioned in the instructions, we used the average provincial incidence).
Description of the CDSS in the Earthquake-Stricken Areas of East Azerbaijan
Three days after the earthquakes, the East Azerbaijan Health Center set up a CDSS in collaboration with the Ministry of Health. Thirty medical teams consisting of a general physician, technician/expert in disease control, technician/expert in environmental health, nurse or midwife, and a driver were deployed to the zones. One officer was chosen as a coordinator for each town. Definition of the diseases under surveillance and the forms required for data collection were extracted from the “National public health disaster and emergency operations plan,”22 copied, and given to the coordinators, who were taught how to complete the forms and report. The officers were asked to provide their health teams with forms and definitions of the cases and to train them on how to complete and report the forms. Collecting and reporting of data started from the fourth day after the earthquake and continued until 98 days after the quakes.
Results
Despite being prone to CD, there was no increase in the incidence of these diseases in the affected area. Only 3 cases of diarrheal outbreaks were reported, which were controlled. Within 1 month of the earthquake, 1315 cases of diarrhea were examined and no positive cases of cholera, salmonella, shigella, or Escherichia coli were reported. In one of the affected cities, some cases of influenza-like syndrome were reported, which were promptly examined by the team. Samples were prepared, were sent to the laboratory, and were found to be negative. During this period, a suspected malaria case was reported, which was not confirmed.
Performance Assessment of the SS
To assess SS performance qualitatively, we interviewed 30 persons who were involved in collecting, summarizing, and analyzing the reports in the health teams (n=15, or 50%), health centers in cities (n=6, or 20%), and provincial health center (n=9, or 30%). Of these, 8 were women (26.66%) and the rest were men. Their education levels were general practitioner (n=17, or 56.66%), graduate student (n=2, or 6.66%), bachelor of science degree (n=8, or 26.66%), and associate of science degree (n=3, or 10%). Ten (33.33%) interviewees had less than 5 years, 6 (20%) had 5 to 10 years, 4 (13.33%) had 10 to 15 years, 4 (13.33%) had 15 to 20 years, 1 (3.33%) had 20 to 25 years, and 5 (16.66%) had over 25 years of work experience.
Usefulness
All interviewees believed that the SS was useful. The SS could collect and analyze the required data to examine the health status of the region both daily and weekly. The results of this analysis were used for planning environmental health activities, supplying required medicines, and developing preventive measures. Early in the response phase when the senior managers and health officials were worried about outbreaks of CD, the reports of the SS assured them that health status was under control. This was a strong response to the rumors, and the data obtained satisfied the needs of the media also. There was no increase in the trend of CD incidence from the early settlement of CDSS, and this was the most useful attribute of the SS from the participants’ viewpoint. The SS demonstrated the effectiveness of health interventions as well.
Simplicity
The SS was simple and completely consistent with the routine SS with which the staff members were familiar. It was set up by using the existing facilities. Its structure was very simple, and only the syndromes or diseases with a risk of outbreak were included. Aside from the health sector, other organizations were not involved in data collection (although this could also be a weakness).
The data collection methods of the SS required no advanced equipment. Telephone and fax were used for daily reports. No software was used for summarizing and analyzing the data. The data were collected, summarized, and sent by use of a pen and paper. The provincial health center compiled the data into spreadsheets. When the data were summarized, a trend graph was drawn and analyzed and feedback was provided. The system did not incur any cost for setting up the SS. The data were collected daily in a short time by the staff, and it was possible to update the SS by spending a couple of hours on the system daily.
Flexibility
The SS was flexible and designed on the basis of local needs. Some changes were made in the forms and the definitions of diseases and syndromes on September 20 on the basis of needs. Although some interviewees (7 interviewees, or 23.33%) considered this a weakness and criticized these changes, most interviewees believed that this system could also be used for other hazards and areas. Detailed results of simplicity, flexibility, usefulness, and data quality assessment are highlighted in Table 1.
Results of Simplicity, Flexibility, Usefulness, and Data Quality Assessment of the Surveillance SystemFootnote a
a Abbreviation: SS, surveillance system.
Data Quality
To examine data quality, 10% percent of the daily report forms and completed forms of reported cases were randomly reviewed.Reference Choudhary, Zane and Beasley 16 Between August 23 and September 22; a total of 93 report forms containing 11 items were required to be completed daily (11×93=1023), of which 352 items were not completed (34.4%). Major items that were not completed included the team number, first name, last name, signature, and stamp of the individual who completed the form. Because the case reporting forms were destroyed, it was not possible for the research team to study their completion rate.
Acceptability
During the period in which the CDSS was active, a total of 30 medical teams, 146 health houses, 98 health relief posts, 3 hospitals, 8 medical diagnostic laboratories, and 11 volunteer teams were providing health services in the affected area. Only 30 health teams reported formally, which means that the acceptability of the SS was 10.63%. Since the number of private physicians offices was not known to the research team, they were excluded from the study.
Although health houses and health relief posts were set up, they did not report daily or regularly. They only reported the occurrence of any suspected case of reportable disease and outbreaks. The proportion of infectious diseases reported by hospitals, laboratories, and the public, as an indicator of SS acceptability, was zero. In the qualitative part of acceptability assessment, some interviewees referred to the rejection of the definitions by physicians.
Sensitivity and Positive Predictive Value
PPV and sensitivity were not calculated owing to the absence of a gold standard. However, the incidence rate of disease and syndrome cases identified by the SS were calculated and compared with the expected national and provincial incidence rates (Table 2).
Comparison of the Number of Diseases and Syndromes Identified by the Surveillance System With Expected Incidences
a National expected incidence ratio.
b Provincial expected incidence ratio (East Azerbaijan).
Representativeness
The SS was representative. The medical teams sent by the provincial health center (30 health teams) actively covered 100% of the affected population and the whole affected area (272 affected villages). However, this 100% coverage was not daily. On average, 45 villages were covered per day, and all the affected villages were covered in nearly a week.
Timeliness
The interval between the incidences of urgent reportable CD until the health center expert got the quick report was an average of 6 hours. The average time for sending a daily report was 26 hours and that for written reports was 7 days. In terms of the outbreaks, the average time for the diagnoses by the SS to reach the provincial health center was 8 hours.
Stability
The CDSS was active for 97 days. By September 22, daily and weekly reports were regularly sent. A total of 37 reports were sent by September 22, and 8 reports were sent in a period of 2 months (from September 23 to November 21). Therefore, the stability of the SS was perfect (100% of expected reports were sent). [Stability=number of sent reports/number of expected reports×100.]
Discussion
In this study, the performance of the CDSS that was set up in the earthquake-stricken areas of East Azerbaijan in Iran was assessed by using the updated guideline of the CDC. This assessment showed the SS to be useful, simple, representative, flexible, stable, and timely. However, data quality (65.6%) was low, and acceptability was very low (10.63%). Sensitivity and PPV could not be calculated owing to the lack of a gold standard.
The SS provided the needed data for the prevention and control of CD and the designing of preventive interventions. Owing to widespread destruction caused by the earthquakes, the infrastructure and several health facilities were disrupted. A large number of the population was settled in tents. Thus, authorities and other people were very concerned about CD outbreaks in the affected region. The main objectives of setting up the SS were to bring reliability and peace of mind, dispel any rumors, and convince the media, which were effectively managed. This SS was the only active system in the region, and like other systems set up during disasters, it was useful and beneficial.Reference Noha, Farag and Rey 17 , Reference Bagheri Lankarani, Alavian and Peyman 21
Although the health workers had no previous experience of working in disasters and were not officially trained, they could set up the SS very quickly and proceed. This is because the SS was simple and the workers were familiar with its structure, forms, instructions, and definitions. A brief training while the SS was operational met most of the health workers’ needs. This system was similar to and consistent with the routine SS. The forms used were also very easy to complete. As mentioned in the literature, any SS should be simple, and our study showed that this system, like other systems set up during previous disasters, was simple.Reference Noha, Farag and Rey 17 , Reference Bagheri Lankarani, Alavian and Peyman 21
The flexibility of the SS was evaluated and confirmed in previous studies. On the basis of health conditions, the forms and diseases and syndromes under surveillance in the SS were changed and these changes were easily accepted by the staff. Although some staff were critical and believed that these changes should not be applied, this is a well-known feature of any SS. 11 , Reference Noha, Farag and Rey 17 , Reference Bagheri Lankarani, Alavian and Peyman 21 SS flexibility has always been evaluated in previous studies, and previous assessments indicated flexibility of the studied SS.
Data quality is another important feature by which an SS can be judged, and completeness of the forms is one of the best examples of data quality. A notable weakness of the SS was the lack of completeness of the fields. Although this study only examined the completeness of the daily report forms, the completeness of the fields was less than in previous studies. In the study by Choudhary, 18 the level of completeness was excellent (98%). In a study by Farag,Reference Noha, Farag and Rey 17 most of the fields related to demographics (90%-97%) were completed, and only fields related to ethnicity and race (which was difficult to extract) were incomplete.
Another noticeable weakness of the setup of the SS was its low acceptability. The main cause of this low score was the lack of coordination between the various sectors participating in the response. The Red Cross, medical community, and volunteer health teams did not participate in the SS. Indeed, the way they worked was different, and they used their own specific guidelines. Even health relief posts that were set up with the participation of the behvarz in the villages did not participate in the SS. The main reason for this challenge was the lack of previous coordination and preparation. However, the acceptability of the SS was very high in previous studies, and health organizations participated actively in the SS.Reference Noha, Farag and Rey 17 , Reference Bagheri Lankarani, Alavian and Peyman 21
Owing to the lack of a gold standard and no other SS with which to make comparisons, sensitivity and PPV were not calculated. This shortcoming was also observed in other studies. When the sensitivity of an SS is low but stable, the results can be used to monitor disease trends and health events.Reference Myint, Kaewkungwal and Singhasivanon 6 Although we cannot judge the degree of sensitivity in the assessed SS, the results were used to monitor CD trends in the affected areas, and the trend analysis was performed daily.
The SS must be representative to judge the health status of a region by use of its outputs. The SS was representative because it covered the entire affected population and earthquake-stricken area. The health teams were actively attending to all areas and assessing the situation.
The SS was designed immediately on the basis of the experience of the Bam earthquake and was operational from the fourth day after the earthquake and provided the necessary data. It was timely during its activity, and one of the timeliest systems in providing data.
An important aspect of the SS was its continuity and stability. The system was operational during the entire period of its activity. The complete stability of the SS may be related to the good structure of the Iranian rural health system and its primary health care system. As previously mentioned, the primary health care system of Iran, particularly in rural areas, is one of the most prominent SS and providers of health care services. 19 , 20 Because simple technology was used, providing resources and requirements was also simple, and thus continuity was easily possible.
Conclusions
Setting up an SS in order to provide strong and stable data for monitoring health-related events, especially CD, is a vital attribute of response to disasters. An SS is an essential tool for health care organizations at all levels. Thus, an active SS was set up in East Azerbaijan after the twin earthquakes. Evaluation of the SS showed that it was useful, simple, flexible, representative, stable, and timely. However, the system had low acceptability and data quality issues. Therefore, the health system should attempt to coordinate the responses of all health organizations and institutions involved in a health disaster by applying the same structure and similar forms, and taking advantage of sharing their data to boost acceptability. Training and continuous practice of the staff should be considered to improve data quality.
Acknowledgment
This study was part of a PhD thesis/dissertation supported by Tehran University of Medical Science. The study was also funded by I.R. Iran’s National Institute of Health Research (NIHR), Tehran University of Medical Science, contract no. 241/M/91375.
