A decision analytic model often comprises a significant part of a health technology assessment. As health technology assessment in the hospital setting evolves, there is an increased need for modeling methods that account for patient care pathways and interactions between patients and their environment. For example, an evaluation of a computed tomography (CT) scanner for a new indication would need to consider the current and increased demand of the machine and how that may affect service in other areas of the hospital. This problem solving approach views “problems” through a systems perspective.
Systems analysis techniques have been developed over decades through operations research and industrial engineering fields (Reference Hillier and Lieberman19). Under systems analysis, mathematical modeling techniques involve mapping a system or process from the real world to a more simplified representation using a set of variables and equations. These models have been identified for use in health technology assessment (Reference Stahl44), mainly because they allow decision makers to simulate hypothetical scenarios without making actual changes to the system. Such models enable the analysis of “what-if scenarios” and provide the opportunity to identify optimized solutions under constraints (e.g., resources, budget, benchmarks). Measures of systems behavior include waiting time, throughput, and resource utilization. Waiting times can be of particular interest due to adverse events and current pressure from the public to receive timely care.
The hospital emergency department (ED) is of particular importance because it is a dominant source of acute care and the main route of admission to the hospital for a large percentage of the population. Long waiting times lead to overcrowding and have been a widely documented problem in EDs (Reference Bond, Ospina and Blitz5). Overcrowding has been associated with increased risk in mortality and re-admission, higher probability of leaving without being seen, and delayed or non-receipt of antibiotics for patients with community-acquired pneumonia (Reference Bernstein, Aronsky and Duseja3;Reference Guttmann, Schull, Vermeulen and Stukel18;Reference Pines, Localio and Hollander39). As such, identifying causes of overcrowding is an essential step to improving safety and outcomes.
ED patient care is complex and relies on several human, physical, and organizational elements (e.g., patients and their relatives, buildings and equipment, management systems). At its most basic level, it is a system consisting of patients, resources (e.g., beds, physicians) and processes (e.g., triage). Generally, patients flow through the following order of processes: triage, registration, placement in an ED bed, clinical assessment, treatment, and/or diagnostics/laboratories followed by disposition. Waiting times in the ED exist for several reasons: capacity does not meet demand (e.g., overcrowding, insufficient number of beds), sub-optimal management of capacity or demand (e.g., scheduling, flow), significant variability over time in demand for services, and differences in patient acuity (Reference Jun, Jacobson and Swisher23). The complexity of care within the ED compounded with the multifaceted issues associated with excessive waiting times lends itself well to systems analysis. Although several comprehensive reviews have outlined the application of mathematical modeling in health care (Reference Fone, Hollinghurst and Temple16;Reference Jun, Jacobson and Swisher23;Reference Mustafee, Katsaliaki, Taylor and SJE37), these reviews were not specific to the ED setting. Despite the increased pressure to reduce waiting times, there are no recent reviews analyzing the use of mathematical models for evaluating the ED. To better inform future HTA and decision making in the hospital setting, the purpose of this study was to evaluate the literature from the perspective of both the methods used (i.e., modeling techniques) and the empirical findings (i.e., study results) perspective. The specific objectives were (i) to identify recent mathematical modeling techniques that have been used to evaluate strategies for decreasing waiting times in the hospital emergency department; (ii) to compare mathematical modeling techniques; and (iii) to identify commonly modeled strategies and to summarize their impact on waiting times.
METHODS
Literature Search
A search strategy was developed to identify the published literature evaluating waiting times in a hospital ED using mathematical modeling techniques. Individualized search strategies (Appendix 1) were developed for several electronic databases using relevant subject headings supplemented by keywords. Due to the scope of this research, medical, engineering, and business (operational research) databases were searched: OVID MEDLINE and EMBASE, Engineering Village 2 Compendex and Inspec, and EBSCOhost Business Source Complete. Subject headings were derived using the thesaurus in each database and were searched individually to assess their added value to the overall strategy. This resulted in certain terms being dropped (e.g., mathematical techniques, emergency physicians). Search strategies were also developed in consultation with a health science librarian and an engineering librarian. Each strategy was limited to English, peer-reviewed journals published between January 2000 and July 2010. These dates were chosen because at the time this review was conducted, past reviews had not included studies after 2000. Additionally, there was a steady publication increase after 2000 of mathematical model healthcare applications (Reference Mustafee, Katsaliaki, Taylor and SJE37). Conference proceedings from Compendex and Inspec were included because engineering conference proceedings are typically published in the format of an article with preliminary results.
Study Selection
Inclusion criteria were adapted from the study by Hoot and Aronsky (Reference Hoot and Aronsky21): (i) implemented a mathematical modeling technique; (ii) analyzed data; (iii) studied waiting times from the perspective of general emergency medicine; (iv) studied waiting times with respect to typical daily arrival rates and patient demands (i.e., no catastrophic events or patient simulation studies); and (v) the primary outcome measures were waiting/process times in the ED, length of stay, or proportion of patients meeting a waiting time target in the ED. For inclusion, a study had to meet all five criteria.
Using pre-determined inclusion/exclusion criteria, two reviewers, an economist (M.L.), and an engineer (K.L.), using Reference Manager v.11 Network, screened titles and abstracts of identified studies for potential inclusion (1st level screening). The kappa statistic was calculated to assess reviewer agreement at this screening level. Full text versions of the published articles were obtained for those studies that met the inclusion criteria and also for those studies where suitability for the review could not be determined based on the title and abstract. One reviewer (M.L.) conducted the full-text screening (2nd level) using the same criteria as the first level screening to determine final inclusion for data abstraction and analysis. Consensus with a second reviewer (J.E.T.) was obtained when it was uncertain if a study met the inclusion criteria. The second reviewer also performed a full-text screening of a 20 percent random selection. A bibliographic search of the included studies was also completed to ensure that all relevant studies were identified.
Data Abstraction and Analysis
A data abstraction form was created to record study information. In addition to recording the mathematical modeling technique, basic study information such as country, objectives, main performance measures, and findings/conclusions were abstracted. To compare the different modeling techniques, each technique was assessed based on 10 model assumptions: analytical or simulation, deterministic or stochastic, discrete or continuous, performance measures, diagrams, capability of handling multiple resource constraints, memory, level of data abstraction, model building time, and developed software (Reference Law and Kelton31). Table 1 explains these concepts and their relevance to the hospital ED. Each study was also analyzed in terms of strategies used to reduce waiting times in the ED. Strategies for waiting time reduction were categorized into scheduling (staff and operational), demand management (methods to re-distribute patients), resource allocation (i.e., beds and staff), change in process times, and other. Two reviewers (M.L., J.E.T.) abstracted the data separately using a Microsoft Excel® template with predefined categories (Appendix 2).
Description and Significance of Different Model Assumptions Used for Comparison
RESULTS
Literature Search
The literature search identified 1,795 unique citations following the removal of duplicates. After screening titles and abstracts, 1,712 citations were excluded, mainly because the articles evaluated (i) a simulated environment where trainees practice techniques on standardized patients or part-task trainers rather than computer simulation; (ii) evaluated catastrophic/infectious disease; (iii) did not include data analysis; or (iv) evaluated prediction scores for triaging an illness within the ED. For the first level of screening, a kappa coefficient of 0.73, reflecting good agreement, was calculated between the two reviewers. A full text review of the remaining eighty-three articles excluded fifty-four additional citations, resulting in twenty-nine studies (fifteen journal articles and fourteen conference papers). No additional articles were identified from searching the references of the included studies. For the second level of screening, a kappa coefficient of 0.88 was calculated between the two reviewers for the random 20 percent sample. Figure 1 summarizes the study selection process. Included and excluded studies from the second level of screening are in Appendix 3.
Diagram of included and excluded studies from the literature review.
Approximately half of the journal articles were published in health science journals and half in operational research or systems management journals. The conference proceedings were all presented at the Winter Simulation Conferences (the primary international outlet for disseminating advances in the field of system simulations). The studies were set in various countries: the United Kingdom (n = 7), the United States (n = 10), Canada (n = 3), Finland (n = 1), Norway (n = 1), Kuwait (n = 1), France (n = 1), Taiwan (n = 1), Japan (n = 1), Trinidad and Tobago (n = 1), Spain (n = 1), and unknown (n = 1).
Mathematical Modeling Techniques
The included studies used four different mathematical modeling techniques: queuing analytic model (n = 4) (Reference Cochran and Roche12;Reference Laskowski, McLeod, Friesen, Podaima and Alfa30;Reference Mayhew and Smith34;Reference Puente, Gomez, Parreno and de la Fuente40), discrete event simulation (n = 20) (Reference Beck, Balasubramanian and Henneman2;Reference Brailsford, Lattimer, Tarnaras and Turnbull9;Reference Coats and Michalis11;Reference Connelly and Bair13–Reference Ferrin, McBroom and Miller15;Reference Gunal and Pidd17;Reference Holm and Dahl20;Reference Hung, Whitehouse, O'Neill, Gray and Kissoon22;Reference Khadem, Bashir, Al-Lawati and Al-Azri25–Reference Komashie and Mousavi28;Reference Mahapatra, Koellig and Patvivatsiri33;Reference Medeiros, Swenson and DeFlitch35;Reference Meng and Spedding36;Reference Nielsen, Hilwig, Kissoon and Teelucksingh38;Reference Ruohonen, Neittanmaki and Teittinen42;Reference Takakuwa and Shiozaki46;Reference Tao, Guinet, Belaidi and Besombes47), discrete event simulation in combination with optimization (n = 2) (Reference Ahmed and Alkhamis1;Reference Yeh and Lin51), system dynamics (n = 2) (Reference Lane, Monefeldt and Rosenhead29;Reference Storrow, Zhou, Gaddis, Han and Miller45), and agent based modeling (n = 2) (Reference Laskowski, McLeod, Friesen, Podaima and Alfa30;Reference Wang50) (note: total adds to thirty instead of twenty-nine because one study used a queuing model and an agent based model). Table 2 presents a comparison of the mathematical modeling techniques with respect to the 10 model assumptions listed in Table 1.
Comparison of Mathematical Modeling Techniques by Model Assumption
GoF, Goodness of Fit Test
Queuing models are characterized over time by an arrival process, a service process (e.g., treatment), the number of servers (e.g., doctors), a constraint on the number of patients allowed to enter the queue and a queue discipline (Reference Thomas, Wilson and Zandin48). The queue discipline is the rule that a server uses to choose the next patient. Examples of queuing disciplines include: first-in, first-out (FIFO); last-in, first-out (LIFO); service in random order (SIRO); priority (PR), and general discipline (GD). One of the studies combined a queuing model with the use of fuzzy numbers which incorporates a level of uncertainty into the model.
Twenty of the studies used discrete event simulation (DES) to meet their objectives. DES is characterized by several concepts: entities that move through the model (e.g., patients), attributes that are characteristics of the entities (e.g., sex), resources that are seized by the entities (e.g., staff), queues (e.g., waiting lines), and events or processes (e.g., triage) that the entity will flow through (Reference Kelton, Sadowski and Sturrock24). Essentially, DES represents a network of queues for services that a patient flows through where attributes determine the pathway of the patient. This technique is unique because it has a simulation clock that keeps track of the passage of time allowing analysts to control the start and end points (Reference Fone, Hollinghurst and Temple16).
Two studies combined DES with optimization. DES computes a set of performance measures based on defined inputs, however, combined with optimization the model can retrieve the best inputs based on an objective function. For example, the objective may be to re-allocate resources to ensure that all low acuity patients do not have a length of stay longer than 8 hours. One study specified the use of a Genetic Algorithm optimization method, which applies a class of evolutionary algorithms to derive solutions from populations (Reference Yeh and Lin51). For instance, a new solution is taken and used to form another solution in hopes that this population will be better than the old one and eventually used to derive an optimal solution.
Two studies applied system dynamics modeling. System dynamics is composed of either a qualitative component or both a qualitative and quantitative component. The qualitative phase involves developing an understanding of the system not only by the research team but also by the stakeholders in the system (Reference Brailsford7). A causal loop diagram is developed with the aim of understanding both direct and indirect relationships between important variables within the structure of the system (Reference Brailsford8). The variables may not necessarily be quantifiable (e.g., disease advocate group pressure). The resulting causal loop diagram could be the end result of a system dynamics model, however, analysts can choose to add a quantitative component to estimate performance measures. To quantify the model, the causal loop diagram is converted into a stock and flow diagram (Reference Brailsford8). Conceptually, this diagram can closely resemble the ED process.
The remaining two studies used an agent based modeling (ABM) approach. ABM consists of a set of agents (e.g., patient, physician) where each agent is governed by a set of behaviors (e.g., treat patient), interactions (e.g., patient can interact with physician), and rules (e.g., maximize patient health) (Reference Macal and North32). Agents are autonomous in that each agent has its own decision-making process. Interactions occur within a pre-defined topographical space that includes resources. ABM is unique because it can capture emergent phenomena (e.g., collective behavior) and agents can adapt and learn (Reference Bonabeau4).
Study Findings and Waiting Time Reduction Strategies
A different hospital ED was evaluated in each of the studies under review. Their individual objectives, performance measures and findings are summarized in Table 3. Below is a brief summary of study findings by common strategy used for reduction of waiting times.
Summary of Study Characteristics, Objectives, Performance Measures, and Findings
Acronyms: UK, United Kingdom; US, United States; NR, Not reported; ED, Emergency Department; ART, Acuity Ratio Triage; LOS, Length of Stay
*Discrete event simulation model within a system dynamics model.
Scheduling. Six studies, all DES, evaluated different staff shift patterns or operational hours as strategies to improve ED efficiency (Reference Ahmed and Alkhamis1;Reference Coats and Michalis11;Reference Duguay and Chetouane14;Reference Mahapatra, Koellig and Patvivatsiri33;Reference Meng and Spedding36;Reference Yeh and Lin51). All resulted in reduced patient waiting times or increased throughput.
Demand Management. Four studies found that fast-tracking low acuity patients through the ED could have both positive and negative effects (Reference Brailsford, Lattimer, Tarnaras and Turnbull9;Reference Cochran and Roche12;Reference Connelly and Bair13;Reference Tao, Guinet, Belaidi and Besombes47). The studies indicated that any improvements for low acuity patients were at the expense of high acuity patients or it decreased door to doctor time, but only if staffing resources were concurrently re-allocated. By altering the triage process, re-allocating an extra triage nurse dependent on patient demand, using a triage team or including a physician at triage, reduced average patient throughput time (Reference Medeiros, Swenson and DeFlitch35;Reference Nielsen, Hilwig, Kissoon and Teelucksingh38;Reference Ruohonen, Neittanmaki and Teittinen42;Reference Wang50). Triage to bed time decreased if a holding area, ED discharge lounge, and observation unit were added (Reference Kolb and Peck27). Bedside registration was not found to be an effective intervention at decreasing length of stay (Reference Beck, Balasubramanian and Henneman2).
Resource Allocation. Altering the number of staff (e.g., physician, nurse, clerks), beds, and/or rooms (Reference Duguay and Chetouane14;Reference Hung, Whitehouse, O'Neill, Gray and Kissoon22;Reference Komashie and Mousavi28;Reference Puente, Gomez, Parreno and de la Fuente40) showed reductions in patient waiting times, with the exception of two studies that found no change (Reference Gunal and Pidd17;Reference Khare, Powell, Reinhardt and Lucenti26).
Process Times. Six of the studies altered inputs to the model to determine whether waiting times decreased if the proportion of patients waiting decreased. Diagnostic/laboratory process times were shortened (Reference Ferrin, McBroom and Miller15;Reference Meng and Spedding36;Reference Storrow, Zhou, Gaddis, Han and Miller45;Reference Wang50), which was significantly associated with a decreased ED length of stay with the exception of one study that found no change (Reference Gunal and Pidd17). Increasing the rate of inpatient admission was successful in decreasing the length of stay in the ED (Reference Khare, Powell, Reinhardt and Lucenti26).
Other. Khadem et al. (Reference Khadem, Bashir, Al-Lawati and Al-Azri25) altered the entire layout of the hospital ED to determine the most efficient layout with respect to waiting times. Mayhew and Smith (Reference Mayhew and Smith34) evaluated whether a change in the discharge definition would decrease process completion time. They re-defined discharge as occurring when the patient is referred or becomes an inpatient as opposed to once they are transferred. Using this definition resulted in faster completion times. Takakuwa and Shiozaki (Reference Takakuwa and Shiozaki46) simulated an increase in number of patients to reallocate resources based on increased waiting times. Laskowski et al. (Reference Laskowski, McLeod, Friesen, Podaima and Alfa30) investigated the use of agent based modeling to evaluate resource optimization and workflow and an analytic queuing model to evaluate waiting times.
DISCUSSION
Faced with the pressure to reduce resource use and improve quality of service under fixed budgets, decision makers are presented with the difficult task of reducing patient waiting times. Because of the various factors involved from both the demand and supply sides, collecting descriptive data regarding waiting times is helpful to inform whether targets are being met, but is likely to be insufficient to understand the systemic issues related to waiting times in the healthcare system ED. Mathematical modeling has an important role to play as they can consider all system components and their interactions in the same model. To address this issue we conducted a literature review to determine the use of these techniques in evaluating waiting time reduction strategies in the ED. The review revealed that twenty-two studies presented DES models (where two used optimization), two system dynamics models, four queuing analytic models and two agent based modeling. Common strategies to decrease waiting times in the ED included altering scheduling, resource utilization, and process times. Only a few studies indicated that results from the mathematical models were implemented into practice.
Selecting a mathematical modeling technique depends on several factors. The group Research Into Global Healthcare Tools (RIGHT) has recently developed a selection framework for modeling and simulation techniques (41). This framework consists of two main criteria: project life cycle stage (e.g., needs and issues identification or performance evaluation) and type of output (e.g., system interaction or comprehensive system behavior). Selection can also be characterized by the amount of time, money, knowledge and data that are available for the model.
Specifically, queuing models are more useful for modeling simple systems because as complexity is added the analytical solutions become less attainable. Arguments against queuing models focus on their theoretical assumptions. Queuing models make the assumption of Poisson distributed arrival times, exponentially distributed inter-arrival times, an infinite queue length, one server and linear relationships. Frequently, this is not the case in the hospital ED. It is possible to extend queuing models outside of these basic assumptions (e.g., multiple simultaneous servers); however, this involves the use of simulation as the analytical solutions are no longer plausible. Puente et al. (Reference Puente, Gomez, Parreno and de la Fuente40) combined simulation with a simple queuing model.
System dynamics modeling is an attractive technique for strategic planning of large populations because of causal loop diagrams and the use of aggregate level data to populate the model (Reference Mayhew and Smith34). The causal loop diagram is flexible because both tangible (e.g., increased waiting time) and intangible (e.g., stakeholder pressure) effects can be incorporated (Reference Mayhew and Smith34). The tradeoff of using this approach is that it lacks memory and patient individuality is lost because of the indivisibility of using continuous variables (Reference Brailsford8). As such, system dynamics is not an optimal tool for understanding detailed workings of the ED. DES models may be preferred for those needing an exact or very accurate understanding of comprehensive system behavior (i.e., resource allocation, implementation, evaluation).
DES models allow modeling the hospital ED in greater detail because they are stochastic, have memory and use discrete inputs. They can also be used to identify causes of bottlenecks and queues or to simultaneously evaluate performance changes based on changes in scheduling, triage and the addition/subtraction of resources. This technique is unique because it has a simulation clock that keeps track of the passage of time allowing analysts to control the start and end points. This is important for dynamic systems like emergency departments where analysts are interested in the steady state. It also allows for analysis of a system where interest is in long-run behavior. DES is a valuable tool in modeling complex systems with non-linear patient flow, typical of the ED. It is also flexible in its ability to manage patients with numerous characteristics (e.g., patients with different acuities, illnesses, sex and age). Additionally, it is possible to model interactions between resources (e.g., physician with residents) using a form of pseudo-agent-based modeling combined with DES. Of the studies identified in the review, DES was the most frequently used technique and was used to address multiple issues simultaneously. For instance, Meng and Spedding (Reference Meng and Spedding36) simultaneously modeled whether changes in process times, the addition of beds and a change in operational hours would reduce waiting times. The main drawback is the time required to collect appropriate and accurate data. Data requirements depend on the amount of modeling necessary to answer the proposed question. A DES model may require the following: resource shift and break schedules, time stamp data (arrival, triage, in bed, seen by nurse, seen by physician, disposition), and transfer times (e.g., how long does it take to walk between the triage and waiting rooms). It is important to assess data requirements and timelines before building a mathematical model.
Similar to DES, ABMs can be used to understand comprehensive system behavior. Agents are programmed at the micro-level (e.g., patients, physicians, organizations) to determine macro-level effects (e.g., performance measures, agent interactions) ABMs have similar modeling advantages as DES: stochastic, discrete, simulation clock and the ability to model non-linear pathways. The main difference is that there is no global system behavior in ABM. Behavior is defined at the individual level and global behavior emerges from interactions between agents and with the environment. Additionally, contrary to DES, resources such as physicians and nurses have autonomous decision-making behaviors. For instance, the physician has the ability to prioritize tasks (e.g., multi-task, interact with residents) rather than act as only a server to patients. Drawbacks are also similar to DES: large data requirements and long model building time. ABM also requires a greater understanding of the agents within the system because individual behaviors need to be programmed in the model. Additionally, there are few user friendly softwares and, therefore, computer programming skills (e.g., C++) may be required.
Our review identified two studies using optimization to model waiting times despite the fact that optimization techniques are commonly used for scheduling staff and appointments in the service sector (Reference Carter and Lapierre10;Reference Topaloglu49). Optimization is a technique that can be used alone or in conjunction with simulation. It has a limited capacity to characterize complex systems (Reference Jun, Jacobson and Swisher23) but is efficient because it only requires one experimental run (Reference Jun, Jacobson and Swisher23). In the past, the computing complexity of hybridizing these two techniques discouraged use (Reference Jun, Jacobson and Swisher23), however, standard simulation packages (e.g., Arena) are now offering options for combining optimization and simulation. This is a very fertile area of future research for waiting time targets. To further understand the utility of these models, it is important to assess implementation of recommendations derived from the models into practice. The primary purpose of five of the identified studies was to describe the development of a model rather than to analyze scenarios or policy changes, however, two studies discussed applying model recommendations. Implementing an extra staff shift resulted in reduced waiting time for a patient to be seen and expedited throughput (Reference Hung, Whitehouse, O'Neill, Gray and Kissoon22). In another case, a hospital implemented split flow and achieved a 61 percent reduction in patients leaving without treatment (Reference Cochran and Roche12). Further research needs to be conducted into this area to assess the usefulness of these modeling techniques for decision making.
Mathematical modeling techniques commonly used for economic evaluations (i.e., decision trees and Markov models) were not found in the literature review. This is a result of their limited ability to handle non-homogeneous populations and highly variable medical systems. They require more programming to include elements such as memory that are already built into other modeling techniques. The usefulness of decision trees and Markov models are limited in their capability to handle systems such as the emergency department.
A few limitations were associated with this study. First, several challenges arose when developing the literature search strategy because of the cross-disciplinary nature of systems analysis (i.e., operations research, industrial engineering, health services research). Indexing in the engineering and business databases is far less detailed than in the health literature databases and therefore their searching tools were more basic. This could have resulted in the indirect exclusion of some articles as they would not have been captured by the search terms. However, reference lists of the included studies were searched to identify additional papers. In addition, results were based on robust search strategies developed in consultation with librarians where two independent reviewers conducted the screening and abstracted the data.
Although several biases can be adjusted for in the modeling process (e.g., temporality) only two studies indicate effectiveness from implementation into practice. As such, individual study findings should be used with caution as the results were based on computer simulations. Unlike other interventional health studies (i.e., CONSORT (Reference Schulz, Altman and Moher43) and STARD (Reference Bossuyt, Reitsma and Bruns6)), computer simulation studies do not have recommended standards of reporting. Similarly, there exist no validated quality assessment tools for such models. Finally we limited the scope of our review to publications over the last 10 years as previous simulation reviews included publications up to 1999 (Reference Fone, Hollinghurst and Temple16;Reference Jun, Jacobson and Swisher23). Despite these limitations, this report provides evidence regarding the use of mathematical models to study waiting times in the ED. Results also call for an improvement in reporting and transparency in presenting results.
CONCLUSION
The literature search resulted in twenty-nine studies published over the last decade which use four mathematical modeling techniques to evaluate waiting time reduction strategies in the hospital ED. Although each modeling technique has strengths and weaknesses, DES was the most frequently used method because of its ability to model complex systems, staff shifts, patient history, and multiple resource constraints, its transparency for decision makers and the wealth of software available for implementation. Scheduling and altering the number of staff according to surges in patient demand showed reductions in ED waiting time. Fast-tracking low acuity patients was also found to be effective in decreasing waiting times, but only at the expense of high acuity patients or decreasing turnaround laboratory times or using point of care testing.
Ultimately, mathematical modeling is a strategy that can be used for the continuous quality improvement and safe delivery of health care without placing patients at risk. It is able to mirror, anticipate, or amplify real situations within a safe environment for healthcare practitioners. There is potential for mathematical models to be used to evaluate the cost-effectiveness of different strategies and new technologies. From the promising results found in the included studies this area of healthcare could greatly benefit from the use of mathematical modeling.
CONTACT INFORMATION
Morgan E. Lim, MA, PhD Candidate, PATH Research Institute, Tim Nye, PhD, Professor, Department of Mechanical Engineering, James M. Bowen, MSc, PharmD, Professor, PATH Research Institute, Jerry Hurley, PhD, Professor, Department of Economics, Ron Goeree, MA, Professor, PATH Research Institute, Jean-Eric Tarride, PhD, Professor, PATH Research Institute, McMaster University, Hamilton, Ontario, Canada
CONFLICTS OF INTEREST
Tim Nye has not declared his conflicts of interest. The other authors report they have no potential conflicts of interest.
Appendix 1: Search Strategies of all electronic databases
COMPENDEX
(((((((({System theory} OR {Decision theory} OR {Systems analysis} OR {Scheduling}) WN CV)) AND (2000-2010 WN YR)) OR
(((({Queueing theory} OR {Operations research} OR {Queueing networks}) WN CV)) AND (2000-2010 WN YR)) OR
(((({Computer simulation} OR {Discrete event simulation} OR {Mathematical models} OR {Simulation}) WN CV)) AND (2000-2010 WN YR)) OR
((((stochastic NEAR/2 model OR process* NEAR/2 model OR theor* NEAR/2 model OR mathematical NEAR/2 model OR computer NEAR/2 model OR emergency NEAR/2 model OR triage NEAR/2 model OR queu* NEAR/2 model OR {patient flow} NEAR/2 model) WN KY)) AND (2000-2010 WN YR)) OR
((((stochastic NEAR/2 simulation OR process* NEAR/2 simulation OR theor* NEAR/2 simulation OR mathematical NEAR/2 simulation OR computer NEAR/2 simulation OR emergency NEAR/2 simulation OR triage NEAR/2 simulation OR queu* NEAR/2 simulation OR {patient flow} NEAR/2 simulation) WN KY)) AND (2000-2010 WN YR)) OR
((((model NEAR/2 simulation) WN KY)) AND (2000-2010 WN YR)) OR
(((({discrete event}) WN KY)) AND (2000-2010 WN YR)) OR
((((queu* NEAR/2 theory OR {patient flow} NEAR/2 theory) WN KY)) AND (2000-2010 WN YR))))) AND
((((((({Emergency rooms}) WN CV)) AND (2000-2010 WN YR)) OR
((($triage) WN KY) AND (2000-2010 WN YR)) OR
((((((Health) OR (Health Care)) WN CV) AND emergency WN KY)) AND (2000-2010 WN YR)) OR
((((emergency NEAR/2 room OR emergency NEAR/2 department OR emergency NEAR/2 ward OR emergency NEAR/2 unit OR emergency NEAR/2 triage) WN KY)) AND (2000-2010 WN YR))))))
INSPEC
((((((((((({digital simulation} OR {simulation} OR {discrete event simulation}) WN CV)) AND (2000-2010 WN YR)) OR (((({queueing theory}) WN CV)) AND (2000-2010 WN YR)) OR
(((({operations research} OR {systems analysis} OR {system theory}) WN CV)) AND (2000-2010 WN YR)) OR
(((({scheduling}) WN CV)) AND (2000-2010 WN YR)) OR
(((((stochastic NEAR/2 model OR process NEAR/2 model OR mathematical NEAR/2 model OR computer NEAR/2 model OR emergency NEAR/2 model OR triage NEAR/2 model OR queueing NEAR/2 model OR {patient NEAR/2 flow} NEAR/2 model) WN KY))) AND (2000-2010 WN YR)) OR
(((((stochastic NEAR/2 simulation OR process NEAR/2 simulation OR mathematical NEAR/2 simulation OR computer NEAR/2 simulation OR emergency NEAR/2 simulation OR triage NEAR/2 simulation OR queueing NEAR/2 simulation OR {patient NEAR/2 flow} NEAR/2 simulation OR dynamic NEAR/2 simulation OR discrete NEAR/2 simulation) WN KY))) AND (2000-2010 WN YR)) OR
((({operations research}) WN KY) AND (2000-2010 WN YR)) OR
((({discrete event}) WN KY) AND (2000-2010 WN YR)) OR
(((((queueing NEAR/2 theory OR {patient NEAR/2 flow} NEAR/2 theory) WN KY))) AND (2000-2010 WN YR)) OR
((((model NEAR/2 simulation) WN KY)) AND (2000-2010 WN YR)) OR
((({queueing networks}) WN KY) AND (2000-2010 WN YR))))) OR
(((({mathematical programming}) WN CV)) AND (2000-2010 WN YR))))) AND
((((((((health care) WN CV) AND (emergency) WN KY)) AND (2000-2010 WN YR)) OR
(((((emergency NEAR/2 room OR emergency NEAR/2 department OR emergency NEAR/2 ward OR emergency NEAR/2 unit OR emergency NEAR/2 triage) WN KY))) AND (2000-2010 WN YR)) OR
((((triage) WN KY)) AND (2000-2010 WN YR))))))
BUSINESS SOURCE COMPLETE
1. ((DE “DECISION theory” or DE “DISCRETE choice models” or DE “MANAGEMENT science” or DE “OPERATIONS research”) OR (DE “SYSTEM theory” or DE “MATHEMATICAL optimization” or DE “PROGRAMMING (Mathematics)” or DE “SIMULATION methods” or DE “QUEUING theory”)) OR (DE “MATHEMATICAL models” or DE “SIMULATION models”)
2. TX stochastic N2 simulation OR process N2 simulation OR mathematical N2 simulation OR computer N2 simulation OR emergency N2 simulation OR triage N2 simulation OR queueing N2 simulation OR patient N2 flow N2 simulation OR dynamic N2 simulation OR discrete N2 simulation
3. TX emergency N2 room OR emergency N2 department OR emergency N2 ward OR emergency N2 unit OR emergency N2 triage OR triage
4. (1 OR 2) AND 3
MEDLINE
1. exp decision theory/ or exp operations research/
2. mathematical computing/ or exp computer simulation/ or exp probability/
3. ((stochastic or process* or theor* or math* or comput* or emergency or triage or queu* or (patient adj2 flow)) adj2 (model* or simulation* or microsimulation*)).ti,ab.
4. (model* adj2 simulation*).ti,ab.
5. ((queu* or patient flow) adj2 theor*).ti,ab.
6. “discrete event”.ti,ab.
7. or/1-6
8. exp Emergency Service, Hospital/
9. (emergency adj2 (department* or ward* or room* or triage)).ti,ab.
10. 8 or 9
11. 7 and 10
EMBASE
1. exp decision theory/ or exp system analysis/
2. process model/ or exp theoretical model/ or exp mathematical model/ or exp simulation/ or exp probability/
3. exp mathematical computing/
4. ((stochastic or process* or theor* or math* or comput* or emergency or triage or queu* or (patient adj2 flow)) adj2 (model* or simulation* or microsimulation*)).ti,ab.
5. (model* adj2 simulation*).ti,ab.
6. ((queu* or patient flow) adj2 theor*).ti,ab.
7. “discrete event”.ti,ab.
8. or/1-7
9. exp Emergency Ward/
10. (emergency adj2 (department* or ward* or room* or triage)).ti,ab.
11. exp Emergency Care/
12. or/9-11
13. 8 and 12
Appendix 2: Data Abstraction Form
Appendix 3: All References
Included References
1.Ahmed, MA, Alkhamis, TM. Simulation optimization for an emergency department healthcare unit in Kuwait. European Journal of Operational Research 2009 November 1;198 (3):936–42.
2.Beck, E, Balasubramanian, H, Henneman, PL. Resource management and process change in a simplified model of the emergency department. Proceedings of the 2009 Winter Simulation Conference (WSC 2009); Univ. of Massachusetts Amherst, Amherst, MA, USA. Piscataway, NJ, USA: IEEE; 2009 p. 1887-95.
3.Brailsford, SC, Lattimer, VA, Tarnaras, P, Turnbull, JC. Emergency and on-demand health care: modelling a large complex system. Journal of the Operational Research Society 2004 January;55 (1):34–42.
4.Coats, TJ, Michalis, S. Mathematical modelling of patient flow through an accident and emergency department. Emergency Medicine Journal 2001;18 (3):190–2.
5.Cochran, JK, Roche, KT. A multi-class queuing network analysis methodology for improving hospital emergency department performance. Computers & Operations Research 2009 May;36 (5):1497–512.
6.Connelly, LG, Bair, AE. Discrete event simulation of emergency department activity: A platform for system-level operations research. Academic Emergency Medicine 2004;11 (11):1177–85.
7.Duguay, C, Chetouane, F. Modeling and improving emergency department systems using discrete event simulation. Simulation 2007 April;83 (4):311–20.
8.Ferrin, D, McBroom, D, Miller, M. Maximizing hospital financial impact and emergency department throughput with simulation. Proceedings of the 2007 Winter Simulation Conference; 2007 p. 1566-73.
9.Gunal, M, Pidd, M. Understanding accident and emergency department performance using simulation. Proceedings for the 2006 Winter Simulation Conference; 2006 p. 446-52.
10.Holm, LB, Dahl, FA. Simulating the effect of physician triage in the emergency department of Akershus University Hospital. Proceedings of the 2009 Winter Simulation Conference (WSC 2009); Helse Sor-Ost Health Service Res. Centre, Akershus Univ. Hosp., Lorenskog, Norway. Piscataway, NJ, USA: IEEE; 2009 p. 1896-905.
11.Hung, GR, Whitehouse, SR, O'Neill, C, Gray, AP, Kissoon, N. Computer modeling of patient flow in a pediatric emergency department using discrete event simulation. Pediatric Emergency Care 2007;23 (1):5–10.
12.Khadem, M, Bashir, H, Al-Lawati, Y, Al-Azri, F. Evaluating the layout of the emergency department of a public hospital using computer simulation modeling: A case study. Proceedings of the 2008 IEEE IEEM; 2010 p. 1709–13.
13.Khare, RK, Powell, ES, Reinhardt, G, Lucenti, M. Adding more beds to the emergency department or reducing admitted patient boarding times: which has a more significant influence on emergency department congestion? Annals of Emergency Medicine 2009;53 (5):575–85.
14.Kolb, E, Peck, J. Reducing emergency department overcrowding - Five patient buffer concepts in comparison. Proceedings of the 2008 Winter Simulation Conference; 2008 p. 1516-25.
15.Komashie, A, Mousavi, A. Modeling emergency departmens using discrete event simulation techniques. Proceedings from the 2005 Winter Simulation Conference; 2005 p. 2681-5.
16.Lane, DC, Monefeldt, C, Rosenhead, JV. Looking in the wrong place for healthcare improvements: A system dynamics study of an accident and emergency department. Journal of the Operational Research Society 2000;51 (5):518–31.
17.Laskowski, M, McLeod, RD, Friesen, MR, Podaima, BW, Alfa, AS. Models of emergency departments for reducing patient waiting times. PLoS ONE [Electronic Resource] 2009;4 (7):e6127.
18.Mahapatra, S, Koellig, C, Patvivatsiri, L, Fraticelli, B, Eitel, D et al. Pairing emergency severity index level triage data with computer aided system design to improve emergency department access and throughput. Proceedings of the 2003 Winter Simulation Conference; 2003 p. 1917-25.
19.Mayhew, L, Smith, D. Using queuing theory to analyse the government's 4-H completion time target in accident and emergency departments. Health Care Management Science 2008;11 (1):11–21.
20.Medeiros, D, Swenson, E, DeFlitch, C. Improving patient flow in a hospital emergency department. Proceedings of the 2008 Winter Simulation Conference; 2008 p. 1526-31.
21.Meng, L-Y, Spedding, T. Modeling patient arrivals when simulating an accident and emergency unit. Proceedings of the 2008 Winter Simulation Conference; 2008 p. 1509-15.
22.Nielsen, AL, Hilwig, H, Kissoon, N, Teelucksingh, S. Discrete event simulation as a tool in optimization of a professional complex adaptive system. Studies in Health Technology & Informatics 2008;136:247–52.
23.Puente, J, Gomez, A, Parreno J, de la FD. Applying a fuzzy logic methodology to waiting list management at a hospital emergency unit: A case study. International Journal of Healthcare Technology and Management 2003;5 (6):432–42.
24.Ruohonen, T, Neittanmaki, P, Teittinen, J. Simulation model for improving the operation of the emergency department of special healthcare. Proceedings of the 2006 Winter Simulation Conference; 2006 p. 453-8.
25.Storrow, AB, Zhou, C, Gaddis, G, Han, JH, Miller, K et al. Decreasing lab turnaround time improves emergency department throughput and decreases emergency medical services diversion: A simulation model. Academic Emergency Medicine 2008;15 (11):1130–5.
26.Takakuwa, S, Shiozaki, H. Functional Analysis for Operating Emergency Department of a General Hospital. 2004 p. 2003-11.
27.Tao, W, Guinet, A, Belaidi, A, Besombes, B. Modelling and simulation of emergency services with ARIS and Arena. Case study: the emergency department of Saint Joseph and Saint Luc Hospital. Production Planning and Control 2009 September;20 (6):484–95.
28.Wang, L. An Agent-Based Simulation for Workflow in the Emergency Department. Proceedings of the 2009 IEEE Systems and Information Engineering Design Symposium; 2009 Apr 24; 2009 p. 19-23.
29.Yeh, J-Y, Lin, WS. Using simulation technique and genetic algorithm to improve the quality care of a hospital emergency department. Expert Systems with Applications 2007 May;32 (4):1073–83.
Excluded References
1.Ahmed, S. Accident and emergency section simulation in hospital. WSEAS Transactions on Computers 2003 January;2 (1):91–5.
2.Alvarez, AM, Centeno, MA. Simulation-based decision support for emergency rooms systems. International Journal of Healthcare Technology and Management 2000;2 (5-6):523–38.
3.Asplin, BR, Flottemesch, TJ, Gordon, BD. Developing Models for Patient Flow and Daily Surge Capacity Research. Academic Emergency Medicine 2006;13 (11):1109–13.
4.Bard, JF, Purnomo, HW. Short-Term Nurse Scheduling in Response to Daily Fluctuations in Supply and Demand. Health Care Management Science 2005 November;8 (4):315–24.
5.Beaulieu, H, Ferland, JA, Gendron, B, Michelon, P. A mathematical programming approach for scheduling physicians in the emergency room. Health Care Management Science 2000;3 (3):193–200.
6.Behr, JG, Diaz, R. A System Dynamics Approach to Modeling the Sensitivity of Inappropriate Emergency Department Utilization. Advances in Social Computing. Proceedings Third International Conference on Social Computing, Behavioral Modeling, and Prediction, SBP 2010; Virginia Modeling, Anal. Simulation Center,VMASC, Old Dominion Univ., Suffolk, VA, United States. Berlin, Germany: Springer Verlag; 2010 p. 52-61.
7.Carter, MW, Lapierre, SD. Scheduling Emergency Room Physicians. Health Care Management Science 2001 December;4 (4):347–60.
8.Ceglowski, R, Churilov, L, Wasserthiel, J. Combining data mining and discrete event simulation for a value-added view of a hospital emergency department. Journal of the Operational Research Society 2007 February;58 (2):246–54.
9.Chauvet, J, Gourgand, M, Rodier, S. Methodological approach and decision-making aid tool for the hospital systems: Application to an emergency department. Modelling and Simulation 2009. The European Simulation and Modelling Conference 2009. ESM 2009; Blaise Pascal Univ., Aubiere, France. Ostend, Belgium: EUROSIS-ETI Publications; 2009 p. 197-204.
10.Cooke, MW, Arora, P, Mason, S. Discharge from triage: Modelling the potential in different types of emergency department. Emergency Medicine Journal 2003;20 (2):131–3.
11.Costa, AX, Ridley, SA, Shahani, AK, Harper, PR, De S, V et al. Mathematical modelling and simulation for planning critical care capacity. Anaesthesia 2003;58 (4):320–7.
12.Daknou, A, Zgaya, H, Hammadi, S, Hubert, H. A dynamic patient scheduling at the emergency department in hospitals. 2010 IEEE Workshop on Health Care Management (WHCM); LAGIS UMR 8146, Ecole Centrale de Lille, Lille, France. Piscataway, NJ, USA: IEEE; 2010 p. 6.
13.De Bruin, AM, Koole, GM, Visser, MC. Bottleneck analysis of emergency cardiac in-patient flow in a university setting: An application of queueing theory. Clinical and Investigative Medicine 2005;28 (6):316–7.
14.De Bruin, AM, van Rossum, AC, Visser, MC, Koole, GM. Modeling the emergency cardiac in-patient flow: an application of queuing theory. Health Care Management Science 2007;10 (2):125–37.
15.Di Mascio, R. Service process control: conceptualising a service as a feedback control system. Journal of Process Control 2002 March;12 (2):221–32.
16.DiDomenico, PB, Pietzsch, JB, Pate-Cornell, ME. Bayesian assessment of overtriage and undertriage at a level I trauma centre. Philosophical Transactions of the Royal Society London, Series a (Mathematical, Physical & Engineering Sciences) 2008;366 (1874):2265–77.
17.Dronzek. Improving critical care. IIE Solutions 2001 November;33 (11):42.
18.Fabio, F, Alfredo, L, Daithi, F. Computer simulation and swarm intelligence organisation into an emergency department: A balancing approach across ant colony optimisation. International Journal of Services Operations and Informatics 2008;3 (2):142–61.
19.Facchin, P, Rizzato, E, Romanin-Jacur, G. Emergency department generalized flexible simulation model. 2010 IEEE Workshop on Health Care Management (WHCM); Dept. of Paediatrics, Univ. of Padova, Padova, Italy. Piscataway, NJ, USA: IEEE; 2010 p. 6.
20.Farinha, R, Oliveira, MD, De Sa, AB. Networks of primary and secondary care services: How to organise services so as to promote efficiency and quality in access while reducing costs. Quality in Primary Care 2008;16 (4):249–58.
21.Fletcher, A, Worthington, D. What is a ‘generic’ hospital model?–a comparison of ‘generic’ and ‘specific’ hospital models of emergency patient flows. Health Care Management Science 2009;12 (4):374–91.
22.Glick, ND, Blackmore, CC, Zelman, WN. Extending simulation modeling to activity-based costing for clinical procedures. Journal of Medical Systems 2000;24 (2):77–89.
23.Green, LV, Soares, J, Giglio, JF, Green, RA. Using queueing theory to increase the effectiveness of emergency department provider staffing. Academic Emergency Medicine 2006;13 (1):61–8.
24.Green, M, Bjork, J, Forberg, J, Ekelund, U, Edenbrandt, L et al. Comparison between neural networks and multiple logistic regression to predict acute coronary syndrome in the emergency room. Artificial Intelligence in Medicine 2006;38 (3):305–18.
25.Green, M, Ekelund, U, Edenbrandt, L, Bjork, J, Forberg, JL et al. Exploring new possibilities for case-based explanation of artificial neural network ensembles. Neural Networks 2009;22 (1):75–81.
26.Gunal, MM, Pidd, M. Understanding target-driven action in emergency department performance using simulation. Emergency Medicine Journal 2009;26 (10):724–7.
27.Hoot, N, Aronsky, D. An early warning system for overcrowding in the emergency department. AMIA Annu Symp Proc 2006;339–43.
28.Hoot, NR, LeBlanc, LJ, Jones, I, Levin, SR, Zhou, C et al. Forecasting Emergency Department Crowding: A Discrete Event Simulation. Annals of Emergency Medicine 2008;52 (2):116–25.
29.Jung, H, Cohen, R. A model for reasoning about interaction with users in dynamic, time critical environments for the application of hospital decision making. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Cheriton School of Computer Science, University of Waterloo, Canada. Ottawa, ON, Canada: Springer Verlag; 2010 p. 319-23.
30.Kolker, A. Process modeling of emergency department patient flow: Effect of patient length of stay on ED diversion. Journal of Medical Systems 2008;32 (5):389–401.
31.Lamiri, M, Xiaolan, X, Dolgui, A, Grimaud, F. A stochastic model for operating room planning with elective and emergency demand for surgery. European Journal of Operational Research 2008 March 16;185 (3):1026–37.
32.Lamiri, M, Xie, X, Zhang, S. Column generation approach to operating theater planning with elective and emergency patients. IIE Transactions (Institute of Industrial Engineers) 2008;40 (9):838–52.
33.Lattimer, V, Brailsford, S, Turnbull, J, Tarnaras, P, Smith, H et al. Reviewing emergency care systems I: Insights from system dynamics modelling. Emergency Medicine Journal 2004;21 (6):685–91.
34.Levin, SR, Dittus, R, Aronsky, D, Weinger, MB, Han, J et al. Optimizing cardiology capacity to reduce emergency department boarding: A systems engineering approach. American Heart Journal 2008;156 (6):1202–9.
35.Litvak, N, van Rijsbergen, M, Boucherie, RJ, van Houdenhoven, M. Managing the overflow of intensive care patients. European Journal of Operational Research 2008;185 (3):998–1010.
36.Lucas, CE, Buechter, KJ, Coscia, RL, Hurst, JM, Meredith, JW et al. Mathematical modeling to define optimum operating room staffing needs for trauma centers. Journal of the American College of Surgeons 2001;192 (5):559–65.
37.Malakooti, B, Malakooti, NR, Ziyong, Y. Integrated group technology, cell formation, process planning, and production planning with application to the emergency room. International Journal of Production Research 2004 May 1;42 (9):1769–86.
38.Miro, O, Sanchez, M, Espinosa, G, Coll-Vinent, B, Bragulat, E et al. Analysis of patient flow in the emergency department and the effect of an extensive reorganisation. Emergency Medicine Journal 2003;20 (2):143–8.
39.Raikundalia, GK, Bain, CA, Mehta, S. Towards an advanced computing solution for hospital management using discrete event simulation. International Review on Computers and Software 2009;4 (1):15–25.
40.Raunak, M, Osterweil, L, Wise, A, Clarke, L, Henneman, P. Simulating patient flow through an emergency department using process-driven discrete event simulation. 2009 ICSE Workshop on Software Engineering in Health Care (SEHC 2009); Univ. of Massachusetts, Amherst, MA, USA. Piscataway, NJ, USA: IEEE; 2009 p. 73-83.
41.Sampath, R, Darabi, H, Buy, U, Liu, J. Control reconfiguration of discrete event systems with dynamic control specifications. IEEE Transactions on Automation Science and Engineering 2008;5 (1):84–100.
42.Sinreich, D, Marmor, Y. Emergency department operations: The basis for developing a simulation tool. IIE Transactions 2005 March;37 (3):233–45.
43.Spath, P. Before implementing changes . . . simulate! Hospital Peer Review 2003;28 (7):101–4.
44.Srijariya, W, Riewpaiboon, A, Chaikledkaew, U. System dynamic modeling: An alternative method for budgeting. Value in Health 2008;11 (Suppl 1):S115–S123.
45.Stainsby, H, Taboada, M, Luque, E. Towards an agent-based simulation of hospital emergency departments. 2009 IEEE International Conference on Services Computing (SCC); Comput. Archit. Oper. Syst. Dept. (CAOS), Univ. Autonoma de Barcelona, Barcelona, Spain. Piscataway, NJ, USA: IEEE; 2009 p. 536-9.
46.Topaloglu, S. A multi-objective programming model for scheduling emergency medicine residents. Computers and Industrial Engineering 2006;51 (3):375–88.
47.Van Oostrum, JM, Van, HM, Vrielink, MM, Klein, J, Hans, EW et al. A simulation model for determining the optimal size of emergency teams on call in the operating room at night. Anesthesia & Analgesia 2008;107 (5):1655–62.
48.Vinson, DR, Berman, DA, Patel, PB, Hickey, DO. Outpatient management of deep venous thrombosis: 2 Models of integrated care. American Journal of Managed Care 2006;12 (7):405–10.
49.Walczak, S. Artificial neural network medical decision support tool: Predicting transfusion requirements of ER patients. IEEE Transactions on Information Technology in Biomedicine 2005;9 (3):468–74.
50.Windmeijer, F, Gravelle, H, Hoonhout, P. Waiting lists, waiting times and admissions: An empirical analysis at hospital and general practice level. Health Economics 2005;14 (9):971–85.
51.Wullink, G, Van, HM, Hans, EW, Van Oostrum, JM, Van Der, LM et al. Closing emergency operating rooms improves efficiency. Journal of Medical Systems 2007;31 (6):543–6.
52.Yang, CC, Lin, WT, Chen, HM, Shi, YH. Improving scheduling of emergency physicians using data mining analysis. Expert Systems with Applications 2009;36 (2 PART 2):3378–87.
53.Zhong, M, Shi, C, Fu, T, He, L, Shi, J. Study in performance analysis of China Urban Emergency Response System based on Petri net. Safety Science 2010;48 (6):755–62.
54.Zilm, F. Estimating emergency service treatment bed needs. Journal of Ambulatory Care Management 2004;27 (3):215–23.
