Nomenclature
- UAVs
-
unmanned aerial vehicles
- TAN-BN
-
tree augmented naive Bayes network
- BN
-
Bayesian network
- AAIB
-
Aviation Accident Investigation Bureau
- NTSB
-
National Transportation Safety Board
- ASID
-
aviation safety incident database
- FAA
-
Federal Aviation Administration
- JARUS
-
Joint Authorities for Rulemaking of Unmanned Systems
- SORA
-
Specific Operations Risk Assessment
- CPT
-
conditional probability table
- MSE
-
mean squared error
- TP
-
true positive
- FP
-
false positive
- TN
-
true negative
- FN
-
false negative
- LOC
-
loss of control (in-flight)
- DAA
-
detect and avoid
- UAS
-
unmanned aerial system
- ISAM
-
integrated safety assessment model
- CNS/ATM
-
communication, navigation, surveillance/air traffic management
Symbols
-
${R_i}$
-
${i_{th}}$
risk factor
-
${R_{\rm{j}}}$
-
${{\rm{j}}_{th}}$
risk factor
-
${r_{ii}}$
-
${i_{th}}$
state of the
${i_{th}}$
risk factor
${R_i}$
-
${c_t}$
-
probability of UAV accident/incident
-
$T(a)$
-
number of accidents
-
${T_{fh}}$
-
total flight time of drones (in hours)
-
${T_{fgh}}$
-
total flight hours (consistent with
${T_{fh}}$
, used for calculating prior probability of risk factors) -
$\omega $
-
risk factor
-
${c_i}$
-
${i_{th}}$
specific risk factor (e.g. brake failure, adverse weather)
-
$\alpha $
,
$\beta $
-
shape parameters of Beta distribution, used to fit conditional probabilities of multi-factor risks
-
$\Gamma $
-
Gamma function
-
$B( {\alpha ,\beta })$
-
Beta function (normalisation factor of Beta distribution), with parameters
$\alpha $
and
$\beta $
-
$\lambda $
-
key parameter representing the contribution ratio of each risk factor to the accident/incident probability
-
${I_p}$
-
conditional mutual information between risk factors, used to assess the dependency between two risk factors
1.0 Introduction
With the increasing application of unmanned aerial vehicles (UAVs) in airspace, UAVs have not only improved the efficiency of transportation and entertainment activities but also posed threats to airspace safety. Since 2017, UAV incidents/accidents have been reported worldwide, and some accidents have even resulted in severe consequences [Reference Sun, Hu, Qin and Zhang1]. Hence, it is urgent to analyse the mechanism of UAV accidents and propose targeted countermeasures.
Recently, UAV-related accidents/incidents have been seen near civil aviation airports [2]. These UAVs can dangerously approach manned aircraft during takeoff or landing, potentially leading to collisions between UAVs and manned aircraft. Even when operating below normal cruising speeds, small UAVs still pose a high risk of damaging manned aircraft windshields [Reference Pyrgies3]. Additionally, UAVs are sometimes affected by failures of critical UAV systems, such as the flight control system, power system or airframe. These failures may result in loss of control of the UAV and collisions with terrain, which also pose threats to ground personnel and devices, especially in locations near the airport. Therefore, risk assessment for UAV-related accidents/incidents is necessary.
Currently, risk assessment methods for aviation safety are categorised into qualitative [Reference Oh and Yoon4, Reference Millar, Hashemi, Mahmoodi, Meyer and Laliberte5] and quantitative approaches [Reference Fan, Blanco-Davis, Yang, Zhang and Yan6]. The Specific Operations Risk Assessment (SORA) method [7, Reference Nikodem, Dittrich and Bierig8], proposed by Joint Authorities for Rulemaking of Unmanned Systems (JARUS), provides a qualitative risk assessment process for UAV operations. It conducts initial risk assessments and provides relevant operational units with safety assurances. Due to its convenience and straightforward process, SORA has been widely used in different safety management programmes for UAV operation. However, the SORA approach still has limitations. One of the most obvious disadvantages is that it cannot uncover the inherent relations among risk factors and accidents/incidents, consequently hindering the development of corresponding countermeasures and strategies.
An alternative approach is to use a structural causal model, such as a Bayesian network, to figure out the potential relationships between risk factors [Reference Allouch, Koubâa, Khalgui and Abbes9]. The Bayesian network approach is a widely used method for quantitative risk assessment [Reference Allouch, Koubâa, Khalgui and Abbes9, Reference Harkleroad, Vela, Kuchar, Barnett and Merchant-Bennett10]. It can identify and estimate the probabilities of risks and the paths of risk propagation under various circumstances, which is quite popular in analysing risks of aviation accidents. For instance, Harkleroad et al. [Reference Harkleroad, Vela, Kuchar, Barnett and Merchant-Bennett10] and Luxhoj [Reference Luxhøj11] used Bayesian network methods for evaluating aviation risks; Lu and Meng [Reference Lu and Meng12] employed Bayesian networks to analyse the dependencies between risk factors contributing to aviation accidents and point out the pathways of risk propagation. Bauranov and Rakas [Reference Bauranov and Rakas13] analysed aviation incident risks using a Bayesian network integrated in the Integrated Safety Assessment Model (ISAM) framework. Han et al. [Reference Han, Yang, Zhao, Guan and Wang14] used a Bayesian network model to study accident risks associated with drones in cargo scenarios.
The construction of the Bayesian network relies largely on structures made up of relationships between risk factors. Many studies use expert ratings and fuzzy mathematical methods to determine the relationships between risk factors and then develop the Bayesian network [Reference Jones and Reveley15–Reference Basavaraj, Rangan and Rajgopal19]. However, they still have difficulties in eliminating the biases led by subjective decisions, as this empirical knowledge from expert opinions may not always be correct. If the subjective differences are obvious, the proposed Bayesian networks will be different as well, leading to inconsistent assessments. Furthermore, it is hard to quantitatively evaluate which structure of the Bayesian network is better, making it difficult to accurately assess the true effects of risk factors on accidents.
Compared with analysis in traditional aviation safety, the Bayesian network is rarely used in modeling UAV accidents/incidents [Reference Xiao, Li, Luo and Liu20]. It would be constructive to quantify the relationships between environmental/technological/human factors and these risk events to unveil the mechanism of UAV accidents/incidents near the airport. To fill this gap, this paper establishes an advanced Bayesian network using UAV accident/incident reports collected near the airport. Potential risk factors are extracted from the reports to build the Bayesian network [Reference Allouch, Koubâa, Khalgui and Abbes9]. Based on the analysis results, several technological risk mitigation measures are proposed and assessed. Also, the proposed Bayesian network method can serve as both a prediction method [Reference Lu and Meng12] for identifying the consequences of a UAV risk event (accident or incident) and an explanation tool for evaluating the safety of UAV operations.
2.0 Data manipulation
2.1 Data collection
UAV accident reports exploited in this study are collected from different sources worldwide, including those from the Aviation Accident Investigation Bureau (AAIB), the National Transportation Safety Board (NTSB), the Aviation Safety Incident Database (ASID), and the drone accident database of the Aviation Safety Network (ASN). The reports include both collisions with other objects and single-UAV accidents, as well as many near-miss or hazardous approach incidents involving UAVs and manned aircraft/other objects. The reports cover a range of drone types, including consumer models, professional-grade rotary drones, large military reconnaissance drones and small UAVs. Nearly all the collected UAV accident or incident samples are observed near the airport. To ensure the data quality, the original reports are initially filtered according to two criteria: their proximity to airport terminal areas and the integrity and clarity of the reports’ information. Following the filtering process, we gathered a total of 2,130 drone accident and incident investigation reports from the data sources above. The reports contain information on the date and location of the accident/incident, information on the drones, flight process and narratives of the events.
2.2 Risk variable identification
To systematically extract risk factors from the textual reports, a semi-automatic text mining procedure was adopted. First, a bag-of-words model was applied to automatically extract candidate keywords from the textual narratives of the 2,130 accident and incident reports. Second, through manual analysis based on the knowledge in aviation safety, the author team merged synonyms, removed irrelevant terms and refined the keyword set. Finally, the remaining keywords were treated as potential risk factors contributing to UAV accidents near airports.
The airport terminal area includes runways, taxiways, approach and departure paths and the surrounding airspace [Reference Wiegmann and Shappell17]. Most UAVs recorded in the accident/incident reports are used for commercial, recreational and industrial purposes [Reference Baraldi, Podofillini, Mkrtchyan, Zio and Dang18]. To analyse the causes of UAV encounters with manned aircraft/objects in terminal area scenarios, potential risk factors are sorted as UAV system, operational environment and human factors [Reference Basavaraj, Rangan and Rajgopal19, Reference Boyd21]. Table 1 shows the detailed descriptions of these risk factors and their prior probabilities.
-
(1) UAV system factors: The possible risk factors of UAV systems include electromagnetic interference, remote communication failure, flight control system failure, brake failure and loss of electrical power. The failures of these critical UAV systems may result in loss of control in-flight, which causes risky encounters near manned aircraft and potential collisions [Reference Clothier, Walker, Fulton and Campbell22, Reference Ghasri and Maghreb23].
-
(2) Operational environment factors: The possible risk factors of operational environment factors include adverse weather (low-visibility, winds, precipitation, etc.), bird strike, other aircraft approaching and obstacles. Adverse weather is one of the most common risk factors increasing the risk of UAV operation, as this factor can impair the stability of both drones and manned aircraft [Reference Shao24]. Bird strikes, other aircraft approaches and obstacles can be viewed as in-flight conflicts between UAVs and other objects, which may cause collisions if the conflicts cannot be tackled [Reference Oztekin and Luxhøj25].
-
(3) Human factors: The possible risk factors of human factors include pilot error, violations, and operating skill level. Pilot errors refer to the misjudgement, distraction or inadequate situational awareness that can lead to the inability to predict and avoid potential encounters between drones and manned aircraft/other objects [Reference Neff and Garman26]. Violation refers to a subjective decision of the manipulator that invades the restricted airspace using drones [Reference Vierhauser, Islam, Agrawal, Cleland-Huang and Mason27]. The lack of skill level means the drone manipulator has insufficient awareness and skills to safely operate the drones, potentially leading to instability of flight and biased trajectories that may trigger a risk approach with manned aircraft/other objects.
Risk classification near the airport terminal area, detailed description and priori probability table for each accident.
Table 1 Long description
A table categorizing risk factors for UAV encounters with manned aircraft or objects in terminal area scenarios, detailing descriptions and prior probabilities. The table has four columns: Factors classification, Factor, Factor description, and Prior probability. It includes three main categories of risk factors: Environmental factors, UAS system factors, and Human factors. Each category lists specific factors with their descriptions and prior probabilities. Environmental factors include Adverse weather, Bird strike, Other aircraft approaching, and Obstacles. UAS system factors include Electromagnetic interference, Remote communication failure, Flight control system failure, Brake failure, and Loss of electrical power. Human factors include Pilot error, Violations, and Operating skill level. Each row provides the factor description and its prior probability.
3.0 Methods
3.1 Method framework
Figure 1 outlines the framework of methods used in this study, including hazard identification, tree-augmented naive Bayes network (TAN-BN) structure learning, risk evaluation and evaluation of risk mitigation measures. The details of the four processes are explained as follows:
-
(1) Hazard identification: The hazard identification involves analysing accident reports by aviation safety experts to extract potential operational information and hazards. This process is done in Section 2.2.
-
(2) TAN-BN structure learning: To identify the risk factors that contribute to drone accidents near the airport and to understand the mechanism, this study employs a TAN-BN algorithm to unveil the structural causal relations between risk factors and accidents. This approach involves applying the tree algorithm for structural learning and uses learning algorithms for parameter estimation.
-
(3) Risk evaluation: Based on the TAN-BN model, sensitivity analysis and diagnostic checks of the network model are applied to explain the risk factors and measure their potential impact on overall safety. The model’s performance in accident prediction is examined by model metrics.
-
(4) Evaluation of risk mitigation measures: Finally, based on the analysis of risk factors, this study proposes different types of risk mitigation measures and tests their effects on safety. Additionally, a real-world case with UAV accident/incident data is used for validating the proposed analytic model to show its generalisability.
The method framework.
3.2 TAN-BN and structure learning
To provide a comprehensive analysis and unveil the complicated structural relationships between risk factors, this study proposes a quantitative risk analysis method based on TAN-BN [Reference Jun and Kim28]. Typically, a BN model consists of target nodes, observable nodes and intermediate nodes [Reference Kwag, Gupta and Dinh29]. The target node represents the probability distribution of the variable, i.e. the probability of a drone accident. Other nodes are used to form a structure that provides causal relations among the factors.
The structure of the BN model is the key to unveiling the relationships of risk factors. In this study, we apply the tree-based augmentation algorithm to learn and create the Bayesian network structure. The structure is dependent on nodes and edges that link the risk factors. Then, the model is updated by a parameter learning algorithm to establish the individual conditional probabilities of nodes and represent the edges. The task of learning the structure of TAN-BN is treated as an optimisation problem. Throughout this process, we use conditional mutual information (as illustrated in Equation (1)) to assess the dependency between two risk factors. The mutual information of each pair of factors is used as weights for the arcs in order to maximise the overall sum of all arcs in forming the network.
\begin{align}{I_p}( {{R_i},{R_j}{\rm{|}}C}) = \mathop \sum \nolimits_{{r_{ii}},{r_{ji}},{c_t}} P( {{r_{ii}},{r_{ji}},{c_t}}){{\log }_b}\frac{{P( {{r_{ii}},{r_{ji}}{\rm{|}}{c_t}})}}{{P( {{r_{ii}}{\rm{|}}{c_t}})P( {{r_{ji}}{\rm{|}}{c_t}})}} \end{align}
where
${I_p}$
defines the conditional mutual information between any risk factor
${R_i}$
and
${R_j}$
in the structure,
${r_{ii}}$
represents the
${i^{{\rm{th}}}}$
state of risk variable
${i_i}$
,
${r_{ji}}$
denotes the
${i^{{\rm{th}}}}$
state of risk factor
${r_j}$
,
${c_t}$
indicates the probability of UAV accident/incident, with b typically set to 2.
The TAN learning process is employed subsequently to construct the BN structure using the mutual information gained in Equation (1). Consequently, it establishes the relationships between risk factors and their connections to the occurrence of UAV accidents/incidents. Subsequently, parameter-learning algorithms are used to learn the parameters for each node in the model, allowing the computation of conditional probability tables (CPT) for each node. The mathematical measure of conditional probability is presented in Section 3.4.
3.3 Estimation of prior probabilities
The analysis requires prior probabilities for both UAV accident/incident and risk factors. Since each drone’s total operation time/frequency is not practically collected, obtaining the exact probabilities of UAV accidents/incidents is not possible. Therefore, we estimate the prior probabilities of the nodes based on the frequency of their parent node in the dataset. Also, approximated total flight hours derived from drone pilot registration are considered to be the exposure to ensure theoretical validity. The risk factors used for estimating the nodes’ prior probabilities include the number of registered pilots, flight years and average annual flight hours per pilot.
-
(1) Prior probability of UAV accident
According to the findings of the Federal Aviation Administration (FAA), registered drone pilots fly an average of 15 times per month, with each flight lasting an average of 20 minutes. Based on this, the annual flight time per pilot is 3,600 minutes (12 months × 15 flights/month × 20 minutes/flight). According to the FAA’s survey, there were 375,226 registered commercial drone pilots in 2024. Thus, the total flight time for registered pilots is approximately 22,513,560 hours (1.351×109 minutes). The statistics also indicate that the recorded UAV accidents during the observation period are 690. Therefore, the accident rate for drones is 3 × 10−5 per flight hour, which corresponds to three accidents per 100,000 flight hours (Equation (2)),
\begin{align}P(a) = \frac{{T(a)}}{{{T_{fh}}}} = \frac{{690}}{{2.251 \times {{10}^7}}} \approx 3 \times {{10}^{ - 5}} \end{align}
where T(a) is the number of accidents, and
${T_{fh}}$
is the total flight time of drones (in hours).
-
(2) Prior probability of risk factor
The prior probabilities of each risk factor in the accidents, represented by the frequency of each parent node, are estimated using the total flight hours and the total flight time that includes specific risk factors. The risk factors’ prior probabilities are calculated by
\begin{align}P(e) = \frac{{T(e)}}{{{T_{fgh}}}} \end{align}
where
${T_{fh}}$
and T(e) represent the total flight time and the number of occurrences of the event, respectively. The estimated prior probability values can be seen in Table 1.
3.4 Estimation of the conditional probability table
In addition to prior probabilities, this study computes the conditional probability [Reference Zhang and Mahadevan30] of different risk factors, which is the probability of a risk factor given the occurrence of another factor. It is defined as in Equation (4),
\begin{align}P( {{\rm{\omega |}}{c_1},{c_2}, \ldots ,{c_m}}) = \frac{{P( {{\rm{\omega }},{c_1},{c_2}, \ldots ,{c_m}})}}{{P( {{c_1},{c_2}, \ldots ,{c_m}})}} \end{align}
where
$P( {{\rm{\omega |}}{c_1},{c_2}, \ldots ,{c_m}})$
represents the conditional probability of different types of UAV accident, risk factor
${\rm{\omega }}$
given the occurrence of
${c_1},{c_2}, \ldots ,{c_m}$
,
$\;P\left( {{\rm{\omega }}|{c_1},{c_2}, \ldots ,{c_m}} \right)$
represents the joint probability of the factor
${\rm{\omega \;\;and}}$
$P( {{c_1},{c_2}, \ldots ,{c_m}})$
represents the probability of
${c_1},{c_2}, \ldots ,{c_m}$
occurring simultaneously.
A common challenge in constructing Bayesian networks is to derive conditional probabilities from limited data and generate CPT, as the data cannot cover all different types of risk factors or the probabilities of all accidents. To address this issue, we derive the conditional probabilities of multi-factor influences using the ratio of the joint probability of risk factors and the single risk factors [Reference Zhang and Mahadevan30], given by
\begin{align}P( {{\rm{\omega }}|{c_i}}) = \frac{{P( {{\rm{\omega }},{c_i}})}}{{P( {{c_i}})}} \end{align}
where,
$P\left( {{\rm{\omega }},{c_i}} \right)$
represents the joint probability of risk event
${\rm{\omega }}$
and risk factor
${c_i}$
occurring together, while
$P\left( {{c_i}} \right)$
represents the probability of factor
${c_i}$
occurring. Both
$P\left( {{\rm{\omega }},{c_i}} \right)$
and
$P\left( {{c_i}} \right)$
can be estimated from the data in the UAV accident/incident database.
For example, considering the risk factor of loss of control (LOC) leading to a UAV accident, it may be caused by various factors such as brake system failure, electromagnetic interference, flight control system failure and remote communication failure. Using data from the collected UAV accident database, we can derive the following conditional probabilities for different types of accidents/incidents.
\begin{align}P( {{\rm{\omega }} = {\rm{Loc}} | c = {\rm{brake\;failure}}}) = \frac{3}{{677}} = 4.43 \times {{10}^{ - 3}} \\[-28pt] \nonumber \end{align}
\begin{align}P( {{\rm{\omega }} = {\rm{Loc}} | c = {\rm{electro\;magnetic\;interference}}}) = \frac{1}{{677}} = 1.48 \times {{10}^{ - 3}} \\[-28pt] \nonumber \end{align}
\begin{align} P( {{\rm{\omega }} = {\rm{Loc}} | c = {\rm{flight\;control\;system\;failure}}}) = \frac{{80}}{{677}} = 11.8 \times {{10}^{ - 2}} \\[-28pt] \nonumber \end{align}
\begin{align} P( {{\rm{\omega }} = {\rm{Loc}} | c = {\rm{remote\;communication\;failure}}}) = \frac{{28}}{{677}} = 4.13 \times {{10}^{ - 2}} \\[-28pt] \nonumber\end{align}
\begin{align} P( {{\rm{\omega }} = {\rm{Loc}} | c = {\rm{loss\;of\;electrical\;power}}}) = \frac{7}{{677}} = 1.03 \times {{10}^{ - 2}} \\[-28pt] \nonumber \end{align}
\begin{align} P( {{\rm{\omega }} = {\rm{Loc}}| c = {\rm{adverse\;weather}}}) = \frac{4}{{677}} = 5.91 \times {{10}^{ - 3}} \\[-28pt] \nonumber \end{align}
\begin{align} P( {{\rm{\omega }} = {\rm{Loc}} | c = {\rm{pilot\;error}}}) = \frac{{23}}{{677}} = 3.39 \times {{10}^{ - 2}} \\[0pt] \nonumber \end{align}
If
$\omega $
is conditioned on multiple factors, each type of accident contributes to the occurrence of
$\omega $
to varying degrees, assuming that risk factors
${c_1},{c_2}, \ldots ,{c_m}$
are independent. Therefore,
$\omega $
requires a function
$f$
to support the estimation of multi-dimensional conditional probability. The development of the function
$f$
should include the following considerations:
-
(1) Monotonicity: The function should be monotonically increasing when the number of risk-event conditioning
$\omega $
increases. -
(2) Contribution of a single accident to the occurrence of
${\rm{\omega }}$
:
$P\left( {{\rm{\omega }},{c_i}} \right)$
should be included in the function. -
(3) Generalisability: The function should be extendable to several special cases.
The condition function used in this study is the Beta distribution. It is highly versatile and can be used to model variables ranging from 0 to 1. In this paper, we use the cumulative distribution function (CDF) of the Beta distribution as a nonlinear function (Equation (13)) to help estimate the conditional probability of ω given multiple risk factors. The CDF of the Beta distribution with two parameters α and β is expressed as follows:
\begin{align}f( {x;\,{\rm{\alpha }},{\rm{\beta }}}) = \frac{{{x^{{\rm{\alpha }} - 1}}{{( {1 - x})}^{{\rm{\beta }} - 1}}}}{{B( {{\rm{\alpha }},{\rm{\beta }}})}} \end{align}
where
$B\left( {{\rm{\alpha }},{\rm{\beta }}} \right)$
is the Beta function, which can be expressed as
$B\left( {\alpha ,\beta } \right) = \frac{{\Gamma \left( \alpha \right)\Gamma \left( \beta \right)}}{{\Gamma \left( {\alpha + \beta } \right)}}$
,
${\rm{\Gamma }}$
is the Gamma function.
For some risk events that may be caused by multiple factors, e.g. LOC, we apply the CDF of the Beta distribution to calculate the conditional probability of a specific type of risk event with multiple risk factors, which is expressed as
\begin{align}P( {{\rm{\omega |}}{c_1},{c_2}, \ldots ,{c_n}} ) = \frac{{\mathop \smallint \nolimits_0^{{{\rm{\lambda }}_i}} {{( {{{\rm{\lambda }}_i}} )}^{{\rm{\alpha }} - 1}}{{( {1 - {{\rm{\lambda }}_i}} )}^{{\rm{\beta }} - 1}}}}{{B( {{\rm{\alpha }},{\rm{\beta }}} )}},\;\alpha ,\beta \gt 0 \end{align}
The definition of the key parameter
$\lambda $
depends on the contribution of each risk factor
${c_i}$
to the probability of accident/incident
$P\left( {{\rm{\omega |}}{c_i}} \right)$
, given by
\begin{align}\lambda = \frac{{\mathop \sum \nolimits_{i = 1}^n P( {{\rm{\omega |}}{c_i}} )}}{{\mathop \sum \nolimits_{i = 1}^m P( {{\rm{\omega |}}{c_i}} )}} \end{align}
Combining the above two equations, the formula for estimating the conditional probability under multiple factors of a specific risk event can be written as
\begin{align}P( {{\rm{\omega }}|{c_1},{c_2}, \ldots ,{c_n}} ) = \frac{{\dfrac{{\mathop \sum \nolimits_{i = 1}^n f( {{c_i}} )}}{{\mathop \sum \nolimits_{i = 1}^m f( {{c_i}} )}}\mathop \smallint \nolimits_0^{\frac{{\mathop \sum \nolimits_{i = 1}^n f( {{c_i}} )}}{{\mathop \sum \nolimits_{i = 1}^m f( {{c_i}} )}}} {{\left( {\dfrac{{\mathop \sum \nolimits_{i = 1}^i f( {{c_i}} )}}{{\mathop \sum \nolimits_{i = 1}^m f( {{c_i}} )}}} \right)}^{{\rm{\alpha }} - 1}}{{\left( {1 - \dfrac{{\mathop \sum \nolimits_{i = 1}^i f( {{c_i}} )}}{{\mathop \sum \nolimits_{i = 1}^m f( {{c_i}} )}}} \right)}^{{\rm{\beta }} - 1}}dt}}{{B( {{\rm{\alpha }},{\rm{\beta }}} )}} \end{align}
The Beta CDF has two unknown parameters,
$\alpha $
and
$\beta $
. The conditional probability
$P\left( {\omega {\rm{|}}{c_i}} \right)$
can be used to calibrate the two parameters. The contribution rate of each risk factor can be calculated based on the frequency of each risk factor in the dataset. Then, the conditional probabilities are input into Equation (17) to solve the minimisation problem and obtain
$\alpha $
and
$\beta $
.
\begin{align}\min h = \mathop \sum \nolimits_{i = 1}^k {{\left[ {f( {{{\rm{\lambda }}_i}}) - {y_i}}\right]}^2} = \mathop \sum \nolimits_{i = 1}^k {{\left[ {\frac{{\mathop \smallint \nolimits_0^{{{\rm{\lambda }}_i}} {\rm{\lambda }}_i^{{\rm{\alpha }} - 1}{{( {1 - {\lambda _i}})}^{{\rm{\beta }} - 1}}dt}}{{B( {{\rm{\alpha }},{\rm{\beta }}})}} - {y_i}}\right]}^2} \end{align}
Based on the optimisation, the
$\alpha $
and
$\beta $
are determined as 0.92797 and 0.014121, respectively.
3.5 Model prediction performance
The prediction performance of the proposed TAN-BN model is evaluated and includes the metrics of accuracy, precision, recall and F1-score. The dataset used in this study is divided into a training set for BN model construction, which contains 70% of the data, and a test set, which contains the remaining 30%. Accuracy is a commonly used metric for assessing the overall correctness, while precision and recall measure the ratio of true positive samples among predicted positive samples and true positive samples. F1-score provides a balanced assessment of both precision and recall. The higher the four metrics are, the better the model is. Based on the true positive (TP), false positive (FP), true negative (TN) and false negative (FN) samples in the confusion matrix, the model’s four metrics are given by
\begin{align}{\rm{Accuracy}} = \frac{{{T_p} + {T_N}}}{{{T_p} + {T_N} + {F_p} + {F_N}}} \end{align}
\begin{align}{\rm{Precision}} = \frac{{{T_p}}}{{{T_p} + {F_p}}} \end{align}
\begin{align}{\rm{Recall}} = \frac{{{T_p}}}{{{T_p} + {F_N}}} \end{align}
\begin{align}F1 - {\rm{score}} = \frac{{2 \times {\rm{Precision}} \times {\rm{Recall}}}}{{{\rm{Precision}} + {\rm{Recall}}}} \end{align}
4.0 Results and discussions
4.1 Mutual information of risk factors
Table 2 displays the mutual information values between the drone accidents and 13 risk factors. It is found that the risk factor ‘Flight control system failure’ has the largest mutual information value (0.08145), implying that this factor poses the greatest impact on the occurrence of drone accidents. Other factors with large mutual information include ‘Remote communication failure’ (0.06347) and ‘Other aircraft approaching’ (0.04675), ‘Loss of electrical power’ (0.04163) and ‘Adverse weather’ (0.04026). The mutual information of the remaining risk factors is less than 0.04, indicating less importance in determining the drone accidents. These risk factors are used for constructing the BN model and selected for further sensitivity analysis.
Mutual information value of “accidents”
4.2 BN structure
Figure 2 illustrates the proposed TAN-BN model for evaluating drone accident risks. The BN structure reveals the relationships between environmental factors, UAV system factors and human factors.
UAV accident Bayesian network structure.
For environmental factors, 32% of the accidents are linked to the presence of other aircraft, with weather-related issues following at 9%. This highlights the significant influence of environmental factors on drone safety, particularly from other aircraft and adverse weather conditions. For UAV system factors, 43% of LOC incidents stem from failures in flight control systems (36%) and communication systems (7%). Additionally, 25.3% of the accidents are related to issues with the drone’s power systems, while 17% are attributed to design flaws. These technical elements significantly impact drone safety. As for human factors, 22.5% of the accidents are associated with unsafe behaviours by pilots, with 20.4% stemming from regulatory violations. This emphasises the critical role that human factors play in UAV operation safety.
For the BN structure, the risk factors of ‘Other aircraft approaching’, ‘System malfunction’, ‘LOC’, ‘Pilot error’, ‘Violation’ and ‘Operating skill level’ are closely linked to the UAV accident, which can be defined as key events preceding the accident. Following these events, ‘Obstacle’, ‘Adverse weather’ and ‘Bird strike’ are the reasons for ‘System malfunction’; Moreover, ‘Adverse weather’, ‘Loss of electrical power’, ‘Remote communication failure’, ‘Flight control system failure’, ‘Brake failure’, ‘Electromagnetic interference’ and ‘Pilot error’ are associated with ‘LOC’. The nodes connecting ‘System malfunction’ and ‘LOC’ explain how the two defining events trigger and then evolve into a UAV accident.
4.3 Model performance on prediction
4.3.1 Performance on fitting risk factors
As mentioned in Section 3.4, the risk factors’ conditional probabilities are fitted by Beta functions. The MSE between the fitted probabilities and the actual values of operating system failures is 3.56608×10−7, indicating that the Beta function precisely fits the observed conditional probability for this risk factor. However, the fitness of the Beta function for ‘Loss of control’ and ‘UAV accident’ reaches 1.06784 × 10−4 and 2.65632 × 10−7, respectively. The low prediction errors indicate that the developed Beta functions have generalised performance in approximating the conditional probabilities of these risk factors. As listed in Table 3, it can be observed that the function fits all data points well in approximating the conditional probabilities of risk factors. In addition, the conditional probability table of LOC is illustrated in Table 4.
The mean squared error for the three child nodes
A simplified CPT example for the child node LOC
4.3.2 Performance on predicting risk events
To examine the performance of the trained BN model’s generalisability, this study selects 30% samples (i.e. 639 samples) from the dataset as the test dataset. The confusion matrix is displayed in Table 5. It can be found that the BN model achieves an overall accuracy of 94.68%, among which the accuracy for accident samples and incident (non-accident) samples are 94.9% and 81.81%, respectively.
Confusion matrix for predicted results
Table 6 presents the prediction performance metrics for each accident case. It can be observed that the F1-score, precision and recall for accident samples are greater than 0.95, indicating that the proposed BN model is effective in identifying UAV accidents, while it is less effective in identifying incidents (the metrics are all below 0.75).
Predictive performance indicator values for each accident
4.4 Sensitivity analysis
This study performs sensitivity analysis to evaluate the effects of a risk factor (in a parent node) on its connected factor (child node) based on the BN model structure. In the sensitivity analysis, an increase or decrease in the prior probability of the parent node leads to a corresponding change in the posterior probability of the child node. The Genie software allows the selection of a target node and any number of other nodes, referred to as sensitivity nodes. Therefore, we set the drone incident as the target node and conduct the sensitivity analysis.
Figure 3 shows the tornado diagram that illustrates the sensitivities of different risk factors on UAV accidents. The length of the horizontal bars in the tornado diagram corresponds to the sensitivity of each node (risk factor), which actually is the contribution of this risk factor to the target node (accident probability). Consequently, it can be found that the top bar, which represents the loss of control caused by the flight control system, has the largest contribution to the occurrence of drone accidents/incidents. Also, if the adverse weather is observed, the probability of UAV accident also increases noticeably, indicating that adverse weather has a great impact on determining accident occurrence. The results also show that loss of control caused by other reasons remarkably contributes to the occurrence of accidents, while violations by the manipulator or brake failure alone have little contribution to accidents. These manifest that the defining events that lead to loss of control of the drone are the most significant reasons for UAV accidents, while human factors and the system failures that are not associated with loss of control have limited effects.
Tornado graph which nodes most frequent risk of accident.
Figure 3. Long description
A horizontal bar graph comparing different conditions and their sensitivity values for accidents equals yes. The graph has 10 horizontal bars, each representing a different condition. The horizontal axis is labeled with sensitivity values ranging from 0.25 to 0.32. The vertical axis lists the conditions with labels such as LOC equals yes, airspace equals yes, and others. Each bar is color-coded with a gradient from red to green, indicating the sensitivity value. The current value is marked at 0.289033 with a reachable range from 0.251248 to 0.328818. The conditions and their corresponding sensitivity values are as follows: 1: LOC equals yes with a value around 0.29, 2: airspace equals yes with a value around 0.29, 3: Accident equals yes with a value around 0.29, 4: LOC equals yes with a value around 0.29, 5: flight control system failure equals yes with a value around 0.29, 6: LOC equals yes with a value around 0.29, 7: Accident equals yes with a value around 0.29, 8: Accident equals yes with a value around 0.29, 9: violations equals yes with a value around 0.29, 10: brake equals yes with a value around 0.29.
4.5 Effects of risk mitigation measures
Based on the established BN structure and estimated posterior probabilities of nodes, the quantified contributions of risk factors on UAV accidents and potential risk mitigation measures can be simultaneously assessed. The effects of risk factors or mitigation measures on UAV accidents are represented by changes in the posterior probability of accident occurrence. As shown in Fig. 4, the initial baseline probability of UAV accidents is 0.38436, with all the probabilities of risk factors (yes=presence of this risk factor) equal to their default values in the accident data. This figure also illustrates the five proposed risk mitigation measures aimed at reducing the likelihood of specific risk factors.
-
(1) M1: Detect and Avoid (DAA)
Baseline accident probability and mitigation measures.
The Detect and Avoid (DAA) is a commonly used mitigation measure that addresses conflicts between aircraft. UAV can be equipped with DAA equipment to detect the obstacles and aircraft that may have a traffic event with itself in advance and conduct manoeuvres to avoid collision. Based on the BN structure, a UAV with DAA will result in a 0% probability for the ‘Approaching other aircraft’ (Yes=0%) and ‘Obstacles’ (Yes=0) nodes (Fig. 5).
Propagation mechanism diagram with DAA mitigation measure.
Hence, the updated posterior probability of the UAV accident under the changes in ‘Approaching other aircraft’ and ‘Obstacles’ decreases by 16% (from 38.43% to 22%). However, it should be noted that the reduction in accident probability is mainly due to the removal of ‘Approaching other aircraft’ rather than ‘Obstacles’. This is because the original probability of ‘Obstacle (Yes)’ is too small (1%), leading to the fact that removing ‘Obstacles’ does not significantly decrease the probability of ‘System malfunction’ (from 5.24% to 5.19%), which has a limited effect on determining the UAV accident probability.
-
(2) Other mitigation measures
Additional mitigation measures include the NextGen-enhanced 4D weather cube, drone geofencing, electromagnetic protection technology, and automatic emergency brake. The effects of the four mitigation measures are listed as below:
M2: NextGen-enhanced 4D weather cube: It is able to remove the risks of adverse weather and potential effects on the flight control system, resulting in 0% of the ‘Adverse weather (Yes)’ and ‘Flight control system failure (Yes)’.
M3: Drone geofencing: It is able to remove the risk of violation of a UAV manipulator, resulting in 0% of the ‘Violation (Yes)’.
M4: Electromagnetic protection technology: It is able to remove the risk of interference caused by electromagnetic conditions/devices, resulting in 0% of the ‘electromagnetic interference’.
M5: Automatic emergency brake: It is able to remove the risk of brake failure of the UAV, resulting in 0% of the ‘Brake failure (Yes)’.
Ultimately, when all five hypothetical mitigation measures (DAA and the above four measures) are incorporated into the evaluation, as depicted in Fig. 6, the accident probability decreases by 17.95% (from 38.43% to 20.48%), signifying that 46.73% of the UAV accidents near the airport are effectively removed. According to the contributions of the five mitigation measures on accident prevention, the DAA plays the most important role, followed by electromagnetic protection technology and drone geofencing. This comparison also provides a guide that DAA could be primarily considered in future safety improvement of drone-manned aircraft interaction. Meanwhile, enhancements on electromagnetic protection and geofencing are also important.
To further quantify the risk reduction performance of each individual mitigation measure, Table 7 presents the specific risk reduction values for each strategy, directly reflecting the contribution of each measure to the reduction in accident probability.
Mitigation measure risk reduction table
Transmission mechanism diagram after all five mitigation measures are implemented.
Figure 6. Long description
The flowchart illustrates the transmission mechanism diagram for aviation safety risk assessment after implementing five mitigation measures. The overall process starts with an ‘Accident’ node, which branches out into several potential causes and factors. Key nodes include ‘system malfunction‘, ‘LOC‘, ‘pilot error‘, ‘operating skill level‘, ‘other aircraft approaching‘, ‘obstacle‘, ‘adverse weather‘, ‘bird strike‘, ‘loss of electrical power‘, ‘flight control system‘, ‘remote communication‘, ‘brake failure‘, and ‘electromagnetic’. Each node has decision points indicating ‘yes’ or ‘no’ outcomes with corresponding percentages. For instance, ‘system malfunction’ leads to ‘LOC’ if yes, and ‘LOC’ further branches into ‘loss of electrical power’ and ‘flight control system’. ‘Pilot error’ is influenced by ‘operating skill level’ and ‘violations’. The flowchart also shows that ‘other aircraft approaching‘, ‘obstacle‘, ‘adverse weather‘, ‘bird strike‘, ‘remote communication‘, ‘brake failure‘, and ‘electromagnetic’ have minimal impact with ‘no’ outcomes being predominant. The relationships and flow between these nodes depict the interconnected risk factors and their contributions to accidents or incidents in aviation safety.
4.6 Case study
To validate the applicability and generalisability of the proposed TAN-BN method in evaluating the operation risk of UAV near the airport, this study conducts a case study at the Guanghan airport (GHN) in China. The study scenario is illustrated in Fig. 7, including the airport and its surrounding area. The airport is located in a region with relatively stable weather conditions. There is a UAV operation airspace near the airport, which includes drones used for airport facility inspections.
Conceptual diagram of UAV operation at Guanghan Airport.
The characteristics or conditions of UAV operations are summarised as:
-
(1) The UAVs operate within a designated area near the airport terminal zone, at an altitude ranging from 80 to 120 m.
-
(2) The drones cruising speed is about 15 kilometers per hour (4.2 m per second) during the tasks.
-
(3) The ambient temperature is 25°C, with wind speeds recorded at 0∼4 kilometers per hour.
-
(4) The average operational duration of the drone is 20 minutes.
-
(5) The operation is classified as visual flight.
After examining the potential of airport facilities and devices, this study proposes two risk mitigation measures that could be used to decrease the accident probability. First, the airport can install high-resolution radars to monitor not only manned aircraft, but also unmanned aircraft near the airport; second, the manufacturers of UAVs will set a drone geofencing to restrict the operation of the UAV. Hence, the proposed BN structure is illustrated in Fig. 8, which combines both risk factors and mitigation measures. The overall structure is in line with that constructed in Section 4.2. The proposed risk mitigation ‘M1: Drone geofencing’ is connected to ‘Violations’, while ‘M2: Airport radar detection and human communication’ is linked to both ‘Other aircraft approaching’ and ‘Obstacles’.
Accident top-level model with two mitigation measures.
Figure 8. Long description
A flowchart illustrating the top-level model of accidents in aviation safety, highlighting mitigation measures and risk factors. The flowchart starts with two mitigation measures: Detect and Avoidance (D A A) and Drone geofencing. D A A leads to potential obstacles such as other aircraft approaching, bird strike, and adverse weather, which can cause system malfunctions. Drone geofencing addresses violations and operating skill levels. System malfunctions and pilot errors can lead to Loss of Control (L O C), which is further influenced by factors like electromagnetic interference, brake failure, loss of electrical power, remote communication failure, and flight control system failure. These factors can result in accidents.
As illustrated in Fig. 9 and Table 8, using the two risk mitigation measures can remove the three risk factors, including ‘Violations’, ‘Other aircraft approaching’ and ‘Obstacle’. Specifically, the application of M3 decreases the accident probability by 1.6% (from 28.08% to 26.48%), while the decrease by using M1 is 7.59% (from 28.08% to 20.49%). As a result, the simultaneous application of both measures decreases the probability of a UAV accident by 8.73% (from 28.08% to 19.35%). The results of this case study demonstrate that the proposed evaluation approach based on the BN model can unveil the risk factors and assess the effectiveness of risk mitigation measures.
Mitigation measure risk reduction table
Transmission mechanism diagram after two mitigation measures are implemented.
5.0 Conclusions
UAV operations near airports have been recognised as one of the new threats to aviation safety in recent years, as they may trigger conflicts with manned aircraft. To understand the threats of UAVs near airports, this study collects historical risk event reports that include both accidents and incidents (near-misses). This study identifies key risk factors associated with the safety of UAV operations. To quantify the impact of risk factors on UAV accidents, a Bayesian network model is constructed and optimised by the TAN algorithm to capture the relationships among the risk factors. Based on this model, the sensitivity of risk factors and contributions of potential mitigation measures are evaluated. Thereafter, a case study using UAV operation data at Guanghan airport is used for validating the applicability and reproducibility of the Bayesian network. Several key findings are summarised as follows:
-
(1) The proposed Bayesian network model has excellent prediction performance on identifying UAV accidents, while it is less effective in identifying incidents.
-
(2) The sensitivity analysis reveals that loss of control in flight is the most important risk factor that leads to UAV accidents, implying that the risk factors connected to loss of control in flight are also determinative.
-
(3) Among the risk mitigation measures, DAA is demonstrated as the most effective in decreasing accident probability, followed by electromagnetic protection technology and drone geofencing.
-
(4) Case study reveals that the proposed BN model can be effectively generalised to field studies and evaluate the contributions of risk mitigation measures.
Several study limitations are also acknowledged. First, this study does not distinguish some risk factors in detail, e.g. adverse weather. This may lead to difficulties in explaining the mechanism of how UAV accidents form and evolve. Second, although this study collected a large number of risk event reports near the airport, only a few reports actually reflect UAV accidents, which leads to a data imbalance between accident and incident samples and impairs the robustness of risk analysis [Reference Zhou, Fu, Wei, Zhou, Li and You31]. Also, the dataset is limited to open-source reports up to 2024, lacks recent cases (2025–2026) and includes very few domestic incidents from China, constraining its temporal and geographical diversity. Third, the proposed Bayesian network model relies solely on observed data and does not account for sociotechnical factors or emergent interactions in complex airport environments, which certainly limits its ability to capture functional variabilities. Future work may adopt systemic methods such as the Functional Resonance Analysis Method (FRAM) to better model interactions among objects and unexpected system behaviours. Hence, further studies are suggested to record more operational data from UAV flights to identify factors that may lead to UAV operational failures. Additionally, advanced data augmentation methods may be considered in further studies to reduce the impact of data imbalance and raise the quality of risk analysis for UAV accident analysis.
Acknowledgement
This work is jointly supported by the National Natural Science Foundation of China (Grant No.52502410) and the Open Project Program of State Key Laboratory of CNS/ATM (No.2025B32). The Independent Research Project of Sichuan Provincial Engineering Research Center of Domestic Civil Aircraft Flight and Operation Support (GCMJ202505). Project of Civil Aviation Flight Technology and Safety Research Base (F2024F05C).
Competing interest
The authors declare no conflict of interest.
Author contributions
Chenglong Li: conceptualisation, project administration, funding acquisition, resources, supervision; Jiameilin Lin: methodology, software, formal analysis, data curation, writing – original draft, investigation; Xuejun Zhang: validation, formal analysis, investigation, data curation, writing – review and editing; Yue Zhou: supervision, data curation, formal analysis, writing – review and editing; Shuwen Zhang: visualisation, investigation, data curation; Yuan Zheng: validation, writing – review and editing.
Institutional review board statement
Not applicable.
Informed consent statement
Not applicable.
