1. INTRODUCTION
There is an extensive existing body of research in the field of collision avoidance as an aid for navigators in manned craft and also for use in autonomous unmanned craft (Belkhouche and Bendjilali, Reference Belkhouche and Bendjilali2013; Lambert et al., Reference Lambert, Gruyer and Pierre2008, Plamen Angelov et al., Reference Plamen Angelov, Xideas, Patchett, Ansell and Michael Everett2008). Among this research there is a broad consensus concerning the need to acquire a precise representation of the environment surrounding the craft and, most importantly, of processing the acquired data for assessing risk of collision before any decision can be made. Some deterministic approaches compute the risk of collision as the rate of change of the relative bearing to craft in the vicinity, either indirectly, by considering the Distance at Closest Point of Approach (DCPA) and Time to Closest Point of Approach (TCPA) as in the appraisal index proposed by Kearon (Reference Kearon1977), or by directly computing the derivative of the relative bearing to the craft, e.g. Plamen Angelov et al. (Reference Plamen Angelov, Xideas, Patchett, Ansell and Michael Everett2008).
This is also the approach recommended by the International Regulations for Prevention of Collisions at Sea (COLREGS) (International Maritime Organization (IMO), 1972), which in rule 7, part D, defines risk of collision as a function of time and bearing as follows:
“In determining if risk of collision exists the following considerations shall be among those taken into account: (i) such risk shall be deemed to exist if the compass bearing of an approaching vessel does not appreciably change; (ii) such risk may sometimes exist even when an appreciable bearing change is evident, particularly when approaching a very large vessel or a tow or when approaching a vessel at close range.”
Current industry standards use a DCPA-TCPA concrete relationship alarm system in line with the above where arbitrary minimum thresholds are set for DCPA and TCPA. Should a target's DCPA and TCPA trespass such limits, an alarm will be given (IMO, 2004).
Hilgert and Baldauf (Reference Hilgert and Baldauf1997) proposed a rule-based concrete model to standardise the meaning of risk given by the COLREG using data provided by Automatic Radar Plotting Aid (ARPA) and defining four crisp risk classes. Alternatively, numerous proposals to add a layer of fuzzy logic to the risk model and or actions to avoid collisions have been suggested. For example, Bukhari et al. (Reference Bukhari, Tusseyeva, Lee and Kim2013) proposed a fuzzy model to capture the relationship between DCPA, TCPA and change in bearing in order to assist Vessel Traffic Service (VTS) centres in making decisions. In addition, Perera et al. (Reference Perera, Carvalho and Soares2012) applied both fuzzy inference and Bayesian methods in order to assess the risk of collision and to take evasive action and Goerland et al. (Reference Goerlandt, Montewka, Kuzmin and Kujala2015) presented a comprehensive rule-based expert system with fuzzy inference where the knowledge domain has been defined by consultation with professional mariners. Furthermore, a number of probabilistic approaches have been proposed that take account of unknown factors in order to predict risk and possible trajectories; see Belkhouche and Bendjilali (Reference Belkhouche and Bendjilali2013) or Lambert et al. (Reference Lambert, Gruyer and Pierre2008) and Simsir et al. (Reference Simsir, Amasyalı, Bal, Çelebi and Ertugrul2014) for the application of Artificial Neural Networks for predicting positions of vessels in a collision alert system. These methods add a layer of sophistication to the geometric approach. Chin and Debnath (Reference Chin and Debnath2009) describe an ordinal model of risk based on data from a survey of pilots, again taking account of the relationship between DCPA and TCPA and incorporating day or night navigation and ship tonnage as an indicator of manoeuvrability.
Common to all these studies is the direct association of risk with the spatial possibility of a collision or the conceptualisation of risk as a consequence of collision only. For these methods, a craft that is navigating in parallel to the observing vessel would not account for risk, as for instance in a traffic separation scheme as defined by the International Maritime Organization (IMO, 2013). However, the navigator's judgement of the risk posed by a neighbouring craft seems to involve factors other than just the geometric calculation of trajectories (Goerlandt and Montewka, Reference Goerlandt and Montewka2015; Curtis, Reference Curtis1986). Furthermore, in “many to many” scenarios, where many craft are at risk of colliding with one another, perception of risk seems to become even more complex and depends on how different craft interact with each another and on their particular idiosyncrasies. Hence, craft can be perceived as posing a risk for the mariner even when a collision is not imminent or even when it is impossible.
In this paper, we present a machine learning approach to assessing inherent risk for craft in the vicinity of a vessel. This approach will infer a model deriving from human judgement and experience, which provides a distribution of risk levels for a given context that can be incorporated as a cost function in navigation or avoidance algorithms. This work is a part of a larger research project in collision avoidance in which it is a key part of the intelligent risk assessment in navigation and collision avoidance algorithms.
To elicit information on the mariners' judgement of risk, a questionnaire has been designed to record data on a number of scenarios. The questionnaire was completed by 425 professional marine navigators, who were asked to assess the risk of individual craft in different scenarios with one or many craft and used an ordinal ranking scale from one to five to quantify their perceived level of risk. A non-linear regression model, Ordered Probit, has then been used to learn and to estimate risk for craft in new scenarios based on a number of attributes. The resulting estimation has the form of a probability vector for a distribution of risk over the values one to five. The objective is not to find an exact value for risk of collision but to learn an operational model of the inherent risk of a neighbouring craft in a given scenario based on the experience of the professional mariner.
An outline of the remainder of the paper is as follows: Section 2 describes in detail the method chosen to elicit the data necessary to define and train our model. The model is then presented and explained in Section 3. Section 4 discusses the performance of the model in comparison with a Naïve-Bayes classifier and proposes a possible application of the resulting estimation. Finally, the conclusions, Section 5, briefly discuss the limitations of our model and suggest directions for further research.
2. QUESTIONNAIRE DESIGN AND DATA COLLECTION
A professional mariner's judgement of risk develops through training and through experience gained in many different encounters with other craft during navigation. This implicit knowledge of good seamanship can only be acquired by training, practice and actual experience. With the objective of capturing the mariner's judgement of neighbouring craft's risk, a questionnaire was designed so as to provide a craft's risk ranking on a scale from one, very low, to five, very high, for different scenarios. The questionnaire was composed of two parts. The first part presented single craft situations, one-on-one encounters between the mariner's vessel and a neighbour craft, and described them textually with the following characteristics being provided: Closest Point of Approach (CPA), Time to Closest Point of Approach (TCPA), Relative Situation, Colour, and Trajectory Variability (see Figure 1). For this first part, no graphical representation was presented.
Presentation of textual questions.
The set of variables was selected through consultation with professional mariners. DCPA, presented to the mariner as CPA, and TCPA are spacio-temporal relational values between a given craft and the vessel being navigated by the mariner and are functions of bearing, range, speed and course of the two craft involved; DCPA being the closest distance between two objects should the trajectories not change and TCPA the time to reach such a point. Relative Situation can take any of the values defined by the COLREGS (International Maritime Organization (IMO), 1972): Head on, crossing, overtaking and/or overtaken. Colour can be either green, red or undefined when it is not clear or cannot be determined, and is the method used by mariners to relatively position themselves with respect to another craft's side; either on the starboard or port side of the reference craft respectively. The Trajectory Variability of a craft indicates the confidence in the prediction of its future positions, thus affecting the certainty in DCPA and TCPA. The professional mariner does this intuitively by observing how erratic a craft's trajectory is. The questionnaire simplifies this to a Boolean variable i.e. erratic or not erratic. In a later application of this research, Kalman filters are used to process the positional data of craft to predict trajectories, and a threshold on the resulting error covariance matrix of the state estimate is used to determine the value of this Boolean variable in real time. Particularities of the individual craft were not presented in this questionnaire, hence variables like tonnage or manoeuvrability, that have been considered in other works, are not contemplated.
The set of variables are deemed to be inherent to the craft and not dependent on the environment. Accordingly, the questionnaire did not include geographical features, visibility constraints - it was assumed to occur in good visibility- or weather conditions, for example, as these were presumed to be common to all nearby craft in a scenario and are hence assumed constants. How these variables would affect the absolute risk is beyond the scope of this paper. The COLREGS convention is a fundamental pillar of maritime training and as such is always part of any assessment made in collision avoidance at sea. Ideally, we would have liked our data to have been independent of this set of rules, for our wish is to learn inherent risk of craft which we could then use to apply any set of rules, and we made this clear when the questionnaire was presented to the mariners. But our results suggest that COLREGS may still have influenced the participants' judgement of risk, for example in their different assessments of a green or red craft in most cases. The model presented in this paper aims to learn about inherent risk of other craft that could then be used as a variable to account when applying the COLREGS at a higher layer.
This first part of the questionnaire consisted of 100 different scenarios from which 20 were randomly selected and presented to each participant. This approach allowed us to collect data on a wider range of scenarios while at the same time limiting the length of the sessions.
The second part of the questionnaire was comprised of simple graphical representations of scenarios, including single craft, one-on-one, and multi-craft, many-on-many situations. The questions were accompanied by a concise supporting text with the objective of eliminating possible ambiguity resulting from the simplicity of the graphics. The graphical scenarios were depicted as a simple polar coordinate representation of Radio Detection And Ranging (RADAR) or Electronic Chart Display and Information System (ECDIS) screens, showing neighbouring craft with vectors for speed and course and a tracked trajectory. It was not explicitly specified if the course and speed vectors were true or relative (see Figure 2), however, the responses show a unimodal distribution with low variance suggesting that a common interpretation has been adopted across the participants. We note that a number of the scenarios presented in this part of the questionnaire corresponded to the graphical representation of some of the text-based questions included in the first section.
Presentation of graphical questions.
In total there were 50 different graphical scenarios, from which a subset of ten were randomly selected to present to each participant for assessment. They were then asked to evaluate the risk for each craft represented and the overall risk for the scenario itself. Other works present a textual representation of risk in line with the International Maritime Organization recommendations for alerts in Integrated Navigaton Sytems (IMO, 2007); see Hilgert and Baldauf (Reference Hilgert and Baldauf1997), Goerlandt et al. (Reference Goerlandt, Montewka, Kuzmin and Kujala2015) and Chin and Debnath (Reference Chin and Debnath2009) for examples. However, in our work the participants had to respond to the questions using a numerical scale ranging from one to five, rating the risk posed by each craft. This scale and range gave adequate resolution and at the same time helped the participants to identify an appropriate risk level with equally spaced categories. Also, it avoided any semantic confusion associated with the use of labels and their interpretation; see Wildt and Mazis (Reference Wildt and Mazis1978) for a discussion on the latter, Schwarz et al. (Reference Schwarz, Knauper, Hippler, Noelle-Neumann and Clark1991) for a study on how interactions between labels and rating scales modify the meaning and Preston and Colman (Reference Preston and Colman2000) for a study in optimal number of categories.
The questionnaire was distributed online and it was accessible anonymously through the University of Bristol website. The server's software stored the responses directly into a database and logged them with a random session identification number. The participants were recruited mainly through maritime organisations, shipping companies and training centres, providing over 8000 observations over the period during which the questionnaire was active. The sample size for each question/scenario varies but on average is 39, with a minimum of 24 and a maximum of 65 participants.
3. DATA ANALYSIS
As expected, there is variation between the participants regarding their judgement of the risk posed by the craft in a given scenario. This may be due to differences in experience or understanding or may simply be natural variation between individuals. After a first analysis of the data, we observed an underlying normal distribution to the responses to the questions and since neither of the continuous independent variables used satisfies the properties of superposition for the dependent variable, risk, this suggests that there is a non-linear relationship between them and a normally distributed error. This, added to the ordered scale proposed for the responses, suggested the use of an Ordered Probit.
3.1. Use of Ordered Probit Model
The Ordered Probit Model has its roots in bio-statistics (Aitchison and Silvey, Reference Aitchison and Silvey1957) and was introduced into the social sciences by McKelvey and Zavoina (Reference McKelvey and Zavoina1975). It has often been used since for the analysis and prediction of dependent ordered variables with an underlying, continuous and nonlinear metric. Indeed, there are many applications of the Ordered Probit model, ranging from accident injury prediction (O'Donnell and Connor, Reference O'Donnell and Connor1996) to estimation of customer's enjoyment of films by the Netflix Prize's winner (Koren, Reference Koren2009; Andreas Toscher, Reference Andreas Toscher2009; Piotte and Chabbert, Reference Piotte and Chabbert2009) and has previously been used by Chin and Debnath (Reference Chin and Debnath2009) to model risk of collision in port water navigation. The model assumes an underlying linear relationship characterised by
$$Y = X\beta + \varepsilon $$
where Y is the continuous latent unobserved variable, X is a vector of independent variables defining the data, β is an unknown coefficient for x and ε is a random disturbance term which is assumed to be normally distributed according to ~N(0, 1).
The central idea is that a latent real variable is underlying the ordinal set of responses and that we observe these responses instead of the latent variable. The real line is then divided into variable regions that represent the ordinal categories; in our case five regions. The observed ordinal variable Z takes the values 1, …, 5 corresponding to the set of risk categories z ∈ {1, …, 5}.
$$Z = z \Leftrightarrow \mu _{z - 1} \lt Y \le \mu _z$$
where μ are the bounds of the regions on Y which define the values of Z as intervals of the real line. The probability of observing a particular ordinal outcome in an Ordered Probit model is therefore given by:
$$P[Z = z] = \Phi \left( {\mu _z - X\beta} \right) - \Phi \left( {\mu _{z - 1} - X\beta} \right)$$
where Φ is the cumulative normal distribution and z = 1, …, 4 and, for z = 5, as ε is assumed to be multivariate normal with σ 2 = 1, then:
$$P\left[ {Z = 5} \right] = 1 - \Phi \left( {\mu _4 - X\beta} \right)$$
The β parameters and the μ bounds of the model are estimated by means of Maximum Likelihood Estimation (MLE), normally using the Newton-Raphson method on the log-likelihood function, which for the Ordered Probit model is:
$$\log (L) = \sum\limits_{i = 1}^N {\log} \left[ {\Phi \left[ {\mu _z - x_i\beta} \right] - \Phi \left[ {\mu _{z - 1} - x_i\beta} \right]} \right]$$
where x is the value of vector X in the sample i, for a sample of size N.
For further details regarding this method see McKelvey and Zavoina (Reference McKelvey and Zavoina1975) or Becker and Kennedy (Reference Becker and Kennedy1992).
3.2. Independent Variables for estimating risk
Let Z be the perceived risk, which has the format of an ordinal polychotomous variable taking on values from one to five. For each craft, there will be a probability that its perceived risk takes each of the values from one to five. Let R i be the vector of probabilities of perceived risk for the given i'th craft defined by the vector of independent variables X i :
$$R_i = \left\langle {{\rm P}\left( {Z = 1 \vert X_i} \right),{\rm P}\left( {Z = 2 \vert X_i} \right), \ldots, {\rm P}\left( {Z = 5 \vert X_i} \right)} \right\rangle $$
The selection of independent variables for the model is intended to isolate the craft from the environment and only those descriptive of the craft's relations with other craft in the neighbourhood are considered. Also, particulars of the craft such as size, type of vessel or navigational circumstances, such as method of propulsion, are not contemplated for this research. Each craft is assumed to be a two dimensional point in the Euclidean plane. The initial set of independent variables was chosen in consultation with professional mariners as described in Table 1.
Initial set of independent variables.
We have compared a number of different models using diverse selections of variables from the list in Table 1 in terms of the Schwarz Bayesian Information Criterion (SBIC) (Schwarz, Reference Schwarz1978) and by calculating Average Marginal Effects of each variable for each value from the possible outcome scale. As a result of this process we selected a final model which has the continuous variables DCPA and TCPA and the discrete (dichotomous) variables Red, Green, Course Erratic, Head on, Crossing, Overtaking. This model offers a balance between goodness of fit and economy of calculation for real time applications.
The results in Table 2 show that DCPA has a large effect on the perceived risk, which concurs with the established methods for assessing risk of collision. However, TCPA on its own seems to have little impact on the risk assessments of the navigators in our study. This will be discussed in more detail in the next section of this paper.
3.3. Parameter estimates
Maximum likelihood estimates of the structural parameters β and the bounds μ on Y are shown in Table 2. Average Marginal effects of the independent variables, the discrete change in probability for each of the values of a given ith variable averaged across the observed values of the rest of the variables in the model (Bartus, Reference Bartus2005), are shown in Table 3 for all possible outcomes of our risk scale. Note that Average Marginal Effects in non-linear models are not constant and should only be taken into consideration as an indicator and not an estimator (Ai and Norton, Reference Ai and Norton2003, Greene, Reference Greene2010).
Parameter estimation for the independent variables and μ bounds on Y.
Standard errors in parentheses
*** p < 0·01, ** p < 0·05, * p < 0·1
Average Marginal Effects of variables for different outcomes.
Standard errors in parentheses
*** p < 0·01, ** p < 0·05, * p < 0·1
Some of our previous assumptions are supported by the estimation of the β parameters but there are also surprises, as in the case of the low absolute value of the parameter for variable TCPA.
As expected, DCPA has a significant effect on the perceived risk; the large negative parameter estimate indicates a significant increase of perceived risk as the value of DCPA decreases, which is consistent with the message that a value 0 for a craft's DCPA anticipates a collision. As mentioned, this is also the approach of well-established methods for assessing risk of collision i.e. a DCPA with value 0 is equivalent to a steady bearing with a craft. However, our data and the TCPA's parameter estimate gives us a different insight. One might intuitively think that a smaller TCPA would also have a significant effect, increasing significantly the perceived risk, but this is not clear from its estimated parameter value. The low absolute value of the latter suggests that only a slight increase in perceived risk results from a decrease in TCPA.
Another parameter that has a large effect on the perceived risk is the steadiness or uncertainty of the trajectory of the craft, included in our model as a discrete dichotomous variable. For an erratic course, the perceived risk increases notably as indicated by the high estimated parameter value.
Furthermore, Table 2 gives a relatively high absolute value to the Head On variable, suggesting that head on encounters are judged to have more risk than crossing or overtaking encounters. In this case, it is possible that the COLREGS are affecting the response and this situation creates more uncertainty in the mariner than the others, for there is not a ‘stand on’ or ‘give way’ vessel defined in the rules or a clear Colour of the craft. This contrasts with the other two situations where it is made clear in the COLREGS which vessel must ‘give way’ in relation to their Colours; the ‘head on’ situation requires an avoiding action from the two craft involved. Red aspect craft, those on the starboard bow of the craft observing, are also perceived as carrying more risk. Again, this may be the effect of the COLREGS on the mariners' judgement as the rules will always give them ‘stand on’ rights in normal visibility situations. This seems to leave the burden for action on the mariner and a judgement of risk is perhaps associated with this responsibility.
4. RESULTS
The performance of the model has been evaluated using ten-fold cross validation (Kohavi, Reference Kohavi1995) over the original questionnaire's responses data set and has been compared against the well-known Gaussian Naïve-Bayes classifier (Hand and Yu, Reference Hand and Yu2001).
4.1. Benchmarking with Naïve-Bayes
Naïve-Bayes classifiers are a popular class of probabilistic classifiers that make use of Bayes' theorem with strong assumptions of independence between the variables used, see Hand and Yu (Reference Hand and Yu2001) for an interesting description of its effectiveness or Rish (Reference Rish2001) for an empirical perspective. It is important to note that we are using Gaussian Naïve-Bayes as a probability estimator and not as a classifier. There is evidence in the literature (Lowd and Domingos, Reference Lowd and Domingos2005) to suggest that Naïve-Bayes is in general a reasonably accurate and efficient general approach and hence provides a good benchmark for our proposed model.
We use the Kullback-Liebler divergence (Kullback and Leibler, Reference Kullback and Leibler1951) between observed and predicted distributions as a measure to compare the performance of the models and also a simple Euclidean distance between the predicted data during cross-validation and observed data. The Kullback-Leibler divergence, or relative entropy, is a non-symmetrical measure of the difference between two probability distributions expressed by:
$$D_{KL}\left( {P \vert \vert Q} \right) = \sum\limits_i {\ln \left( {\displaystyle{{P\left( i \right)} \over {Q\left( i \right)}}} \right)P\left( i \right)} $$
This metric is widely used in Information Theory since its results can be interpreted as the average number of extra units of information required to encode data generated by one distribution, P, using coding from a different distribution, Q.
Figure 3 shows the results of this test with the values D KL in the Y axis and the scenario in the X axis. It can be clearly seen that the Ordered Probit model significantly outperforms the Gaussian Naïve-Bayes model on predicting risk for new craft using our dataset. The values D KL , in Nats, for every scenario depicted in the questionnaire are better (lower values better) in almost every case and in providing consistent and homogeneous results.
Nats for each scenario for O Probit and Naïve-Bayes.
Figure 4 and Figure 5 show the Euclidean distance between the observed data and the predictions obtained during the cross validation process for Ordered Probit model and for the Gaussian Naïve-Bayes model respectively. It is clear that there is a larger dispersion for the Gaussian Naïve-Bayes model. The line x = y would represent a perfect fit of a model.
Cross validation dispersion for Ordered Probit model.
Cross validation dispersion for Gaussian Naïve-Bayes model.
4.2. Apply the Risk Model
Using the presented model as a risk estimator provides a vector of probabilities representing the inherent risk distribution of any given craft. We propose a method to embed such a vector of risk in the navigation space where applicable. The resulting estimation can be converted into a spacio-temporal risk cost function by means of nested areas representing different risk levels for a given craft. This cost function can then be employed to define risk-shaped pseudo-static ‘obstacles’ incorporated into all sorts of path finding algorithms. We present a three dimensional example of this application where the radius of such areas is set to a constant and their height is defined by the predicted probability for the level of risk. Note that it could also be possible to work with two dimensional areas of variable radius defined by the risk.
Let the craft Tg have a circular domain of an arbitrary diameter at time cero t 0. With a simple projection of the craft's domain to a possible point of collision given Tg's vector and our own craft's (my Object) speed, we can calculate a likely area in a time interval during which our craft could potentially be invading Tg's domain if a course between V 1 and V 2 where taken (see Figure 6). Course V c would invariably lead to a collision at a future time should the course and speed of both, craft Tg and my Object, not change.
Projection of a given craft's domain to a collision time.
At t 0, the craft's estimated risk is mapped concentrically at equal distances, being risk one at the centre and risk five at the periphery. The risk acquired when crossing a risk interval for a given craft can be easily calculated in the actual related craft's domain. Thus, the trajectory that receives the higher risk is the one that would collide with the target, which crosses the five zones at its maximum chord or its diameter.
The height, value of risk, of each one of our stacked zones is defined as:
$$H = \displaystyle{{D_i} \over {\left( {\left\vert {\mathop {Tg}\limits^ \rightharpoonup - {\mathop V\limits^ \rightharpoonup}_c} \right\vert} \right){\rm P}\left( {\,pR_i \vert X} \right)}}\;i = \left( {1, \ldots ,5} \right)$$
where D
i
is the diameter of the given zone,
$({\vert {\mathop {Tg}^{\rightharpoonup}} {\mathop {V}^{\rightharpoonup}}\vert})$
is the relative vector magnitude for a collision and
${\rm P}\left( {pR_i \vert X} \right)$
the probability for the given estimated risk value.
The cost function to find the risk that a trajectory is acquiring, aR, when crossing a risk interval of a given craft:
$$aR = \sum\limits_{i = 1}^5 {2\sin \left( {\displaystyle{{\theta _i} \over 2}} \right)H_i} $$
where θ is the angle between the two radius defined by the centre and the intersections of the relative trajectory at each risk map into Tg's domain.
Thus, given a risk vector 〈0·21, 0·18, 0·23, 0·18, 0·20〉 for Tg, the zones would look as in Figure 7.
Projection of a given craft's domain to a collision time with embedded risk.
The above spacio-temporal risk cost function provides a framework that can be employed into path finding and optimisation algorithms to avoid the projected risk interval pyramids and to find efficient and safe navigation routes between traffic.
5. CONCLUSIONS
Avoiding collisions is ultimately the objective of assessing risk for a given craft and it can be achieved by any of the established methods for determining risk of collision. However, we claim that a method which considers all neighbouring craft, not only those with low DCPA values, and that provides a probabilistic model of risk can help the mariner to improve their decision making. In particular, it can potentially allow for the optimisation of routes taking account of potential risk, especially in situations in which prioritising between neighbouring vessels is required, uncertainty is present or where some risk must be accepted and managed in order to successfully navigate through them. We show a simple example of application for mapping the obtained risk vector to the space-time and a cost function to be used in path finding and optimisation algorithms.
Humanlike understanding of a craft's risk in its complexity, including quantifying uncertainty, offers a powerful tool for Intelligent Navigation Systems. The analysis in this paper is obviously limited to the training dataset collected and further development, including a continuous learning capability, would be necessary for real world applications. This first approach offers a rather simple model which should be expanded to include possible interactions between neighbouring craft, their explicit changes in speed or course, i.e. somehow contemplated in trajectory variability, size of craft and rate of turn i.e. implying manoeuvrability, for instance.
The dataset obtained in our questionnaire can potentially offer responses to some of these variables, interactions for instance, but does not contain enough information to be able to learn from craft's size or manoeuvrability. Eliciting new data with a new questionnaire would be desirable to further advance in learning and modelling risk.
