1. Introduction
The use of Extended Reality (XR) technologies in teaching and training has increased remarkably (Marougkas et al., Reference Marougkas, Troussas, Krouska and Sgouropoulou2023). This covers XR in its broadest sense: in addition to less immersive forms of virtual reality, such as Augmented Reality (AR) (Dunleavy et al., Reference Dunleavy, Dede and Mitchell2008), Cave Automatic Virtual Environments (CAVE) (Ojajuni et al., Reference Ojajuni2023) or even Mixed Reality (MR) (Tang et al., Reference Tang, Au, Lau, Ho and Wu2020), the most advanced fully immersive virtual reality (IVR) techniques, where Head Mounted Displays (HMDs) and controllers are used, are also having an impact on training and education (Freina and Ott, Reference Ott and Freina2015; Pellas et al., Reference Pellas, Mystakidis and Kazanidis2021). The research literature on the added value of XR technologies for learning has grown significantly (Checa and Bustillo, Reference Checa and Bustillo2019; Yin et al., Reference Yin, Chng, Wong, Ho, Chua and Chui2020). Although it has been proven in several studies that less immersive forms of XR can also offer added value in learning (Wörner et al., Reference Wörner, Kuhn and Scheiter2023; Zhao et al., Reference Zhao, Ni, Dong, Tian and Chen2023), this study will limit itself to investigating the added value of fully immersive virtual reality in maritime safety training. Since the mid-2010s, the continuous innovation and commercialisation of more affordable IVR tools makes this technology more easily accessible for many educational situations (Cecotti, Reference Cecotti2022). Immersive Virtual Reality aims to give the user the impression that they have ‘stepped inside the synthetic world’ (Small, Reference Small and Furht2008) that has a high level of psychological fidelity; in other words, creating for the user a psychological experience of ‘being there’ (presence) (Bailenson, Reference Bailenson2018; Cummings and Bailenson, Reference Cummings and Bailenson2015). The use of IVR technologies in education will only increase in the future as these offer solutions to pedagogical constraints and situations such as time, space and infrastructure limitations, dangerous learning environments, or unethical conditions (Bailenson, Reference Bailenson2018; Freina and Ott, Reference Ott and Freina2015). This is also an often-cited advantage in marketing campaigns (‘safe training, anywhere, anytime’) by companies to promote VR products or services (AtlasVR, n.d.; Cortiical, n.d.; Delaware, n.d.; OneBonsai, 2023; VRTS, n.d.). The present research makes a comparison between IVR learning and traditional hands-on lessons in maritime safety education. It is a first initiative at Antwerp Maritime Academy (AMA) to investigate whether VR technology can be a possible added value for maritime training and an impetus to explore future opportunities in this field. To this purpose, a Meta Quest 2 HMD was used to make a quantitative assessment of procedural performance results in a lifeboat launching case study. Although some previous studies concluded that high immersive HMD VR training has a positive influence on learning outcomes (Checa and Bustillo, Reference Checa and Bustillo2019; Liu et al., Reference Liu, Wang, Lei, Wang and Ren2020, Wu et al., Reference Wu, Yu and Gu2020), the review of Jensen and Konradsen (Reference Jensen and Konradsen2017) stated that the use of HMDs offers no added value for certain types of learning and might even be counterproductive when compared with less immersive technologies or traditional instruction.
Because of the importance of safety in the maritime industry (Arslan et al., Reference Arslan, Kurt, Turan and De Wolff2016; Lloyd’s List Intelligence, 2022), but mainly because of the specific nature of the lifeboat safety course, an IVR application for the training of lifeboat launching and recovery procedures was developed. Lifeboats, along with life rafts, are the main lifesaving equipment on board merchant ships to save lives at sea in the event of maritime accidents such as collision, fire and grounding (Akyuz, Reference Akyuz2016; International Maritime Organization, 2019). Periodic maintenance, inspections and lifeboat drills need to be carried out on a regular basis to familiarise and train ship crews, which entails a risk of lifeboat incidents with potential consequences such as damage to equipment, financial implications, human injury and even death (Shen and Huang, Reference Shen and Huang2011). The main causes for lifeboat incidents identified to date are a lack of knowledge, and insufficient training and experience (Danaci and Yildirim, Reference Danacı and Yildirim2023; SIRC, 2018), which underlines the importance of quality practical education in this field.
Divided into several practical lessons, AMA students spend a total of 4 h on hands-on lifeboat training in their practical training group where a conventionally launched fully enclosed lifeboat with gravity luffing davit is used (Hatecke GmbH, n.d.). The ‘XR lifeboat learning’ application used in this research is the digital twin of this physical lifeboat and davit of AMA, and has been developed in close collaboration with Delaware. XR lifeboat learning has two learning modules: the lifeboat launching module and the lifeboat recovery module. The present research focuses on the lifeboat release procedure, but can be further expanded to recovery in subsequent research.
2. Methodology
2.1. The VR learn model
The way an IVR application is modelled and learning content is delivered is not an easy issue and even small adjustments can have a significant effect on the learning outcomes (Wolfartsberger et al., Reference Wolfartsberger, Zimmermann, Obermeier and Niedermayr2023). According to the IVR model (CAMIL) of Makransky and Petersen (Reference Makransky and Petersen2021), it is not the medium of IVR itself that results in more or less learning, but the method of instruction used in an IVR lesson whereby the unique possibilities of the medium are facilitated. Also in the field of safety training, there are several studies and frameworks that define its objectives and the ways in which safety performance manifests itself efficiently (Burke and Sockbeson, Reference Burke, Sockbeson, Clarke, Probst, Guldenmund and Passmore2015; Griffin and Neal, Reference Griffin and Neal2000; Kirkpatrick and Kirkpatrick, Reference Kirkpatrick and Kirkpatrick2016; Salas et al., Reference Salas, Tannenbaum, Kraiger and Smith-Jentsch2012). Considering this, the ‘XR Lifeboat Learning’ application used in the present study focuses on the acquisition of safety procedural skills, although there is also some input for factual and conceptual (declarative) safety knowledge acquisition. As a basis for the structure behind the application, the Four-Component Instructional Design (4C/ID) model was used. According to this model, educational programs for teaching complex skills or professional competencies are built from four components: learning tasks, supportive information, procedural information and part-task practice (Van Merriënboer and Kester, Reference Van Merriënboer and Kester2014). Practically speaking, this has led to four modes within both a lifeboat release procedure and a lifeboat recovery procedure: Discovery, Learning, Practice and Test (Figure 1). Each mode offers a different way to engage with the learning content that has been divided into smaller sub-tasks to rehearse.
User interface in the ‘XR Lifeboat Learning’ application based on the 4C/ID model.
In addition, the user can start each session for a real emergency abandon ship situation or a drill situation. The actions that must be performed to safely launch a lifeboat during a drill or exercise are somewhat different from launching the same lifeboat in a real abandon ship situation. The application gives the opportunity to train for both situations. In contrast to many other studies that have been conducted so far on VR training in procedural skills (Matovu, Reference Matovu, Ungu, Won, Tsai, Treagust, Mocerino and Tasker2022; Pellas, Reference Pellas, Mystakidis and Kazanidis2021), the lifeboat launching procedure is complex and extensive.
2.2. Organisation of the research
On their introduction day at AMA, first-year bachelor students received an introduction about the upcoming VR research project and had an opportunity to register. A total of 39 students enrolled voluntarily on that day. During the project, one student dropped out, resulting in 38 usable test results. Although this is a rather limited sample size, 38 students enrolling voluntarily in the study is a strong participation rate, representing approximately 65% of the total number of Dutch speaking first-year Bachelor of Nautical Sciences and Marine Engineering students that year. Given the constraints of maritime education and the demanding academic schedule, this high level of engagement underscores the study’s relevance and the participants’ genuine interest in IVR-based training. The present study is limited to an evaluation of the added value of IVR for procedural knowledge, and a real lifeboat and launching davit was used to this purpose. Due to the nature of the practical evaluation, no pretest was administered; therefore, no pre- and post-test comparative analysis could be performed. First-year students have, in general, no prior knowledge about lifeboats and working with a real lifeboat without preliminary training would not have been safe to perform.
The students were randomly divided into two main training groups: VR (13 students) and control (C; 12 students). An additional third group VR+ (13 students) was also created to explore the added complementary value of VR to traditional safety classes. All students of all groups had the opportunity to follow the regular theory safety lessons, but attendance at these lessons is not mandatory and was not registered. In contrast to the theory lessons, there were differences in the requirements regarding the practical lessons between training groups. In group C, the students followed traditional practical lessons, as they would normally do in their first-year bachelor curriculum. In the VR group, the 4 h practice with the physical lifeboat was replaced by VR learning time. The students of the VR+ group received a combination training of the traditional physical lifeboat lessons and an additional 4 h of VR learning time to master the lifeboat procedures. Both the VR training and the traditional hands-on training did not take place all at once, but were spread out at intervals of 1 week between training sessions. The duration of a single VR session was between 40 and 50 min, with 40 min per session counting as effective learning time, and an additional 10 min for re-familiarisation with the material and set-up for the digital environment. This is in line with the suggestion for future research in the literature review by Hamilton et al. (Reference Hamilton, McKechnie, Edgerton and Wilson2020), to account for a longer period of habituation or free navigation to reduce the hindrance of learning outcomes due to unfamiliarity with the technology. Students had to register for four VR sessions, which corresponds to the full 4 h of lifeboat training time. The choice for a maximum duration per session was made deliberately: research shows that VR sessions of less than 2 h have a larger effect than sessions of more than 2 h (Taranilla et al., Reference Taranilla, Olivares, Gutiérrez and González-Calero2022) and that it takes on average 55 to 70 min for users to experience discomfort (Kourtesis et al., Reference Kourtesis, Collina, Doumas and MacPherson2019). To ensure that physical discomfort would not be a limiting factor for this study, these 55 min were shortened with a 25% safety margin, equating to 40 min. Delaware, our partner who created the VR application, also indicated that 45 min is approximately the limit at which they usually perceive sensitive users to experience complaints and motion sickness.
Each student had to take a practical test with a physical lifeboat (Figure 2) where they were assessed by an independent party, who had no knowledge of the background of the participants. These tests were conducted three weeks after training because testing competence and skill acquisition can be more valuable after some time has passed after training (Kim et al., Reference Kim, Ritter and Koubek2011).
Physical lifeboat used for the practical test.
Just before the start, the participants received a clear written description of the scenario: performing an ‘abandon ship exercise’ or in other words, to launch the lifeboat correctly as it should be done on board a seagoing vessel during an exercise. Each student therefore had to: muster, don correct personal protective equipment, independently launch the lifeboat, start the engine and finally free the lifeboat from the ship without endangering the safety of themself or their passenger (the examiner). Recovery of the lifeboat was not part of the test. The examiner was expected to be part of the crew but had no prior knowledge of lifeboat procedures: they did not intervene and only did what the student asked them to do during the test. The assessment was based on an objective exam form with yes/no questions to assess whether the student had correctly performed the various tasks in the procedure. The questions were formulated in such a way that the subtasks could also be assessed. This way, it was possible to investigate more precisely where an error was made. For example: it was not only asked ‘Does the student start the engine properly?’ but also ‘Does the student check the battery?’, ‘Does the student turn on the ignition?’. Questions were also included to assess the chronology of certain acts such as: ‘Does the student start the engine before opening the lifeboat hooks?’.
The yes/no questions were converted into a score table. If the student performed the procedure completely correctly and received a single point for each question on the sheet, this gave a total of 40 points. This simplest approach gives an initial insight into how many (sub)tasks a participant performed correctly and gives the opportunity to examine whether there are significant differences in the Total Score between the different training groups VR, VR+ and C.
2.3. Procedural performance assessment
Makransky and Petersen (Reference Makransky and Petersen2021) recommend when conducting research to distinguish between the different psychological differentiations in knowledge to understand how IVR contributes to which learning outcome. This means that, in addition to a correct pedagogical learning approach, correct corresponding assessment methods for analysing results are to be considered (Bloom, Reference Bloom1956). Although the XR lifeboat learning also focuses (to a limited extent) on factual and conceptual knowledge, it was decided to only assess procedural knowledge in this study. To this end, the motor skills, task performance and the correct order of task performance were assessed in the practical test of the students. Regarding performance evaluation, it was not only checked whether the task has been completed, but also the performance accuracy (was the task completed correctly) and the efficiency of the performance.
Although the Total Score analysis might reveal differences between groups in the number of questions answered with ‘yes’ in the exam sheet, it will not give an answer to the question whether the student succeeded in their assignment, namely, was the student able to get the lifeboat safely in the water and free from the ship? The total score table also gives us no exact insight into how well the student actually performed, because some actions in the procedure are more crucial than others. Even if you forget to start the engine you will still get the lifeboat into the water, but if you forget to pull the automatic release wire to lower the boat, the launching procedure is over immediately. This is both a research and a pedagogical issue: how you evaluate the results depends in the first place on what you want to evaluate (Bloom, Reference Bloom1956). Therefore, the student procedural performances were tested against four questions.
-
Question 1: Can the student independently launch the lifeboat into the water?
-
Question 2: Can the student independently launch the lifeboat into the water and free the lifeboat from the ship?
-
Question 3: Does the student not endanger their own safety or the safety of their passenger(s) during the procedure?
-
Question 4: Can the student independently launch the lifeboat into the water and free the lifeboat from the ship without compromising their own safety or the safety of their passenger(s)?
Questions 1 and 2 do not address the safety aspect during the procedure. In question 3, however, it does not matter whether the student gets the lifeboat into the water, but they are evaluated for breaches of safety. Everything comes together in question 4: the outcome of this test question shows which students (individually) have fully succeeded in correctly and safely carrying out the procedures for launching lifeboats, as befits a real abandon ship safety exercise. To answer these questions, the different (sub)tasks in the exam sheet were divided into three categories: minor, normal and major.
-
A minor task is a task or step that, if the student does not perform it or performs it incorrectly, will still lead to a yes answer to the question. The student will still succeed in their goal.
-
A normal task is a task or step that, if the student does not perform it or performs it incorrectly, can under certain circumstances lead to the answer no to the question. Depending on the circumstances, this error may lead to a negative result, but not always. For example, skipping the step ‘checking landing zone clear of obstructions’ will not necessarily lead to a failure to launch the lifeboat into the water, unless there is a boat, barge or other obstruction alongside the ship during the exercise.
-
A major task is a task or step that, if the student does not perform it or performs it incorrectly, always leads to the answer no to the question. A single major mistake immediately leads to a negative result.
To generate a score out of 100 for each student’s performance on the test questions, weighting to the (sub)tasks was assigned as shown in Table 1. For every test question, the sum of all major tasks counts for 50%. Because there are only three minor errors in the entire procedure for question 4, it was decided to use the distribution 10%–40%–50% for this question; otherwise, this would lead to an abnormal weight distribution. Through this quantitative analysis of the results, we hope to provide insight into the differences in procedural performance between the groups. On the one hand, this is achieved by using a binary approach to the results of the test questions to see how many students passed or failed per group. Major tasks are crucial to achieve a positive answer to the test questions and thus a major error condition (MEC) has been built in. This means that if a major error is made in the procedure, this immediately leads to 0 out of 50 for this part and the student will not have passed the test. On the other hand, the results are weighted to make a more in-depth analysis of the procedure execution by means of categorising (sub)tasks. Moreover, by splitting the entire lifeboat procedure into different sub-questions, an attempt is made to analyse where differences occur between groups in the procedure. In the binary approach, the MEC is necessary to know who passed or failed. However, analysing the percentage results with MEC gave large extremes in scores. Additionally, using the MEC, the number of major mistakes made by a student has no impact to their percentual result. After making a single major error, no more points can be earned with ‘major’ categorised tasks. Thus, analysing differences between groups in percentual results is consequently done without the MEC, to give us an idea of the extent to which the student almost succeeds, whereby a weight is given to certain actions in the procedure. To analyse the binary pass/fail results of the students between groups for all the test questions, the Fisher Exact test is used. Subsequently, Kruskal–Wallis and post hoc Mann–Whitney U tests are used to reveal possible significant differences in student scores without MEC between groups.
Weight distribution for task categories test questions
3. Results
3.1. Total Score
Because the number of subjects per training group is rather small and the data are not normally distributed, the non-parametric Kruskal–Wallis test is used for an analysis of the original 40 points score table. The results show that there is a significant statistical difference in Total Score between the different training groups, H (2, n = 38) = 15.66, p <.001. Consecutive Mann–Whitney U tests indicate that the differences between group VR and group C (U (N VR = 13, N C = 12) = 14.00, z = −3.50, p < .001), and between group VR+ and group C (U (N VR+ = 13, N C = 12) = 19.00, z = −3.22, p = .001) are statistically significant. As can be seen in Figure 3, the groups VR (Md = 34) and VR+ (Md = 34) scored better than group C (Md = 24). The comparison of group VR and VR+ is not significant U (N VR = 13, N VR+ = 13) = 60.50, z = −1.25, p = .021.
Total Scores of the students (n = 38) for the three different training groups VR+ (students who received additional IVR lessons), VR (students who only received IVR lessons) and C (students who received traditional practical lessons).
3.2. Test question 1
For test question 1 (and for all the following test questions), the Fisher Exact test is used to analyse the binary pass/fail results. In total, 30 of the 38 students, of which 12 VR+, nine VR and nine C students, obtained a positive result on test question 1. The results of the Fisher’s exact test (p = .376) do not indicate a significant association between training groups and the number of students who passed or failed test question 1.
However, a Kruskal–Wallis test does reveal a significant difference in scores without MEC between the different groups, H (2, n = 38) = 14.09, p <.001. Mann–Whitney U tests show a significant difference between group VR+ and group C (U (N VR+ = 13, N C = 12) = 22.50, z = −3.02, p = .003), as well as between group VR and group C (U (N VR = 13, N C = 12) = 14.50, z = −3.46, p < .001), but there is no significant difference between group VR and group VR+, U (N VR = 13.00, N VR+ = 13) = 71.50, z = −.67, p = .504. In Figure 4, scores in the VR+ group (Md = 91.83) and VR group (Md = 90.50) are significantly higher compared with the C group (Md = 63.33).
Scores of the students (n = 38) for the three different training groups for test question 1 which examines whether the student can independently launch the lifeboat into the water.
3.3. Test question 2
In Table 2, we read that nine VR+ students, five VR students and only one C student passed test question 2, the other 23 students failed. The Fisher’s exact test shows a significant association between training group and the number of students who passed or failed, p = .007.
Pass/Fail versus training group for test question 2 which examines whether the student can independently launch and free the lifeboat
The Kruskal–Wallis test shows a significant difference in scores without MEC between training groups, H (2, n = 38) = 15.50, p < .001. Mann–Whitney U tests show a significant difference between group VR+ and group C (U (N VR+ = 13, N C = 12) = 20.00, z = −3.16, p = .002), as well as between group VR and group C (U (N VR = 13, N C = 12) = 11.00, z = −3.65, p < .001), but there is no significant difference between group VR and group VR+, U (N VR = 13, N VR+ = 13) = 72.00, z = −.64, p = .521. As we can see in Figure 5, scores in the VR group (Md = 88.67) and the VR+ group (Md = 90.08) were significantly higher than the scores of the C group (Md = 69.5).
Scores of the students (n = 38) for the three different training groups for test question 2 which examines whether the student can independently launch and free the lifeboat.
3.4. Test question 3
In Table 3, we see that only two C students and two VR students passed test question 3, while nine VR+ students passed. Considering only the safety aspect in the procedure, the Fisher exact test shows a statistically significant association between training group and the number of students who passed or failed, p = .008.
Pass/Fail versus group for test question 3 which examines whether the student does not endanger safety
Kruskal–Wallis tests show a significant difference in scores without MEC, H (2, n = 38) = 14.08, p < .001, and in the number of major errors, H (2, n = 38) = 11.22, p = .004, between training groups. Post hoc comparisons with Mann–Whitney showed that significant differences in scores without MEC are situated between groups VR+ and C (U (N VR+ = 13, N C = 12) = 15.00, z = −3.43, p < .001), and groups VR and C (U (N VR = 13, N C = 12) = 32.00, z = −2.50, p = .012). There was no significant difference between groups VR and VR+, U (N VR+ = 13, N VR = 13) = 49.50, z = −1.80, p = .072. In Figure 6, we see that the training groups VR+ (Md = 91.67) and VR (Md = 82.06) scored significantly higher than group C (Md = 67.70).
Scores of the students (n = 38) for the three different training groups for test question 3 which examines whether the student does not endanger safety.
The number of major errors for test question 3 was found to be significantly lower in the VR+ group (M = 0.46, SD = .78) than in the VR group (M = 1.77, SD = 1.36), (U (N VR+ = 13, N VR = 13) = 34.00, z = −2.73, p = .006) and in the C group (M = 2.42, SD = 2.11), (U (N VR+ = 13, N C = 12) = 25.00, z = −3.05, p = .002). There was no significant difference in the number of major errors between training groups VR and C, U (N VR = 13, N C = 12) = 64.50, z = −.75, p = .453.
3.5. Test question 4
Of all 38 participants in the research project, only 10 students passed test question 4. In Table 4, we see that the distribution amounted to eight VR+ students (62 %), 2 VR students (15 %) and 0 C students. The Fisher exact test shows that there is a strong association between training group and succeeding test question 4, p = .001.
Pass/Fail versus group for test question 4 which examines whether the student can independently and safely launch and free the lifeboat
The Kruskal–Wallis test shows a significant difference in scores without MEC between training groups for test question 4, H (2, n = 38) = 15.47, p < .001. This is also the case for the number of major errors, H (2, n = 38) = 12.04, p = .002. The post hoc Mann–Whitney tests show that significant differences in scores without MEC are situated between training groups VR+ and C (U (N VR+ = 13, N C = 12) = 19.00, z = −3.21, p = .001) and between VR and C (U (N VR = 13, N C = 12) = 16.50, z = −3.35, p < .001). There is no significant difference in scores between VR and VR+, U (N VR+ = 13, N VR = 13) = 53.00, z = −1.62, p = .106. As can be seen in Figure 7, scores in the VR+ group (Md = 90.35) and the VR group (Md = 87.45) were significantly higher than the scores of students in the C group (Md = 66.67). As indicated on Figure 8, the number of major errors was found to be significantly lower in the VR+ group (M = 1.31, SD = 5.53) than in the VR group (M = 2.15, SD = 1.52), U (N VR+ = 13, N VR = 13) = 45.50, z = −2.07, p = .038 and in the C group (M = 4.42, SD = 3.78), U (N VR+ = 13, N C = 12) = 20.50, z = −3.20, p = .001. There was no significant difference in the number of major errors between training groups VR and C (U (N VR = 13, N C = 12) = 43.00, z = −1.95, p = .052).
Scores of the students (n = 38) for the three different training groups for test question 4 which examines whether the student can independently and safely launch and free the lifeboat.
Number of major errors made by the students (n = 38) for the three different training groups for test question 4 which examines whether the student can independently and safely launch and free the lifeboat.
4. Discussion of results
The strength of this study lies first in the fact that the actual procedural and safety behaviour on an individual level was assessed effectuating a real lifeboat launch with a physical lifeboat and davit installation. Hamilton et al.’s (Reference Hamilton, McKechnie, Edgerton and Wilson2020) review showed that in most studies in IVR training so far, there was an over reliance on the multiple-choice question, where for 62% of the investigated studies, it was the only method of assessment. This is a consistent conclusion in other reviews, such as that of Checa and Bustillo (Reference Checa and Bustillo2019) and Matovu et al. (Reference Matovu, Ungu, Won, Tsai, Treagust, Mocerino and Tasker2022). The highly relevant study by Jung and Ahn (Reference Jung and Ahn2018), who previously investigated the effect of VR on procedural skills training for launching lifeboats, also limited the evaluation to a description of actions and restricted implementation thereof. The results of Jung and Ahn (Reference Jung and Ahn2018) showed no significant differences for procedural knowledge, while for practical skills, the HMD/sensors group performed significantly better. This contradicts other positive VR research results for procedural skill acquisition, which had led to an initial tentative conclusion that pre-practicing procedures in IVR allows transfer to the real world (Hamilton Reference Hamilton, McKechnie, Edgerton and Wilson2020). It can be argued that the way Jung and Ahn (Reference Jung and Ahn2018) evaluated procedural knowledge was most ideal, since procedural knowledge is defined as knowledge about how to do something and reveals itself through behaviour, rather than conscious memory (Makransky and Petersen, Reference Makransky and Petersen2021). The actual launching of a lifeboat is an impressive and exciting event, and an operation prone to human error, where making a mistake can have serious consequences, even resulting in loss of life (Akyuz, Reference Akyuz2016). The procedure is extensive and to perform this flawlessly without any professional experience is not self-evident. Even for experienced seafarers, this operation always entails the necessary caution and even stress (Shen and Huang, Reference Shen and Huang2011). It should be emphasised that there is a difference between a descriptive methodology and having to perform actions in a stressful situation. Moreover, research shows that even though intentions predict behaviour, people do not always act on their intentions (Fishbein et al., Reference Fishbein, Hennessy, Yzer and Douglas2003). Second, the advantage of this study is that it was conducted in the context of an existing Nautical Science and Marine Engineering education program to investigate the effect of IVR on procedural performance in maritime education, which is not regularly done in other research (Radianti et al., Reference Radianti, Majchrzak, Fromm and Wohlgenannt2020). In addition, the results of an IVR training group were compared with the results of both a traditional hands-on learning group and those of a group in which traditional lessons were supplemented with IVR training.
As explained earlier, it is unfortunate that the sample size in this study is rather limited. However, this constraint is inherent to the context of maritime education at our institution. Our academy has a small student body, making large-scale studies impractical. Furthermore, integrating research into an official academic program posed logistical challenges, as students needed to complete their coursework within strict time frames, ensuring equal opportunities before final exams. The practical lifeboat test itself presented additional constraints: it required individual execution, involved specialised equipment susceptible to wear and tear, and carried inherent safety risks, significantly limiting the number of trials. Lifeboats are designed to save lives, but over the years, many serious casualties have occurred during drills and launching a lifeboat remains a high-risk operation (Gard, 2022). Not surprisingly, other similar studies with lifeboats, such as that of Billard et al. (Reference Billard, Smith and Veitch2020), face the same problems which results in small sample sizes. This underscores the need to compare and add results in future similar research. Despite the size of this study, the sample is diverse enough to provide meaningful insights into the feasibility of IVR-based lifeboat training and the statistical methods applied were specifically designed for small sample groups, ensuring that the findings remain robust and methodologically sound.
The most relevant comparison of results to be made is that between the main groups VR and C. The initial Total Score analysis, which looks purely at the number of questions on which the students were assessed positively by the examiner, already reveals important differences between the two training groups. Students who only followed the VR training scored better than students who followed the real-life practical lifeboat training. When looking further at test question 1, there was no significant difference between group VR and C in the number of students that succeeded in getting the lifeboat in the water. The vast majority of students was able to pass test question 1. However, the scoring results do show a significant difference in the procedural performance for this first chapter in the lifeboat launching procedure as the VR students outperformed the C students. Of all the consecutive test questions that were analysed, question 1 is the least relevant. Getting the lifeboat in the water is a good first step in the overall lifeboat procedure, but if an individual is unable to free the lifeboat from the ship and manoeuvre it away, this is not much use. Test question 2 includes the entire lifeboat launching procedure, but like test question 1, still does not take the safety aspect into account. We can clearly see that the next stage in the procedure, which is opening the hooks and freeing the lifeboat, proves to be a greater challenge for most students. Only five VR students and one C student succeeded test question 2, but again the VR group performed better than the control group in terms of procedural correctness. For test question 3, only two students from both groups VR and C, passed. Also here, there is no significant difference in terms of number of students passing and the number of major errors made in the procedure, but there is a significant difference when looking at the scores without the MEC. The VR students still outperformed the students who received the traditional hands-on training. This means that overall, VR students scored better in pursuing a higher degree of safety when performing a lifeboat launching exercise. The findings of test question 4, which evaluates the entire lifeboat launching procedure including the degree of safety, are similar to those of test questions 2 and 3. There was no significant difference to be found between the number of students who managed to successfully perform an entire lifeboat drill, meaning being able to lower the boat, open the hooks, start the engine and free the lifeboat from the ship, without compromising safety. However, when looking at the scores evaluating the extent of procedural performance, the VR group scored significantly better than the traditional lecture group C.
The above consistent results show that solely IVR training can be used to learn lifeboat launching procedures in maritime education. Students who only receive the IVR lifeboat training even outperform students who receive the lessons physically with a real lifeboat in a traditional practical training setting. In general, the results achieved by the students in the control group were rather disappointing. Not a single student who received the usual traditional practical lessons succeeded in completely launching the lifeboat in the water in a safe manner. If we consider the safety aspect alone, we see that only two out of 12 traditional lecture students do not commit major offences. These results are confrontational, especially since the importance of safety compliance is strongly emphasised in both the theoretical and practical lessons at AMA. To clarify, the lessons the group C students received in this study are the actual lessons as they take place today in the official curriculum. This gives rise to a critical questioning of practical safety lifeboat courses as they are now, organised at various maritime institutes. The training given at AMA meets the current international Standards of Training, Certification and Watchkeeping (STCW, 1978 as amended) (International Maritime Organization, 2012) and pedagogical standards. However, these research results show also that it may not be as opportune to transfer procedural skills in larger practical training groups where students cannot perform all the different psychomotor skills on an individual level. This is the first time that an individual practical test on lifeboat launching has been conducted in this particular way at AMA and also, at other maritime institutions, students are not individually being assessed in this particular way. This is because it is practically impossible to have all first-year students individually perform a full lifeboat launch. As a result, students are being evaluated on their practical skills on a permanent basis during practical classes. It can however be nuanced that novice cadets on board should never perform a lifeboat drill alone and that it would be quite exceptional to have to launch a lifeboat without a superior officer and other crewmembers. Additionally, during their first voyages aboard, cadets should still continue their practical training in line with a cadet training book to become a fully licensed officer of the watch.
Of course, it is rather expedient to assume that practical safety lessons will be replaced by IVR in their entirety, so it is certainly interesting to explore in the future how IVR can positively contribute to existing practical lessons. The intention in this research to create a VR+ group in addition to a VR and C group was a first initiative to analyse whether the combination of regular lessons supplemented with VR training can offer added value for maritime safety training. Most studies have examined the effects of IVR as the sole method of learning and did not use a combined approach (Hamilton et al., Reference Hamilton, McKechnie, Edgerton and Wilson2020; Lamb et al., Reference Lamb, Etopio, Hand and Yoon2019). The use of virtual reality technologies for blended learning in education remains largely unexplored (Kovalenko et al., Reference Kovalenko, Marienko and Sukhikh2021). However, it must be nuanced that the VR+ participants in this study had double learning time, so we can hardly consider this as a hybrid approach, and it goes without saying that extra learning time contributes to an improvement in learning outcomes. Therefore, not surprisingly, when analysing and comparing the results, the VR+ students had a much higher success rate for the procedural execution of a lifeboat launching exercise for test questions 2, 3 and 4. For test question 2, only a total of 15 students (40 %) succeeded, of which nine students belonged to the VR+ group. For test question 3, which only examines the degree of procedural safety, nine out of 13 VR+ students succeeded in not endangering the safety of themselves and the crew during the launch process compared with only two students in both the VR and C group. This was not the case for test question 1; the vast majority of students (80 %) passed and there was no significant difference between groups in the number of students that succeeded. Although the above findings were to be expected, very surprisingly, this study showed no significant difference between VR and VR+ when examining the scores without MEC for all the test questions. This means that when examining the degree of procedural correctness alone, the students of the VR+ group did not perform statistically better than students in the VR group. This is quite remarkable because it suggests that the 3 h of real, live hands-on training with a lifeboat does not necessarily positively affect learning outcomes when students receive the same learning time in VR. The next step can be to examine how much traditional learning time can be surrendered for VR time and which combined approach would be best. Most other research that examined a combined IVR approach also evaluated the impact of IVR as a supplement to traditional lessons rather than complementing it (Hamilton et al., Reference Hamilton, McKechnie, Edgerton and Wilson2020; Nersesian et al., Reference Nersesian, Spryszynski and Lee2019). The study load of students at AMA is already quite high and it is doubtful whether students are willing to take extracurricular IVR lessons. After the test exam, the students of group C were given the opportunity to try out VR as they did not have the opportunity to do so yet. Although this was an opportunity to better prepare for the official theory exam that was yet to be taken, of the 12 students from group C, only two students registered to make use of this possibility.
Given the results and the high failure rate of students for a safety-critical procedure in this study, it certainly seems opportune to use IVR to assist teachers in lifeboat training. IVR can give students the opportunity to train all psychomotor actions and skills at an individual level which enhances procedural learning (Maeda et al., Reference Maeda, McGee and Marigold2018; Yin et al., Reference Yin, Chng, Wong, Ho, Chua and Chui2020), but also offers the possibility to provide targeted feedback on performance. The ‘XR Lifeboat Learning’ test mode could also be used to conduct tests, helping lecturers to objectively assess student performances. The software generates a score and keeps track of all data, thus giving the possibility of review sessions at a later stage, making it possible to optimise individual student learning paths.
5. Conclusion
This research initially compares learning outcomes concerning lifeboat launching procedures of two main student groups having received different types of practical training integrated in the official framework of the educational program at the Antwerp Maritime Academy: IVR only training (VR) and traditional practical training (C). Additionally, learning outcomes are compared with a third VR+ group as an initiative to analyse whether the combination of regular lessons supplemented with IVR training can offer added value for maritime safety training. Despite the limited sample size, the study encompasses a sufficiently diverse group to yield valuable insights into the feasibility of IVR-based lifeboat training. The statistical methods employed are specifically tailored for small samples, ensuring the reliability and validity of the results, providing essential preliminary evidence on the positive effect of IVR training on lifeboat learning procedures in maritime safety education. Students of the VR group perform better on various points compared with students who receive their training with a physical lifeboat in a traditional practical lesson setting. Although there is no indication of a significantly higher success rate of launching a lifeboat safely in the water, a VR student still obtains significantly higher scores in procedure execution. Additionally, when evaluating the safety aspect alone in the procedure, VR students score significantly better than students from the control group. Certainly, when the traditional practical lessons are supplemented with IVR lessons, students will experience a much higher success rate for the procedural execution of a lifeboat launching exercise. The VR+ students have a higher success rate in launching a lifeboat correctly and safely in the water, achieve significantly higher scores and make significantly fewer capital errors. Interestingly, there was no significant difference in procedural correctness scores between VR and VR+, and students who received hands-on training in addition to VR did not score significantly better, despite the longer learning time. While the sample size may limit broad generalisation, these results strongly indicate that IVR has the potential to enhance maritime safety education and improve learning outcomes, particularly in procedural knowledge and skills. Moreover, this study marks the initial phase of an ongoing research effort, providing a foundation for future investigations with larger participant groups. To really determine how IVR is most suitable as a learning tool or can potentially offer an equivalent alternative to bottlenecks in maritime safety education, considerable quantitative and qualitative research still needs to be done. Acceptability of IVR, effects of impact variables (Makransky and Klingenberg, Reference Makransky and Klingenberg2022), negative aspects and ethical challenges (Spiegel, Reference Spiegel2017) are just a few examples that still need to be investigated. This study focused on a single procedural exercise in a particular domain of maritime safety. Since safety on board seagoing vessels goes beyond lifeboats and is very diverse and extensive, research in other areas of safety training would certainly be interesting.
Acknowledgements
This research was supported by Flanders Innovation and Entrepreneurship (VLAIO) within Research and Development project XR Analytics (HBC.2022.0590). Acknowledgements to Nick Thienpondt, Jesse Rebmann, Jonas Vandeput, Sven De Hauwere, David Braem and Dr. Marc Braem. The authors would also like to thank all participants and Yeppe Slootmans for assisting in this research.
Authors contributions
I.H. – Conceptualisation, investigation, writing – original draft. D.L. – Writing – review and editing, validation. G.P. – Funding acquisition, project administration. A.D. – Software.
Competing interests
The authors declare that they have no competing interests.
