International Journal of Disaster Risk Reduction 96 (2023) 103940 Available online 9 August 2023 2212-4209/© 2023 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). A non-immersive virtual reality serious game application for flood safety training Alessandro D’Amico a,b,*, Gabriele Bernardini a, Ruggiero Lovreglio c, Enrico Quagliarini a a Department of Construction, Civil Engineering and Architecture DICEA, Università Politecnica delle Marche, Ancona, 60131, Italy b Department of Civil, Constructional and Environmental Engineering DICEA, Sapienza University of Rome, Rome, 00176, Italy c School of Built Environment, Massey University, Auckland, 0632, New Zealand A R T I C L E I N F O Keywords: Flood Safety Virtual reality Serious game Disaster risk management A B S T R A C T Various methodologies and technologies have been developed and tested to train communities for natural hazards and investigate human behaviour. The combination of Virtual Reality (VR) and Serious Games (SG) represents a promising solution to expose communities to different hazardous scenarios in a totally safe way and without exposing the testers to any real risks. Previous studies tested VR SG for several different natural hazards and safety training scenarios, but only a few applications have been proposed within the context of flood safety training. Furthermore, comprehensive prototyping works aimed at evaluating VR SG applications in terms of knowledge acquisition, self-efficacy and user experience, are still needed. This work proposes a novel non-immersive VR SG in the context of users’ safety in the event of flooding in the urban built environment, pursuing the users’ safety training. The proposed application is based on several modules, which can be combined to form different storylines and training objectives. The VR SG capabilities are demonstrated here by firstly considering one significant storyline. Results show a significant increase in self-efficacy and safety knowledge after the VR experience, thus suggesting the possibility to exploit it for increasing users’ aware- ness and preparedness. Furthermore, results also demonstrate the existence of similarities be- tween real-world behaviours and VR choices by the tested individuals, thus suggesting how an application of this kind could also be used to support the development and validation of flood evacuation simulators. 1. Introduction Floods represent one of the most critical disasters for the Built Environment (BE), and its users are exposed to high possibilities of flood risk because it can have different origins: groundwater, river (fluvial), surface water (pluvial), estuary/coastal (tidal), or from sewer sources. Urban floods are usually generated from a complex combination of causes, both meteorological and hydrological ex- tremes, such as extreme precipitation and flows [1]. Building resilience to these changing conditions has become a common goal for different disciplines concerned with the protection of the BE and the safety of users [2,3]. The safety of the BE users during an urban flood depends on the flood dynamics into the BE layout and on the users’ interactions with the floodwater levels (i.e. the depth and speed of the water can reduce the users’ evacuation speed and pose problems for their * Corresponding author. Università Politecnica delle Marche, Department of Construction, Civil Engineering and Architecture, Ancona, 60131, Italy. E-mail address: alessandro.damico@uniroma1.it (A. D’Amico). Contents lists available at ScienceDirect International Journal of Disaster Risk Reduction journal homepage: www.elsevier.com/locate/ijdrr https://doi.org/10.1016/j.ijdrr.2023.103940 Received 10 May 2022; Received in revised form 3 February 2023; Accepted 8 August 2023 mailto:alessandro.damico@uniroma1.it www.sciencedirect.com/science/journal/22124209 https://www.elsevier.com/locate/ijdrr https://doi.org/10.1016/j.ijdrr.2023.103940 https://doi.org/10.1016/j.ijdrr.2023.103940 http://crossmark.crossref.org/dialog/?doi=10.1016/j.ijdrr.2023.103940&domain=pdf https://doi.org/10.1016/j.ijdrr.2023.103940 http://creativecommons.org/licenses/by-nc-nd/4.0/ International Journal of Disaster Risk Reduction 96 (2023) 103940 2 stability) and the BE elements [4–7]. Pedestrian evacuation is one of the main strategies used today to reduce the impact of floods on BE users, who are asked to move towards safe and raised areas situated in the urban layout (e.g. gathering areas according to the urban emergency plan) or upstairs (e.g. to perform "shelter-in-place" strategies) [8,9]. Works on real-world observations and experiments have pointed out the presence of several behaviours during emergency evacuation which can endanger the users due to their inter- action with the BE and the floodwater conditions [4,6,10–15]. In this general behavioural context, promoting correct behavioural responses and understanding the users’ reactions are two of the main steps in providing strategies for users’ and BE risk reduction [16]. The same approach has also been found to support risk reduction in other kinds of emergencies such as fires [17,18] and earthquakes [19,20]. Different methods for behavioural analysis and educational and training purposes have been applied in literature. These methods also include augmented and Virtual Reality (VR) applications [17,21–23]. Both immersive and non-immersive VR allows researchers to increase the possibility of reproducing different dangerous scenarios, thus allowing users to interact with the digital environment without any risk [20,24]. The literature have shown that both immersive and non-immersive VR training tools can help overcome the pedagogical education of traditional training methods [23,25]. In fact, VR training makes it possible to follow all the behaviours and choices made by users during the training, providing them with a report on what they did correctly [20,26]. Furthermore, VR training solutions provide customisation in training depending on the type of users [19,23]. To date, several applications of VR training have been tested in literature. For instance, several immersive and non-immersive VR applications have been developed and tested for fire safety training [27,28], tsunami and earthquake training [20,29,30], aviation training [31,32] and counter-terrorism safety training [31,33], among other types of safety training. Several studies have also compared the effectiveness of VR training with traditional training methods [27,28,34,35]. These studies have highlighted that a fundamental objective of these tools is to enhance users’ knowledge on the safety procedures to follow during a disaster (i.e., knowledge acquisition) as well as several studies have shown that VR training tools (also known as VR Serious Games) perform better that other traditional training tools in retaining these information (i.e., knowledge retention) [27,28]. Further, existing studies have showed that both immersive and non-immersive VR SG can provide increment of users’ self-efficacy which is defined as the in- dividuals’ belief in their capacity to properly accomplish a task [31,36,37]. Previous works underlined the capabilities of non-immersive VR solutions towards this goal [28,31,38], showing that their efficacy is the same of immersive solutions [34]. In this sense, using such non-immersive solutions can be more effective since they can reach more users, performing training activities on a personal computer or smartphone, and also using online applications. VR training tools can provide a unique opportunity to train communities living in high flood risk areas. A few applications have already been proposed in literature focusing on flood safety training. For instance, Sermet and Demir [39] presented a VR framework that creates a realistic 3D gaming environment to increase public awareness of flood risks. Gamberini et al. [40] investigated how co-design methodologies can be used to identify the contents of a VR simulation for river flood emergency training. Fujimi and Fujimura [41] proposed a VR application to promote early evacuation decisions during flood emergencies. However, this research focuses only on evacuation-promoting river design. Finally, Mol et al. [42] used VR to increase risk perception, coping appraisal, negative emotions and damage-reducing behaviour through a simulated flooding experience. These preliminary studies show the feasibility of using VR for flood safety. Recent works pointed out the growing popularity and promises of Serious Game (SG) in the field of Disaster Risk Management (DRM) and related scientific research [43,44]. The primary purpose of SG is an educational one rather than an entertainment-oriented one, since its success lies in the shift in learning methods from lecturer-based to more engaging approaches in which a strong emphasis is put on emotional aspects. Djaouti et al. [43] provided a brief overview of existing taxonomies of SGs by tracing the evolution of definitions adopted over time by the different research approaches. Although the domain boundaries of the field of SGs are still subject to debate, according to all these definitions, the primary purpose of SGs is an educational one rather than an entertainment-oriented one. With the advent of these technologies, the term "Serious Games" has come to be identified with aspects of simulation games, thus being associated with any piece of software that merges a non-entertaining purpose (serious) with a video game structure (game) [43]. The success of SGs lies in the shift in learning methods from lecturer-based to more engaging approaches in which a strong emphasis is put on emotional aspects. In fact, previous works claimed that "Serious games […] have the potential to reach a wide audience and convey reliable and consistent information regarding DRM, installing disaster awareness, portraying hazards and vulnerabilities, and modelling useful skills across all stages of the DRM cycle" [44]. Among the SGs analysed by this research (45 in total), most focus on floods (27), earthquakes (10), and drought threats (7), these being the most common and deadliest natural disasters. In addition, only 14 out of 45 SGs analysed were developed as single-player computer/VR experiences. These SG approaches are mainly similar to quizzes and usually goal-oriented, thus requiring the player to perform a certain action to get a "reward". They offer more complicated decision-making processes, putting players in control. Although some limits in their impact could be related to users’ technology self-efficacy issues [37], previous works widely remarks that only VR SGs offer the possibility of a more realistic and engaging experience [44]. In fact, they give players a sense of control over what is happening on screen, but they also allow them to more easily identify with the "heroes" of the story, contributing to the empathy of the situation and stimulating interest in learning more about disasters. Finally, considering the evaluation of data processed during the use of SGs, few cases offer documented evidence with detailed information using structured surveys, particularly with respect to the stated objectives, resorting instead to simple discussion and participant observation [44]. However, comprehensive prototyping works that test VR SG applications in terms of knowledge acquisition, self-efficacy and user experience are still needed in literature, as well as the adherence between real-world and VR SGs behaviours and choices by users should be analysed to better remark the capabilities of this kind of approach in DRM-related activities. In view of the above, this paper aims at investigating the effectiveness of the combination between SG and non-immersive VR in A. D’Amico et al. International Journal of Disaster Risk Reduction 96 (2023) 103940 3 improving evacuation behaviour in case of flood pedestrian evacuation in urban BE scenarios. A new non-immersive VR SG is then prototyped by including several training objectives for flood safety training and laboratory tests are carried out by comparing the users’ knowledge and self-efficacy before and after the tests. Furthermore, data from VR SG tests are investigated to verify the adherence of tested behaviour in respect of literature ones, thus analysing if substantial differences between simulated and real-world conditions exists and could affect the users’ experience. 2. Materials and methods Fig. 1 presents a flow chart showing the research design process. The first step is the development of the training application according to the concepts of gamification, non-immersive VR, and evacuation training. In particular, gamification lies in the con- struction of individual scenarios, also called "story modules", whose overall articulation builds the "storyline". A beta test activity was carried out with a small group of experts, selected from professors, researchers, doctoral students and students from two universities in Italy (Polytechnic University of Marche and Sapienza University of Rome) and one in New Zealand (Massey University) to verify the applicability of the experiment and receive technical feedback on the simulation. The beta test involved 30 participants. Following the feedback received in the beta test phase, the appropriate changes were made to the SG and then proceeded to experiments with the VR SG. The different components of the VR application are described in Section 2.1. The final application was then tested using pre-test and post-test questionnaires, as shown in Section 2.2). Furthermore, the behaviour of the participants was also investigated, as explained in Section 2.3. According to the aims of the study, the following specific research hypotheses (H) have been defined: • Aim A: evaluate the effectiveness of an VR SG in improving evacuation behaviour, in terms of knowledge and self-efficacy increment: o H A.1: the use of the SG generates an increase in knowledge in the users; o H A.2: the use of the SG generates an increase in self-efficacy in the users. • Aim B: explore human behaviour facing flood pedestrian evacuation in urban BE scenarios, to assess if differences with real-world behaviours could impact the test results: o H B.1: the choice of safe path is defined by signals or rescuers supporting the evacuation process; o H B.2: the interaction with other people affects the evacuation and motion path; o H B.3: the interaction with floodwater levels affects the evacuation and motion path; o H B.4: the interaction with movable and unmovable elements affects the evacuation and motion path. 2.1. Application development According to the general aims of this work, the proposed VR application aims to train users through flood scenarios relying on possible real-world conditions of urban BEs. In this study, the selected scenario is designed according to previous works on real-world and typological BEs affected by floods [45]. In particular, the scenario is consistent with the typological geometric features of Italian urban BE (i.e. historical ones) of riverine cities prone to flood risk. The scheme of a portion of the urban layout is represented in Fig. 2. The scheme is characterised by a square, some streets connecting the riverfront to the square, and the buildings that define the boundaries of these open spaces. The scenario Fig. 1. Research design scheme. A. D’Amico et al. International Journal of Disaster Risk Reduction 96 (2023) 103940 4 comprises the modelling of these outdoor spaces (see, for instance, Fig. 5), and two indoor environments (to simulate the start of the process from a building and a gathering area located inside a building of the square; see Fig. 13). The floodwater levels of the scenario can be changed to analyse the users’ reaction to the flooding risk by providing them with different training scenarios [45]. In particular, in this work, we assumed a floodwater depth equal to 40 cm with a simulated walking speed of about 0.8 m/s, according to Bernardini et al. [46]. The following subsections offer the specific steps in the VR SG application modelling and development involving this scenario. 2.1.1. Training objectives The first step relates to the definition of training objectives to analyse. According to literature, the best responses, in terms of learning, occur when you have defined objectives in training [47,48]. At the same time, analysing responses to specific behaviours is one of the specific aims of the research in terms of simulating human behaviour in the event of a disaster. The main training objectives have hence been defined according to the behavioural response of VR SG participants, who will act as pedestrians in the flood emergency: a) the path choice: the participants should have the possibility of knowing the actual height of the water for the different alternative paths (thanks to, e.g. depth indicator poles or objects with commonly known dimensions such as cars) [4], or with a wayfinding system (both for indoor and outdoor scenarios) [14]; b) the choice of a safe area: the participants could perform "shelter-in-place" strategies by moving towards higher levels of the building where they are located, or they could move towards gathering areas and shelters provided by emergency plans [14,49]. In this second case, the users could choose a gathering area located in a building (thus reaching the building and then moving upstairs), or located outdoors (e.g. a raised gathering area), or even prefer to select a raised outdoor position perceived as safe (including, for instance, a bench) and stay there until the level becomes critical or until the rescuers’ arrival [4,9]; c) the interaction with obstacles: the presence of mobile or fixed obstacles on a defined path enables users to choose whether or not to avoid them (e.g. cars or small objects in the flow) or move towards them if perceived as possible shelters (i.e. grab bars, raised areas, benches) [4]; d) the interaction with other people (i.e. social interactions): other individuals, modelled as Non-Player Characters (NPCs), could slow down the evacuation of the participants (in case the participants choose to help them), or act as a facilitator element (in case of the presence of a rescuer or a leader during the evacuation) [4]. 2.1.2. Story modules The story modules are defined on the basis of training objectives as defined in Section 2.1.1. Each story module is planned to analyse the participants’ behaviour regarding one or more training objectives. The structure in story modules hence can: • analyse a limited number of human behaviours in emergency response to limit the possibility of confounding factors; • create interchangeable modules within general storylines. This allows researchers to analyse the results of a single module in a direct and specific manner, even if inserted in different storylines. 5 story modules are elaborated as shown in Fig. 3, linking the related identification code ID to the specific story module conditions and training objectives as defined in Section 2.1.1: ID1. Inside a room on the ground floor of a building: this could represent the start of the indoor scenario, and the participants could choose whether to go downstairs, upstairs, or outside. The possibility of going downstairs will lead to "game-over" conditions due to the wrong choice of an unsafe place. The possibility of going upstairs depends on the availability of a higher level, and, in this work, this possibility is not considered so as to force people to move outdoors and investigate related evacuation behaviours. The possibility of going outside connects to another story module; ID2. Street with obstacles: participants need to proceed in the street with different types of obstacles (fixed and mobile ones); ID3. Street with other pedestrians: participants need to proceed in a street where other users (NPCs) are present. These NPCs could interact with the participants as help seekers (i.e. rescuers such as policemen) or pedestrians who are evacuating; ID4. Intersections: participants need to choose the best path also according to the depth indicator poles and objects present at the Fig. 2. Scheme of the base scenario for VR application. Built environment dimensions: square [45 × 75 m]; building block [33 × 67 m]; streets parallel to the river [4 m]; streets perpendicular to the river [6 m]. A. D’Amico et al. International Journal of Disaster Risk Reduction 96 (2023) 103940 5 intersections; ID5. Square: participants need to move in an open space where obstacles and other users are present. Two possibilities are considered: reaching a safe area within the square; or getting inside a building and reaching a higher level. Fig. 3. Scheme of story modules linked to training objectives and possible application in storylines. *: in the indoor story module, the possibilities of going upstairs and downstairs are not presented at the same time: in the first story module, participants can make a choice between going outdoors or downstairs, while there is the possibility of going upstairs in the last story module, after getting inside a building from the square story module. A. D’Amico et al. International Journal of Disaster Risk Reduction 96 (2023) 103940 6 2.1.3. Modelling strategy The story modules defined in Section 2.1.2 are then modelled using the Unity game engine (version 2020.3.10f1). The exposed methodology fits both Immersive VR and non-immersive VR. The modelling involves all the elements of the BE (such as buildings, roads, signs, benches), the NPCs (i.e. the avatars of pedestrians who are evacuating, with their animations), as well as the water element to simulate the flood scenario. Each story module (see section 2.1.2) has been modelled as a separated scene in Unity, scaling the dimension to the whole BE scenario (see section 2.1.1). Following the example of Krishnan [47], some pre-built assets have been used, both paid as well as free assets from the Unity Assets store, for specific purposes: • to recreate the urban BE: "Urban construction" pack, developed by Quantum Theory and "Urban City Pack", developed by PolyPixel: pre-built elements of the built environment (buildings, streets, etc.); • to simulate flood in VR: "WaveMaker", developed by Lidia Martinez. Given the complexity of floodwater spreading simulation as well as considering the solutions of most of the previous works [41,50,51], the modelling strategy is based on a graphical simplification of a sort of flowing plane that rises from the ground level to a defined level; • to improve the "flood-immersive" experience: o "AllSky Free - 10 Sky/Skybox Set", developed by rpgwhitelock, for the setting of the skybox with a storm-like sky; o "Rain Maker - 2D and 3D Rain Particle System for Unity", developed by Digital Ruby (Jeff Johnson), to reproduce the audio-visual sensation of rain thanks to specific scripts and textures; o "Simple Water Shader URP", developed by IgniteCoders, to render the water in a more detailed way; o "Fast Buoyancy", developed by Nathan Gauër, to simulate floating obstacles in the flooded street thanks to specific objects and scripts. The model of the BE based on a real case study could be developed in several ways. The most used methodologies are based on BIM modelling [20], simpler 3D models, i.e. from Sketchup [50], or directly modelled into Unity with the use of pre-built assets [47]. In this research, the basic geometries have been modelled inside the BIM Authoring Software Autodesk Revit to respect the dimension and proportions of HBE scenarios (Fig. 4), and then textures and more complex models have been applied inside Unity, with the use of the aforementioned pre-built assets (Fig. 5). Several scripts in C# language have been used to develop the models of the story modules. Some of them came from the described "assets", and some others came from specific scripts elaborated for the experiment. The scripts were elaborated in VisualStudio 2019. The following scripts have been elaborated for the simulation: • Player Controller and Mouse Handler to ensure the First Person Movement; • Story Module Control, to pass from one story module to another one. The script is based on switching between levels in a video game. It is applied to a virtual plan that marks the physical boundaries of the BE of the single-story module; • Position tracking to record the times and movements of the participants in each single-story module in order to analyse behaviour in the virtual flood risk scenario. For the NPCs, we use models generated by Mixamo [52], powered by Adobe, where the avatars of the people and related ani- mations were selected (i.e., the injured walking). Two types of NPCs are considered in this work: (A) single NPCs moving linearly to safe areas; and (B) small groups of NPCs calling for help. For the latter, the scene also includes a disclaimer that warns participants to move after having helped people. The disclaimer is activated when the participants enter the area of influence of the groups of NPCs. As Fig. 4. Modelling of the scenario in Unity. A. D’Amico et al. International Journal of Disaster Risk Reduction 96 (2023) 103940 7 for the sound, different aspects have been used to simulate: environmental sound (i.e. the rain) to create a more realistic environment, punctual sound sources such as the voice of the policemen’s avatars and police car sirens to draw the user’s attention to a specific point (safe areas). Finally, in this work, a non-immersive system has been adopted for experiments. The general efficacy of non-immersive solutions is the same of immersive solutions [34], but they allow reducing complexities in test performing while applying it to a significant number of participants. In such terms, this choice also allows us to focus on the prototyping of the SG, which is one of the mail research goals, and thus to preliminary understand how users could perceive the experience and if evacuation behavior have been improved by training activities. Tests are carried out on a laptop mounting Intel Core i7-8750H CPU (@ 2.20 GHz) - 32 GB RAM, and Graphics card NVIDIA GeForce RTX2070, Windows 10, and using a Philips V LINE monitor LCD FULL HD (23.8 inchs, 1920 × 1080 pixel). However, these specifications are not related to minimum ones, since other preliminary tests have been carried out on other less performing laptops (e.g. i5-6200, i7-4750HQ and i7-11800H CPU; 8 and 16 GB RAM; NVIDIA GeForce GTX960 M and AMD Radeon RS M330 GPU; Windows 8, 8.1 and 11) guaranteeing that the VR application can be easily run in common PC configuration currently available. Furthermore, the non-immersive approach could ensure easy-to apply safety and health standards for participants also in view of possible sanitary restrictions (i.e. tests were carried out during the period of the COVID-19 pandemic). An example video of the possible scenarios in the SG has been uploaded to show the extent of immersion accomplished and is available at the following link: https://bit.ly/3YiD3JZ. 2.2. Data collection and questionnaire Different protocols of involvement of participants are used in literature [46,48,53]. Usually, a number higher than 30 participants is recommended for this kind of experiment. If the research includes interest in a retention test, the number should be even higher due to the possibility of no response from involved participants after some weeks. For this purpose, the number of involved trained should Fig. 5. Images of two-story modules: "road with obstacles" (left) and " intersection" (right) modelled within Unity software. Table 1 Example of how the answers to the open questions were evaluated. Questions Participants #1 #2 #3 #4 #5 #6 … #55 1. go out 1 upper floors 1 1 1 1 "panic" − 1 stay signs and indications help people looking for help 1 2. avoid moving obstacles 1 1 shelter on fixed obstacles 1 1 1 1 1 stay 1 move close to the walls water level control change route 1 elevated zone 3. looking for help 1 1 1 1 1 1 help people 1 1 1 4. towards shallow water 1 1 1 1 1 1 help people look at elements 5. safe zone 1 1 1 inside the building 1 avoid moving obstacles 1 help people 1 closest place 1 total 5 7 3 5 5 6 … 5 A. D’Amico et al. https://bit.ly/3YiD3JZ International Journal of Disaster Risk Reduction 96 (2023) 103940 8 be higher than 50. Participants are usually volunteers, randomly chosen to be a representative sample of the population, divided by gender and age. As an example, Lovreglio et al. [27] recruited volunteers in Auckland (New Zealand) at the Albany campus of Massey University and in several public libraries. Fujimi and Fujimura [41] recruited the volunteers at University, 103 students from the University of Tokyo. The common procedures in literature rely on two evaluation tests to assess participants’ knowledge before and after the test [54]. This method enables the evaluation of the training efficacy of the elaborated method. To compare the effectiveness of the SG experience, two evaluation tests are designed to evaluate participants’ knowledge acqui- sition, namely pre-test (before the training) and post-test (immediately after the training). The evaluation tests are elaborated as questionnaires, with both close-ended and open-ended questions [27]. The pre-test questionnaire includes three parts: 1. Collection of background information on participants’ age, their previous experience and training with evacuation drills, as well as their gaming experience; 2. Assessment of the participants’ awareness of flooding risk; 3. Focus on the operational steps in which the participants had to describe how they would behave in the different scenarios described (the scenarios described coincide with the story modules). The first and the second parts are elaborated with close-ended questions, while the third part is made up of open-ended questions. The post-test questionaries focus on knowledge acquired during the training due to the evacuation and emergency procedures and self- efficacy levels. The post-test questionnaires are structured to have the exact same parts as the pre-test questionnaire, except for the first part on previous background data, in order to compare the results before and after the SG. In the post-test questionnaire, we add a part concerning the involvement in the simulation and the feedback on the experience of VR SG. Both close-ended and open-ended questions are used. In open-ended questions, the participants need to describe the possible behaviours in an emergency during a flooding risk related to each training objective defined and story module described. The open- ended questions are used to avoid prompting possible answers or limited responses, as in the case of close-ended questions. The open- ended questions are processed through the identification of keywords within the answer and are coded to provide a score (+1 for positive behaviours, i.e. I go upstairs, looking for a policeman; and − 1 in the case of negative behaviours, i.e. I stay put, “panic”). Table 1 shows the coding created by the researchers. The other data are collected through close-ended questions and using a Likert scale, ranking them from 0 to 6 (0 = strongly disagree and 6 = strongly agree). Google form was used to set up the tests and make them available to participants. 2.3. Data analysis The results of the double-questionnaire used in this study are analysed using statistical testing. Participants’ knowledge before and after as well as self-efficacy are compared using u-test or t-test, depending on the normality of the data. The statistical tests are carried out using SPSS Version 27.0 statistical software package. The analysis of participants’ trajectories is performed to detect the usage levels of the open spaces during the evacuation with respect to elements in the BE, such as movable and unmovable elements (i.e. barrels, cars, building walls, benches), safe areas, other people, including rescuers. A 1 m × 1 m grid is selected to discretise the space according to the ranges of distances between unmovable elements in literature [12], being also consistent with the maximum avatar diameter. Trajectories can also point out literature-based phenomena noticed in: (1) real-world observations from flood events, flood evacuation modelling, common behaviours in evacuation [4,9,12,13,15,55,56]; and (2) VR movement depending on the methods for the translational component plus head movements for rotational components, to detect differences with real-world scenarios [57,58]. The percentage of participants in reference to their final position is assessed considering "game over" (GO, that is unsafe) or the arrival in safe areas codified in the evacuation plan. Furthermore, answers to the open questions in the post-test survey are analysed to qualitatively point out behavioural issues in tra- jectories and final positions analysis. Evacuation times are investigated by providing a boxplot representation to assess the overall effects on the time of permeance into the VR outdoor environment. Kolmogorov-Smirnov tests are carried out to assess if evacuation times had a normal distribution [59]. Matlab R2021b is used to perform these analyses [60]. Fig. 6. Graphs describing the sample of volunteers who took part in the experiment. A. D’Amico et al. International Journal of Disaster Risk Reduction 96 (2023) 103940 9 3. Results The results of the testing are presented here in three main sub-sections. The description of the participants is provided in Section 3.1, the awareness assessments and SG training perception are offered in Section 3.2, while behavioural analyses are investigated in Section 3.3. 3.1. Participants The experiment involved 55 volunteers. The sample is composed of 30 male and 25 female volunteers, with an age range ranging from 18 to 68 years. The first interesting fact is the poor preparation in terms of sample evacuation exercises (Fig. 6). 35% have never carried out evacuation drills, 42% carry them out once a year and only 7% twice a year. The percentages are greater in cases of drills aimed at flood risk, where 96% of the sample has never carried out this type of drill. As for familiarity with video games, 27% never play, 40% are casual gamers (less than once a year, at least once a year or once a month), and 33% are instead regular players (at least once a week or more), as shown by Fig. 6. 3.2. Awareness assessments and SG training perception The first part of the pre and post-test questionnaires with closed-ended questions is organised in the same way to compare the evaluation before and after the SG simulation. The answers provided by the volunteers show a variation from constant disagreement answers (values from 0 to 2) to constant agreement answers (from 3 to 6) for questions 1 to 6, i.e. the questions relating to the self- assessment of ability to deal with a situation of danger, while they have an opposite tendency for questions 7 to 9, regarding self- assessment on the possible consequences of risk and personal vulnerability (Figs. 7 and 8). The means of the responses are characterised by a normal distribution (Kolmogorov-Smirnoff test p-value = 0.2 > 0.05), and the difference between before and after performing the SG showed statistical significance (T-Test, p-value <0.001), thus demonstrating the effectiveness of the experiment. Concerning the H A.1 (“the use of the SG generates an increase in knowledge in the users”) and H A.2 (“the use of the SG generates an increase in self-efficacy in the users”) results confirmed the research hypotheses shown in section 2 (Fig. 9). The involvement in the simulation presents optimal results, with high satisfaction percentages of the learning method, compared to traditional methods. Thus, results remark a good response in the increase of preparedness to face the flood risk. Satisfaction values (equal to scores 4,5,6) in percentages higher than 50% are recorded in all the questions [1. = 81%; 2. = 76%; 3. = 65%; 4. = 54%; 5. = 98%; 6. = 89%; 7. = 94%]. The question with the lowest percentage of positive scores is 4. The question "I could easily deal with the situation foreseen in the virtual game" reaches the cumulative value of 54% between answers with values 4, 5 and 6. This result is to be considered linked to an awareness of the danger of flood risk, for which, despite the improvement in preparation, it is possible that the participants in the experiment still understood the seriousness of the flood risk, not being influenced by the ease of management in the virtual world (Fig. 10). Feedback on the simulation also shows good results (Fig. 11). The first three questions concern the participants’ emotional state with respect to feelings of fear, nervousness and anxiety. In this case, about 77% of the participants do not perceive negative feelings (responses with values from 0 to 2) [1. = 79%; 2. = 78%; 3. = 76%], while 93% of the participants find the SG enjoyable and engaging reporting high values for the fourth question (answers with values from 4 to 6). Questions 5 to 8 concern the technical aspects. 93% of the participants find it easy to interact with the non-immersive VR of the SG. 87% feel that the virtual world was adequately realistic, and 85% think that the flood scenario is realistic enough too. 54% of the participants also consider the interaction with other people (NPCs) in the SG to be adequately realistic (question 8), both considering the rescuers and the other pedestrians (answers with values from 4 to 6), although the majority of answers focused on neutral values, about 33% (answers with a value of 3). This data indicates a Fig. 7. Participants’ answers to closed-ended self-assessment questions on flood risk before and after the SG. A. D’Amico et al. International Journal of Disaster Risk Reduction 96 (2023) 103940 10 possibility of improving the modelling of NPCs and related interactions in the SG. 74% of the participants think that they would behave in real life during a flood exactly as they did in the SG (question 9). The last five questions instead concern the perception of danger in the SG and the severity of the flood. 67% of the volunteers perceived the urgency to act during the scenarios proposed in the SG (question 10 - answers with values from 4 to 6). Questions 11 to 13 regarding the sensation perceived in the scenarios showed more distributed values. The feeling of danger is perceived by 58% of participants in the initial story module inside the flooded building; 52% in the story modules comprising the street with obstacles and people; and just 31% instead in the square (answers with values from 4 to 6). This data seems to point out a decline in the perception of danger as the spaces of the BE become less narrow and wider. Finally, 61% of the participants perceived the flood as serious (question 14 - answers with values from 4 to 6). One of the most interesting aspects that emerges in the open-ended question in the pre-test, concerns the attitude towards the Fig. 8. Boxplot for comparing participants’ responses (for questions number are reported in Fig. 7) to closed-ended self-assessment questions before (black boxplots) and after (yellow boxplots) the SG. Fig. 9. Boxplot for comparison of the pre and post SG averages of the answers to the closed-ended self-efficacy questions. Fig. 10. Boxplot of the evaluations on involvement in the simulation by the participants. Questions: 1. Simulated situations in the virtual game are useful for my safety; 2. Simulated situations will allow me to effectively deal with a flood emergency; 3. After simulated situations in the virtual game, I can greatly reduce the likelihood of injury to myself or others during a flood emergency; 4. I could easily cope with the predicted situation in the virtual game; 5. I found this flood simulation more engaging than traditional training tools (such as evacuation drills, health and safety notions, recommendation brochures and seminars); 6. It was easier to remember the actions to be performed during a flood provided in this simulation versus those provided with traditional training tools; 7. I prefer flood simulation to traditional training tools. A. D’Amico et al. International Journal of Disaster Risk Reduction 96 (2023) 103940 11 presence of a rescuer. The same percentage of participants are engaged in searching for help from the rescuer and in collaborating and providing assistance to the other people for social attachment issues. Post-test questionnaires highlight which behaviours are effectively performed, showing greater adherence to scientific literature with respect to real flood risk situations and providing important information regarding the management of the built environment (Table 2). Road signs appear to be the element of greatest influence, together with the presence of other indicators useful for deducing the level of water depth. Finally, the most recurrent criterion for choosing the area to reach for safety is the presence of a rescuer who Fig. 11. Boxplot of participant simulation feedback. Questions: 1. This experience makes me feel scared; 2. This experience makes me feel tense/nervous; 3. This experience makes me anxious; 4. The game was engaging/fun; 5. I found it easy to play this game; 6. The virtual world was adequate/realistic; 7. The virtual flood scenario was adequate/realistic; 8. Interaction with other virtual people was adequate/realistic; 9. I would behave the same way in real life during the flood; 10. I felt the urge to act/do something during the flood; 11. I felt in danger while I was in the virtual building at the beginning of the simulation; 12. I felt in danger while I was in the virtual street; 13. I felt in danger while I was in the virtual square; 14. I thought that the flood was serious. Table 2 Participants’ answers to open questions on behaviour in case of flood risk before and after the SG. PRE POST go outside go upstairs stay inside/ "panic" other go outside – – Game over (underground) 1. You are inside a room on the ground floor of a building and a flood warning is declared. 18% 59% 12% 11% 85% – – 15% avoid moving obstacles shelter by fixed obstacles stop other avoid moving obstacles skirt walls get away from the river Game over (towards the river) 2. You are walking on a flooded road and there are possible fixed and moving obstacles in your path. 46% 21% 7% 15% 80% 24% 43% 15% seeking help help other people Both other seeking help help other people Both – 3. You are walking on a flooded street and on your path, there is an experienced evacuee (or policeman) and people who need help. 44% 44% 8% 4% 59% 14% 27% – towards shallow water I do not know – other Signal attracted by other people other indicators (poles, cars) Game over (wrong way) 4. You are at an intersection and must choose the safest route. There are some elements that help to understand the depth of the water. 87% 6% – 7% 61% 30% 52% 15% Safe zone Building the closest other Safe zone Building The first I’ve seen – 5. You are in a flooded square. There are various obstacles and people seeking shelter. At the end of the square, there is a raised safe area and around some buildings where you can take refuge. 53% 13% 30% 4% 70% 30% 7% – Signal Police officer Safer attracted by other people 11% 36% 27% 18% A. D’Amico et al. International Journal of Disaster Risk Reduction 96 (2023) 103940 12 attracted the attention of the participants. Similar remarks can be then compared to the participants’ trajectories analysis, as pointed out in Section 3.3. According to the scores assigned to the open-ended questions, the overall values are then compared, and the difference between before and after performing the SG shows statistical significance (Mann-Whitney U Test, p-value <0.001), thus demonstrating the effectiveness of the experiment on the improvement of perception, preparedness and response of the risk of disasters by the sample (Fig. 12). 3.3. Behavioural analysis Fig. 13 shows the outdoor space usage in the VR SG tests (absolute usage frequency), tracing the participants’ trajectories according to a 1 m × 1 m grid spatial discretisation. In this sense, Fig. 13 support findings on the hypotheses concerning human behaviour exploration, focusing on the evacuation and motion paths by showing probable trajectories. Dark blue represents non-used patches of the grid, while red refers to the greater number of times the person moves in the patch over time. Different main areas for interactions between the volunteers and the built environment are marked by different letters in Fig. 13 and shown as views of the VR environment in Fig. 14. The evacuation starting point is the building marked by "s" (Fig. 14-a). Participants stop their evacuation in different parts of the environment (compare with Fig. 13): • one of the following "game over" (GO) areas: indoor, downstairs; the river (Fig. 14-b); one of the streets at the intersection with higher waters levels (Fig. 14-c); • one of the two safe areas: SA1 for the building (the safe area is located on the upper floor - Fig. 14-g); SA2 for the raised outdoor platform (Fig. 14-h). The joint analysis of trajectories and open questions in post-test surveys allows for a resumption of the reasons for the choices of final selected areas, and it confirms the general reliability of VR SG tests in collecting data similar to those of real-world flood scenarios and other behaviours that are common to other kinds of emergencies. In particular, literature behaviours are confirmed in respect of the research hypotheses shown in section 2. Concerning H B.1 (“the choice of safe path is defined by signals or rescuers supporting the evacuation process”) and H B.3 (“ the interaction with floodwater levels affects the evacuation and motion path”), confirmed behaviours regard: I. (virtual) environment exploration to collect information on the emergency conditions, i.e. on the water level and on the urban layout [4,13,14,61]. This behaviour is mainly noticed: - at the starting building exit (Fig. 14-b), where exploration prevents volunteers from rapidly leaving the unsafe position and thus depicts a sort of "curiosity effect"; - at the main intersection (Fig. 14-c), where people approaching spend time evaluating the boundary conditions and then select the proper evaluation path; II. in view of the previous point (I), the effectiveness of elements with well-known dimensions (e.g. traffic poles in Fig. 14-c, rectangle 1; cars in Fig. 14-c, rectangle 2) in supporting the water flows levels and so the identification of proper evacuation direction outdoors [4]. These elements could limit the number of people in GO conditions at the first intersection, as shown in Fig. 15; III. in view of point (I), the effectiveness of wayfinding signs for the identification of proper evacuation direction when no other visible people are present [13,15]. As also remarked in Fig. 1515% of volunteers did not leave the building but decided to remain indoors and reached the stairs moving downstairs. They do not correctly understand the wayfinding sign placed on the door of the stairs (stairs with downward arrows). In fact, they believe they are accessing the stairs, and they could then move upstairs, Fig. 12. Comparison boxplot of the scores for the pre and post knowledge. A. D’Amico et al. International Journal of Disaster Risk Reduction 96 (2023) 103940 13 thus confirming the individual’s desire to move upstairs when possible [8,9]. Then, for participants who exit from the building, a non-emergency sign pointing out the river (Fig. 14-b, red rectangle) prevents them from moving toward the risk source. Fig. 15 shows that no people are involved in GO conditions towards the river. Finally, the sign pointing out the square (Fig. 14-c, rectangle 1) significantly limits the number of participants selecting the wrong evacuation path. Concerning H B.1 (“the choice of safe path is defined by signals or rescuers supporting the evacuation process”) and H B.2 (“the interaction with other people affects the evacuation and motion path”), the interaction with other people and rescuers along the evacuation path is confirmed [4,13,56]. The NPCs attract the volunteers while moving because of social interaction phenomena, for both those placed along the second street (as in Fig. 13, rectangle d; Fig. 14-d, rectangle) and placed on the benches in the square (as in Fig. 13, rectangle f; Fig. 14-f, rectangle 1). Finally, the interaction with movable and unmovable obstacles as investigated by H B.4 (“the interaction with movable and unmovable elements affects the evacuation and motion path”) has been confirmed too [4,6,12]. Movable obstacles are perceived as repulsive elements because of risky elements drawn by floodwaters. Participants prefer to adapt their trajectories (Fig. 13, rectangle a) to move far from barrels (Fig. 14-a, rectangle). They prefer to move again close to the wall of the building (preferred distance of 1–2 m) [12] without crossing the street (width of 6 m). A similar effect is connected with the car placed along the second street of the path (Fig. 14-e, rectangle 2), as shown by the distancing trajectories in Fig. 13, rectangle e. Unmovable obstacles are perceived as attractive Fig. 13. Outdoor space use (1 m × 1 m grid) by volunteers while moving from the starting building (S) to one of the safe areas (SA1: building; SA2: raised platform), expressed in terms of absolute usage frequency. Rectangles with white and dashed borders point out the main outdoor areas affecting motion paths, with the identification code according to Fig. 14. Results support testing hypotheses on human behaviour exploration concerning evacuation and motion paths. Fig. 14. Main elements affecting motion paths (and related main elements in the red rectangles) in respect of the testing hypotheses on human behaviour exploration. Each panel offers their position in outdoor areas according to Fig. 13 plan view: a) starting building, whose exit is marked in red with S [movable obstacles]; b) the road to the river [street sign pointing out the river direction]; c) first intersection with three possible selectable directions [1: street sign pointing out the square direction, and barriers and traffic poles as water level markers; 2: car as water level marker]; d) street parallel to the river [other pedestrians]; e) street parallel to the river [1: bench as an unmovable obstacle; 2: car as potentially movable obstacle]; f) square [1: bench as unmovable obstacle, including a standing pedestrian; 2: other benches as unmovable obstacles]; g) SA1, as the safe area in the building, with the entrance marked in green; h) SA2, as the safe area on the raised platform marked in green. A. D’Amico et al. International Journal of Disaster Risk Reduction 96 (2023) 103940 14 elements, as mainly shown by benches placed along the second street (Fig. 13, rectangle e; Fig. 14-e, rectangle 1) and in the square (Fig. 13, rectangle f; Fig. 14-f, rectangles 1 and 2). In particular, participants moving toward the building walls are believed to move to the next sidewalks to reduce the effects of floodwater depths, as noticed in real-world observations [4]. It is worth noticing that the need to become familiar with the navigation methods could partially influence the aforementioned exploration behaviours as well as the participants’ trajectories, although the preliminary testing scene allows volunteers to get trained with movement rules [18,62,63]. In this sense, differences between real-world observations and VR SG data on trajectories seem to mainly emerge considering the square. In the tests, as shown in Fig. 13, volunteers generally preferred to move in a straight line to where they identified as the best safe area (Fig. 14-g, h). Patches closest to the buildings seem to be less used than the central ones, contrary to what has been done along the streets and in real-world observations [12], essentially because moving in a straight line in the space of the square could be really simple for the volunteers. In addition, some participants change their motion direction in the central part of the square by firstly moving towards SA1 and then switching to SA2. Open questions in the post-test surveys point out that they move to SA1 because they first notice the police car, thus confirming the effectiveness of rescuers’ presence at the scene to collect people in a safe area [4]. Anyway, while approaching, they are not convinced about the safety of their choices since they have no additional information on the building site (e.g. "could I move upstairs once they entered the building?"). Thus, they prefer to move towards the visible raised platform because they are also attracted by the evacuation signage (Fig. 14-h). The attraction due to the visible wide raised platform is generally remarked by the higher percentage of people selecting SA2 with respect to SA1, as shown in Fig. 15. The effects of the police car placed in front of SA1 entrance are also remarked by the evacuation time distribution in Fig. 16. The mean value for SA1 (166 ± 61s) is 12% greater than the one for SA2 (145 ± 22s), and both are characterised by a normal distribution of values according to the Kolmogorov-Smirnov normality test. Although the linear path distances from the starting building "s" to SA1 and SA2 are almost the same, a longer time is needed to coordinate the entry movement towards the SA1 entrance door, bypassing the police car (Fig. 14-g). In view of the above, outliers and Q3 to Q4 values in SA1 evacuation times distributions are both affected by this phenomenon and by attraction due to unmovable obstacles (i.e. building walls and benches) in the square (see Fig. 13, rectangle f and Fig. 14-f). 4. Discussion The discussion is organised around three main themes: 1. modularity of the designed story modules; 2. increase in people’s awareness of flood risk; 3. deriving experimental data on behavioural issues according to the comparison of quantitative results of the experiments versus behavioural analysis in real-world scenarios. Firstly, one of the most innovative approaches tested is related to the modularity of the designed storyline. The subdivision into story modules and their structured organisation can allow greater freedom for the user to explore the BE under risky conditions, presenting a greater adherence to reality. At the same time, the use of these modules allows data to be analysed both from each module and in an aggregate way across the entire storyline. In this sense, this work hence represents an example of an application storyline that could be modified in a more articulated manner. For instance, the users may not necessarily incur in the game over conditions while selecting some route choices, and thus they can continue the evacuation process and just lengthen the route to the gathering area. In this way, retrieved data could be analysed by comparing user behaviour in the same modules but articulated differently in the unfolding of the main storyline. Another possible application is to differentially combine story modules by building completely different storylines and testing human behaviour in the urban BE, starting from different points in the layout. The modelling methodology adopted in this work can support these activities because it uses checkpoints at the beginning and end of a given step, according to common video games techniques. Meanwhile, the use of this technique allows us to punctually analyse the results of the simulations both from the qualitative point of view (feedback of the users regarding the training objectives declared) and quantitative (times and followed paths). The second point relates to the increase of awareness in people with respect to flooding-type disaster preparedness issues. Substantial percentages of participants report positive ratings with respect to the easy interaction in the SG and the information gained during the experiment. They also report that the SG was engaging, and no negative feelings (e.g., anxiety and fear) are underlined. Both par- ticipants’ self-efficacy and knowledge acquisition before and after the training are statistically different, thus demonstrating positive results of the experiment. The use of immersive learning techniques can have a long-lasting effect on participants’ memory, allowing them to intuitively recall their experience in virtual training when needed in the real environment. A limitation of this study lies in the lack of retention data that should have been carried out at least four weeks later, but that was not in the objectives of the research. Similar results are reported in literature, confirming that VR training leads to better results and allows longer knowledge retention than traditional learning methods [25,27]. Concerning the third point, this work demonstrates that VR SG appears to be a valuable tool for deriving experimental data on behavioural issues, as trajectory analysis appears to confirm general trends from real-world observations. From the results, it emerges that the elements that have most influenced the choices in all story modules are those of signage (i.e. directional signs or indications of safe areas) or presence recognisable in an immediate way through the audio and visual input of rescuers (i.e. NPCs that indicated their presence by moving their arms, sirens of police cars and voices of policemen that called the attention of users). These outcomes certainly represent interesting suggestions on which to base considerations to make the urban BE safer., such as, for example, the inclusion of appropriate signage related to safety issues, possibly integrated into the existing tourist information to move around the city. Results also suggest that some elements of the BE which can guide the safety of users should be designed in a more recognisable way, such as water depth indicators. These results could be used to develop, calibrate or validate simulation models for pedestrian flood evacuation by mainly taking advantage of possible differences in pedestrian trajectories in the square and of the percentage of A. D’Amico et al. International Journal of Disaster Risk Reduction 96 (2023) 103940 15 users who cannot reach a safe area. In any case, the participants’ sample dimension should be increased by also varying floodwater levels to account for possible differences in paths, BE features and implemented strategies, thanks to the VR SG modularity. In this sense, different wayfinding systems could be tested, as well as the impact of familiarity issues with the BE and the emergency plan. In this sense, preliminary tests could be carried out by allowing volunteers to freely move in the BE before the flood or making them aware of the emergency plan layout before the simulation starts. Nevertheless, differences between immersive and non-immersive VR should be assessed, also in combination with other locomotion methods. Although non-immersive VR has many application advantages, especially due to its simplicity in widespread applications to citizens [31,34], immersive applications could increase the reality of the scenario experience [44], thus promoting a more engaging response in the volunteers and overcoming the limitations of non-immersive VR SG [39]. Three different solutions should be hence included [28,31,38]: (1) the non-immersive VR, as developed in this contribution, by extending it to other personal devices (and online platforms too), so as to increase the possible engaged audience; (2) immersive VR, with the use of head-mounted viewers, or even of cave automatic virtual environment; and (3) an "immersed" strategy, for example in a swimming pool to simulate with greater realism the evacuation limited by the effects of the presence of water. In view of point (2), VR platforms can be also used, since they allow players unmatched freedom of movement and can also be also set up to include the simulation of fatigue effects due to floodwaters. In view of point (3), there are currently few studies on the use of VR in water immersion [64], and in the literature review they were not found specifically in relation to emergency and evacuation conditions in the event of a flood disaster. A central theme of this last application concerns the possibility of using head-mounted viewers for virtual reality not connected to electricity for the safety of the participants in the experiment (i.e. special VR immersion viewers, such as DIVR Ballast Technologies, or waterproof boxes in which to insert water-proof smartphones). Moreover, beside these VR solutions, additional equipment for objective behavioral analysis could be also adopted in tests, e.g. eye-tracking techniques to focus where and how the participants look for information and support in the virtual environment [65]. Finally, the authors are aware of additional limitations of the study that should be solved by further efforts. The social aspects of the SG use could be further explored, such as statistical analysis on the actual data. Multidisciplinary approach issues to the research should be improved, by more deeply exploring data from statistics and social science standpoint, in addition to the engineering one focused on the built environment. This task represents a further step of the research. In this sense, a possible evolution could be linked to the expansion of the use of the VR SG by envisaging a larger sample of users. The current sample size is sufficient for the aim of this first part of the research, but specific efforts should be provided towards users with different technological self-efficiency levels [37], Fig. 15. Percentage of volunteers by distinguishing the areas where their evacuation ended, distinguished by: game over (GO) areas, in black; and safe areas (SA), in yellow. Fig. 16. Evacuation time boxplot for GO intersections, SA1 and SA2 conditions. A. D’Amico et al. International Journal of Disaster Risk Reduction 96 (2023) 103940 16 thus considering differences with respect to age (i.e. young people confident with games and elder people) and also including vulnerable groups, i.e. elderly. Similarly, further works should differentiate the sample also in terms of risk awareness (i.e. planners, rescue teams, general users). 5. Conclusion The safety of the users in the built environment (BE) during an urban flood depends on the flood spreading into the BE layout and on the users’ interactions with the BE elements. During the immediate aftermath of a disaster, human behaviour focuses on evacuation target selection and how to get there. In the specific case of a flooding disaster, movement is obviously influenced by the presence, level and velocity of the water. Emergency activities and evacuation planning can be supported by awareness-raising processes, such as training campaigns. With this perspective, virtual reality (VR) and Serious Games (SGs) training offer powerful tools to prepare users for disasters and improve their emergency response by providing immersive, engaging, and safe methods of learning. This paper aims at investigating the use of SG in combination with non-immersive VR training to enhance participants’ knowledge and self-efficacy in case of a flood disaster. The proposed prototype of a new non-immersive VR application is based on several modules which can be combined to form different storylines and training objectives. One storyline is tested in this work to assess the efficacy of the training solution and investigate participants’ behaviours. The results show a significant increase in self-efficacy and safety knowledge after the VR experience. Furthermore, the results allow for the generation of new behavioural data which could be used in future flood evacuation simulations, thanks to the general adherence between behaviours and in VR conditions and real-world scenarios. Future applications may concern the use of the SG in immersive VR to test the difference in the behaviour compared to non- immersive VR and "immersed in water" to simulate the movements made more difficult by the presence of water. Regarding immersive VR, further research developments will be related to SG customisation, considering the height, gender, and age of the user, to set the movement speed more specifically in the flood disaster scenario, test other combinations of story modules to define different storylines, and implement water simulation to predict varying depth levels as the disaster phases change. While for "immersed" VR, an interesting finding will be to compare visually induced motion sickness and the difference in behaviour between the participant in VR while standing on the ground or floating in the water. There are currently few studies on the use of VR in water immersion and literature, and they were not found specifically in relation to emergency and evacuation conditions in the event of a flood disaster. Ethical approval According to other similar studies also performed by our research group [40,55] and involving VR tests at the Università Politecnica delle Marche - DICEA department, the ethical issues included the necessity to inform volunteers properly and fully about the exper- iment procedures and the analysed data. No sensitive information was collected, and data were anonymised. All the participants signed a waiver before the tests giving consent to the research group. Author contributions statement Conceptualisation, A.D., G.B., R.L. and E.Q.; methodology, A.D. and E.Q.; validation, G.B., R.L. and E.Q.; SG development A.D., formal analysis, A.D. and G.B.; investigation, A.D. and G.B.; resources, E.Q. and G.B.; data curation, A.D., G.B. and R.L.; wri- ting—original draft preparation, A.D.; writing—review and editing, R.L., G.B. and E.Q.; visualisation, A.D. and G.B.; supervision, R.L. and E.Q.; project administration, E.Q.; funding acquisition, E.Q. All authors have read and agreed to the published version of the manuscript. Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Data availability Data will be made available on request. Acknowledgements Go for IT Project - Research Grant "Flood Risk Assessment, Mitigation and Management in Coastal Cities through a Behavioural- Design Approach", funded by Italian Ministry of University and Research, Special Integrative Fund for Research - Contribution of the CRUI Foundation. A. D’Amico et al. International Journal of Disaster Risk Reduction 96 (2023) 103940 17 Appendix. Questionnaires Items Pre-Test Questionnaire Question Type of question General informations 1 Gender close-ended 2 Age (year) open-ended 3 Nationality open-ended 4 How frequently have you practiced in an evacuation drill? close-ended 5 How frequently have you practiced in a flood drill? close-ended 6 How often do you play video games? close-ended Section 1: Specific knowledge Please state your level of agreement with these statements. (0 = strongly disagree and 6 = strongly agree). 1 I am confident that I am able to effectively deal with a flooding emergency close-ended 2 Thanks to my resources, I know how to manage in a flooding emergency close-ended 3 I would be able to deal with a flooding emergency even if the water level is critical and water velocity does not permit me to move easily close-ended 4 I would be able to cope with a flooding emergency even if I find other people along the way close-ended 5 I would be able to deal with a flooding emergency even if the exit is blocked and the water level doesn’t permit me to open doors and going outside close-ended 6 I would be able to cope with a flooding emergency even if I find objects that may harm me along the way close-ended 7 The consequences of a flooding emergency on my safety would be severe close-ended 8 I would be vulnerable during a flood close-ended Section 2: What if … ? please describe how you would behave in these possible situations (open-ended questions) 1 You are inside a room on the ground floor of a building and a flood alarm is declared. open-ended 2 You are walking on a flooded street and there are possible fixed and moving obstacles in your path. open-ended 3 You are walking on a flooded street and on your path, there are people in need who need help and an experienced evacuee (that is, someone who had previous experiences with flood risk and evacuation)/a rescuer (e.g. a policeman). open-ended 4 You are at an intersection and must choose the safest route. Some items that help you to understand the water depth are presents. open-ended 5 You are in a flooded square. There are several obstacles and people seeking shelter. At the end of the square there is a raised safe area and around some buildings to take refuge open-ended Post-Test Questionnaire Question Type of question Section 1: Specific knowledge Please state your level of agreement with these statements. (0 = strongly disagree and 6 = strongly agree). 1 I am confident that I am able to effectively deal with a flooding emergency close-ended 2 Thanks to my resources, I know how to manage in a flooding emergency close-ended 3 I would be able to deal with a flooding emergency even if the water level is critical and water velocity does not permit me to move easily close-ended 4 I would be able to cope with a flooding emergency even if I find other people along the way close-ended 5 I would be able to deal with a flooding emergency even if the exit is blocked and the water level doesn’t permit me to open doors and going outside close-ended 6 I would be able to cope with a flooding emergency even if I find objects that may harm me along the way close-ended 7 The consequences of a flooding emergency on my safety would be severe close-ended 8 I would be vulnerable during a flood close-ended Section 2: What if … ? (why did you behaved the way you did) please describe why you behaved the way you did during the simulation in the various scenarios (open-ended questions) 1 You are inside a room on the ground floor of a building and a flood alarm is declared. open-ended 2 You are walking on a flooded street and there are possible fixed and moving obstacles in your path. open-ended 3 You are walking on a flooded street and on your path, there are people in need who need help and an experienced evacuee (that is, someone who had previous experiences with flood risk and evacuation)/a rescuer (e.g. a policeman). open-ended 4 You are at an intersection and must choose the safest route. Some items that help you to understand the water depth are presents. open-ended 5 You are in a flooded square. There are several obstacles and people seeking shelter. At the end of the square there is a raised safe area and around some buildings to take refuge open-ended Section3: Simulation engagement Please state your level of agreement with these statements. (0 = strongly disagree and 6 = strongly agree). 1 The simulated situations in the virtual game are useful for my safety close-ended 2 The simulated situations will allow me to effectively deal with a flooding emergency close-ended 3 After the simulated situations in the virtual game, I can strongly reduce the probability of injury to myself or others during a flooding emergency close-ended 4 I could easily face out the situation provided in the virtual game close-ended 5 I found this flooding simulation more engaging than traditional training tools (like evactuation drills, health and safety inductions, recommendation leaflets, and seminars) close-ended 6 It was easier to remember the flooding actions provided in this simulation than those provided with traditional training tool close-ended (continued on next page) A. D’Amico et al. International Journal of Disaster Risk Reduction 96 (2023) 103940 18 (continued ) Question Type of question Section 1: Specific knowledge Please state your level of agreement with these statements. (0 = strongly disagree and 6 = strongly agree). 7 I prefer the flooding simulation over traditional training tools close-ended Section 3: Simulation feedback Thinking about the flooding simulation you were just in, please state your level of agreement with the following statements (0 = strongly disagree and 6 = strongly agree). 1 This experience makes me feel scared/fearful close-ended 2 This experience makes me feel tense/nervous close-ended 3 This experience makes me feel anxious close-ended 4 The game was engaging/fun close-ended 5 I found it easy to play this game close-ended 6 The virtual world was adequate/realistic close-ended 7 The virtual flooding scenario was adequate/realistic close-ended 8 The interaction with other virtual people was adequate/realistic close-ended 9 I would behave the same way in real life during the flood close-ended 10 I felt the urgency to act/do something during the flooding close-ended 11 I felt in danger while I was in the virtual building at the beginning of the simulation close-ended 12 I felt in danger while I was in the virtual street close-ended 13 I felt in danger while I was in the virtual square close-ended 14 I thought that the flood was serious close-ended Comments please describe how you would behave in these possible situations (open-ended questions) 1 What is the ONE thing you would do differently in a flooding emergency now that you have been through this virtual experience? open-ended 2 Are there any other comments you wish to express about your virtual experience? open-ended References [1] A.K. Jha, R. Bloch, J. Lamond, Cities and Flooding A Guide to Integrated Urban Flood Risk Management for the 21st Century, The World Bank, GFDRR, 2011. [2] S. Mannucci, F. Rosso, A. D’Amico, G. Bernardini, M. Morganti, Flood resilience and adaptation in the built environment : how far along are we, Sustainability 14 (2022) 1–22, https://doi.org/10.3390/su14074096. [3] A. D’Amico, M. Russo, M. Angelosanti, G. Bernardini, D. Vicari, E. Quagliarini, E. Currà, Built environment typologies prone to risk: a cluster analysis of open spaces in Italian cities, Sustain. Times 13 (2021) 1–32, https://doi.org/10.3390/su13169457. [4] G. Bernardini, S. Camilli, E. Quagliarini, M. D’Orazio, M. D’Orazio, Flooding risk in existing urban environment: from human behavioral patterns to a microscopic simulation model, Energy Proc. 134 (2017) 131–140, https://doi.org/10.1016/j.egypro.2017.09.549. [5] M. Shirvani, G. Kesserwani, P. Richmond, Agent-based modelling of pedestrian responses during flood emergency: mobility behavioural rules and implications for flood risk analysis, J. Hydroinf. 22 (2020) 1078–1092, https://doi.org/10.2166/hydro.2020.031. [6] Y. Baba, T. Ishigaki, K. Toda, Experimental studies on safety evacuation from underground spaces under inundated situations, J. JSCE. 5 (2017) 269–278, https://doi.org/10.2208/journalofjsce.5.1_269. [7] T. Ishigaki, Y. Onishi, Y. Asai, K. Toda, H. Shimada, Evacuation criteria during urban flooding in underground space, in: 11th Int. Conf. Urban Drain, 2008, p. 7. [8] F. Taillandier, P. Di Maiolo, P. Taillandier, C. Jacquenod, L. Rauscher-Lauranceau, R. Mehdizadeh, An agent-based model to simulate inhabitants’ behavior during a flood event, Int. J. Disaster Risk Reduc. 64 (2021), 102503, https://doi.org/10.1016/j.ijdrr.2021.102503. [9] D. Lumbroso, M. Davison, Use of an agent-based model and Monte Carlo analysis to estimate the effectiveness of emergency management interventions to reduce loss of life during extreme floods, J. Flood Risk Manag. 11 (2018) S419–S433, https://doi.org/10.1111/jfr3.12230. [10] H.-K. Lee, W.-H. Hong, Y.-H. Lee, Experimental study on the influence of water depth on the evacuation speed of elderly people in flood conditions, Int. J. Disaster Risk Reduc. 39 (2019), 101198, https://doi.org/10.1016/j.ijdrr.2019.101198. [11] R.J. Cox, M.J. Shand, T.D. Blacka, Australian Rainfall & Runoff Revision Project 10: Appropriate Safety Criteria for People, 2010, https://doi.org/10.1038/ 103447b0. [12] G. Bernardini, M. Postacchini, E. Quagliarini, M. Brocchini, C. Cianca, M. D’Orazio, A preliminary combined simulation tool for the risk assessment of pedestrians’ flood-induced evacuation, Environ. Model. Software 96 (2017) 14–29, https://doi.org/10.1016/j.envsoft.2017.06.007. [13] Y. Zhu, T. Chen, N. Ding, M. Chraibi, W.C. Fan, Follow the evacuation signs or surrounding people during building evacuation, an experimental study, Phys. A Stat. Mech. Its Appl. 560 (2020), 125156, https://doi.org/10.1016/j.physa.2020.125156. [14] J. Lin, R. Zhu, N. Li, B. Becerik-Gerber, How occupants respond to building emergencies: a systematic review of behavioral characteristics and behavioral theories, Saf. Sci. 122 (2020), 104540, https://doi.org/10.1016/j.ssci.2019.104540. [15] S. Yamashita, R. Watanabe, Y. Shimatani, Smart adaptation activities and measures against urban flood disasters, Sustain. Cities Soc. 27 (2016) 175–184, https://doi.org/10.1016/j.scs.2016.06.027. [16] R. Lovreglio, V. Gonzalez, R. Amor, M. Spearpoint, J. Thomas, M. Trotter, R. Sacks, The need for enhancing earthquake evacuee safety by using virtual reality serious games, in: Lean Comput. Constr. Congr. - Vol. 1 Proc. Jt. Conf. Comput. Constr, Heriot-Watt University, Edinburgh, 2017, pp. 381–389, https://doi.org/ 10.24928/JC3-2017/0058. [17] R. Lovreglio, Virtual and augmented reality for human behaviour in disasters: a review emergency evacuation, in: Fire Evacuation Model. Tech. Conf, vol. 2020, 2020, p. 2020. https://www.researchgate.net/publication/343809101. [18] L. Cao, J. Lin, N. Li, A virtual reality based study of indoor fire evacuation after active or passive spatial exploration, Comput. Hum. Behav. 90 (2019) 37–45, https://doi.org/10.1016/j.chb.2018.08.041. [19] Z. Feng, V.A. González, C. Mutch, R. Amor, A. Rahouti, A. Baghouz, N. Li, G. Cabrera-Guerrero, Towards a customizable immersive virtual reality serious game for earthquake emergency training, Adv. Eng. Inf. 46 (2020), 101134, https://doi.org/10.1016/j.aei.2020.101134. [20] R. Lovreglio, V. Gonzalez, Z. Feng, R. Amor, M. Spearpoint, J. Thomas, M. Trotter, R. Sacks, Prototyping virtual reality serious games for building earthquake preparedness: the Auckland City Hospital case study, Adv. Eng. Inf. 38 (2018) 670–682, https://doi.org/10.1016/j.aei.2018.08.018. [21] R. Lovreglio, M. Kinateder, Augmented reality for pedestrian evacuation research: promises and limitations, Saf. Sci. 128 (2020), 104750, https://doi.org/ 10.1016/j.ssci.2020.104750. [22] M. Kinateder, E. Ronchi, D. Nilsson, Virtual reality for fire evacuation research, Fed. Conf. Comput. Sci. Inf. Syst. (2014) 319–321. A. D’Amico et al. http://refhub.elsevier.com/S2212-4209(23)00420-X/sref1 https://doi.org/10.3390/su14074096 https://doi.org/10.3390/su13169457 https://doi.org/10.1016/j.egypro.2017.09.549 https://doi.org/10.2166/hydro.2020.031 https://doi.org/10.2208/journalofjsce.5.1_269 http://refhub.elsevier.com/S2212-4209(23)00420-X/sref7 https://doi.org/10.1016/j.ijdrr.2021.102503 https://doi.org/10.1111/jfr3.12230 https://doi.org/10.1016/j.ijdrr.2019.101198 https://doi.org/10.1038/103447b0 https://doi.org/10.1038/103447b0 https://doi.org/10.1016/j.envsoft.2017.06.007 https://doi.org/10.1016/j.physa.2020.125156 https://doi.org/10.1016/j.ssci.2019.104540 https://doi.org/10.1016/j.scs.2016.06.027 https://doi.org/10.24928/JC3-2017/0058 https://doi.org/10.24928/JC3-2017/0058 https://www.researchgate.net/publication/343809101 https://doi.org/10.1016/j.chb.2018.08.041 https://doi.org/10.1016/j.aei.2020.101134 https://doi.org/10.1016/j.aei.2018.08.018 https://doi.org/10.1016/j.ssci.2020.104750 https://doi.org/10.1016/j.ssci.2020.104750 http://refhub.elsevier.com/S2212-4209(23)00420-X/sref22 International Journal of Disaster Risk Reduction 96 (2023) 103940 19 [23] Y. Zhang, H. Liu, S.-C. Kang, M. Al-Hussein, Virtual reality applications for the built environment: research trends and opportunities, Autom. ConStruct. 118 (2020), 103311, https://doi.org/10.1016/j.autcon.2020.103311. [24] S. Gwynne, M. Amos, M. Kinateder, N. Bénichou, K. Boyce, C. Natalie van der Wal, E. Ronchi, The future of evacuation drills: assessing and enhancing evacuee performance, Saf. Sci. 129 (2020), 104767, https://doi.org/10.1016/j.ssci.2020.104767. [25] S.M.V. Gwynne, E.D. Kuligowski, K.E. Boyce, D. Nilsson, A.P. Robbins, R. Lovreglio, J.R. Thomas, A. Roy-Poirier, Enhancing egress drills: preparation and assessment of evacuee performance, Fire Mater. (2017), https://doi.org/10.1002/fam.2448. [26] Z. Feng, V.A. González, R. Amor, M. Spearpoint, J. Thomas, R. Sacks, R. Lovreglio, G. Cabrera-Guerrero, An immersive virtual reality serious game to enhance earthquake behavioral responses and post-earthquake evacuation preparedness in buildings, Adv. Eng. Inf. 45 (2020), 101118, https://doi.org/10.1016/j. aei.2020.101118. [27] R. Lovreglio, X. Duan, A. Rahouti, R. Phipps, D. Nilsson, Comparing the effectiveness of fire extinguisher virtual reality and video training, Virtual Real. 25 (2021) 133–145, https://doi.org/10.1007/s10055-020-00447-5. [28] A. Rahouti, R. Lovreglio, S. Datoussaïd, T. Descamps, Prototyping and validating a non-immersive virtual reality serious game for healthcare fire safety training, Fire Technol. 57 (2021) 3041–3078, https://doi.org/10.1007/s10694-021-01098-x. [29] C. Li, W. Liang, C. Quigley, Y. Zhao, L.F. Yu, Earthquake safety training through virtual drills, IEEE Trans. Vis. Comput. Graph (2017) 1275–1284, https://doi. org/10.1109/TVCG.2017.2656958. [30] Z. Feng, V.A. González, C. Mutch, R. Amor, G. Cabrera-Guerrero, Instructional mechanisms in immersive virtual reality serious games: earthquake emergency training for children, J. Comput. Assist. Learn. 37 (2021) 542–556, https://doi.org/10.1111/jcal.12507. [31] F. Buttussi, L. Chittaro, Effects of different types of virtual reality display on presence and learning in a safety training scenario, IEEE Trans. Vis. Comput. Graph. 24 (2018) 1063–1076, https://doi.org/10.1109/TVCG.2017.2653117. [32] L. Chittaro, Passengers’ safety in aircraft evacuations: employing serious games to educate and persuade, in: Lect. Notes Comput. Sci. (Including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), Springer, Berlin, Heidelberg, 2012, pp. 215–226, https://doi.org/10.1007/978-3-642-31037-9_19. [33] R. Lovreglio, D.-C. Ngassa, A. Rahouti, D. Paes, Z. Feng, A. Shipman, Prototyping and Testing a Virtual Reality Counterterrorism Serious Game for Active Shooting, SSRN Electron. J., 2021, https://doi.org/10.2139/ssrn.3995851. [34] F. Buttussi, L. Chittaro, A comparison of procedural safety training in three conditions: virtual reality headset, smartphone, and printed materials, IEEE Trans. Learn. Technol. 14 (2021) 1–15, https://doi.org/10.1109/TLT.2020.3033766. [35] R. Sacks, A. Perlman, R. Barak, Construction safety training using immersive virtual reality, Construct. Manag. Econ. 31 (2013) 1005–1017, https://doi.org/ 10.1080/01446193.2013.828844. [36] A. Bandura, Self-efficacy: toward a unifying theory of behavioral change, Psychol. Rev. 84 (1977) 191–215, https://doi.org/10.1037/0033-295X.84.2.191. [37] J.D. Saville, L.L. Foster, Does technology self-efficacy influence the effect of training presentation mode on training self-efficacy? Comput. Hum. Behav. Reports. 4 (2021), 100124 https://doi.org/10.1016/j.chbr.2021.100124. [38] L. Chittaro, F. Buttussi, Exploring the use of arcade game elements for attitude change: two studies in the aviation safety domain, Int. J. Hum. Comput. Stud. 127 (2019) 112–123, https://doi.org/10.1016/j.ijhcs.2018.07.006. [39] Y. Sermet, I. Demir, Flood action VR: a virtual reality framework for disaster awareness and emergency response training, in: ACM SIGGRAPH 2019 Posters, SIGGRAPH 2019, Association for Computing Machinery, Inc, 2019, https://doi.org/10.1145/3306214.3338550. [40] L. Gamberini, A. Bettelli, G. Benvegnù, V. Orso, A. Spagnolli, M. Ferri, Designing “safer water.” A virtual reality tool for the safety and the psychological well- being of citizens exposed to the risk of natural disasters, Front. Psychol. 12 (2021) 1–12, https://doi.org/10.3389/fpsyg.2021.674171. [41] T. Fujimi, K. Fujimura, Testing public interventions for flash flood evacuation through environmental and social cues: the merit of virtual reality experiments, Int. J. Disaster Risk Reduc. 50 (2020), https://doi.org/10.1016/j.ijdrr.2020.101690. [42] J.M. Mol, W.J.W. Botzen, J.E. Blasch, After the virtual flood: risk perceptions and flood preparedness after virtual reality risk communication, Judgm. Decis. Mak. 17 (2022) 189–214. https://sjdm.org/journal/20/200909/jdm200909.pdf. [43] D. Djaouti, J. Alvarez, J.-P. Jessel, Classifying Serious Games, 2011, pp. 118–136, https://doi.org/10.4018/978-1-60960-495-0.ch006. [44] A. Solinska-Nowak, P. Magnuszewski, M. Curl, A. French, A. Keating, J. Mochizuki, W. Liu, R. Mechler, M. Kulakowska, L. Jarzabek, An overview of serious games for disaster risk management – prospects and limitations for informing actions to arrest increasing risk, Int. J. Disaster Risk Reduc. 31 (2018) 1013–1029, https://doi.org/10.1016/j.ijdrr.2018.09.001. [45] G. Bernardini, G. Romano, L. Soldini, E. Quagliarini, How urban layout and pedestrian evacuation behaviours can influence flood risk assessment in riverine historic built environments, Sustain. Cities Soc. 70 (2021), 102876, https://doi.org/10.1016/j.scs.2021.102876. [46] G. Bernardini, E. Quagliarini, M. D’Orazio, M. Brocchini, Towards the simulation of flood evacuation in urban scenarios: experiments to estimate human motion speed in floodwaters, Saf. Sci. 123 (2020), 104563, https://doi.org/10.1016/j.ssci.2019.104563. [47] R.R. Krishnan, S. Sushil, R. Hrishikesh, S. Devadas, A. Ganesh, G. Narayanan, A novel virtual reality game for disaster management applications, in: Proc. 2019 IEEE Int. Conf. Commun. Signal Process. ICCSP 2019, Institute of Electrical and Electronics Engineers Inc., 2019, pp. 254–257, https://doi.org/10.1109/ ICCSP.2019.8697956. [48] S. Irshad, A. Perkis, Serious storytelling in virtual reality: the role of digital narrative to trigger mediated presence and behavioral responses, in: CHI Play 2020 - Ext. Abstr. 2020 Annu. Symp. Comput. Interact. Play, Association for Computing Machinery, Inc, 2020, pp. 267–271, https://doi.org/10.1145/ 3383668.3419908. [49] K. Haynes, M. Tofa, A. Avci, J. van Leeuwen, L. Coates, Motivations and experiences of sheltering in place during floods: implications for policy and practice, Int. J. Disaster Risk Reduc. 31 (2018) 781–788, https://doi.org/10.1016/j.ijdrr.2018.07.011. [50] S. Wang, J. Wang, P. Gourbesville, L. Andres, Visualization of Flood Simulation with Microsoft HoloLens, 2020, pp. 91–101, https://doi.org/10.1007/978-981- 15-5436-0_8. [51] P. Gmelch, R. Lejano, E. O’Keeffe, D.F. Laefer, C. Drell, M. Bertolotto, U. Ofterdinger, J. McKinley, The case for low-cost, personalized visualization for enhancing natural hazard preparedness, XLIV-M-2–2020, ISPRS - Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci (2020) 37–44, https://doi.org/10.5194/ isprs-archives-xliv-m-2-2020-37-2020. [52] Mixamo, (n.d.). https://www.mixamo.com/#/(accessed March 8, 2022). [53] Y. Li, B. Hu, D. Zhang, J. Gong, Y. Song, J. Sun, Flood evacuation simulations using cellular automata and multiagent systems -a human-environment relationship perspective, Int. J. Geogr. Inf. Sci. 33 (2019) 2241–2258, https://doi.org/10.1080/13658816.2019.1622015. [54] L. Fu, J. Zhu, W. Li, Q. Zhu, B. Xu, Y. Xie, Y. Zhang, Y. Hu, J. Lu, P. Dang, J. You, Tunnel vision optimization method for VR flood scenes based on Gaussian blur, Int. J. Digit. Earth. (2021), https://doi.org/10.1080/17538947.2021.1886359. [55] H. Chanson, R. Brown, D. McIntosh, Human body stability in floodwaters: the 2011 flood in Brisbane CBD, in: Hydraul. Struct. Soc. - Eng. Challenges Extrem, The University of Queensland, 2014, pp. 1–9, https://doi.org/10.14264/uql.2014.48. [56] M. Haghani, M. Sarvi, Crowd behaviour and motion: empirical methods, Transp. Res. Part B Methodol. 107 (2018) 253–294, https://doi.org/10.1016/J. TRB.2017.06.017. [57] J. Munafo, M. Diedrick, T.A. Stoffregen, The virtual reality head-mounted display Oculus Rift induces motion sickness and is sexist in its effects, Exp. Brain Res. 235 (2017) 889–901, https://doi.org/10.1007/s00221-016-4846-7. [58] H.K. Kim, J. Park, Y. Choi, M. Choe, Virtual reality sickness questionnaire (VRSQ): motion sickness measurement index in a virtual reality environment, Appl. Ergon. 69 (2018) 66–73, https://doi.org/10.1016/j.apergo.2017.12.016. [59] J.P.M. de Sá, Applied Statistics Using SPSS, STATISTICA, MATLAB and R, Springer-Verlag Berlin Heidelberg, 2007. [60] MathWorks - Archived MathWorks Documentation - MathWorks Italia, (n.d.). https://it.mathworks.com/help/doc-archives.html (accessed December 21, 2021). [61] J. Dijkstra, B. de Vries, J. Jessurun, Wayfinding search strategies and matching familiarity in the built environment through virtual navigation, Transport. Res. Procedia 2 (2014) 141–148, https://doi.org/10.1016/j.trpro.2014.09.018. A. D’Amico et al. https://doi.org/10.1016/j.autcon.2020.103311 https://doi.org/10.1016/j.ssci.2020.104767 https://doi.org/10.1002/fam.2448 https://doi.org/10.1016/j.aei.2020.101118 https://doi.org/10.1016/j.aei.2020.101118 https://doi.org/10.1007/s10055-020-00447-5 https://doi.org/10.1007/s10694-021-01098-x https://doi.org/10.1109/TVCG.2017.2656958 https://doi.org/10.1109/TVCG.2017.2656958 https://doi.org/10.1111/jcal.12507 https://doi.org/10.1109/TVCG.2017.2653117 https://doi.org/10.1007/978-3-642-31037-9_19 https://doi.org/10.2139/ssrn.3995851 https://doi.org/10.1109/TLT.2020.3033766 https://doi.org/10.1080/01446193.2013.828844 https://doi.org/10.1080/01446193.2013.828844 https://doi.org/10.1037/0033-295X.84.2.191 https://doi.org/10.1016/j.chbr.2021.100124 https://doi.org/10.1016/j.ijhcs.2018.07.006 https://doi.org/10.1145/3306214.3338550 https://doi.org/10.3389/fpsyg.2021.674171 https://doi.org/10.1016/j.ijdrr.2020.101690 https://sjdm.org/journal/20/200909/jdm200909.pdf https://doi.org/10.4018/978-1-60960-495-0.ch006 https://doi.org/10.1016/j.ijdrr.2018.09.001 https://doi.org/10.1016/j.scs.2021.102876 https://doi.org/10.1016/j.ssci.2019.104563 https://doi.org/10.1109/ICCSP.2019.8697956 https://doi.org/10.1109/ICCSP.2019.8697956 https://doi.org/10.1145/3383668.3419908 https://doi.org/10.1145/3383668.3419908 https://doi.org/10.1016/j.ijdrr.2018.07.011 https://doi.org/10.1007/978-981-15-5436-0_8 https://doi.org/10.1007/978-981-15-5436-0_8 https://doi.org/10.5194/isprs-archives-xliv-m-2-2020-37-2020 https://doi.org/10.5194/isprs-archives-xliv-m-2-2020-37-2020 https://www.mixamo.com/#/ https://doi.org/10.1080/13658816.2019.1622015 https://doi.org/10.1080/17538947.2021.1886359 https://doi.org/10.14264/uql.2014.48 https://doi.org/10.1016/J.TRB.2017.06.017 https://doi.org/10.1016/J.TRB.2017.06.017 https://doi.org/10.1007/s00221-016-4846-7 https://doi.org/10.1016/j.apergo.2017.12.016 http://refhub.elsevier.com/S2212-4209(23)00420-X/sref59 https://it.mathworks.com/help/doc-archives.html https://doi.org/10.1016/j.trpro.2014.09.018 International Journal of Disaster Risk Reduction 96 (2023) 103940 20 [62] Y.M. Kim, I. Rhiu, A comparative study of navigation interfaces in virtual reality environments: a mixed-method approach, Appl. Ergon. 96 (2021), 103482, https://doi.org/10.1016/j.apergo.2021.103482. [63] R. Lovreglio, E. Ronchi, D. Nilsson, Calibrating floor field cellular automaton models for pedestrian dynamics by using likelihood function optimization, Phys. A Stat. Mech. Its Appl. 438 (2015) 308–320, https://doi.org/10.1016/j.physa.2015.06.040. [64] G. Fauville, A.C.M. Queiroz, E.S. Woolsey, J.W. Kelly, J.N. Bailenson, The effect of water immersion on vection in virtual reality, Sci. Rep. 11 (2021) 1–13, https://doi.org/10.1038/s41598-020-80100-y. [65] M.I. Berkman, Eye tracking in virtual reality, Encycl. Comput. Graph. Games. 1–8 (2019), https://doi.org/10.1007/978-3-319-08234-9_170-1. A. D’Amico et al. https://doi.org/10.1016/j.apergo.2021.103482 https://doi.org/10.1016/j.physa.2015.06.040 https://doi.org/10.1038/s41598-020-80100-y https://doi.org/10.1007/978-3-319-08234-9_170-1 A non-immersive virtual reality serious game application for flood safety training 1 Introduction 2 Materials and methods 2.1 Application development 2.1.1 Training objectives 2.1.2 Story modules 2.1.3 Modelling strategy 2.2 Data collection and questionnaire 2.3 Data analysis 3 Results 3.1 Participants 3.2 Awareness assessments and SG training perception 3.3 Behavioural analysis 4 Discussion 5 Conclusion Ethical approval Author contributions statement Declaration of competing interest Data availability Acknowledgements Appendix Questionnaires Items Pre-Test Questionnaire Post-Test Questionnaire References