Communicating uncertainty of scientific models in disaster risk management : a New Zealand/Aotearoa case study : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Emergency Management at Massey University, Wellington, New Zealand EMBARGOED until 9 October 2027

Abstract

Scientific models play a vital role in supporting decision-making within Disaster Risk Management (DRM). Nevertheless, the uncertainties inherent in these models substantially influence how their outputs are incorporated into decision-making, creating challenges for effective communication. Transparent interaction between scientists and decision-makers is essential for addressing these challenges. This PhD research examines how scientists and decision-makers in DRM engage with modelling and uncertainty communication and proposes a communication tool to enhance these processes. In the first phase, I conducted seventeen in-depth qualitative interviews with scientists and fifteen with decision-makers. Reflective Thematic Analysis was employed to identify key themes. From the scientists’ interviews, four themes were constructed: (a) model development and characterisation, (b) accountability and opinion, (c) uncertainty communication approaches and (d) collaboration for effective uncertainty communication. Similarly, from the decision-makers’ interviews, five themes were created: (a) uncertainty - an inherent aspect of scientific modelling, (b) seeking certainty in uncertainty communication, (c) uncertainty communication is not focused on the decision-maker’s need, (d) uncertainty is one aspect of the complex DRM decision-making, and (e) collaboration and Trust - a way forward for effective communication. The analysis of Scientists’ interview data reveals that scientists take varied approaches to modelling practice and uncertainty communication. These differences are shaped by disciplinary background, experience, and the extent of interaction with decision-makers. A key barrier identified was the absence of standardised guidelines for best-practice uncertainty communication, which limits the effective incorporation of uncertainty into DRM decision-making. The findings also suggested that a sustained collaboration between scientists and decision-makers throughout the model development lifecycle enhances the communication of uncertainty. The analysis of interviews with decision-makers reveals that the communication of uncertainty has a significant influence on decision-making. Existing approaches often fail to align with the needs of decision-makers, thereby limiting their ability to integrate uncertainty into decision-making for DRM. Decision-makers understand that uncertainty is inherent in modelling; yet they tend to prefer certainty in uncertainty communication, which suggests the over-reliance on probabilistic language. This reliance may foster a false sense of confidence in model outputs and reduce the effective use of uncertainty information. The results indicate that participatory and collaborative engagement with scientists helps to build trust and better align uncertainty communication with decision-makers’ needs. Furthermore, the results of this research highlight that while scientists prioritise methodological rigour and employ technical vocabulary and probabilistic language to convey uncertainty, decision-makers seek actionable insights and practical implications, frequently perceiving existing communication as overly complex and misaligned with their needs. These findings underscore the need for tailored communication strategies that balance the scientific intricacies of uncertainty with the practical needs of decision-makers. Since emphasis on collaboration and user-focused uncertainty visualisations is suggested in the first phase, it is a key pathway to improving uncertainty communication in DRM. In the second phase, a communication framework, the “Uncertainty Doughnut,” was developed, grounded in insights from qualitative in-depth interviews with 17 scientists and 15 decision-makers, as well as a comprehensive review of existing uncertainty communication frameworks and visualisation tools. The Uncertainty Doughnut categorises uncertainty in scientific models into five key components/locations: model input, model processing, model output, expert judgement, and deep uncertainty. Each location comprises multiple sources of uncertainty, with their magnitude represented by the relative weighting assigned to each source. The higher the magnitude of the source, the greater its contribution to the overall uncertainty of the model. Furthermore, the “3D visualisation” of this communication framework enhances its interpretability, fostering clearer, more transparent, and effective communication between scientists and decision-makers. To evaluate the Uncertainty Doughnut's usability and effectiveness, 13 feedback interviews (11 individual and 2 group interviews) were conducted. The feedback on the “Uncertainty Doughnut” was mixed, with participants suggesting that the Doughnut has potential to simplify complex information, describing it as intuitive and effective for decision-makers. However, they were also concerned that the level of detail might reduce confidence by drawing too much attention to uncertainty. Nevertheless, they suggest that practical testing in real-world scenarios is a way forward to validate its effectiveness and usability.