Using value chain approaches to evaluate the end-to-end warning chain

The weather information value chain provides a framework for characterising the production, communication, and use of information by all stakeholders in an end-to-end warning system covering weather and hazard monitoring, modelling and forecasting, risk assessment, communication and preparedness activities. Warning services are typically developed and provided through a multitude of complex and malleable value chains (networks), often established through co-design, co-creation and co-provision. In November 2020, a 4-year international project under the World Meteorological Organization (WMO) World Weather Research Programme was instigated to explore value chain approaches to describe and evaluate warning systems for high impact weather by integrating physical and social science. It aims to create a framework with guidance and tools for using value chain approaches, and to develop a database of high impact weather warning case studies for scientists and practitioners to review, analyse and learn from previous experience using value chain approaches.

Here we describe a template for high-impact weather event case study collection that provides a tool for scientists and practitioners involved in researching, designing and evaluating weather-related warning systems to review previous experience of high impact weather events and assess their efficacy.

1 Introduction

There has been an increase in extreme weather events in recent years and decades, which has been attributed to human-induced climate change caused by a significant increase in greenhouse gas emissions (IPCC, 2021). Some examples of extreme weather events that have been linked to climate change include heatwaves, heavy rainfall, wildfires, and storms (IPCC, 2022). These events can have substantial impacts on people and communities, including damage to property, disruption of daily life, and loss of life (Golding, 2022).

Severe weather warnings alert people to potentially dangerous weather conditions that could pose a threat to their safety. These warnings are issued for a variety of small and large-scale severe weather events, including thunderstorms, tornadoes, hurricanes, flash floods, and heatwaves. By issuing severe weather warnings through early warning systems (EWS) with sufficient lead time, authorities can help people and businesses to prepare for and respond to those events, ensuring that they are better able to protect themselves, their families, and their property. Severe weather warnings are typically issued by national hydrometeorological agencies, such as the National Weather Service in the US or the Bureau of Meteorology in Australia, and are broadcast through various channels, including television, radio, social media, and local emergency notification systems (Golding, 2022). In addition to alerting people to the potential danger of severe weather, impact-based warnings can also provide important information about the expected impact of the weather event, such as the anticipated road conditions and related implications on traffic, or potential health outcomes, and any recommended actions that people should take to stay safe (Potter et al., 2021). The United Nations recently proposed an Action Plan with the goal “to reach everyone on Earth with early warnings against increasingly extreme and dangerous weather” in the next five years at COP27 in November 2022 (WMO, 2022b).

The information value chain in the end-to-end warning system

Effective warnings of weather-related hazards result from the successful interaction of many people and organisations, each contributing their specific capability and knowledge. This process can be described by the information value chain which is a general concept referring to the process of how information is created, stored, used, and shared in an organisation or system within a particular context (Lazo and Mills, 2021). In the case of hydrometeorological information services, the weather information value chain provides a framework for characterising the production, communication, and use of information by all stakeholders in an end-to-end warning system covering weather and hazard monitoring, modelling and forecasting, risk assessment, communication and preparedness activities (Lazo et al., 2020; Leviakangas, 2009; Perrels et al., 2012; WMO, 2015). These functions are represented by the mountains in Fig. 1. Good partnership between all actors involved is paramount to ensure that all relevant data and information is received by the subsequent actor. If any of these communication “bridges” (Fig. 1) fail, the best possible hazard forecast is essentially useless as its potential value is not realised by the recipient through informed decision-making (Golding, 2022).

Figure 1 Schematic warning value chain for high impact weather warnings showing the capabilities and outputs (mountains) and information exchanges (bridges) linking the capabilities and their associated communities (Tan et al., 2022). Before and during an actual severe event the flow of information is predominantly downstream, while for post event assessments, implementation of improvements and creation of new services the chain becomes more like a feedback cycle.

While Fig. 1 represents a simple schematic of an end-to-end warning chain, it is a lot more complex in reality. Warning services are typically developed and provided through a multitude of complex and malleable value chains, often established through co-design, co-creation and co-provision. Most studies and post-event reviews analyse, assess, and review components of the warning chain including warning communication, hazard impacts and community response, often proposing improvements and new approaches (Demuth et al., 2012; Kaltenberger et al., 2020; Kelman et al., 2018; Lazo et al., 2020; Martinez, 2020; Merz et al., 2020; Morss et al., 2022; Rodwell et al., 2020; Wu et al., 2020) but rarely is the chain considered as a whole (Cawood et al., 2018; Emerton et al., 2020; Lazo and Mills, 2021; Msemo et al., 2021; Perrels et al., 2012).

The project described in this article aims to bring the pieces together to evaluate the end-to-end warning chain in support of the Sendai Framework for Risk Reduction 2015–2030 and its specific targets (UNDRR, 2015), as well as the early warning systems aspects of the 2015 Paris Agreement (United Nations/Framework Convention on Climate Change, 2015).

2 WWRP value chain project

The World Meteorological Organisation (WMO) World Weather Research Programme (WWRP) established the High Impact Weather (HIWeather) project in 2015 to support the improvement of weather-related warnings worldwide through targeted research (Zhang et al., 2019). Within this project and with input and collaboration from the Societal and Economic Research Applications Working Group (SERA WG), a 4-year flagship project commenced in November 2020 to use value chain approaches to evaluate the end-to-end warning chain. As of January 2023, the project team includes 36 members from the academic, public and private sectors (some members are in the HIWeather project and SERA WG) across fourteen countries. The broad spectrum of the warning value chain from observations and modelling to warning communication and decision-making requires a wide range of expertise. As such, the project team comprises roughly equal numbers of physical scientists and social scientists, each bringing their expertise for some components of the warning value chain.

2.1 Goals

The project's aims are to

• review value chain practices used to describe and understand weather, warning and climate services,

• assess and provide guidance on how to effectively apply value chains in a weather warning context involving multiple users and partnerships, and

• create a searchable warning chain database that researchers and practitioners can use to explore the organisation and performance of actual end-to-end warning chains for high impact events and assess their effectiveness using value chain approaches.

The first of two key outputs of the project is a high-level framework for service providers and decision makers. It will provide guidance and tools on the application of value chain approaches for activities including understanding existing services, designing new services and assessing services for resource allocation and optimising structures. The framework will be a synthesis of literature reviews, workshops and learnings that come from case study analysis as part of the second key output. This is complemented with a glossary to provide a common terminology for the broad spectrum of users in research and operation.

Figure 2 Interaction between the objectives of the project. Dashed boxes group objectives led by SERA WG and HIWeather respectively with the blue arrows pointing to their key outputs.

The second key output is an online database of high impact events which includes rich information spanning the entire warning value chain and is linked to the WMO Catalogue of Hazardous Events (WMO-CHE, WMO, 2022a). The events database will support understanding of best practice, be valuable for the analysis of the value chain for specific events and provide important insights to incorporate in the high-level framework. Figure 2 visualises how the objectives of the two key outputs complement each other. As an interim step, the project team has developed a database template (questionnaire) that scopes information from each section of the warning value chain that is required for an in-depth evaluation which informs the design of an online database. The next section elaborates on the database questionnaire.

2.2 Database questionnaire

Many databases exist that provide information on forecasts (e.g., ECMWF Severe Event Catalogue) or impacts (e.g., EM-DAT, SHELDUS, DesInventar) associated with severe weather events. However, these databases only target some data like economic costs, lives lost and damages or evaluate the forecast performance for an event. This project's database leverages and extends these existing databases by providing a comprehensive picture of the end-to-end production and flow of information and decision making along the warning chain. This will enable in-depth case studies and cross-cutting analysis of end-to-end warning value chains, from simple to complex, to understand effective practices, and support the cycle of review and improvements that would enhance future warnings. The questionnaire was originally designed for weather events but its use is encouraged for other relevant events such as hydrological or geohazard events to support the UNDRR Sendai Framework calling for multi-hazard warning systems (UNDRR, 2015).

Figure 3 Structure of the database questionnaire.

The questionnaire is structured in three main parts (Fig. 3):

1. The essential information table requests brief facts about a particular event, such as what happened, when, and where, impacts and responses. This information will help database users to filter events. These are numerical and short text entries only. Links to this event in other databases and catalogues about this event should be provided if possible.

2. The second part requests supplementary information about the different parts in the warning value chain. This more detailed information and analysis about the weather/hazard source, hazards, impacts, warning communication and warning response will help users understand what was unique about the warning chain for this event. The questions in Part 2 probe many aspects of the warning chain but are not exhaustive. Information here might include:

   • a.

     graphics (for example, forecast charts, reanalysis maps, warning graphics, photos of impacts, etc.; subject to copyright);

   • b.

     videos (for example, from social media, weather service outlooks, etc.);

   • c.

     free-form text (for example, description of meteorology, extracts from reports, data analysis, tables, etc.);

   • d.

     links (e.g., to external reports, media, national archives, policy documents, protocols, meeting records, etc.).

   Part 2 also supports the analysis of the whole warning chain through questions about information and workflows, strongest and weakest links, and lessons learned.

3. The subjective assessment asks contributors to rate the effectiveness of the individual elements of the end-to-end warning chain, and its overall effectiveness, on a scale of 1 (poor) to 5 (excellent). This is intended only to assist users of the database in choosing cases for further examination.

The accompanying guide provides detailed descriptions of the information requested in the database questionnaire. It contains advice and examples to help contributors provide consistent, high-quality data and information that will enable researchers and practitioners to make effective use of the database. Contributors from National Meteorological and Hydrological Services (NMHS), emergency management, relevant partner agencies and research institutions are the main focus group for using this questionnaire, but anyone interested is welcomed to participate.

The database questionnaire and guide are available on the HIWeather website at http://hiweather.net/Lists/130.html (last access: 15 February 2023).

2.3 High impact weather events studied

Since the start of the project in November 2020, many severe weather events have caused billions of dollars of damage and significant loss of lives (Christian Aid, 2021, 2022). During the two years, the project team has flagged 33 events of interest and started collecting data and information for 10, with two case studies fully completed and analysed. The collection comprises of severe events including heavy rain and flash flooding in Henan, China in July 2021, Hurricanes Ida (August 2021) and Ian (September 2022) in the US, Black Summer bushfires 2019/2020 in south-eastern Australia, and the Hunga Tonga Hunga Ha'apai volcano eruption in Tonga in January 2022, among others. Some of these and other case studies are briefly analysed and compared by Golding et al. (2022).

More detailed findings and lessons learnt using the concept of the warning value chain were analysed by project members for a cold-front induced ultramarathon tragedy in Jingtai County of Baiyin City, China in May 2020 (Zhang et al., 2021). Project members at the Bureau of Meteorology held an internal workshop with pre-workshop surveys interviewing forecasters, embedded meteorologists, communication specialists and managers to assess the performance of each part of the warning value chain for a heavy rain and flood event in eastern Australia in March 2021. The UK Met Office used the questionnaire to evaluate the performance of their warning system for severe windstorm Eunice in February 2022. The database questionnaire has proven to be a useful tool for these case study analyses.

The questionnaire was also used as an educational tool at the University of Miami in an undergraduate tropical meteorology course to study the warning value chain for two hurricanes as an assignment and was found to enhance the students' critical thinking about hurricane forecasting, impacts, warning communication and response (Majumdar et al., 2023). Future publications will elaborate on these case studies.

Challenges in doing case studies

During the wider use of the questionnaire, a couple of challenges were identified when collecting data and information for case study events. First, data access and availability in general is highly variable. While some information on weather forecasts, observations and satellite imagery is often easy to get via international databases, similar information on hazards is much less accessible since such forecasts are mostly performed with regional models and not stored meticulously. Hazard forecasts, and especially impact forecasts, are in their infancy and not operationally used by many NMHS which limits their accessibility further.

Secondly, the communication bridges between forecasts and warnings and other stages in the warning value chain are rarely documented and need input from the relevant institutions to get necessary details. Impact information beyond economic damages, injuries and lives lost that is not captured in other databases can be derived from photos and videos posted by observers on social media, however, this information is buried quickly and difficult to recover some time after the event. Similarly, warnings issued via different communication channels quickly disappear after the event (e.g., warning notifications on phones and apps). On the other hand, a lot of information on a case study only becomes available weeks, months, or even years after the event when economic damages and social impacts have been assessed, and – in case of a particularly severe event – a post-event assessment has been performed by the NMHS and partner agencies (e.g., emergency services). Generally, the collection of information can split into two time frames, (i) during and immediately after the event unfolded to collect “perishable” data from social media and warning communication, and (ii) much later after the event for accurate and more in-depth information. Care must be taken when using media reports as they can sensationalise events and need to be balanced with post-event surveys conducted by NMHS and/or partner agencies; however, the latter are rarely made public. In addition, copyrights should not be violated when collecting media reports, photos and videos that document the event which is in many cases, especially on social media platforms, often not clearly labelled.

By bringing attention to these challenges, we hope to inform future collaborators of these associated difficulties when taking on a case study and encourage investigators to work with the involved agencies to collect and provide this information more openly to help inform the value chain analysis.

2.4 Progress and next steps of the value chain project

Since its inception, the project has made significant progress to achieve its aims through contributions from a continuously growing, interdisciplinary project team. In addition to the interim database template for high-impact weather event case study collection and analysis, the project team also created a glossary of common terminology used by social and physical scientists and is drafting a conceptual high-level value chain framework which is planned for release at the end of 2023. A bibliography of relevant value chain literature, including for natural hazard events and warnings, was recently released to support the framework development and provide a resource for anyone interested in the concept of the value chain. We encourage researchers and practitioners to use these resources – available on the HIWeather website at http://hiweather.net/Lists/130.html (last access: 15 February 2023).

The collection and assessment of severe event warning value chains is an on-going activity in the project which benefits from any outside collaborations that help expand this collection. It will build a solid basis for in-depth analysis across events for certain parts of the value chain, evaluation of the value chain for events of similar type and/or impacted location or similar events in different countries. Parallel to these activities, we plan to transform the interim database questionnaire into an online tool and searchable database (likely by keyword) to create a more convenient resource that outlives the project. Ultimately, the knowledge obtained by analysing warnings through the lens of the information value chain will provide NMHSs and their partners with better information to collaboratively design and implement more effective warning services.