Using value chain approaches to evaluate the end-to-end warning chain
The weather information value chain provides a framework for characterising the production, communication, and use of information by all stakeholders in an end-to-end warning system covering weather and hazard monitoring, modelling and forecasting, risk assessment, communication and preparedness activities. Warning services are typically developed and provided through a multitude of complex and malleable value chains (networks), often established through co-design, co-creation and co-provision. In November 2020, a 4-year international project under the World Meteorological Organization (WMO) World Weather Research Programme was instigated to explore value chain approaches to describe and evaluate warning systems for high impact weather by integrating physical and social science. It aims to create a framework with guidance and tools for using value chain approaches, and to develop a database of high impact weather warning case studies for scientists and practitioners to review, analyse and learn from previous experience using value chain approaches.
Here we describe a template for high-impact weather event case study collection that provides a tool for scientists and practitioners involved in researching, designing and evaluating weather-related warning systems to review previous experience of high impact weather events and assess their efficacy.
There has been an increase in extreme weather events in recent years and decades, which has been attributed to human-induced climate change caused by a significant increase in greenhouse gas emissions (IPCC, 2021). Some examples of extreme weather events that have been linked to climate change include heatwaves, heavy rainfall, wildfires, and storms (IPCC, 2022). These events can have substantial impacts on people and communities, including damage to property, disruption of daily life, and loss of life (Golding, 2022).
Severe weather warnings alert people to potentially dangerous weather conditions that could pose a threat to their safety. These warnings are issued for a variety of small and large-scale severe weather events, including thunderstorms, tornadoes, hurricanes, flash floods, and heatwaves. By issuing severe weather warnings through early warning systems (EWS) with sufficient lead time, authorities can help people and businesses to prepare for and respond to those events, ensuring that they are better able to protect themselves, their families, and their property. Severe weather warnings are typically issued by national hydrometeorological agencies, such as the National Weather Service in the US or the Bureau of Meteorology in Australia, and are broadcast through various channels, including television, radio, social media, and local emergency notification systems (Golding, 2022). In addition to alerting people to the potential danger of severe weather, impact-based warnings can also provide important information about the expected impact of the weather event, such as the anticipated road conditions and related implications on traffic, or potential health outcomes, and any recommended actions that people should take to stay safe (Potter et al., 2021). The United Nations recently proposed an Action Plan with the goal “to reach everyone on Earth with early warnings against increasingly extreme and dangerous weather” in the next five years at COP27 in November 2022 (WMO, 2022b).
The information value chain in the end-to-end warning system
Effective warnings of weather-related hazards result from the successful interaction of many people and organisations, each contributing their specific capability and knowledge. This process can be described by the information value chain which is a general concept referring to the process of how information is created, stored, used, and shared in an organisation or system within a particular context (Lazo and Mills, 2021). In the case of hydrometeorological information services, the weather information value chain provides a framework for characterising the production, communication, and use of information by all stakeholders in an end-to-end warning system covering weather and hazard monitoring, modelling and forecasting, risk assessment, communication and preparedness activities (Lazo et al., 2020; Leviakangas, 2009; Perrels et al., 2012; WMO, 2015). These functions are represented by the mountains in Fig. 1. Good partnership between all actors involved is paramount to ensure that all relevant data and information is received by the subsequent actor. If any of these communication “bridges” (Fig. 1) fail, the best possible hazard forecast is essentially useless as its potential value is not realised by the recipient through informed decision-making (Golding, 2022).
While Fig. 1 represents a simple schematic of an end-to-end warning chain, it is a lot more complex in reality. Warning services are typically developed and provided through a multitude of complex and malleable value chains, often established through co-design, co-creation and co-provision. Most studies and post-event reviews analyse, assess, and review components of the warning chain including warning communication, hazard impacts and community response, often proposing improvements and new approaches (Demuth et al., 2012; Kaltenberger et al., 2020; Kelman et al., 2018; Lazo et al., 2020; Martinez, 2020; Merz et al., 2020; Morss et al., 2022; Rodwell et al., 2020; Wu et al., 2020) but rarely is the chain considered as a whole (Cawood et al., 2018; Emerton et al., 2020; Lazo and Mills, 2021; Msemo et al., 2021; Perrels et al., 2012).
The project described in this article aims to bring the pieces together to evaluate the end-to-end warning chain in support of the Sendai Framework for Risk Reduction 2015–2030 and its specific targets (UNDRR, 2015), as well as the early warning systems aspects of the 2015 Paris Agreement (United Nations/Framework Convention on Climate Change, 2015).
The World Meteorological Organisation (WMO) World Weather Research Programme (WWRP) established the High Impact Weather (HIWeather) project in 2015 to support the improvement of weather-related warnings worldwide through targeted research (Zhang et al., 2019). Within this project and with input and collaboration from the Societal and Economic Research Applications Working Group (SERA WG), a 4-year flagship project commenced in November 2020 to use value chain approaches to evaluate the end-to-end warning chain. As of January 2023, the project team includes 36 members from the academic, public and private sectors (some members are in the HIWeather project and SERA WG) across fourteen countries. The broad spectrum of the warning value chain from observations and modelling to warning communication and decision-making requires a wide range of expertise. As such, the project team comprises roughly equal numbers of physical scientists and social scientists, each bringing their expertise for some components of the warning value chain.
The project's aims are to
review value chain practices used to describe and understand weather, warning and climate services,
assess and provide guidance on how to effectively apply value chains in a weather warning context involving multiple users and partnerships, and
create a searchable warning chain database that researchers and practitioners can use to explore the organisation and performance of actual end-to-end warning chains for high impact events and assess their effectiveness using value chain approaches.
The first of two key outputs of the project is a high-level framework for service providers and decision makers. It will provide guidance and tools on the application of value chain approaches for activities including understanding existing services, designing new services and assessing services for resource allocation and optimising structures. The framework will be a synthesis of literature reviews, workshops and learnings that come from case study analysis as part of the second key output. This is complemented with a glossary to provide a common terminology for the broad spectrum of users in research and operation.
The second key output is an online database of high impact events which includes rich information spanning the entire warning value chain and is linked to the WMO Catalogue of Hazardous Events (WMO-CHE, WMO, 2022a). The events database will support understanding of best practice, be valuable for the analysis of the value chain for specific events and provide important insights to incorporate in the high-level framework. Figure 2 visualises how the objectives of the two key outputs complement each other. As an interim step, the project team has developed a database template (questionnaire) that scopes information from each section of the warning value chain that is required for an in-depth evaluation which informs the design of an online database. The next section elaborates on the database questionnaire.
2.2 Database questionnaire
Many databases exist that provide information on forecasts (e.g., ECMWF Severe Event Catalogue) or impacts (e.g., EM-DAT, SHELDUS, DesInventar) associated with severe weather events. However, these databases only target some data like economic costs, lives lost and damages or evaluate the forecast performance for an event. This project's database leverages and extends these existing databases by providing a comprehensive picture of the end-to-end production and flow of information and decision making along the warning chain. This will enable in-depth case studies and cross-cutting analysis of end-to-end warning value chains, from simple to complex, to understand effective practices, and support the cycle of review and improvements that would enhance future warnings. The questionnaire was originally designed for weather events but its use is encouraged for other relevant events such as hydrological or geohazard events to support the UNDRR Sendai Framework calling for multi-hazard warning systems (UNDRR, 2015).
The questionnaire is structured in three main parts (Fig. 3):
The essential information table requests brief facts about a particular event, such as what happened, when, and where, impacts and responses. This information will help database users to filter events. These are numerical and short text entries only. Links to this event in other databases and catalogues about this event should be provided if possible.
The second part requests supplementary information about the different parts in the warning value chain. This more detailed information and analysis about the weather/hazard source, hazards, impacts, warning communication and warning response will help users understand what was unique about the warning chain for this event. The questions in Part 2 probe many aspects of the warning chain but are not exhaustive. Information here might include:
graphics (for example, forecast charts, reanalysis maps, warning graphics, photos of impacts, etc.; subject to copyright);
videos (for example, from social media, weather service outlooks, etc.);
free-form text (for example, description of meteorology, extracts from reports, data analysis, tables, etc.);
links (e.g., to external reports, media, national archives, policy documents, protocols, meeting records, etc.).
Part 2 also supports the analysis of the whole warning chain through questions about information and workflows, strongest and weakest links, and lessons learned.
The subjective assessment asks contributors to rate the effectiveness of the individual elements of the end-to-end warning chain, and its overall effectiveness, on a scale of 1 (poor) to 5 (excellent). This is intended only to assist users of the database in choosing cases for further examination.
The accompanying guide provides detailed descriptions of the information requested in the database questionnaire. It contains advice and examples to help contributors provide consistent, high-quality data and information that will enable researchers and practitioners to make effective use of the database. Contributors from National Meteorological and Hydrological Services (NMHS), emergency management, relevant partner agencies and research institutions are the main focus group for using this questionnaire, but anyone interested is welcomed to participate.
The database questionnaire and guide are available on the HIWeather website at http://hiweather.net/Lists/130.html (last access: 15 February 2023).
2.3 High impact weather events studied
Since the start of the project in November 2020, many severe weather events have caused billions of dollars of damage and significant loss of lives (Christian Aid, 2021, 2022). During the two years, the project team has flagged 33 events of interest and started collecting data and information for 10, with two case studies fully completed and analysed. The collection comprises of severe events including heavy rain and flash flooding in Henan, China in July 2021, Hurricanes Ida (August 2021) and Ian (September 2022) in the US, Black Summer bushfires 2019/2020 in south-eastern Australia, and the Hunga Tonga Hunga Ha'apai volcano eruption in Tonga in January 2022, among others. Some of these and other case studies are briefly analysed and compared by Golding et al. (2022).
More detailed findings and lessons learnt using the concept of the warning value chain were analysed by project members for a cold-front induced ultramarathon tragedy in Jingtai County of Baiyin City, China in May 2020 (Zhang et al., 2021). Project members at the Bureau of Meteorology held an internal workshop with pre-workshop surveys interviewing forecasters, embedded meteorologists, communication specialists and managers to assess the performance of each part of the warning value chain for a heavy rain and flood event in eastern Australia in March 2021. The UK Met Office used the questionnaire to evaluate the performance of their warning system for severe windstorm Eunice in February 2022. The database questionnaire has proven to be a useful tool for these case study analyses.
The questionnaire was also used as an educational tool at the University of Miami in an undergraduate tropical meteorology course to study the warning value chain for two hurricanes as an assignment and was found to enhance the students' critical thinking about hurricane forecasting, impacts, warning communication and response (Majumdar et al., 2023). Future publications will elaborate on these case studies.
Challenges in doing case studies
During the wider use of the questionnaire, a couple of challenges were identified when collecting data and information for case study events. First, data access and availability in general is highly variable. While some information on weather forecasts, observations and satellite imagery is often easy to get via international databases, similar information on hazards is much less accessible since such forecasts are mostly performed with regional models and not stored meticulously. Hazard forecasts, and especially impact forecasts, are in their infancy and not operationally used by many NMHS which limits their accessibility further.
Secondly, the communication bridges between forecasts and warnings and other stages in the warning value chain are rarely documented and need input from the relevant institutions to get necessary details. Impact information beyond economic damages, injuries and lives lost that is not captured in other databases can be derived from photos and videos posted by observers on social media, however, this information is buried quickly and difficult to recover some time after the event. Similarly, warnings issued via different communication channels quickly disappear after the event (e.g., warning notifications on phones and apps). On the other hand, a lot of information on a case study only becomes available weeks, months, or even years after the event when economic damages and social impacts have been assessed, and – in case of a particularly severe event – a post-event assessment has been performed by the NMHS and partner agencies (e.g., emergency services). Generally, the collection of information can split into two time frames, (i) during and immediately after the event unfolded to collect “perishable” data from social media and warning communication, and (ii) much later after the event for accurate and more in-depth information. Care must be taken when using media reports as they can sensationalise events and need to be balanced with post-event surveys conducted by NMHS and/or partner agencies; however, the latter are rarely made public. In addition, copyrights should not be violated when collecting media reports, photos and videos that document the event which is in many cases, especially on social media platforms, often not clearly labelled.
By bringing attention to these challenges, we hope to inform future collaborators of these associated difficulties when taking on a case study and encourage investigators to work with the involved agencies to collect and provide this information more openly to help inform the value chain analysis.
2.4 Progress and next steps of the value chain project
Since its inception, the project has made significant progress to achieve its aims through contributions from a continuously growing, interdisciplinary project team. In addition to the interim database template for high-impact weather event case study collection and analysis, the project team also created a glossary of common terminology used by social and physical scientists and is drafting a conceptual high-level value chain framework which is planned for release at the end of 2023. A bibliography of relevant value chain literature, including for natural hazard events and warnings, was recently released to support the framework development and provide a resource for anyone interested in the concept of the value chain. We encourage researchers and practitioners to use these resources – available on the HIWeather website at http://hiweather.net/Lists/130.html (last access: 15 February 2023).
The collection and assessment of severe event warning value chains is an on-going activity in the project which benefits from any outside collaborations that help expand this collection. It will build a solid basis for in-depth analysis across events for certain parts of the value chain, evaluation of the value chain for events of similar type and/or impacted location or similar events in different countries. Parallel to these activities, we plan to transform the interim database questionnaire into an online tool and searchable database (likely by keyword) to create a more convenient resource that outlives the project. Ultimately, the knowledge obtained by analysing warnings through the lens of the information value chain will provide NMHSs and their partners with better information to collaboratively design and implement more effective warning services.
The database questionnaire, glossary, and bibliography are publicly available at http://hiweather.net/Lists/130.html (WMO, 2023).
Conceptualization, methodology, and writing – review and editing: DH, EEE, CM, BG, and SP. Writing – original draft preparation: DH and EEE. Visualization: DH, EEE, and BG. Project administration: EEE, BG, SP, DH, and CM. Funding acquisition: EEE, BG, SP. All authors have read the manuscript and agreed to be accountable for the content of the work.
The contact author has declared that none of the authors has any competing interests.
Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This article is part of the special issue “EMS Annual Meeting: European Conference for Applied Meteorology and Climatology 2022”. It is a result of the EMS Annual Meeting: European Conference for Applied Meteorology and Climatology 2022, Bonn, Germany, 4–9 September 2022. The corresponding presentation was part of session OSA1.2: Value Chains for Early Warning Systems.
We would like to acknowledge the great contributions to the value chain project from all project team members as well as the support from the WMO World Weather Research Program. Please see the full list of all team members at http://hiweather.net/Lists/138.html (WMO, 2023).
This research has been supported by the World Meteorological Organization HIWeather Trust Fund.
This paper was edited by Chiara Marsigli and reviewed by Sharanya Majumdar and Linus Magnusson.
Cawood, M., Keys, C., and Wright, C.: The total flood warning system: What have we learnt since 1990 and where are the gaps, Aust. J. Emerg. Manage., 33, 47–52, 2018.
Christian Aid: Counting the cost 2021: A year of climate breakdown (Issue December), https://reliefweb.int/report/world/counting-cost-2021-year-climate-breakdown-december-2021 (last access: 15 February 2023), 2021.
Christian Aid: Counting the Cost 2022 – A year of climate breakdown (Issue December), https://mediacentre.christianaid.org.uk/new-report-top-10-climate-disasters-cost-the-world-billions-in (last access: 15 February 2023), 2022.
Demuth, J. L., Morss, R. E., Morrow, B. H., and Lazo, J. K.: Creation and communication of hurricane risk information, B. Am. Meteorol. Soc., 93, 1133–1145, https://doi.org/10.1175/BAMS-D-11-00150.1, 2012.
Emerton, R., Cloke, H., Ficchi, A., Hawker, L., de Wit, S., Speight, L., Prudhomme, C., Rundell, P., West, R., Neal, J., Cuna, J., Harrigan, S., Titley, H., Magnusson, L., Pappenberger, F., Klingaman, N., and Stephens, E.: Emergency flood bulletins for Cyclones Idai and Kenneth: A critical evaluation of the use of global flood forecasts for international humanitarian preparedness and response, Int. J. Disast. Risk Reduct., 50, 101811, https://doi.org/10.1016/j.ijdrr.2020.101811, 2020.
Golding, B.: Towards the “Perfect” Weather Warning, in: Towards the “Perfect” Weather Warning, Springer, https://doi.org/10.1007/978-3-030-98989-7, 2022.
Golding, B., Potter, S., Ebert, E., and Hoffmann, D.: Preparing for the unprecedented, in: EMS Annual Meeting 2022, 5–9 September 2022, Bonn, Germany, EMS2022-284, https://doi.org/10.5194/ems2022-284, 2022.
IPCC: Summary for Policymakers, in: Climate Change 2021: The Physical Science Basis, Contribution of Working Group I to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change, edited by: Masson-Delmotte, V., Zhai, P., Pirani, A., Connors, S. L., Péan, C., Berger, S., Caud, N., Chen, Y., Goldfarb, L., Gomis, M. I., Huang, M., Leitzell, K., Lonnoy, E., Matthews, J. B. R., Maycock, T. K., Waterfield, T., Yelekçi, O., Yu, R., and Zhou, B., Cambridge University Press, Cambridge, UK and New York, NY, USA, 3–32, https://doi.org/10.1017/9781009157896.001, 2021.
IPCC: Summary for Policymakers, in: Climate Change 2022: Impacts, Adaptation, and Vulnerability, Contribution of Working Group II to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change edited by: Pörtner, H.-O., Roberts, D. C., Poloczanska, E. S., Mintenbeck, K., Tignor, M., Alegría, A., Craig, M., Langsdorf, S., Löschke, S., Möller, V., Okem, A., and Rama, B., Cambridge University Press, Cambridge, UK and New York, NY, USA, 3–33, https://doi.org/10.1017/9781009325844.001, 2022.
Kaltenberger, R., Schaffhauser, A., and Staudinger, M.: “What the weather will do” – results of a survey on impact-oriented and impact-based warnings in European NMHSs, Adv. Sci. Res., 17, 29–38, https://doi.org/10.5194/asr-17-29-2020, 2020.
Kelman, I., Ahmed, B., Esraz-Ul-Zannat, M., Saroar, M. M., Fordham, M., and Shamsudduha, M.: Warning systems as social processes for Bangladesh cyclones, Disast. Prev. Manage., 27, 370–379, https://doi.org/10.1108/DPM-12-2017-0318, 2018.
Lazo, J. K. and Mills, B.: Weather-Water-Climate Value Chain(s): Giving VOICE to the Characterization of the Economic Benefits of Hydro-Met Services and Products. An AMS Policy Program Study, AMS, Washington, DC, https://www.ametsoc.org/ams/assets/File/policy/WWC_Value_Chain_Economic_Benefits.pdf (last access: 15 February 2023), 2021.
Lazo, J. K., Hosterman, H. R., Sprague-Hilderbrand, J. M., and Adkins, J. E.: Impact-Based Decision Support Services and the Socioeconomic Impacts of Winter Storms, B. Am. Meteorol. Soc., 101, E626–E639, https://doi.org/10.1175/BAMS-D-18-0153.1, 2020.
Leviakangas, P.: Valuing meteorological information, Meteorol. Appl., 101, 91–101, https://doi.org/10.1002/met.122, 2009.
Majumdar, S. J., Ebert, E., Golding, B. W., and, Hoffmann, D.: Undergraduate Classroom Evaluation of the Value Chain for High-Impact Weather Events, in: 32nd Conference on Education, AMS Annual Meeting, 8–12 January 2023, Denver, https://ams.confex.com/ams/103ANNUAL/meetingapp.cgi/Session/63489 (last access: 7 July 2023), 2023.
Martinez, A. B.: Forecast accuracy matters for hurricane damage, J. Econom., 8, 18, https://doi.org/10.3390/econometrics8020018, 2020.
Merz, B., Kuhlicke, C., Kunz, M., Pittore, M., Babeyko, A., Bresch, D. N., Domeisen, D. I. V, Feser, F., Koszalka, I., Kreibich, H., Pantillon, F., Parolai, S., Pinto, J. G., Punge, H. J., Rivalta, E., Schröter, K., Strehlow, K., Weisse, R., and Wurpts, A.: Impact Forecasting to Support Emergency Management of Natural Hazards, Rev. Geophys., 58, 1–52, https://doi.org/10.1029/2020RG000704, 2020.
Morss, R. E., Vickery, J., Lazrus, H., Demuth, J., and Bostrom, A.: Improving Tropical Cyclone Forecast Communication by Understanding NWS Partners' Decision Timelines and Forecast Information Needs, Weather Clim. Soc., 14, 783–800, https://doi.org/10.1175/wcas-d-21-0170.1, 2022.
Msemo, H. E., Taylor, A. L., Birch, C. E., Dougill, A. J., and Hartley, A.: The value of weather and climate information to the tanzanian disaster risk reduction sector using nonmonetary approaches, Weather Clim. Soc., 13(, 1055–1068, https://doi.org/10.1175/WCAS-D-21-0005.1, 2021.
Perrels, A., Nurmi, V., and Nurmi, P.: Weather service chain analysis (WSCA) –An approach for appraisal of the social-economic benefits of improvements in weather services, in: Proceedings of the 16th International Road Weather Conference, SIRWEC 2012, Helsinki, Finland, 23–25 May 2012, https://www.researchgate.net/publication/233776312_Weather_service_chain_analysis_WSCA_-_An_approach_for_appraisal_of_the_social-economic_benefits_of_improvements_in_weather_services (last access: 15 February 2023), 2012.
Potter, S., Harrison, S., and Kreft, P.: The Benefits and Challenges of Implementing Impact-Based Severe Weather Warning Systems: Perspectives of Weather, Flood, and Emergency Management Personnel, Weather Clim. Soc., 13, 303–314, https://doi.org/10.1175/wcas-d-20-0110.1, 2021.
Rodwell, M. J., Hammond, J., Thornton, S., and Richardson, D. S.: User decisions, and how these could guide developments in probabilistic forecasting, Q. J. Roy. Meteorol. Soc., 146, 3266–3284, https://doi.org/10.1002/qj.3845, 2020.
Tan, M. L., Hoffmann, D., Ebert, E., Cui, A., and Johnston, D.: Exploring the potential role of citizen science in the warning value chain for high impact weather, Front. Commun., 7, 949949, https://doi.org/10.3389/fcomm.2022.949949, 2022.
UNDRR: Sendai Framework for Disaster Risk Reduction 2015–2030, WMO publications, https://www.undrr.org/publication/sendai-framework-disaster-risk-reduction-2015-2030 (last access: 15 February 2023), 2015.
United Nations/Framework Convention on Climate Change: Adoption of the Paris Agreement, in: 21st Conference of the Parties, Towards a Climate-Neutral Europe: Curbing the Trend, An Official Publication, United Nations, Paris, https://doi.org/10.4324/9789276082569-2, 2015.
WMO: Valuing Weather and Climate, World Meteorological Organization, ISBN 978-92-63-11153-1, 2015.
WMO: UN seeks to track hazardous events and disaster losses and damages, https://public.wmo.int/en/media/news/un-seeks-track-hazardous-events-and-disaster-losses-and-damages (last access: 7 November 2022), 2022a.
WMO: Early Warnings for All Action Plan unveiled at COP27, https://public.wmo.int/en/media/press-release/early-warnings-all-action-plan-unveiled-cop27#:~:text=TheExecutive Action Plan for,50 billion in adaptation financing (last access: 7 November 2022), 2022b.
WMO: World Meteorological Organization: HIWeather Value Chain Project, http://hiweather.net/Lists/130.html (last access: 15 February 2023), 2023.
Wu, W., Emerton, R., Duan, Q., Wood, A. W., Wetterhall, F., and Robertson, D. E.: Ensemble flood forecasting: Current status and future opportunities, WIREs Water, 7, 1–32, https://doi.org/10.1002/wat2.1432, 2020.
Zhang, Q., Li, L., Ebert, B., Golding, B., Johnston, D., Mills, B., Panchuk, S., Potter, S., Riemer, M., Sun, J., Taylor, A., Jones, S., Ruti, P., and Keller, J.: Increasing the value of weather-related warnings, Sci. Bull., 64, 647–649, https://doi.org/10.1016/j.scib.2019.04.003, 2019.
Zhang, Q., Ng, C.-P., Dai, K., Xu, J., Tang, J., Sun, J., and Mu, M.: Lessons Learned from the Tragedy during the 100 km Ultramarathon Race in Baiyin, Gansu Province on 22 May 2021, Adv. Atmos. Sci., 38, 1803–1810, https://doi.org/10.1007/s00376-021-1246-0, 2021.