Forecaster perspectives on model quality and verification at the Norwegian Meteorological Institute
Development Centre for Weather Forecasting, Norwegian Meteorological Institute, Oslo, Norway
Morten Køltzow
Development Centre for Weather Forecasting, Norwegian Meteorological Institute, Oslo, Norway
Andrew Singleton
Development Centre for Weather Forecasting, Norwegian Meteorological Institute, Oslo, Norway
Lene Østvand
Development Centre for Weather Forecasting, Norwegian Meteorological Institute, Oslo, Norway
Mette Sundvor Skjerdal
Development Centre for Weather Forecasting, Norwegian Meteorological Institute, Bergen, Norway
Bjørn Gilje Lillegraven
Development Centre for Weather Forecasting, Norwegian Meteorological Institute, Oslo, Norway
Gunnar Noer
Development Centre for Weather Forecasting, Norwegian Meteorological Institute, Tromsø, Norway
Related authors
Jonathan J. Day, Gunilla Svensson, Barbara Casati, Taneil Uttal, Siri-Jodha Khalsa, Eric Bazile, Elena Akish, Niramson Azouz, Lara Ferrighi, Helmut Frank, Michael Gallagher, Øystein Godøy, Leslie M. Hartten, Laura X. Huang, Jareth Holt, Massimo Di Stefano, Irene Suomi, Zen Mariani, Sara Morris, Ewan O'Connor, Roberta Pirazzini, Teresa Remes, Rostislav Fadeev, Amy Solomon, Johanna Tjernström, and Mikhail Tolstykh
Geosci. Model Dev., 17, 5511–5543, https://doi.org/10.5194/gmd-17-5511-2024, https://doi.org/10.5194/gmd-17-5511-2024, 2024
Short summary
Short summary
The YOPP site Model Intercomparison Project (YOPPsiteMIP), which was designed to facilitate enhanced weather forecast evaluation in polar regions, is discussed here, focussing on describing the archive of forecast data and presenting a multi-model evaluation at Arctic supersites during February and March 2018. The study highlights an underestimation in boundary layer temperature variance that is common across models and a related inability to forecast cold extremes at several of the sites.
Lukas Frank, Marius Opsanger Jonassen, Teresa Remes, Florina Roana Schalamon, and Agnes Stenlund
Earth Syst. Sci. Data, 15, 4219–4234, https://doi.org/10.5194/essd-15-4219-2023, https://doi.org/10.5194/essd-15-4219-2023, 2023
Short summary
Short summary
The Isfjorden Weather Information Network (IWIN) provides continuous meteorological near-surface observations from Isfjorden in Svalbard. The network combines permanent automatic weather stations on lighthouses along the coast line with mobile stations on board small tourist cruise ships regularly trafficking the fjord during spring to autumn. All data are available online in near-real time. Besides their scientific value, IWIN data crucially enhance the safety of field activities in the region.
Samuel Ephraim, Paquita Zuidema, Aaron Bansemer, Lintong Cai, Markus Petters, Elise Rosky, Jefferson R. Snider, Zhien Wang, Sarah Woods, Kevin Barry, Theresa Campos, Owen Cruikshank, Sabine Eckhardt, Nikolaos Evangeliou, Romanos Foskinis, Jeffrey R. French, Bart Geerts, Coltin Grasmick, Silvia Henning, Varun Kumar, Andreas Massling, Camille Mavis, Greg M. McFarquhar, Athanasios Nenes, Gunnar Noer, Ryan Patnaude, Russell Perkins, Lise Lotte Sorensen, Henrik Skov, Tyler Tatro, Florian Tornow, and Lu Zhang
EGUsphere, https://doi.org/10.5194/egusphere-2026-2940, https://doi.org/10.5194/egusphere-2026-2940, 2026
Short summary
Short summary
The processes that establish how mixed-phase closed-cell clouds transition to more open cellular structures are poorly known. First-of-its kind aircraft observations document such a transition in the presence of anomalously high aerosol concentrations over the Nordic Seas at cloud temperatures < -15 °C. The reduces the drop size, discouraging riming. Eventually, ice precipitation produces surface cold pools that drive the convective transition, despite strong counteracting surface fluxes.
Jonathan J. Day, Gunilla Svensson, Barbara Casati, Taneil Uttal, Siri-Jodha Khalsa, Eric Bazile, Elena Akish, Niramson Azouz, Lara Ferrighi, Helmut Frank, Michael Gallagher, Øystein Godøy, Leslie M. Hartten, Laura X. Huang, Jareth Holt, Massimo Di Stefano, Irene Suomi, Zen Mariani, Sara Morris, Ewan O'Connor, Roberta Pirazzini, Teresa Remes, Rostislav Fadeev, Amy Solomon, Johanna Tjernström, and Mikhail Tolstykh
Geosci. Model Dev., 17, 5511–5543, https://doi.org/10.5194/gmd-17-5511-2024, https://doi.org/10.5194/gmd-17-5511-2024, 2024
Short summary
Short summary
The YOPP site Model Intercomparison Project (YOPPsiteMIP), which was designed to facilitate enhanced weather forecast evaluation in polar regions, is discussed here, focussing on describing the archive of forecast data and presenting a multi-model evaluation at Arctic supersites during February and March 2018. The study highlights an underestimation in boundary layer temperature variance that is common across models and a related inability to forecast cold extremes at several of the sites.
Lukas Frank, Marius Opsanger Jonassen, Teresa Remes, Florina Roana Schalamon, and Agnes Stenlund
Earth Syst. Sci. Data, 15, 4219–4234, https://doi.org/10.5194/essd-15-4219-2023, https://doi.org/10.5194/essd-15-4219-2023, 2023
Short summary
Short summary
The Isfjorden Weather Information Network (IWIN) provides continuous meteorological near-surface observations from Isfjorden in Svalbard. The network combines permanent automatic weather stations on lighthouses along the coast line with mobile stations on board small tourist cruise ships regularly trafficking the fjord during spring to autumn. All data are available online in near-real time. Besides their scientific value, IWIN data crucially enhance the safety of field activities in the region.
Cited articles
Ainslie, B., So, R., and Chen, J.: Operational Evaluation of a Wildfire Air Quality Model from a Forecaster Point of View, Weather Forecast., 37, 681–698, https://doi.org/10.1175/WAF-D-21-0064.1, 2022. a, b
Bouallegue, Z. B., Haiden, T., Weber, N. J., Hamill, T. M., and Richardson, D. S.: Accounting for Representativeness in the Verification of Ensemble Precipitation Forecasts, Mon. Weather Rev., 148, 2049–2062, https://doi.org/10.1175/MWR-D-19-0323.1, 2020. a
Bröcker, J., Driscoll, S., Necker, T., Rodríguez, J., Dacre, H., Harvey, N., and Ben Bouallègue, Z.: Verification of AI-based environmental forecasting systems: What can we do, what do we need to do, and what are the challenges?, J. Eur. Meteorol. Soc., 4, 100032, https://doi.org/10.1016/j.jemets.2026.100032, 2026. a
Casati, B., Wilson, L. J., Stephenson, D. B., Nurmi, P., Ghelli, A., Pocernich, M., Damrath, U., Ebert, E. E., Brown, B. G., and Mason, S.: Forecast verification: current status and future directions, Meteorol. Appl., 15, 3–18, https://doi.org/10.1002/met.52, 2008. a, b
Casati, B., Loveday, N., Bouallègue, Z. B., Bröcker, J., Wahl, S., de Elía, R., Pic, R., Subich, C., Separovic, L., Brunet, D., and McTaggart-Cowan, R.: Operational Verification of Artificial Intelligence Weather Prediction, B. Am. Meteorol. Soc., BAMS–D–26–0054.1, https://doi.org/10.1175/BAMS-D-26-0054.1, 2026. a
Demuth, J., Morss, R., Jankov, I., Alcott, T., Alexander, C., Nietfeld, D., Jensen, T., Novak, D., and Benjamin, S.: Recommendations for Developing Useful and Usable Convection-Allowing Model Ensemble Information for NWS Forecasters, Weather Forecast., 35, 1381–1406, https://doi.org/10.1175/WAF-D-19-0108.1, 2020. a, b, c
Dorninger, M., Gilleland, E., Casati, B., Mittermaier, M. P., Ebert, E. E., Brown, B. G., and Wilson, L. J.: The Setup of the MesoVICT Project, B. Am. Meteorol. Soc., 99, 1887–1906, https://doi.org/10.1175/BAMS-D-17-0164.1, 2018. a
Doswell III, C.: Weather Forecasting by Humans—Heuristics and Decision Making, Weather Forecast., 19, 1115–1126, https://doi.org/10.1175/WAF-821.1, 2004. a, b, c
ECMWF: IFS Documentation CY49R1, https://www.ecmwf.int/en/publications/ifs-documentation (last access: 4 August 2026), 2024. a
Eresmaa, R., Andrae, U., Berggren, L., Bremnes, J. B., Fortelius, C., Frogner, I., Gregow, E., Grote, R., Ridal, M., Vignes, O., Ansper, I., Azad, R., Begens, J., Bergholt, L., Blyverket, J., Daniel, L., Engdahl, B. J., Erlandsen, H., Eshagh, M., Hasu, M., Homleid, M., Ideström, P., Ivarsson, K., Lerner-Vilu, A., Mets, A., Moldengauere, S., Noer, G., Nurmi, E., Partio, M., Priedītis, G., Saarikalle, E., Schönach, D., Shapkalijevski, M., Sild, K., SitĊihins, V., Sokka, N., Spjelkavik, S., Tack, A., Vaštšenko, A., Wettergren, A., Ylinen, K., and Zandovskis, U.: The Operational Forecast Process at MetCoOp, B. Am. Meteorol. Soc., 107, E946–E963, https://doi.org/10.1175/BAMS-D-25-0225.1, 2026. a
Frogner, I.-L., Singleton, A. T., Køltzow, M., and Andrae, U.: Convection-permitting ensembles: Challenges related to their design and use, Q. J. Roy. Meteor. Soc., 145, 90–106, https://doi.org/10.1002/qj.3525, 2019. a
Fundel, V. J., Fleischhut, N., Herzog, S. M., Göber, M., and Hagedorn, R.: Promoting the use of probabilistic weather forecasts through a dialogue between scientists, developers and end-users, Q. J. Roy. Meteor. Soc., 145, 210–231, https://doi.org/10.1002/qj.3482, 2019. a, b
Gilleland, E., Ahijevych, D., Brown, B. G., Casati, B., and Ebert, E. E.: Intercomparison of Spatial Forecast Verification Methods, Weather Forecast., 24, 1416–1430, https://doi.org/10.1175/2009WAF2222269.1, 2009. a
Göber, M., Zsótér, E., and Richardson, D. S.: Could a perfect model ever satisfy a naïve forecaster? On grid box mean versus point verification, Meteorol. Appl., 15, 359–365, https://doi.org/10.1002/met.78, 2008. a, b
Kain, J., Janish, P., Weiss, S., Baldwin, M., Schneider, R., and Brooks, H.: Collaboration between Forecasters and Research Scientists at the NSSL and SPC: The Spring Program, B. Am. Meteorol. Soc., 84, 1797–1806, https://doi.org/10.1175/BAMS-84-12-1797, 2003. a, b, c, d
Køltzow, M., Casati, B., Bazile, E., Haiden, T., and Valkonen, T.: An NWP Model Intercomparison of Surface Weather Parameters in the European Arctic during the Year of Polar Prediction Special Observing Period Northern Hemisphere 1, Weather Forecast., 34, 959–983, https://doi.org/10.1175/WAF-D-19-0003.1, 2019. a, b, c, d
Køltzow, M., Casati, B., Haiden, T., and Valkonen, T.: Verification of Solid Precipitation Forecasts from Numerical Weather Prediction Models in Norway, Weather Forecast., 35, 2279–2292, https://doi.org/10.1175/WAF-D-20-0060.1, 2020. a, b
Køltzow, M., Schyberg, H., Støylen, E., and Yang, X.: Value of the Copernicus Arctic Regional Reanalysis (CARRA) in representing near-surface temperature and wind speed in the north-east European Arctic, Polar Res., 41, https://doi.org/10.33265/polar.v41.8002, 2022. a
Køltzow, M., Øestad, M., Azad, R., Rugaard Furevik, B., Olsen, A.-M., and Remes, T.: How inaccurate offshore wind observations impact the quality of operational numerical weather prediction, Q. J. Roy. Meteor. Soc., 150, 5337–5355, https://doi.org/10.1002/qj.4872, 2024. a
Lillegraven, B. G., Noer, G., and Østvand, L.: Verification of Operational Weather Prediction Models: September to November 2024, METinfo 21/2025, Norwegian Meteorological Institute, ISSN 1894-759X, https://www.met.no/publikasjoner/met-info/ (last access: 4 August 2026), 2025a. a
Lillegraven, B. G., Noer, G., and Østvand, L.: Verification of Operational Weather Prediction Models: December 2024 to February 2025, METinfo 31/2025, Norwegian Meteorological Institute, ISSN 1894-759X, https://www.met.no/publikasjoner/met-info/ (last access: 4 August 2026), 2025b. a
Lillegraven, B. G., Noer, G., and Østvand, L.: Verification of Operational Weather Prediction Models: March to May 2025, METinfo 36/2025, Norwegian Meteorological Institute, ISSN 1894-759X, https://www.met.no/publikasjoner/met-info/ (last access: 4 August 2026), 2025c. a
Lillegraven, B. G., Noer, G., and Østvand, L.: Verification of Operational Weather Prediction Models: September to November 2025, METinfo 18/2026, Norwegian Meteorological Institute, ISSN 1894-759X, https://www.met.no/publikasjoner/met-info/ (last access: 4 August 2026), 2026. a
Müller, M., Batrak, Y., Kristiansen, J., Køltzow, M. A., Noer, G., and Korosov, A.: Characteristics of a Convective-Scale Weather Forecasting System for the European Arctic, Mon. Weather Rev., 145, 4771–4787, https://doi.org/10.1175/MWR-D-17-0194.1, 2017. a
Murphy, A.: What Is a Good Forecast? An Essay on the Nature of Goodness in Weather Forecasting, Weather Forecast., 8, 281–293, https://doi.org/10.1175/1520-0434(1993)008<0281:WIAGFA>2.0.CO;2, 1993. a, b
Padilla, L. M. K., Powell, M., Kay, M., and Hullman, J.: Uncertain About Uncertainty: How Qualitative Expressions of Forecaster Confidence Impact Decision-Making With Uncertainty Visualizations, Front. Psychol., 11, 579267, https://doi.org/10.3389/fpsyg.2020.579267, 2021. a, b, c
Pagano, T. C., Casati, B., Landman, S., Loveday, N., Taggart, R., Ebert, E. E., Khanarmuei, M., Jensen, T. L., Mittermaier, M., Roberts, H., Willington, S., Roberts, N., Sowko, M., Strassberg, G., Kluepfel, C., Bullock, T. A., Turner, D. D., Pappenberger, F., Osborne, N., and Noble, C.: Challenges of Operational Weather Forecast Verification and Evaluation, B. Am. Meteorol. Soc., 105, E789–E802, https://doi.org/10.1175/BAMS-D-22-0257.1, 2024. a, b
Pagano, T. C., Ebert, E. E., and Khanarmuei, M.: Enhancing Forecast Verification in National Meteorological and Hydrological Services, Meteorol. Appl., 32, e70051, https://doi.org/10.1002/met.70051, 2025. a
Remes, T., Køltzow, M., and Kähnert, M.: Spatial variability of near-surface air temperature in the Copernicus Arctic regional reanalysis, Q. J. Roy. Meteor. Soc., 151, e4941, https://doi.org/10.1002/qj.4941, 2025. a
Steenburgh, W. J., Schultz, D. M., Snyder, B. J., and Meyers, M. P.: Bridging the Gap Between Operations and Research to Improve Weather Prediction in Mountainous Regions, in: Mountain Weather Research and Forecasting, Springer, https://doi.org/10.1007/978-94-007-4098-3_12, 2013. a, b
Vigna, I., Jeuring, J., and Doksæter Sivle, A.: Melkeværet: A serious game to bridge climate service providers and agricultural users, Tech. Rep. MET report 16/2024, Norwegian Meteorological Institute, https://www.met.no/publikasjoner/met-report/met-report-2024 (last access: 4 August 2026), 2024. a
Short summary
Success of weather models depends on how forecasters use and trust them. We held workshops with Norwegian forecasters to understand how they evaluate model quality and use it in their work. Forecasters rely on experience, peer discussions, and past events. They need tools that show model quality in specific situations and help them make decisions during shifts. Their views should guide tool development. This study highlights the value of collaboration between model developers and forecasters.
Success of weather models depends on how forecasters use and trust them. We held workshops with...