Evaluating co-creation of knowledge: from quality criteria and indicators to methods
Susanne Schuck-Zöller
CORRESPONDING AUTHOR
Climate Service Center Germany, Helmholtz-Zentrum Geesthacht, 20095
Hamburg, Germany
Jörg Cortekar
Climate Service Center Germany, Helmholtz-Zentrum Geesthacht, 20095
Hamburg, Germany
Daniela Jacob
Climate Service Center Germany, Helmholtz-Zentrum Geesthacht, 20095
Hamburg, Germany
Related authors
No articles found.
Colin G. Jones, Fanny Adloff, Ben B. B. Booth, Peter M. Cox, Veronika Eyring, Pierre Friedlingstein, Katja Frieler, Helene T. Hewitt, Hazel A. Jeffery, Sylvie Joussaume, Torben Koenigk, Bryan N. Lawrence, Eleanor O'Rourke, Malcolm J. Roberts, Benjamin M. Sanderson, Roland Séférian, Samuel Somot, Pier Luigi Vidale, Detlef van Vuuren, Mario Acosta, Mats Bentsen, Raffaele Bernardello, Richard Betts, Ed Blockley, Julien Boé, Tom Bracegirdle, Pascale Braconnot, Victor Brovkin, Carlo Buontempo, Francisco Doblas-Reyes, Markus Donat, Italo Epicoco, Pete Falloon, Sandro Fiore, Thomas Frölicher, Neven S. Fučkar, Matthew J. Gidden, Helge F. Goessling, Rune Grand Graversen, Silvio Gualdi, José M. Gutiérrez, Tatiana Ilyina, Daniela Jacob, Chris D. Jones, Martin Juckes, Elizabeth Kendon, Erik Kjellström, Reto Knutti, Jason Lowe, Matthew Mizielinski, Paola Nassisi, Michael Obersteiner, Pierre Regnier, Romain Roehrig, David Salas y Mélia, Carl-Friedrich Schleussner, Michael Schulz, Enrico Scoccimarro, Laurent Terray, Hannes Thiemann, Richard A. Wood, Shuting Yang, and Sönke Zaehle
Earth Syst. Dynam., 15, 1319–1351, https://doi.org/10.5194/esd-15-1319-2024, https://doi.org/10.5194/esd-15-1319-2024, 2024
Short summary
Short summary
We propose a number of priority areas for the international climate research community to address over the coming decade. Advances in these areas will both increase our understanding of past and future Earth system change, including the societal and environmental impacts of this change, and deliver significantly improved scientific support to international climate policy, such as future IPCC assessments and the UNFCCC Global Stocktake.
Bjorn Stevens, Stefan Adami, Tariq Ali, Hartwig Anzt, Zafer Aslan, Sabine Attinger, Jaana Bäck, Johanna Baehr, Peter Bauer, Natacha Bernier, Bob Bishop, Hendryk Bockelmann, Sandrine Bony, Guy Brasseur, David N. Bresch, Sean Breyer, Gilbert Brunet, Pier Luigi Buttigieg, Junji Cao, Christelle Castet, Yafang Cheng, Ayantika Dey Choudhury, Deborah Coen, Susanne Crewell, Atish Dabholkar, Qing Dai, Francisco Doblas-Reyes, Dale Durran, Ayoub El Gaidi, Charlie Ewen, Eleftheria Exarchou, Veronika Eyring, Florencia Falkinhoff, David Farrell, Piers M. Forster, Ariane Frassoni, Claudia Frauen, Oliver Fuhrer, Shahzad Gani, Edwin Gerber, Debra Goldfarb, Jens Grieger, Nicolas Gruber, Wilco Hazeleger, Rolf Herken, Chris Hewitt, Torsten Hoefler, Huang-Hsiung Hsu, Daniela Jacob, Alexandra Jahn, Christian Jakob, Thomas Jung, Christopher Kadow, In-Sik Kang, Sarah Kang, Karthik Kashinath, Katharina Kleinen-von Königslöw, Daniel Klocke, Uta Kloenne, Milan Klöwer, Chihiro Kodama, Stefan Kollet, Tobias Kölling, Jenni Kontkanen, Steve Kopp, Michal Koran, Markku Kulmala, Hanna Lappalainen, Fakhria Latifi, Bryan Lawrence, June Yi Lee, Quentin Lejeun, Christian Lessig, Chao Li, Thomas Lippert, Jürg Luterbacher, Pekka Manninen, Jochem Marotzke, Satoshi Matsouoka, Charlotte Merchant, Peter Messmer, Gero Michel, Kristel Michielsen, Tomoki Miyakawa, Jens Müller, Ramsha Munir, Sandeep Narayanasetti, Ousmane Ndiaye, Carlos Nobre, Achim Oberg, Riko Oki, Tuba Özkan-Haller, Tim Palmer, Stan Posey, Andreas Prein, Odessa Primus, Mike Pritchard, Julie Pullen, Dian Putrasahan, Johannes Quaas, Krishnan Raghavan, Venkatachalam Ramaswamy, Markus Rapp, Florian Rauser, Markus Reichstein, Aromar Revi, Sonakshi Saluja, Masaki Satoh, Vera Schemann, Sebastian Schemm, Christina Schnadt Poberaj, Thomas Schulthess, Cath Senior, Jagadish Shukla, Manmeet Singh, Julia Slingo, Adam Sobel, Silvina Solman, Jenna Spitzer, Philip Stier, Thomas Stocker, Sarah Strock, Hang Su, Petteri Taalas, John Taylor, Susann Tegtmeier, Georg Teutsch, Adrian Tompkins, Uwe Ulbrich, Pier-Luigi Vidale, Chien-Ming Wu, Hao Xu, Najibullah Zaki, Laure Zanna, Tianjun Zhou, and Florian Ziemen
Earth Syst. Sci. Data, 16, 2113–2122, https://doi.org/10.5194/essd-16-2113-2024, https://doi.org/10.5194/essd-16-2113-2024, 2024
Short summary
Short summary
To manage Earth in the Anthropocene, new tools, new institutions, and new forms of international cooperation will be required. Earth Virtualization Engines is proposed as an international federation of centers of excellence to empower all people to respond to the immense and urgent challenges posed by climate change.
Swantje Preuschmann, Tanja Blome, Knut Görl, Fiona Köhnke, Bettina Steuri, Juliane El Zohbi, Diana Rechid, Martin Schultz, Jianing Sun, and Daniela Jacob
Adv. Sci. Res., 19, 51–71, https://doi.org/10.5194/asr-19-51-2022, https://doi.org/10.5194/asr-19-51-2022, 2022
Short summary
Short summary
The main aspect of the paper is to obtain transferable principles for the development of digital knowledge transfer products. As such products are still unstandardised, the authors explored challenges and approaches for product developments. The authors report what they see as useful principles for developing digital knowledge transfer products, by describing the experience of developing the Net-Zero-2050 Web-Atlas and the "Bodenkohlenstoff-App".
Dmitry V. Sein, Anton Y. Dvornikov, Stanislav D. Martyanov, William Cabos, Vladimir A. Ryabchenko, Matthias Gröger, Daniela Jacob, Alok Kumar Mishra, and Pankaj Kumar
Earth Syst. Dynam., 13, 809–831, https://doi.org/10.5194/esd-13-809-2022, https://doi.org/10.5194/esd-13-809-2022, 2022
Short summary
Short summary
The effect of the marine biogeochemical variability upon the South Asian regional climate has been investigated. In the experiment where its full impact is activated, the average sea surface temperature is lower over most of the ocean. When the biogeochemical coupling is included, the main impacts include the enhanced phytoplankton primary production, a shallower thermocline, decreased SST and water temperature in subsurface layers.
Kevin Sieck, Christine Nam, Laurens M. Bouwer, Diana Rechid, and Daniela Jacob
Earth Syst. Dynam., 12, 457–468, https://doi.org/10.5194/esd-12-457-2021, https://doi.org/10.5194/esd-12-457-2021, 2021
Short summary
Short summary
This paper presents new estimates of future extreme weather in Europe, including extreme heat, extreme rainfall and meteorological drought. These new estimates were achieved by repeating model calculations many times, thereby reducing uncertainties of these rare events at low levels of global warming at 1.5 and 2 °C above
pre-industrial temperature levels. These results are important, as they help to assess which weather extremes could increase at moderate warming levels and where.
Bettina Steuri, Tanja Blome, Katharina Bülow, Juliane El Zohbi, Peter Hoffmann, Juliane Petersen, Susanne Pfeifer, Diana Rechid, and Daniela Jacob
Adv. Sci. Res., 17, 9–17, https://doi.org/10.5194/asr-17-9-2020, https://doi.org/10.5194/asr-17-9-2020, 2020
Short summary
Short summary
The goal of an interdisciplinary team of scientists at the Climate Service Center Germany (GERICS) was to make the findings of the special report IPCC SR1.5 more accessible to the citizens of Hamburg. Therefore, a flyer was created that is understandable to non-climate scientists, visually attractive and generates interest.
In this article, the authors provide insights into their teamwork, the underlying guiding principles as well as lessons learned that are of great value for future projects.
Erik Kjellström, Grigory Nikulin, Gustav Strandberg, Ole Bøssing Christensen, Daniela Jacob, Klaus Keuler, Geert Lenderink, Erik van Meijgaard, Christoph Schär, Samuel Somot, Silje Lund Sørland, Claas Teichmann, and Robert Vautard
Earth Syst. Dynam., 9, 459–478, https://doi.org/10.5194/esd-9-459-2018, https://doi.org/10.5194/esd-9-459-2018, 2018
Short summary
Short summary
Based on high-resolution regional climate models we investigate European climate change at 1.5 and 2 °C of global warming compared to pre-industrial levels. Considerable near-surface warming exceeding that of the global mean is found for most of Europe, already at the lower 1.5 °C of warming level. Changes in precipitation and near-surface wind speed are identified. The 1.5 °C of warming level shows significantly less change compared to the 2 °C level, indicating the importance of mitigation.
Joni-Pekka Pietikäinen, Tiina Markkanen, Kevin Sieck, Daniela Jacob, Johanna Korhonen, Petri Räisänen, Yao Gao, Jaakko Ahola, Hannele Korhonen, Ari Laaksonen, and Jussi Kaurola
Geosci. Model Dev., 11, 1321–1342, https://doi.org/10.5194/gmd-11-1321-2018, https://doi.org/10.5194/gmd-11-1321-2018, 2018
Short summary
Short summary
The regional climate model REMO was coupled with the FLake lake model to include an interactive treatment of lakes. Using this new version, the Fenno-Scandinavian climate and lake characteristics were studied. Our results show that overall the new model version improves the representation of the Fenno-Scandinavian climate in terms of 2 m temperature and precipitation and that the model can reproduce surface water temperature, ice depth and ice season length with reasonably high accuracy.
William J. Gutowski Jr., Filippo Giorgi, Bertrand Timbal, Anne Frigon, Daniela Jacob, Hyun-Suk Kang, Krishnan Raghavan, Boram Lee, Christopher Lennard, Grigory Nikulin, Eleanor O'Rourke, Michel Rixen, Silvina Solman, Tannecia Stephenson, and Fredolin Tangang
Geosci. Model Dev., 9, 4087–4095, https://doi.org/10.5194/gmd-9-4087-2016, https://doi.org/10.5194/gmd-9-4087-2016, 2016
Short summary
Short summary
The Coordinated Regional Downscaling Experiment (CORDEX) is a diagnostic MIP in CMIP6. CORDEX builds on a foundation of previous downscaling intercomparison projects to provide a common framework for downscaling activities around the world. The CORDEX Regional Challenges provide a focus for downscaling research and a basis for making use of CMIP6 global output to produce downscaled projected changes in regional climates, and assess sources of uncertainties in the projections.
B. Eggert, P. Berg, J. O. Haerter, D. Jacob, and C. Moseley
Atmos. Chem. Phys., 15, 5957–5971, https://doi.org/10.5194/acp-15-5957-2015, https://doi.org/10.5194/acp-15-5957-2015, 2015
E. Katragkou, M. García-Díez, R. Vautard, S. Sobolowski, P. Zanis, G. Alexandri, R. M. Cardoso, A. Colette, J. Fernandez, A. Gobiet, K. Goergen, T. Karacostas, S. Knist, S. Mayer, P. M. M. Soares, I. Pytharoulis, I. Tegoulias, A. Tsikerdekis, and D. Jacob
Geosci. Model Dev., 8, 603–618, https://doi.org/10.5194/gmd-8-603-2015, https://doi.org/10.5194/gmd-8-603-2015, 2015
S. Kotlarski, K. Keuler, O. B. Christensen, A. Colette, M. Déqué, A. Gobiet, K. Goergen, D. Jacob, D. Lüthi, E. van Meijgaard, G. Nikulin, C. Schär, C. Teichmann, R. Vautard, K. Warrach-Sagi, and V. Wulfmeyer
Geosci. Model Dev., 7, 1297–1333, https://doi.org/10.5194/gmd-7-1297-2014, https://doi.org/10.5194/gmd-7-1297-2014, 2014
A. I. Hienola, J.-P. Pietikäinen, D. Jacob, R. Pozdun, T. Petäjä, A.-P. Hyvärinen, L. Sogacheva, V.-M. Kerminen, M. Kulmala, and A. Laaksonen
Atmos. Chem. Phys., 13, 4033–4055, https://doi.org/10.5194/acp-13-4033-2013, https://doi.org/10.5194/acp-13-4033-2013, 2013
Cited articles
Bergmann, M., Jahn, T., Knobloch, T., Krohn, W., Pohl, C., Brohmann, B., Hoffmann, E., Loibl, M. C., Rehaag, R., Schramm, E., and Voß, J. P.: Quality Criteria of Transdisciplinary Research – A Guide for the Formative Evaluation of Research Projects, ISOE-Studientexte, 13, Institute for Social-Ecological Research (ISOE), 78 pp., Frankfurt/M., Germany, 2005.
Bergmann, M., Jahn, T., Knobloch, T., Krohn, W., Pohl, C., and Schramm, E.: Methods for Transdisciplinary Research, Campus, Frankfurt/M., Germany, 2012.
Climate Service Center Germany: 29 publications referring to the evaluation of criteria and indicators, measuring co-creation processes, availabe at: http://www.gerics.de/imperia/md/content/csc/gerics/literaturliste_review.pdf, last access: 26 October 2017.
Daschkeit, A. and Loibl, M. C.: Projektspezifische Evaluation human-ökologischer Nachhaltigkeitsforschung, in: Evaluation inter- und transdisziplinärer Forschung, edited by: Pohl, C. and Stoll-Kleemann, S., Edition Humanökologie, 5, Oekom, München, Germany, 157–173, 2007.
European Commission: A European research and innovation Roadmap for Climate Services, Bruxelles, available at: http://www.wmo.int (last access: 20 December 2016), 2015.
Godin, B. and Doré, C.: Measuring the impacts of science – beyond the economic dimension, Urbanisation INRS, Culture et Société, available at: http://www.csiic.ca/PDF/Godin_Dore_Impacts.pdf (last access: 20 December 2016), 2005.
Hassenforder, E., Smajgl, A., and Ward, J.: Towards understanding participatory processes: Framework, application and results, J. Environ. Manage., 157, 84–95, 2015.
Jahn, T., Bergmann, M., and Keil, F.: Transdisciplinarity between mainstreaming and marginalization, Ecological Economics, 79, 1–10, 2012.
Jahn, T. and Keil, F.: An actor-specific guideline for quality assurance in transdisciplinary research, Futures, 65, 195–208, 2015.
Kaufmann, A. and Kasztler, A.: Differences in publication and dissemination practices between disciplinary and transdisciplinary science and the consequences for research evaluation, Science and Public Policy, 36, 215–227, 2009.
Klein, J. T.: Evaluation of interdisciplinary and transdisciplinary research, Am. J. Prev. Med., 35, S116–123, 2008.
Masse, L. C., Moser, R. P., Stokols, D., Taylor, B. K., Marcus, S. E., Morgan, G. D., Hall, K. L., Croyle, R. T., and Trochim, W. M.: Measuring collaboration and transdisciplinary integration in team science, Am. J. Prev. Med., 35, S151–160, 2008.
Mauser, W., Klepper, G., Rice, M., Schmalzbauer, B. S., Hackmann, H., Leemans, R., and Moore, H.: Transdisciplinary global change research: the co-creation of knowledge for sustainability, Current Opinion in Environmental Sustainability, 5, 420–431, 2013.
OECD: Glossary of key terms in evaluation and results based management, available at: http://www.oecd.org/development/peer-reviews/2754804.pdf (last access: 14 July 2016), 2002.
Pohl, C. and Hirsch Hadorn, G.: Principles for designing transdisciplinary research. A proposition by the Swiss Academies of Arts and Sciences, Oekom, München, Germany, 120 pp., 2007.
Rafols, I., Laydesdorff, L., O'Hare, A., Nightingale, P., and Stirling, A.: How journal rankings can suppress interdisciplinary research, Res. Policy, 41, 1262–1282, https://doi.org/10.1016/J.respol.2012.03.015, 2012.
Research Councils UK: Evaluation – Practical Guidelines, available at: http://www.rcuk.ac.uk/documents/publications/evaluationguide-pdf/ (last access: 15 June 2017), 2011.
Schuck-Zöller, S., Bowyer, P., Jacob, D., and Brasseur, G.: Inter- und transdisziplinäres Arbeiten im Klimaservice, in: Anpassung an regionale Klimafolgen kommunizieren, edited by: Beese, K., Fekkak, M., Katz, C., Körner, C., and Molitor, H., Oekom, 99–116, München, Germany, 2014.
Schuck-Zöller, S., Brinkmann, C., and Rödder, S.: Integrating Research and Practice – What climate services can learn from other fields, in: Communicating climate change information for decision-making, edited by: Serrao-Neumann, S., Coudrain, A., and Coulter, L., Springer, Heidelberg/New York, in press, 2018.
Spaapen, J., Dijstelbloem, H., and Wamelink, F.: Evaluating research in context, consultative committee of sector councils for research and development (COS), The Hague, Netherlands, 156 pp., 2007.
Stauffacher, M., Flüeler, T., Krütli, P., and Scholz, R. W.: Analytic and dynamic approach to collaboration: A transdisciplinary case study on sustainable landscape development in a Swiss prealpine region, Syst. Pract. Action Res., 21, 409–422, https://doi.org/10.1007/s11213-008-9107-7, 2008.
Vaughan, C. and Dessai, S.: Climate services for society: origins, institutional arrangements, and design elements for an evaluation framework, WIREs Clim. Change, 5, 587–603, https://doi.org/10.1002/wcc.290, 2014.
Wall, T. U., Meadow, A. M., and Horganic, A.: Developing evaluation indicators to improve the process of coproducing usable climate science, Weather, Climate, and Society, 9, 95–107, 2017.
Walter, A., Helgenberger, S., Wiek, A., and Scholz, R.: Measuring societal effects of transdisciplinary research projects: Design and application of an evaluation method, Eval. Program Plann., 30, 325–338, 2007.
Wolf, B., Lindenthal, T., Szerenscitis, M., Holbrook, J. B., and Heß, J.: Evaluating research beyond scientific impact, GAIA, 22/2, 104–114, 2013.
Short summary
Climate services and other fields, that are used to integrate the users in research activities (co-creation), are pledging for existing evaluation methods to be widened up. The authors harmonize the different elements of evaluation in an evaluation
cascade, scaling down from very general evaluation dimensions to tangible assessment methods and suggest how to proceed in developing evaluation criteria and indicators. Two examples demonstrate how co-creation of knowledge could be assessed.
Climate services and other fields, that are used to integrate the users in research activities...