Learning and teaching about seasonal climate forecasts: a Mediterranean educational experience toward operational climate services

During the World Climate Conference-3, Capacity Development has been acknowledged as a transversal component underpinning all the other Pillars of the Global Framework for Climate Services. Within the Mediterranean basin, the interest of climate services based on seasonal climate forecasts is rising because they provide an opportunity for developing a proactive approach towards water management.

In 2014, the Regional Training Center (RTC) in Italy, in agreement with World Meteorological Organization (WMO) and member countries of Region I and VI, identified seasonal climate forecasts as a strategic subject of capacity development for the Mediterranean Region.

Following design-based research methods, this paper presents the evolution of the training approaches adopted, from classroom lessons to a blend of practical and theoretical classroom and distance learning. This evolution, as well as the rising satisfaction of trainees' expectations encouraged WMO and the RTC to widen the spectrum of beneficiaries and to make the resulting course materials available for other regions and RTCs as a course package. The course package provides essential guidelines to facilitate adoption and adaptation of the course by different institutions and instructors, including those in other WMO Regions, based on regional or institutional learning needs and standards, while also serving the needs of individual learners.

1 Introduction

Climate variability and associated risks are affecting water availability. An increasing water scarcity will have global-level impacts on different economic sectors, and particularly on food security. This threat requires research centers, central and local government agencies, businesses, and end-users to find new and innovative ways to interpret, apply and disseminate climate information for decision-making (Bruno Soares and Dessai, 2016). Climate information is a valuable resource for planning and decision-making, but the challenge is to transform information to services tailored to specific users (WMO, 2011). It is a two-fold challenge: on one side acquiring the capacity to transform meteo-climatic information into useful “information products” tailored on users' needs, on the other hand implementing training initiatives and knowledge-sharing tools to allow key users to hone their skills and competences.

In 2009, World Climate Conference-3 acknowledged Capacity Development as a transversal component underpinning all the other pillars of the Global Framework for Climate Services (GFCS; WMO, 2014). The global scale of learning needs for climate services calls for innovative solutions, collaborative projects, a range of flexible modalities to reach learners in a variety of ways, and for sharing resources and successful strategies within the global community. Within the World Meteorological Organization (WMO) Education and Training Programme, the network of Regional Training Centers (RTC) plays a major role in helping member countries develop operational climate services (CS).

The Mediterranean climate is characterized by a large spatio-temporal variability: synoptic to mesoscale spatial variability, and inter-seasonal and multi-decadal to centennial temporal variability. These characteristics raise the demand for CS that apply seasonal forecasting in the Mediterranean region to a wide spectrum of sectors. Seasonal to inter-annual hydro-climatic predictions provide an opportunity for developing a proactive approach to water management (Finnessey et al., 2016; Goddard et al., 2010). Seasonal forecast information is thus relevant in a large variety of decision making processes, but their diffusion as a practical tool is limited by two main factors: (a) a complex learning effort, including fundamental knowledge gaps such as climate modeling, statistical inference, must be conducted before an effective adoption in decision-making chains; (b) the relevant spatio-temporal heterogeneity of seasonal forecast reliability. This key point is further made difficult by the fact that quality of the resulting information should be assessed in term of the probabilistic reliability of ensemble-based forecasts (Weisheimer and Palmer, 2014).

The RTC in Italy, in agreement with WMO (WMO, 2015) and member countries of Region I and VI, in 2014 identified the application of seasonal forecasts as a strategic area for capacity development for the Mediterranean Region. The RTC in Italy is hosted and managed by the Institute of Biometeorology of the Italian National Research Council (IBIMET-CNR) in Florence – Italy, having been designated as RTC by the WMO Executive Council in 1997 (WMO-No. 867; WMO, 1997).

Following design-based research methods, this paper presents the evolution of approaches adopted for training on the application of seasonal forecasts in the Mediterranean region, from classroom lessons to a blend of practical and theoretical classroom and distance learning,. The RTC has applied a case-by-case participatory approach, involving different stakeholders and beneficiaries: National Hydro Meteorological Services (NMHSs), Regional Climate Outlook Forums (such as MedCOF, PRESANORD and SEECOF), Regional Associations and various WMO programmes. The course held in 2016 adopted a blended solution: a short classroom course with workshops, seminars and practical sessions in a 50∕50 ratio of theory and practice, coupled with an online phase, mostly or fully asynchronous (mostly self-directed, with limited or no live sessions).

This evolution of training solutions as well the rising satisfaction of trainees drove WMO and the RTC to widen the spectrum of beneficiaries, making the course available for institutions and also individuals in other regions as a course package. This evolution is consistent with the WMO Global Campus initiative (https://learn.wmo.int, last access: 30 November 2018), and progressive improvement of a training intervention based on evaluation results and newly recognized needs, which forms the structure of this paper, is consistent with the “design-based research” methodology now prominent in the learning sciences (Barab, 2006).

2 Developing a blended-learning course

The training approach adopted by CNR-IBIMET for developing its courses follows a sequence frequently articulated as the ADDIE model (Hess and Greer, 2016), which is consistent with many other linear instructional development models (Braden, 1997). It begins with identification and analysis of the learning needs of the targeted learners and closes with an evaluation of the effectiveness of the training, with design, development, and implementation steps between. This model is also often depicted as a cycle, since continual improvement is one of its goals. The institution ensures a participatory approach involving all stakeholders in the process, including trainees of past courses, the targeted audience of trainees, and experts from other institutions such as NMHSs, Regional Climate Outlook Forums and WMO. The needs analysis for the 2015 International Training Course on Seasonal Forecasts in the Mediterranean was a first testing ground to involve beneficiary institutions in a structured process aimed at a better definition of the course contents. First, a questionnaire was prepared and distributed to the participants of the previous course. We received responses from seven countries around the Mediterranean. Following this, a focus group was organized during the PRESANORD08¹ Forum in Alger from 20t to 24 March 2015, where learning needs were discussed with experts from the National Meteorological Services of North Africa Countries. The process was successful and led to clearly identified learning outcomes of the course, including these high-level outcomes: Learners will be able to (a) understand general aspects of the production and use of seasonal forecasts; (b) list sources of predictability; (c) diagnose the requirements for water management applications of seasonal forecasts; (d) apply seasonal forecasts operationally to water management in different sectors, and (e) make decisions based on their limitations due to the intrinsic seasonal forecast reliability.

Figure 1 Overall evaluation of training courses 2014–2016.

After clarifying objectives and determining that a blended learning solution would be appropriate, the design and development of learning activities and resources was completed with interaction between trainers and Education and Training Office of WMO (WMO ETR). The distance-learning phase aims to provide a common, basic knowledge of the topics of the course and to familiarize participants with tools and software that will be used in the practical part of the workshop. Distance learning was delivered through a Moodle platform, a common course management system used within the WMO community. Moodle was also used for sharing training materials and providing a logistic information for the face-to-face workshop. All the workshops (2014–2015–2016) were structured in morning lessons and hands-on practical sessions in the afternoon, and every workshop focused on specific aspects related to seasonal forecasts in the Mediterranean: Seasonal Forecasts for Agriculture in the Mediterranean (2014), Seasonal Forecasts and Water Management in the Mediterranean Basin: Integrated Approaches (2015), Verification of Operational Seasonal Forecasts in the Mediterranean region (2016). Workshops were delivered at CNR premises in Florence (2014 and 2015) and in Rome (2016). Morning sessions were conducted in IBIMET lecture halls, while hands-on sessions were carried out in computer laboratories, where students and trainers could have high levels of interaction. Coffee breaks and lunches were important components, being valued as opportunities for exchanging experiences and building interactions among the group, which increased the aspects of the course aimed at building a community of practice.

Last component of the training process aims to evaluate the effectiveness of training by evaluating the training processes and assessing learning.. An evaluation of training process, according to the first level “reaction” of the Kirkpatrick's training evaluation model (Kirkpatrick, 1994), was carried out through two different questionnaires handled with the Moodle platform, one for the distance learning and one for the front workshop. A comparative analysis of training evaluations 2014/2015/2016 shows an improvement of the perception participants had of their training experience from the first to the last course (Fig. 1).

Moreover, for the course of 2016 an assessment of the acquaintance of specific technical aspects (questionnaire on programming skills with R Language) was carried out, addressing the second level of the Kirkpatrick's model “Learning”. The results showed that 13 % of participants acquired advanced knowledge through distance learning lessons, 73 % basic knowledge and 13 % scarce knowledge.

3 Developing a course package

After the completion of the third course, a new WMO-IBIMET joint initiative was initiated, in collaboration with MeteoSwiss, to develop an online course package, Training and Operational Principles (TOP) for Seasonal Forecasts, based on the training materials produced during the face-to-face courses (https://training.climateservices.it/rtc-ibimet/t-o-p-seasonal-forecast/, last access: 30 November 2018).

The experience acquired in the previous training initiatives, the positive feedback of the participants on the distance learning courses, and the great interest on the subject of seasonal forecasts were the driving factor to set up a permanent and accessible web resource. In fact, the evaluation survey results highlighted that the majority of the trainees expressed a high level of satisfaction on the distance-learning course, as it is “a useful tool to improve their knowledge and competences”. In addition, the blending of a variety of content format and media can turn “passive content” into an engaging learning experience.

The resulting course package is a set of online resources providing theoretical and practical knowledge on seasonal forecasts models and predictability of Mediterranean climate variables, climate and data analysis, forecast verification, and specific application of seasonal forecasts for agriculture and water management. The training materials, created by the well-known researchers and scholars who contributed to the previous courses, have been reorganized in a new conceptual structure and “developed in a consistent and reliable fashion” (Molenda et al., 2003) following sound instructional design principles. Expected outcomes defined for the package are that after completing the course, learners should be able to:

• identify and manipulate seasonal forecast products and related information suitable for their specific interests;

• set up a verification analysis both for probabilistic and deterministic forecasts;

• identify proper strategies to transform model outputs into climate services for specific applications.

gain according to the ADDIE model (Hess and Greer, 2016), the first phase, Analysis, began with a task and topic analysis of the learning needs. This analysis contributed to the course plan, which is the output of the design phase. This existing Moodle sites used in the previous courses offered a precise overview of the materials produced by the trainers for all three courses, facilitating the production of a new course plan that described the combined learning objectives hierarchy, taking into account the prerequisite skills for competency development. The detailed content outline was a further step in the course design, providing all the content information, broken down into modules, lessons, activities and assignments, based on an analysis of the overall content needs.

Another goal was to follow a collaborative process in the design phase. Therefore, the course plan was created using structured Google Drive documents, where each module listed learning objectives and links to resources such as lessons, case studies, seminars, exercises, readings, and a structured bibliography. This free web tool afforded a collaborative environment in which team members could each provide their input based on their respective expertise.

The final versions of the course resources were organized in a Google Site (https://sites.google.com/view/top-seasonal-forecast/home, last access: 30 November 2018) for a final review, allowing team members to discuss and propose revisions or integrations.

After the final revision process, the course package materials and resources were transferred into a dedicated website (https://training.climateservices.it/rtc-ibimet/t-o-p-seasonal-forecast/, last access: 30 November 2018) for external use. Any user can easily access the course package and download the training resources, published under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License to ensure open access, yet preserve the intellectual property rights of the different authors, unless specified otherwise.

The course package, designed for forecasters and climatologists with knowledge of general meteorology as well as physical and dynamic meteorology, can be used either by education and training institutions, as the basis for group courses, or individually by NHMS staff members who wish to improve their climate services competencies or to specialize in seasonal forecasting. According to the WMO Competency Framework for the Provision of Climate Services, approved by the WMO Executive Council of June 2016 (WMO-No. 49), the course package addresses Competency 3: create and/or interpret climate forecasts, climate projections and model output.

The course package is intended to be used by other institutions around the world in the creation of new courses adapted to specific local training needs. To allow widespread use of the training content, the IBIMET-CNR training team developed an eBook with essential guidelines to facilitate the course adoption or adaptation by different institutions. To address the heterogeneous knowledge levels of potential learner audiences, each course unit describes prerequisites and proposes external resources to fill knowledge gaps. This allows individual learners to create a personalized learning path, taking into account different levels and needs, assuming that this is more effective than “one size fits all”. Future enhancements to this or new course packages could include options for tracking progress and opportunities for group or individual self-assessment.

In early 2018, the course package inspired, and part of it was used by, the RTC Peru and the CLIMANDES Project for the creation of a module of the course “Aplicaciones prácticas de la información climática para una agricultura resiliente y sustentable” for the North Western South America Andes region. Many resources from the RTC IBIMET course package were used as presented, others were adapted for the unique needs of this very different climatic regime, and new course content was developed as well. This reuse demonstrates the appeal of well-conceived course packages to address global learning needs in delivering operational climate services. Additional regional centers have examined the course package for reuse. In the summer of 2018, the course package was made available through the WMOLearn section of the WMO E-Library (https://library.wmo.int, last access: 30 November 2018), which will host a growing number of resources to address the various WMO competency frameworks, including the critical set of competencies for provision of climate services. WMOLearn is a set of resources and tools being made available to all WMO Members through the WMO Global Campus initiative.

As highlighted by Google Analytics Reports (period January–July 2018), the WMO E-Library has been an important channel of dissemination of the TOP Course Package, as well as the IBIMET RTC Moodle platform particularly in the second part of the period analyzed. Figure 2 shows the referrals to the TOP Course Package from other web pages.

At July 2018, the subscribed users with the role of student are 27 and the number of downloaded presentations is 64.

4 Conclusions

Capacity development is acknowledged as a critical component of the GFCS. The experience of the WMO ETR and the RTC confirms that capacity development is an enabler to trigger a wider application of CS in public organizations. The use of seasonal climate forecasts in the Mediterranean is developing, but has not yet reached operational and wide-scale application. The reasons relate to both the early developmental stage of forecast skill, but also to limited knowledge and skills of climate services staff members for their operational implementation. RTC-Italy intended to bridge explicitly this double gap by, on one side, strengthening the skill of personnel for the application of seasonal forecasts, but also building capacity for the evaluation of forecasts' skill (a specific module addresses this issue).

Figure 2 Google Analytics Report September 2017–July 2018 of referrals to the TOP Course package.

The course package was intended to be used primarily by other institutions around the world in the creation of new courses adapted to specific local training needs. Feedbacks are encouraging, during the first six months, the TOP had 27 registered students, 64 presentations have been downloaded and the package was partially reused by the RTC in Peru.

To strengthen the latter impacts, the Theory and Operational Principles Course Package established essential guidelines to facilitate the adoption of this and future course packages on other topics by widest number of institutions and instructors: to provide additional content to better meet students' needs, and to modify content to fulfil their regional and national or institutional education standards.

The full range of TOP impacts have not yet been assessed, giving room for further research both on the development of learning approaches and the adaptation to different socio-economic environments.