Here we present a preliminary (beta) version of the empirical seasonal forecasting system. The purpose of this beta version is twofold: first, establish a reference version based on standard predictors making use of known sources of predictability and, second, compare its skill over the Mediterranean with the state-of-the-art dynamical systems. Results coming from ongoing work within the project providing new specific predictors for the Mediterranean region can be easily added to this preliminary version. The skill of the new system will be evaluated with respect to this reference version. Eden et al. (2015) developed a global empirical seasonal forecasting system based on Multi Linear Regression (MLR), using a few global climate indices as predictors, and producing a probabilistic output using the residuals from regression. This system shows ability to produce skilful forecasts over several world regions, despite the reduced number of predictors used. Wang et al. (2017) showed, using MLR too, that a careful selection of predictors can produce skilful prediction of winter NAO.

The beta version of the empirical seasonal forecasting system here described follows the same procedure based on MLR suggested by these two studies as this kind of models only requires very modest computing resources and has the additional advantage of being easy to modify. The second version of the empirical seasonal forecasting system, incorporating results from MEDSCOPE project findings, will be developed and evaluated in the second part of the project.

Given the extension of the Mediterranean domain, its great complexity (both orographic and land-ocean distribution), and location, sub regions within the domain are affected by different factors at different times of the year. In order to improve the skill of the system, the empirical model will use different sets of predictors for every climatologically homogeneous defined sub region within the Mediterranean domain and every season. Selection of predictors, from the pool listed in Table 1, is based on their high correlation with precipitation over large areas within the sub region and for each particular season. One issue with this type of models where predictors may change spatially is the extreme noisiness of the forecasts showing very different results for neighbour grid points. For example, probability values for lower tercile over a grid point in southwestern Turkey are calculated for 2018 FMA using 2 different sets of predictors (Table S1 in the Supplement, row FMA, columns Turkey and East Mediterranean), obtaining 61 % and 37 %. As a compromise between selecting the best predictors for every point and forecasting smooth synoptic scale anomaly patterns, the domain is divided in sub regions (see Table 3), and a subset of predictors from the pool (see Table 1) will be selected for each one of them as a whole. To avoid abrupt transitions in space and time, predictors will be restricted to partially match among neighbour sub regions and consecutive seasons (for example, January–March and February–April). To further smooth out transitions among sub regions, they have been defined with a high amount of overlap. Those grid points belonging to more than one sub region will be assigned a weighted average from values from different sub regions, based on distance to respective borders.

In short, sub regions have been defined seeking a compromise between encompassing main land areas and countries (adding an overlap area among them) and climatological homogeneity. In order to meet this last feature, empirical orthogonal functions (EOFs) for yearly precipitation have been calculated and their patterns taken into account for defining sub regions. Figure 1 depicts these three EOFs with chosen sub regions superimposed. Table S1 provides the exact definition of 5 sub-regions employed.

As we intend to deliver synoptic scale anomaly patterns, low-resolution predictands will be selected for this beta version of the system. Precipitation data from the Global Prediction Climate Centre (GPCC) dataset (Schneider et al., 2017) will be used (2.5∘ version). Data consist of a blend of GPCC v7 (until 2013) and its monitoring (v5) from 2014 onwards. Surface temperature is obtained from the ERA-interim reanalysis (Dee et al., 2011). The period selected is 1979–2016. In both cases, predictands will consist of the three months average for every season and grid point. The empirical model runs every month with one-month lead time, i.e., computing a forecast for the following season (3 months) and for both predictands. For example, in January, the forecast will be calculated for February–March–April.

This beta version of the empirical model makes use of global climate indices provided by external sources. The initial pool of predictors includes 25 monthly time series of indices associated to atmospheric and oceanic climate variability indices, ocean heat content and snow cover (see Table 1). Additionally, for each index, a new monthly time series is generated calculating the incremental value from the previous month (e.g., February incremental value would be February minus January). The purpose of such incremental series is try to find additional sources of predictability by analysing if a rapid change in the state of a certain indicator could be linked to anomalous atmospheric circulation. Before exploring these 50 indices (25 climate indices plus their 25 incremental series), moving averages from 2 to 6 months are applied, to better capture mechanisms from different time scales.

The selection process of predictors for each sub region starts by the computation for each season of the correlation coefficient between predictand and predictors from the pool applying different lead times (from 1 to 12 months). Correlation is also computed for six different options of moving average (up to 6 months). So, for a particular grid point, predictor and season, a collection of correlation coefficient values will be obtained. A predictor is finally selected considering the percentage of grid points with significant (for p-value ≤0.10) correlation. As an example, Table 2 shows percentage of points with significant correlation for the predictor DMI for different lead times and moving averages.

Bearing in mind the known relative low predictability, and consequently skill, for the studied area, this procedure will try to unveil the best possible signal that a predictor from the pool can offer playing with different lead times and moving averages. Considering correlation data from the different predictors in Table 1, and applying constrains for the sake of continuity among regions, a set of predictors is finally selected for every region (see Table S1).

The model makes use of multiple linear regression (MLR), as described in Wilks (2006), over every grid point, using the set of predictors selected for each specific sub region. Trend is removed before calculating regression and then added to regression results. In order to express the forecast in probabilistic terms, first, terciles are calculated for predictands at every grid point. Then, probabilities are assigned to every tercile, using a normal distribution, centred in the deterministic output of the MLR and using information from residuals to adjust its width. This distribution represents the expected probability density function (pdf) for the forecasted predictand value. The computation of the area below this curve and between observed terciles provides the probability of the forecasted value to be in every one of them (Eden et al., 2015).

This procedure requires predictands to adjust to a normal distribution. As this is not the case with precipitation, square root is previously applied over this predictand, to transform it into a normal-like distribution (Pasqui et al., 2007). Additionally, the system performs a series of checks to ensure that certain required assumptions are met, such as: no collinearity among predictors, no overfitting, residuals are normally distributed and homoscedastic and do not show autocorrelation.

The performance of the empirical model is assessed by verifying against observational grids for a hindcast calculated for the period 1983–2014. Additionally, the verification scores computed for the empirical system are compared with those of some state-of-art seasonal forecasting systems based on dynamical models for a common hindcast period to check the possible benefit of the empirical method here described. Regression is trained for same period, using “Leave-One-Out” technique (Wilks, 2006), excluding a total of 5 years from the series (two before and two after the year we are forecasting) to avoid possible autocorrelation (Breusch, 1978 and Godfrey, 1978). Section 3.3 describes several verification indices calculated for this version of the empirical system, and their comparison with the corresponding indices calculated from dynamical models.

Figure 2 shows a few examples of forecasts maps for precipitation. Although some noisy features can still be seen over certain areas, mainly Egypt and Arabian Peninsula, observed patterns are synoptic scale and continuous, generally speaking. Borders between sub regions are not evident, either, so forecast maps are reasonably shaped, and defined sub regions and proposed constraints for predictors seem to work well (all this set up were imposed to produce smooth synoptic scale anomaly patterns) This particular example corresponds for the 1-month lead time 2014 DJF and JAS forecast, and anomaly patterns resemble in terms of structures and spatial variability those shown by dynamical models. In order to compare results, the empirical model is also run using temperature as predictand and using the same predictors as for precipitation, under the assumption that same anomalous circulation captured by these predictors can affect to temperature as well. Results can be seen in Fig. 3. The same conclusions about the forecasts appearance also applies for temperature.

Verification scores are computed and visualized following (Sánchez-García et al., 2018). For every predictand (temperature and precipitation), season, score and verifying sub region, results from a selection of seasonal forecasting systems based on dynamical models and the empirical system are put together in a table for easy comparison.

The dynamical models and versions used for this comparison were operational when this project started (2017) and are the following: ECMWF system 4, Météo-France system 5, Met-Office system 9 (GloSea5), National Center for Enviromental Prediction (NCEP) system version 2, Canadian Seasonal to Inter-annual Prediction System (CanSIPS) and Japanese Seasonal Forecasting System 2. Both probabilistic and deterministic scores have been computed for dynamical models and the empirical system: Ranked Probability Skill Score (RPSS), Relative Operating Characteristic (ROC) area and Brier Skill Score (BSS), for upper and lower tercile probabilities and anomaly correlation. Statistical significance of all computed scores has been quantified by the p-value estimated using a bootstrapping non-parametric method (see details in Wilks, 2006).

To compare the skill of this first version of the empirical model against this selection of dynamical models is necessary to choose a common period of hindcast data, to ensure that differences in scores are not attributable to changes in predictability over the years. So, although hindcast for the empirical system is available for a longer period, skill is evaluated for the common period 1997–2009. Table 4 shows some examples for the scores anomaly correlation, RPSS and ROC area (lower and upper tercile) computed for precipitation over France and for temperature over the Eastern Mediterranean region. For these particular areas, the empirical system performs especially well. Generally speaking, precipitation scores for the empirical system are better than for dynamical models. Temperature scores are roughly the same level for dynamical models and empirical system. Additionally to tables for specific sub regions, Fig. 4 depicts some examples of anomaly correlation maps for precipitation calculated for 1983–2014 and for autumn (SON) and spring (MAM) seasons. Certain areas like northern France, Benelux, parts of Germany or Morocco, large areas of Eastern Mediterranean and west of Black Sea show good correlation with observations for these particular two examples. However, other places, e.g., most of Iberian Peninsula and Italy, most of the Eastern part of the domain for SON and large parts of Northern Africa for MAM, hardly show any. The relatively high skill reached over some regions show the potential of the empirical system. As there is still room for improvement, additional efforts should focus on the identification of good predictors for large areas still showing low skill.