Quality assessment of heterogeneous surface radiation network data

Abstract. The DWD national radiation measurement network comprises 82 automatic sites, 29 manned sites with shaded and unshaded pyranometer and the BSRN station at Lindenberg. The quality assessment routinely applied takes into account the basic astronomical and empirical considerations as well as some interdependencies like total to diffuse flux relation and cross checking with sunshine duration. A more advanced quality assessment approach attempts to routinely utilise timeseries of clear sky radiative transfer simulations for every site. For that purpose a link to cloud coverage obtained from Meteosat second generation geostationary satellite data, highly resolved in time and space, was established. There is a predefined calibration cycle of 30 month for automatic stations. Data analysis on this timescale allows for the detection of sensor degradation, wrong calibration or configuration and other possible local disturbances. Furthermore using satellite cloud mask enables the identification of larger clear sky regions characterized by similar atmospheric conditions. Thus, in a regionalization step correction or recalibration of moderate quality data to a higher level can be considered. The paper provides an overview of DWD surface radiation network and the current activities to improve automatic quality assessment using remotely sensed data and clear sky modeling for the upgrading of radiation data.


Introduction
Irradiance is a key parameter in Earth's weather and climate system.Accurate observations of the components of the near surface radiation budget are therefore essential IPCC (2007).For the achievement of a global coverage the utilisation of satellite-based retrievals have become an important part.Such retrievals are developed and data are provided in the framework of EUMETSAT's Satellite Application Facilities like SAF on Climate Monitoring (Mueller et al., 2009) and LandSAF (Trigo et al., 2011).Even though not needed as a direct input but indispensable for testing, validation and adaptation of satellite-based algorithms are accurate ground-based observations.High quality ground-based measurements are performed in the context of the baseline surface radiation network (BSRN, Ohmura et al., 1998).The Deutscher Wetterdienst (DWD) is operating the national ra-diation network which is a heterogeneous collection of sites measuring downward shortwave radiation.The network is hosted by Meteorological Observatory Lindenberg.
This paper deals with the progress made in quality assessment of the radiation network data due to the following activities: -by using satellite data input of cloud coverage -by comparing measured quantities to simulated ones in clear-sky conditions

The ground-based network
The basic idea of the radiation network is to achieve a good coverage of whole Germany by means of radiation measurements, i.e. to avoid gaps.It comprises manned as well as fully automatic stations.Manned stations (in 2011: 29) are Published by Copernicus Publications.equipped with two Kipp & Zonen CM11 or CM21 pyranometer (Kipp & Zonen, 2004) to measure diffuse (shaded) and total (unshaded) downward shortwave radiation (0.3-2.8 µm).Instrumentation is categorised as secondary standard according to ISO 9060 or high quality with respect to WMO classification.CM 21 response time is 5 s for 95 % of responses, less than 15 s for CM 11, respectively.11 sites are equipped additionally with Kipp & Zonen CG4 pyrgeometer to cover the spectral range from 4.5 to 45 µm (Kipp & Zonen, 2001).SCAPP (scanning pyrheliometer and pyranometer) is a low-cost instrument to measure diffuse and direct irradiances developed by DWD and manufactured by Siggelkow Germany (Fig. 1).It was designed for automatic mode and operates without outer sun shading and outer moving parts.The spectral coverage is limited to 0.3 to 1.1 µm due to the use of a silicon detector.For daily sums of downward shortwave radiation an accuracy of 10 % is expected (20 % for hourly sums, Bergholter and Dehne, 1994).Figure 2 provides an overview of the DWD ground-based radiation measurement network.
The calibration of the field/network pyranometers is performed at the RRC/NRC at Meterological Observatory Lindenberg (MOL) indoor referring to ISO 9847:1992 "Solar energy -Calibration of field pyranometers by comparison to a reference pyranometer".SCAPPs are calibrated at MOL locally too, but outdoor only between the Northern Hemisphere spring and autumn equinoxes using direct and diffuse irradiance measured by the station pyrheliometer and shaded pyranometer.All calibrations are tracable to World Radiation Reference (WRR).
Indoor calibration capability -to get independent from the season -is under development.A more detailed analysis of SCAPP functionality is given in Behrens and Grewe (2005).

Scheme for automatic quality assessment
The core conventional data analysis currently operated refers to the proposal given by Long and Dutton (2002) with regard to BSRN (Ohmura et al., 1998).This will be complemented by a procedure checking overall data status (sensor degra- dation, systematic disturbances like restrictions of full sky view) described in the following sections.It makes use of an independant approach for detection of clear-sky episodes at measurement site operating daily (Sect.3.1).For the sites labelled as clear simulation of expected downward shortwave radiation will be performed using a simplified radiative transfer model setup (Sect.3.2).First results using BIAScorrected reference values are given in Sect. 4.

Utilisation of satellite data
Current geostationary satellite imagery is provided with temporal resolution of 15 min and a footprint size of 3 km at nadir view (Schmetz et al., 2002).EUMETSAT's Satellite Application Facility on Nowcasting and Very Short Range Forecasting is developing algorithms to routinely process data of MSG-SEVIRI.Retrieval of cloud mask and cloud type is based on pixel-by-pixel analysis on original satellite projection by means of multispectral thresholding method (Derrien and LeGleau, 2005) where N ave is the averaged total cloud coverage at location and DatAvail is the ratio of valid MSG cloud cover estimates in the predefined time window.This is to ensure that averaged cloud cover represents the whole time frame and is not biased due to greater lack of data.Figure 3 displays examplary output of total cloud cover calculation.Referring to satellite data to characterise the cloud coverage at site instead of taking the radiation measurements itself yields some advantages: -conventional methods like evaluation of the ratio of diffuse and total irradiation might fail if at least one parameter is disturbed, potential detection of such misbehaviour could be prevented -a homogeneous method is applied for all ground station independant of its instrumentation -evaluation of satellite pixels enables a full sky view, i.e. all sky segments as seen by upward-looking instrument are checked for cloud contamination Some potential drawbacks need to be mentioned here, too.
The footprint (sampling distance) of SEVIRI radiometer is 3 km at nadir view and resolution gets coarser towards north and south.It might effect in an underestimation of small cumulus clouds (Cumulus humilis).To exclude such cases from clear-sky evaluation the variability of the ground-based signal is checked additionally.Secondly, very thin cirrus from time to time remains undetected, but no strong affection is expected here.

Simulation vs. measurement
Clouds strongly affect the radiation field and therefore influence the measurements of shortwave total and diffuse irradiance.Simulations of arbitrary meteorological scenes requires detailed knowledge of microphysical cloud properties that can hardly be provided in appropriate quality for the whole region of interest.Best matches between calculated and measured quantities can be expected for clear-sky conditions when direct component of solar radiation dominates the total (global) radiation.The focus was set on the insolation time frame ranging from 10:00 to 13:00 UTC.Calculation of incoming shortwave radiation is done using radiation transfer code Streamer (Key, 2001).The setup for the model only roughly distinguishes between winter and summer midlatitude atmosphere.It runs with climatological ozone and background tropospheric aerosol.
For the comparison study data of pyranometer sites of the period May 2010 to August 2011 were selected, resulting in a total number of 560 cases.Time series of calculated total radiation in general shows a good correspondence to measured

Simulation vs. measurement
Clouds strongly affect the radiation field and therefore influence the measurements of shortwave total and diffuse irradiance.Simulations of arbitrary meteorological scenes requires detailed knowledge of microphysical cloud properties that can hardly be provided in appropriate quality for the whole region of interest.Best matches between calculated and measured quantities can be expected for clear-sky conditions when direct component of solar radiation dominates the total (global) radiation.The focus was set on the insolation time frame ranging from 10 to 13 UTC.Calculation of incoming shortwave radiation is done using radiation transfer code Streamer ([5]).The setup for the model only roughly distinguishes between winter and summer midlatitude atmosphere.It runs with climatological ozone and background tropospheric aerosol.For the comparison study data of pyranometer sites of the period May 2010 to August 2011 were selected, resulting in a total number of 560 cases.Time series of calculated total radiation in general shows a good correspondence to measured quantities (figure 4).Periods of enduring bad  quantities (Fig. 4).Periods of enduring bad weather conditions cause a further reduction of evaluable data.The distribution of differences simulated vs. measured is displayed in the left of Fig. 5.The model tends to give higher estimates even though some negative deviations are present, too.Same is indicated by the relative bias of 1.02.
Figure 6 provides a more station specific view on the relation of calculations and measurements for pyranometer sites.Almost all stations show an overestimation by the model, except two mountaineous sites located on top of low mountain summits (Fichtelberg 1214 m a.s.l., Hohenpeissenberg 988 m).In other words, lowland stations are receiving less radiation than calculated, however mountaineous get more.It is explained by the fact that calculations were performed assuming a fixed height above sea level (120 m a.s.l.).This finding means that the model setup will need further adjustment for actual station elevation.
A summary of the results is provided in Table 1.The model generally slightly overestimates, but filtering for instrument category draws a different picture for pyranometer and SCAPP: the systematic deviation is almost doubled for the latter.This is possibly due to less long-term stability of the sensors, but needs to be further investigated with regard to calibration procedure too.However, the higher variability  meets our expectations and is inline with instrument specifications (Bergholter and Dehne, 1994).

First applications
Monitoring the differences in the timeseries of calculated and measured shortwave radiances reveales possible errors in station configuration, wrong calibration or sensor degradation.
For SCAPP a calibration cycle of 30 month was predefined according to manufactorers advices.Several years of operational service have shown that in practice the cycle must be set shorter.The comparison model-measurement helps to identify the right time for instrument substitution (see Fig. 7).
Instrument status is a valuable information for operators and basic level quality assessment, too. Figure 8 displayes an example for a graphical realisation of instrument status for site Magdeburg (Fig. 8).

Conclusions and outlook
The study presented here investigates relations between measured downward shortwave radiation and calculated values in clear sky conditions to improve automatic data quality checks.Based on a dataset of real clear-sky conditions identyfied using Meteosat/SEVIRI cloud coverage distributed over more than one year dedicated relations to simulations according to instrument group (pyranometer, SCAPP) could be found.These slight but systematic differences can be considered and used to assess the quality of realtime data.Currently a simple setup for the model is used.It can be expected that feeding of more realistic atmospheric profiles achieved by providing radiosoundings, filter radiometry (if available) or forecasted/analysed profiles of numerical weather prediction model will lead to more refined and precised results.The future consideration of the site elevation will surely cause a positive bias at the elevated stations (Fichtelberg, Hohenpeissenberg), too.As a consequence the spread would decrease.

Adv
The approach presented here will be checked for applicability for overcast sky conditions.

Figure 2 .
Figure 2. Ground based network for irradiance measurements, location and characterisation of sites.Green circles indicate shortwave total (G) and diffuse (D) component measured by pyranometers, pink indicates the same plus atmospheric longwave radiation (A).Blue: sites equipped with SCAPP.

Figure 3 :
Figure 3: Total cloud coverage from MSG based on pixelby-pixel analysis using NWCSAF algorithm, mapped onto COSMO-DE model domain and calculated for a 9x9 pixel matrix Figur down at th data Augu

Figure 3 .Figure 4 .
Figure 3.Total cloud coverage from MSG based on pixel-bypixel analysis using NWCSAF algorithm, mapped onto COSMO-DE model domain and calculated for a 9 × 9 pixel matrix.

Figure 5 .
Figure 5. Systematic deviations of calculated and measured irradiances averaged from 10:00-13:00 UTC for pyranometer sites.A distinct peak can be found at positive differences (measured subtracted from modeled).

Figure 6 .
Figure 6.Systematic discrepancy and standard deviation of model calculations of downward total shortwave radiation from measured quantities, given for all pyranometer sites, expressed in percent.

Figure 7 .
Figure 7. Shortwave radiation (total) measured at site Magdeburg/ Saxony-Anhalt with SCAPP (red) and calculated (green).Note that identification of strong overestimation in July possibly due to wrong station setup caused a release of instrument substitution end of August.
. Static and dynamic thresholds are precomputed from geographical, climatological and forecasted data.
MSG cloud mask and cloud type in a pixel window centered at ground site are used to identify insolation episodes at Adv. Sci.Res., 8, 93-97, 2012 www.adv-sci-res.net/8/93/2012/noon.Then following simple extraction criteria are applied:

Table 1 .
Statistics of comparison model-measurement, instrument classes treated separately.The model overestimates for both categories, but difference is larger for SCAPP.