Selected results of a model validation exercise

The concentration fields calculated with three Gaussian models and one Lagrangian dispersion model are validated against a set of SF6 concentration data provided by the German environmental programme BWPLUS. The source was a pig fattening unit in fairly flat terrain. The results reveal that, in flat terrain with steady undisturbed flow, the use of Gauss models is still justified, whereas Lagrangian models should be used whenever the flow is modified by obstacles or topography.


Introduction
Atmospheric dispersion models play an important role in environmental assessment studies to investigate and to quantify the effects of human activity on air quality.Thus, the model users and decision makers require validated models to increase confidence in the model results (Schatzmann and Britter, 2005).Model developers ideally provide their users with results of validation studies e. g. on homepages.Confidence in model performance is however more increased via independent model validation exercises.The department of Environmental Meteorology at ZAMG, over the years, has taken part in several such studies (Pechinger and Petz, 1995;Hirtl and Baumann-Stanzer, 2007;Hirtl et al., 2007;Baumann-Stanzer et al., 2008).Recently, validation experiments have been conducted by the COST-Action 732 (http://www.mi.uni-hamburg.de/Home.484.0.html) and by the Austrian Working Group on "Dispersion Modelling in the Near Field", organized by the Austrian Ministry of Economic Affairs.
Here we report on a model comparison with a data set from the German environmental programme BWPLUS within the project "Odour emission and spread" (Baechlin et al., 2002).With a pig fattening unit in fairly flat terrain as odour source, validation data sets for 15 experiments consisting of SF 6 concentration data (both at the stack and in the near field), meteorological observations as well as odour intensity data estimated by a panel were created.The experimental set-up is shown in Fig. 1.All experiments took place under neutral conditions, wind speeds varying between 2, 5 and 7 m s −1 .
Correspondence to: M. Piringer (martin.piringer@zamg.ac.at)This data set is used to validate the dispersion models routinely used at ZAMG.

Material and methods
The three Gaussian models ONGAUSSplus (the Austrian regulatory Gaussian model), AODM (the Austrian Odour Dispersion Model, Schauberger et al., 2002 and2000), the Gaussian model ADMS 3.1 (CERC, 2001) with advanced boundary layer physics (in contrast to the former two models), and the Lagrangian model LASAT 3.0 (Janicke, 2007) are validated against the BWPLUS data set.AODM is in fact the Austrian regulatory model extended with a peak-tomean approach for the calculation of odour concentrations; short-term peak concentrations downwind of the source depend on wind velocity and stability of the atmosphere and are calculated for 5 seconds, the average duration of a single human breath (Piringer et al., 2007).For LASAT, two different model runs are performed, one without and one with considering the small terrain inhomogeneities in the vicinity of the pig fattening unit.
The comparison with observations is done unpaired in space as quantile-quantile and residual plots.This is appropriate especially for Gaussian models because plume propagation in these models is along a straight line for the time interval in which the concentration field is calculated.In flows disturbed by topography or buildings, the exact location of the concentration maximum can therefore not be reproduced with these models; nevertheless, due to their widespread use, it is of interest whether they are able to give a reliable estimate of the magnitude of the maximum concentration.In addition, statistical performance measures as geometric mean value (MG) and geometric variance (VG), fractional bias (FB) and normalised mean-square error (NMSE) are Published by Copernicus Publications.calculated.Values for these measures are interpreted in light of acceptability criteria given by Chang and Hanna (2004).

Results
The results of the comparison of SF 6 concentration data and modelled concentrations for all 15 experiments are shown in Fig. 2 in the form of quantile-quantile plots, i.e. unpaired in space.Figure 2 clearly shows differences in model performance.The best results are obtained with LASAT, based on a diagnostic wind field model.Run LASAT-Terrain is slightly superior over the run without considering the terrain features.
Of the Gaussian runs, AODM performs best, followed by the regulatory Gauss model ONGAUSSplus which however under-estimates large concentrations.ADMS systematically and most severely under-estimates in this exercise.The use of a peak-to-mean ratio as for AODM would improve the result, but a considerable under-estimation would remain.Model runs undertaken by other members of the aforementioned Austrian Working Group on "Dispersion Modelling in the Near Field" confirm the under-estimation of ADMS in this particular case.The somewhat surprising large differences in the results within the Gauss model family can probably be explained by the fact that the models use different approaches concerning the estimation of effective stack height and plume propagation and shape.In this exercise, no attempt has been undertaken to "tune" the models with respect to input parameters and settings to improve their performance; instead, the same input was used for all the model runs.In Fig. 3, residual plots as suggested by Hanna et al. (2003) are used to compare the data-sets.The lower and upper ends of the vertical bars represent the 2nd and 98th percentile of the residuals (model value to observation).The lower and upper limits of the boxes indicate the 16th and 84th percentiles of the residuals, the horizontal line in between the median.At least 50% of the model data should be within a factor of 2 to the observations (within the dotted lines) according to Chang and Hanna (2004).
AODM achieves on average a closer agreement between model and observations (median close to 1) than the regulatory Gauss model ONGAUSSplus (abbreviated with ONG) but a comparable range of residuals.ADMS significantly underestimates the observations (more than 98 percent of the model values smaller than the observations (residuals less than 1).The two LASAT runs, one without and the other with terrain data, are similar to AODM, with the median near 1 and by far more than 50 percent of the model values within a factor of two to the observations (most of the box within the dotted lines).The smallest range of residuals of all the runs is obtained with LASAT without considering the small terrain inhomogeneities.Whereas the quantile-quantile plots in Fig. 2 show LASAT-Terrain to be slightly superior to the run without terrain, the residual plot of Fig. 3 shows the opposite; validation results apparently depend also on the measure applied.Introducing terrain apparently slightly increases the range of concentration differences.
Statistical performance measures -together with their range of acceptability according to Chang and Hanna (2004) -are given in Table 1.The best values are highlighted in bold.For all runs except ADMS, MG, VG and NMSE are within the proposed range of acceptability.For ADMS, the statistical measures are somewhat higher than the suggested limits in this case.FB is acceptable for AODM and the two LASAT runs only.LASAT, according to

Conclusions
As one result of the model comparison with the BWPLUS data set, relatively simple Gauss models are able to give reliable estimates of maximum concentrations, depending on the model configuration; they can be used with some confidence as "first guess"-or "screening"-models.The Lagrangian model LASAT, designed for flow and concentration fields disturbed by obstacles or topography, is however superior even in this simple BWPLUS case.Nevertheless, due to the wide application of Gauss models, it is of some interest that also these kinds of models can perform well under selected conditions.If however a more precise location of the maximum in the concentration field is desired and if the straight propagation of the plume is deflected by an obstacle, a careful selection of an appropriate model is necessary.In flat terrain with steady undisturbed flow the use of Gauss models is still justified, whereas Lagrangian models should be used whenever the flow is modified by obstacles or topography.Both model types can be used to calculate yearly averages as well as percentiles of concentrations based on longer meteorological time series.Due to the widespread use of such models, the proof of their reliability is of utmost necessity.This is best achieved via independent validation data sets, more of which are further needed to increase the trust in the performance of these models.

Figure 1 :Figure 1 .
Figure 1: Set-up of the BWPLUS experiment showing the locations of the source, the anemometer, and the receptors.Receptors are along two traverses perpendicular to the prevailing wind direction.

Table 1 ,
performs best (3 best values without, 2 best values with terrain), AODM and LASAT-Terrain perform best according to MG.