Any improvements in a process producing
forecast output like NWP, statistical adaptations or, perhaps most importantly,
the final forecasts to the users, can be discovered and measured only by
reliable and comprehensive verification procedures. Finnish Meteorological
Institute’s (FMI) operational weather service has boasted a well-established
verification system both at its central and regional levels for quite some
time. An operational real-time verification software package was developed and
implemented in the early-mid 1990s (Nurmi & Rissanen, 1993; Nurmi, 1995 a,
b; 1996). Additionally, there exist
verifiable datasets of (final) forecast output since the late 1970s which
enable, e.g. trend analysis. The present operational verification package,
however, covers only a limited number of verification locations. Moreover, the
overall FMI forecast production process and the relevant tools have been
renewed since the introduction of the verification system. This has put a
pressure on a thorough revision of the verification framework. The new system
under development will maintain as a backbone all the good features and
functions of the replaceable scheme. These include close to real-time
monitoring of the very latest forecast output, option for personal verification
statistics, an exhaustive set of quality measures and statistics, coverage of
all guidance as well as forecasters’ final output, inclusion of specific target
scores used for official monitoring of forecast performance, printable
extensive periodical verification reports, and a human-friendly user interface.
The underlying novelty in the forecast production environment - and the associated verification process - is a transition from points to fields. A forecasters’ grid editor tool was introduced to the forecasting community a couple of years ago and has recently reached the operational status. The grid editor provides as a first guidance to the forecasters various meteorological fields originating from different sources. These include, e.g. mesoscale numerical analyses, precipitation analyses and very short-range forecasts based on radar composite output, and NWP charts from ECMWF as well as from the national HIRLAM model. The editor is equipped with a nifty and versatile set of editing tools to edit the guidance fields in space and time. All operational forecast generation is integrated into the grid editor and, consequently, all final forecasts to the customers and to the public originate from this environment, automatically. Eventually, all relevant forecast information to serve the verification purposes will also originate from these data. As opposed to the previous (i.e. present) verification system with statistics available only for a limited number of locations, the new framework can produce similar results for hundreds of locations, again, automatically. Pre-operational tests have indicated that there are no significant differences in forecast quality originating from either system (Kilpinen et al., 2001). Vice versa, the test results have, as yet, not provided any indication of improvements in forecasts quality since the implementation of the grid editor. Nevertheless, there are clear signals of potential strengths of the grid editor in the forecast production rationalization process and, moreover, a verification approach proves to be a fine means to discern potential improvements in new forecasting system developments.
References:
Kilpinen, J, A. Sarkanen, P. Kukkonen, M. Pietarinen and
V. Kangasniemi, 2001: The Grid Editor And Verification Results. Proc. 8th ECMWF Workshop on
Meteorological Operational Systems (ECMWF, Reading, UK).
Nurmi, P. and J. Rissanen, 1993: A State-of-the-Art Real-Time Verification Scheme. Preprints, 1st European
Conference on Applications of Meteorology (Oxford, UK).
Nurmi, P., 1995 a:
Operational Real-Time Forecast Verification: Impacts on Forecast
Quality. Preprints, 14th Conference on
Weather Analysis and Forecasting (Dallas, Texas, USA) (AMS, Boston), 130.
Nurmi, P., 1995 b:
Influence of Real-Time Forecast Verification on the Quality of Local
Weather Forecasts. Preprints, 2nd European
Conference on Applications of Meteorology (Toulouse, France), 140-143.
Nurmi, P., 1996: Real-Time Verification of Operational Weather Forecasts at the Finnish Meteorological Institute. (Invited lecture at the) Workshop on the Evaluation of Space Weather Forecasts (Boulder, Colorado, USA), 135‑139.