Why is it so hard??
National Center for
Atmospheric Research, Research Applications Program
In this paper we explore the difficulties of verifying convective forecasts. Statistically and intuitively meaningful verification results for convective storm forecasts are often difficult to obtain. The very rapid non-linear evolution of the precipitation field (Wilson et al. 1998) makes both nowcasting and verification of storms difficult. This rapid evolution of the precipitation field is often due to boundary layer and convective scale processes, such as outflow boundary development and interaction with other convergence boundaries. For 2-6 hr nowcast periods, there is general agreement among the science community that nowcast will need to be probabilistic, however the characteristics of these probabilistic forecast has yet to be evaluated and understood by the user communities. The goal of this talk is to discuss the issues associated with validation of deterministic and probabilistic convective storm nowcasts.
In the first part of the talk, statistics are shown to illustrate how relatively minor errors or assumptions in determining the observation field can have major effects on the standard contingency table based statistics. A "perfect forecast" (what actually occurred) is evaluated based on observations that are varied with time, space, and intensity. Results illustrate the sensitivity of the forecast statistics to relatively minor variations in observations.
In the second part of the talk, predictability of storms and potential for probability nowcast products are discussed. Examples of convection with varying predictability will be shown. These examples will illustrate that it is not necessarily the area coverage of the storms that determine predictability but the convective regime in which the storms form. Some of the nowcast uncertainties are tied to the predictability of the trigger or forcing mechanism, (a squall line associated with a cold front is predictable whereas as a complex of storms triggered by the interaction of meso-scale boundaries is not). Another key component of longer-term nowcasts of convection is how to deal with secondary convection associated with gust fronts from storms that are not present when the forecast is issued. Finally, at longer time periods, uncertainties exist in the algorithms themselves for example, extrapolations are not perfect and these errors magnify with increasing forecast time. Potential probability nowcasts will be illustrated, in order to provide a first guess of verification requirements.