Practically Perfect Hindcasts for Forecast Verification and Evaluation

Mike Kay

Cooperative Institute for Research in Environmental Sciences (CIRES)

University of Colorado/NOAA Research-Forecast Systems Laboratory

Boulder, Colorado

Arial forecasts such as those produced by the Aviation Weather Center and the Storm Prediction Center are often issued with the understanding that certain portions of the forecast area contain no verifying observations or reports. Traditional verification statistics such as the probability of detection and false alarm ratio are heavily impacted by this issue, and thus do not range from 0 to 1 in practice. Because of this limitation, the accuracy of forecasts often appears to be very low, and forecast performance judged to be quite poor. Stated differently, the baselines for skill are dependent upon forecast difficulty, which varies from situation to situation.

A method for dealing with these difficulties has been developed and is dubbed the "practically-perfect" method. The term "practically-perfect" implies a forecast that is consistent with one that a forecaster would make given prior perfect knowledge of the events. Such a forecast can be used to obtain minimum and maximum scores that could reasonably be obtained for a given forecast. This provides a much more useful range over which forecaster performance may be assessed. This concept is not limited to a particular score or set of scores, and may be applied to both dichotomous and probabilistic forecasts.