Algorithms vs. Forecasts:
Aviation Weather Center,
NOAA
Algorithms which forecast the state of the atmosphere are necessarily different from human forecasts providing a user service. The process differences result in different information and products which require different verification methods to judge skill.
An algorithm is an attempt to model the physics of the atmosphere. An example would be an objective thunderstorm forecast algorithm. The algorithm attempts to model the instability of the atmosphere, the triggering mechanism, the convective cloud, and the response of the atmosphere to the thunderstorm cloud such as hail, lightning, turbulence, wind gusts, etc. The accuracy of the algorithm is limited by available data and parameter assumptions of the algorithm. The scale of the algorithm is limited to the scale of the grids of data that input to the algorithm.
A subjective forecast is an attempt to satisfy users' needs for information about a potential atmospheric hazard. An example would be a thunderstorm forecast such as the Convective SIGMET. The Convective SIGMET provides information to pilots of en route aircraft regarding thunderstorms of sufficient extent and/or intensity as to be potentially hazardous to aircraft operations. The scale of the Convective SIGMET is defined as a line of thunderstorms at least 60 miles long with thunderstorms affecting at least 40 % of its length or an area of thunderstorms covering at least 40% of a 3,000 square mile area or larger.
The scale of the subjective forecast is limited by regulation and human ability to resolve the phenomenon in question in space and time while the scale of the algorithm is limited by the resolution of data processing techniques. Ideally the thunderstorm algorithm should forecast every detected thunderstorm, while the subjective forecast should ideally forecast every thunderstorm of interest to the user. The verification of the algorithm can use every observed thunderstorm in the verification database, while the verification of the subjective forecast must first filter the observations to eliminate those that do not create an impact on the users' operations. Since airplanes can easily fly around isolated thunderstorms, they are not of interest to the user and should be eliminated from the data set before verification services are applied.
The rules of verification should be the same as the rules of the forecast service. Each forecast service has specifications for product form, product communications, product length, as well as product content. These rules must be the basis for the verification processing techniques as much as is possible.
The end goal of algorithm verification is to establish how closely the algorithm can predict events in nature. The end goal of forecast verification is to establish whether information needs of the users are being provided. Since these are different goals, they require different verification methodologies.