The Calibration of Probability Forecasts: A Perspective
from Signal Detection Theory
32 Hensman St
Latham ACT
2615, Australia
ibmason@bigpond.com
Formulation of a numerical probability forecast can be described in terms of scaling from a latent quantity representing the judgment of the forecaster to the probability interval [0,100]%. The shape and location of a 'normative’ scaling function which would give good calibration is fixed by the parameters delta-m and s of the signal detection model fitted by standard methods to hit and false alarm rates, and by the climatological probability of the event. Systematic miscalibration can be regarded as due to use of an inappropriate scaling function, and the parameters of this 'substantive' function can be estimated from verified forecasts. This provides a means of modeling the reliability diagram and frequency-of-use histogram of the forecasts, and suggests some guidelines for the practice of numerical probability forecasting.