SEARCH

SEARCH BY CITATION

Keywords:

  • Baseline distribution;
  • Economic forecast evaluation;
  • Probability of bankruptcy;
  • Scoring rules

Summary.  Financial institutions and regulatory agencies direct much effort and expertise towards estimating probabilities of bankruptcy. By comparison, the techniques that are used to evaluate probability estimates have attracted little attention and remain somewhat ad hoc. The most common approach is to count misclassifications, based on an arbitrary classification threshold. Some use is made of conventional probability score functions, such as the Brier score (from meteorology) but these are not standard practice. Our purpose is to introduce a family of economic probability score functions designed to capture the utility obtained by a user, with a specified utility function, who uses the estimated probabilities to make hypothetical bets against a rival forecaster or model. The conceptual appeal of these score functions is that probability forecasts are evaluated neither in abstract, nor in isolation, but by whether they would hypothetically have ‘made money’ for a given user, with specified risk aversion, against comparable forecasts or market betting prices.