Standard Article

Bagging and Boosting

  1. Richard De Veaux

Published Online: 15 JUL 2005

DOI: 10.1002/0470011815.b2a13003

Encyclopedia of Biostatistics

Encyclopedia of Biostatistics

How to Cite

De Veaux, R. 2005. Bagging and Boosting. Encyclopedia of Biostatistics. 1.

Author Information

  1. Williams College, Williamstown, MA, USA

Publication History

  1. Published Online: 15 JUL 2005


Bagging and boosting are examples of ensemble learning methods from data mining. These methods combine the predictions of many different models (usually of the same type) into an ensemble prediction. For classification problems, the normal combining technique is voting, taking the modal prediction of all the models as the ensemble prediction. For regression problems, averaging or other summaries of the various model outputs can be used. In principle, bagging and boosting can be applied to any class of models, although the most widely studied class of models to which bagging and boosting have been applied are decision trees. The result of bagging is an estimate that has reduced variance when compared to the predictions of the individual models. Boosting, by contrast, appears to be able to reduce both the bias and the variance of the predictions. Both methods share the disadvantage, however, that the resulting model is not as interpretable as the constituent models because they average over a large collection of predictions.


  • machine learning;
  • data mining;
  • model averaging;
  • bootstrap;
  • committee of models