Bagging and Boosting
Published Online: 15 JUL 2005
Copyright © 2005 John Wiley & Sons, Ltd
Encyclopedia of Biostatistics
How to Cite
De Veaux, R. 2005. Bagging and Boosting. Encyclopedia of Biostatistics. 1.
- Published Online: 15 JUL 2005
Bagging and boosting are examples of ensemble learning methods from data mining. These methods combine the predictions of many different models (usually of the same type) into an ensemble prediction. For classification problems, the normal combining technique is voting, taking the modal prediction of all the models as the ensemble prediction. For regression problems, averaging or other summaries of the various model outputs can be used. In principle, bagging and boosting can be applied to any class of models, although the most widely studied class of models to which bagging and boosting have been applied are decision trees. The result of bagging is an estimate that has reduced variance when compared to the predictions of the individual models. Boosting, by contrast, appears to be able to reduce both the bias and the variance of the predictions. Both methods share the disadvantage, however, that the resulting model is not as interpretable as the constituent models because they average over a large collection of predictions.
- machine learning;
- data mining;
- model averaging;
- committee of models