Stepwise Regression

Stepwise regression is both a general term and a specific method for choosing predictor variables from a larger pool of possible predictors in multiple regression. Related approaches include forward selection and backward elimination. All three procedures may identify different and suboptimal sets of predictors for a given sample of data. Some regression packages can identify the best possible one-predictor model, two-predictor model, and so on, according to the criteria specified by the user. This approach is computer intensive but is feasible with sets of at least 50 or 60 predictors. However they are selected, the best set of predictors in one sample will often not be the best in another sample or in the underlying population. Significance tests and R² values may be far more liberal than their nominal values. Some of the estimated coefficients are also likely to be substantially overstated relative to their population values. Researchers using stepwise regression procedures will generally get better results when they have large samples of observations and relatively small sets of potential predictors; when they consider multiple procedures and criteria for selecting predictors; when they cross-validate findings with new data or with random subsamples from the available data; and when they avoid self-deception in interpreting the findings.

Keywords: regression; correlation; collinearity; variable selection; forward selection; backward elimination