Avoiding the Look-Ahead Pathology of Decision Tree Learning
Article first published online: 6 JUN 2013
© 2013 Wiley Periodicals, Inc.
International Journal of Intelligent Systems
Volume 28, Issue 10, pages 974–987, October 2013
How to Cite
Last, M. and Roizman, M. (2013), Avoiding the Look-Ahead Pathology of Decision Tree Learning. Int. J. Intell. Syst., 28: 974–987. doi: 10.1002/int.21612
- Issue published online: 1 AUG 2013
- Article first published online: 6 JUN 2013
Most decision-tree induction algorithms are using a local greedy strategy, where a leaf is always split on the best attribute according to a given attribute-selection criterion. A more accurate model could possibly be found by looking ahead for alternative subtrees. However, some researchers argue that the look-ahead should not be used due to a negative effect (called “decision-tree pathology”) on the decision-tree accuracy. This paper presents a new look-ahead heuristics for decision-tree induction. The proposed method is called look-ahead J48 ( LA-J48) as it is based on J48, the Weka implementation of the popular C4.5 algorithm. At each tree node, the LA-J48 algorithm applies the look-ahead procedure of bounded depth only to attributes that are not statistically distinguishable from the best attribute chosen by the greedy approach of C4.5. A bootstrap process is used for estimating the standard deviation of splitting criteria with unknown probability distribution. Based on a separate validation set, the attribute producing the most accurate subtree is chosen for the next step of the algorithm. In experiments on 20 benchmark data sets, the proposed look-ahead method outperforms the greedy J48 algorithm with the gain ratio and the gini index splitting criteria, thus avoiding the look-ahead pathology of decision-tree induction.