On the complexity of Rocchio's similarity-based relevance feedback algorithm

Abstract

Rocchio's similarity-based relevance feedback algorithm, one of the most important query reformation methods in information retrieval, is essentially an adaptive learning algorithm from examples in searching for documents represented by a linear classifier. Despite its popularity in various applications, there is little rigorous analysis of its learning complexity in literature. In this article, the authors prove for the first time that the learning complexity of Rocchio's algorithm is O(d + d²(log d + log n)) over the discretized vector space {0,…, n − 1}^d, when the inner product similarity measure is used. The upper bound on the learning complexity for searching for documents represented by a monotone linear classifier over {0,…, n − 1}^d can be improved to, at most, 1 + 2k (n − 1) (log d − log(n − 1)), where k is the number of nonzero components in q. Several lower bounds on the learning complexity are also obtained for Rocchio's algorithm. For example, the authors prove that Rocchio's algorithm has a lower bound on its learning complexity over the Boolean vector space {0, 1}^d.