How Many Iterations are Sufficient for Efficient Semiparametric Estimation?
Article first published online: 3 APR 2013
© 2013 Board of the Foundation of the Scandinavian Journal of Statistics
Scandinavian Journal of Statistics
Volume 40, Issue 3, pages 592–618, September 2013
How to Cite
CHENG, G. (2013), How Many Iterations are Sufficient for Efficient Semiparametric Estimation?. Scandinavian Journal of Statistics, 40: 592–618. doi: 10.1002/sjos.12005
- Issue published online: 7 AUG 2013
- Article first published online: 3 APR 2013
- Received November 2011, in final form October 2012
- generalized profile likelihood;
- higher order asymptotic efficiency;
- k-step estimation;
- Newton–Raphson algorithm;
- semiparametric models
Abstract. A common practice in obtaining an efficient semiparametric estimate is through iteratively maximizing the (penalized) full log-likelihood w.r.t. its Euclidean parameter and functional nuisance parameter. A rigorous theoretical study of this semiparametric iterative estimation approach is the main purpose of this study. We first show that the grid search algorithm produces an initial estimate with the proper convergence rate. Our second contribution is to provide a formula in calculating the minimal number of iterations k* needed to produce an efficient estimate . We discover that (i) k* depends on the convergence rates of the initial estimate and the nuisance functional estimate, and (ii) k* iterations are also sufficient for recovering the estimation sparsity in high dimensional data. The last contribution is the novel construction of which does not require knowing the explicit expression of the efficient score function. The above general conclusions apply to semiparametric models estimated under various regularizations, for example, kernel or penalized estimation. As far as we are aware, this study provides a first general theoretical justification for the ‘one-/two-step iteration’ phenomena observed in the semiparametric literature.