On-line expectation–maximization algorithm for latent data models



This article is corrected by:

  1. Errata: Corrigendum: On-line expectation–maximization algorithm for latent data models Volume 73, Issue 5, 792, Article first published online: 18 October 2011

Olivier Cappé, TELECOM ParisTech, 46 rue Barrault, 75013 Paris cedex 13, France.
E-mail: cappe@telecom-paristech.fr


Summary.  We propose a generic on-line (also sometimes called adaptive or recursive) version of the expectation–maximization (EM) algorithm applicable to latent variable models of independent observations. Compared with the algorithm of Titterington, this approach is more directly connected to the usual EM algorithm and does not rely on integration with respect to the complete-data distribution. The resulting algorithm is usually simpler and is shown to achieve convergence to the stationary points of the Kullback–Leibler divergence between the marginal distribution of the observation and the model distribution at the optimal rate, i.e. that of the maximum likelihood estimator. In addition, the approach proposed is also suitable for conditional (or regression) models, as illustrated in the case of the mixture of linear regressions model.