SEARCH

SEARCH BY CITATION

Keywords:

  • maximum likelihood;
  • EM algorithm;
  • numerical maximisation;
  • constrained optimisation;
  • Poisson mixtures;
  • Markov chains;
  • hidden Markov models

Abstract

There is by now a long tradition of using the EM algorithm to find maximum-likelihood estimates (MLEs) when the data are incomplete in any of a wide range of ways, even when the observed-data likelihood can easily be evaluated and numerical maximisation of that likelihood is available as a conceptually simple route to the MLEs. It is rare in the literature to see numerical maximisation employed if EM is possible. But with excellent general-purpose numerical optimisers now available free, there is no longer any reason, as a matter of course, to avoid direct numerical maximisation of likelihood. In this tutorial, I present seven examples of models in which numerical maximisation of likelihood appears to have some advantages over the use of EM as a route to MLEs. The mathematical and coding effort is minimal, as there is no need to derive and code the E and M steps, only a likelihood evaluator. In all the examples, the unconstrained optimiser nlm available in R is used, and transformations are used to impose constraints on parameters.

I suggest therefore that the following question be asked of proposed new applications of EM: Can the MLEs be found more simply and directly by using a general-purpose numerical optimiser?