Numerical Maximisation of Likelihood: A Neglected Alternative to EM?
Article first published online: 24 FEB 2014
© 2014 The Authors. International Statistical Review © 2014 International Statistical Institute.
International Statistical Review
Volume 82, Issue 2, pages 296–308, August 2014
How to Cite
MacDonald, I. L. (2014), Numerical Maximisation of Likelihood: A Neglected Alternative to EM?. International Statistical Review, 82: 296–308. doi: 10.1111/insr.12041
- Issue published online: 4 AUG 2014
- Article first published online: 24 FEB 2014
- Manuscript Accepted: 17 SEP 2013
- Manuscript Revised: 30 AUG 2013
- Manuscript Received: 23 JAN 2013
- maximum likelihood;
- EM algorithm;
- numerical maximisation;
- constrained optimisation;
- Poisson mixtures;
- Markov chains;
- hidden Markov models
There is by now a long tradition of using the EM algorithm to find maximum-likelihood estimates (MLEs) when the data are incomplete in any of a wide range of ways, even when the observed-data likelihood can easily be evaluated and numerical maximisation of that likelihood is available as a conceptually simple route to the MLEs. It is rare in the literature to see numerical maximisation employed if EM is possible. But with excellent general-purpose numerical optimisers now available free, there is no longer any reason, as a matter of course, to avoid direct numerical maximisation of likelihood. In this tutorial, I present seven examples of models in which numerical maximisation of likelihood appears to have some advantages over the use of EM as a route to MLEs. The mathematical and coding effort is minimal, as there is no need to derive and code the E and M steps, only a likelihood evaluator. In all the examples, the unconstrained optimiser nlm available in R is used, and transformations are used to impose constraints on parameters.
I suggest therefore that the following question be asked of proposed new applications of EM: Can the MLEs be found more simply and directly by using a general-purpose numerical optimiser?