Chapter 4. Bayesian Estimation

Summary

• 4.1 Bayesian estimation theory: basic definitions

• 4.2 Bayesian estimation

• 4.3 The estimate–maximise method

• 4.4 Cramer–Rao bound on the minimum estimator variance

• 4.5 Design of mixture Gaussian models

• 4.6 Bayesian classification

• 4.7 Modeling the space of a random process

• 4.8 Summary

Bayesian estimation is a framework for the formulation of statistical inference problems. In the prediction or estimation a parameter, the Bayesian philosophy is based on combining the evidence contained in the signal with prior knowledge of the probability distribution of the process. Bayesian methodology includes the classical estimators such as maximum a posteriori (MAP), maximum-likelihood (ML), minimum mean square error (MMSE) and minimum mean absolute value of error (MAVE) as special cases. This chapter begins with an introduction to the basic concepts of estimation theory, and considers the statistical measures that are used to quantify the performance of an estimator. We study Bayesian estimation methods and consider the effect of using a prior model on the mean and the variance of an estimate. The estimate–maximise (EM) method for the estimation of a set of unknown parameters from an incomplete observation is studied, and applied to the mixture Gaussian modelling of the space of a continuous random variable. This chapter concludes with an introduction to the Bayesian classification of discrete or finite-state signals, and the K-means clustering method.