A combined theory for PCA and PLS

Abstract

We present here an algorithmic approach to modelling data that includes principal component analysis (PCA) and partial least squares (PLS). In fact, the numerical algorithm presented can carry out PCA or PLS. The algorithm for linear analysis and extensions to non-linear analysis applies to both PCA and PLS. The algorithm allows for combination of PCA and PLS types of models and therefore extends modelling to new types of models that involve combination of regression models and ‘selection of variation’ models, which is the idea of PCA-type models. The fact that the algorithm carries out both PCA and PLS shows that PCA and PLS are based on the same theory. This theory is based on the H-principle of mathematical modelling. The algorithm allows tests for outliers, sensitivity analysis and tests of submodels. These aspects of the algorithm are treated in detail. We compute various measures of sizes, e.g. of components, of the covariance matrix, of its inverse, etc. that show how much the algorithm has selected at each step. The analysis is illustrated by data from practice.