Geometric consistency of principal component scores for high‐dimensional mixture models and its application

Abstract

In this article, we consider clustering based on principal component analysis (PCA) for high‐dimensional mixture models. We present theoretical reasons why PCA is effective for clustering high‐dimensional data. First, we derive a geometric representation of high‐dimension, low‐sample‐size (HDLSS) data taken from a two‐class mixture model. With the help of the geometric representation, we give geometric consistency properties of sample principal component scores in the HDLSS context. We develop ideas of the geometric representation and provide geometric consistency properties for multiclass mixture models. We show that PCA can cluster HDLSS data under certain conditions in a surprisingly explicit way. Finally, we demonstrate the performance of the clustering using gene expression datasets.

1 INTRODUCTION

High‐dimension, low‐sample‐size (HDLSS) data situations occur in many areas of modern science such as genetic microarrays, medical imaging, text recognition, finance, chemometrics, and so on. In recent years, substantial work has been done on HDLSS asymptotic theory, where the sample size n is fixed or n/d→0 as the data dimension d→∞. Hall, Marron, and Neeman (2005), Ahn, Marron, Muller, and Chi (2007), Yata and Aoshima (2012), and Lv (2013) explored several types of geometric representations of HDLSS data. Jung and Marron (2009) showed inconsistency properties of the sample eigenvalues and eigenvectors in the HDLSS context. Yata and Aoshima (2012, 2013) developed the noise‐reduction methodology to provide consistent estimators of both the eigenvalues and eigenvectors together with principal component (PC) scores in the HDLSS context. Shen, Shen, Zhu, and Marron (2016) and Hellton and Thoresen (2017) also provided several asymptotic properties of the sample PC scores in the HDLSS context.

The HDLSS asymptotic theory was created under the assumption of either the population distribution is Gaussian or the random variables in a sphered data matrix have a ρ‐mixing dependency. However, Yata and Aoshima (2010) developed an HDLSS asymptotic theory without such assumptions. Moreover, they created a new principal component analysis (PCA) called the cross‐data‐matrix methodology that is applicable to construct an unbiased estimator in HDLSS nonparametric settings. Based on the cross‐data‐matrix methodology, Aoshima and Yata (2011) developed a variety of inference for HDLSS data such as given‐bandwidth confidence region, two‐sample test, classification, variable selection, regression, pathway analysis, and so on (see Aoshima et al., 2018 for the review).

PCA is an important visualization and dimension reduction technique for high‐dimensional data. Furthermore, PCA is quite popular for clustering high‐dimensional data (see section 9.2 in Jolliffe, 2002 for details). For clustering HDLSS gene expression data, see Armstrong et al. (2002) and Pomeroy et al. (2002). Liu, Hayes, Nobel, and Marron (2008) and Ahn, Lee, and Yoon (2012) gave binary split‐type clustering methods for HDLSS data. Borysov, Hannig, and Marron (2014) considered hierarchical clustering for high‐dimensional data. Li and Yao (2018) considered a model‐based clustering for a high‐dimensional mixture. Given this background, we decided to focus on high‐dimensional structures of multiclass mixture models via PCA. In this article, we consider asymptotic properties of PC scores for high‐dimensional mixture models to apply for cluster analysis in HDLSS settings. The main contribution of this article is that we give theoretical reasons why PCA is effective for clustering HDLSS data.

Suppose there are independent and d‐variate populations, Π_i,i=1,…,k, having an unknown mean vector μ_i and unknown (positive‐semidefinite) covariance matrix ∑_i for each i. We consider a mixture model to classify a dataset into k (≥2) groups. We assume that any sample is taken with mixing proportions ε_is from Π_is, where ε_i∈(0,1) and ${\sum }_{i=1}^k{\epsilon }_i=1$ but the label of the population is missing. We assume that k and ε_is are independent of d. We consider a mixture model whose probability density function (or probability function) is given by

$$\begin{array}{ll}f(\mathit{x})={\displaystyle \sum _{i=1}^k}{\epsilon }_i{\pi }_i(\mathit{x};{\mathit{\mu }}_i,{\mathbf{\sum }}_i),& \end{array}$$ (1)

where $\mathit{x}\in {\mathbb{R}}^d$ and π_i(x;μ_i,∑_i) is a d‐dimensional probability density function (or probability function) of Π_i having a mean vector μ_i and covariance matrix ∑_i. Suppose we have a d×n data matrix X=[x₁,…,x_n], where x_j,j=1,…,n, are independently taken from Equation 1. We assume n≥k. Let

$$n_i=\#\{j|{\mathit{x}}_j\in {\mathrm{\Pi }}_i\text{for}j=1,\dots ,n\}\text{and}{\eta }_i=n_i/n\text{for}i=1,\dots ,k,$$

where #A denotes the number of elements in a set A. We assume that n and n_is are independent of d. Let μ and ∑ be the mean vector and the covariance matrix of Equation 1, respectively. Then, we have that

$$\mathit{\mu }={\displaystyle \sum _{i=1}^k}{\epsilon }_i{\mathit{\mu }}_i\text{and}\mathbf{\sum }={\displaystyle \sum _{i=1}^{k-1}}{\displaystyle \sum _{j=i+1}^k}{\epsilon }_i{\epsilon }_j({\mathit{\mu }}_i-{\mathit{\mu }}_j){({\mathit{\mu }}_i-{\mathit{\mu }}_j)}^{\mathrm{T}}+{\displaystyle \sum _{i=1}^k}{\epsilon }_i{\mathbf{\sum }}_i.$$

We note that E(x|x∈ Π_i)=μ_i and Var(x|x∈ Π_i)= ∑_i for i=1,…,k. We denote the eigendecomposition of ∑ by ∑=HΛH^T, where Λ=diag(λ₁,…,λ_d) having eigenvalues λ₁≥…≥λ_d≥0 and H=[h₁,…,h_d] is an orthogonal matrix of the corresponding eigenvectors. Let ${\mathit{x}}_j-\mathit{\mu }=\mathit{H}{\mathbf{\Lambda }}^{1/2}{(z_{1j},\dots ,z_{dj})}^{\mathrm{T}}$ for j=1,…,n. Then, (z_1j,…,z_dj)^T is a sphered data vector from a distribution with the identity covariance matrix; E{(z_1j,…,z_dj)^T}=0 and $\mathrm{Var}\{{(z_{1j},\dots ,z_{dj})}^{\mathrm{T}}\}={\mathit{I}}_d$, where I_d denotes the d‐square identity matrix. The ith true PC score of x_j is given by

$${\mathit{h}}_i^{\mathrm{T}}({\mathit{x}}_j-\mathit{\mu })={\lambda }_i^{1/2}{z}_{ij}\text{(hereafter called}{s}_{ij}).$$

We note that Var(s_ij)=λ_i for all i,j. Let Δ_i=||μ_i||² for i=1,…,k, where ||·|| denotes the Euclidean norm. Here, we assume that

$${\mathrm{\Delta }}_1\ge {\mathrm{\Delta }}_2\ge \mathrm{···}\ge {\mathrm{\Delta }}_k,$$

without loss of generality. We also assume that

$${\mathrm{\Delta }}_k=0(\text{i.e.,}{\mathit{\mu }}_k=\mathbf{0}),$$

for the sake of simplicity.

Remark 1.When μ_k≠0, let ${\mathit{\mu }}_i^{\prime }={\mathit{\mu }}_i-{\mathit{\mu }}_k$ for i=1,…,k. Then, it holds that ${\sum }_{i=1}^{k-1}{\sum }_{j=i+1}^k{\epsilon }_i{\epsilon }_j({\mathit{\mu }}_i-{\mathit{\mu }}_j){({\mathit{\mu }}_i-{\mathit{\mu }}_j)}^{\mathrm{T}}={\sum }_{i=1}^{k-1}{\sum }_{j=i+1}^k{\epsilon }_i{\epsilon }_j({\mathit{\mu }}_i^{\prime }-{\mathit{\mu }}_j^{\prime }){({\mathit{\mu }}_i^{\prime }-{\mathit{\mu }}_j^{\prime })}^{\mathrm{T}}$. Hence, for any inference of ∑ by the sample covariance matrix, one can assume μ_k=0 without loss of generality.

As the sign of an eigenvector is arbitrary, we assume that ${\mathit{h}}_i^{\mathrm{T}}{\mathit{\mu }}_i\ge 0$ for i=1,…,k−1, without loss of generality. In addition, we assume the cluster means are more spread than the within class variation in the sense that:

Condition 1. $\frac{{\max }_{i=1,\dots ,k}{\lambda }_{\max }({\mathbf{\sum }}_i)}{{\mathrm{\Delta }}_{k-1}}\to 0$ as d→∞.

Here, ${\lambda }_{\max }(\mathit{M})$ denotes the largest eigenvalue of any positive‐semidefinite matrix, M. We consider clustering x₁,…,x_n into one of Π_is in HDLSS situations. When k=2, Yata and Aoshima (2010) gave the following result: we denote the angle between two nonzero vectors x and y by $\text{Angle}(\mathit{x},\mathit{y})={\cos }^{-1}\{{\mathit{x}}^{\mathrm{T}}\mathit{y}/(||\mathit{x}||·||\mathit{y}||)\}$. By noting that μ₂=0, under Condition 1, it holds that as d→∞

$$\begin{array}{ll}\frac{{\lambda }_1}{{\epsilon }_1{\epsilon }_2{\mathrm{\Delta }}_1}\to 1\text{and}\text{Angle}({\mathit{h}}_1,{\mathit{\mu }}_1)\to 0,& \end{array}$$ (2)

from the fact that ${\lambda }_1/\mathrm{\Delta }={\mathit{h}}_1^{\mathrm{T}}\mathbf{\sum }{\mathit{h}}_1/\mathrm{\Delta }={\epsilon }_1{\epsilon }_2{({\mathit{h}}_1^{\mathrm{T}}{\mathit{\mu }}_1)}^2+o(1)$ as d→∞ under Condition 1. Furthermore, for the normalized first PC score $s_{1j}/{\lambda }_1^{1/2}(=z_{1j})$, it follows that

$$\underset{d\to \infty }{\mathrm{plim}}\frac{s_{1j}}{{\lambda }_1^{1/2}}=\left\{\begin{array}{ll}{({\epsilon }_2/{\epsilon }_1)}^{1/2}& \text{when}{\mathit{x}}_j\in {\mathrm{\Pi }}_1,\\ -{({\epsilon }_1/{\epsilon }_2)}^{1/2}& \text{when}{\mathit{x}}_j\in {\mathrm{\Pi }}_2,\end{array}\right.$$ (3)

for j=1,…,n. Here, “plim” denotes the convergence in probability. This result is a special case of Theorem 2 in Section 3. See Remark 8. One would be able to cluster x_js into two groups if s_1j is accurately estimated in HDLSS situations.

In this article, we consider asymptotic properties of sample PC scores for Equation 1 in the HDLSS context that d→∞ while n is fixed. In Section 2, we first derive a geometric representation of HDLSS data taken from the two‐class mixture model. With the help of the geometric representation, we give geometric consistency properties of sample PC scores in the HDLSS context. We show that PCA can cluster HDLSS data under certain conditions in a surprisingly explicit way. In Section 3, we investigate asymptotic behaviors of true PC scores for the k(≥3)‐class mixture model and provide geometric consistency properties of sample PC scores when k≥3. In Section 4, we demonstrate the performance of clustering based on sample PC scores using gene expression datasets. We show that the real‐HDLSS datasets hold the geometric consistency properties.

2 PC SCORES FOR TWO‐CLASS MIXTURE MODEL

In this section, we consider PC scores for the two‐class (k=2) mixture model.

2.1 Preliminary

The sample covariance matrix is given by $\mathit{S}={(n-1)}^{-1}(\mathit{X}-\bar{\mathit{X}}){(\mathit{X}-\bar{\mathit{X}})}^{\mathrm{T}}={(n-1)}^{-1}{\sum }_{j=1}^n({\mathit{x}}_j-{\bar{\mathit{x}}}_n){({\mathit{x}}_j-{\bar{\mathit{x}}}_n)}^{\mathrm{T}}$, where ${\bar{\mathit{x}}}_n=n^{-1}{\sum }_{j=1}^n{\mathit{x}}_j$ and $\bar{\mathit{X}}={\bar{\mathit{x}}}_n{\mathbf{1}}_n^{\mathrm{T}}$ with ${\mathbf{1}}_n={(1,\dots ,1)}^{\mathrm{T}}\in {\mathbb{R}}^n$. Then, we define the n×n dual sample covariance matrix by ${\mathit{S}}_{\mathrm{D}}={(n-1)}^{-1}{(\mathit{X}-\bar{\mathit{X}})}^{\mathrm{T}}(\mathit{X}-\bar{\mathit{X}})$. We note that rank(S_D)≤n−1. Let ${\widehat{\lambda }}_1\ge \mathrm{···}\ge {\widehat{\lambda }}_{n-1}\ge 0$ be the eigenvalues of S_D. Then, we define the eigendecomposition of S_D by

$${\mathit{S}}_{\mathrm{D}}={\displaystyle \sum _{i=1}^{n-1}}{\widehat{\lambda }}_i{\widehat{\mathit{u}}}_i{\widehat{\mathit{u}}}_i^{\mathrm{T}},$$

where ${\widehat{\mathit{u}}}_i={({û}_{i1},\dots ,{û}_{in})}^{\mathrm{T}}$ denotes a unit eigenvector corresponding to ${\widehat{\lambda }}_i$. As the sign of ${\widehat{\mathit{u}}}_i$s is arbitrary, we assume ${\widehat{\mathit{u}}}_i^{\mathrm{T}}{\mathit{z}}_i\ge 0$ for all i without loss of generality, where z_i is defined by z_i=(z_i1,…,z_in)^T. Note that S and S_D share the nonzero eigenvalues. Let

$${\widehat{z}}_{ij}={û}_{ij}{n}^{1/2}\text{for}i=1,\dots ,n-1;j=1,\dots ,n.$$

We note that ${\widehat{z}}_{ij}$ is an estimate of $s_{ij}/{\lambda }_i^{1/2}(=z_{ij})$ for i=1,…,n−1;j=1,…,n from the facts that

$${\widehat{z}}_{ij}={\{n/(n-1)\}}^{1/2}{\widehat{\mathit{h}}}_i^{\mathrm{T}}({\mathit{x}}_j-{\bar{\mathit{x}}}_n)/{\widehat{\lambda }}_i^{1/2}\text{and}{\displaystyle \sum _{j=1}^n}{\widehat{z}}_{ij}^2/n=1\text{if}{\widehat{\lambda }}_i>0,$$

where ${\widehat{\mathit{h}}}_i$ denotes a unit eigenvector of S corresponding to ${\widehat{\lambda }}_i$. Let ${\mathit{X}}_0=\mathit{X}-\mathit{\mu }{\mathbf{1}}_n^{\mathrm{T}}$ and ${\mathit{P}}_n={\mathit{I}}_n-n^{-1}{\mathbf{1}}_n{\mathbf{1}}_n^{\mathrm{T}}$. We note that ${\mathit{S}}_{\mathrm{D}}={\mathit{P}}_n{\mathit{X}}_0^{\mathrm{T}}{\mathit{X}}_0{\mathit{P}}_n/(n-1)$. We consider the sphericity condition: tr(∑²)/tr(∑)²→0 as d→∞. Note that the sphericity condition is equivalent to “λ₁/tr(∑)→0 as d→∞.” When one can assume that X is Gaussian or Z=(z_ij) is ρ‐mixing and the fourth moments of each variable in Z are uniformly bounded, under the sphericity condition, Jung and Marron (2009) suggested a geometric representation as follows:

$$\begin{array}{ll}\underset{d\to \infty }{\mathrm{plim}}\frac{{\mathit{X}}_0^{\mathrm{T}}{\mathit{X}}_0}{\text{tr}(\mathbf{\sum })}={\mathit{I}}_n,\text{so that}\underset{d\to \infty }{\mathrm{plim}}\frac{(n-1){\mathit{S}}_{\mathrm{D}}}{\text{tr}(\mathbf{\sum })}={\mathit{P}}_n.& \end{array}$$ (4)

Remark 2.Yata and Aoshima (2012) showed that Equation 4 holds under the sphericity condition and Var(||x_j−μ||²)/tr(∑)²→0 as d→∞.

From Equation 4, we observe that the eigenvalue becomes deterministic as the dimension increases, whereas the eigenvector of S_D does not uniquely determine the direction. In addition, Hellton and Thoresen (2017) present asymptotic properties of the sample PC scores when Z is ρ‐mixing. We note that Equation 1 does not presuppose the assumption that X is Gaussian or Z is ρ‐mixing. See section 4.1.1 in Qiao, Zhang, Liu, Todd, and Marron (2010) for details. In the present article, we present new asymptotic properties of the sample PC for Equation 1.

2.2 Geometric representation and consistency property of PC scores when k = 2

We will find a geometric representation for Equation 1 and the finding is completely different from Equation 4. We assume the following conditions:

Condition 2. $\frac{{\max }_{i=1,\dots ,k}\text{tr}({\mathbf{\sum }}_i^2)}{{\mathrm{\Delta }}_{k-1}^2}\to 0$ as d→∞.

Condition 3. $\frac{{\max }_{i=1,\dots ,k}\mathrm{Var}(||\mathit{x}-{\mathit{\mu }}_i|{|}^2|\mathit{x}\in {\mathrm{\Pi }}_i)}{{\mathrm{\Delta }}_{k-1}^2}\to 0$ as d→∞.

Condition 4. $\frac{\text{tr}({\mathbf{\sum }}_i)-\text{tr}({\mathbf{\sum }}_j)}{{\mathrm{\Delta }}_{k-1}}\to 0$ as d→∞ for all i,j=1,…,k(i<j).

Remark 3.Condition 2 is stronger than Condition 1 as it holds that ${\{{\lambda }_{\max }({\mathbf{\sum }}_i)\}}^2\le \text{tr}({\mathbf{\sum }}_i^2)$ for i=1,…,k. Let β(>0) be a constant such that ${\text{lim inf}}_{d\to \infty }$ (Δ_k−1/d^β)>0. Let λ_i1≥···≥λ_id≥0 be eigenvalues of ∑_i for i=1,…,k. For a spiked model such as

$${\lambda }_{ij}=a_{ij}{d}^{{\alpha }_{ij}}(j=1,\dots ,t_i)\text{and}{\lambda }_{ij}=c_{ij}(j=t_i+1,\dots ,d),$$

with positive constants, a_ij, c_ij, and α_ij (not depending on d), and a positive integer t_i (not depending on d), Condition 1 holds when α_i1<β for i=1,…,k. Also, Condition 2 holds when β>1/2 and α_i1<β for i=1,…,k. See Yata and Aoshima (2012) for the details of the spiked model.

Remark 4.If Π_is are Gaussian, it holds that $\mathrm{Var}(||\mathit{x}-{\mathit{\mu }}_i|{|}^2|\mathit{x}\in {\mathrm{\Pi }}_i)=O\{\text{tr}({\mathbf{\sum }}_i^2)\}$ for i=1,…,k, so that Condition 3 naturally holds under Condition 2.

Remark 5.When k=2, Condition 4 holds if tr(∑₁)/tr(∑₂)→1 as d→∞ and ${\text{lim inf}}_{d\to \infty }\{{\mathrm{\Delta }}_1/\text{tr}(\mathbf{\sum })\}>0$.

We define that

$$r_j={(-1)}^{i+1}(1-{\eta }_i)\text{according to}{\mathit{x}}_j\in {\mathrm{\Pi }}_i\text{for}j=1,\dots ,n.$$

The following result gives a geometric representation for Equation 1 when k=2.

Theorem 1.Assume Δ₁/tr(∑)→c(>0) as d→∞. Under Conditions 2, it holds

$$\underset{d\to \infty }{\mathrm{plim}}\frac{(n-1){\mathit{S}}_{\mathrm{D}}}{\mathrm{tr}(\mathbf{\sum })}=c\mathit{r}{\mathit{r}}^{\mathrm{T}}+(1-{\epsilon }_1{\epsilon }_{2}c){\mathit{P}}_n,$$ (5)

where r=(r₁,…,r_n)^T.

When S_D≠O, we note that ${\widehat{\mathit{u}}}_1^{\mathrm{T}}{\mathbf{1}}_n=0$, so that ${\widehat{\mathit{u}}}_1^{\mathrm{T}}{\mathit{P}}_n={\widehat{\mathit{u}}}_1^{\mathrm{T}}$. Thus from Equation 5, the first eigenvector of S_D uniquely determines the direction. In fact, by noting ${\mathit{r}}^{\mathrm{T}}{\mathbf{1}}_n=0$ and ||r||²=nη₁η₂, we have the following results for the first eigenvector and PC scores when k=2. Using Corollary 1, one can cluster x_js into two groups by the sign of ${\widehat{z}}_{1j}$s:

Corollary 1.Under Conditions 2, it holds that for n_i>0,i=1,2

$$\begin{array}{ll}& \underset{d\to \infty }{\mathrm{plim}}{\widehat{\mathit{u}}}_1=\frac{\mathit{r}}{{(n{\eta }_1{\eta }_2)}^{1/2}}\mathrm{and}\\ & \underset{d\to \infty }{\mathrm{plim}}{\widehat{z}}_{1j}=\left\{\begin{array}{ll}{({\eta }_2/{\eta }_1)}^{1/2}& \mathrm{when}{\mathit{x}}_j\in {\mathrm{\Pi }}_1,\\ -{({\eta }_1/{\eta }_2)}^{1/2}& \mathrm{when}{\mathit{x}}_j\in {\mathrm{\Pi }}_2,\end{array}\right.\text{for}j=1,\dots ,n.\end{array}$$

We considered an easy example such as Π_i:N_d(μ_i,∑_i),i=1,2, with μ₁=1_d, μ₂=0, ${\mathbf{\sum }}_1=(0.3^{|i-j{|}^{1/3}})$, and ${\mathbf{\sum }}_2=\mathit{B}(0.3^{|i-j{|}^{1/3}})\mathit{B}$, where B=diag[−{0.5+1/(d+1)}^1/2,{0.5+2/(d+1)}^1/2,…,(−1)^d{0.5+d/(d+1)}^1/2]. We note that Δ₁=d and ∑₁≠ ∑₂ but tr(∑₁)=tr(∑₂)=d. Then, Conditions 1 hold. We set n₁=1 and n₂=2. We took n=3 samples as x₁∈ Π₁ and x₂,x₃∈ Π₂. In Figure 1, we displayed scatter plots of 20 independent pairs of $\pm {\widehat{\mathit{u}}}_1$ when (a) d=5, (b) d=50, (c) d=500, and (d) d=5,000. We denoted r=(2/3,−1/3,−1/3)^T by the solid line and 1_n=(1,1,1)^T by the dotted line. We note that Angle $({\widehat{\mathit{u}}}_1,{\mathbf{1}}_n)=\pi /2$ when S_D≠O. We observed that all the plots of $\pm {\widehat{\mathit{u}}}_1$ gather on the surface of the orthogonal complement of 1_n. Also, the plots appeared close to r as d increases. Thus, one can cluster x_js into two groups by the sign of ${\widehat{z}}_{1j}$s.

Figure 1

Toy example to illustrate the geometric representation of $\pm {\widehat{\mathit{u}}}_1$ on the unit sphere when k=2 and n=3. We plotted 20 independent pairs of $\pm {\widehat{\mathit{u}}}_1$ when x₁∈ Π₁ and x₂,x₃∈ Π₂. The solid line denotes r=(2/3,−1/3,−1/3)^T and the dotted line denotes 1_n=(1,1,1)^T [Color figure can be viewed at wileyonlinelibrary.com]

Next, we investigated robustness of Corollary 1 against Condition 4 by some simulation studies. Let Δ_∑=|tr(∑₁)−tr(∑₂)|. We considered an easy example such as Π_i:N_d(μ_i,∑_i),i=1,2, with μ₁=(1,…,1,0,…,0)^T whose first ⌈d^3/4⌉ elements are 1, μ₂=0, ${\mathbf{\sum }}_1=\gamma (0.3^{|i-j{|}^{1/3}})$, and ${\mathbf{\sum }}_2=\mathit{B}(0.3^{|i-j{|}^{1/3}})\mathit{B}$, where γ≥1. Here, ⌈·⌉ denotes the ceiling function. Note that Δ_∑=(γ−1)d. We set d=5,000, n=10, n₁=4, and n₂=6. We took n=10 samples as x₁,…,x₄∈ Π₁ and x₅,…,x₁₀∈ Π₂. In Figure 2, we displayed scatter plots of $({\widehat{z}}_{1j},{\widehat{z}}_{2j})$, j=1,…,n, when (a) γ=1+2d^−1/2, (b) γ=1+2d^−1/4, and (c) γ=3. From Corollary 1 we denoted (3/2)^1/2 and −(2/3)^1/2 by dotted lines. Note that Δ_∑/Δ₁≈2d^−1/4 for (a), Δ_∑/Δ₁≈2 for (b), and Δ_∑/Δ₁≈2d^1/4 for (c). Thus, Condition 4 holds for (a), while it does not hold for (b) and (c). For (a) and (b), we observed that the estimated PC scores give good performances. On the other hand, the first PC score did not gather around (3/2)^1/2 or −(2/3)^1/2 for (c). However, $({\widehat{z}}_{1j},{\widehat{z}}_{2j})$s were concentrated on the origin (0,0) for x_j∈ Π₂.

Figure 2

Toy example to illustrate asymptotic behaviors of the estimated principal component scores when k=2. We plotted $({\widehat{z}}_{1j},{\widehat{z}}_{2j})$ which is denoted by small circles when x_j∈ Π₁ and by small triangles when x_j∈ Π₂. The theoretical convergent points, (3/2)^1/2 and −(3/2)^1/2, are denoted by dotted lines [Color figure can be viewed at wileyonlinelibrary.com]

When Δ₁/Δ_∑→0 as d→∞, we give the following result to explain the reason of the phenomenon in Figure 2c. Under the assumptions of Proposition 1, one can cluster x_js into two groups by the size of ${\widehat{z}}_{ij}$s even when Condition 4 is not met:

Proposition 1.Assume k=2, $n_{l_{\ast }}\ge 2$ and $n_{l_{\ast }^{\prime }}\ge 1$, where l_∗(≠l^′_∗) is an integer such that $\mathrm{tr}({\mathbf{\sum }}_{l_{\ast }})>\text{tr}({\mathbf{\sum }}_{l_{\ast }^{\prime }})$. Assume also that ${\max }_{l=1,2}\mathrm{tr}({\mathbf{\sum }}_l^2)/{\mathrm{\Delta }}_{\mathrm{\sum }}^2\to 0$, ${\max }_{l=1,2}\mathrm{Var}(||\mathit{x}-{\mathit{\mu }}_l|{|}^2|\mathit{x}\in {\mathrm{\Pi }}_l)/{\mathrm{\Delta }}_{\mathrm{\sum }}^2\to 0$ and Δ₁/Δ_∑→0 as d→∞. Then, it holds that

$$\begin{array}{ll}& \underset{d\to \infty }{\mathrm{plim}}|{\widehat{z}}_{ij}|>0\mathrm{when}{\mathit{x}}_j\in {\mathrm{\Pi }}_{l_{\ast }}\mathrm{for}\mathrm{some}i\in [1,n_{l_{\ast }}-1]\mathrm{and}\\ & \underset{d\to \infty }{\mathrm{plim}}{\widehat{z}}_{ij}=0\mathrm{when}{\mathit{x}}_j\in {\mathrm{\Pi }}_{l_{\ast }^{\prime }}\mathrm{for}i=1,\dots ,n_{l_{\ast }}-1.\end{array}$$

Remark 6.For k≥3, we do not give any consistency property when Condition 4 is not met because the sufficient conditions of Proposition 1 become quite complicated for k≥3. Detailed study for the case when k≥3 is left for a future work.

The assumptions of Proposition 1 hold for (c) of Figure 2. Thus, $({\widehat{z}}_{1j},{\widehat{z}}_{2j})$s were concentrated on the origin (0,0) for x_j∈ Π₂ in (c).

3 PC SCORES FOR MULTICLASS MIXTURE MODEL

In this section, we consider PC scores for the k(≥3)‐class mixture model.

3.1 Asymptotic behaviors of true PC scores when k ≥ 3

Let

$${\epsilon }_{(0)}=0\text{and}{\epsilon }_{(i)}={\displaystyle \sum _{j=1}^i}{\epsilon }_j\text{for}i=1,\dots ,k.$$

We assume the following condition:

Condition 5. $\text{Angle}({\mathit{\mu }}_i,{\mathit{\mu }}_j)\to \frac{\pi }{2}$ and $\frac{{\mathrm{\Delta }}_j}{{\mathrm{\Delta }}_i}\to 0$ as d→∞ for i,j=1,…,k−1(i<j).

Remark 7.We consider the case when all elements of μ_is are constants (not depending on d) such as μ_i=(μ_i1,…,μ_ip,0,…,0) with μ_is≠0 (not depending on d) for s=1,…,p. If all elements of μ_is are constants, the condition “Angle(μ_i, μ_j) →π/2 as d→∞” holds for i<j under Δ_j/Δ_i→0 as d→∞, so that Condition 5 holds under Δ_i+1/Δ_i→0 as d→∞ for i=1,…,k−2. See the settings of Figures 3 and 4. Note that Δ₁≫···≫Δ_k−1 under Condition 5. We emphasize that Conditions 1 become strict as k increases under Condition 5.

Figure 3

Toy example to illustrate the asymptotic behaviors of true principal component scores when k=3. We plotted (z_1j,z_2j) which is denoted by small circles when x_j∈ Π₁, by small triangles when x_j∈ Π₂, and by small squares when x_j∈ Π₃. The dashed triangle consists of three vertices, namely, (1,0), (−1,2^1/2), and (−1,−2^1/2), which are theoretical convergent points [Color figure can be viewed at wileyonlinelibrary.com]

Figure 4

Toy example to illustrate the asymptotic behaviors of true principal component scores when k=4. We plotted (z_1j,z_2j,z_3j). The dashed triangular pyramid was given by Equation 6 with k=4 [Color figure can be viewed at wileyonlinelibrary.com]

We have the following results.

Theorem 2.Under Conditions 1 and 5, it holds that for i=1,…,k−1;j=1,…,n

$$\begin{array}{ll}\underset{d\to \infty }{\mathrm{plim}}\frac{s_{ij}}{{\lambda }_i^{1/2}}=\left\{\begin{array}{ll}0& \mathrm{when}i\ge 2\mathrm{and}{\mathit{x}}_j\in {\displaystyle {\cup }_{m=1}^{i-1}}{\mathrm{\Pi }}_m,\\ {\left(\frac{1-{\epsilon }_{(i)}}{{\epsilon }_i(1-{\epsilon }_{(i-1)})}\right)}^{1/2}& \mathrm{when}{\mathit{x}}_j\in {\mathrm{\Pi }}_i,\\ -{\left(\frac{{\epsilon }_i}{(1-{\epsilon }_{(i)})(1-{\epsilon }_{(i-1)})}\right)}^{1/2}& \mathrm{when}{\mathit{x}}_j\in {\displaystyle {\cup }_{m=i+1}^k}{\mathrm{\Pi }}_m.\end{array}\right.& \end{array}$$ (6)

Remark 8.The consistency in Equation 3 is equivalent to Equation 6 with k=2 and i=1.

Corollary 2.Under Conditions 1 and 5, it holds that for i=1,…,k−1

$$\begin{array}{ll}\frac{{\lambda }_i}{{\epsilon }_i(1-{\epsilon }_{(i)}){\mathrm{\Delta }}_i/(1-{\epsilon }_{(i-1)})}\to 1\mathrm{and}\text{Angle}({\mathit{h}}_i,{\mathit{\mu }}_i)\to 0asd\to \infty .& \end{array}$$

For example, when k=3, from Equation 6, we have that for j=1,…,n

$$\begin{array}{lcr}& & \underset{d\to \infty }{\mathrm{plim}}\frac{s_{1j}}{{\lambda }_1^{1/2}}=\left\{\begin{array}{ll}{\{(1-{\epsilon }_1)/{\epsilon }_1\}}^{1/2}& \text{when}{\mathit{x}}_j\in {\mathrm{\Pi }}_1,\\ -{\{{\epsilon }_1/(1-{\epsilon }_1)\}}^{1/2}& \text{when}{\mathit{x}}_j\notin {\mathrm{\Pi }}_1\end{array}\right.\text{and}\\ & & \underset{d\to \infty }{\mathrm{plim}}\frac{s_{2j}}{{\lambda }_2^{1/2}}=\left\{\begin{array}{ll}0& \text{when}{\mathit{x}}_j\in {\mathrm{\Pi }}_1,\\ {[{\epsilon }_3/\{{\epsilon }_2(1-{\epsilon }_1)\}]}^{1/2}& \text{when}{\mathit{x}}_j\in {\mathrm{\Pi }}_2,\\ {-[{\epsilon }_2/\{{\epsilon }_3(1-{\epsilon }_1)\}]}^{1/2}& \text{when}{\mathit{x}}_j\in {\mathrm{\Pi }}_3.\end{array}\right.\end{array}$$

One can check whether x_j∈ Π₁ or not by the first PC score. If x_j∉ Π₁, one can check whether x_j∈ Π₂ or x_j∈ Π₃ by the second PC score. In general, one can cluster x_js using at most the first k−1 PC scores.

We considered a toy example such as Π_i:N_d(μ_i,∑_i),i=1,…,4, where μ₁=1_d, μ₂=(1,…,1,0,…,0)^T whose first ⌈d^3/4⌉ elements are 1, μ₃=(1,…,1,0,…,0)^T whose first ⌈d^1/2⌉ elements are 1, and μ₄=0. We set ${\mathbf{\sum }}_1=(0.3^{|i-j{|}^{1/3}})$, ${\mathbf{\sum }}_2=\mathit{B}(0.3^{|i-j{|}^{1/3}})\mathit{B}$, ∑₃=0.8∑₁, and ∑₄=1.2∑₂, where B is defined in Section 2.2. Then, Conditions 1 and 5 hold. We first considered the case when k=3:Π_i,i=1,2,3, having (ε₁,ε₂,ε₃)=(1/2,1/4,1/4). We set n=20 and (n₁,n₂,n₃)=(10,5,5). From Theorem 2, one can expect that $(z_{1j},z_{2j})(=(s_{1j}/{\lambda }_1^{1/2},s_{2j}/{\lambda }_2^{1/2}))$ becomes close to (1,0) when x_j∈ Π₁, (−1,2^1/2) when x_j∈ Π₂, and (−1,−2^1/2) when x_j∈ Π₃. In Figure 3, we displayed scatter plots of (z_1j,z_2j), j=1,…,n, when (a) d=100, (b) d=1,000, and (c) d=10,000. We observed that the scatter plots appear close to those three vertices as d increases.

Next, we considered the case when k=4: Π_i,i=1,…,4, having ε₁=···=ε₄=1/4. We set n=20 and n₁=···=n₄=5. In Figure 4, we displayed scatter plots of (z_1j,z_2j,z_3j), j=1,…,n, when (a) d=100, (b) d=1,000 and (c) d=10,000. From Theorem 2, we displayed the triangular pyramid given by Equation 6 with k=4. As expected theoretically, we observed that the scatter plots appear close to four vertices of the triangular pyramid as d increases. They seemed to converge slower in Figure 4 than in Figure 3. This is because the conditions of Theorem 2 become strict as k increases. See Remark 7.

3.2 Consistency property of PC scores when k ≥ 3

Let

$${\eta }_{(0)}=0\text{and}{\eta }_{(i)}={\displaystyle \sum _{j=1}^i}{\eta }_j\text{for}i=1,\dots ,k.$$

We assume the following condition:

Condition 6. $\frac{{\max }_{i=1,\dots ,k-2}({\mathit{\mu }}_i^{\mathrm{T}}{\mathbf{\sum }}_j{\mathit{\mu }}_i)}{{\mathrm{\Delta }}_{k-1}^2}\to 0$ as d→∞ for j=1,…,k.

Remark 9.From the fact that ${\mathit{\mu }}_i^{\mathrm{T}}{\mathbf{\sum }}_j{\mathit{\mu }}_i\le {\mathrm{\Delta }}_i{\lambda }_{\max }({\mathbf{\sum }}_j)$, Condition 6 holds under ${\mathrm{\Delta }}_1{\lambda }_{\max }({\mathbf{\sum }}_j)/{\mathrm{\Delta }}_{k-1}^2\to 0$ as d→∞ for j=1,…,k.

As for the estimated PC scores, we have the following result. From Theorem 3, one can cluster x_js into k groups by the elements of ${\widehat{\mathit{u}}}_i,i=1,\dots ,k-1$:

Theorem 3.Under Conditions 2, it holds that for n_l>0, l=1,…,k

$$\begin{array}{ll}\underset{d\to \infty }{\mathrm{plim}}{\widehat{z}}_{ij}=\left\{\begin{array}{ll}0& \mathrm{when}i\ge 2\mathrm{and}{\mathit{x}}_j\in {\displaystyle \underset{m=1}{\overset{i-1}{\cup }}}{\mathrm{\Pi }}_m,\\ {\left(\frac{1-{\eta }_{(i)}}{{\eta }_i(1-{\eta }_{(i-1)})}\right)}^{1/2}& \mathrm{when}{\mathit{x}}_j\in {\mathrm{\Pi }}_i,\\ -{\left(\frac{{\eta }_i}{(1-{\eta }_{(i)})(1-{\eta }_{(i-1)})}\right)}^{1/2}& \mathrm{when}{\mathit{x}}_j\in {\displaystyle \underset{m=i+1}{\overset{k}{\cup }}}{\mathrm{\Pi }}_m,\end{array}\right.& \end{array}$$ (7)

for i=1,…,k−1;j=1,…,n.

Condition 5 is essential for the consistency properties given in Theorems 2 and 3. We investigated the robustness of Theorem 3 against Condition 5 by some simulation studies. We considered a toy example such as Π_i:N_d(μ_i,∑_i),i=1,2,3, where μ₁=1_d, μ₂=(1,…,1,0,…,0)^T whose first ⌈d/ζ⌉ elements are 1, μ₃=0, ${\mathbf{\sum }}_1=(0.3^{|i-j{|}^{1/3}})$, ${\mathbf{\sum }}_2=\mathit{B}(0.3^{|i-j{|}^{1/3}})\mathit{B}$, and ${\mathbf{\sum }}_3=(0.4^{|i-j{|}^{1/3}})$. We set d=5,000, n=20, and (n₁,n₂,n₃)=(10,5,5). In Figure 5, we displayed scatter plots of $({\widehat{z}}_{1j},{\widehat{z}}_{2j})$, j=1,…,n, when (a) ζ=1/5, (b) ζ=2/5, and (c) ζ=4/5. Also, we displayed the triangle given by Equation 7 with k=3. Note that Angle(μ₁,μ₂)=0.352π and Δ₂/Δ₁=1/5 for (a), Angle(μ₁,μ₂)=0.282π and Δ₂/Δ₁=2/5 for (b), and Angle(μ₁,μ₂)=0.148π and Δ₂/Δ₁=4/5 for (c). For (a) and (b), we observed that the estimated PC scores give good performances. On the other hand, the estimated PC scores seemed not to converge to the theoretical points for (c). This is because Condition 5 is not met. However, we could find three separate clusters for Π_i,i=1,2,3. See Appendix B for the reason.

Figure 5

Toy example to illustrate asymptotic behaviors of the estimated principal component scores when k=3. We plotted $({\widehat{z}}_{1j},{\widehat{z}}_{2j})$ which is denoted by small circles when x_j∈ Π₁, by small triangles when x_j∈ Π₂, and by small squares when x_j∈ Π₃. The dashed triangle consists of three vertices, namely, (1,0), (−1,2^1/2), and (−1,−2^1/2), which are the theoretical convergent points [Color figure can be viewed at wileyonlinelibrary.com]

4 REAL‐DATA EXAMPLES

We demonstrate the performance of clustering, based on sample PC scores, using gene expression datasets.

4.1 Clustering when k = 2

We analyzed microarray data by Chiaretti et al. (2004) in which the dataset consists of 12,625 (=d) genes and 128 samples. The dataset has two tumor cellular subtypes, Π₁: B cell (95 samples) and Π₂: T cell (33 samples). Refer to Jeffery, Higgins, and Culhane (2006) as well. We checked behaviors of the PC scores using several samples from the two tumor cellular subtypes. We considered three cases: (a) n=10 samples consist of the first five samples from both Π₁ and Π₂ (i.e., n₁=5 and n₂=5), (b) n=40 samples consist of the first 20 samples from both Π₁ and Π₂ (i.e., n₁=20 and n₂=20), and (c) n=128 samples consist of n₁=95 samples from Π₁ and n₂=33 samples from Π₂. In the top panels of Figure 6, we displayed scatter plots of the first two PC scores, $({\widehat{z}}_{1j},{\widehat{z}}_{2j})$s, for (a), (b), and (c). From Corollary 1, we denoted (η₂/η₁)^1/2 and −(η₁/η₂)^1/2 by dotted lines. For (a), we observed that the estimated PC scores give good performances. The first PC scores gathered around (η₂/η₁)^1/2 or −(η₁/η₂)^1/2. For (b), the estimated PC scores gave adequate performances except for the two points from Π₂. Those two samples, which are the ninth and twentieth samples of Π₂, are probably outliers. In fact, the two points are far from the cluster of Π₂. The other 38 samples were perfectly classified into the two groups by the sign of the first PC scores. As for (c), although there seemed to be two clusters except for the two samples, we could not classify the dataset by the sign of the first PC scores. This is probably because η₁ and η₂ are unbalanced. From Equation 2, when the mixing proportions are unbalanced, λ₁ becomes small. The first eigenspace was possibly affected by the other eigenspaces, so that the first PC scores appear in the wrong direction. We tested the clustering except for the outlying two samples. We used the remaining 31 samples for Π₂. We considered the following three cases for samples from Π₁: (d) the first 16 samples from Π₁, so that n₁=16,n₂=31,n=47, and η₁/η₂≈0.5; (e) the first 31 samples from Π₁, so that n₁=31,n₂=31,n=62, and η₁/η₂=1; and (f) the first 62 samples from Π₁, so that n₁=62,n₂=31,n=93, and η₁/η₂=2. In the bottom panels of Figure 6, we displayed scatter plots of $({\widehat{z}}_{1j},{\widehat{z}}_{2j})$s for (d), (e), and (f). For (d) and (e), we observed that the estimated PC scores give good performances. As for (f), although there seemed to be two clusters, we could not classify the dataset by the sign of the first PC scores. Note that η₁ and η₂ are unbalanced in (d) and (f). Even though (d) is an unbalanced case, the estimated PC scores worked well for the case. We had an estimate for the ratio of the largest eigenvalues, ${\lambda }_{\max }({\mathbf{\sum }}_1)/{\lambda }_{\max }({\mathbf{\sum }}_2)$, as 1.598 by the noise‐reduction methodology given by Yata and Aoshima (2012). The first eigenspace of ∑ in (d) is less affected by the first eigenspace of ∑_is than in (f) as $\mathbf{\sum }={\epsilon }_1{\epsilon }_2({\mathit{\mu }}_1-{\mathit{\mu }}_2){({\mathit{\mu }}_1-{\mathit{\mu }}_2)}^{\mathrm{T}}+{\epsilon }_1{\mathbf{\sum }}_1+{\epsilon }_2{\mathbf{\sum }}_2$. This is probably the reason why the estimated PC scores gave good performances even in (d).

Figure 6

Scatter plots of the first two principal component scores, supposing k=2 in the dataset of Chiaretti et al. (2004). We denoted them by small circles when x_j∈ Π₁ and by small triangles when x_j∈ Π₂. The theoretical convergent points, namely, (η₂/η₁)^1/2 and −(η₁/η₂)^1/2, are denoted by dotted lines. The two samples, encircled by dots in (b) and (c), are probably outliers [Color figure can be viewed at wileyonlinelibrary.com]

4.2 Clustering when k ≥ 3

We analyzed microarray data by Bhattacharjee et al. (2001) in which the dataset consisted of five lung carcinomas types with d=3,312. We only used four classes as Π₁: pulmonary carcinoids (20 samples), Π₂: normal lung (17 samples), Π₃: squamous cell lung carcinomas (21 samples), and Π₄: adenocarcinomas (20 samples), so that n₁=20,n₂=17,n₃=21, and n₄=20. Note that Π₄ originally had 139 samples. We used only the first 20 samples from Π₄ in order to keep balance in sample sizes with the other classes. We first considered clustering when k=3 under the following setups: (a) the dataset consists of Π₁, Π₂, and Π₃ (n=58); (b) the dataset consists of Π₁, Π₂, and Π₄ (n=57); and (c) the dataset consists of Π₁, Π₃, and Π₄ (n=61). In Figure 7, we displayed scatter plots of the first two PC scores, $({\widehat{z}}_{1j},{\widehat{z}}_{2j})$s, for each of (a), (b), and (c). Also, we displayed the triangle given by Equation 7 with k=3 using Theorem 3. We observed that the estimated PC scores give good performances. The three clusters gathered around each vertex for (a), (b), and (c).

Figure 7

Scatter plots of the first two principal component scores, supposing k=3 in the dataset of Bhattacharjee et al. (2001). We denoted them by small circles when x_j∈ Π₁, by small triangles when x_j∈ Π₂, by small squares when x_j∈ Π₃, and by small inverted triangles when x_j∈ Π₄. The theoretical convergent points are denoted by the vertices of the triangle [Color figure can be viewed at wileyonlinelibrary.com]

Next, we considered clustering when k=4: Π_i,i=1,…,4, so that n=78. In Figure 8, we displayed scatter plots of the first three PC scores. The dataset seemed not to converge to the theoretical convergent points given by Equation 7 in Theorem 3. This is probably because the conditions of Theorem 3 become strict as k increases. See Remark 7. Thus, the convergence is slower than in the case when k=3 as in Figure 7. However, there seemed to be four separate clusters of each Π_i.

Finally, we introduce an example using next generation sequencing datasets. Shen, Shen, Zhu, and Marron (2012, 2016) gave a scatter plot of first two PC scores for the next generation sequencing cancer data by Wilhelm and Landry (2009) in which the dataset consists of three curves with d=1,709 and n=180. See Figure 9 which was given in figure 1 of Shen et al. (2012). The three clusters seem to compose of a triangle such as Figure 7.

Figure 8

Scatter plots of the first three principal component scores, supposing k=4 in the dataset of Bhattacharjee et al. (2001). The dashed triangles and triangular pyramid were given by Equation 7 with k=4 [Color figure can be viewed at wileyonlinelibrary.com]

Figure 9

Shen et al. (2012) gave a scatter plot of first two principal component scores for the next generation sequencing cancer data. [Color figure can be viewed at wileyonlinelibrary.com]

4.3 Clustering: Special case

We analyzed microarray data by Armstrong et al. (2002) in which the dataset consists of three leukemia subtypes having 12,582 (=d) genes. We used two classes such as Π₁: acute lymphoblastic leukemia (24 samples) and Π₂: mixed‐lineage leukemia (20 samples), so that n₁=24,n₂=20, and n=44. In Figure 10, we displayed scatter plots of the first three PC scores.

Figure 10

Scatter plots of the first three principal component scores, supposing k=2 in the dataset of Armstrong et al. (2002) [Color figure can be viewed at wileyonlinelibrary.com]

We observed that the dataset is perfectly separated by the sign of the second PC scores. This figure looks completely different from Figure 6. This is probably because the largest eigenvalue, ${\lambda }_{\max }({\mathbf{\sum }}_1)$ or ${\lambda }_{\max }({\mathbf{\sum }}_2)$, is too large. When k=2, we give the following result to explain the reason of the phenomenon in Figure 10. Under the assumptions of Proposition 2, one can cluster x_js into two groups by some ith PC score even when Condition 1 is not met:

Proposition 2.Assume that ${\max }_{i=1,2}({\mathit{\mu }}_1^{\mathrm{T}}{\mathbf{\sum }}_i{\mathit{\mu }}_1)/{\mathrm{\Delta }}_1^2\to 0$ as d→∞. Then, there exists some positive integer i_⋆ such that

$$\frac{{\lambda }_{i_{\star }}}{{\epsilon }_1{\epsilon }_2{\mathrm{\Delta }}_1}\to 1\text{as}d\to \infty .$$

Furthermore, assume that ${\lambda }_{i_{\star }}$ is distinct in the sense that

$$\underset{d\to \infty }{\lim \inf }\left|\frac{{\lambda }_{i^{\prime }}}{{\lambda }_{i_{\star }}}-1\right|>0\text{for}{i}^{\prime }=1,\dots ,d(i^{\prime }\ne i_{\star }).$$

Then, if ${\mathit{h}}_{i_{\star }}^{\mathrm{T}}{\mathit{\mu }}_1\ge 0$, it holds that Angle $({\mathit{h}}_{i_{\star }},{\mathit{\mu }}_1)\to 0$ as d→∞ and for j=1,…,n

$$\underset{d\to \infty }{\mathrm{plim}}\frac{s_{i_{\star }j}}{{\lambda }_{i_{\star }}^{1/2}}=\left\{\begin{array}{ll}{({\epsilon }_2/{\epsilon }_1)}^{1/2}& \mathrm{when}{\mathit{x}}_j\in {\mathrm{\Pi }}_1,\\ -{({\epsilon }_1/{\epsilon }_2)}^{1/2}& \mathrm{when}{\mathit{x}}_j\in {\mathrm{\Pi }}_2.\end{array}\right.$$ (8)

We estimated the largest eigenvalue using the noise‐reduction methodology given by Yata and Aoshima (2012). By noting Remark 1, we considered Δ₁ as ${\mathrm{\Delta }}_1=||{\mathit{\mu }}_1^{\prime }|{|}^2=||{\mathit{\mu }}_1-{\mathit{\mu }}_2|{|}^2$. We estimated Δ₁ using an unbiased estimator given by Aoshima and Yata (2014). Then, we obtained the estimates of $({\lambda }_{\max }({\mathbf{\sum }}_1)/{\mathrm{\Delta }}_1,{\lambda }_{\max }({\mathbf{\sum }}_2)/{\mathrm{\Delta }}_1)$ as (0.465,0.787), so that Condition 1 is not met obviously. In addition, by estimating ε_is by η_is, we had ${\epsilon }_2{\lambda }_{\max }({\mathbf{\sum }}_2)>{\epsilon }_1{\epsilon }_2{\mathrm{\Delta }}_1$. Thus, the first eigenspace of ∑ is probably the first eigenspace of ∑₂ as $\mathbf{\sum }={\epsilon }_1{\epsilon }_2({\mathit{\mu }}_1-{\mathit{\mu }}_2){({\mathit{\mu }}_1-{\mathit{\mu }}_2)}^{\mathrm{T}}+{\epsilon }_1{\mathbf{\sum }}_1+{\epsilon }_2{\mathbf{\sum }}_2$. Thus, i_⋆ in Proposition 2 must be 2. This is the reason why the dataset can be separated by the sign of the second PC scores in Figure 10.

5 CONCLUDING REMARKS

In this article, we considered the mixture model by Equation 1 in high‐dimensional settings. We studied asymptotic properties of both the true PC scores and the sample PC scores for the high‐dimensional mixture model. We gave conditions under which PCA is very effective for clustering HDLSS data. We showed that HDLSS data can be classified by the sign of the first several PC scores theoretically. However, we have to say, in actual HDLSS data analyses, one may encounter cases such as in Figures 6c and 10, where the dataset is not always classified by the sign of the first several PC scores. Several reasons should be considered: (i) actual HDLSS datasets often include several outliers, (ii) the regularity conditions are not met, and (iii) the mixing proportions ε_is are quite unbalanced. Thus, we recommend the following three steps: (i) apply PCA to HDLSS data; (ii) using PC scores, map the dataset onto a feature space such as the first three eigenspaces, and (iii) apply general clustering methods such as the k‐means method to the feature space. However, the number of clusters k is unknown in general. We emphasize that the first k−1 eigenvalues are quite spiked for the model 1. Recently, Jung, Lee, and Ahn (2018) proposed a test of the number of spiked components for high‐dimensional data. Thus, one may apply the test to the choice of k for clustering.

ACKNOWLEDGEMENTS

We would like to thank the two anonymous referees for their constructive comments.

Appendix A: Lemmas and their proofs

Throughout, let μ_i,j=μ_i−μ_j and Δ_i,j=||μ_i,j||² for i,j=1,…,k(i<j). Let u_i=(u_i1,…,u_in)^T, where

$$u_{ij}=\left\{\begin{array}{ll}0& \text{when}i\ge 2\text{and}{\mathit{x}}_j\in {\displaystyle \underset{m=1}{\overset{i-1}{\cup }}}{\mathrm{\Pi }}_m,\\ {[(1-{\eta }_{(i)})/\{n{\eta }_i(1-{\eta }_{(i-1)})\}]}^{1/2}& \text{when}{\mathit{x}}_j\in {\mathrm{\Pi }}_i,\\ {-[{\eta }_i/\{n(1-{\eta }_{(i)})(1-{\eta }_{(i-1)})\}]}^{1/2}& \text{when}{\mathit{x}}_j\in {\displaystyle \underset{m=i+1}{\overset{k}{\cup }}}{\mathrm{\Pi }}_m,\end{array}\right.$$

for i=1,…,k−1;j=1,…,n. Let ${\mathit{\nu }}_i={\sum }_{m=1}^k{\eta }_m({\mathit{\mu }}_i-{\mathit{\mu }}_m)$ for i=1,…,k. Let V=[ν₍₁₎,…,ν_(n)], where ν_(j)=ν_i according to x_j∈ Π_i for j=1,…,n. Note that $\mathit{V}{\mathbf{1}}_n={\sum }_{j=1}^n{\mathit{\nu }}_{(j)}=\mathbf{0}$. We define the eigendecomposition of V^TV/n by ${\mathit{V}}^{\mathrm{T}}\mathit{V}/n={\sum }_{i=1}^{k-1}{\tilde{\lambda }}_i{\tilde{\mathit{u}}}_i{\tilde{\mathit{u}}}_i^{\mathrm{T}}$ from the fact that rank(V)≤k−1, where ${\tilde{\lambda }}_1\ge \mathrm{···}\ge {\tilde{\lambda }}_{k-1}\ge 0$ are eigenvalues of V^TV/n and ${\tilde{\mathit{u}}}_i={({ũ}_{i1},\dots ,{ũ}_{in})}^{\mathrm{T}}$ is a unit eigenvector corresponding to ${\tilde{\lambda }}_i$ for each i. We assume ${\tilde{\mathit{u}}}_i^{\mathrm{T}}{\mathit{u}}_i\ge 0$ for i=1,…,k−1, without loss of generality.

Lemma 1.When k=2, it holds that under Conditions 2

$$\underset{d\to \infty }{\mathrm{plim}}\frac{(n-1){\mathit{S}}_{\mathrm{D}}-\mathrm{tr}({\mathbf{\sum }}_1){\mathit{P}}_n}{{\mathrm{\Delta }}_1}=\mathit{r}{\mathit{r}}^{\mathrm{T}}.$$

Proof.As μ₂=0, we can write that ${\mathit{x}}_j-{\eta }_1{\mathit{\mu }}_1=({\mathit{x}}_j-{\mathit{\mu }}_i)+{(-1)}^{i+1}(1-{\eta }_i){\mathit{\mu }}_1$ for i=1,2;j=1,…,n. From the fact that ${\lambda }_{\max }({\mathbf{\sum }}_i)\le \text{tr}{({\mathbf{\sum }}_i^2)}^{1/2}$, we have that $\mathrm{Var}\{{({\mathit{x}}_j-{\mathit{\mu }}_i)}^{\mathrm{T}}{\mathit{\mu }}_1|{\mathit{x}}_j\in {\mathrm{\Pi }}_i\}={\mathit{\mu }}_1^{\mathrm{T}}{\mathbf{\sum }}_i{\mathit{\mu }}_1\le {\mathrm{\Delta }}_1{\lambda }_{\max }({\mathbf{\sum }}_i)=o({\mathrm{\Delta }}_1^2)$ as d→∞ for j=1,…,n;i=1,2 under Condition 2. Also, we have that $\mathrm{Var}\{{({\mathit{x}}_j-{\mathit{\mu }}_i)}^{\mathrm{T}}({\mathit{x}}_{j^{\prime }}-{\mathit{\mu }}_{i^{\prime }})|{\mathit{x}}_j\in {\mathrm{\Pi }}_i,{\mathit{x}}_{j^{\prime }}\in {\mathrm{\Pi }}_{i^{\prime }}\}=\text{tr}({\mathbf{\sum }}_i{\mathbf{\sum }}_{i^{\prime }})\le \text{tr}{({\mathbf{\sum }}_i^2)}^{1/2}\text{tr}{({\mathbf{\sum }}_{i^{\prime }}^2)}^{1/2}=o({\mathrm{\Delta }}_1^2)$ for all j≠j′ and i,i′ =1,2 under Condition 2. Then, using Chebyshev's inequality, for any τ>0, under Condition 2, it holds that for all j≠j′ and i,i′ =1,2

$$\begin{array}{ll}& P\{|{({\mathit{x}}_j-{\mathit{\mu }}_i)}^{\mathrm{T}}({\mathit{x}}_{j^{\prime }}-{\mathit{\mu }}_{i^{\prime }})/{\mathrm{\Delta }}_1|>\tau |{\mathit{x}}_j\in {\mathrm{\Pi }}_i,{\mathit{x}}_{j^{\prime }}\in {\mathrm{\Pi }}_{i^{\prime }}\}=o(1)\text{and}\\ & P\{|{({\mathit{x}}_j-{\mathit{\mu }}_i)}^{\mathrm{T}}{\mathit{\mu }}_1/{\mathrm{\Delta }}_1|>\tau |{\mathit{x}}_j\in {\mathrm{\Pi }}_i\}=o(1),\end{array}$$ (A1)

so that ${({\mathit{x}}_j-{\mathit{\mu }}_i)}^{\mathrm{T}}({\mathit{x}}_{j^{\prime }}-{\mathit{\mu }}_{i^{\prime }})/{\mathrm{\Delta }}_1=o_P(1)$ and ${({\mathit{x}}_j-{\mathit{\mu }}_i)}^{\mathrm{T}}{\mathit{\mu }}_1/{\mathrm{\Delta }}_1=o_P(1)$ when x_j∈ Π_i and ${\mathit{x}}_{j^{\prime }}\in {\mathrm{\Pi }}_{i^{\prime }}$ (j≠j^′). We note that $E(||{\mathit{x}}_j-{\mathit{\mu }}_i|{|}^2|{\mathit{x}}_j\in {\mathrm{\Pi }}_i)=\text{tr}({\mathbf{\sum }}_i)$. Similar to A1, under Condition 3, it holds that $\{||{\mathit{x}}_j-{\mathit{\mu }}_i|{|}^2-\text{tr}({\mathbf{\sum }}_i)\}/{\mathrm{\Delta }}_1=o_P(1)$ when x_j∈ Π_i for i=1,2;j=1,…,n. By noting that {tr(∑₁)−tr(∑₂)}/Δ₁=o(1) under Condition 4, we have that

$$\underset{d\to \infty }{\mathrm{plim}}\frac{{(\mathit{X}-{\eta }_1{\mathit{\mu }}_1{\mathbf{1}}_n^{\mathrm{T}})}^{\mathrm{T}}(\mathit{X}-{\eta }_1{\mathit{\mu }}_1{\mathbf{1}}_n^{\mathrm{T}})-\text{tr}({\mathbf{\sum }}_1){\mathit{I}}_n}{{\mathrm{\Delta }}_1}=\mathit{r}{\mathit{r}}^{\mathrm{T}},$$

under Conditions 1. By noting that ${\mathit{P}}_n{(\mathit{X}-{\eta }_1{\mathit{\mu }}_1{\mathbf{1}}_n^{\mathrm{T}})}^{\mathrm{T}}(\mathit{X}-{\eta }_1{\mathit{\mu }}_1{\mathbf{1}}_n^{\mathrm{T}}){\mathit{P}}_n/(n-1)={\mathit{S}}_{\mathrm{D}}$ and ${\mathit{r}}^{\mathrm{T}}{\mathit{P}}_n={\mathit{r}}^{\mathrm{T}}$ from ${\mathit{r}}^{\mathrm{T}}{\mathbf{1}}_n=0$, we conclude the result.

Lemma 2.Let ${\stackrel{\text{'}}{\mathit{\mu }}}_{i,i+1}={\mathit{\mu }}_{i,i+1}/{\mathrm{\Delta }}_{i,i+1}^{1/2}$ for i=1,…,k−1 and let Δ_(i,j)= Δ_j,j+1/Δ_i,i+1 for i,j=1,…,k−1(i<j). Under Conditions 1 and 5, it holds that as d→∞

$$\begin{array}{ll}& \frac{{\lambda }_i}{{\mathrm{\Delta }}_{i,i+1}}=\frac{{\epsilon }_i(1-{\epsilon }_{(i)})}{1-{\epsilon }_{(i-1)}}+o(1)\mathrm{and}{\mathit{h}}_i^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{i,i+1}=1+o(1)\mathrm{for}i=1,\dots ,k-1;\\ & {\mathit{h}}_i^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{i-1,i}=-\frac{1-{\epsilon }_{(i)}}{1-{\epsilon }_{(i-1)}}{\mathrm{\Delta }}_{(i-1,i)}^{1/2}\{1+o(1)\}\mathrm{for}i=2,\dots ,k-1\mathrm{when}k\ge 3;and\\ {\mathit{h}}_j^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{i,i+1}& =o({\mathrm{\Delta }}_{(i,j)}^{1/2})\text{for}i,j=1,\dots ,k-1(i+1<j)\mathrm{when}k\ge 3.\end{array}$$

Proof.From the fact that $|{\mathit{\mu }}_i^{\mathrm{T}}{\mathit{\mu }}_j|\le {({\mathrm{\Delta }}_i{\mathrm{\Delta }}_j)}^{1/2}$, under Condition 5 it holds that as d→∞

$$\frac{{\mathrm{\Delta }}_{i,i+1}}{{\mathrm{\Delta }}_i}=\frac{{\mathrm{\Delta }}_i+{\mathrm{\Delta }}_{i+1}+O\{{({\mathrm{\Delta }}_i{\mathrm{\Delta }}_{i+1})}^{1/2}\}}{{\mathrm{\Delta }}_i}\to 1\text{for}i=1,\dots ,k-2.$$

Then, under Condition 5, it holds that

$$\frac{{\mathit{\mu }}_{i,i+1}^{\mathrm{T}}{\mathit{\mu }}_{j,j+1}}{{({\mathrm{\Delta }}_{i,i+1}{\mathrm{\Delta }}_{j,j+1})}^{1/2}}=\frac{{\mathit{\mu }}_i^{\mathrm{T}}{\mathit{\mu }}_j+O\{{({\mathrm{\Delta }}_i{\mathrm{\Delta }}_{j+1})}^{1/2}+{({\mathrm{\Delta }}_{i+1}{\mathrm{\Delta }}_j)}^{1/2}\}}{{({\mathrm{\Delta }}_i{\mathrm{\Delta }}_j)}^{1/2}\{1+o(1)\}}=o(1),$$

for i,j=1,…,k−1(i<j). Hence, under Condition 5, we claim that

$${\stackrel{\text{'}}{\mathit{\mu }}}_{i,i+1}^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{j,j+1}\to 0\text{and}\frac{{\mathrm{\Delta }}_{j,j+1}}{{\mathrm{\Delta }}_{i,i+1}}\to 0\text{as}d\to \infty \text{for}i,j=1,\dots ,k-1(i<j).$$ (A2)

Let ${\mathit{e}}_d(\in {\mathbb{R}}^d)$ be an arbitrary unit vector. From ${\mathit{e}}_d^{\mathrm{T}}\left({\sum }_{i=1}^k{\epsilon }_i{\mathbf{\sum }}_i\right){\mathit{e}}_d\le {\sum }_{i=1}^k{\lambda }_{\max }({\mathbf{\sum }}_i)$, it holds that

$$\frac{{\mathit{e}}_d^{\mathrm{T}}\mathbf{\sum }{\mathit{e}}_d}{{\mathrm{\Delta }}_{k-1}}=\frac{{\mathit{e}}_d^{\mathrm{T}}\left({\sum }_{i=1}^{k-1}{\sum }_{j=i+1}^k{\epsilon }_i{\epsilon }_j{\mathit{\mu }}_{i,j}{\mathit{\mu }}_{i,j}^{\mathrm{T}}\right){\mathit{e}}_d}{{\mathrm{\Delta }}_{k-1}}+o(1),$$ (A3)

under Condition 1. Note that ${\mathit{\mu }}_{i,j}={\sum }_{m=i}^{j-1}{\mathit{\mu }}_{m,m+1}$ for i,j=1,…,k(i<j). Thus, it holds that

$$\begin{array}{ll}{\displaystyle \sum _{i=1}^{k-1}}{\displaystyle \sum _{j=i+1}^k}{\epsilon }_i{\epsilon }_j{\mathit{\mu }}_{i,j}{\mathit{\mu }}_{i,j}^{\mathrm{T}}=& {\displaystyle \sum _{i=1}^{k-1}}{\epsilon }_{(i)}(1-{\epsilon }_{(i)}){\mathit{\mu }}_{i,i+1}{\mathit{\mu }}_{i,i+1}^{\mathrm{T}}+{\displaystyle \sum _{i=1}^{k-2}}{\displaystyle \sum _{j=i+1}^{k-1}}{\epsilon }_{(i)}(1-{\epsilon }_{(j)})({\mathit{\mu }}_{i,i+1}{\mathit{\mu }}_{j,j+1}^{\mathrm{T}}+{\mathit{\mu }}_{j,j+1}{\mathit{\mu }}_{i,i+1}^{\mathrm{T}}).\end{array}$$ (A4)

From the facts that ${\lambda }_1={\mathit{h}}_1^{\mathrm{T}}\mathbf{\sum }{\mathit{h}}_1={\max }_{{\mathit{e}}_d}({\mathit{e}}_d^{\mathrm{T}}\mathbf{\sum }{\mathit{e}}_d)$ and Δ_k−1= Δ_k−1,k, by combining Equation A3 with Equations A2 and A4, we have that

$$\frac{{\lambda }_1}{{\mathrm{\Delta }}_{1,2}}=\underset{{\mathit{e}}_d}{\max }\left\{{\epsilon }_{(1)}(1-{\epsilon }_{(1)}){({\mathit{e}}_d^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{1,2})}^2+o(1)\right\}={\epsilon }_{(1)}(1-{\epsilon }_{(1)})+o(1),$$

under Conditions 1 and 5. Hence, by noting that Δ_1,2/Δ₁=1+o(1) and ${\mathit{h}}_1^{\mathrm{T}}{\mathit{\mu }}_1\ge 0$, it holds that ${\mathit{h}}_1^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{1,2}={\mathit{h}}_1^{\mathrm{T}}{\mathit{\mu }}_1/{\mathrm{\Delta }}_1^{1/2}+o(1)=1+o(1)$.

Next, we consider λ₂ and h₂. From A2, we note that ${\stackrel{\text{'}}{\mathit{\mu }}}_{i,i+1}^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{j,j+1}=o(1)$ and Δ_(i,j)=o(1) for i,j=1,…,k−1(i<j) under Condition 5. Then, under Conditions 1 and 5, it holds that for j≥2

$$\begin{array}{lcr}0=\frac{{\mathit{h}}_1^{\mathrm{T}}\mathbf{\sum }{\mathit{h}}_j}{{\mathrm{\Delta }}_{1,2}}={\epsilon }_{(1)}(1-{\epsilon }_{(1)})\{1+o(1)\}{\stackrel{\text{'}}{\mathit{\mu }}}_{1,2}^{\mathrm{T}}{\mathit{h}}_j+{\epsilon }_{(1)}(1-{\epsilon }_{(2)}){\stackrel{\text{'}}{\mathit{\mu }}}_{2,3}^{\mathrm{T}}{\mathit{h}}_j{\mathrm{\Delta }}_{(1,2)}^{1/2}+o({\mathrm{\Delta }}_{(1,2)}^{1/2}),& & \end{array}$$

from Equations A3 and A4 and ${\mathit{h}}_1^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{2,3}=o(1)$, so that for j≥2

$${\mathit{h}}_j^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{1,2}=-\{(1-{\epsilon }_{(2)})/(1-{\epsilon }_{(1)})\}{\stackrel{\text{'}}{\mathit{\mu }}}_{2,3}^{\mathrm{T}}{\mathit{h}}_j{\mathrm{\Delta }}_{(1,2)}^{1/2}+o({\mathrm{\Delta }}_{(1,2)}^{1/2}).$$ (A5)

By combining Equation A3 with Equations A4 and A5, we have that

$$\begin{array}{ll}& \frac{{\lambda }_2}{{\mathrm{\Delta }}_{2,3}}=\frac{{\mathit{h}}_2^{\mathrm{T}}\mathbf{\sum }{\mathit{h}}_2}{{\mathrm{\Delta }}_{2,3}}=\frac{{\mathit{h}}_2^{\mathrm{T}}\{{\sum }_{i=1}^2{\epsilon }_{(i)}(1-{\epsilon }_{(i)}){\mathit{\mu }}_{i,i+1}{\mathit{\mu }}_{i,i+1}^{\mathrm{T}}+2{\epsilon }_{(1)}(1-{\epsilon }_{(2)}){\mathit{\mu }}_{1,2}{\mathit{\mu }}_{2,3}^{\mathrm{T}}\}{\mathit{h}}_2}{{\mathrm{\Delta }}_{2,3}}+o(1)\\ & ={\epsilon }_{(2)}(1-{\epsilon }_{(2)}){({\stackrel{\text{'}}{\mathit{\mu }}}_{2,3}^{\mathrm{T}}{\mathit{h}}_2)}^2+{\epsilon }_{(1)}(1-{\epsilon }_{(1)})\frac{{({\stackrel{\text{'}}{\mathit{\mu }}}_{1,2}^{\mathrm{T}}{\mathit{h}}_2)}^2}{{\mathrm{\Delta }}_{(1,2)}}+2{\epsilon }_{(1)}(1-{\epsilon }_{(2)})\frac{({\stackrel{\text{'}}{\mathit{\mu }}}_{1,2}^{\mathrm{T}}{\mathit{h}}_2)({\stackrel{\text{'}}{\mathit{\mu }}}_{2,3}^{\mathrm{T}}{\mathit{h}}_2)}{{\mathrm{\Delta }}_{(1,2)}^{1/2}}+o(1)\\ & ={\epsilon }_{(2)}(1-{\epsilon }_{(2)})-\frac{{\epsilon }_{(1)}{(1-{\epsilon }_{(2)})}^2}{1-{\epsilon }_{(1)}}+o(1)=\frac{{\epsilon }_2(1-{\epsilon }_{(2)})}{1-{\epsilon }_{(1)}}+o(1),\end{array}$$ (A6)

under Conditions 1 and 5. Hence, from the assumption that ${\mathit{h}}_2^{\mathrm{T}}{\mathit{\mu }}_2\ge 0$, it holds that ${\mathit{h}}_2^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{2,3}={\mathit{h}}_2^{\mathrm{T}}{\mathit{\mu }}_2/{\mathrm{\Delta }}_2^{1/2}+o(1)=1+o(1)$.

Next, we consider λ₃ and h₃. Note that ${\mathit{h}}_j^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{2,3}=o(1)$ for j≥3 from ${\mathit{h}}_2^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{2,3}=1+o(1)$. Then, under Conditions 1 and 5, we have that for j≥3

$$\begin{array}{lc}0=\frac{{\mathit{h}}_1^{\mathrm{T}}\mathbf{\sum }{\mathit{h}}_j}{{\mathrm{\Delta }}_{1,2}}& ={\epsilon }_{(1)}(1-{\epsilon }_{(1)})\{1+o(1)\}{\stackrel{\text{'}}{\mathit{\mu }}}_{1,2}^{\mathrm{T}}{\mathit{h}}_j+{\epsilon }_{(1)}(1-{\epsilon }_{(2)})\{1+o(1)\}{\stackrel{\text{'}}{\mathit{\mu }}}_{2,3}^{\mathrm{T}}{\mathit{h}}_j{\mathrm{\Delta }}_{(1,2)}^{1/2}\\ & +{\epsilon }_{(1)}(1-{\epsilon }_{(3)}){\stackrel{\text{'}}{\mathit{\mu }}}_{3,4}^{\mathrm{T}}{\mathit{h}}_j{\mathrm{\Delta }}_{(1,3)}^{1/2}+o({\mathrm{\Delta }}_{(1,3)}^{1/2})\text{and}\end{array}$$ (A7)

$$\begin{array}{lc}0=\frac{{\mathit{h}}_2^{\mathrm{T}}\mathbf{\sum }{\mathit{h}}_j}{{\mathrm{\Delta }}_{2,3}}& ={\epsilon }_{(1)}(1-{\epsilon }_{(1)})\frac{{\mathit{h}}_2^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{1,2}{\stackrel{\text{'}}{\mathit{\mu }}}_{1,2}^{\mathrm{T}}{\mathit{h}}_j}{{\mathrm{\Delta }}_{(1,2)}}+{\epsilon }_{(1)}(1-{\epsilon }_{(2)})\frac{{\mathit{h}}_2^{\mathrm{T}}({\stackrel{\text{'}}{\mathit{\mu }}}_{1,2}{\stackrel{\text{'}}{\mathit{\mu }}}_{2,3}^{\mathrm{T}}+{\stackrel{\text{'}}{\mathit{\mu }}}_{2,3}{\stackrel{\text{'}}{\mathit{\mu }}}_{1,2}^{\mathrm{T}}){\mathit{h}}_j}{{\mathrm{\Delta }}_{(1,2)}^{1/2}}\\ & +{\epsilon }_{(1)}(1-{\epsilon }_{(3)})\frac{{\mathit{h}}_2^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{1,2}{\stackrel{\text{'}}{\mathit{\mu }}}_{3,4}^{\mathrm{T}}{\mathit{h}}_j}{{\mathrm{\Delta }}_{(1,2)}^{1/2}}{\mathrm{\Delta }}_{(2,3)}^{1/2}+{\epsilon }_{(2)}(1-{\epsilon }_{(2)})\{1+o(1)\}{\stackrel{\text{'}}{\mathit{\mu }}}_{2,3}^{\mathrm{T}}{\mathit{h}}_j\\ & +{\epsilon }_{(2)}(1-{\epsilon }_{(3)}){\stackrel{\text{'}}{\mathit{\mu }}}_{3,4}^{\mathrm{T}}{\mathit{h}}_j{\mathrm{\Delta }}_{(2,3)}^{1/2}+o({\mathrm{\Delta }}_{(2,3)}^{1/2})\\ & =\frac{{\epsilon }_2(1-{\epsilon }_{(2)})}{1-{\epsilon }_{(1)}}\{1+o(1)\}{\stackrel{\text{'}}{\mathit{\mu }}}_{2,3}^{\mathrm{T}}{\mathit{h}}_j+\frac{{\epsilon }_2(1-{\epsilon }_{(3)})}{1-{\epsilon }_{(1)}}{\stackrel{\text{'}}{\mathit{\mu }}}_{3,4}^{\mathrm{T}}{\mathit{h}}_j{\mathrm{\Delta }}_{(2,3)}^{1/2}+o({\mathrm{\Delta }}_{(2,3)}^{1/2})+{\stackrel{\text{'}}{\mathit{\mu }}}_{1,2}^{\mathrm{T}}{\mathit{h}}_j\times o({\mathrm{\Delta }}_{(1,2)}^{-1/2}),\end{array}$$ (A8)

from Equations A2 to A5, ${\mathit{h}}_1^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{2,3}=o(1)$, ${\mathit{h}}_1^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{3,4}=o(1)$, and ${\mathit{h}}_2^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{3,4}=o(1)$. Then, by combining Equations A7 and A8, under Conditions 1 and 5, it holds that for j≥3

$$\begin{array}{ll}{\mathit{h}}_j^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{1,2}=o({\mathrm{\Delta }}_{(1,3)}^{1/2})\text{and}{\mathit{h}}_j^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{2,3}=-\frac{1-{\epsilon }_{(3)}}{1-{\epsilon }_{(2)}}{\stackrel{\text{'}}{\mathit{\mu }}}_{3,4}^{\mathrm{T}}{\mathit{h}}_j{\mathrm{\Delta }}_{(2,3)}^{1/2}+o({\mathrm{\Delta }}_{(2,3)}^{1/2}).& \end{array}$$ (A9)

Similar to Equation A6, by combining Equation A3 with Equations A4 and A9, under Conditions 1 and 5, we have that

$$\begin{array}{ll}\frac{{\lambda }_3}{{\mathrm{\Delta }}_{3,4}}& ={\epsilon }_{(3)}(1-{\epsilon }_{(3)}){({\stackrel{\text{'}}{\mathit{\mu }}}_{3,4}^{\mathrm{T}}{\mathit{h}}_3)}^2+{\epsilon }_{(2)}(1-{\epsilon }_{(2)})\frac{{({\stackrel{\text{'}}{\mathit{\mu }}}_{2,3}^{\mathrm{T}}{\mathit{h}}_3)}^2}{{\mathrm{\Delta }}_{(2,3)}}+2{\epsilon }_{(2)}(1-{\epsilon }_{(3)})\frac{({\stackrel{\text{'}}{\mathit{\mu }}}_{2,3}^{\mathrm{T}}{\mathit{h}}_3)({\stackrel{\text{'}}{\mathit{\mu }}}_{3,4}^{\mathrm{T}}{\mathit{h}}_3)}{{\mathrm{\Delta }}_{(2,3)}^{1/2}}+o(1)\\ & ={\epsilon }_{(3)}(1-{\epsilon }_{(3)})-\frac{{\epsilon }_{(2)}{(1-{\epsilon }_{(3)})}^2}{1-{\epsilon }_{(2)}}+o(1)=\frac{{\epsilon }_3(1-{\epsilon }_{(3)})}{(1-{\epsilon }_{(2)})}+o(1),\end{array}$$

so that ${\mathit{h}}_3^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{3,4}=1+o(1)$ from the assumption that ${\mathit{h}}_3^{\mathrm{T}}{\mathit{\mu }}_3\ge 0$.

In a way similar to λ₃ and h₃, as for λ_i and h_i (4≤i≤k−1), we have that λ_i/Δ_i,i+1=ε_i(1−ε_(i))/(1−ε_(i−1))+o(1), ${\mathit{h}}_i^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{i,i+1}=1+o(1)$ and ${\mathit{h}}_i^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{i-1,i}=-\{(1-{\epsilon }_{(i)})/(1-{\epsilon }_{(i-1)})\}{\mathrm{\Delta }}_{(i-1,i)}^{1/2}\{1+o(1)\}$ together with ${\mathit{h}}_j^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_{i,i+1}=o({\mathrm{\Delta }}_{(i,j)}^{1/2})$ for i,j=1,…,k−1 (i+1<j) under Conditions 1 and 5. It concludes the results.

Lemma 3.Under Conditions 1 and 5, it holds that for i=1,…,k−1

$$\begin{array}{ll}\underset{d\to \infty }{\lim }{\mathit{h}}_i^{\mathrm{T}}{\displaystyle \sum _{m=1}^k}\frac{{\epsilon }_m({\mathit{\mu }}_{i^{\prime }}-{\mathit{\mu }}_m)}{{\lambda }_i^{1/2}}=\left\{\begin{array}{ll}0& \mathrm{when}i\ge 2\mathrm{and}{i}^{\prime }<i,\\ {[(1-{\epsilon }_{(i)})/\{{\epsilon }_i(1-{\epsilon }_{(i-1)})\}]}^{1/2}& \mathrm{when}{i}^{\prime }=i,\\ {-[{\epsilon }_i/\{(1-{\epsilon }_{(i)})(1-{\epsilon }_{(i-1)})\}]}^{1/2}& \mathrm{when}{i}^{\prime }>i.\end{array}\right.& \end{array}$$

Proof.We write that

$$\begin{array}{ll}& {\displaystyle \sum _{m=1}^k}{\epsilon }_m({\mathit{\mu }}_1-{\mathit{\mu }}_m)={\displaystyle \sum _{m=1}^{k-1}}(1-{\epsilon }_{(m)}){\mathit{\mu }}_{m,m+1},\\ & {\displaystyle \sum _{m=1}^k}{\epsilon }_m({\mathit{\mu }}_k-{\mathit{\mu }}_m)=-{\displaystyle \sum _{m=1}^{k-1}}{\epsilon }_{(m)}{\mathit{\mu }}_{m,m+1}\text{and}\\ & {\displaystyle \sum _{m=1}^k}{\epsilon }_m({\mathit{\mu }}_i-{\mathit{\mu }}_m)={\displaystyle \sum _{m=i}^{k-1}}(1-{\epsilon }_{(m)}){\mathit{\mu }}_{m,m+1}-{\displaystyle \sum _{m=1}^{i-1}}{\epsilon }_{(m)}{\mathit{\mu }}_{m,m+1},\end{array}$$ (A10)

for i=2,…,k−1. Using Lemma 2, under Conditions 1 and 5, we have that as d→∞

$$\begin{array}{ll}{\mathit{h}}_1^{\mathrm{T}}{\displaystyle \sum _{m=1}^k}\frac{{\epsilon }_m({\mathit{\mu }}_1-{\mathit{\mu }}_m)}{{\mathrm{\Delta }}_{1,2}^{1/2}}& ={\mathit{h}}_1^{\mathrm{T}}\frac{(1-{\epsilon }_{(1)}){\mathit{\mu }}_{1,2}}{{\mathrm{\Delta }}_{1,2}^{1/2}}+o(1)=1-{\epsilon }_{(1)}+o(1)\text{and}\\ {\mathit{h}}_1^{\mathrm{T}}{\displaystyle \sum _{m=1}^k}\frac{{\epsilon }_m({\mathit{\mu }}_{i^{\prime }}-{\mathit{\mu }}_m)}{{\mathrm{\Delta }}_{i,i+1}^{1/2}}& =-{\mathit{h}}_1^{\mathrm{T}}\frac{{\epsilon }_{(1)}{\mathit{\mu }}_{1,2}}{{\mathrm{\Delta }}_{1,2}^{1/2}}+o(1)=-{\epsilon }_{(1)}+o(1)\text{for}{i}^{\prime }=2,\dots ,k,\end{array}$$

from Equation A10. Also, using Lemma 2, under Conditions 1 and 5, we have that for i=2,…,k−1;i′ =i+1,…,k;i′′ =1,…,i−1

$$\begin{array}{ll}{\mathit{h}}_i^{\mathrm{T}}{\displaystyle \sum _{m=1}^k}\frac{{\epsilon }_m({\mathit{\mu }}_i-{\mathit{\mu }}_m)}{{\mathrm{\Delta }}_{i,i+1}^{1/2}}& ={\mathit{h}}_i^{\mathrm{T}}\frac{(1-{\epsilon }_{(i)}){\mathit{\mu }}_{i,i+1}-{\epsilon }_{(i-1)}{\mathit{\mu }}_{i-1,i}}{{\mathrm{\Delta }}_{i,i+1}^{1/2}}+o(1)\\ & =(1-{\epsilon }_{(i)})+\frac{{\epsilon }_{(i-1)}(1-{\epsilon }_{(i)})}{1-{\epsilon }_{(i-1)}}+o(1)\\ & =\frac{1-{\epsilon }_{(i)}}{1-{\epsilon }_{(i-1)}}+o(1),\\ {\mathit{h}}_i^{\mathrm{T}}{\displaystyle \sum _{m=1}^k}\frac{{\epsilon }_m({\mathit{\mu }}_{i^{\prime }}-{\mathit{\mu }}_m)}{{\mathrm{\Delta }}_{i,i+1}^{1/2}}& =-{\epsilon }_{(i)}+\frac{{\epsilon }_{(i-1)}(1-{\epsilon }_{(i)})}{1-{\epsilon }_{(i-1)}}+o(1)\\ & =-\frac{{\epsilon }_i}{1-{\epsilon }_{(i-1)}}+o(1),\text{and}\\ {\mathit{h}}_i^{\mathrm{T}}{\displaystyle \sum _{m=1}^k}\frac{{\epsilon }_m({\mathit{\mu }}_{i^{\prime \prime }}-{\mathit{\mu }}_m)}{{\mathrm{\Delta }}_{i,i+1}^{1/2}}& =o(1).\end{array}$$

Thus, from Lemma 2, we can conclude the results.

Lemma 4.Assume Conditions 2. Then, under the condition:

$$0<\underset{d\to \infty }{\mathrm{plim}}\frac{{\tilde{\lambda }}_i}{{\mathrm{\Delta }}_{i,i+1}}<\infty \mathrm{for}i=1,\dots ,k-1,$$ (A11)

it holds that

$$\underset{d\to \infty }{\mathrm{plim}}{\widehat{\mathit{u}}}_i^{\mathrm{T}}{\tilde{\mathit{u}}}_i=1\mathrm{for}{\widehat{\mathit{u}}}_i^{\mathrm{T}}{\tilde{\mathit{u}}}_i\ge 0,i=1,\dots ,k-1.$$

Proof.From the fact that ${\lambda }_{\max }({\mathbf{\sum }}_i)\le \text{tr}{({\mathbf{\sum }}_i^2)}^{1/2}$, we have that $\mathrm{Var}\{{\mathit{\mu }}_{k-1}^{\mathrm{T}}({\mathit{x}}_j-{\mathit{\mu }}_i)|{\mathit{x}}_j\in {\mathrm{\Pi }}_i\}={\mathit{\mu }}_{k-1}^{\mathrm{T}}{\mathbf{\sum }}_i{\mathit{\mu }}_{k-1}\le {\lambda }_{\max }({\mathbf{\sum }}_i){\mathrm{\Delta }}_{k-1}=o({\mathrm{\Delta }}_{k-1}^2)$ as d→∞ for i=1,…,k;j=1,…,n under Condition 2. Then, we have that for i=1,…,k−1; i′=1,…,k;j=1,…,n

$$\begin{array}{ll}\mathrm{Var}\{{\mathit{\mu }}_{i,i+1}^{\mathrm{T}}({\mathit{x}}_j-{\mathit{\mu }}_{i^{\prime }})|{\mathit{x}}_j\in {\mathrm{\Pi }}_{i^{\prime }}\}={\mathit{\mu }}_{i,i+1}^{\mathrm{T}}{\mathbf{\sum }}_{i^{\prime }}{\mathit{\mu }}_{i,i+1}& =O({\mathit{\mu }}_i^{\mathrm{T}}{\mathbf{\sum }}_{i^{\prime }}{\mathit{\mu }}_i+{\mathit{\mu }}_{i+1}^{\mathrm{T}}{\mathbf{\sum }}_{i^{\prime }}{\mathit{\mu }}_{i+1})=o({\mathrm{\Delta }}_{k-1}^2),\end{array}$$

under Conditions 2 and 6. Then, similar to Equation A1, under Conditions 2 and 6, it holds that ${\mathit{\mu }}_{i,i+1}^{\mathrm{T}}({\mathit{x}}_j-{\mathit{\mu }}_{i^{\prime }})/{\mathrm{\Delta }}_{k-1}=o_P(1)$ when ${\mathit{x}}_j\in {\mathrm{\Pi }}_{i^{\prime }}$ for i=1,…,k−1;i^′=1,…,k;j=1,…,n. In addition, under Conditions 2 and 3, we can claim that ${({\mathit{x}}_j-{\mathit{\mu }}_i)}^{\mathrm{T}}({\mathit{x}}_{j^{\prime }}-{\mathit{\mu }}_{i^{\prime }})/{\mathrm{\Delta }}_{k-1}=o_P(1)$ and $||{\mathit{x}}_j-{\mathit{\mu }}_i|{|}^2/{\mathrm{\Delta }}_{k-1}=\text{tr}({\mathbf{\sum }}_i)/{\mathrm{\Delta }}_{k-1}+o_P(1)$ when x_j∈ Π_i and ${\mathit{x}}_{j^{\prime }}\in {\mathrm{\Pi }}_{i^{\prime }}$ for all j≠j^′ and i,i^′=1,…,k. Here, we write that x_j−μ_η=(x_j−μ_i)+ν_i for i=1,…,k;j=1,…,n, where ${\mathit{\mu }}_{\eta }={\sum }_{i=1}^k{\eta }_i{\mathit{\mu }}_i$. Then, by noting Equation A10 with ε_i=η_i and ε_(i)=η_(i), i=1,…,k, under Conditions 1, and 6, we have that

$$\begin{array}{ll}\frac{||{\mathit{x}}_j-{\mathit{\mu }}_{\eta }|{|}^2}{{\mathrm{\Delta }}_{k-1}}=\frac{||{\mathit{\nu }}_i|{|}^2+\text{tr}({\mathbf{\sum }}_i)}{{\mathrm{\Delta }}_{k-1}}+o_P(1)\text{and}\frac{{({\mathit{x}}_j-{\mathit{\mu }}_{\eta })}^{\mathrm{T}}({\mathit{x}}_{j^{\prime }}-{\mathit{\mu }}_{\eta })}{{\mathrm{\Delta }}_{k-1}}=& \frac{{\mathit{\nu }}_i^{\mathrm{T}}{\mathit{\nu }}_{i^{\prime }}}{{\mathrm{\Delta }}_{k-1}}+o_P(1),\end{array}$$

when x_j∈ Π_i and ${\mathit{x}}_{j^{\prime }}\in {\mathrm{\Pi }}_{i^{\prime }}$ for all j≠j^′ and i,i^′=1,…,k. Thus, under Conditions 1, and 6, it holds that

$$\underset{d\to \infty }{\mathrm{plim}}\frac{{(\mathit{X}-{\mathit{\mu }}_{\eta }{\mathbf{1}}_n^{\mathrm{T}})}^{\mathrm{T}}(\mathit{X}-{\mathit{\mu }}_{\eta }{\mathbf{1}}_n^{\mathrm{T}})-\text{tr}({\mathbf{\sum }}_1){\mathit{I}}_n-{\mathit{V}}^{\mathrm{T}}\mathit{V}}{{\mathrm{\Delta }}_{k-1}}=\mathit{O}.$$ (A12)

Let ${\mathit{e}}_{n\ast }(\in {\mathbb{R}}^n)$ be an arbitrary random unit vector such that ${\mathit{e}}_{n\ast }^{\mathrm{T}}{\mathbf{1}}_n=0$. We note that ${\mathit{P}}_n{(\mathit{X}-{\mathit{\mu }}_{\eta }{\mathbf{1}}_n^{\mathrm{T}})}^{\mathrm{T}}(\mathit{X}-{\mathit{\mu }}_{\eta }{\mathbf{1}}_n^{\mathrm{T}}){\mathit{P}}_n/(n-1)={\mathit{S}}_{\mathrm{D}}$. Then, by noting ${\mathit{e}}_{n\ast }^{\mathrm{T}}{\mathit{P}}_n={\mathit{e}}_{n\ast }^{\mathrm{T}}$, under Equation A11, Conditions 1, and 6, we have that

$$\begin{array}{ll}{\mathit{e}}_{n\ast }^{\mathrm{T}}\frac{(n-1){\mathit{S}}_{\mathrm{D}}-\text{tr}({\mathbf{\sum }}_1){\mathit{P}}_n}{{\mathrm{\Delta }}_{k-1}}{\mathit{e}}_{n\ast }& =\frac{{\sum }_{i=1}^{n-1}(n-1){\widehat{\lambda }}_i{\mathit{e}}_{n\ast }^{\mathrm{T}}{\widehat{\mathit{u}}}_i{\widehat{\mathit{u}}}_i^{\mathrm{T}}{\mathit{e}}_{n\ast }-\text{tr}({\mathbf{\sum }}_1)}{{\mathrm{\Delta }}_{k-1}}\\ & ={\mathit{e}}_{n\ast }^{\mathrm{T}}\frac{{(\mathit{X}-{\mathit{\mu }}_{\eta }{\mathbf{1}}_n^{\mathrm{T}})}^{\mathrm{T}}(\mathit{X}-{\mathit{\mu }}_{\eta }{\mathbf{1}}_n^{\mathrm{T}})-\text{tr}({\mathbf{\sum }}_1){\mathit{I}}_n}{{\mathrm{\Delta }}_{k-1}}{\mathit{e}}_{n\ast }\\ & ={\mathit{e}}_{n\ast }^{\mathrm{T}}\frac{{\mathit{V}}^{\mathrm{T}}\mathit{V}}{{\mathrm{\Delta }}_{k-1}}{\mathit{e}}_{n\ast }+o_P(1)\\ & =\frac{{\sum }_{i=1}^{k-1}n{\tilde{\lambda }}_i{\mathit{e}}_{n\ast }^{\mathrm{T}}{\tilde{\mathit{u}}}_i{\tilde{\mathit{u}}}_i^{\mathrm{T}}{\mathit{e}}_{n\ast }}{{\mathrm{\Delta }}_{k-1}}+o_P(1),\end{array}$$ (A13)

from Equation A12. We note that ${\tilde{\mathit{u}}}_i^{\mathrm{T}}{\mathbf{1}}_n=0$ for i=1,…,k−1 in case of rank(V)=k−1. Also, from Equation A2, we note that ${\tilde{\lambda }}_i,i=1,\dots ,k-1$, are distinct under Condition 5 and Equation A11 for a sufficiently large d. Thus, from Equation A13, if ${\widehat{\mathit{u}}}_i^{\mathrm{T}}{\tilde{\mathit{u}}}_i\ge 0$ for i=1,…,k−1, we have that ${\widehat{\mathit{u}}}_i^{\mathrm{T}}{\tilde{\mathit{u}}}_i=1+o_P(1)$ for i=1,…,k−1. It concludes the result.

Lemma 5.Assume Condition 5. For n_i>0,i=1,…,k, it holds that

$$\underset{d\to \infty }{\mathrm{plim}}\frac{{\tilde{\lambda }}_i}{{\mathrm{\Delta }}_{i,i+1}}=\frac{{\eta }_i(1-{\eta }_{(i)})}{1-{\eta }_{(i-1)}}\mathrm{and}\underset{d\to \infty }{\mathrm{plim}}{\tilde{\mathit{u}}}_i^{\mathrm{T}}{\mathit{u}}_i=1\mathrm{for}i=1,\dots ,k-1.$$

Proof.By noting Equation A10 with ε_i=η_i and ε_(i)=η_(i), i=1,…,k, we can write that

$$\begin{array}{ll}\frac{\mathit{V}{\mathit{V}}^{\mathrm{T}}}{n}=& {\displaystyle \sum _{i=1}^{k-1}}{\eta }_{(i)}(1-{\eta }_{(i)}){\mathit{\mu }}_{i,i+1}{\mathit{\mu }}_{i,i+1}^{\mathrm{T}}+{\displaystyle \sum _{i=1}^{k-2}}{\displaystyle \sum _{j=i+1}^{k-1}}{\eta }_{(i)}(1-{\eta }_{(j)})({\mathit{\mu }}_{i,i+1}{\mathit{\mu }}_{j,j+1}^{\mathrm{T}}+{\mathit{\mu }}_{j,j+1}{\mathit{\mu }}_{i,i+1}^{\mathrm{T}}).\end{array}$$ (A14)

We have the eigendecomposition of VV^T/n by $\mathit{V}{\mathit{V}}^{\mathrm{T}}/n={\sum }_{i=1}^{k-1}{\tilde{\lambda }}_i{\tilde{\mathit{h}}}_i{\tilde{\mathit{h}}}_i^{\mathrm{T}}$, where ${\tilde{\mathit{h}}}_i$ is a unit eigenvector corresponding to ${\tilde{\lambda }}_i$ for each i. We note that η_i>0,i=1,…,k for n_i>0,i=1,…,k. Then, by noting Lemmas 2 and 3 and the fact that Equation A14 is same as Equation A4 with ε_(i)=η_(i),i=1,…,k−1, under Condition 5, we have that for i=1,…,k−1

$$\underset{d\to \infty }{\mathrm{plim}}\frac{{\tilde{\lambda }}_i}{{\mathrm{\Delta }}_{i,i+1}}=\frac{{\eta }_i(1-{\eta }_{(i)})}{1-{\eta }_{(i-1)}}\text{and}\underset{d\to \infty }{\mathrm{plim}}\frac{{\tilde{\mathit{h}}}_i^{\mathrm{T}}{\mathit{\nu }}_{(j)}}{{\tilde{\lambda }}_i^{1/2}}=u_{ij}{n}^{1/2},$$

if ${\tilde{\mathit{h}}}_i^{\mathrm{T}}{\mathit{\mu }}_i\ge 0$. We note that ${ũ}_{ij}={\tilde{\mathit{h}}}_i^{\mathrm{T}}{\mathit{\nu }}_{(j)}/{(n{\tilde{\lambda }}_i)}^{1/2}$ from the fact that ${\tilde{\mathit{u}}}_i={\mathit{V}}^{\mathrm{T}}{\tilde{\mathit{h}}}_i/{(n{\tilde{\lambda }}_i)}^{1/2}$ for i=1,…,k−1. Hence, we can conclude the result.

Appendix B: Additional Proposition

When Condition 5 is not met, Theorem 3 does not hold. However, in Figure 5c, we could find three separate clusters of Π_i,i=1,2,3, even though Condition 5 is not met. To explain the reason of this phenomenon, we give the following result.

Proposition 3.Assume Conditions 2 and 6. Then, under the condition:

$$0<\underset{d\to \infty }{\mathrm{plim}}\frac{{\tilde{\lambda }}_{k-1}}{{\mathrm{\Delta }}_{k-1}}<\infty ,$$

it holds that for i=1,…,k−1, as d→∞

$${\widehat{\mathit{u}}}_i^{\mathrm{T}}\frac{(n-1){\mathit{S}}_{\mathrm{D}}}{{\mathrm{\Delta }}_{k-1}}{\widehat{\mathit{u}}}_i=\frac{\mathrm{tr}({\mathbf{\sum }}_1)}{{\mathrm{\Delta }}_{k-1}}+{\widehat{\mathit{u}}}_i^{\mathrm{T}}\frac{{\mathit{V}}^{\mathrm{T}}\mathit{V}}{{\mathrm{\Delta }}_{k-1}}{\widehat{\mathit{u}}}_i+o_P(1).$$

Proof.By noting that ${\widehat{\mathit{u}}}_i^{\mathrm{T}}{\mathbf{1}}_n=0$ for i=1,…,k−1 when rank(S_D)≥k−1, from Equation A13, we can conclude the result.

By noting that ${\widehat{\mathit{u}}}_i={({\widehat{z}}_{i1},\dots ,{\widehat{z}}_{in})}^{\mathrm{T}}/n^{1/2}$, from Proposition 3, for sufficiently large d, the estimated PC scores depend only on the structure of V^TV even when Condition 5 is not met. Then, as rank(V^TV)=k−1, there must be k separate clusters for Π_i,i=1,…,k, in the first k−1 PC spaces as seen in Figure 5c.

Appendix C: Proofs of Theorems, Corollaries, and Propositions

Proofs of Theorem 1 and Corollary 1.We note that tr(∑₁)/tr(∑)→(1−ε₁ε₂c) as d→∞ under Condition 4 and Δ₁/tr(∑)→c(>0) as d→∞. Then, using Lemma 1, we can conclude the result of Theorem 1.

Next, we consider the proof of Corollary 1. From the fact that ${\mathbf{1}}_n^{\mathrm{T}}{\mathit{S}}_{\mathrm{D}}{\mathbf{1}}_n=0$, it holds that ${\widehat{\mathit{u}}}_1^{\mathrm{T}}{\mathbf{1}}_n=0$ when S_D≠O, so that ${\mathit{P}}_n{\widehat{\mathit{u}}}_1={\widehat{\mathit{u}}}_1$. Also, note that ||r||²=nη₁η₂ and ${\mathit{r}}^{\mathrm{T}}{\mathbf{1}}_n=0$. Then, using Lemma 1, under Conditions 1, it holds that ${\widehat{\mathit{u}}}_1^{\mathrm{T}}\{(n-1){\mathit{S}}_{\mathrm{D}}-\text{tr}({\mathbf{\sum }}_1){\mathit{P}}_n\}{\widehat{\mathit{u}}}_1/{\mathrm{\Delta }}_1=n{\eta }_1{\eta }_2+o_P(1)$. Hence, from Equation 3 and the assumption that ${\widehat{\mathit{u}}}_1^{\mathrm{T}}{\mathit{z}}_1\ge 0$, we have that ${\widehat{\mathit{u}}}_1^{\mathrm{T}}\{{(n{\eta }_1{\eta }_2)}^{-1/2}\mathit{r}\}=1+o_P(1)$ for n_i>0, i=1,2. In view of the elements of r, we can conclude the result of Corollary 1.

Proof of Proposition 1.We assume x_j∈ Π₁ for j=1,…,n₁, x_j∈ Π₂ for j=n₁+1,…,n, and tr(∑₁)≥tr(∑₂) without loss of generality. Similar to the proof of Lemma 1, under the assumptions of Proposition 1, we have that

$$\underset{d\to \infty }{\mathrm{plim}}\frac{(n-1){\mathit{S}}_{\mathrm{D}}-\text{tr}({\mathbf{\sum }}_2){\mathit{P}}_n}{{\mathrm{\Delta }}_{\mathrm{\sum }}}={\mathit{P}}_n{\mathit{D}}_n{\mathit{P}}_n,$$

where D_n=diag(1,…,1,0,…,0) whose first n₁ diagonal elements are 1. Note that the first n₁−1 eigenvalues of P_nD_nP_n are multiple. Also, note that the eigenspace for the multiple eigenvalue consists of the n₁−1 vectors,

$${(1,-1,0,\dots ,0)}^{\mathrm{T}},{(1,0,-1,0,\dots ,0)}^{\mathrm{T}},\dots ,{(1,0,\dots ,0,-1,0,\dots ,0)}^{\mathrm{T}}.$$

Thus, by noting that ${\widehat{\mathit{u}}}_i^{\mathrm{T}}{\mathbf{1}}_n=0$ for i=1,…,n₁−1, we can conclude the result.

Proofs of Theorem 2 and Corollary 2.We write that ${\mathit{x}}_j-\mathit{\mu }=({\mathit{x}}_j-{\mathit{\mu }}_i)+{\sum }_{m=1}^k{\epsilon }_m({\mathit{\mu }}_i-{\mathit{\mu }}_m)$ for j=1,…,n; i=1,…,k. We note that $\mathrm{Var}\{{\mathit{e}}_d^{\mathrm{T}}({\mathit{x}}_j-{\mathit{\mu }}_i)/{\mathrm{\Delta }}_{k-1}^{1/2}|{\mathit{x}}_j\in {\mathrm{\Pi }}_i\}={\mathit{e}}_d^{\mathrm{T}}{\mathbf{\sum }}_i{\mathit{e}}_d/{\mathrm{\Delta }}_{k-1}\le {\lambda }_{\max }({\mathbf{\sum }}_i)/{\mathrm{\Delta }}_{k-1}=o(1)$ as d→∞ under Condition 1 for j=1,…,n;i=1,…,k, where ${\mathit{e}}_d(\in {\mathbb{R}}^d)$ is an arbitrary unit vector. Then, under Condition 1, when x_j∈ Π_i, it holds that

$$\frac{{\mathit{e}}_d^{\mathrm{T}}({\mathit{x}}_j-\mathit{\mu })}{{\mathrm{\Delta }}_{k-1}^{1/2}}=\frac{{\mathit{e}}_d^{\mathrm{T}}\{{\sum }_{m=1}^k{\epsilon }_m({\mathit{\mu }}_i-{\mathit{\mu }}_m)\}}{{\mathrm{\Delta }}_{k-1}^{1/2}}+o_P(1).$$

Then, using Lemmas 2 and 3, we can conclude the result of Theorem 2.

For the proof of Corollary 2, by noting that Δ_i,i+1/Δ_i=1+o(1) and ${\mathit{h}}_i^{\mathrm{T}}{\mathit{\mu }}_{i,i+1}/{\mathrm{\Delta }}_{i,i+1}^{1/2}={\mathit{h}}_i^{\mathrm{T}}{\mathit{\mu }}_i/{\mathrm{\Delta }}_i^{1/2}+o(1)$ for i=1,…,k−1, under Condition 5, from Lemma 2, the results are obtained straightforwardly.

Proof of Theorem 3.By combining Lemmas 4 and 5, from Theorem 2 and the assumption that ${\widehat{\mathit{u}}}_i^{\mathrm{T}}{\mathit{z}}_i\ge 0$ for all i, the result is obtained straightforwardly.

Proof of Proposition 2.Let ∑_(∗)=ε₁∑₁+ε₂∑₂. Then, we define the eigendecomposition of ∑_(∗) by ${\mathbf{\sum }}_{(\ast )}={\sum }_{i=1}^d{\lambda }_{i(\ast )}{\mathit{h}}_{i(\ast )}{\mathit{h}}_{i(\ast )}^{\mathrm{T}}$, where λ_1(∗)≥···≥λ_d(∗)≥0 are eigenvalues of ∑_(∗) and h_i(∗) is a unit eigenvector corresponding to λ_i(∗) for each i. Let λ=ε₁ε₂Δ₁ and ${\stackrel{\text{'}}{\mathit{\mu }}}_1={\mathit{\mu }}_1/{\mathrm{\Delta }}_1^{1/2}$. Then, from $\mathbf{\sum }=\lambda {\stackrel{\text{'}}{\mathit{\mu }}}_1{\stackrel{\text{'}}{\mathit{\mu }}}_1^{\mathrm{T}}+{\mathbf{\sum }}_{(\ast )}$, under ${\max }_{i=1,2}({\stackrel{\text{'}}{\mathit{\mu }}}_1^{\mathrm{T}}{\mathbf{\sum }}_i{\stackrel{\text{'}}{\mathit{\mu }}}_1)/{\mathrm{\Delta }}_1\to 0$ as d→∞, it holds that ${\stackrel{\text{'}}{\mathit{\mu }}}_1^{\mathrm{T}}\mathbf{\sum }{\stackrel{\text{'}}{\mathit{\mu }}}_1/\lambda \to 1$, so that

$${\displaystyle \sum _{i=1}^d}\frac{{\lambda }_{i(\ast )}{({\mathit{h}}_{i(\ast )}^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_1)}^2}{\lambda }=o(1).$$ (C1)

Let κ(i)=λ_i(⋆⋆)−λ for i=1,…,d. For a sufficiently large d, when κ(1)>0, there exists some positive integer i_⋆⋆ such that

$$i_{\star \star }=\max \{i|\kappa (i)>0\text{for}i=1,\dots ,d\}.$$

Then, from Equation C1, we have that ${\sum }_{i=1}^{i_{\star \star }}{({\mathit{h}}_{i(\ast )}^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_1)}^2=o(1)$, so that ${\lambda }_{i_{\star }}/\lambda =1+o(1)$ with i_⋆=i_⋆⋆+1. When κ(1)≤0 for a sufficiently large d, it holds that ${\lambda }_{i_{\star }}/\lambda =1+o(1)$ with i_⋆=1. In addition, under ${\text{lim inf}}_{d\to \infty }|{\lambda }_{i^{\prime }}/{\lambda }_{i_{\star }}-1|>0$ for i^′=1,…,d(i^′≠i_⋆), it holds that ${\mathit{h}}_{i_{\star }}^{\mathrm{T}}{\stackrel{\text{'}}{\mathit{\mu }}}_1=1+o(1)$ from ${\mathit{h}}_{i_{\star }}^{\mathrm{T}}{\mathit{\mu }}_1\ge 0$. Then, from the fact that ${\mathit{h}}_{i_{\star }}^{\mathrm{T}}{\mathbf{\sum }}_i{\mathit{h}}_{i_{\star }}/\lambda \to 0$ as d→∞ for i=1,2, in a way similar to Equation A1, we have that $s_{i_{\star }j}/{\lambda }_{i_{\star }}^{1/2}={\mathit{h}}_{i_{\star }}^{\mathrm{T}}({\mathit{x}}_j-\mathit{\mu })/{\lambda }_{i_{\star }}^{1/2}={\mathit{h}}_{i_{\star }}^{\mathrm{T}}({\mathit{\mu }}_i-\mathit{\mu })/{\lambda }_{i_{\star }}^{1/2}+o_P(1)$ when x_j∈ Π_i for j=1,…,n;i=1,2. We can conclude the results.