Histopathological image analysis for centroblasts classification through dimensionality reduction approaches

Authors

  • Evgenios N. Kornaropoulos,

    Corresponding author
    1. Informatics and Telematics Institute—Centre for Research and Technology Hellas (ITI—CERTH), Thessaloniki, Greece
    • Correspondence to: E. N. Kornaropoulos, Department of Computer Vision, Informatics and Telematics Institute—Centre for Research and Technology Hellas (ITI—CERTH), 1st km Thermi—Panorama, 57001, Thessaloniki, Greece. E-mail: ekornaro@gmail.com

    Search for more papers by this author
  • M. Khalid Khan Niazi,

    1. Department of Biomedical Informatics, Ohio State University, Columbus, Ohio
    Search for more papers by this author
  • Gerard Lozanski,

    1. Department of Pathology, Ohio State University, Columbus, Ohio
    Search for more papers by this author
    • Gerard Lozanski and Metin N. Gurcan contributed equally to this article.

  • Metin N. Gurcan

    1. Department of Biomedical Informatics, Ohio State University, Columbus, Ohio
    Search for more papers by this author
    • Gerard Lozanski and Metin N. Gurcan contributed equally to this article.

    • Senior member, IEEE.


Abstract

We present two novel automated image analysis methods to differentiate centroblast (CB) cells from noncentroblast (non-CB) cells in digital images of H&E-stained tissues of follicular lymphoma. CB cells are often confused by similar looking cells within the tissue, therefore a system to help their classification is necessary. Our methods extract the discriminatory features of cells by approximating the intrinsic dimensionality from the subspace spanned by CB and non-CB cells. In the first method, discriminatory features are approximated with the help of singular value decomposition (SVD), whereas in the second method they are extracted using Laplacian Eigenmaps. Five hundred high-power field images were extracted from 17 slides, which are then used to compose a database of 213 CB and 234 non-CB region of interest images. The recall, precision, and overall accuracy rates of the developed methods were measured and compared with existing classification methods. Moreover, the reproducibility of both classification methods was also examined. The average values of the overall accuracy were 99.22% ± 0.75% and 99.07% ± 1.53% for COB and CLEM, respectively. The experimental results demonstrate that both proposed methods provide better classification accuracy of CB/non-CB in comparison with the state of the art methods. © 2013 International Society for Advancement of Cytometry

Follicular lymphoma (FL) is the second most common lymphoid malignancy in the western world [1]. FL is a disease with highly variable clinical course and is currently only curable with stem cell transplant. In a subset of patients, FL behaves as chronic indolent disease that may never require therapy, whereas in another subset it acts aggressively and if not treated with chemoimmunotherapy will result in patient's rapid death. Moreover, in a subset of patients with an initial indolent presentation, FL may transform into the aggressive type where symptomatic patients must be treated. The reason why chemoimmunotherapy does not apply in indolent cases of the disease is due to the lack of evidence in how beneficial it will be. Therefore, in order to avoid unnecessary toxicity, treatment is reserved for symptomatic patients and for those with the aggressive form of the disease.

Oncologist's clinical decisions at the time of initiation and type of therapy are guided by risk stratification. Risk stratification for each individual patient is based on a combination of clinical and laboratory findings, including morphological characteristics of tumor tissue.

The World Health Organization (WHO) adopted the method proposed by Mann and Berard, which is the most established and recommended one for morphological risk stratification of FL [2]. In this method, a pathologist stratifies FL cases into histological grades based on the number of large cancer cells, called centroblasts (CB). CBs are counted manually in standard microscopic high power fields (HPFs) from Hematoxilin and Eosin (H&E) stained tissue biopsies of FL. Due to the time constraints, CBs are only counted in 10 random HPFs, sampled from sections of malignant follicles [3]. Using this method, FL cases are categorized into three histological grades according to the average CB count per HPF; Grade I (0–5 CB/HPF), Grade II (6–15 CB/HPF), and Grade III (>15 CB/HPF). Grade III cases are further subclassified into Grade IIIa and IIIb. Follicles of Grade IIIa contain CBs along with small cells called centrocytes, while follicles of Grade IIIb contain pure populations of CBs without admixed centrocytes [1]. Grades I and II are considered as low risk cases and may not require treatment unless the patient is symptomatic. Grades IIIa and IIIb though are evaluated as high risk and indicate an aggressive type of disease that requires immediate life-saving chemotherapy.

Manual histological grading of FL is highly subjective and it requires considerable effort, along with an extensive training. Furthermore, the analysis of even one HPF under a light microscope by the pathologist is a time-consuming process. This laborious and time demanding nature of the method is the reason why only 10 HPF are analyzed per case. This limitation makes the method highly vulnerable to sampling bias in cases of tumors with high tissue heterogeneity. Moreover, due to the subjective nature of this method, some prognostic clues are not easily observed by all pathologists. Indeed, inter- and intrapathologist variability has been reported, which normally ranges from 61% to 73% [4, 5]. All these issues directly affect the clinical decisions about the timing and the type of therapy. Therefore, increasing the reliability and reproducibility of the process of histological grading is of great importance.

Computer-aided diagnosis (CAD) is increasingly used in medicine to aid with detection, diagnosis, and prognosis of diseases both in radiology and pathology [6-10, 43, 44]. The quantitative analysis of histological tissues is the subject of several earlier studies and it has been successfully applied in the diagnosis of prostate cancer [11], neuroblastoma (NB) [12, 13], and breast cancer [14, 39-41]. To better understand the different stages of CAD and some of its applications, a comprehensive review can be found in Ref. 15.

CAD has also been reported to be beneficial in classifying tissue subtypes associated with various grades of FL [16-24]. The classification is often achieved by extracting features from H&E stained images of FL. These features can distinguish the cells from each other, and thereby classify them into different categories. Several morphological and topological features [21], or their combination with texture features [16, 20, 45], as well as with graph-based features [22] have been introduced. The most discriminating subset of these feature vectors are often identified in a lower dimensional space, created by employing principal component analysis (PCA) that captures the main modes of variations in the data. Using this approach, classification accuracies ranging from 75% to 85% are reported [16, 20, 21], but this level of performance may not be sufficient enough for certain clinical applications. Selection of the best and clinically meaningful feature sets, as well as the design of classifiers, is an active area of research [16, 17, 20, 21, 46, 47].

In the current study, our aim is to classify the cells in FL images in a similar manner as perceived by the pathologists when they review tissue of FL under a microscope. For this purpose, we set out to develop a tool called Kyttaro (meaning cell in Greek) that acts as a content-based image retrieval system. This system brings the most relevant cell images from its library of cell images, which are already classified into CB or non-CB categories.

In clinical practice, pathologists identify several features of CB, such as size, circularity, coarse texture, multiple nucleoli, vesicular chromatin, and accentuated nuclear membrane. Moreover, pathologists also take into account the structures around the cell, while making a decision. However, not every pathologist uses these features; part of the knowledge is implicit. Therefore, we concluded that we should consider the whole image of a cell with its surroundings as a feature vector. In that way, we incorporate all the features mentioned by the pathologist. Furthermore, redundant features are removed by linear and nonlinear dimensionality reduction methods.

The section to follow provides detailed information about the database used in the current study. Method section describes the proposed classification methods along with a preprocessing step necessary to suppress noise from the images. The training process of each proposed classifier, as well as its comparative analysis with the state of the art methods are presented in Experimental Results section. This is followed by a comprehensive discussion in Discussion section. Finally, the conclusions are given in Conclusions section.

Image Database

Tissue biopsies of FL stained with H&E, from 17 different patients were scanned using a high-resolution whole slide scanner (Aperio—Image Scope). Three board-certified hematopathologists selected 500 HPF images of follicular lymphoma out of the scanned tissue biopsies. These 500 images are then examined by two expert pathologists by using a remote viewing and annotation tool, developed in our lab, to mark CB cells on the HPF images. Using these markings, a set of images of CB cells was created. Each image contains the CB cell at its center and is of size 71×71 pixels (Fig. 1a). Similarly, a second set of same size images of cells that were not marked by any pathologist as CB was generated. These images are called non-CB cells and typically include centrocytes, histoicytes, dendric cells (Fig. 1b). All together, the database is composed of 213 CB and 234 non-CB images. These cases were selected from the archives of The Ohio State University with Institutional Review Board (IRB) approval (Protocol 2007 C0069, renewed May 13, 2013).

Figure 1.

Images of a CB cell (left image) and non-CB cells (right image). The scanner's resolution at ×40 magnification is 0.25 μm/pixel, therefore the yellow lines indicate a physical length of 4 μm in the tissue. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

METHOD

In this section, we describe the process of noise removal in the cell images, as well as the two proposed methods of cells classification in FL images. While the first method extracts discriminative features by utilizing linear dimensionality reduction, the second one uses a nonlinear dimensionality reduction to extract the discriminative features. The test image is first projected into a low-dimensional space (discriminating feature space). Then the class label of the image is determined by a distance function. The image retrieval system of Kyttaro tool will be based on the most efficient of the two classification methods.

Noise Removal

Microscopic images show variation within them or between them due to the conditions under which they were acquired. Tissue cutting, processing, and staining during slide preparation are some of the steps that cause these variations, making it difficult to perform consistent quantitative analysis on these images [25]. Therefore, all the images in our database were first converted to grayscale and then standardized to partially compensate for these differences. The new image after standardization is a centered, scaled version of the grayscale image of a cell. Moreover, to reduce some salt-and-pepper type of noise while preserving the inherent texture characteristics, we applied median filtering with a kernel size of 5-by-5, to the standardized gray-scale images (see Fig. 2).

Figure 2.

RGB image of a CB cell (top left image), its intensity representation in gray scale (top right image), the gray scale image after applying standardization (bottom left image) and the filtered with median filter the same gray scale standardized image (bottom right). [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Classification Using Orthogonal Bases (COB)

The design of this classifier is inspired by the theoretical framework provided by Elden [26]. According to this framework, the CB/non-CB classification problem can be formulated as a minimization problem, where the objective is to minimize the least-square error between a given image of CB/non-CB and its low rank approximation. We assume that we have n images of CB and the same number of images of non-CB cells. Let any (i.e. CB or non-CB) cell image be a vector in a m-dimensional vector space, where m is equal to the number of pixels in each image. Here, the vector entries are the intensity values at each pixel. Within the m-dimensional vector space of all the images, there lie two different hypothetical subspaces, one for the CB cells and the other for the non-CB cells. We will now present a detailed methodology to determine each hypothetical subspace which will enable us to differentiate between CB and non-CB cell images.

Let math formula math formula be the matrix

display math(1)

where columns are formed by concatenating n m-dimensional vectors of CB or non-CB cells in the training set. So, essentially each column corresponds to a unique CB or non-CB image from the set of n images. Unlike other popular methods, we created two different matrices, one for CB and another one for non-CB.

To remove the redundancy from each of these rectangular matrices, we can write Ai as:

display math(2)

where math formula math formula and math formula math formula are the orthogonal matrices, whose columns represent the left and right orthonormal eigenvectors for Ai, respectively. Here, math formula corresponds to the eigenvalues of Ai. Further, suppose that Ai has rank r. Then, matrices Ai can be written as:

display math(3)

where math formula is the kth math formula-dimensional orthonormal eigenvector of math formula, math formula is the kth math formula-dimensional orthonormal eigenvector of math formula and math formula is the kth eigenvalue of math formula. Since math formula » math formula, in case Ai has a full column rank, r would be equal to n. In that case r should be substituted by n in Eq. (3) and the following equations.

Let:

display math(4)
display math

Then math formula can be written as an outer product expansion as:

display math(5)

Each of the terms math formula is an outer product of vectors uj and vj, weighted by the eigenvalue math formula.

Equation (5) shows us that any new image of a cell z can be expressed in terms of the orthogonal basis math formula as:

display math(6)

Since math formula is a scalar, we can rearrange the order of the factors to:

display math(7)

Now math formula is expressed as a linear combination of left singular vectors math formula. The left singular eigenvectors are an orthogonal basis for the column space of math formula so for the “image space of CB” or the “image space of non-CB.” We will refer to these eigenvectors as “singular images.” Each coefficient in Eq. (7) is a product of two factors, math formula and math formula. The factor math formula, where math formula denoted the inner product of the two vectors, is jth component of math formula relative to the orthonormal basis math formula. Under the action of math formula, each math formula component of math formula becomes a math formula component after scaling by the proper math formula. Hence, the coefficients in Eq. (7) express the coordinates of the image math formula in terms of the singular images math formula.

The ordering of the singular images is based on the value of their corresponding eigenvalues. The eigenvalues of math formula are proportional to the values of the covariance matrix among the images, which represent the second dimension of math formula. Therefore the eigenvector with the highest eigenvalue points to the highest variance among the images. We assume that the discriminating features of CB/non-CB would be revealed in the directions pointing towards the highest variance among the images. We therefore order the singular images on a descending order of their corresponding eigenvalues.

In Figure 3 we present the grayscale version of the first three singular images in the “image space of CB” and the first three singular images in the “image space of non-CB.” Reviewing these images, one can note that the first singular image may represent the size feature of the cells because CB cells are typically bigger than non-CB cells in size. However, no immediate associations can be made with the features represented by the following singular images.

Figure 3.

Grayscale version of the first three singular images in the “image space of CB” (top three images) and the first three singular images in the “image space of non-CB.” (bottom three images). [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

The singular images make up the bases of the subspace of CB and non-CB images. Any unknown cell image can be classified as CB or non-CB by calculating representations of the image in terms of CB or non-CB bases. The cell class whose bases approximate the unknown image better in terms of least square error is selected as the class of the image.

To determine which of the two bases of cell images better represent an unknown cell image math formula, the norm of the residual vector is calculated in the least squares sense:

display math(8)

where math formula denotes the norm of the residual vector math formula in each basis (it can be math formula or math formula when we have math formulaand math formula corresponding to CB and non-CB singular basis vector images) .

This problem can be restated in the following form:

display math(9)

where math formula and math formula is the linear combination of weights that need to be assigned to each singular image math formula. As mentioned before, the singular images, which consist of the columns of math formula, are orthonormal, therefore math formula. The solution of this problem is now given by replacing the factor math formula in Eq. (9) by math formula. The nice thing about this formulation is that we do not need to compute the inverse while solving the least square problem, which is generally the case. This makes it applicable to data residing in very high dimensional space. In the end, we are calculating the norm of the residual vector as follows:

display math(10)

The classification criterion is defined like this:

  • Classification algorithm of method 1:

  • Training: For the training set of images of known CB and non-CB cells compute the SVD of each of the set of cells of the two kinds.

  • Classification: If math formula then math formula belongs to CB class otherwise math formula belongs to non-CB class.

The reason for assigning the unknown image to CB class in cases of a tie is because we want to be conservative in the decision and leave the final decision to the pathologist.

Classification Based on Laplacian Eigenmaps (CLEM)

A cell can be considered as a collection of numbers, each specifying light intensity. But this collection also specifies the Cartesian coordinates of a point with respect to a set of axes [27]. Therefore, any cell image can be identified with a point in an abstract image space.

The second classifier is based on preserving the similarities among the images of CB/non-CB cells by using Laplacian Eigenmaps (LEM). LEM is a nonlinear dimensionality reduction method which approximates the lower-dimensional manifold embedded in the abstract image space, while preserving the intrinsic spatial relationship among cell images [28, 29]. In that way, it overcomes the limitation of linear dimensionality methods in which the low dimensional space is approximated by projecting the data onto the eigenvectors. The preservation of spatial relationship in LEM makes it relatively insensitive to outliers and noise.

In order to achieve spatial relation preservation, an undirected, weighted graph math formula of cells is constructed. This is the most crucial step of the LEM algorithm. The cell images constitute the nodes, or vertices ( math formula) of the graph and the approximate similarities among the cells are represented by the edges ( math formula) of the graph. Weights are assigned to the edges based on the similarity values among the images of the cells, resulting in a weight matrix math formula. Two metrics were considered, a dissimilarity measure (the Euclidean distance metric) and a similarity measure (the linear cross-correlation coefficient metric), both preferred for their simplicity.

Given math formula number of images of CB and non-CB cells math formula in the abstract image space math formula, we construct a weighted graph with math formula nodes, one for each image, and the set of edges connecting neighboring images to each other. An edge is put between nodes math formula and math formula if math formula and math formula are “similar.” To define similarity, typically either the math formula-neighborhoods or the math formula-nearest neighbors approach is used. In math formula-neighborhood approach, the similarity is defined by a threshold math formula without taking into account the number of the neighbors that one node might have. In the math formula-nearest neighbors approach, on the other hand the math formula most similar images are chosen as the most similar neighbors. The former approach is more geometrically motivated but may lead to several connected images, in case of a bad choice of a low threshold math formula. Although the math formula nearest neighbors is less geometrically motivated, it is simpler in implementation. Additionally, in the math formula-nearest neighbors approach, an isolated node might end up having neighbors that have in fact very low similarity with it. In order to define similarity, we use a combination of the math formula-neighborhoods approach and the math formula-nearest neighbors approach. In our method the math formula node is connected to the math formula node if it is in the group of its math formula ( math formula=10) closest neighbors (and vice versa), but also satisfies the following criterion:

display math(11)

where math formula is either the Euclidean distance or the cross-correlation coefficient metric, math formula with math formula being the mean score of the Euclidean distances or the cross-correlation coefficients, depending on the metric used, among all the training images. In our case, the cross-correlation coefficient was chosen empirically as the metric to reflect the similarities among the cell images, as it provided better classification results. In the end, our approach is a hybrid of math formula-neighborhoods and the math formula-nearest neighbors approaches. The value of math formula was determined experimentally as 1.15 with the assumption that the threshold should be close to the value math formula. Therefore, we searched the optimal value of threshold in the range of math formula math formula in incremental steps of 0.05.

Moreover, the weights are assigned by a heat kernel:

display math(12)

By using the heat kernel approach while assigning weights to the edges of a graph, we end up in a similarity preserving character of classification that emphasizes the natural clusters in the images. Equation (14) holds only for small math formula so as the heat kernel to become increasingly localized and be characterized by Dirac's δ-function [30]. Therefore, this approach is limited by the necessity to choose a value for the parameter math formula. In our work, the value for math formula was chosen empirically as 5.

After constructing the graph, the next step of the method is the creating of the manifold on which the cell images reside. The manifold is created by connecting image-nodes based on their similarity and it spans a lower dimensional space. The problem of defining the manifold can be considered as generalized eigenvector problem. It can then be stated as a problem of finding the eigenvectors that satisfy the following equation:

display math(13)

where math formula are the generalized eigenvectors, math formula are the generalized eigenvalues, math formula is the diagonal weight matrix with its entries being column (or row since math formula is symmetric) sums of math formula( math formula) and math formula is the Laplacian matrix or the Laplacian operator of the graph. The use of the Laplacian operator of the graph in order to compute the eigenvectors is the key to achieve the optimal embedding of the manifold. As proved in Ref. 30 the Laplacian operator of the graph is an approximation of the Laplace-Beltrami operator defined on the manifold. The use of the Laplacian operator of the graph results in the optimal embedding of the manifold; therefore, its intrinsic geometric structure is reflected.

The ordering of the eigenvectors is based on the criterion of mapping the weighted connected graph math formula to a line so that two nodes math formula, which are connected to each other, stay as close as possible after the mapping. Let math formula be the map from the graph to the real line. Then, the optimal eigenvectors are computed based on the minimization of the following objective function:

display math(14)

As proven in Ref. 30, for any math formula,

display math(15)

Therefore, the vector math formula that minimizes Eq. (14) is given by the minimum eigenvalue solution to the generalized eigenvalue problem of Eq. (13). In the end, the optimal eigenvector is the one that satisfies the following equation:

display math(16)

where math formula is a constraint to remove an arbitrary scaling factor in the embedding and math formula is a second constraint applied to prevent the collapse of all vertices of math formula onto the real number 1. This might happen in case of a connected graph with the value of 1 assigned to each vertex. It is easy to see that 1 is an eigenvector with eigenvalue 0 and in case of a connected graph, is the only eigenvector for… Therefore, this constraint of orthogonally should be added. The solution math formula is finally given by the eigenvector with the smallest non-zero eigenvalue.

Having computed the eigenvectors that consists of the low dimensional space spanned by the manifold, the classification is achieved by computing the Euclidean distance among the embedded cell images on the manifold (Fig. 4). We use the math formula-nearest neighbors concept ( math formula) to achieve a more accurate estimation of the “closeness” between two embedded images. Then we create two distance variables, math formula for the Euclidean distance of any tested image from CB cells and math formula corresponding to the closest images of non-CB cells from the training set. The classification criterion can be then summarized as:

  • Classification algorithm of method 2:

  • Training: For the training set of images of known CB and non-CB cells, create the manifold in which the images lie by computing the optimum generalized eigenvectors of the Laplacian equation.

  • Classification: For the jth image, if math formula then image j belongs to CB class otherwise image j belongs to non-CB class., where math formula and math formula the Euclidean distance of image j from the closest to it images of CB cells and non-CB cells from the training set respectively.

Figure 4.

The cell images (CB image cells shown in a hue of blue, whereas Non-CB image cells shown in a hue of red) that lie on the three dimensional manifold (it is consisted of the three first eigenvectors) and the tested image (shown in black) which resides on the top left of the manifold. This specific tested image cell would be classified as a CB image cell since it resides closer to CB image cells. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Classification Approaches Implemented for Comparison

For comparison purposes, in addition to our two classification methods, eight more (both supervised and unsupervised) classification methods were implemented. Two of these also use dimensionality reduction in order to reveal the most discriminating features of images of CB and non-CB cells. The difference among all the implemented classification methods is the space on which classification is performed, which is either the image or a low-dimensional feature space and the final decision for the classification is based on the similarity criteria.

Unsupervised methods

Euclidean distance (ED)

The simplest way to classify the cells is using distance (dissimilarity) or similarity metrics. The most commonly used dissimilarity metric is the Euclidean distance between the corresponding pixels in two images [28, 48]. The classification in that case is the same approach as of the math formula-nearest neighbors. We experimentally determined the value of math formula. Therefore, each test image is assigned a class label based on the label of its three nearest neighbors.

Correlation coefficient (CC) and mutual information (MI)

As for other similarity metrics, the cross-correlation coefficient, as well as the mutual information (MI) [31] between two images of cells were examined. Again math formula nearest neighbors approach was used, experimentally setting math formula equal to 17 and 5 for cross-correlation coefficient and MI methods, respectively. As MI is based on entropy, which is a measure of histogram dispersion, this metric is intensity-based, in contrast to the Euclidean distance and cross-correlation coefficient, which are both color-based measures.

k-means

This is one of the most commonly used clustering methods and falls into the general category of variance-based clustering [32, 33]. Given the set of math formula images of cells, in real math formula-dimensional space math formula( math formula), and an integer k that defines the number of classes considered for classification (i.e. in our case math formula corresponding to CB and non-CB cells), the problem is to determine a set of math formulapoints in math formula, called centers, so as to minimize the mean squared distance from each image to its nearest center [34].

Fuzzy C-means(FCM)

FCM is based on minimization of the following objective function:

display math(17)

where math formula is weighting exponent for math formula which controls the fuzziness of resulting clusters and can be any real number greater than 1, math formula is the degree of membership of math formula image in the cluster j, math formula is the ith of math formula-dimensional images (N images in total), math formula is the math formula-dimensional center of the cluster (C classes in total) and ||*|| is any norm expressing the similarity between any measured data and the center, which in our case was the Euclidean distance among the images. Each test image is assigned a class label based on the membership function [35, 42].

Supervised methods

Classification by linearly projection in low-dimensional space (CLPLDS)

SVD was used again in another, simpler way for dimensionality reduction and cell classification. Using all the training set of images of both CB and non-CB cells, we calculate the eigenvectors that span the orthogonal eigenspace of all the images of both kinds of cells, and we then project the unknown image on this low dimensionality eigenspace. Again, the eigenvectors were ordered in a sequence of the descending eigenvalues. The projection of the image onto the subspace spanned by only a few eigenvectors would well characterize the cells into the CB and non-CB. Classification was achieved by calculating the dissimilarity among the projected images, using the Euclidean distance. The smaller the score of the Euclidean distance among the projected unknown image and the projected image of CB from the training set, the more chances the unknown image to be a CB (and vice versa when comparing to a non-CB from the training set). math formula-nearest neighbors ( math formula) approach was once again used in order to estimate the closeness of each unknown image to each class of cells.

Classification using linear discriminant analysis (CLDA)

Since LDA is a data classification rather than a feature extraction method (as SVD), it was used along with SVD in order to maximize the ratio of between-class variance to the within-class variance in the training data set, guaranteeing in that way maximal separability. The projected training and testing images in the eigenspace spanned with the eigenvectors calculated by SVD were used as input to the LDA algorithm. The measure of closeness is the log of the probability density function of the testing image, multiplied by the class' prior probabilities. In the same time, the probability density function of the testing image is based on the number of the neighboring to it images in each class. LDA then assigns labels (CB, non-CB) to each of the testing images of cells, classifying them to one of the two classes. Using LDA, the distribution of the features extracted through SVD is better differentiable.

Classification by optimizing on the least squares objective function with a penalty on the L1-norm of the parameters (Lasso)

Bridge regression, a special family of penalized regressions of a penalty function math formulawith math formula is considered in this work as a classifier for CB and non-CB images. Consider the linear regression problem math formula, where math formula is a math formula-dimensional test image, math formula is a math formula-by- math formula matrix formed by concatenating the training CB or non-CB cell images, math formula is the math formula-dimensional vector of parameters and math formula a math formula-dimensional vector of independent and identically distributed random errors. Our goal is to calculate the math formula parameters that minimize the squared error between math formula and then use the calculated math formula parameters to estimate this squared error. The class, in which the squared error is smaller, is the one that better represents the test cell image. We implemented the approach mentioned in Ref. 36 to solve bridge regression for math formula. Particularly they developed a simple algorithm for the lasso by studying the structure of the bridge estimators. The non-differentiability of the objective function in lasso was handled with “Shooting” algorithm which cycles through the parameters in order. The shrinkage parameter math formula and the tuning parameter math formula are selected via generalized cross-validation (GCV).

Quadratic discriminant analysis (QDA) based color features classification

Finally, the performance of a recently proposed quantitative methodology was also examined here [20]. This method was designed specifically for classifying CB and non-CB cells in FL. The method was based on training and testing of a quadratic discriminant analysis (QDA) classifier. The novel aspects of that method were the identification of the CB object with prior information, and the introduction of the principal component analysis (PCA) in the spectral domain to extract color texture features. Both geometric and texture features were used to achieve the classification. The average classification rate of that classifier was equal to 82.56%. We will refer to this method as Color Features Classification (CFC).

Experimental Results

In this section, we first describe the procedure to train both COB and CLEM classifiers and subsequently we present the results of their performance in comparison with the corresponding results of the state of the art classification methods.

Training the Classifiers

In order to train the classifiers, we divided the images of CB and non-CB into training and test sets using an 80% (training): 20% (test) ratio. Due to the higher dimensional feature vectors with limited sample size, the reliability of the proposed methods in extracting the most discriminant features can be questioned. Therefore in order to provide a better estimation of the performance of the tested methods, the pre-mentioned division was repeated K times to yield the “Hold-out K-folds” cross-validation approach [37]. This is considered as the outer loop of the procedure.

The CB and non-CB images allocated for training were further subdivided into 90% to 10% portions, resulting in an inner loop. The division is again repeated K times. During each iteration of the inner loop, the optimal number of eigenvectors and the optimal manifold of embedded training images of CB and non-CB classes were extracted based on the optimal number of generalized eigenvectors while training COB and CLEM, respectively. The optimal number of eigenvectors during training of COB and the optimal manifold during training of CLEM are chosen based on classification result of the 10% subtesting images. Moreover, in case of training COB, all the training images (i.e. 80%) are used at the outer loop, whereas in case of training CLEM, only the 90% of the training images are used at the outer loop.

Empirically, K is set to 10 in both approaches. An overall flow chart of the proposed classification scheme can be viewed in Figure 5.

Figure 5.

Flow chart of the proposed scheme for classification of tested image cells into CB and non-CB classes. The classification method used was either COB or CLEM, both extracting discriminatory features used in the classification of the tested image cells.

Demonstration of the Results

In order to evaluate the performance of various approaches described in the Methods section, we used the two commonly used metrics: precision and recall. Furthermore, the overall accuracy was computed to evaluate the classifier.

Precision is defined as the ratio of the number of correctly classified cells to the total number of classified cells:

display math(18)

where math formula (true-positive) corresponds to the sum of the classified CB or non-CB cells and math formula (false-positive) corresponds to the sum of the misclassified CB or non-CB cells.

Recall is defined as the ratio of the number of classified cells to the number of cells expected to be classified:

display math(19)

where math formula corresponds to the sum of the classified CB and non-CB cells and math formula (false-negative) corresponds to the sum of the non-classified CB and non-CB cells.

Overall accuracy (OA) is defined as the ratio of the number of correctly classified CB and non-CB cells to the total number of classified cells

display math(20)

where math formula corresponds to the number of cells correctly classified as CB, math formula corresponds to the number of cells correctly classified as non-CB, math formula is the total number of classified CB cells and math formula is the total number of classified non-CB cells.

Table 1 demonstrates the results from the COB when no noise removal is applied. Table 2 on the other hand demonstrates the results of classification using COB when only standardization was applied, whereas in Table 2, the corresponding results after applying median filtering are demonstrated. Tables 4 through 6 show the corresponding results for the CLEM approach. The results in all six tables correspond to the average performance of each iteration during which the images were divided into training and testing sets with 80% to 20% ratio.

Table 1. Testing performance of COB without standardization and noise removal
 CBNCB 
SetPRPROA
  1. Results from the 80% to 20% testing procedure followed for the evaluation of the performance of COB classifier in case of standardizing the cell images. Ten iterations were performed. P corresponds to precision, R corresponds to recall and OA corresponds to overall accuracy.

179.0789.0780.8580.8580.00
284.0986.0586.9685.1185.56
395.1290.7091.8495.7593.33
492.5086.0588.0093.6290.00
584.2174.4278.8587.2381.11
690.4888.3789.5891.4990.00
794.6081.4084.9195.7588.89
890.2486.0587.7691.4988.89
992.5086.0588.0093.6290.00
1080.3995.3594.8778.7286.67
Mean88.3285.3587.1689.3687.44
Std5.895.914.756.104.19
Table 2. Testing performance of COB after standardization
 CBNCB 
SetPRPROA
  1. Results from the 80% to 20% testing procedure followed for the evaluation of the performance of COB classifier in case of standardizing the cell images. Ten iterations were performed. P corresponds to precision, R corresponds to recall and OA corresponds to overall accuracy.

195.1290.7091.8495.7593.33
284.0986.0584.9685.1085.56
391.1195.3595.5691.4993.33
488.6490.7091.3089.3690.00
597.3786.0588.4697.8792.22
686.6790.7091.1187.2388.89
786.6790.7091.1187.2388.89
895.2393.0293.7595.7594.44
994.8786.0588.2495.7591.11
1090.0083.7286.0091.4987.78
Mean90.9889.3090.4391.7090.56
Std4.503.673.014.422.83

In Table 7 the classification results of the other methods are presented for comparison purposes. In case of the methods CLPLDS, CLDA, Lasso, CFC, COB, and CLEM, which are all supervised methods, the best classification results achieved during 10 testing iterations are presented. It should be noted that the particular pre-processing steps, which improves the performance, are part our proposed methods. Therefore, no preprocessing was applied to the images in case of all the implemented methods. Furthermore, it is interesting to see how the classification depends on the number of terms in the CB-basis or non-CB-basis, in COB, CLDA and CLPLDS. In Figure 6 we show how recall changes depending on the number of terms (up to a hundred) in the two bases.

Figure 6.

Results of CB precision (top), CB recall (second from top), non-CB precision (third from top), non-CB recall (fourth from top), as well as the overall accuracy results (bottom) relative to the number of eigenvectors (or singular images in case of COB) used in the four supervised classification methods that use dimensionality reduction . Here we plot the results of only the first 15 eigenvector, since the rest do not add any discriminative information of the data. Results of methods that use linear dimensionality reduction (CLPLRDS,CLDA and COB) are shown in green, red, and blue respectively, whereas for CLEM (nonlinear dimensionality reduction) results are in black. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Finally, we examined the consistency of COB and CLEM. Consistency in our case is apprehended as the degree of agreement when applying a classification method in different images of a cell, created based on different points inside its body (see Fig. 7a). We examined the consistency of our methods in 80 different cells (40 CB and 40 non-CB). Knowing the optimal center-point of each cell, marked by the pathologist, we tested the methods in images of the cell created based on the closest to this, marking points. Closeness is defined based on a radius around the original center of the cell (see Fig. 7b). Taking into account points which lie up to a radius of 7 pixels away from the original center (in Fig. 7b, Dmax = 7), we ended up with 224 different images of each cell (8 created based on the points that lie 1 pixel away, 16 based on pixels lie 2 pixels away, 24 based on point that lie 3 pixels away, etc).

Figure 7.

(a) Images of a CB cell created based on different points inside its body. The middle image is the image created based on the pathologist's marking. (b) Image of a cell used on the examination of COB and CLEMs consistency. New images of the cell were created based on pixels that lie on the colorful rectangulars shown in the figure (created based on a distance d from the cell's center, shown with a white arrow). [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

The results in the consistency are shown in Figure 8. These results were obtained after applying all the necessary preprocessing steps (noise removal and standardization of the images), as these led to the best results during training COB and CLEM (see Tables 3 and 6). Out of the 40 tested original CB images, 78% and 88% were classified correctly in case of COB and CLEM respectively. The classification results are reasonable considering that 40 images represent quite a small sample (19%) of the total amount of cell images used to derive the results in all the presented tables.

Figure 8.

Average results in consistency of COB (blue) and CLEM (black) for 40 CB cells (top) and 40 Non-CB (middle). Results in consistency are nothing more than the classification accuracy of the new images of the cells, created based on pixels that lie radius pixels away from the original center of the cell (0 point in x-axis). The bars represent the average results and their range represents the standard deviation from this mean value. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Table 3. Testing performance of COB after standardization and noise removal
 CBNCB 
SetPRPROA
  1. Results from the 80% to 20% testing procedure followed for the evaluation of the performance of COB classifier in case of standardizing the cell images. Ten iterations were performed. P corresponds to precision, R corresponds to recall and OA corresponds to overall accuracy.

1100.0095.3595.92100.0097.78
2100.0097.6797.92100.0098.89
3100.0097.6797.92100.0098.89
4100.00100.00100.00100.00100.00
5100.00100.00100.00100.00100.00
6100.00100.00100.00100.00100.00
7100.0097.6797.92100.0098.89
8100.00100.00100.00100.00100.00
9100.0097.6797.92100.0098.89
10100.0097.6797.92100.0098.89
Mean100.0098.3798.55100.0099.22
Std0.001.571.390.000.75
Table 4. Testing performance of CLEM without standardization and noise removal
 CBNCB 
SetPRPROA
  1. Results from the 80% to 20% testing procedure followed for the evaluation of the performance of COB classifier in case of standardizing the cell images. Ten iterations were performed. P corresponds to precision, R corresponds to recall and OA corresponds to overall accuracy.

178.2683.7282.5076.7480.23
275.0076.7476.1974.4275.58
382.3597.6797.1479.0788.37
487.1879.0780.8588.3783.72
578.7286.0584.6276.7481.40
677.7897.6796.8872.0984.89
781.8383.7283.3381.4082.56
888.2369.7775.0090.7080.23
981.0869.7773.4783.7276.74
1071.7476.7475.0069.7773.26
Mean80.2282.0982.5079.3080.70
Std5.079.878.586.804.56
Table 5. Testing performance of CLEM after standardization
 CBNCB 
SetPRPROA
  1. Results from the 80% to 20% testing procedure followed for the evaluation of the performance of COB classifier in case of standardizing the cell images. Ten iterations were performed. P corresponds to precision, R corresponds to recall, and OA corresponds to overall accuracy.

187.5081.4082.6188.3784.89
287.5097.6897.3786.0591.86
382.6188.3787.5081.4084.89
485.3781.4082.2286.0583.72
587.5081.4082.6188.3784.89
690.4888.3788.6490.7089.54
781.4081.4081.4081.4081.40
8100.0076.7481.13100.0088.37
982.9379.0780.0083.7281.40
1085.0079.0780.4486.0582.56
Mean87.0383.4984.3987.2185.35
Std5.336.255.395.403.52
Table 6. Testing performance of CLEM after standardization and noise removal
 CBNCB 
SetPRPROA
  1. Results from the 80% to 20% testing procedure followed for the evaluation of the performance of CLEM classifier in case of standardizing the cell images and applying median filtering on top. Ten iterations were performed. P corresponds to precision, R corresponds to recall and OA corresponds to overall accuracy.

1100.00100.00100.00100.00100.00
295.56100.00100.0095.3597.67
397.73100.00100.0097.6798.84
4100.00100.00100.00100.00100.00
597.5693.0293.3397.6795.35
6100.00100.00100.00100.00100.00
7100.0097.6797.73100.0098.84
8100.00100.00100.00100.00100.00
9100.00100.00100.00100.00100.00
10100.00100.00100.00100.00100.00
Mean99.0899.0799.1199.0799.07
Std1.582.252.151.631.53
Table 7. Classification performance of previous and our proposed methods
 Performance (%)
 CBNCB 
ClassifierPRPROA
  1. Classification results from the various methods examined in this study. Each classifier uses a different method. ED holds for Euclidean distance, CC holds for crosscorrelation, MI holds for Mutual Information, FCM holds for Fuzzy C-means, CLPLDS holds for Classification by Linear Projection in Low Dimensional Space, CLDA holds for Classification using Linear Discriminant Analysis, CFC holds for Color Feature-based Classifier, COB for Classification using Orthogonal Bases and CLEM holds for Classification based on Laplacian Eigenmaps. P corresponds to precision, R corresponds to recall, and OA corresponds to overall accuracy.

ED85.560.571.590.576
CC8687888787
MI83235895.561
k-means53605952.556
FCM4141464643.5
CLPLDS88.591918990
CLDA9195.595.591.593
Lasso9360.572.59679
CFC8795958791
COB100100100100100
CLEM100100100100100

On the other hand, the classification results in case of non-CB were slightly higher (90% and 85% for COB and CLEM respectively). Furthermore, one can notice that both in the cases of CB, as well as non-CB the classification accuracy is always above the threshold of 50%, even for 7 pixels away from the original center of the cell. On the contrary, it remains consistent (almost equal to the accuracy in the original image) for up to 2 pixels away from the center of the cell. Finally, the significant drop in accuracy in the Figure 8 of non-CB cells in large distances from the cell's center (radius of 6 and 7) can be reasonably attributed to the small size of a non-CB. Due to that, there is a high chance that 6 or 7 pixels away from the original center of the cell are not part of its body anymore.

Based on the results in Figure 8, we thought of examining the accuracy of our methods, when taken into account the classification results of images of the cell created by the closest (up to 7 radius distance) to original center pixels. In Figure 9 we see that, even considering the results of points lying 3 pixels away from the center, accuracy still remains high and close to the one of the original cell image, both for when applying COB or CLEM. Moreover, this holds for both CB and non-CB cell.

Figure 9.

Results of classification's accuracy in COB (blue) and CLEM (black) for 40 CB (top plot) and non-CB (bottom plot), when taking into account the results of the cell's images created based on the close to the cells' center pixels in various radius away from it. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Discussion

The results in Tables 1 to 6 indicate the effectiveness and reproducibility of our proposed methods for the CB and non-CB cell classification. They are effective because of high recall and precision rates. They are reproducible because changing the training and test sets result in similar performances. We observed that the subspaces spanned by PCA of each cell class did not change drastically for different training sets, as this would result in low precision and recall results in some iterations. Therefore, high-dimensional, low-sample size (HDLSS) did not have a negative impact for this particular problem. These tables also demonstrate the usefulness of the steps to remove the noise existed in the images (i.e. standardization and median filtering). Preprocessing the input image to COB and CLEM with the median filtering increased the average overall accuracy rate by 11.78% and 18.37% while reducing the variance by 3.44% and 3.03%, respectively. These results justify our initial hypothesis that FL images suffer from noise due to inconsistencies in the preparation phase. Therefore, noise removal is necessary. Although we used median filtering for this study, other non-linear filters with similar characteristics such as anisotropic diffusion filter (ADF) can also be considered. ADF has the advantage of preserving the boundaries of the cells in the image, however, this kind of filter is computationally expensive.

Time efficiency is very important in this application as the pathologist has to examine huge images for every individual patient. Time efficiency is another quality of both COB and CLEM, since only 750 ms and 900 ms are required for the classification of an unknown cell respectively. It should be noted that Kyttaro was developed in Matlab ® (Mathworks, Natick, MA) and it was not optimized for efficiency.

The results in Table 7 indicate that our methods outperform some of the commonly used or previously developed methods. Those methods use only features extracted from the image while our methods utilize the whole images in the decision. Moreover, the results show the importance of dimensionality reduction to reveal the most discriminative features. As expected, the methods that used this kind of approach (CLPLDS, CLDA, COB, and CLEM) showed the four best results among all the methods.

Furthermore, the results from Figure 8 indicate the consistency of our two proposed methods with some tolerance to the selection of the cell location by the pathologist. These results show that the pathologists do not need to point to the exact same location to receive the same results. Not every pathologist would identify the same exact pixel in the HPF FL image as the center of that cell. In fact, as discussed later, intervariability among pathologists is quite common. In addition to that, the shape of the cells is not uniform. Therefore based on the results in Figure 9, we concluded that it is better to take into account not only the marking of each pathologist, but also the points around it in a radius to 3 pixels away, without risking on the accuracy of the result. Experimental results indicate that CLEM method is very susceptible to the values of its parameters. Therefore, the low recall results for non-CB cells may be due to non-optimal selection of its parameters. Optimization of the parameters of different methods is beyond the scope of this study. In our future work, we will optimize the parameters of our system and validate them on a larger dataset using some optimization techniques [38].

In order to demonstrate the usefulness of the system, Belkacem-Boussaid et al., used the input of two experienced pathologists to complete two different experiments on the test set data [20]. In the first experiment the pathologists graded the images without the assistance of our proposed CAD algorithms, whereas in the second experiment, they displayed the results of the designed CAD on the same data set and asked them if either they agreed with the results or not. In case there is a disagreement of judgment between the pathologist and the CAD algorithms, the pathologist were free to change the grading. It was also noted that these experiments were administered on different days for each pathologist. These experiments revealed inter- and intrareader variability between the grading of the two pathologists. They noticed that the pathologists are not in concurrent in all cases. Inter- and intra-reader variability errors are introduced during their subjective reading. This study revealed that the intra- and inter-reader variability can be more than 25%.

Conclusions

In this article, we developed a new quantitative methodology to classify cells in a FL image into one of two categories of cells (CB vs. non-CB) using linear and nonlinear dimensionality reduction.

The proposed methods could be proven to be a useful diagnostic tool that facilitates pathologists in accurate and reproducible FL grading. Moreover, Kyttaro could be beneficial to inexperienced pathologists by retrieving the most similar cells to the ones they have chosen to examine. The proposed methods also incorporate the features mentioned by an expert pathologist while performing classification. The performance of the developed methods was compared with some commonly used and previously developed methods. The current overall classification rates (99.22% ± 0.75% and 99.07% ± 1.53% obtained by COB and CLEM classifier, respectively) are very encouraging considering inter and intra-reader variability of the pathologists which is over 25%.

As a future work, the presented methods will be further refined, so that the selection of the cells in Kyttaro will be done automatically, instead of a manual selection by a user. This will be done by integrating Kyttaro with previously developed algorithms for the detection of cells from FL images [16, 18-22]. This integration needs to be carried out in such a way to take advantage of the common features of both systems to improve the classification accuracy and efficiency. Moreover, we will further develop an overall automated system for the quantitative analysis of the pathological images of FL, by incorporating the classification methods proposed in this study. Our vision is to translate these efficient classification methods to clinical use. Toward this vision, computer-assisted systems will be regularly used in clinical practice for cancer grading and prognosis. Such a system would have whole-slide FL images as input and the analysis of the images will be done in different scales, starting from a coarse level, in which follicles need to be segmented. Then, moving to a finer scale, HPF images, created around these follicle regions, will be analyzed after automatically detecting the cells in them. After the identification of the cells, the region around each cell would be encoded by one of our proposed methods in order to assess whether the cell belongs to a CB class or a non-CB class. Depending on the number of CB cells computed in the image, a grade for whole image would be assigned. We are in the process of setting up a clinical trial to assess the effectiveness of this system.

Acknowledgments

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Cancer Institute, or the National Institutes of Health.

Ancillary