SEARCH

SEARCH BY CITATION

Keywords:

  • image analysis;
  • cell segmentation;
  • optimization;
  • graph cut;
  • fast marching level set;
  • fluorescence imaging;
  • macrophage spreading

Abstract

  1. Top of page
  2. Abstract
  3. OBJECTIVE
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONCLUSION
  8. SOFTWARE AND DATA AVAILABILITY
  9. Literature Cited

To evaluate macrophage spreading in immunofluorescence images of macrophages for surface protein CD11b and nuclear counterstaining with DAPI, it is necessary to measure the size of the macrophages at different time points after stimulation. Manual evaluation of fluorescent micrographs is usually a time-consuming and error-prone task, with poor reproducibility. Automatic image analysis methods can be used to improve the results. The quality of the analysis with these methods mainly depends on the quality of the image segmentation. A segmentation and quantification scheme based on shading correction, k-means clustering, and fast marching level sets has been developed for the purpose. An initial application of this approach showed that separating touching and overlapping cells in particular suffers severely in the inevitably blurred conditions, leading to partly erroneous measurements of macrophage spreading. An alternative method of segmentation in fluorescent micrographs was therefore investigated and evaluated in this study. The proposed approach uses a methodology that separates foreground objects from background objects on the basis of Boykov's graph cuts. In this process, a rough estimation of background pixels is used for background seeds. To identify foreground seeds, a difference of Gaussian band pass filter based workflow is developed. Information on foreground and background seeds is then used for a gradient magnitude based graph cut resulting in a robust figure–ground separation method. In addition, a fast marching level set approach is used in the post-processing step, which makes it possible to split touching cells by incorporating information about the cell nuclei. An evaluation based on a total of 553 manually labeled macrophages depicted in 21 micrographs showed that the proposed method significantly improves segmentation and splitting performance for fluorescent micrographs of LPS-stimulated macrophages and reduces the rate of error in automated analysis of macrophage spreading in comparison with alternative methods. © 2013 International Society for Advancement of Cytometry


OBJECTIVE

  1. Top of page
  2. Abstract
  3. OBJECTIVE
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONCLUSION
  8. SOFTWARE AND DATA AVAILABILITY
  9. Literature Cited

To evaluate macrophage spreading in immunofluorescence images of macrophages for surface protein CD11b and nuclear counterstaining with 4,6-diamino-2-phenylindole (DAPI), it is necessary to measure the size of the macrophages at different time points after stimulation. Unless semi-automatic image analysis tools are used, each macrophage has to be manually delineated to allow measurement and quantification of its size. Automated image analysis methods are needed in order to speed up this very time-consuming and error-prone step of analysis.

Figure 1 shows a typical sequence for macrophage spreading at several time points (t = 0 (control), t = 15 min, t = 1 h, t = 2 h, and t = 24 h) and the corresponding ground truth, illustrating the challenge represented by increasingly touching and overlapping cells.

thumbnail image

Figure 1. Typical temporal sequence of macrophage spreading at points of time: t = 0 (control) (A,F), t = 15 min (B,G), t = 1 h (C,H), t = 2 h (D,I), and t = 24 h (E,J), and the corresponding ground truth depicting the challenge of increasingly touching and overlapping cells. Ground truth cells crossing the image boundary and cells that could not be outlined manually were assigned to a rejection class (red cells). Scale bar: 10 μm.

Download figure to PowerPoint

In this type of automated image analysis system, cell segmentation (separating and delineating the depicted cells from the image background and from each other) is the most challenging part of the process, as macrophages may touch each other or even overlap, especially with increasing time steps. In extreme cases, these overlaps cannot even be resolved correctly by experienced biologists. In particular, separating cells becomes even more challenging if the macrophages are out of focus or appear blurred, or have low contrast relative to the image background. To facilitate automated cell segmentation in micrographs, and in particular to support the splitting of macrophages for further analysis, additional DAPI staining is used to visualize and localize the cell nuclei. This information can then be used to segment the macrophages, both by humans and machines.

State of the Art

Various approaches have been proposed in the literature for segmentation of fluorescent micrographs incorporating information about the position and extension of the cell nuclei. Most of these methods use a process known as the seeded watershed transform (WT). Palmieri et al. (1) describe such a scheme for the analysis of cellular phosphatidylinositol (3, 4, 5)-triphosphate levels and distribution in HEK293T and LIM1215 cell lines. For segmentation of the nuclei, a threshold is selected interactively. Cells are then segmented using a seeded WT. Lindblad et al. (2) describe a fully automatic scheme for segmentation of Chinese hamster ovary (CHO) nuclei and cells. A contrast-based threshold selection method is applied to segment the nuclei. The resulting binary image is then distance-transformed. Next, touching or overlapping nuclei are separated by applying a WT to the distance-transformed image. After segmentation of the nuclei, CHO cells are segmented by applying a seeded WT. A very similar scheme has been described by Bengtsson and Wahlby (3) for segmentation of tumor cells stained with green fluorescent protein (GFP).

A different approach for incorporating nucleus information into the segmentation of fluorescently labeled cells is based on level sets (LS). Yan et al. (4) used a modified WTation for extraction of the cell nuclei. For segmentation of the cells, they used independent LS as well as LS that include an interaction model based on the cell nuclei. The evaluations clearly demonstrate that independent LS are not suitable for this segmentation task. Including the interaction model allows high-quality segmentation of the cells. Yu et al. (5) combined the LS method with topology-preserving constraints that prevent cells from merging or splitting. This method was improved using Voronoi diagrams (6). Srinivasa et al. (7) described an active mask-based scheme for segmentation of HeLa cells and the Golgi body. The same group (8) demonstrated that the active mask-based scheme outperforms the seeded WT for segmentation of fluorescence cells. A scheme based on fast marching LS for splitting of touching and overlapping macrophage cells has been described by Held et al. (9). Wittenberg et al. (10) made multiple (global and local) use of Otsu's thresholding approach for segmentation of cervical cell nuclei in DAPI-stained images, the location and extension of which were used to delineate and separate strongly overlapping and touching Papanicolaou-stained epithelial cells using a region-growing approach. Using the cell nuclei as indicators of cell centers, overlap regions between two adjacent cells were identified and resolved using a densitometric model.

Proposed Approach

To allow automated analysis for measurement of Cd11b/APC-stained and DAPI-stained micrographs in various time steps, we propose a chain of conditioning (preprocessing) and segmentation steps that allows automated measurement and quantification of the macrophage size at several time points. Specifically, we propose a segmentation approach for the cell nuclei in the DAPI channel consisting of a combination of k-means clustering, distance, and WT (11). Information about the size, location, and extension of nuclei is then used to initialize segmentation of the macrophages in the Cd11b/APC staining. We have shown in previous studies (9) that an approach for segmenting and separating macrophage cells in the Cd11b/APC channel based on Gaussian smoothing, k- means clustering, and fast marching LS is possible. As shown by Held et al. (9), this approach also has limitations with increasing overlaps among neighboring cells, as splits between adjacent cells tend to produce a leakage phenomenon (Fig. 2). In addition, the threshold selection method based on k-means clustering fails for images with strong blurring or illumination artifacts. Even if a background correction step is applied, the accuracy of the threshold selection remains poor due to noise and blur (Fig. 2B).

thumbnail image

Figure 2. Blurred macrophages (A) are segmented using k-means clustering–based figure–ground separation (B) and graph cut–based figure–ground separation (C). Using the k-means clustering–based approach, blurred cell boundaries result in oversegmentation of the image (white arrows), which causes leaking out of the subsequent FMLS–based cell splitting method. Improved separation of foreground and background (C) reduces leakage and hence improves the splitting of adjacent cells. Scale bar: 10 μm.

Download figure to PowerPoint

We have therefore now developed and evaluated a new approach based on Boykov's graph cut (GC) algorithm (12–15). The only study that to our knowledge has previously used a graph cut approach for segmentation of fluorescent micrographs was by Maška et al. (16), who used GCs for segmentation of touching cell nuclei. To separate the nuclei from the image background, an initial graph cut (12) using Riemannian metric-based edge capacities and histogram analysis is applied and a binary image is obtained. For separation of cell clusters, a distance transform is applied to the binary image and peaks are detected. A second multilabel graph cut is then used to split touching cells. Although the study by Maška et al. (16) shows that the graph cut method is flexible and offers boundary accuracy, the method cannot be applied directly to macrophage cells with a more complex morphology or varying intensities, as in Figure 1. This study therefore describes and evaluates a graph cut based on an alternative energy functional for the separation of fluorescent macrophages from the image background (figure–ground separation), which can be used as an alternative to k-means thresholding. Figure 3 illustrates the resulting image-processing workflows for segmentation of nuclei and cells.

thumbnail image

Figure 3. Workflow for segmentation of cells and nuclei. It should be noted that the novel graph cut–based method replaces previous k-means clustering–based figure–ground separation of the Cd11b/APC stained macrophages.

Download figure to PowerPoint

MATERIALS AND METHODS

  1. Top of page
  2. Abstract
  3. OBJECTIVE
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONCLUSION
  8. SOFTWARE AND DATA AVAILABILITY
  9. Literature Cited

Image Data and Image Acquisition

The dataset consists of 21 CD11b/APC-stained and DAPI-stained images of murine bone marrow macrophages that were stimulated with LPS. The images were acquired with a Zeiss Axiovert microscope. To obtain a reliable ground truth, 553 macrophage cells were manually annotated by an experienced user. Cells touching or exceeding the image boundary or cells whose boundaries could not be determined correctly by the human user were labeled and assigned to a rejection class (Fig. 1). These macrophages were excluded from comparison of the algorithms described.

Methods Overview

A multichannel segmentation scheme is used for segmentation of the CD11b/APC-stained and DAPI-stained macrophages. Figure 3 provides an overview of the workflow.

Preprocessing/conditioning.

To enhance the image quality and reduce noise and artifacts in the images, a two-step preprocessing chain is applied before segmenting the CD11b/APC-stained cell or DAPI-stained nuclei images. The preprocessing starts with a shading correction by subtracting a minimum filtered image with radius rpre from the original image. A Gaussian smoothing filter with radius σpre is then applied for noise level reduction.

Nuclei segmentation.

First, segmentation of the cell nuclei is carried out in the DAPI channel. After preprocessing (for details, see previous section), k-means clustering based on image intensities is applied to binarize the nucleus images. The darkest cluster is therefore interpreted as the image background. Touching or overlapping nuclei are then separated by a WT on the distance-transformed binary image. When this method is used to split touching nuclei, the convex shape of the nuclei is used for splitting.

Macrophage segmentation.

The macrophages are segmented using two different segmentation schemes, in which the first scheme serves as a reference and the second denotes the proposed approach. Both segmentation schemes apply a common preprocessing chain; the reference segmentation approach uses a k-means clustering–based threshold selection method (9). A graph cut–based figure–ground separation method is now proposed as an alternative.

Graph Cut–Based Figure–Ground Separation

Described below is a new approach for figure–ground separation of fluorescent micrographs based on the graph cut approach as defined by Boykov et al. (12–15). The graph cut method provides a framework that can be used for image segmentation by minimization of an energy function. For each pixel p contained in a set of connected pixels P and a neighborhood system N(P) containing a set of neighboring pixels {p,q}, a cost function E(A) is minimized. In this process, A describes a binary partitioning of the image, whereas Ap and Aq denote the labels of a specific pixel p or q. In this contribution, the pixel set P contains every pixel in the input image and N denotes a four-neighborhood system. Solving the graph cut corresponds to minimization of the following energy function:

  • equation image

where Rp (A) denotes a penalty for the assignment of a pixel p to a label l out of a set of labels L. For figure–ground separation, the two labels “foreground” and “background” (or figure and ground) are considered. The smoothness term Esmooth(A) denotes a penalty assigned for a discontinuity between p and q, whereas δ(Ap,Aq) denotes a Dirac function that is 1 if ApAq and 0 otherwise.

For this cut to be used for segmentation, the functions Edata (A) and Esmooth (A) have to be chosen in an appropriate way. We describe below a methodology for estimating both terms on the basis of image data that allows robust figure–ground separation even in blurred images with illumination artifacts.

Smoothness term.

For this application, the macrophage cells are separated from the image background using the locations of strong gradients. Hence, the smoothness energy term Esmooth (A) depends on the gradient magnitude ∇I, which is normalized to a range of [0, 1]. The resulting image is denoted as ∇I[0,1]. Including an exponential weighting factor α > 0 leads to:

  • equation image

The ∇ operator is implemented by using a differential of Gaussian filter kernel with standard deviation σdog.

Data term.

The data term Rp evaluates how well a specific labeling fits to a given data model. In this application, Rp is defined by a set of foreground and background (figure and ground) pixels that are very likely to be part of foreground or background objects. The penalty term Rp is therefore set to 1 if p is a foreground seed, but is assigned to the class background by the graph cut. Similarly, Rp is also set to 1 if p is a background seed but is labeled as foreground by the graph cut. If p is neither a foreground nor a background seed, nor erroneously assigned by the graph cut, Rp is set to 0. An overview of the workflow of graph cut–based figure–ground separation is provided in Figure 4.

thumbnail image

Figure 4. Workflow for the graph cut–based figure–ground separation method. For an input image (A), background seeds (B) are automatically estimated. A difference of Gaussian (DoG) filter (C) is applied to identify potential foreground seeds. Foreground seeds (E) are obtained by setting negative values to 0 (D), threshold selection, and removal of small objects. On the basis of these seeds, a graph cut is applied for figure–ground separation of the macrophages (F).

Download figure to PowerPoint

Identification of Background Seeds for the Graph Cut Data Term

The background seeds can be identified by analyzing global image intensity values. To improve this method, the previously described preprocessing chain is applied, reducing both the noise level in the image as well as blurring and illumination artifacts.

It should be noted that it is crucial that the identified background seeds should most likely belong to the image background, since otherwise the graph cut method may be misled. This is why a conservative threshold is applied for identification of the background seeds. This step is implemented using a k-means clustering–based threshold selection method with a high number of clusters, which can be adjusted to the image data and is usually between 5 and 20. After the clustering, the darkest cluster is assumed to represent background seeds (Fig. 4B). In contrast to Otsu's threshold selection, for example, k-means clustering offers sufficient flexibility to ensure that only background pixels are selected.

Identification of Foreground Seeds for the Graph Cut Data Term

The identification of foreground seeds poses a challenge, as the intensity distribution may vary for each cell. This is why global threshold selection methods fail with this task. A further challenge is represented by blurring or illumination artifacts, which make it more difficult to identify the macrophages.

Instead of using global intensity features, we propose selecting pixels that are relatively bright relative to the image background. To estimate the image background, the input image I is smoothed with a strong Gaussian filter function with σstrong. In order to select bright objects and reduce liability to noise, this difference imaging scheme is extended to a DoG-based band pass filter (Fig. 4C) by:

  • equation image

where Ibp denotes the band pass filtered image and σweak and σstrong represent the standard deviations of Gaussian filter kernels with σweak < σstrong. By removing pixels with a negative filter response, a result image Imath image is obtained (Fig. 4D).

Separation of relevant cell pixels from background and noise structures in the DoG-filtered image Imath image again poses a challenge. As k-means clustering and locally adaptive thresholds fail with this task for varying image data, a nonparametric method incorporating information on background seeds has been developed.

For robust identification of foreground seeds, a dual threshold (17) followed by removal of small objects is applied. For the dual threshold, a low and a high threshold value are determined on the basis of the background seeds identified (Fig. 4B). The lower threshold is chosen in such a way that most (in this study, 98%) of the background seeds exhibit a lower intensity in comparison with the threshold value. Only non-background seeds are analyzed to estimate the higher threshold value. The higher threshold value is then set to the median intensity of the foreground pixels. For both computations, pixels that hold Imath image = 0 are ignored.

Additionally, small objects corresponding to noise pixels or background structures are removed. As the lower threshold value has been chosen in such a way that some background noise is still contained in the image, all objects smaller than the average object size can be removed to obtain the foreground seeds required for estimation of Edata(A) (Fig. 4E).

Using the workflow described above to identify foreground seeds usually does not require adjustment of quantile values to determine lower and higher threshold values and to remove small objects if the image data change. Instead, parameters for DoG filtering are adjusted. Preliminary tests show that this procedure also allows figure–ground separation for micrographs of different fluorescent imaging domains without adjustment of quantile values.

Cell Splitting

After macrophages have been separated from the image background, both image processing workflows (Fig. 3) apply a fast marching level set (FMLS) method for the splitting of touching and overlapping cells. More specifically, the FMLS approach incorporates information on the positions and extensions of the previously obtained cell nuclei. The FMLS was chosen because it allows time-efficient and topology-preserving splitting of adjacent cells. The quality of this splitting step depends on the LS speed function F. For this study, the speed function F combines information on gradient magnitude ∇I, the image background, and the local curvature of the energy front κ. Information on the image background is obtained after k-means clustering–based or graph cut–based figure–ground separation and prevents the FMLS method from running into the image background. This information is stored in a binary image IB that holds IB = 0 for background pixels and IB = 1 for foreground pixels. The method proposed by Nilsson and Heyden (18) has been used to estimate the local curvature of the front. This method efficiently estimates the curvature of the front κ based on the pixels in a local neighborhood and returns κ ∈ [−0.5,0.5] with small values indicating leaking out of the front. Including this term in the speed function reduces the leaking effect of the FMLS method. Combining the pieces of information described results in the following speed function for the FMLS:

  • equation image

Here, α1 ∈ [0,2] denotes a weighting factor adjusting impact of the curvature term. Using this speed function for propagation and the cell nuclei position and extension for initialization of the FMLS, touching or overlapping cells can be split as seen in Figures 2B and 2C.

For further information on the FMLS based cell splitting method, see Held et al. (9).

Performance Metric and Optimization

The modules for k-means clustering and the graph cut–based workflow show several free parameters. To improve automated assessment of the macrophages, each module's free parameters are optimized with respect to a manually provided ground truth by using genetic algorithms (19). The GALib library developed by Wall (20) has been used here. To avoid memorization and thus overfitting, a threefold cross-validation is included. The complete dataset is split into three disjunctive subsets. Two of the subsets are then used for training of the proposed image processing algorithms, while the remaining third subset is used for evaluation. This procedure is repeated for all three combinations of training and testing data. Using this cross-validation scheme, training and testing are always performed on different images.

For evaluation of the segmentation performance, each segmented cell region Si,i = 1,…,n is compared with the nearest and best-matching manually annotated ground truth cell Tj,j = 1,…,m, where the number of segmented cells is denoted with n, and m describes the number of ground truth cells. For evaluation of the segmentation accuracy of each cell, mapping is performed in which the best-fitting segmented cell is searched for the nearest ground truth cell. For the fitting and for performance evaluation, the overlap between two cells, also known as the Jaccard similarity measure Jij (21), is used:

  • equation image

The Jaccard measure yields high values for a good overlap and 1 for complete identity, while low values denote bad overlaps and 0 corresponds to disjunctive regions.

Segmented objects that best fit to boundary cells or cells that could not be accurately delineated (red cells in Fig. 1) are not considered for evaluation. As a result of this, parameter optimization by the genetic algorithm as well as evaluation are not biased by such cells.

RESULTS

  1. Top of page
  2. Abstract
  3. OBJECTIVE
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONCLUSION
  8. SOFTWARE AND DATA AVAILABILITY
  9. Literature Cited

For comparison of the segmentation performance, each module's parameters were automatically optimized relative to the manually annotated ground truth. The resulting measurements showed that the proposed graph cut–based segmentation method outperforms the reference k-means clustering algorithm (Fig. 5). To determine whether the segmentation performance differs significantly from the results obtained with the reference method, a nonparametric Friedman test was applied, which showed a high level of statistical significance at P < 0.0001.

thumbnail image

Figure 5. Comparison of the Jaccard overlap for the k-means clustering–based workflow and the graph cut–based workflow. Using the graph cut–based workflow, the median Jaccard similarity increases from 0.73 to 0.80.

Download figure to PowerPoint

For evaluation of macrophage spreading, it is essential to investigate the way in which these segmentation errors affect the measurements of macrophage size obtained at different time points. The complete dataset was therefore split according to the corresponding times after stimulation (control, 15 min, 1 h, 2 h, 24 h). Box plots for the resulting cell sizes comparing manual annotation, the k-means–based workflow, and the graph cut–based workflow are shown in Figure 6. Analysis of these box plots indicated that the median cell size measured using the graph cut–based segmentation scheme was closer to the median cell size of the manually annotated macrophages. In order to investigate whether the cell sizes observed differed significantly for a specific time point, a statistical Friedman test was performed. This test showed that the measurements obtained were significantly different for the control group, at P < 0.0001. For observations at 15 min and 24 h after stimulation, measurements were significantly different at P < 0.01, whereas 1 h after stimulation, the differences between the segmentation methods were not significant (P > 0.01). To determine whether size measurements can be significantly improved using graph cut–based figure–ground separation, a pairwise Wilcoxon test was performed for each subset of data (Table 1). This test showed that measurements obtained with the graph cut method were more similar to the manually annotated data for all data subsets.

thumbnail image

Figure 6. Automatically determined cell sizes by manual annotation of the data, the k-means clustering–based workflow, and the graph cut–based workflow for the different subsets “control (0 min)” (A), “15 min” (B), “1 h” (C), and “24 h” (D) after stimulation.

Download figure to PowerPoint

Table 1. P values resulting from a pairwise Wilcoxon test for comparison of the similarity of cell sizes obtained
SubsetControl (0 min)15 min1 h24 h
  1. Cell sizes obtained by manual annotation (M) and k-means clustering (KM) and the graph cut-based method (GC) are compared.

M/KM0.130.310.800.5
M/GC0.900.690.800.5

To allow comparison between the figure–ground separation methods described and the state of the art using data from different fluorescence imaging domains, the algorithms described were compared with standard methods provided by the CellProfiler (22), using publicly available image data. For this comparison, a representative training image and a testing image were selected from publicly available 3T3 cells (23), the image set BBBC008v1 (22) available from the Broad Bioimage Benchmark Collection (24), and from fibroblasts obtained from the prototype database (25). For this experiment, the “IdentifyPrimaryObjects” routine in the CellProfiler was used for segmentation of the training images. Each of the figure–ground separation routines provided by the CellProfiler was thus applied. Specifically, the parameters of the segmentation routines were manually adjusted and the segmentation routine offering the best segmentation was selected. Similarly, the parameters were adapted for the proposed k-means clustering and the graph cut figure–ground separation method. After training, figure–ground separation routines were applied to the testing image. The images resulting in this experiment are shown in Figure 7. In addition to the experiments using publicly available fluorescent image data, the experiment described was repeated using our macrophage dataset as well as degenerated macrophage data. The resulting segmentation results are presented in Figure 8.

thumbnail image

Figure 7. Comparison of different figure–ground separation methods based on macrophage data from this study (AH) and macrophage data overlaid with additional noise and illumination artifacts (IP). On the basis of a training image (A,I), the parameters of the segmentation methods can be adjusted until qualitative segmentation does not improve. This results in images (B,J) for the k-means clustering–based workflow, images (C,K) for the graph cut–based figure-ground separation algorithm, and images (D,L) for the CellProfiler. On the basis of the resulting optimal parameterization of the segmentation methods, the testing images (E,M) are segmented using k-means clustering (F,N), graph cuts (G,O), or the CellProfiler (H,P) for figure–ground separation. It should be noted that the CellProfiler figure–ground separation routine “MoG” is used for images (D,H,L,P). Some errors resulting from automated segmentation are highlighted with red arrows. Scale bar: 10 μm.

Download figure to PowerPoint

thumbnail image

Figure 8. Comparison of different figure–ground separation methods using publicly available data from 3T3 cells (AH) (18), the image set BBBC008v1 (IP) (22, 24) and fibroblasts (QX) (1). On the basis of a training image (A,I,Q), the parameters of the segmentation methods can be adjusted until qualitative segmentation does not improve. This results in images (B,J,R) for the k-means clustering–based workflow, images (C,K,S) for the graph cut–based figure–ground separation algorithm, and images (D,L,T) for the CellProfiler. On the basis of the resulting optimal parameterization of the segmentation methods, the testing images (E,M,U) are segmented using k-means clustering (F,N,V), graph cuts (G,O,W), or the CellProfiler (H,P,X) for figure–ground separation. It should be noted that the CellProfiler figure– ground separation routine “RobustBackgroundPerObject” is used for images (D,H), “OtsuGlobal” for (L,P), and “RidlerCalvard” for (T,X). Some errors resulting from automated segmentation are highlighted with red arrows.

Download figure to PowerPoint

The k-means clustering–based method described in this study yields results very similar to those with the methods previously published by Held et al. (9). Nevertheless, the measurements obtained using the Jaccard measure were slightly higher in the previous study (9) for a very similar dataset. This was due to the effect that in the manually obtained ground truth annotations used in the preliminary study, only a subset of cells (421 cells vs. 553 cells used in this study) was annotated to carry out research on macrophage growth. To compare potential errors using the automated methods with manual data analysis, a complete annotation of the macrophage dataset was created for this study. The macrophage dataset, including the ground truth data, can be obtained from the senior author.

DISCUSSION

  1. Top of page
  2. Abstract
  3. OBJECTIVE
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONCLUSION
  8. SOFTWARE AND DATA AVAILABILITY
  9. Literature Cited

Comparison of automated measurements of macrophage size relative to manual measurements for different time points shows that the most significant errors are observed for the control group (t = 0 min). In contrast to this finding, Wenzel et al. (26) observed greater errors for later time points when the cells had become more confluent and were overlapping each other. This apparent contradiction can be explained by differences in the datasets used for the studies. In the image dataset used for the present investigation, the macrophages in the control group (t = 0 min) were among the most challenging micrographs for automatic segmentation, with more out-of-focus cells and stronger overlaps among adjacent macrophages than in micrographs captured 15 min, 1 h, and 24 h after infection.

Our own experiments and those of Wenzel et al. (26) show that overlapping cells are one of the most important reasons for differences between automatically generated and manually generated measurements. As cells are assumed to be touching and not overlapping with most automated segmentation methods, the object sizes measured show a negative bias. These errors do not cancel out if a large population of cells is observed. Further sources of error are induced by objects that are not correctly separated from the image background due to blurring or shading artifacts.

Using the macrophage data to compare the different figure–ground separation methods shows that the graph cut–based segmentation routine allows more robust segmentation of the original macrophage data (Figs. 7A–7H). For these data, the k-means clustering–based approach and the CellProfiler lead to oversegmentation of some cells and are not capable of separating dark cells from the image background. Comparison of the different methods for a degenerated macrophage image that was overlaid with additional noise and illumination artifacts (Figs. 7I–7M) shows that the graph cut–based method outperforms both the CellProfiler and the k-means clustering–based technique.

Comparing the k-means and the graph cut–based approach with the CellProfiler using 3T3 cells (Figs. 8A–8H) (18) shows that the k-means clustering–based segmentation method as well as the CellProfiler result in oversegmentation of the very bright cells, whereas the graph cut–based method is able to perform high-quality segmentation of both the brighter and the darker cells. For the image set BBBC008v1 (Figs. 8I–8P), all of the methods have comparable results, as this dataset can be nicely segmented using Otsu's threshold selection method. For the fibroblast data (Figs. 8Q–8X), all of the methods allow good segmentation of the data, but only the graph cut–based method is able to segment the extensions of the cells correctly.

CONCLUSION

  1. Top of page
  2. Abstract
  3. OBJECTIVE
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONCLUSION
  8. SOFTWARE AND DATA AVAILABILITY
  9. Literature Cited

This study presents a novel segmentation method based on graph cuts that is applicable to fluorescent micrographs. In comparison with a k-means clustering–based method using macrophage data, the segmentation routine presented here shows a significant improvement in segmentation performance. A comparison of the proposed graph cut–based method with k-means clustering and the CellProfiler based on publicly available fluorescent image data showed that the graph cut–based method outperformed the alternative methods for different fluorescent micrographs. In detail, the graph cut method allowed more robust handling of cells showing blurring and low contrast against the image background. In addition, segmentation of micrographs showing cells with strongly varying intensities was also improved. This results in improved segmentation of tiny cell extensions.

The improvement in segmentation performance is crucial for many experiments that require segmentation of the image for evaluation (1, 8, 26, 27). In addition, classifiers using cell shape as a feature benefit from the improved segmentation of cell boundaries and particularly cell extensions.

Future research will involve application of the graph cut–based method to different fluorescent image datasets. Experiments aimed at improving segmentation by including a phase-contrast channel will also be carried out.

SOFTWARE AND DATA AVAILABILITY

  1. Top of page
  2. Abstract
  3. OBJECTIVE
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONCLUSION
  8. SOFTWARE AND DATA AVAILABILITY
  9. Literature Cited

It should be noted that the software is still under development. A dongled evaluation version of the software can be obtained for from the senior author. The macrophage dataset, including the manual cell outlines, has been made publicly available as image set BBBC020 in the Broad Bioimage Benchmark Collection (24).

Literature Cited

  1. Top of page
  2. Abstract
  3. OBJECTIVE
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONCLUSION
  8. SOFTWARE AND DATA AVAILABILITY
  9. Literature Cited
  • 1
    Palmieri M,Nowell CJ,Condron M,Gardiner J,Holmes AB,Desai J,Burgess AW,Catimel B.Analysis of cellular phosphatidylinositol (3,4,5)-trisphosphate levels and distribution using confocal fluorescent microscopy.Anal Biochem2010;406: 4150.
  • 2
    Lindblad J,Wählby C,Bengtsson E,Zaltsman A.Image analysis for automatic segmentation of cytoplasms and classification of Rac1 activation.Cytometry A 2004;57A:2233.
  • 3
    Bengtsson E,Wahlby C.Robust cell image segmentation methods.Pattern Recognit Image Anal 2004;14:157167.
  • 4
    Yan P,Zhou X,Shah M,Wong STC.Automatic segmentation of high-throughput RNAi fluorescent cellular images.IEEE Trans Inf Technol Biomed 2008;12:109117.
  • 5
    Yu W,Lee HK,Hariharan S,Bu W,Ahmed S.Quantitative neurite outgrowth measurement based on image segmentation with topological dependence.Cytometry A 2009;75A:289297.
  • 6
    Yu W,Lee HK,Hariharan S,Bu W,Ahmed S.Evolving generalized Voronoi diagrams for accurate cellular image segmentation.Cytometry A 2010;77:379386.
  • 7
    Srinivasa G,Fickus MC,Guo Y,Linstedt AD,Kovacevic J.Active mask segmentation of fluorescence microscope images.IEEE Trans Image Process 2009;18:18171829.
  • 8
    Srinivasa G,Fickus MC,Gonzalez-Rivero MN,Hsieh SY,Guo Y,Linstedt AD,Kovacevic J.Active mask segmentation for the cell-volume computation and Golgi-body segmentation of hela cell images. In:Institute of Electrical and Electronic Engineers, editors.Biomedical Imaging: From Nano to Macro. 5th IEEE International Symposium. Paris, May 14–17, 2008.Piscataway, NJ:IEEE;2008. pp 348351.
  • 9
    Held C,Wenzel J,Webel R,Marschall M,Lang R,Palmisano R,Wittenberg T.Using multimodal information for the segmentation of fluorescent micrographs with application to virology and microbiology. In: Institute of Electrical and Electronic Engineers, editors.Proceedings of the 33rd Annual International Conference of the IEEE EMBS. Boston, Massachusetts, August 30 to September 3, 2011.Piscataway, NJ:IEEE;2011. pp 64876490.
  • 10
    Wittenberg T,Grobe M,Münzenmayer C,Kuziela H,Spinnler K.A semantic approach to segmentation of overlapping objects.Methods Inform Med 2004;43:343353.
  • 11
    Held C,Palmisano R,Häberle L,Hensel M,Wittenberg T.Comparison of parameter-adapted segmentation methods for fluorescence micrographs.Cytometry A 2011;79A:933945.
  • 12
    Boykov Y,Funka-Lea G.Graph cuts and efficient N-D image segmentation.Int J Comput Vis 2006;70:109131.
  • 13
    Boykov Y,Kolmogorov V.An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision.IEEE Trans Pattern Anal Mach Intell 2004;26:11241137.
  • 14
    Boykov Y,Kolmogorov V.Computing geodesics and minimal surfaces via graph cuts. In: Institute of Electrical and Electronic Engineers, editors. Proceedings of the 9th IEEE International Conference on Computer Vision, ICCV '03. Washington, DC, October 13–16, 2003, Vol2.Piscataway, NJ:IEEE;2003. pp 2633.
  • 15
    Boykov Y,Veksler O,Zabih R.Fast approximate energy minimization via graph cuts.IEEE Trans Pattern Anal Mach Intell 2001;23:12221239.
  • 16
    Maška M,Daněk O,Ortiz-de-Solórzano C,Muñoz-Barrutia A,Kozubek M,García IF.A two-phase segmentation of cell nuclei using fast level set-like algorithms. In:Salberg AB,Hardeberg JY,Jenssen R, editors.Image Analysis. Proceedings of 16th Scandinavian Conference, SCIA 2009.Berlin:Springer;2009. pp 390399.
  • 17
    Buie HR,Campbell GM,Klinck RJ,MacNeil JA,Boyd SK.Automatic segmentation of cortical and trabecular compartments based on a dual threshold technique for in vivo micro-CT bone analysis.Bone 2007;41:505515.
  • 18
    Nilsson B,Heyden A.A fast algorithm for level set-like active contours.Pattern Recognit Lett 2003;24:13311337.
  • 19
    Goldberg DE.Genetic Algorithms in Search, Optimization, and Machine Learning.Reading, MA:Addison-Wesley;1989.
  • 20
    Wall M.GAlib: A C++ Library of Genetic Algorithm Components. Version 2.4.7. Available at:http://lancet.mit.edu/ga/.
  • 21
    Van Rijsbergen CJ.Information Retrieval,2nd ed.London:Butterworths;1979.
  • 22
    Carpenter AE,Jones TR,Lamprecht MR,Clarke C,Kang IH,Friman O,Guertin DA,Chang JH,Lindquist RA,Moffat J, et al.CellProfiler: Image analysis software for identifying and quantifying cell phenotypes.Genome Biol 2006;7:R100.
  • 23
    Osuna EG,Hua J,Bateman NW,Zhao T,Berget PB,Murphy RF.Large-scale automated analysis of location patterns in randomly-tagged 3T3 cells.Ann Biomed Eng 2007;35:10811087.
  • 24
    Ljosa V,Sokolnicki KL,Carpenter AE.Annotated high-throughput microscopy image sets for validation.Nat Methods 2012;9:637.
  • 25
    Plant AL,Elliott JT,Bhat TN.New concepts for building vocabulary for cell image ontologies.BMC Bioinformatics 2011;12:487.
  • 26
    Wenzel J,Palmisano R,David JP,Wittenberg T,Lang R.Measurement of TLR-induced macrophage spreading by automated image analysis: Differential role of Myd88 and MAPK in early and late responses.Front Physiol 2011;2:71.
  • 27
    Webel R,Milbradt J,Auerochs S,Schregel V,Held C,Nöbauer K,Razzazi-Fazeli E,Jardin C,Wittenberg T,Sticht H,Marschall M.Two isoforms of the protein kinase pUL97 of human cytomegalovirus are differentially regulated in their nuclear translocation.J Gen Virol 2011;92:638649.