This article is a US government work and, as such, is in the public domain in the United States of America.
Automatic segmentation and supervised learning-based selection of nuclei in cancer tissue images†
Article first published online: 31 JUL 2012
Published 2012 Wiley Periodicals, Inc.
Cytometry Part A
Volume 81A, Issue 9, pages 743–754, September 2012
How to Cite
Nandy, K., Gudla, P. R., Amundsen, R., Meaburn, K. J., Misteli, T. and Lockett, S. J. (2012), Automatic segmentation and supervised learning-based selection of nuclei in cancer tissue images . Cytometry, 81A: 743–754. doi: 10.1002/cyto.a.22097
- Issue published online: 22 AUG 2012
- Article first published online: 31 JUL 2012
- Manuscript Accepted: 12 JUN 2012
- Manuscript Revised: 18 MAY 2012
- Manuscript Received: 1 MAR 2012
- National Cancer Institute, National Institutes of Health. Grant Number: HHSN261200800001E
- Intramural Research Program of the National Institutes of Health
- National Cancer Institute
- Center for Cancer Research
- Department of Defense Breast Cancer Idea Award to Tom Misteli
- tissue segmentation;
- artificial neural network;
- cancer diagnosis;
- image analysis;
- pattern classification
Analysis of preferential localization of certain genes within the cell nuclei is emerging as a new technique for the diagnosis of breast cancer. Quantitation requires accurate segmentation of 100–200 cell nuclei in each tissue section to draw a statistically significant result. Thus, for large-scale analysis, manual processing is too time consuming and subjective. Fortuitously, acquired images generally contain many more nuclei than are needed for analysis. Therefore, we developed an integrated workflow that selects, following automatic segmentation, a subpopulation of accurately delineated nuclei for positioning of fluorescence in situ hybridization-labeled genes of interest. Segmentation was performed by a multistage watershed-based algorithm and screening by an artificial neural network-based pattern recognition engine. The performance of the workflow was quantified in terms of the fraction of automatically selected nuclei that were visually confirmed as well segmented and by the boundary accuracy of the well-segmented nuclei relative to a 2D dynamic programming-based reference segmentation method. Application of the method was demonstrated for discriminating normal and cancerous breast tissue sections based on the differential positioning of the HES5 gene. Automatic results agreed with manual analysis in 11 out of 14 cancers, all four normal cases, and all five noncancerous breast disease cases, thus showing the accuracy and robustness of the proposed approach. © Published 2012 Wiley Periodicals, Inc.