By continuing to browse this site you agree to us using cookies as described in About Cookies
Notice: Wiley Online Library will be unavailable on Saturday 7th Oct from 03.00 EDT / 08:00 BST / 12:30 IST / 15.00 SGT to 08.00 EDT / 13.00 BST / 17:30 IST / 20.00 SGT and Sunday 8th Oct from 03.00 EDT / 08:00 BST / 12:30 IST / 15.00 SGT to 06.00 EDT / 11.00 BST / 15:30 IST / 18.00 SGT for essential maintenance. Apologies for the inconvenience.
Conventional histopathology is the gold standard for allograft monitoring, but its value proposition is increasingly questioned. “-Omics” analysis of tissues, peripheral blood and fluids and targeted serologic studies provide mechanistic insights into allograft injury not currently provided by conventional histology. Microscopic biopsy analysis, however, provides valuable and unique information: (a) spatial-temporal relationships; (b) rare events/cells; (c) complex structural context; and (d) integration into a “systems” model. Nevertheless, except for immunostaining, no transformative advancements have “modernized” routine microscopy in over 100 years. Pathologists now team with hardware and software engineers to exploit remarkable developments in digital imaging, nanoparticle multiplex staining, and computational image analysis software to bridge the traditional histology—global “-omic” analyses gap. Included are side-by-side comparisons, objective biopsy finding quantification, multiplexing, automated image analysis, and electronic data and resource sharing. Current utilization for teaching, quality assurance, conferencing, consultations, research and clinical trials is evolving toward implementation for low-volume, high-complexity clinical services like transplantation pathology. Cost, complexities of implementation, fluid/evolving standards, and unsettled medical/legal and regulatory issues remain as challenges. Regardless, challenges will be overcome and these technologies will enable transplant pathologists to increase information extraction from tissue specimens and contribute to cross-platform biomarker discovery for improved outcomes.
Limitations and Strengths of Conventional Histopathology
Biopsies used for allograft monitoring are invasive, expensive, potentially morbidity- and mortality-producing, an unpleasant patient experience, subject to sampling error, and interpretations are prone to bias, subjectivity and inter-observer variability (1). Compared to less-invasive monitoring techniques, such as peripheral blood and fluid mRNA arrays and proteomics/metabolomics (2,3), the risk-benefit ratio and relative “value proposition” of routine histopathology are decreasing. Despite shortcomings, however, conventional histology quickly provides a wealth of difficult to replace information about structural integrity, spatial and temporal relationships, and rare events/cells, which depend on traditional light and fluorescence microscopes. Advances in digital imaging techniques, robotics and computing are providing new “toolkits” enabling pathologists to extract more information from tissue samples and increase the histopathology value proposition.
The Digital Imaging Revolution
Digital (anatomic) pathology (DP) rapidly evolved (4) from basic microscope-mounted cameras for individual microscope field sharing to automated scanning of entire glass slides to create WSI files (4–7). Aggregate progress in DP evolved from individual advancements in robotics, digital imaging and computer resources (4–7).
Current slide scanners are software-driven, robotically controlled microscope systems: a stage moves continuously under a uni-ocular microscope equipped with a high-speed camera that captures each field. Focusing algorithms and robotics adjust the objective along tissue z-plane topography while moving the slide continuously in x- and y-planes. Optimally focused individual fields or image strips are then “quilted” together to create seamless two-dimensional (2D) WSIs (8). This “traditional microscope” image-capture approach inherently results in a scanning “bottleneck” limiting throughput.
Scan time, or time required to convert a glass slide into a WSI, depends largely on the scanner's objective lens: higher resolution native 40× scans have smaller fields of view and more z-planes that require more image captures and result in longer scan times and larger files sizes than lower resolution 20× WSI. Commercial vendors usually quote 20× scan time per cm2 of tissue. This setup offers a compromise of resolution, larger field area, file size, and scan times practical enough to approach clinical applicability. In the senior author's experience, however, native 40× scans are needed to resolve kidney and liver transplantation pathology challenges, such as inflammatory cell subtypes (e.g. plasma cells), viral inclusions and glomerular basement membrane abnormalities (Figure 1); others report similar problems and pathologists reluctance to relinquish native 40× scans (5,9).
Capturing “in-focus” z-plane images is a major challenge made worse by poor slide preparation (thick sections, folds, etc.). Various problems lead to scan failure rates (unreadable WSI) of ∼ 2–5% (5,7). Z-stacking algorithms, using high resolution lenses to capture images from various focal planes over a several-microns range, can improve 2D WSI quality by selecting the “optimally focused” image for each address. This requires more focus operations due to a smaller depth of field and greatly adds to scan time. 3D WSI can be created by expanded z-stacking and retaining all levels (e.g. cytology; Ref. 10). Newer generation scanners offer the option of multiple interchangeable objectives to optimize throughput: robotic switching systems exchange lenses for detailed scanning of only selected areas/cases.
Scanners and services
Comparison of cost, scan time, image resolution, hardware robustness, slide holder capacity, image focusing and stitching algorithms, z-stack acquisition modes, brightfield versus fluorescence, file compression methods/formats/size, and application capacity are issues to understand and discuss with vendors (11). Systems currently range from $30 000–$250 000, depending on scanner capabilities and supporting software complexity. Two access avenues are available: (1) outright equipment purchase; and (2) vendor contract-based WSI scanning services. Integrated service providers offer combined scanning, hosting and image and data analysis (Table 1).
Table 1. Selection of vendors providing whole slide imaging products/services
Imaging modality (service)
BF = brightfield; FL = fluorescence; Host = hosting service; Service = imaging service available.
Compromise between scan time and image quality must be carefully examined and understood when considering clinical WSI implementation. Current scanners complete native 40× needle biopsy scans in 4–15 min, depending on the total tissue area. Reducing scan time while maintaining resolution is a major marketplace challenge/focus. Improvements in optical design, digital camera throughput and computing speed have enabled large vendors (e.g. Leica, Omnyx, Philips) to reduce scan time to ∼1 min for 20× scans; the expectation is for continual improvement. Robust testing of any scanning system under intended usage conditions is strongly suggested before purchasing.
In our opinion, epi-fluorescence capabilities are absolutely necessary for maximum DP exploitation, especially for transplantation research (12,13). Fluorescence scanners usually support multiple filters either via mechanical switching or tunable LED excitation; they capture multiple signals (channels) and assemble composite images similar to conventional confocal microscopy. Multichannel WSI scanners can either (a) capture the entire tissue area with one filter, and then change filters, etc., which saves scan time; or (b) switch filters at each capture location, which is superior for channel colocalization. Multipass filters enable simultaneous multiple fluorophores imaging followed by color component separation, but this approach limits fluorophore selection.
Fluorescence WSI systems performance characteristics should be understood before purchasing (e.g. alternative imaging modalities epi-fluorescence). Current filter switching technology requires 2–4 h scan time for five fluorophore Qdot®-stained biopsy slides depending on exposure time. Scan time can be shortened to < 2 h by limiting focus range, digitally enhancing emission signals and changing automated focus adjustment frequency, but only if these are built-in capabilities.
Potential implementation for clinical practice mandates changes in workflow, which can affect scanner choice. Important considerations include slide loader capacity, scanner system robustness, slide acceptance promiscuity, attention to histology quality (14), and slide barcoding (see below). Slide and tissue thickness variation influence z-plane focusing algorithms: thin tissue sections (4 μm) generally produce better WSI. Manual WSI file labeling results in unacceptably high error rates (∼10%): complex filenames needed to encode appropriate information promote errors that render WSI inaccessible. Barcoding links WSI to LIS data early in the workflow for nearly error-free case navigation.
WSI files are complex image composites that require freely downloadable, vendor-supplied, viewing software, which encodes fast switching between composites resulting in “magnification changes” that resemble traditional microscopes. Recent movement toward “open” WSI formats enables many vendor solutions to read images from a variety of devices. Rapid remote viewing is enabled by a combination of image compression algorithms, transmitting only the current field of view, and caching algorithms that predict image navigation (15). Remote viewing is superior to transferring entire WSI files to a local server, which is impractical for large slide volumes because of file size. 100 Mbit networking speed is generally sufficient for WSI viewing, as are speeds provided by residential broadband connectivity. Although image data transmission is optimized, latency (i.e. time required for signal to navigate telecommunication components between source server and viewer), and to a lesser extent bandwidth, issues become rate limiting for intercontinental WSI viewing. Direct server/file access for those outside an institution can be challenging because of institutional internet firewalls that present logistical/bureaucratic obstacles.
Data and equipment infrastructure
Large WSI files (range ∼ 200 MB to ∼5 GB) ultimately reside on servers linked both to scanners and to local or remote viewers. Huisman (16) provides an overview of storage and IT implications of housing large volumes of “post-signout” 20× WSI files. IT storage in large academic institutions is often physically remote, necessitating high-speed, high-bandwidth networking connections for rapid, effective and efficient data file movement between scanning systems and archiving. Our data connectivity environment consists of 1 gigabit (preferred) and 100 megabit (workable) networking between the WSI devices and the server storage. A typical needle biopsy 1 gigabyte WSI file can be transferred from the scanning system PC to web-accessible archives in ∼1–2 min. Compression and caching technology provide efficient review speeds for external access (crossing firewalls). Network and interface traffic during periods of high data usage can slow data transfer, so close communication between the WSI storage infrastructure vendor and local IT teams is critical to anticipate and resolve issues.
System “robustness” and redundancy (scanners, server and delivery) are needed for anticipated clinical service applications. Scanners need periodic maintenance and repair; servers crash and perform suboptimally because of software issues; and network outages can prevent slide access. Failure at any point often completely stops diagnostic rendering. High availability (HA) systems can be setup where a second, redundant server exists when the primary unit fails. Storage and network access can be similarly designed resulting in minimal unanticipated downtime. We deployed a HA digital slide system for a low volume trial setting (2–20 slides/day) and have been running continuously for 4+ years with 99.9% uptime.
Importance of Combining Multiplex Staining and High Resolution WSI
Multiplex staining (simultaneous identification of three or more protein antigens), provides a critical link between “-omics” discovery science and empowering aspects of DP. Proteins representing structural and/or signaling pathways involved in injury and repair can be linked to contextual cues embedded in tissue structure. Multiplexing has previously been expensive and inconvenient because it: (a) relies on traditional fluorophores with inherent drawbacks; (b) requires expensive and inconvenient fluorescent microscopes; and (c) depends on tedious image capturing steps and subjective interpretation. Spatially overlapping signals derived from brightfield chromogen multiplexing are harder to separate and quantify unless multispectral imaging is used (17). We (13) found Qdots (12,18) ideal for multiplexing because of (1) wideband absorption, narrow-band emission spectra, and a wide Stokes' shift that enables microscopists to (a) excite with a single UV range light, and obtain crisp and clean images (19); (b) easier unmixing when >3 antigens are targeted (18); (c) panoramic overview at low magnifications; and (2) photostability (Figure 2).
Multiplex challenges include: careful antibody titration to prevent cross-reaction and signal bleed-through; incompatible antigen-recovery techniques for paraffin-embedded tissues; and uneven background auto-fluorescence excitation. An example technology to limit nonspecific emissions, the Nuance system (CRI, Woburn, MA) uses liquid crystal tunable filter scanning 420–720 nm at a set microscopic field of view, separating out specific spectra including autofluorescence. Resultant images are created from specific wavelengths; traditional emission and excitation filters are not required. Although powerful, this technique has performance issues and typically used for small areas of interest.
Transformation of microscopically observed parameter reporting from semiquantitative, estimated values, to objective data previously required ocular micrometers for measuring distance, random point counting grids, or individual image capture for morphometric analysis. All are labor intensive. It is far easier, and nearly as accurate, to estimate biopsy percentages involved by inflammation or fibrosis or count structures than to conduct morphometric studies.
Slide scanners replace many repetitive mechanical operator/pathologist functions and morphometric software programs are beginning to accept WSI. Analytic software ranges from relatively inexpensive basic macro-driven software for color quantification, through expensive and complex, trainable model-based applications for recognizing and quantifying tissue patterns (Reviewed in Refs. 20,21).
Analytic software can be layered into three successive tiers, each providing more complex data:
1Pixel-based (equivalent of manual random point counting): automated software recognition of pixel color enabling assessments such as percent fibrosis area based on trichrome or immunohistochemistry stains. Public domain solutions (e.g. ImageJ http://rsbweb.nih.gov/ij) and specifically optimized commercial products support a variety of stains (e.g. H&E, Trichrome, PAS, and immunohistochemistry) in AEC or DAB (e.g. Aperio's Slide Analysis Toolbox).
2Simple morphometry builds on pixel-based color analyses: similarly colored pixels grouped together define biologically relevant structures (e.g. nuclei) if they satisfy relevant predefined criteria (e.g. shape and/or size). This enables individual cell identification and associated analyte measurements.
3Complex imaging analysis systems combine pixel-based and simple morphometry into higher order relationships using rule sets to define complex tissue structures (e.g. glomeruli). The open-source FARSIGHT project (http://www.farsight-toolkit.org) displays scatter plots that highlight nuclear size, centroids and associated analytes; commercial products such as Definiens (http://www.definiens.com) and CompuCyte (http://www.compucyte.com) organize individual cells to higher order structures, while preserving the ability for tabular data output and downstream analyses (Table 2). Once relevant structures are identified, users can select from a variety of tools to analyze quantitative data. These processes, however, can be complex and time consuming for nonexperts. Supplemental Table 1 shows methods, software and parameter assessment for transplant-specific morphometric studies.
Table 2. Example image analysis software systems
Color based /morphometry based selection, WSI enabled
Multiplexed WSI analytics require flexible viewing functions to show/hide combinations of image layers to delineate structures of interest. The DAPI layer segments nuclei and Qdot-stained layers provide intranuclear, cytoplasmic, or cell surface measurements for each cell, enabling quantification of molecular analytes over chosen subpopulations. Resulting data can be displayed for visual inspection as a scatter plot and/or saved to tables/database (Figure 3).
Potential advantages of automated algorithm WSI processing are substantial: percentage fibrosis areas, number/percent of sclerotic glomeruli, etc. could be calculated before pathologist viewing and reporting. However, such algorithms are computer resource intensive and would require powerful grid facilities or cloud computing environments. This could also enable content-based image retrieval (CBIR) systems: user select structures or regions of interest (ROI) and then use software to identify similar areas in the same or other WSI.
WSI Usage, Workflow Issues and Enabling Features
WSI systems are used primarily for education, basic and translational research, quality assurance, archiving, second opinion consultations (7) and clinical trials (22). Many traditional pathology processes are facilitated by brightfield WSI (Table 3): especially noticeable in transplant pathology are (1) case collation and previous biopsies are immediately available for review electronically; (2) split screen side-by-side and multiplex panel comparisons (Figure 1); (3) easy sharing of problematic cases electronically for consultation, clinicopathological correlation and quality assurance; and (4) easier analytics. An efficiency study completed specifically for transplantation pathology revealed a potential time savings of 15% (23). Concordance rates between “digital diagnosis” versus traditional glass slide microscopy for various specialties is generally > 90% (reviewed in (4)). Improved pathologist and case distribution efficiency, diagnostic accuracy, and value proposition because of new toolsets are potential savings areas that might help offset significant implementation costs (5).
Table 3. “Head-to-head” comparison of digital and traditional approach to transplant pathology
Key aspects of pathology reviews
Reliance on glass slides
Inferior early, but superior late: WSI are dependent on good quality histology, but scanning is currently a “bottleneck”, dependent on scan time, image quality, network issues. Once high quality WSI is complete, WSI access is usually superior to re-reviewing the glass slides
Superior early, but inferior late: Glass slides are immediately available for review after preparation, but once high-quality WSI is complete “slide” access for WSI and retention of image quality over time is inferior for glass slides, which can be lost, broken, misplaced or deteriorate
Case organization and evaluation
Superior: Pathologists currently assume many “clerical” functions such as matching slides to requisitions, assuring slides receipt, and comparison to previous specimens. Many of these tasks are either completed electronically or assumed by others in DP systems
Inferior: See the left box for WSI; these functions are currently assumed by pathologists
Image quality (combination of resolution, color fidelity)
Slightly inferior: Dependant on imaging setup and objective lens resolution. Currently, highest resolution WSI are slightly inferior to optical microscopes, and restrict access to modalities such as oil-immersion for increased resolution
Superior early, inferior late: Initially glass slide quality is better than WSI, but glass slides degrade and stains fade over time.
Slide navigation time
Inferior: Magnification and field of view changes require more time than microscope and time required is directly proportional to observer distance from WSI server – can be problematic for international arrangements
Superior: magnification and fields of view changes affected almost immediately
Slide navigation capabilities
Superior: Enables square panoramic field of view and overview functions, low-power examination, finely granular digitally selected magnifications
Inferior: Smaller round field of view, minimal overview functions, predetermined magnification, very low magnification tend not to get used
Significantly inferior and not generally possible to the same extent
Currently not possible
Multiple simultaneous viewers in geographically disparate locations
Inferior: Possible only with live camera feed from microscope with desktop sharing software
Multiple simultaneous local viewers (multiheaded slide conference)
Superior: Multiheaded microscope conference difficult to reproduce digitally
Review of multiplex-stained slides
Vastly superior: obviates need for specialized IF microscopes, schedules, inconvenient location, complexity, photobleaching, multiple channels, etc.
Significantly inferior: Traditional fluorescence microscopes are expensive, complex, difficult to use, shared resource, inconvenient location, etc.
Substrate for (automated) morphomtery
Vastly superior: WSI greatly facilitate triage to automated morphometry including brightfield, fluorescence, and multiplex stained slides. Scanner generally replaces time-consuming tasks of operator, such as capturing images, channels, saving files, exporting, etc.
Significantly inferior: Morphometric studies generally cumbersome and requires capturing digital imaging or ocular objective micrometers, fluorescence and multiplex stained slides very cumbersome to manually convert to images for morphometric examination
Integration into electronic medical records
Vastly superior: All information already exists in digital format. How much of this promise actually materializes should be a focus of further study
Significantly inferior: Pathologist/operator has to assume task of scanner by capturing and exporting images
As a practical example, all Immune Tolerance Network (http://www.immunetolerance.org) liver allograft tissue and slides obtained over the last 6 years have immediately been incorporated into a completely digital workflow (scanning, viewing, reporting and archiving). Current and previous biopsy immediate availability (regardless of viewer location) and side-by-side comparisons have been invaluable for assessing consequences of changes in immunosuppression (22), HCV-related fibrosis progression, and correlative case discussions. Multiplex staining enabled exact localization of C4d deposits in relation to endothelial cells (24), determination leukocytes subsets and morphometric analysis of liver structure (25). Integration of pathology images/data with mRNA expression arrays, flow analysis of peripheral blood leukocytes and DSA determinations will enable cross-platform patient- and study-centric data views.
Complete workflow integration is the focus of Omnyx, LLC, which should facilitate workload distribution. Practical workflow solutions for specific tasks also exist for WSI data (e.g. Aperio's Spectrum Data Management Software). While rich in functionality the challenge of deep integration into the complex and commercially competitive clinical IT environment usually requires programming expertise or dedicated resources before widespread adoption (4,11,26). Large vendors (Omnyx, Philips) use radiology PACS experience as a model (“medical device” concept) forcing system synchronization via interfaces thus enabling an integrated user workflow.
Standardization and Regulatory and Medical-Legal Considerations
Novelty and rapid hardware and software evolution result in unsettled medical/legal (27), regulatory (28) and standardization issues (4,14,26,29) and undetermined hardware, software specifications, and calibration processes, but like an impressionistic painting, images are beginning to appear. For example, licensing is state regulated and regulations vary considerably with respect to telemedicine. States with large rural populations generally have complex telemedicine regulations because of needs-based experience, while densely populated east coast states might not (27). Important issues include whether an interaction is considered primary or second-opinion telepathology; resident state of physician and patient and how that determination and the practice of pathology is defined; and licensure requirements (27).
Compromise between standardization and flexibility resulted in NEMA DICOM Working Group 26 (ftp://medical.nema.org/medical/dicom/final/sup145_ft.pdf) and the CAP initiating discussion groups that provided recommendations about specifications for imaging format unity and Diagnostic Workstation prototypic specifications (http://www.cap.org/apps/docs/snomed/documents/diagnostic_work_station_information_sheet.pdf). Envisaged is a vendor-neutral, 3-monitor, heterogeneous environment that allows pathologist customization, collaborative EMR exchanges (4,29,30), real-time consultation, structured reporting (26), clinical decision support (26,28), and online reference materials. Exchanges highlighted areas needing consideration: display monitor standardization, including pixel quality (e.g. resolution, color reproduction/standardization), refresh speed, automated software detection of adjustment needs, and image adjustment notifications, when appropriate. Various calibration tools (e.g. “spiders”) can monitor color uniformity across multiple workstations. We have attempted to internally standardize image quality by using higher end IPS displays and standard color temperature settings (e.g. 6500K). Yagi (14) introduced optimization methods for histology and WSI imaging color quality.
US DP and clinical automated image analysis algorithms are regulated by the Food and Drug Administration (FDA) (28). Two major processes enable US device marketability: (1) FDA Clearance (510(k) process); and (2) FDA Approval. Three FDA regulatory medical device classifications include (a) class I: present minimal potential user harm and are generally exempt from the pre-market notification process; (b) class II: devices for which existing methods, standards and guidance documents are available to provide assurances of safety and effectiveness; these typically require pre-market notification via FDA Clearance; (c) class III: usually support or sustain human life, are of substantial importance in preventing the impairment of human health, and present a potential unreasonable risk of patient illness or injury. Pre-Market Approval (PMA) submission to the FDA is required to allow marketing of class III medical devices.
The FDA remains undecided whether an application for primary diagnosis DP would be a class II or class III device (28). The CAP convened a work group to address the question: “What needs to be done to ‘validate’ a WSI system before it is placed in clinical service?” (http://www.cap.org/apps/docs/membership/transformation/new/validating.pdf); recommendations are due soon. The FDA has, however, already classified and cleared, as class II devices, WSI systems that combine digital imaging with automated image analysis for Her2/neu, Ki67 and p53 (http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPMN/pmn.cfm). New image analysis algorithms require a substantial equivalence study for safety and effectiveness with another lawfully marketed model such as conventional manual microscopy and/or an already FDA-cleared algorithm.
Drivers and Challenges for Implementation of DP
DP drivers assuring continual development include improved pathologist efficiency and accuracy; freedom from glass slides after initial scanning; more efficient case distribution; and enhanced “value proposition” because of new toolsets. Challenges, which adversely affect return on investment, include (1) the scanning/image acquisition ‘bottleneck’; (2) additional equipment, supply, and dedicated personnel (e.g. digital path assistant) costs without decreasing other supply costs (e.g. glass slides); (3) large WSI files (∼10× typical radiograph) resulting in storage and transmission issues; and (4) software vendors competition inhibiting fluid digital EMR integration. Regardless, we fully expect clinical implementation soon for relatively low volume specialties, such as transplantation pathology.
We would like to thank the staff of the Research Histology Service from the Thomas E. Starzl Transplantation Institute, members of the Demetris Laboratory and the Roysam Laboratory, and Dr. William M. Lee of the Department of Medicine, Abramson Cancer Center, University of Pennsylvania.
Funding source: This study was supported by National Institutes of Health (NIH) R01-AI-081678, N01-AI-15416, U01-A1-077867, P01-A1-064343 (all for A.J.D), R01-EB-005157, R01-CA-135509 (all for B.R. and W.M.L.). Also it was supported by S-IDEA grant W81XWH-07-1-0325 (B.R).
The authors of this manuscript have conflicts of interest to disclose as described by the American Journal of Transplantation. A.J.D. is a member of the clinical development team of Omnyx (http://www.omnyx.com), a corporation developing a complete DP solution. A.L. serves as a consultant for Carl Zeiss MicroImaging, LLC.