3D parallel coordinate systems—A new data visualization method in the context of microscopy-based multicolor tissue cytometry

Abstract

Background:

Presentation of multiple interactions is of vital importance in the new field of cytomics. Quantitative analysis of multi- and polychromatic stained cells in tissue will serve as a basis for medical diagnosis and prediction of disease in forthcoming years. A major problem associated with huge interdependent data sets is visualization. Therefore, alternative and easy-to-handle strategies for data visualization as well as data meta-evaluation (population analysis, cross-correlation, co-expression analysis) were developed.

Methods:

To facilitate human comprehension of complex data, 3D parallel coordinate systems have been developed and used in automated microscopy-based multicolor tissue cytometry (MMTC). Frozen sections of human skin were stained using the combination anti-CD45-PE, anti-CD14-APC, and SytoxGreen as well as the appropriate single and double negative controls. Stained sections were analyzed using automated confocal laser microscopy and semiquantitative MMTC-analysis with TissueQuest 2.0. The 3D parallel coordinate plots are generated from semiquantitative immunofluorescent data of single cells. The 2D and 3D parallel coordinate plots were produced by further processing using the Matlab environment (Mathworks, USA).

Results:

Current techniques in data visualization primarily utilize scattergrams, where two parameters are plotted against each other on linear or logarithmic scales. However, data evaluation on cartesian x/y-scattergrams is, in general, only of limited value in multiparameter analysis. Dot plots suffer from serious problems, and in particular, do not meet the requirements of polychromatic high-context tissue cytometry of millions of cells. The 3D parallel coordinate plot replaces the vast amount of scattergrams that are usually needed for the cross-correlation analysis. As a result, the scientist is able to perform the data meta-evaluation by using one single plot. On the basis of 2D parallel coordinate systems, a density isosurface is created for representing the event population in an intuitive way.

Conclusions:

The proposed method opens new possibilities to represent and explore multidimensional data in the perspective of cytomics and other life sciences, e.g., DNA chip array technology. Current protocols in immunofluorescence permit simultaneous staining of up to 17 markers. Showing the cross-correlation between these markers requires 136 scattergrams, which is a prohibitively high number. The improved data visualization method allows the observation of such complex patterns in only one 3D plot and could take advantage of the latest developments in 3D imaging. © 2006 International Society for Analytical Cytology

The major advantage of microscopy-based multicolor tissue cytometry (MMTC) (1) is the generation of observer-independent quantitative data on cellular and molecular markers. Such markers can be measured simultaneously in tissue sections as well as cell preparations. The ability to determine multiple markers/parameters simultaneously but on a single cell basis is the fundamental prerequisite in cytomics (2–4), a novel discipline that is expected to constitute the basis for a predictive medicine in the future (5, 6). Because of the simultaneous analysis of several/many markers, interrelationships can be visualized, but how? Although more data for the visualization process will lead to better understanding of interdependencies and, in the clinical context, to better decisions for medical diagnoses, an increasing amount of data conflicts with the nature of visualization techniques in general. The more the parameters, the higher the degree of difficulty in terms of data presentation.

In this article current methods for data visualization and data exploration in tissue cytometry are evaluated and problems are discussed. All of these commonly used techniques suffer from serious problems, which are related to human comprehension of complex data sets. When analysing multicolor immunofluorescence images, multiple parameter combinations and cell populations occur, which are difficult to visualize. Many different scattergrams, showing two parameters each, can be generated, but comprehension of multidimensional parameters and significant cell populations is hard to achieve. To overcome these problems, an alternative method for visualization of complex data is presented, and its application in tissue cytometry is outlined.

METHODS

Datasets

The measured data events used in Figures 1–7 were generated by confocal laser scanning microscopy (LSM 510 META, Carl Zeiss, Germany). Automated microscopy-based multicolor tissue cytometry (MMTC) was performed on frozen sections of human skin that were stained using the triple combination of anti-CD45-PE, anti-CD14-APC, and SytoxGreen as well as the appropriate single and double negative controls. Stained sections were analyzed using TissueQuest 2.0 software and analysis system (TissueGnostics, Austria). The dataset presented in Figure 8 was randomly generated using Matlab environment (Mathworks, USA). They represent three clusters of events with 10 parameters.

Scattergrams, 2D and 3D Parallel Coordinate Diagrams

The scattergrams presented throughout the article contain measured events from the frozen sections mentioned earlier and the analysis and scattergram tools of the TissueQuest 2.0 software. The 3D parallel coordinate plots were generated either from the same data set as the scattergrams or randomly computer-generated 10-dimensional data.

The 2D parallel coordinates were plotted using integrated function from the Matlab environment. The innovation of the 3D parallel coordinate systems consists in the new approach and technique for generating the density isosurface and was programmed under Matlab.

STATE OF THE ART

Scattergrams (Dot Plots)

Each point in the diagram indicates one measure event (see Fig. 1). The 2D display can be easily overlooked and understood, and is appropriate for direct interaction with 2D input devices (e.g. the mouse) (7). Another major benefit of scattergrams is the high acceptance by the scientific community. Scattergrams have been widely used and became the most commonly used visualization method in cytometry. Scattergrams are able to show only two data dimensions at a time (8). Because of the continuously increasing number of simultaneously applicable markers, simple scattergrams do not meet the requirements of visualization tools for multiparametric data sets, i.e., they are inadequate for description of complex interrelationships among and between multicolor-labeled cell populations.

Figure 1. Scattergram. Reactivity with anti-CD45-PE and anti-CD14-APC is depicted on x- and y-axis, respectively. Accumulated measure events are quite frequent. In this case overlapping events are drawn in the lower left corner of the plot. More than 40% of the events are displayed on one single point, which results in a loss of the intuitive understanding. The two marked groups near to the axes also expose the same issue. The occluded points create dense lines. [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.]

Overview Problem in Relation to Multicolor Staining

The fast development in MMTC will result in an even more pronounced problem regarding the data meta-evaluation. In tissue cytometry, the data of interest consist of several parameters for each measure-event. In terms of multicolor labeled tissue, each marker leads to at least one additional parameter or rather to one additional dimension. Assuming that the number of markers on the same tissue sample will rise up to 50 or more, it is foreseeable that a vast amount of scattergrams (i.e., 50 × 49/2 = 1225 to visualize all pair wise interactions) is needed to process the cross-correlation analysis.

In the case of multivariate data mining, a collection of scatter plots has to be created where several parameters are paired to each other. As the number of dimensions grows, it is getting increasingly difficult for the user to keep track of the plenty of scattergrams and their interdependences. In the worst case, the set of variables in all combinations ends in a complete scatter matrix (9). This raises the risk of losing the overview, and thus, it represents another argument against the usage of scattergrams.

Aggregation Problem Caused by the Large Amount of Data

Based on the selected magnification, the digital camera mounted to the microscope is able to acquire only a limited field of view, which is substantially smaller than the entire sample. The acquisition of the total area of a tissue section may require several thousands of single images, which results in a problematic large data set. In the field of biomedicine, up to 1,000,000 measure events per analysis is not extraordinary. In considerations of short response times of the application, the lack of performance has to be kept in mind. The continuous feedback to the observer during the direct user interaction provides a powerful tool for efficient and fast data mining, which we cannot afford to lose.

The problem occurs when different measure-events are drawn on the same coordinate in the scattergram. Occlusion is a significant concern when processing large data, since the probability of overlapping of two or more measure events rapidly grows with the amount of items in the display. As a result of this fact, the observer has problems to figure out the number of events (10). The outcome of this is maybe a fatal wrong medical diagnosis. An example of the scattergram that suffers from the aggregation problem is provided in Figure 1. At the considered data quantity the probability of massive point collisions is high.

The display of additional histograms for each parameter of the scatter plot is of course the most obvious solution for getting the aggregation problem under control. A histogram is a specialized plot where the data range is split up into equally sized nonoverlapping intervals. Imagine each interval as a bin where the number of points that falls into each bin is counted. The purpose lies in the presentation of population distribution and is handy for a fast retrieval of outliers in the data. An example of histograms is provided in Figure 2. However, in a multidimensional data set, the use of extra histograms for every scatter plot is problematic and raises new problems. To explore the multidimensional data space using the cross-correlation analysis, plenty of plots are necessary. Thus, valuable screen space is lost by displaying the multiplicity of additional histograms. Needless to say that the possibility of optionally histogram hiding gives no effective advancement but rather amplifies the user's confusion.

Figure 2. Scattergram–histogram combination. Scattergram depicts fluorescence 1 on the x-axis and fluorescence 2 on the y-axis using logarithmic scales. The corresponding histograms add the density information for both parameters separately. [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.]

Another way to solve the aggregation problem is the utilization of 2D density plots or contour plots, which are also known from flow cytometry. The methods display the density information mapped in the color space. We used a color map ranging from blue to red, while others (11–13) used grayscale color maps in combination with density estimation in parallel coordinates. The “hot” (red) regions versus “cold” (blue) regions approach is similar to the method used in scintigraphy in the nuclear medicine.

Still the color is needed for the cross-correlation analysis to brush the gated subset.

IMPROVED METHODS OF VISUALIZATION

2D Parallel Coordinate Systems

Parallel coordinates represent a visualization method designed for multivariate data that maps into 2D space without losing any information. The technique was introduced by Inselberg and Dimsdale in the context of computational geometry (14, 15), and Wegman and Luo discussed the application to multidimensional data analysis (16). A good explanation is provided by Siirtola in (17): An n-dimensional data tuple {x₁, x₂, x₃, ..., x_n} is visualized in parallel coordinates as a polygonal line, connecting the points x₁, x₂, x₃, ..., x_n in n parallel y-axes. For a large set of tuples, this technique will produce a compact 2D visualization of the whole multidimensional data set. The orthographic lines that represent the axes are uniformly spaced. The respective scale is assigned as usual to the axis. The user should keep clearly in mind that not all axes mandatorily follow the same scale division what may cause misinterpretation. The number of dimensions is only bounded by the available space of the display area or rather the screen. For the pattern recognition from user side, the distance of the axes is of vital importance.

The utilization of 2D parallel coordinate systems solves one of the major problems caused by scattergrams. Only one plot is sufficient for displaying the complete data set including all parameters. All other plots are no longer necessary (see Fig. 3). This approach saves space and is a solution for the described overview problem of scattergrams. Another improvement is given by the fact that only one object (polygonal line) is used for one measure event. Thus, it is possible to notice the relationship between the parameters for each cell.

Figure 3. Comparison between three scattergrams and one 2D parallel coordinate plot, using a clustered population from a dataset with 4 parameters. The example shows a clustered population of real measure events. Each event has 4 different parameters: area, DNA mean intensity, CD45 mean intensity, and CD14 mean intensity. (a) The cluster in the first scattergram has low values for area and high values for the nuclei intensity. (b) The CD45 parameter for the same population has lower mean than the DNA and is more dispersed. Still it is high enough to consider the population as being CD45 positive. (c) The mean of the CD14 is even lower than the CD45, indicating a negative population in respect to CD14. (d) The same data set can be represented by the use of one 2D parallel coordinate plot. For one measure event, three points on three different scattergrams are needed. In this case, the observer can lose the track of one entity across the different plots. The improvement of the 2D parallel coordinate plot lies in the fact that one polygonal line represents one event across the complete parameter set. The first scattergram (a) is replaced by the correlation between the first and the second axis, as indicated by the arrow. The same procedure happens for scattergrams (b) and (c), respectively. [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.]

The use of parallel coordinates does not solve all the problems (13). Novotny (10) explained one unpleasant drawback. The display gets cluttered during the rendering process of a large number of samples. Interaction and even mere understanding of such a display is complicated (see Fig. 4). Features like axis swapping, axis adding, and axis deletion implicates positive influences on the user interaction process. Especially axis swapping is essential because parameters with interrelationships can benefit from a side-by-side position. An axis switch operation is also a way for the reduction of crossings inside the plot. However, these improvements cannot solve the problems caused by the huge data amount. This fact leads to a loss of the data significance. We have developed a promising solution by using the 3D space that can deal with the bulk of data.

Figure 4. Major disadvantage of 2D parallel coordinate systems. Basically the same procedure as described in Fig. 3 is used again to transform the scattergrams into the 2D parallel coordinate plots. The data set consists of three populations, highlighted with green, blue, and red. The difference lies in the data amount. The dataset consists of approximately 1,000 measure events. Even though this number of measure events is not extraordinary for analyses in tissue cytometry, the 2D parallel coordinate plot is reaching its limit. Virtually no clear patterns or groups can be observed. This cluttering effect is strongest in the section between the CD45 and CD14 axis. No real distinction between high populations and equal distributed low dense populations can be made. The three major populations visualized in the last scattergram (CD45/CD14) can easily be observed in the parallel plot. The mean density values of the populations are represented by three lines with different styles (dashed, dot-dashed, and dotted). [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.]

3D Parallel Coordinate Systems

To solve the problem with the huge amount of data to be visualized, we propose a new data visualization method that solves both of the earlier described problems of scattergrams. The overview problem can be solved using the 2D version. The 2D parallel coordinate systems succeed in the reduction of plots and are therefore a corrective for the lost overview problem (Fig. 5a).

Figure 5. The plot evolution starting with 2D parallel coordinates toward the 3D surface version. The example illustrates the effect of unrecognizable population distribution in case of a multitude of data (a). In (b) color-coded density information is added to the plot. The same data set is represented in (c) as a density isosurface in the 3D space. The color coding in the 3D plot (c) is only for a better perception of the image in the paper, but unnecessary in a 3D software environment. The density plot in (b) is the x/y-view of the 3D parallel surface plot from (c). [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.]

The negative effects from the aggregation problem can be reduced by using colors in the density plot. The information about the population distribution could be preserved by mapping a color space correspondingly to the density onto the plot (Fig. 5b). Hereby, a color vector, running from dark blue to dark red, is assigned to the plot depending on the population. Thus, it appears that highly dense populations are visualized red, and low density populations appear blue.

However, the color is more adequate for showing gated populations than densities. The proposed 3D parallel coordinate plot solves the aggregation problem by creating an isosurface (Fig. 5c). The x/y-plane represents the 2D version of the parallel coordinates, while the z-dimension represents the density of the events. Imagine an isosurface as a landscape. Low event accumulations are mapped to flat areas (blue), and high density populations are mapped to peaks in the surface (red). The user is able to rotate the plot on demand in all directions in the 3D space. Therefore, it is possible to explore the cell populations in a convenient way. Some previous 3D approaches on the parallel coordinate plot used the temporal order on the x- or z-axis for expressing the evolution of a certain event (18, 19). Our proposed method does not aim for a case history, but mainly for visualizing clustered events. The z-values (the height of the surface) are given by a density estimator applied on the x/y-plane from the parallel coordinates (20). This plotting generates a smooth surface depicting clusters as trends or peaks.

APPLICATION OF 3D PARALLEL COORDINATES IN HISTOLOGY

Phenotypic characterization of tissue-infiltrating leukocytes represents a major field of application for tissue cytometry, as the immune reaction associated with disease plays an essential role for understanding the process of disease formation as well as treatment. Therefore, detailed data on the composition of leukocytic populations are important in many fields of research and clinical routine (21–25). Figure 6 shows an example of a three-color MMTC-analysis of inflamed skin and provides quantitative data on cell types and numbers as well as localization within the tissue sample. Scattergrams are well suited to visualize the relation between two markers. As the amount of scattergrams increases by the factor of n × (n − 1)/2, with n being the number of parameters to be displayed, the number of plots becomes unmanageable soon. Visualizing the same data set with 2D and 3D parallel coordinate systems adds a whole new dimension, as the relations between all parameters can be visualized simultaneously in one graph (Fig. 7). Column 1 shows simple 2D parallel coordinate systems. When the data set contains many cells, the use of this type of graph is limited, as the lines make up dense areas and the information about relations between parameters is largely lost. Color coding of event numbers (Fig. 7, column 2) adds information about the “mainstreams” in the relationships among the parameters shown. The most efficient way to visualize such complex data is depiction of color-coded event numbers in 3D (Fig. 7, column 3). This tool of data visualization will support the recognition of interdependencies within multidimensional parameter-sets.

Figure 6. Tissue cytometric analysis of tissue infiltrating leukocytes using standard scattergrams. Sections of inflamed skin were triple stained using anti-CD45-PE, anti-CD14-APC, and SytoxGreen for labeling the DNA. Automated image acquisition and single-cell measurement was done using MMTC. The analysis procedure determines several parameters comprising size and mean relative intensity of all fluorescent markers. (a)–(c) shows three different scattergrams of one triple stained section with (a) plotting DNA intensity versus area/nucleus size, (b) DNA intensity versus CD45-PE reactivity, and (c) CD45-PE versus CD14-APC. Gate 1 depicts the CD45+/CD14+ monocyte population in blue. Please note that each dot represents a single leukocyte in situ. All cells belonging to this gate are also labeled in blue in the scattergrams (a) and (b). Gate 2 contains all nonleukocytes in the analyzed specimen, here primarily epithelial cells. Gate 3 shows those CD45⁺ leukocytes, which are CD14⁻. To control staining specificity, isotype-matched negative controls have been performed showing (d) CD45-PE versus IgG1-APC (negative control for CD14), (e) CD14-APC versus IgG1-PE (negative control for CD45), and (f) both controls against each other. Please note that the scaling for the nucleus area is linear while all other parameters are depicted with logarithmic scaling. [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.]

Figure 7. On the basis of the data set obtained with MMTC, which is described in Methods, three gates (gate 1, gate 2, and gate 3) are depicted (see Fig. 6c). In the first column the 2D parallel coordinate method is used, which produces hard interpretable cluttered plots. The plots in the second column add the density information to the 2D version. Plots which are created with the new 3D parallel coordinate method are presented in the last column. The plots in row (a) visualize the data set without any gate. Of course, the combination of the complete data set in one plot is the hardest to interpret, but also the most sophisticated approach. In rows (b)–(d) only one gated subset is taken, respectively. [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.]

DISCUSSION AND CONCLUSIONS

Experience revealed that the widely-used scattergrams are not sufficient in the described biological context. The main direction of the proposed methods is clear: there is no way around 3D in life science. The capabilities of the computers are by far developed enough to handle the additional computation amount that is caused by the 3D rendering.

For the acceptance of the proposed visualization technique, it is substantial to provide the new method in parallel to the scattergrams. As a result of this approach, the observer can get used to it by comparing the well-known scattergrams with the 3D parallel coordinate systems. In this concern, the visual connection between the two methods is important. Identical parts of a data set in separate views (plots) should be visually identifiable (26). Thus, the user can set a gate in a scattergram and see the influence on the 2D and 3D parallel coordinate plots immediately.

Although the brushing of gated populations is made by coloring of the polygonal lines (27, 28) for the 2D parallel coordinates, future research has to explore an appropriate approach for the 3D parallel plot. Gating methods directly in the 3D plot should be further investigated and evaluated in comparison with the conventional scattergram gating methods.

Today, we face cytometry experiments with more than eight different markers (1). In such cases, eight dimensions of the data come only from the markers' mean intensity. One can assume that other morphological parameters are taken into account, such as area, eccentricity, perimeter, major and minor axis length of the nucleus or of the cell. Then the number of dimensions is increased with up to 10 (for 18 dimensions 306 scattergrams are needed). With current methods of staining and observation, simultaneous staining of 12 markers are possible (132 scattergrams). We can easily imagine that soon we will have 20 markers (380 scattergrams), and the number can only increase in time.

Moreover, consequent extension of cytomics into the next organizational level in living systems (i.e. tissues and tissue systems) also requires an extended acknowledgement of the spatial aspect (which is usually ignored in most of the current techniques in microscopy) and lead to the concept of “tissomics” (29, 30). By using current visualization techniques (scattergrams), data meta-evaluation becomes rapidly unacceptably difficult. The numbers show that a substantial improvement of data visualization is necessary for the scientific progress in this field. The invention proposed brings the required boost by condensing all the scattergrams in one plot, solving both aggregation and overview problems. To illustrate this, Figure 8 displays a computer-generated data set with 10 dimensions in a 3D parallel coordinate plot, under different angles. Even for a lower number of markers, say four, the advantage of replacing 10 scattergrams with only one 3D parallel coordinate plot is substantial. In tissues, comparative analyses are often needed. One can imagine the case of comparison between two different regions of the stained section (i.e. normal regions versus tumor regions, or infiltrated regions). This would require finding differences and similarities between sets of tens or hundreds of scattergrams. By using the 3D parallel coordinates, the user has to compare only two diagrams, each 3D diagram representing the full data set of the selected region of tissue.

Figure 8. A computer-generated data set with 10 dimensions (parameters) is visualized in a 3D parallel coordinate system. There are 3 independent populations. The images represent the same diagram from above (a) and rotated (b)–(f) with 30°, 60°, 90°, 180°, and 210°, respectively. Notice the streams (“mountains”) showing the population. The peaks represent crowded event regions, while the flat areas represent low densities in the parallel axis spatial domain. [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.]

As stated in previous sections, a major disadvantage of the 2D plots like scattergrams or 2D parallel coordinate systems is the aggregation problem, which makes the user misinterpret the number of events in a certain population. In clinical terms, in most of the cases, this density represents exactly the clinical relevance of a result. The invention consists of the third dimension added to the 2D parallel coordinate system and represents the density of the events in the population, which is most likely, also the clinical relevance of the measured events. The observer can easily overview the 3D plot and identify the peaks (high density clinical relevant regions) and their evolution throughout the rest of the parameters. Having clinical relevance overview throughout all parameters is the major benefit of the new 3D parallel coordinate plot. With a brief look, the clinician can recognize the presence or absence of proved disease patterns or, in the case of a research experiment, identify important results.

In the forthcoming years, virtual reality (VR) (31) will have a considerable impact on the scientific world. Using this new approach the data visualization can take place in a virtual 3D environment. An additional step further goes the augmented reality (AR), where the computer generated data (in our case the 3D parallel coordinates) are combined with a real world environment (32). Imagine a 3D plot that is displayed in the space in front of the user. By using the hands or a kind of pointing device, the user is able to interact directly in the plot and perform actions like setting of gates, zooming, etc. With respect to the rapid development in 3D displays, VR, and AR, the 3D parallel coordinate method has a potential that is worth exploring.