The conceptual separation of photon data and metadata has several advantages for image data. Multidimensional image data sets are often very large (a multifocal plane, time-lapse recording can produce gigabytes of binary photon data). However, metadata typically consists of descriptions of the experimental parameters used, the number and extent of each dimension of data, the photon data file name and a set of searchable keyword tags. These data are naturally in character format and can easily be incorporated into a database to facilitate searching and retrieval. The OME system has a built-in image viewer that can view 4D data sets within the browser. For large data set visualization or analysis, OME has a connection framework that allows it to be interfaced with client-side analysis tools such as MATLAB and VisBio (discussed below).
VisBio is a computer application that we are developing for the interactive graphical display and quantitative analysis of biological image data of arbitrary dimensionality (31). Through the VisBio interface, users are able to import microscope data in any file format and interactively explore and measure the data within 4D recordings of specimens. VisBio goes beyond the capabilities of current commercial and public domain software because it is being specially tailored to the demands of handling and animating massive data sets fluidly. Furthermore, VisBio enables the interactive representation of recordings in which each spatiotemporal pixel element contains multiple dimensions, e.g. emission intensity, color spectrum, and fluorescence excited state lifetime. The program will thereby satisfy demands that are being generated by current systems and new imaging systems under development. Over the last couple of years, we have been developing a pilot implementation of VisBio in response to the need for advanced analysis tools to process data generated by novel imaging instrumentation being developed by our group and others. This implementation is available from the VisBio website as the v2.31 stable release.
Meanwhile, to guarantee the flexibility necessary for effective analysis of more exotic multidimensional data types such as spectra and lifetime, we have begun work on VisBio's third major revision, v3.00. We have remodeled the architecture of VisBio to be even more powerful and flexible. Core data and display code have been refactored, we have begun work on a VisBio–OME interface and support for several additional microscopy formats has been added. Because this new feature set offers numerous advantages over VisBio v2.31, we are releasing a series of beta versions as new functionality is implemented and existing features are ported from v2.31.
Built on existing software. VisBio has been built with the VisAD scientific visualization toolkit (http://www.ssec.wisc.edu/~billh/visad.html) and includes ImageJ; it therefore inherits a multitude of features from both tools. Work done to improve or enhance ImageJ or VisAD automatically benefits VisBio. More importantly, as additional or improved functionality in VisBio has been needed, we have implemented it in the core VisAD package, benefiting not only the biological community, but also all users of VisAD in general.
File formats. Because very few image analysis packages support every format, we have made it a priority to support every major microscopy imaging format, including multipage TIFF, Bio-Rad PIC, Zeiss LSM, Zeiss ZVI, Metamorph STK, Olympus Fluoview, Openlab LIFF and QuickTime movie, with planned support for several others (Leica, Nikon, IPLab). Support for these formats has been wrapped into the VisAD toolkit itself, so that all VisAD-based applications can take advantage of our work.
Fully multidimensional. VisBio can import image data of any dimensionality. It provides sophisticated subsampling features, and enables visualization by mapping dimensional axes to the “image stack” and “animation” parameters. Flexible export features allow data sets to be saved to a group of files in Bio-Rad PIC, multipage TIFF or QuickTime movie format.
Generalized data engine. VisBio can work with any number of data objects simultaneously, imported from file groups on disk or computed from other data objects using built-in “data transforms.” A “subsampling” data transform enables visualization of a subset of a data object. Also implemented are a preliminary multispectral mapping algorithm that allows interactive weighting of each spectral channel (Fig. 2), a transform that computes maximum intensity projections across a given dimensional axis, one for creating image overlays in 2D and one for performing arbitrary slicing of an image stack in 3D. We have designed the data transform functionality to handle any algorithmic visualization need, including fluorescence lifetime curve fitting, spectral analysis routines, denoising and other image manipulation routines.
Figure 2. Seventeen-channel spectral image of a methyl green-stained section of uterus imaged with MP excitation. VisBio displays image chrominance as a weighted function of wavelength values in each channel. The left-hand image is colored according to VisBio's “best guess” mapping, with the first 1/3 of the channels equally weighted toward red, the second 1/3 toward green and the last 1/3 toward blue. The right-hand image illustrates the discrimination of nuclei by overweighting the contribution of channel 1 (660 nm) and negatively weighting channel 5 (600 nm) with respect to the red color component. This scheme reveals nuclei as red and surrounding tissue as turquoise. (Uterus section was prepared by Al Kutchera, Midwest Microtech.)
Modular display logic. VisBio's display logic allows the creation of any number of displays. Each data object can be added to any number of displays, and each display can contain any number of data objects. Thus it is possible to visualize two or more data sets in 3D, side by side, or two image stacks simultaneously within the same display window. The dimensional axes mapped to Z-axis and animation can be quickly switched on the fly. The user can query pixel values at any position within the display. Projection options for zooming, rotation, panning and aspect ratio control are available.
Variable resolution. VisBio makes use of multiresolution functionality, displaying data in lower resolution to improve animation speed, but “burning in” data at full resolution when more detail is needed for closer inspection. This approach not only improves rendering and animation speed, but also cuts down on memory use, because only the currently displayed time step must be maintained in memory at full resolution at any given time.
Flexible color mapping. VisBio provides precise control over how colors are mapped. A data set may have more than one color channel associated with each pixel. VisBio allows for complete control over each channel's color table. It also provides shortcuts to compute a color composite from all channels; to map channels to the red, green and blue color components of an RGB color space; or map them to the hue, saturation and value components of an HSV color space.
Arbitrary slicing (Fig. 3). Planes can be placed at any orientation in the 3D image stack, “slicing” it at any angle. Data along these arbitrary slices are interpolated and displayed at a user-defined resolution. Arbitrary slice computation is fast enough that animating these slices at reasonable speeds is practical.
Figure 3. VisBio performing an arbitrary slicing of a C. elegans embryo undergoing cell fusion and imaged by multiphoton microscopy. The resultant slice is seen superimposed in the 3D display, as well as in the 2D display, (data set provided by Dr. William Mohler of the University of Connecticut, Farmington, CT.)
Volume rendering (Fig. 4). Support for rendering each image stack as a semitransparent volume is implemented. Control over the level of transparency is provided in order to locate the optimum visual setting for eliminating noise but preserving the important aspects of the data.
Figure 4. Three-dimensional reconstruction of a confocal data set of mouse vasculature, (data set provided by Dr. Rich Halberg, Mr. Lance Rodenkirch and Dr. William Dove of the University of Wisconsin-Madison)
Measurement tools. A set of tools for performing measurements on the data are available. The distance between any two points in an image stack can be computed, and important events in the data can be flagged with markers. Measurements can also be set to be standard across each slice of every time step. Lastly, if more complex analysis is desired, the measurements can be saved to a text file formatted for easy importing into Excel or other spreadsheet application.
OME import interface (Fig. 5). We have added the ability to upload a data set from VisBio to an OME image database across the network. This functionality provides OME with an interactive, client-side, multidimensional data import procedure that works with any of the growing number of VisBio-supported microscopy file formats. We have also created a preliminary adaptation of this interface as a plug-in for ImageJ, so that image stacks within ImageJ can be imported into the OME system as well.
Figure 5. ImageJ and VisBio interacting with OME. The browser displays multiple data sets that have been uploaded to our OME database using VisBio and ImageJ.