Gradients play an important role in 2D image processing. Many edge detection algorithms are gradient-based. We are interested in 3D boundary detection which can be considered as an extension of 2D edge detection in 3D space. In this paper, an algorithm to automatically and quantitatively measure the suitability of gradient magnitudes in detection of 3D boundary points of confocal image stacks is presented. A Measurement Function is defined to evaluate the suitability of each gradient magnitude chosen to be the threshold for 3D boundary detection. The application of Gauss's Divergence Theorem provides a solution to calculate the Measurement Function numerically. The gradient magnitude at which the maximum of the Measurement Function is achieved can be utilized as the most appropriate threshold for gradient-based boundary detection and other operations like volume visualization.
Various forms of microscopy deliver a substantial amount of 3D cell biological information. However, datasets delivered by, for example, stacks of fluorescence confocal sections, are usually very large, noisy and often contain superfluous information. Segmentation is then necessary to identify and extract the objects of interest. Once identified, these objects can be further processed by volume visualization techniques such as volume rendering and surface rendering (Eils & Athale, 2003).
Boundary detection is of great importance in object identification in both 2D and 3D spaces. For 2D images, Canny (1986) developed a classic and effective method for edge detection using the first and second derivative of the intensity values. In 3D space, the aim of boundary detection is to extract surfaces of volumetric objects in image stacks. This can be considered as the 3D extension of 2D edge detection (Liu, 1977; Bomans et al., 1990; Monga et al., 1990). Several approaches have been developed specifically for the 3D boundary detection of confocal image stacks (Pudney et al., 1995; Fang et al., 2000).
In 3D space, a gradient is an important feature of volumetric points. A gradient can be used for boundary detection of confocal image stacks. Currently, however, the selection of the appropriate gradient threshold is usually implemented through trial-and-error or experimental analysis. The precise and automatic selection of an appropriate gradient threshold still remains a challenge for boundary surface detection of 3D confocal image stacks due to the complexity of confocal fluorescence images. It is crucial for the gradient threshold used in the boundary detection to locate as many correct boundary surface points as possible. In addition, it should be able to remove non-boundary points. Based on these criteria, in the present paper we describe a method to quantitatively and automatically evaluate the suitability of gradient magnitudes to be chosen as the threshold for detection of 3D boundary surface points. The present method relies on a quantitative function that we refer to as a Measurement Function. By defining the Measurement Function, it is possible to find the gradient magnitude at which the maximum of the Measurement Function is achieved. The gradient magnitude may then be subsequently used as the most appropriate gradient threshold for gradient-based detection of 3D boundary surface points. It is important to stress that the most appropriate gradient magnitude does not have to be the maximal gradient magnitude. However, the value of the Measurement Function for the most appropriate (optimal) gradient magnitude reaches maximum when compared with the values of the Measurement Function for other gradient magnitudes.
Materials and methods
10 µm frozen sections of embryonic day 13.5 mouse brain were collected on slides, permeabilized in 0.3% TritonX-100 for 5 min and then washed in PBS three times for 5 min. The sections were treated with RNAse (0.2 mm in PBS) for 1 h at RT before incubation with propidium iodide (50 µg mL−1 in PBS) for 15 min. After a 2 min wash in PBS, the sections were mounted in Dako mounting medium. A Bio-Rad MRC-600 confocal fluorescence microscope equipped with a krypton/argon laser was used for fluorescence microscopy. Z stacks were collected using Bio-Rad COMOS software. Although propidium iodide fluorescence did not fade appreciably during collection of the stacks, the images were processed to correct for any fading that might have occurred and to reduce noise. ImageJ software publicly available at http://rsb.info.nih.gov/ij/was used (Rasband et al.). To correct for any loss of fluorescence, images were processed using a ‘Bleach Correction’ filter after which they were normalized using the ‘Normalize’ function. Finally, to reduce noise, each section within the image stacks was convolved with a 3 × 3 × 3 median filter.
Results and discussion
A gradient is understood here as a vector giving the direction of the data changes whose direction is perpendicular to the surface of the object boundary. The magnitude of the gradient shows the rate of data changes and it is commonly used in classification and boundary detection of volumetric datasets. However, gradient magnitudes for volumetric points contained within objects of interest in any given biological dataset may cover a large value range due to a variety of technical problems related to data collection by fluorescence microscopy itself (Dobrucki, 2004). Consequently, it is difficult to identify the appropriate gradient magnitude that would serve as the threshold for boundary surface points detection. If the gradient threshold with a high value is chosen, the boundary will become more fragmented, although fewer false boundary points will be obtained. On the other hand, if we choose a gradient threshold with a low value, some points not belonging to boundary points will be selected and a false boundary will be generated. Furthermore, traditional selection of threshold based on trial-and-error or experimental analysis is time-consuming and not accurate. In order to find the most appropriate gradient threshold, we introduce a Measurement Function with which the suitability of a gradient magnitude to be chosen as the threshold in 3D boundary detection can be measured. After we calculate the Measurement Function for all the gradient magnitudes, the gradient magnitude at which the maximum of the Measurement Function is achieved can be used as the threshold and subsequently employed for boundary detection and volume visualization.
Given an intensity function f(x, y, z) in 3D space, the gradient g(x, y, z) of f(x, y, z) is:
where gx(x, y, z), gy(x, y, z) and gz(x, y, z) are the three components of g(x, y, z) in the x, y and z direction, respectively. The magnitude of g(x, y, z) is usually calculated as:
Consider a random gradient magnitude G. Within a volumetric confocal image stack, there are some points whose gradient magnitudes are equal to G. To measure the suitability of G as the potential threshold for 3D boundary detection, we assume that those points with gradient magnitudes equal to G are located on the boundary surface. A good gradient threshold should be able to detect as many correct boundary points as possible. As a result, the detected boundary surface should have a large area. But this is not sufficient. Meanwhile, the points located on the boundary surface should have a large data change rate. Thus, we define the Measurement Function for G to be the surface integral of the dot product of the gradient with the surface area for all the points whose gradient magnitudes are equal to G. So the Measurement Function for G can be written in form of:
where M(G) is the value of the Measurement Function for the gradient magnitude G, g(x, y, z) is the gradient for a point and its magnitude is equal to G, and ds is the surface area vector at a point whose gradient magnitude is equal to G. The direction of ds is parallel to its normal direction. Figure 1 shows a simple model of a confocal fluorescence object in the x-y plane. Red circles indicate background points, while green circles represent points belonging to the object of interest. A yellow line connects the yellow boundary points and demarcates the boundary contour. The blue arrow N represents the direction of the contour normal at a boundary point, while the black arrow g represents the direction of the gradient vector at that point. In confocal fluorescence images, the intensity values of objects are always larger than those of background points. Therefore, the direction of the gradient vector at the boundary point is opposite to the direction of the contour normal. In 3D space, the direction of the gradient vector is also opposite to the direction of the boundary surface normal for a point located on the boundary surface. Thus there is a minus sign in the Measurement Function.
According to Gauss's Divergence Theorem (Morse & Feshbach, 1953), the surface integral of a vector g(x, y, z) over the boundary surface and the volume integral of the divergence of the vector g(x, y, z), ∇·g(x, y, z), over the region surrounded by the boundary surface are related by:
where V is the group of points that is surrounded by the boundary surface (including the boundary surface points), dv is the volume size for each point within V, and ∇ is the gradient operator ∇ = (/x, /y, /z).
The ‘dot product’ of the vector operator ∇ with vector g(x, y, z), namely the divergence of g(x, y, z), can be calculated as:
Now the Measurement Function becomes:
In our confocal object boundary model (Fig. 1), we assume that the points surrounded by the boundary surface have the same intensity value and thus the divergences of gradients for those points are zero (0). Only the points located on the boundary surface have valuable gradient divergences. In addition, because a volumetric confocal dataset comprises discrete intensity values, the divergences of gradients ∇ · g(x, y, z), are discrete values as well. Usually a numerical method is employed to evaluate integrals for discrete datasets. In a volumetric dataset, each point is considered to have a unit cubic volume. Therefore, dv is equal to one (1). So now the Measurement Function can be replaced by:
The above equation shows that the Measurement Function for a gradient magnitude G could be calculated by summing up the divergences of the gradients for all the points that have the same gradient magnitude as G. For the other gradient magnitudes, the values of their Measurement Function can be calculated in the same way and recorded as a quantitative clue to reflect their suitability of being the threshold for detection of 3D boundary surface points. The gradient magnitude at which the maximum of the Measure Function is achieved can be considered to be the most appropriate threshold for gradient-based detection of 3D boundary surface points.
We tested our algorithm on several stacks of images of fluorescently labelled nuclei (‘round’ objects), intermediate filaments (long thin objects), microvascular endothelium (long tubular objects) and endoplasmic reticulum (irregular cytoplasmic objects) and obtained satisfactory segmentations in each case. To illustrate the present method, we have chosen image stacks of propidium iodide-stained nuclei from a frozen section of mouse brain. Figure 2(a) shows a volume rendered image (Levoy, 1988) of a confocal fluorescence dataset comprising 50 images of propidium iodide-stained nuclei. Each image within the dataset has 768 × 512 pixels and the size of the whole image stack is 18.7MB. When dealing with this image stack, we first calculated the gradient magnitude for each volumetric point inside this image stack using the Central Difference Method (Bentum et al., 1996) in 4.7 s with a PC equipped with 1.7GHz CPU and 512MB RAM. Then, for each existing gradient magnitude, we calculated and recorded the value of the Measurement Function by summing up the divergences of the gradients for the points with gradient magnitude equal to the given gradient magnitude. This took us 4.9 s. Figure 2(b) is a graph showing the obtained values of the Measurement Function for each gradient magnitude for the image stack shown in Fig. 2(a). The maximum value of the Measurement Function is achieved when the gradient magnitude is equal to 47 (the vertical line).
To testify the result, we conducted the gradient-based boundary detection on the dataset shown in Fig. 2(a). The points whose gradient magnitudes are equal to, or bigger than 47, are to be located on the boundary surface. Figure 3(a) shows the section #30 of the image stack shown in Fig. 2(a). Figure 3(b) shows the boundary surface points detected using the most appropriate gradient threshold equal to 47. For comparison, Fig. 3(c and d) show the boundary surface points detected on the section #30 with another two gradient thresholds that have lower values of the Measurement Function.
The boundary points detected with our algorithm can also be employed in volume visualization. Currently, the derivation of the appropriate iso-value for surface rendering of 3D confocal fluorescence datasets still remains a challenge. According to Fang et al. (1998), the intensity value where the maximum number of boundary surface points occurs can be considered as the optimal iso-value. Therefore, after we obtained the boundary surface points detected with the most appropriate gradient threshold, which is identified through calculating the Measurement Function, we generated a histogram based on the intensity values of all boundary surface points. The intensity value where the histogram reaches its local maximum can be used for surface rendering. Figure 4(a) shows another stack of confocal images of nuclei consisting of 90 optical sections. The curve showing the values of the Measurement Function for each gradient magnitude for this stack is illustrated in Fig. 4(b). Next, after the detection of the boundary surface points with the most appropriate gradient threshold, we generated a histogram of intensity values for all the detected boundary surface points as shown in Fig. 4(c). Figure 4(d) shows the rendered image using surface rendering with the iso-value obtained from Fig. 4(c) (the vertical line). The boundary surface points obtained using the most appropriate gradient threshold can also be applied to improve the visualization in volume rendering by designing a two-dimensional transfer function. Transfer function design in volume rendering is a procedure to assign optical properties, like colour and transparency values, to volumetric points based on their numerical values before rendering (He et al., 1996; Kniss et al., 2002). Transfer functions can facilitate the user's visual perception of the relationship between subcellular structures. In the example shown in Fig. 4, we first assigned two different colours to the boundary surface points and the internal points, respectively, based on their gradient magnitudes. Next, we set the boundary surface points to be semitransparent by assigning an appropriate transparency value for them. The final visualization of the dataset shown in Fig. 4(a) after application of the transfer function is shown in Fig. 5.
In this paper, we defined a Measurement Function to evaluate the suitability of gradient magnitudes in 3D boundary points detection for confocal images stacks. The application of Gauss's Divergence Theorem offers a practical solution to calculate the Measurement Function numerically. The calculated values of the Measurement Function help to identify the most appropriate gradient threshold for 3D boundary points detection. In addition, we illustrated how the detected 3D boundary points can be applied in volume visualization.
This study was supported by grants from the Agency for Science, Technology and Research (A*STAR) of Singapore to Y.Y. Cai and from the Canadian Institutes of Health Research to M. Opas. M. Opas is a member of Heart & Stroke/Richard Lewar Centre of Excellence.