Generation of digital phantoms of cell nuclei and simulation of image formation in 3D image cytometry

Abstract

Image cytometry still faces the problem of the quality of cell image analysis results. Degradations caused by cell preparation, optics, and electronics considerably affect most 2D and 3D cell image data acquired using optical microscopy. That is why image processing algorithms applied to these data typically offer imprecise and unreliable results. As the ground truth for given image data is not available in most experiments, the outputs of different image analysis methods can be neither verified nor compared to each other. Some papers solve this problem partially with estimates of ground truth by experts in the field (biologists or physicians). However, in many cases, such a ground truth estimate is very subjective and strongly varies between different experts. To overcome these difficulties, we have created a toolbox that can generate 3D digital phantoms of specific cellular components along with their corresponding images degraded by specific optics and electronics. The user can then apply image analysis methods to such simulated image data. The analysis results (such as segmentation or measurement results) can be compared with ground truth derived from input object digital phantoms (or measurements on them). In this way, image analysis methods can be compared with each other and their quality (based on the difference from ground truth) can be computed. We have also evaluated the plausibility of the synthetic images, measured by their similarity to real image data. We have tested several similarity criteria such as visual comparison, intensity histograms, central moments, frequency analysis, entropy, and 3D Haralick features. The results indicate a high degree of similarity between real and simulated image data. © 2009 International Society for Advancement of Cytometry

CURRENT biomedical research relies strongly on computer-based evaluation, as the vast majority of commonly used acquisition techniques produce large numbers of numerical or visual data. Each individual technique (optical microscopy, PET, MRI, CT, ultrasound, etc. (1)) has its advantages that make it suitable for use in selected fields of research. However, each technique also has its own drawbacks (blur, noise, various aberrations) (2–4). In this sense, one should keep in mind that biomedical data supplied by scientific or diagnostic instruments always suffer from some imperfection and therefore cannot be directly used for further analysis, evaluation or measurement. To guarantee more accurate results, some preprocessing is required.

Let us focus on the area of image reconstruction. Here, the search for ground truth approximation is called an image restoration (5–7), and it constitutes the best possible method for image recovery. It is done in time-reverse order; that is, the image defect caused by the phenomenon that appeared in the whole acquisition process as the first one must be eliminated as the last one. The same holds for the second, the third and the following defects. As soon as the captured image is restored, it can be further submitted to measurement or segmentation algorithms. If all the defects and the noise were removed during image restoration, we should get the original unaffected ground truth image data as well as derived ground truth analysis results. In practice, this is not possible. The task is too complex, so the imperfection is diminished rather than totally pruned away. Moreover, due to the nonexistence of ground truth image data, it is difficult to judge whether the restoration has been done correctly and hence whether the final evaluation is valid.

Let us avoid searching for inverse sequences and instead try to follow the chronological order, that is, the acquisition process. First, let us create a digital phantom of an object that we want to observe. Then let us submit this object to all the phenomena that appeared during the real acquisition. The final synthetic image will be very similar to the image acquired under real conditions. In this way, we design a simulator: a tool for digital phantom generation and simulation of its acquisition. A user can then use the simulator to evaluate the correctness of various image restoration (or image analysis) algorithms by comparing their results with ground truth image data (or ground truth analysis results) derived from the digital phantom (see Fig. 1).

Figure 1. Quality control (QC) in biomedical image data processing and the advantage of using simulators rather than expert know-how for this purpose. From the image processing point of view, ground truth derived from expert knowledge is imprecise and laborious to obtain, whereas phantom-based ground truth is exact and easy to generate in large quantities. On the other hand, from the biological point of view, expert-based approximate ground truth might be more precise because it corresponds to real objects, not synthetic digital phantoms. In this article, however, we concentrate on image processing QC, not QC related to modeling a biological system.

It is noteworthy that the quality of each simulator depends strongly on the demands of its designer. The simulator can be simple or very sophisticated. At present, the simulators appear in nearly every field of biomedical research. Although many authors designed their simulators as compact toolboxes, the simulation process can always be split into three principal parts: (I) digital phantom object generation, (II) simulation of signal transmission, and (III) simulation of signal detection and image formation.

(I) Phantom Generation

Macroscopic objects like the kidneys, heart, brain, muscles, or blood vessels are examples of objects that are relatively easy to model, as their shape and behavior are well understood. The problem is to choose the model that fits the observed data to the greatest extent possible. Generation of the static models has already been studied extensively when creating the reference phantom image for the brain (8–12). Jensen et al. proposed simple models of the kidneys and the fetus (13). When handling these types of image data, no movement is expected. Completely different object types represent the heart or blood vessels, where activity is studied and kinetic models (14–17) are needed.

In contrast with the macroscopic world, when handling microscopic objects like cells, the main issue is that their components and DNA structure cannot be simply observed with the naked eye. The observer needs a special technique in order to acquire at least some visual draft of the studied objects. As there are many unwanted phenomena causing image imperfection in this case, the final image is strongly affected by these degradations, and some amount of image restoration is required. Unfortunately, the process of restoration is not exactly reciprocal to acquisition; one must admit that the restored image is an estimate rather than the original image data. That is why no one knows the exact shape of microscopic structures; hence, there is no ground truth describing the exact shape of these objects. For many microscopic objects, either they are expected to have a very simple structure, or their shape is deliberately simplified.

Regarding the simplest point-like objects such as FISH (18, 19) spots, Grigoryan et al. proposed a simulation toolbox for generating large sets of spots (20). Each spot was represented by a sphere randomly placed in 3D space. Two individual spheres were allowed to overlap, but only under specific conditions. Manders et al. also addressed the issue of virtual spot generation (21). They verified a novel region-growing segmentation algorithm over a large set of Gaussian-like 3D objects arranged in a grid.

When creating cell-like or nucleus-like objects, the algorithm uses the most popular shapes, such as circles and ellipses in 2D and spheres and ellipsoids in 3D space. To check the quality of the new cell nuclei segmentation algorithm, Lockett et al. (22) generated a set of artificial spatial objects in the shape of curved spheres, ellipsoids, discs, bananas, satellite discs, and dumbbells. Lehmussola et al. (23) designed a more complex simulator called SIMCEP that could produce large cell populations. However, the toolbox was designed for 2D images only, and its extension to higher dimensions was not straightforward. Later, Svoboda et al. (24) extended this model to manipulate fully 3D image data, but with a limited number of generated objects.

Up to now, the majority of authors have focused on the design of spots or nuclei. Recently, Zhao et al. (25) designed an algorithm for generating an entire 2D digital cell phantom. Here, they modeled the whole cell, including the nucleus, proteins, and cell membrane, using machine learning from real data. This is a different approach than that of other studies that use basic shapes and their deformations. The advantage of the machine learning approach is that it might extract a more precise shape model from real data, but the disadvantage is that the model cannot be described in precise mathematical terms.

Although the previously mentioned papers and articles were interested in individual object modeling, Graner et al. (26) focused on generating large cell populations. They adopted the statistical large-Q Potts model to simulate the reorganization of uniformly distributed cell-like objects to guarantee the natural shape and distribution of the cells.

(II) Signal Transmission

The second stage covers the period during which a signal is transmitted through the environment. One of the most typical environment characteristics is the impulse response of the system, often called the point spread function (PSF) (2), which is common in optical microscopy as well as in PET and ultrasound imaging. This is the pivotal phenomenon drastically affecting the quality of the final results, and it is usually simulated (27) by convolving the incoming signal with the given PSF. However, the vast majority of the authors reduced this simulation to the simple Gaussian kernel (20, 21, 28, 29), as it is commonly understood to be a good approximation of any PSF. This phenomenon is not the only one affecting the transmitted signal. In optical microscopy, for example, an incorrect position of the light source, uneven illumination (24), chromatic (30) and monochromatic aberrations (31), or some reflection or refraction on lens surfaces may create some artifacts as well. One should keep in mind that signal reflection and refraction happen in other modalities as well. Ultrasound imaging is an example of a technique using these phenomena.

(III) Signal Detection and Image Formation

The final stage corresponds to the detection of the signal with the device sensors and its conversion to the digital representation, which naturally has some problems. The use of sensors introduces Poisson noise (22, 23, 27, 28), which is an artifact typically seen even with the naked eye. Aside from Poisson noise, the A/D converter and amplification electronics introduce a certain level of additive white Gaussian noise (22, 29). If the equipment is not properly cooled, the level of dark current noise increases. The CCD detectors, for example, are well known for their fixed-pattern noise and blooming effect. Finally, one should keep in mind that the arrangement of the signal sensors determines the final image sampling.

Examples of Simulators

Depending on the acquisition device, simulators can easily be split into several groups: optical microscopy, CT, PET, MRI, and ultrasound imaging (see Table 1).

Table 1. An overview of the recent toolboxes and papers dealing with the generation of phantoms of biomedical objects and their acquisition

Simulated		Toolbox Name	Author(s)	Stages	Dims	Reference	Year
Object	Technique
Spots	Fluorescence microscopy	–	Manders et al.	I,III	3D	(21)	1996
Spots	Fluorescence microscopy	–	Grigoryan et al.	I,II,III	3D	(20)	2002
Nuclei	Fluorescence microscopy	–	Lockett et al	I,III	3D	(22)	1998
Nuclei	Fluorescence microscopy	–	Solorzano et al.	I,II,III	3D	(29)	1999
Nuclei	Fluorescence microscopy	–	Svoboda et al.	I,II,III	3D	(24)	2007
Cells	Fluorescence microscopy	SIMCEP	Lehmussola et al	I,II,III	2D	(23, 32)	2005
Cells	Fluorescence microscopy	–	Dufour et al.	I,II,III	3D+time	(33)	2003
Cells	Fluorescence microscopy	–	Zhao et al	I	2D	(25)	2007
*	Fluorescence microscopy	SVI Huygens	Voort et al.	II,III	3D	(27)	1995
Cells		CompuCell3D	Graner and Glazier	I	3D+time	(26)	1992
*	CT	CTSIM	Rosenberg	II,III	2D	(34)	2002
Brain	PET	PETSIM	Ma et al.	II,III	3D	(28)	1993
Brain	MRI	–	Collins et al.	I	3D	(12)	1998
Brain	MRI	–	Tofts et al.	I	3D	(8)	1997
Heart motion	MRI	–	Waks et al.	I,II,III	3D	(14)	1996
Brain	MRI	–	Rexilius et al.	I	3D	(9, 10)	2003
Brain	MRI	Brain Web	Kwan et al.	I,II,III	3D	(35, 36)	1996
Tissue motion	Ultrasound	–	Schalaikjer et al.	I	3D	(16, 17)	1998
Heart motion	Ultrasound	–	Rabben et al.	I	3D	(15)	1993
Heart motion	Ultrasound	–	D'hooge et al.	I,II,III	2D	(37)	2003
*	Ultrasound	Ultrasim	Holm	II,III	2D	(38)	2001
*	Ultrasound	Field II	Jensen et al.	I,II,III	3D	(13, 39, 40)	1996
*	Ultrasound	–	Wójcik et al.	II,III	3D	(41)	1997

In the area of ultrasound imaging, Jensen et al. (13, 39) designed a very complex and flexible engine called Field II, based on manipulation with a 3D linear acoustic field. An alternative approach is Ultrasim (38), which could generate at most 2D data. Unlike other authors, Wójcik et al. (41) designed a nonlinear model of acoustic wave propagation in tissues. In the field of elastography, D'hooge (37) focused particularly on simulating and tracking heart movement.

In the MRI area, Cocosco et al. designed a web interface called BrainWeb (36), which also included a very useful brain phantom database (11). Waks designed a complex cardiac motion simulator (14), focusing on tagged MRI data.

Ma (28) found MRI phantom data very useful when designing a PET simulator of brain images. In CT imaging, Rosenberg designed a general purpose CT simulation toolbox (34).

In optical microscopy, great progress in simulating image data has been made mainly during the past 3 years, during which several papers (20, 23, 24, 33) have given straightforward recipes or even offered useful software toolboxes. However, the development of simulators in this area is still beginning; hence, great progress might be expected.

In this article, we present a novel fully 3D technique simulating the whole process of image acquisition from optical microscopes. The whole process is split into three main independent consecutive sub-processes: digital phantom generation, simulation of signal transmission, and simulation of signal detection and image formation, outlining the structure of section 3. Section 4 presents the results. In the figures, we present the quality of our method as well. In the conclusion, we discuss the goals that we have achieved.

REFERENCE DATA

Before starting to develop any simulation toolbox, one should have expert knowledge or at least access to a sufficiently large database of real reference data. Such a source of information is very important: digital phantom generation is pointless without knowledge of the nature and the structure of the images of real objects. In particular, this knowledge is helpful when tuning the final simulator behavior.

Equipment

In this article, the optical microscope Zeiss Axiovert 200M with Yokogawa CSU-10 confocal unit and Andor iXon 887 back illuminated EM CCD camera connected to standard PC were used to acquire real image data. Concerning the microscope settings, the alpha Plan-Fluar objective (100×/1.45 NA) was used.

For the time being, the simulator (developed in GNU C/C++) can imitate an optical system comprising four different CCD cameras (see Table 2) and four optical microscopes (see Table 3) with two types of confocal units and various objectives. In general, there is no restriction put on the whole optical system; that is, the component list can be extended arbitrarily. The only requirement is that each component has to be described properly.

Table 2. List of simulated cameras

Vendor	Camera Name
Andor	iXon 887 BI
Photometrics	CoolSNAP HQ
Photometrics	Quantix KAF-1400
Princeton Instruments	MicroMax 1300-YHS

Table 3. List of simulated microscopes

Vendor	Microscope Name	Optional Extension
Zeiss	Axiovert 200M	CSU-10 confocal unit
Zeiss	Axiovert 100S	CARV confocal unit
Leica	DMRXA	CSU-10 confocal unit
Leica	DMIRE2	–

We evaluated the results with two servers. The first one had two Intel Core 2 Quad 2.0 GHz processors, 8GB RAM and the SUSE Linux operating system. The latter one had an Intel Core 2 Quad 2.4 GHz processor, 4 GB RAM, and the Microsoft Windows Server 2003 operating system. We programmed the system in Matlab (Mathworks) with the DIPimage toolbox (Delft University of Technology, Netherlands).

Biological Material

Microspheres

TetraSpeck™ microspheres (Molecular Probes, Eugene, OR) of 0.1 μm (T-7279), 0.2 μm (T-7280), and 0.5 μm (T-7281) diameter were used in this study. Each microsphere is stained with four different fluorescent dyes simultaneously: blue (365/430 nm), green (505/515 nm), orange (560/580 nm), and dark red (660/680 nm). In this study, we observed only the green channel. For illustration, see Figure 2, in which the given 3D image data are visualized.

Figure 2. A sample image containing microspheres of 0.5 μm diameter with peak excitation/emission wavelengths of 560/580nm. This 3D figure consists of three individual images: the top-left image contains a selected xy-slice, the top-right image corresponds to a selected yz-slice, and the bottom one depicts a selected xz-slice. Three mutually orthogonal slice planes are shown with ticks.

HL-60

One cell line representing tissue cultures was used: human promyelocytic leukemia cells HL-60 (European Collection of Cell Cultures) maintained between 1 and 5 × 10⁵ cells/ml (see Fig. 3). These cells grow in RPMI-1640 medium containing 10% fetal bovine serum, penicillin (100 U/ml), and streptomycin (100 mg/ml) at 37°C in a 5% CO₂ atmosphere. Dense suspension of cells in PBS (50–100 μl) was dropped onto poly-L-lysine coated microscopic slides. After attachment to the slide (about 15 min), cells were fixed with 3.7% paraformaldehyde in 250 mM HEPEM (RT;12 min) and washed in 1× PBS (3 × 5 min). After washing, chromatin in the cell nuclei was stained with DAPI.

Figure 3. An example of four real HL-60 cell nuclei stained with DAPI. White arrows show the location of one selected nucleolus. Each 3D figure consists of three individual images: the top-left image contains a selected xy-slice, the top-right image corresponds to a selected yz-slice, and the bottom one depicts a selected xz-slice. Three mutually orthogonal slice planes are shown with ticks.

Granulocytes

Granulocytes are a category of terminally differentiated white blood cells characterized by the presence of granules in their cytoplasm. Granulocytes are also called polymorphonuclear leukocytes (PML) because of the varying shapes of the nucleus. Human leukocytes that contain granulocytes (see Fig. 4) were isolated from heparinized peripheral donor blood from healthy patients using density gradient sedimentation composed from Telebrix (Guerbert, France) and a 3.8% solution of dextran (Sigma) in a 2:8 ratio. Granulocytes were isolated from the upper layer with leukocytes using Histopaque-1077 double gradient density centrifugation (Sigma). Residual erythrocytes were removed by an erythrocyte lysis buffer. Concerning fixation and staining of the granulocytes, the same method was used as in the case of HL-60 cells.

Figure 4. An example of four real granulocyte nuclei stained with DAPI. Each 3D figure consists of three individual images: the top-left image contains a selected xy-slice, the top-right image corresponds to a selected yz-slice, and the bottom one depicts a selected xz-slice. Three mutually orthogonal slice planes are shown with ticks.

METHOD

Following the introduction, the whole toolbox is split into three independent parts: digital phantom generation, signal transmission, and signal detection and image formation. Each one represents a specific stage appearing during the final image synthesis. The text of this section will follow this structure.

Digital Phantom Generation

Initially, the toolbox selects the type of the object to be simulated. Currently, the toolbox can simulate three types of objects.

HL-60

In the experiments using the HL-60 cell line, investigators are usually interested in studying the cell nucleus that occupies most of the cell volume. Therefore, the following text will focus on modeling nucleus shape and texture.

When simulating the appearance of the HL-60 standard cell line, we can presume the shape of the initial object to be spherical, as this object type is topologically equivalent to a sphere. However, the basic objects like spheres or ellipsoids are too simple and regular. Because the aim is to simulate real objects, a certain amount of irregularity is required. For this purpose, we used the PDE-based method to distort the object shape. The idea is based on viewing the object boundary as a deformable surface. The deformation is realized with fast level set methods (42) using artificial noise (43) as a speed function (44). An example of such a deformation process result can be seen in Figures 5a and 5b.

Figure 5. HL-60 nucleus digital phantom generation: (a) the original ellipsoid representing the rough primitive mask of a generated cell nucleus and (b) the same object after the 3D PDE-based deformation. (c) Object equipped with texture defining the internal structure. (d) The image after passing through the optical system. Each 3D figure consists of three individual images: the top-left image contains a selected xy-slice, the top-right image corresponds to a selected yz-slice, and the bottom one depicts a selected xz-slice. Three mutually orthogonal slice planes are shown with ticks.

Besides the shape, the texture of the nucleus image profile reveals important information about the cell activity. In each stage of the cell cycle, chromatin has different properties, and hence when stained, it looks different. For these purposes, the study and the measurement of heterogeneity of chromatin is an important task (45). Essentially, there are two ways to generate synthetic texture: algorithms for texture synthesis (46–48) and methods for procedural texture modeling (43). Here, we decided to use the latter one. The texture function is defined as a sum of several Perlin's noise functions (43):

• (1)

where β controls flickering of the texture, α is responsible for smoothness, and N controls whether the result is still coarse (for N < 5) or fine enough (see Fig. 5c). However, certain nucleus parts may not contain chromatin and hence may remain unstained. These locations are either left blank (without any texture) or defined as very dark. The latter case corresponds to an unwanted staining effect. The nucleoli (see Fig. 3) might be an example of such an object type that typically appears as a dark (not stained) place in the image of a nucleus. It was discovered empirically that there is only one nucleolus per healthy nucleus in human cells. As for cancerous cells, there might be more than one nucleolus. The shape of such a nucleolus is mostly spherical or slightly deformed. Because of this property, its generation follows the same idea as the generation of the whole HL-60 nucleus.

Granulocyte

Granulocyte is a type of cell containing a nucleus with highly condensed chromatin. Such a nucleus typically has three to five lobes connected by very narrow, barely perceivable channels. The shape of each lobe resembles a slightly deformed sphere. Concerning the inner structure, each lobe contains a cavity, as the chromatin is mostly concentrated in the nucleus periphery. Similarly to the HL-60 cell line, we are mainly studying the nucleus context, and hence the digital phantom designed in the following text is expected to represent only the nucleus.

First of all, note that the lobes are distributed within a very small area and are sometimes even tightly pressed to each other. When generating the granulocyte structure, a center of mass is either given by the user or generated randomly within the image space. Subsequently, the position of each lobe follows the Gaussian distribution with a mean in the given center and a very low standard deviation. Because of the similarity of the basic lobe profile with the HL-60 cell nucleus, the same approach as in the previous paragraph was used to generate each lobe. Finally, the lobes were connected with spatial cubic splines simulating the channels between the individual lobes. The simulation of chromatin concentration near the nucleus periphery was managed with a distance transform weighted by the procedural texture (see Fig. 6c).

Figure 6. Granulocyte digital phantom generation: (a) the initial group of ellipsoids representing the rough primitive estimate of lobes; (b) each lobe slightly deformed; (c) the lobes equipped with cavities and nonhomogeneous structure of chromatin; (d) image after passing through the optical system. Each 3D figure consists of three individual images: the top-left image contains a selected xy-slice, the top-right image corresponds to a selected yz-slice, and the bottom one depicts a selected xz-slice. Three mutually orthogonal slice planes are shown with ticks.

Microsphere

Microspheres are spherical objects of known diameter without any texture or distortion. Hence, a simple sphere with a given diameter can represent each digital bead phantom. The microspheres are typically used when calibrating the optical system. Here, they can be used to verify the correctness of the selected PSF and hence the plausibility of the final synthetic image.

Notation

To summarize the previous paragraphs, the generation of any digital phantom can be described simply as:

• (2)

where I_background is an initial empty image with the nonspecific staining effect, and P is a multiset of all the available phantom types. When staining the objects using fluorescent dyes, some parts of the specimen or cover glass may unwillingly be stained. This phenomenon manifests itself as a barely perceptible veil covering the whole glass. Finally, MakePhantom(.) is any function responsible for creation of a selected digital phantom. As three types of objects are currently ready for simulation, the variable p can be substituted for HL-60, granulocyte, or microsphere.

Signal Transmission

Light conditions

In traditional light microscopy, both laser and mercury lamps are used for visualization purposes, but neither type can spread light within the specimen uniformly. Therefore, the distribution of the light intensity is assumed to be quadratic (49) over each xy-slice:

• (3)

where vector a = (a₀, a₁, a₂, a₃, a₄, a₅) defines the shape of a quadratic surface typically widely opened. In some cases, there may even be two quadratic profiles in the image. This happens if the mercury lamp arc and its mirrored image are located far from each other. Very often, the quadratic surface peak is not centered with respect to the image center. This is usually the case if the camera or lamp is shifted relative to optical axis of the microscope.

This phenomenon makes objects that are placed farther from the optical axis look darker than the others. Cropping such an object from this large image usually leads to neglecting this effect because within this small sub-image the effect is hardly recognized with the naked eye. Submitting such an image to the discrete Fourier transform (DFT), one can reveal the uneven illumination again, though this is not seen with the naked eye. The explanation is based on DFT properties. Because the Fourier domain is strictly periodic, one should keep in mind that the right side of an image is virtually joined with the left side and the upper side with the lower one. If there is at least a small amount of light gradient across the image, due, for example, to uneven illumination, the left side does not join the right side smoothly. This phenomenon manifests itself as a centered cross in the Fourier domain. The stronger the light gradient, the stronger the cross appears (see Fig. 7).

Figure 7. Uneven illumination of the specimen that occurs during image acquisition causes the Fourier transform counterpart to contain a cross centered in the position of zero frequency: (a) the original image; (b) its Fourier power spectrum.

When eliminating this phenomenon, a quadratic surface is typically fitted to the original image. The result is then obtained as the pixel-wise ratio of the original image and the given surface. Because in the case of simulation we solve the inverse problem, we can apply the estimated quadratic surface to the given image simply by multiplying.

Impulse response

Each optical system can be described by a point spread function (PSF) that is the impulse response of this system determining the amount and the characteristics of image blur. The PSF can either be measured empirically (real PSF) or estimated (theoretical PSF (50)). The theoretical PSF is usually based on prior knowledge of the optical system properties (confocal or wide-field microscope, objective, wavelength, etc.). Here, we used a real PSF. The PSF was subsequently used as a convolution kernel applied to the given image (see Fig. 5d or 6d).

Notation

The consecutive processes of this stage can be described formally as follows:

• (4)

where I_phantoms is the image containing pure digital phantoms, and I_light is the image representing the light decay effect. The operator “.” corresponds to pixel-wise multiplication, and the operator “*” corresponds to convolution.

Signal Detection and Image Formation

Signal detector resolution

One should keep in mind that there are several factors affecting the image resolution. The signal detector contributes with its pixel size, which, together with total magnification of the optical system, defines spatial sampling frequency. In our application, we establish the correct image resolution based on this information.

Dark current signal

There is a signal generated internally in the detector even if no photon is coming, for example, in the darkroom. This phenomenon is called dark current signal and is linearly proportional to the exposure time. The detector vendor specifies dark current generated per pixel per unit time (in electrons/pixel/s). For example, the best CCD cameras generate less than 0.1 electrons/pixels/s. If this constant is multiplied by the exposure (acquisition) time, the total dark current signal is obtained. This type of signal is strongly dependent on the detector temperature: the lower the temperature, the lower the number of unwillingly generated electrons.

Fixed pattern noise

Every CCD chip produces a small number of hot pixels, which appear as very bright pixels in the final image matrix, and dead pixels, which appear as very dark pixels. The first case can be simulated by filling corresponding pixel positions with an overwhelming number of electrons. The dead pixels are simulated as pixel positions with no electrons.

Quantification uncertainty

The most important noise in low-light imaging (which is typical for fluorescence microscopy) is shot noise, which is usually modeled with a Poisson distribution. This phenomenon stems from physical laws that say that the relative measurement error increases as signal intensity decreases. This noise, also known as Poisson noise, is neither additive nor multiplicative. Its value depends directly on the input signal in each position. Mean noise level equals the square root of the quantified signal (number of electrons). The higher the value of the input signal, the lower is the relative amount of noise present.

Readout noise

The signal amplification process produces a certain amount of readout noise, which is modeled as additive white Gaussian noise. The amount of this noise is also specified by the detector vendor (in electrons per pixel). The best CCD cameras generate less than 10e⁻/pixel. Subsequently, the A/D converter is used; that is, the electrons representing the signal are quantized into analog-digital units (ADU). Each such ADU defines a particular intensity level in the final digital image.

Notation

To summarize the processes of this stage, we can write:

• (5)

where r(.) stands for signal sampling, η_p(.) defines the Poisson noise, η_g defines the additive Gaussian noise, I_d defines the dark current signal including hot/dead pixels, and “ADC[.]” stands for the quantization process of conversion from electron units to ADUs. Theoretically, I_blurred should be a continuous signal, and r(.) should be a real sampling function defined by the detector properties. However, we cannot handle a continuous signal. To simulate this, we can up-sample the initial image I_background to a higher resolution. Afterwards, the r(.) function is simply a down-sampling filter. This approach allows generating images with sub-pixel precision.

RESULTS AND DISCUSSION

Currently, the simulator can generate three different types of objects. The simplest one is the microsphere, which is usually used to measure optical system properties. The latter two objects are HL-60 cell nuclei (see Fig. 8) and granulocyte nuclei (see Fig. 9). As these two types of objects are quite complex, one needs a lot of experience to set all the required simulator parameters correctly in order to get the expected shape and the internal structure. Close collaboration with biologists helped us tune the parameters of individual methods in the process of digital phantom generation. In this way, we obtained the most suitable values. They are part of the source code, which is freely available under the GNU GPL.

Figure 8. An example of four synthetic images of HL-60 cell line. These images should be compared with those in Fig. 3. Each 3D figure consists of three individual images: the top-left image contains a selected xy-slice, the top-right image corresponds to a selected yz-slice, and the bottom one depicts a selected xz-slice. Three mutually orthogonal slice planes are shown with ticks.

Figure 9. An example of four synthetic images of granulocytes. These images should be compared with those in Fig. 4. Each 3D figure consists of three individual images: the top-left image contains a selected xy-slice, the top-right image corresponds to a selected yz-slice, and the bottom one depicts a selected xz-slice. Three mutually orthogonal slice planes are shown with ticks.

The plausibility of the results can be assessed in many ways. The most common method for biomedical image data comparison is visual inspection by an expert. On one hand, this approach is important for a coarse estimate when deciding whether the given image is visually similar to the class of real images. On the other hand, it is tedious and cumbersome, and the human eye may be simply deceived. This is especially true with spatial (3D) image data.

Another way of comparison is more straightforward, more exact and faster. The idea is based on image descriptors. Such a descriptor can, for example, characterize the objects contained within the image, measure some local texture properties, or evaluate some simple statistics over the whole image. In the past, various measures (also called descriptors or features) have been introduced, and some of them have been in principle accepted as a standard. The measures have been designed either for generic image data (51, 52) (especially the texture measures) or for segmented image data (51, 53). The latter include, for example, the number of objects within an image, the circularity or size of the objects, or the distance between the objects. Until recently, the vast majority of descriptors have been designed and consequently evaluated only over 2D image data. Lately, due to progress in the development of computer hardware, most of the features have been extended to 3D (54, 55).

In this work, we focused on evaluating a selected set of global texture features that includes 2nd to 6th central moment (51) and 3D Haralick features (55). We compared the features computed from real image data with those computed from simulated image data directly, whereas a recent study (25) compared the real and simulated data indirectly based on classification results inferred from the features.

We also studied image intensity histograms and entropy. Aside from these measurements, we also validated some of the optical system properties with a Fourier analysis (we already discussed this issue in Section 3.2).

Measurements

Central moments

The nth central moment is a statistical measure evaluated over a given discrete random variable.

• (6)

where

• 

Here, the variable is marked as z_i and denotes the particular intensity level present in the image. Hence, the sum covers the range of all the image intensity levels (L). Each moment (μ_n) has its own specific meaning. The second moment is known as the variance of the given data. The third moment expresses the skewness of the measured data, and the fourth moment describes the relative flatness.

Entropy

Entropy [see Eq. (7)] comes from information theory and defines the amount of uncertainty (the amount of information) hidden in the measured data. The uniform area has zero entropy, whereas scattered data carry more information and hence can be characterized with a higher entropy level.

• (7)

Haralick features

The Haralick features (52, 55) are one of the most popular texture feature sets. They are a collection of statistics calculated from a so-called co-occurrence matrix. Haralick features include angular second moment, contrast, correlation, variance, inverse second difference moment, etc. Each of these statistics relates to a certain image feature. For example, the statistic “contrast” measures the amount of local variation in an image.

Intensity histograms

The intensity histogram (56) is a graph specifying how many pixels within a given image have the same intensity. Typically, the zero (leftmost) position indicates the number of pixels with the lowest intensity, and the rightmost histogram position indicates the number of pixels with the highest intensity. It is clear that similar images should have nearly the same histogram. Moreover, rotating or shifting the objects within the image does not affect the histograms. Hence, slightly shifted or rotated objects still produce the same histograms.

Evaluation

It is common to evaluate a large number of features over each image to get a so-called feature vector. These feature vectors are typically computed for each individual image and submitted to a suitable decision tool. Such a tool can either be a classifier (e.g., a neural network, support vector machine, clustering methods) or a statistical method (e.g., distribution plot, goodness-of-fit test). In this work, we used the latter approach.

To show that the synthetic images produced by the simulation toolbox are almost the same as the real images acquired from the optical microscope, we evaluated all the introduced measurements over 500 synthetic images and real images (individually for each object). Tables 4 and 5 show the results. To illustrate that both the real and the synthetic data sets come from populations with a common distribution, we use quantile–quantile plots (57) (see Figs. 10 and 11). Regarding quantile–quantile plots, if the two sets come from a population with the same distribution, then the points should fall approximately along the 45-degree reference line. The greater the deviation from this reference line, the greater the evidence for the conclusion that the two data sets came from populations with different distributions.

Figure 10. Comparison of descriptors computed from real and synthetic HL-60 images. Quantile–quantile plots illustrate whether the measured datasets come from populations with similar distributions. Generally, if the two sets come from a population with the same distribution, the points should fall approximately along the reference line.

Figure 11. Comparison of descriptors computed from real and synthetic granulocyte images. Quantile–quantile plots illustrate whether the measured datasets come from populations with similar distributions. Generally, if the two sets come from a population with the same distribution, the points should fall approximately along the reference line.

Table 4. Evaluation of central moments and entropy over the real and simulated image data representing HL-60 cell line

Measure	Real Image	Simulated Image
Each measure is expressed as a mean value over the whole dataset including standard deviation.
2^nd c.m.	0.1304 ± 7.6 × 10^−4	0.1569 ± 5.1 × 10^−4
3^rd c.m.	0.0069 ± 1.7 × 10^−5	0.0083 ± 1.9 × 10^−5
4^th c.m.	0.0012 ± 0.8 × 10^−6	0.0013 ± 1.0 × 10^−6
5^th c.m.	0.0004 ± 1.3 × 10^−7	0.0004 ± 1.7 × 10^−7
6^th c.m.	0.0001 ± 1.9 × 10^−8	0.0001 ± 2.4 × 10^−8
Entropy	5.384 ± 0.22	5.682 ± 0.12

Table 5. Evaluation of central moments and entropy over the real and simulated image data representing granulocytes

Measure	Real Image	Simulated Image
Each measure is expressed as a mean value over the whole dataset including standard deviation.
2^nd c.m.	0.4978 ± 1.2 × 10^−3	0.4667 ± 1.0 × 10^−3
3^rd c.m.	0.0140 ± 1.7 × 10^−5	0.0126 ± 3.0 × 10^−5
4^th c.m.	0.0031 ± 1.6 × 10^−6	0.0024 ± 1.7 × 10^−6
5^th c.m.	0.0012 ± 3.0 × 10^−7	0.0010 ± 4.0 × 10^−7
6^th c.m.	0.0004 ± 1.0 × 10^−7	0.0003 ± 1.0 × 10^−7
Entropy	6.134 ± 0.07	6.295 ± 0.13

The individual quantile–quantile plots in Figure 10 show that the feature vectors for both real and synthetic HL-60 data follow very similar distributions. In the case of granulocyte nuclei (see Fig. 11), the same is valid except for the last quantile–quantile plot, which is characterized by a slight deviation from the reference 45-degree line.

Concerning 3D Haralick texture features, we evaluated 19 features in total. In (55), where the evaluation was done locally, some texture features were found to be highly correlated. Therefore, only 15 features were considered, and later in (58), the number of independent features was further reduced to eight. In our study, we simplified the method presented in (55) to get global texture features. In that way, we found 9 features to be uncorrelated. These features include: sum average, texture entropy, texture contrast, texture correlation, texture homogeneity, maximum probability, sum entropy, difference entropy, and information measure of correlation. There is a close correspondence between our selection and the previously (58) mentioned subset of eight Haralick features (six of them are common to both studies).

Similarly to the evaluation of central moments over the image data, we generated nine quantile–quantile plots for each image dataset. For HL-60 dataset, eight of nine plots exhibited similarity between individual features for both real and synthetic images. Regarding the granulocyte dataset, five of nine plots also showed that the feature vectors for both real and synthetic data follow very similar distributions. The rest of the quantile–quantile plots did not follow the expected 45-degree line, which was also the case for the last plot in Figure 11. In future work, we will search for the differences between the real and the synthetic data that caused the imperfection of some quantile–quantile plots.

Figures 12 and 13 show the intensity histograms illustrating the similarities between the two image datasets. Notice that the corresponding histogram pairs exhibit a similar shape.

Figure 12. Mean of log intensity histograms for (left) real HL-60 images, (right) synthetic HL-60 images.

Figure 13. Mean of log intensity histograms for (left) real granulocyte images, (right) synthetic granulocyte images.

CONCLUSION

In this article, we presented a very complex and efficient tool for generating phantom biomedical images. Furthermore, it simulates the whole process of image acquisition from an optical microscope, from the very beginning when preparing the specimen, to the very end when converting the analog signal into its digital form and storing it to a hard drive. This toolbox generates 3D objects but does not have any limitations; hence, it can be extended to any higher dimension (time sequences or spectral imaging) and generate other types of cells. The whole simulation toolbox can be split into three independent parts: digital phantom generation, signal transmission, and signal detection and image formation.

For the time being, the toolbox generates three types of objects (HL-60 nuclei, granulocyte nuclei, and microspheres). In the future, we intend to generate more types of objects: cells (nucleus, cell shape, and selected sub-cellular components) and cell clusters that typically appear in tissue scans.

The simulator proposed in this article is freely available under the GNU GPL at: http://cbia.fi.muni.cz/simulator/.

Acknowledgements

Thanks are also due to Martin Maška for his implementation of PDE-based methods, Honza Skalický for the microscope management, and the HCILAB from the Faculty of Informatics at Masaryk University for computational support. We also greatly appreciated the help of Ludvík Tesař from the Institute of Information Theory and Automation (Czech Academy of Sciences, Prague) with the 3D Haralick feature implementation. A talk based on this paper has been given at the ISAC XXIV Congress in Budapest (May 2008). The simulation toolbox is freely available under GNU GPL at http://cbia.fi.muni.cz/simulator/.