Megan T. Valentine, Department of Mechanical Engineering, University of California, Santa Barbara, CA 93106, U.S.A. Tel: 805–893-2594; fax: 805–893-8651; e-mail: firstname.lastname@example.org
We propose a novel and algorithmically simple Hough transform method that exploits the geometric properties of ellipses to enable the robust determination of the ellipse position and properties. We make use of the unique features of the evolute created by Hough voting along the gradient vectors of a two-dimensional image to determine the ellipse centre, orientation and aspect ratio. A second one-dimensional voting is performed on the minor axis to uniquely determine the ellipse size. This reduction of search space substantially simplifies the algorithmic complexity. To demonstrate the accuracy of our method, we present analysis of single and multiple ellipsoidal particles, including polydisperse and imperfect ellipsoids, in both simulated images and electron micrographs. Given its mathematical simplicity, ease of implementation and reasonable algorithmic completion time, we anticipate that the proposed method will be broadly useful for image processing of ellipsoidal particles, including their detection and tracking for studies of colloidal suspensions, and for applications to drug delivery and microrheology.
The detection of ellipses, circles and lines is an integral part of experimental image analysis for a wide range of biological, chemical and mechanical engineering applications. Of these three classic object shapes, the most general is the ellipse, as the remaining two objects can be expressed as special cases in which the major and minor axes lengths are equal or the length of the minor axis is zero. Ellipse detection enables the size, shape and orientation of naturally occurring anisotropic structures (i.e. red blood cells and bacteria) to be determined. Further, it allows the characterization of synthetic ellipsoidal particles, which have become increasing important in colloidal studies (Keville et al., 1991; Ho et al., 1993; Sacanna et al., 2006; Champion et al., 2007; Crassous et al., 2012). The shape anisotropy of ellipsoids gives rise to novel interparticle and hydrodynamic interactions, and allows ordered liquid crystalline phases to form at high volume fraction (Melle et al., 2002; Han et al., 2009; Madivala et al., 2009; Güell et al., 2010; Zheng & Han, 2010). Ellipsoidal particles have broad engineering applications, ranging from the self-assembly of novel photonic band gap structures to the geometrically controlled uptake of drug-infused structures through cellular phagocytosis (Velikov et al., 2002; Champion & Mitragotri, 2006; Ding et al., 2009).
Due to the clear technological importance, a number of algorithms to detect and characterize ellipses in digital images have been developed (Kawaguchi & Nagata, 1998; Fitzgibbon et al., 1999; Yu et al., 2004; Hahn et al., 2008). Unlike spherical particle detection, in which a small number of robust centroid tracking algorithms have emerged as standard tools, the computational requirements and algorithm complexity of ellipse detection software has largely limited its practical use in colloid science (Crocker & Grier, 1996; Jenkins & Egelhaaf, 2008). One of the most widely used computer vision tools is the Hough transform (HT) method, in which the features (i.e. size, shape, orientation) of a real-space image are transformed into discrete points of a parameter space (Hough, 1962; Duda & Hart, 1972). This multidimensional parameter space is then searched to uniquely identify and characterize the original object. In this paper, we report a new implementation of the HT that exploits the geometric features of ellipses to reduce algorithm complexity, provides rapid analysis of image quality and accurately determines the shape, size and orientation of even imperfect ellipses. We demonstrate its utility using both simulated and experimental images of ellipsoidal colloids. Our method provides accuracy and speed that is comparable to that of direct fitting by a least squares minimization approach, but additionally allows for rapid assessment of image quality and enables identification and characterization of adjacent or touching objects.
Prior HT methods have been used successfully in the detection of spheroidal objects in two-dimensional images (Jenkins & Egelhaaf, 2008; Irmscher et al., 2012). Circles are represented using three independent parameters: the centre coordinates (xc, yc) and the radius r. The resulting Hough space is thus three-dimensional, and the classical circular HT algorithm has O(N3) space requirements and O(N4) time complexity for an image I(x,y) of size N×N pixels. The memory requirements and completion time can be reduced by completing the search in two consecutive steps (Illingworth & Kittler, 1987; Yoo & Sethi, 1993; Bennett et al., 1999; Kanatani & Ohta, 2004; Peng et al., 2007). First, the gradient of the image is determined by calculating the centred differences in intensity. For a circle, the gradient consists of a radially symmetric array of vectors that all intersect at the centre. Thus, it is possible to identify the circle centre by identifying the most likely point of intersection. In practice, this is achieved by creating a new accumulation array (typically N×N) in which the gradient vector at each pixel in the original image is represented by a line that emanates in both directions from the same pixel in the accumulation array, at an angle (Peng et al., 2007). Each line extends across the entire accumulation array, and each pixel intersected by the line receives one weighted vote, where the weight function is given by the gradient magnitude . If the expected range of circle radii is known, the length of the gradient vector can be truncated to minimize potential interference from neighbouring objects. The resulting accumulation array can be displayed graphically as an image. Pixels that accumulate the most votes when the contributions of all gradient vectors are summed qualify as candidate centres.
Once the candidates for the centres are detected, the voting for the radius is performed. To accomplish this, each pixel is assigned a value equal to the radial distance r to the detected circle centre, rounded down to the nearest integer. The one-dimensional accumulation array is constructed by summing the image gradient magnitudes of all pixels with a given assigned value of r, then normalizing by the number of pixels in the set. This is repeated for each possible radius, and the value of r that accumulates the most votes (i.e. the highest normalized sum) is considered the best candidate as the circle radius.
Applications of the HT to ellipse detection are substantially more complex, because ellipses are parameterized by five independent parameters: centre position (xc, yc), orientation θ and the lengths of the major and minor axes, a and b, respectively (Tsuji & Matsumoto, 1978; Muammar & Nixon, 1989; Pao et al., 1993; Yoo & Sethi, 1993; Aguado et al., 1996; Xie & Ji, 2002). This typically requires a five-dimensional parameter space with O(N5) space requirements and O(N6) time complexity. In this work, we reduce algorithmic complexity to O(N2) in space and O(N3) in time by using a gradient-based HT as described in (Peng et al., 2007), and by making use of the fixed points and the symmetry of ellipses. We further develop this method by exploiting the properties of the evolute of the ellipse. The evolute describes the envelope of surface normals of a geometrical object. The gradient vectors along the perimeter of the ellipse create a radiant field in the shape of an astroid, as shown in Figure 1. For purposes of ellipse tracking, the most important points on this astroid evolute are the four cusps (indicated by red dots) where the normal rays of the ellipse are most concentrated. When the evolute is drawn over the original ellipse, two cusps lie along the major axis of the ellipse. These ‘major cusps’ are always found within the bounding perimeter of the ellipse, making them particularly good targets for identification by image processing. The two ‘minor cusps’ can be found within or outside of the ellipse perimeter. Cusp positions are given by
where ca and cb are the distances from the ellipse centre to the position of the major and minor cusps, respectively. Once a pair of major cusps has been identified, the centre parameters (xc, yc) and the orientation θ are immediately determined. Equation (1) provides a unique relationship between the two remaining parameters a and b, which determine the shape of the ellipse. Therefore, for each valid pair, the ellipse detection is finalized by carrying out an additional one-dimensional accumulator on an appropriately chosen variable. We choose this parameter to be the length of the minor axis, b, as its range is relatively small as compared to the alternatives, such as the length of the major axis (a), focal distance () or the minor cusp distance (cb).
In practice, ellipses are detected using custom-written algorithms implemented in MATLAB, following the methods described in (Peng et al., 2007). The original image is smoothed using a Gaussian kernel, and the image gradient calculated using centred differences at each nonzero pixel. The two-dimensional accumulation array is obtained by voting along the gradient direction, with weighting by the gradient magnitudes. This array is convolved with a local maximum finding filter, such as a Laplacian of Gaussian kernel or an equivalent disk filter where when when r0 < r < r1, and = 0 when r > r1. Here, r0= 0.6r1 and r1 is a user-defined input that scales with the size of the object of interest. By contrast to Gaussian filters, which are commonly used for maxima detection, we find the Laplacian of Gaussian filter naturally suppresses the background offset in pixel intensity and enhances the signal to noise ratio (Neycenssac, 1993).
Cusp detection is carried out in two stages. As a first step, we multiply pixel-wise the values of the local maximum-filtered array and the original accumulator array, then threshold the result with a cutoff magnitude of ∼10–25% of the maximum value. All pixels with values below this cutoff are set to zero, generating a two-dimensional image of bright ‘islands’ on a dark background. We then determine the eccentricity ɛ using a built-in MATLAB function1 that assigns ɛ≈ 0 to circular objects and ɛ≈ 1 to lines. To isolate bright features that likely contain cusps, we reject candidates with ɛ > 0.9. In a second step, we threshold the original two-dimensional accumulation array a cutoff magnitude of 60% of the maximum value. For each region identified in step 1, we calculate the centroid of the equivalent region in the thresholded accumulator array. The resulting positions form a list of candidate cusp coordinates.
From this list, we seek to identify valid pairs of major cusps, while eliminating minor cusps and image artefacts. Theoretically, any two peaks can be chosen to make a pair; however, if the range of expected ellipse shapes and sizes is known, we impose an upper and a lower bound to the interaction distance between maxima. Incorporating such a priori knowledge of the ellipse parameters not only constrains the possible size of the evolute to aid in cusp pairing, but also reduces the computation and storage requirements by reducing the search space.
For each remaining pair, we then perform a one-dimensional vote on the minor axis (b). For each possible value of b, there is a unique value of a, given the observed value of ca (using Eq. 1). To generate the accumulation array, we sum the image gradient magnitudes of all pixels that reside on an ellipse with this unique (a, b) pair with a given assigned value of b, then normalize by the number of pixels in the set. The best candidate of b is the value that accumulates the most votes by this normalized sum. In some cases, the detected cusps will not be a valid pair of major cusps, and the one-dimensional accumulation array will fail to produce a strong peak value; such pairs are eliminated.
In the limit that a≈b, the four cusps collapse into a single point at the centre of the circle. Therefore, the gradient-based circular HT method can be treated as a special case of the gradient-based evolute HT method proposed here. Although prior descriptions of elliptical HTs have described the evolute, to our knowledge a direct analysis of cusp positions has not been performed (Luo et al., 1995; Kanatani & Ohta, 2004).
Results and discussion
To demonstrate the utility of our method in detecting and characterizing the properties of multiple and polydisperse ellipses, we test our algorithm using simulated ellipses and experimental images of ellipsoids. First, we generate a simulated image containing four randomly oriented elliptical rings with different sizes and shapes (Fig. 2). All four ellipses are successfully detected by our algorithm; however, we find that the highly asymmetric distribution of the voting lines in the proximity of the cusp leads to a systematic underestimate of ca. Through careful analysis of a much larger ensemble of ellipses, we determined the measured ca to be underestimated by ∼10% for all ellipse shapes and sizes. Therefore, in the implementation of the method, the detected cusp distances are corrected according to this criterion. With this correction in place, the parameters of the detected ellipses are in excellent agreement with the values used to generate the ellipses (see Table S1). The uncertainty in centre finding is approximately one pixel and the orientation is reliably determined to within one degree.
To test the performance of our algorithm under more realistic conditions, we detect and characterize the properties of experimental electron micrographs of micrometre-scale polydisperse ellipsoids, prepared as described in (Champion et al., 2007). Images were obtained using an FEI XL40 Sirion FEG digital scanning electron microscope (SEM; Hillsboro OR, USA). In the case of an almost perfectly ellipsoidal single particle, our evolute-based algorithm accurately determines ellipse position, orientation, shape and size (Fig. 3). Since the precise shape of the ellipsoid is not known prior to imaging, we verify the accuracy of our characterization method by comparing our results to those obtained using an ellipsoidal fit using a least squared minimization routine. Least squares fitting is performed by eroding the gradient image and removing the isolated and spur pixels (i.e. bright pixels that are connected to other bright features by 0 or 1 pixels), as described in detail in (Fitzgibbon et al., 1999). As shown in Figure 3 and Table S2, the approximation by our method and the least squares fit are practically indistinguishable.
To further test this approach, we analyse an SEM image containing multiple imperfectly ellipsoidal particles on an uneven background. As demonstrated in Figure 4, these particles are also easily detected by our evolute-based HT method. In this example image, nine particles of different sizes and aspect ratios are located within the field of view. For each ellipse, reasonable estimates of particle position, orientation, shape and size are produced, and no false particles are detected. To verify the accuracy of our method, we again compare our results to those obtained with an ellipsoidal fit using a least squares minimization approach. We again find very good agreement between the results obtained with these two methods for (xc, yc), b and θ, with errors of less than 1 pixel or 1 degree, respectively (see Table S3). Since these particles are not perfect ellipses, but are truncated along their major axis to a blunt end, we find larger uncertainties in our determination of a than we observed for more perfect ellipses. Through visual inspection, we find that the HT method systematically underestimates the length of the major axis whereas the least squares method overestimates it. Thus, in this case, we independently determine the end-to-end length of the ellipses by a manual measurement of the edge-detected and eroded image, and use this value to determine the error in a (Table S3).
The overall performance of the evolute-based HT method is similar to that of direct fitting of ellipsoids using a least squares minimization approach. Each provides reasonable estimates of ellipse position and features. The detection of these nine particles in the 1303-by-725 pixel image displayed in Figure 4 is completed in less than 2 s by either method using a Windows 7 OS desktop computer with a Pentium Dual Core E5700 3GHz CPU and 3GB RAM. In principle, our method reduces the space complexity of the elliptical HT from O(N5) to O(N2). However, in order to achieve execution times of ∼2 s, we have used a vectorized implementation that avoids the use of for loops, which are computationally costly in an interpreted language like MATLAB. This increases the space requirements to O(N3). If implemented in a compiled language (i.e. C++), this issue can be avoided, albeit at the loss of the numerous built-in functions for the easy display and manipulation of images that are available in MATLAB.
There are unique advantages of the evolute-based HT method we present. First, visual inspection of the quality of the evolutes in the accumulation array image provides an immediate and direct readout of our ability to detect ellipsoids with this technique. This allows the effects of various illumination schemes, contrast microscopy methods, etc. to be quickly screened and the best imaging conditions to be determined without full analysis of all data. In addition, as shown in Figure 5, the evolute-based approach allows for analysis of objects that are physically touching. This is possible since the major cusp positions are always found within the ellipse perimeter, and the parameter space used for feature detection is related to, but distinct from, the real space image. At very high volume fractions, the performance of HT methods tends to degrade, as it does for most particle detection algorithms, due to interference effects between neighbouring objects during the Hough voting step; however, a priori knowledge of ellipse size and shape can substantially limit the search area, improving computational performance and detection accuracy.
We have presented a new HT method, which reduces the complexities of the classical elliptical HT by detecting the cusps on the ellipse evolute that is generated using the gradient information already contained in the image. This method accurately detects and characterizes multiple and polydisperse particles in a reasonable computational time, making it a useful alternative for a wide range of image processing applications in colloid science, materials science and biomedical engineering.
Authors thank Aaron Anselmo and Prof. Samir Mitragotri of UCSB for fabricating the ellipsoidal particles and for assistance in SEM imaging, and Dr. Emre Akbas of UCSB for useful discussions. Authors gratefully acknowledge the support of a Career Award at the Scientific Interface from the Burroughs Wellcome Fund (to M.T.V.). This work made use of MRL Central Facilities supported by the MRSEC Program of the NSF under Award No. DMR-1121053.