- Top of page
- General methods
- Supporting Information
Colour patterns play a key role in many animal interactions (Cott 1940). Signal receivers must discriminate among patterns with significant information content, such as social dominance (Møller 1988) or toxicity (Brodie 1993). To understand such decisions fully, we need a way of extracting the information contained in the colour pattern observed. There exist effective standardised methods for comparing colours (Endler & Mielke 2005; Stoddard 2012), but ways of quantifying the spatial aspect of a pattern are less well developed.
Humans have extremely good visual capabilities and cognition (Pinker 1984) and hence, when studying colour patterns, often find it easy to make qualitative decisions about the information in a signal and classify the patterns appropriately. However, if asked to articulate rigorously and precisely the reasons behind such decisions, we find this far more difficult. This means that the outcome of such comparisons is subjective and inconsistent. Furthermore, humans, like all species, perceive the world in a way that is subtly constrained by their sensory and cognitive abilities. Differences in factors such as spatial resolution and spectral sensitivity, as well as higher processing, mean that the human perception on which a judgement is based may not accurately reflect that of the most relevant signal receiver (Endler 1990).
An alternative is therefore to use automated, computer-based methods for comparing patterns. Traditional methods of shape analysis are inappropriate, as they rely on the selection, either manual or automatic (Boyer et al. 2011), of homologous landmarks. These are unlikely to exist in colour patterns, especially when comparing disparate taxa, such as will often be the case in examples of mimicry. To capture pattern information in a more flexible way, there are two main types of approach, both well developed in computer science. The first involves ‘feature extraction’: creating a statistical summary for each of the patterns under study, often based on properties of the image in the frequency domain using Fourier transforms (Zhang & Lu 2002), or on ‘moment invariants’ (Khotanzad & Yaw Hua 1990). A simple measure such as Euclidean distance between two summary vectors can then give the dissimilarity between two patterns (Zhang & Lu 2003). This type of method is commonly used in image retrieval algorithms, where an image is sought within a large database that shares similar properties to a target image (Rui, Huang & Chang 1999).
The second approach involves direct comparison of individual pixels or regions of pixels through, for example, cross-correlation (Briechle & Hanebeck 2001) or the sum of absolute differences (Goshtasby 2005). This method, known as template matching, is commonly used for image registration (Zitová & Flusser 2003). The key point here is that it compares whole images, pixel by pixel, before the information is summarised. By contrast, in feature extraction, the images are first summarised and then compared. Thus, template matching is sensitive to differences in specific features within the pattern (e.g. particular spots or stripes), while feature extraction is based on the overall attributes of the pattern.
In a biological context, several authors have used Fourier transforms to facilitate comparison of general pattern properties, such as in striped coats of mammals (Godfrey, Lythgoe & Rumball 1987), cuttlefish displays (Barbosa et al. 2008) and spots on bird eggs (Stoddard & Stevens 2010). Endler (2012) suggests a slightly different approach, counting the number of transitions from one colour to another in ‘adjacency analysis’. However, all of these cases adopt an ‘image retrieval’-type approach, in which the image is simplified to a few summary values before the comparison with another image takes place. In doing so, specific spatial information regarding the location of particular pattern features is discarded.
In some cases, this is not a problem and could even be an advantage. For example, in egg mimicry, the pattern on each host egg is unique. The mimetic egg therefore resembles the overall features and type of pattern of a host egg rather than the exact locations of pigment blotches (Stoddard & Stevens 2010), and so the use of summary variables is more appropriate than a consideration of individual pattern elements. On the other hand, for strongly stereotyped patterns, such as those seen in many insect mimicry complexes (see e.g. Cott 1940; Ruxton, Sherratt & Speed 2004), the precise shape and position of pattern features may be important. In summarising, we risk discarding pertinent information, and what is retained will depend upon the statistics chosen, increasing subjectivity.
By contrast, the ‘template matching’ approach of comparing specific features within an image has very rarely been applied to biological colour patterns. A simple version was used by Williams (2007), who subdivided bumblebee patterns into 27 regions, each of which, within individuals, is usually occupied by a uniform colour. The patterns can then be compared region by region. This approach can be effective, but is limited by the fact that colour boundaries in a given individual may not exactly coincide with the predefined regions; it relies upon creating discrete homologous categories in patterns which are often continuous in nature. Williams' (2007) division of a bumblebee pattern into 27 regions is able to detect the presence or absence of a spot or stripe, but not its exact outline.
To make the subdivision method more sensitive, we can use more regions. The logical extreme is to divide the pattern into as many separate regions as possible; in practical terms, for a pattern recorded as a 2D digital image, regions would be the individual pixels of the image. We can then score the dissimilarity at each pixel location and take the sum of all such values; this method is known as the sum of absolute differences (Goshtasby 2005). The dissimilarity score could be a simple match or mismatch for binary images or a difference in brightness for greyscale. Unfortunately, increasing the number of regions introduces a different problem, which is that common features between the two images must line up exactly to be recognised as similar. A stripe offset by just a few pixels would be counted as just as strong a mismatch as its complete absence – in fact more so, since both locations will count as mismatches. Although the method has been used to produce a rough measure of mimetic accuracy in hoverflies (Dittrich et al. 1993; Azmeh et al. 1998), it gives several anomalies. For example, Azmeh (1999) observed that all-black hoverflies are given unrealistically high measures of similarity to wasps.
Here, we describe a new, holistic method for the measurement of similarity between two or more biological colour patterns. The method is similar to the sum of absolute differences in that it uses information from the whole pattern at the level of the individual pixel, but it is more robust to small spatial variations among images because it is based on the distance transform (Borgefors 1986). In the distance transform of a binary image, each pixel is weighted by the minimum distance to the nearest white pixel. Distance transforms have been used on one previous occasion to analyse biological colour patterns (Anderson et al. 2010) but to make qualitative decisions (identification) rather than the quantitative comparisons of similarity we seek. The method we describe is applicable to any pattern that consists of clearly separated colours (as opposed to colours blending from one to another), for example, those seen in the abdominal patterns of many insects, cetacean markings, wing patterns of many butterflies and moths, amphibian aposematic signals and body patterns of reef fish. We demonstrate the utility and versatility of the method by applying it to three ecological examples.
- Top of page
- General methods
- Supporting Information
We have demonstrated the use of distance transforms to generate a measure of similarity between two colour patterns. This is not the first use of distance transforms for image comparison in biology. However, in the only other instance of which we are aware, it was used in the context of individual recognition, to detect whether two patterns of spots were the same or different (Anderson et al. 2010). In this qualitative approach, the pattern variation was used as a means for identifying individuals rather than being of interest in itself. We have sought to show that there is much more potential in the method for studying variation in patterns. To our knowledge, this is the first such study to use distance transforms where the magnitude of the difference is biologically relevant.
Our method captures far more pattern information than in previous analyses of the three systems studied here, since it does not characterise a pattern using summary variables. Instead, it uses the full pattern, pixel by pixel, to calculate a similarity value. The advantage gained is clear when we look at the hoverfly example: despite using information only from the abdominal colour pattern, the distance transform method gave an assessment of similarity that correlated extremely well with human perception of whole insect similarity (r = 0·87). The multivariate measure from Penney et al. (2012), which included summary variables for the pattern, as well as morphometric data from other body parts, still gave a significant correlation but explained a lower proportion of the variance (r = 0·56, from Penney et al. 2012). It is worth noting, though, that even if the calculated dissimilarity did not correlate with a predator's perception of the pattern, it would still form a useful comparison; we could then ask the question: why does the predator not make use of the available information to distinguish models from mimics?
Similarly, in the case of Polistes clypeal badges, the brokenness measure (Tibbetts & Dale 2004) captures only a limited amount of information about the shape of the badge. In the case of the Portuguese population used in this study, the distance transform method retains almost all of the brokenness information, while adding more detail along a second dimension (Fig. 5). Despite the extra information, we still detect no association between mass and badge shape. As the measure we have used is more comprehensive, it strengthens the conclusion that the lack of a detected association is due to the genuine absence of a link rather than failure to capture the relevant variation. This supports the findings of Cervo et al. (2008) and Green et al. (2013) in other European populations. A further advantage of the distance transform method is that, because it is based on an image of the whole clypeus rather than just the badge, it captures variation in clypeus outline as well as the badge itself. This may be especially useful in populations where a large proportion of individuals have no black badge at all (Cervo et al. 2008).
The relatively objective nature of our technique helps minimise a researcher's reliance on assumptions based on their own perception. The data we present on Müllerian mimicry in heliconiines largely confirm previous assumptions (Joron et al. 1999), but do reveal that in absolute terms, a few mimetic pairings may not be as clear cut as they initially appear. In particular, a consistent difference is picked up between subspecies of Melinaea marseus and their Heliconius numata comimics (Fig. 4a). This difference is connected largely with shape rather than pattern, since it is reduced considerably when shape is standardised (Fig. 4b). The difference may or may not be relevant to a consideration of mimicry since predators may not attend to shape information, difficult to interpret reliably with different wing angles at rest. Predators, such as birds, may well perceive the same striking pairings as suggested by humans, but this should be explicitly tested, given that alternative groupings, or a more continuous mimicry ring, also form plausible descriptions. This example is also a good demonstration of the potential scope of the technique; it can compare patterns with more than two colours and detect differences of outline as well as pattern.
We recognise that in our example analyses, the use of RGB photographs and illustrations may introduce a bias towards the human perception of colour patterns and that a more thorough analysis would take into account the visual abilities of the likely signal receivers. However, our focus is on the method of pattern comparison, whatever technique was used to collect and prepare the images. Furthermore, there is no evidence in any of the examples to suggest that human perception of the colour boundaries is any different from that of other animals. In the case of hoverflies, for example, responses of pigeons were no different when presented with naturally lit specimens (Green et al. 1999) as opposed to RGB photographs (Dittrich et al. 1993).
One limitation of the distance transform method is that it captures spatial variation in patterns only, not variation in colour hue. This is because the image is converted into binary format, with a colour classed as either present or absent; any more subtle information on hue or brightness is lost. While restricting the scope of the method slightly, this also brings benefits, since brightness of colours can in some cases vary over time, both during an animal's lifetime and after death. If images are taken from museum specimens, then fading of colours can be a major problem. However, provided major colour boundaries are still detectable, fading will not affect the binary images produced. If data on hue or brightness are thought to be relevant to the system under study, for example, the brightness of an aposematic signal, then this information would need to be included through a separate analysis such as recording of spectral reflectance values (Endler & Mielke 2005).
While we have aimed to develop an objective and consistent similarity measure, it will never be possible to remove subjectivity from the process entirely. The very process of representing a 3D colour pattern in a 2D digital image inevitably changes the nature of the pattern in some ways. The choice of colours used to segment the pattern (as in the Heliconius example) will also inevitably alter the outcome.