Using comprehensive machine‐learning models to classify complex morphological characters

Abstract Recognizing and classifying multiple morphological features, such as patterns, sizes, and textures, can provide a comprehensive understanding of their variability and phenotypic evolution. Yet, quantitatively measuring complex morphological characters remains challenging. We provide a machine learning‐based pipeline (SVMorph) to consider and classify complex morphological characters in multiple organisms that have either small or large datasets. Our pipeline integrates two descriptors, histogram of oriented gradient and local binary pattern, to meet various classification needs. We also optimized feature extraction by adding image data augmentation to improve model generalizability. We tested SVMorph on two real‐world examples to demonstrate that it can be used with small training datasets and limited computational resources. Comparing with multiple CNN‐based methods and traditional techniques, we show that SVMorph is reliable and fast in texture‐based individual classification. Thus, SVMorph can be used to efficiently classify multiple morphological characters in distinct nonmodel organisms.


| INTRODUC TI ON
As intuitive reflections of biodiversity, morphological characteristics provide valuable taxonomic insights into phenotypic evolution and its underlying genetic mechanisms (Grant & Grant, 2006;Joron et al., 2006;Peichel et al., 2001). Therefore, comparative morphology is generally the indispensable, first step of organismal feature identification and investigation. Yet, quantitatively measuring morphological variation remains challenging.
Fortunately, imaging techniques have advanced so that morphological characteristics can be acquired easily. Commonly, morphological features of interest are measured manually and then classified based on measured data, and the usual geometric morphometrics describes individuals through relative landmark positions (Bookstein, 1996). For example, Drosophila wings are two-dimensional structures with clear venation patterns, so the veins and their crossings are usually considered landmarks whose distances and angles can be measured precisely (Cavicchi et al., 1981;Guerra et al., 1997).
Similarly, some automated systems including machine learningbased methods use fundamental wing morphological features to classify specific insects. For example, using machine learningbased approaches Crnojević et al. (2014) detected vein junctions to discriminate hoverfly species. Yang et al. (2015) developed DAIIS, which identifies owlfly species according to wing outlines. Also, color pattern modeling characterizes Heliconius butterfly phenotypes by comparing color patterns (Le Poul et al., 2014); a machine learning-based approach classifies guenon face patterns (Allen & Higham, 2015); and Patternize, another machine learning-based method that works for multiple organisms, uses color pattern variations (Van Belleghem et al., 2018). While those approaches are useful, either for focal organisms or for processing a specific morphological feature, they are not easily used with organisms other than their original targets or for detecting and classifying more complex features that may harbor a wider range of morphological variability in a combination of patterns, shapes, and textures. A comprehensive and efficient solution that considers and handles such integrative features is still required.
As illustrated above, machine learning has recently provided remarkable image processing solutions that can both efficiently identify complex features and accomplish classification tasks (Lürig et al., 2021). Note that although deep-learning algorithms are widely applied to feature extraction and classification, they require relatively larger training sets and cannot appropriately handle classification problems with fewer samples. In contrast, support vector machine (SVM) universally handles generalized problems and performs well when confronted by a limited or imbalanced sample size, for the decision boundary can be determined by a few support vectors (Tang et al., 2009). Therefore, we developed an SVM-based pipeline, SVMorph, for complex morphological classification. To begin, SVMorph tackles many classification needs by combining two optional descriptors, histogram of oriented gradient (HOG) (Dalal & Triggs, 2005) and local binary pattern (LBP) (Baraldi & Panniggiani, 1995), to extract morphological features from images, and it then trains an SVM classifier to carry out classification tasks. We also employ data augmentation to improve model generalizability, which aims to increase the dataset size by appropriately modifying existing dataset. To illustrate SVMorph's usage and performance, we used it to classify two groups of organisms that possess distinct and polymorphic features. Ultimately, SVMorph demonstrated its considerable potential and universality by excellently and proficiently handling complex morphological classification.

| S VMORPH WORKFLOW
We developed SVMorph in the MATLAB environment (MathWorks Inc.). The overall workflow for a typical task generally includes data preprocessing, feature extraction, and classification ( Figure 1). When conducting a new task, a classifier must be trained and established before new data may be processed.

| DATA ACQU IS ITI ON AND PREPRO CE SS ING
While raw images may be captured using digital cameras that can provide better image resolution, we recommend two-dimensional structures, such as insect wings, be photoscanned for better imaging homogeneity. For example, using a regular scanner (HP LaserJet Pro M227fdw) with a resolution of 600 dpi, we obtained a single forewing image of about 1,000 × 1,000 pixels. While using a digital camera (NIKON D850) with a 1:1 macro lens, we could acquire a single forewing image of over 3,000 × 3,000 pixels with much higher resolution. However, SVMorph or other deep learning-based algorithms only require 256 × 256-pixel or 224 × 224-pixel images as input, and even lower resolutions work as well. Therefore, the need for image resolution is easily satisfied, so we are more concerned with image homogeneity. When the images come from different sources, it may be difficult to ensure the consistency of the photographing equipment or conditions. Relatively, scanners can provide stable photographing conditions and low operational complexity to obtain images with similar quality. Nevertheless, when digital cameras are properly set up, image homogeneity will not be a critical issue. In addition, data augmentation has the potential to reduce the effects of unstable photographing conditions. Therefore, ordinary photographing devices and normal conditions can satisfy the requirements of SVMorph. After data acquisition, SVMorph requires all input image data be preprocessed as follows. Given that both HOG and LBP feature descriptors extract feature information from eight-bit grayscale images, raw images should be transformed into eight-bit grayscale format (Figure 2a,b) and the appropriate brightness, contrast, and exposure are required ( Figure 2a) to ensure optimum classification accuracy. For our sample datasets, we adjusted the butterfly images with the following parameters: brightness 10 and contrast −50, and we did not adjust the spider images. Note that the adjustments in this step are optional. For photoscanned images, the adjustment in this step is equivalent to setting these parameters correctly when photographing with a digital camera. We want the morphological features we care most about to be well expressed in the images. Therefore, adjustments such as lighting and exposure can depend on the visual effects of the images, that is, whether key morphological features can be clearly distinguished visually. In general, there are no particular restrictions on photographing, but the conditions need to be consistent for a classification task.
Regarding the sample size, as mentioned above, SVM generally performs well compared to other algorithms when faced with a limited or imbalanced sample size, because the decision boundary of SVM is determined only by a few samples (support vectors). Besides, the data augmentation step can improve the classification to some extent as well. Therefore, SVMorph is less affected by the limited sample size. In practice, the criterion of sufficient sample size is difficult to predict and needs to be determined based on the actual model performance. Nevertheless, the model performance generally improves with the increasing sample size, so larger sample size is always preferred. On the other hand, when adding samples, highly imbalanced class distributions of samples should be avoided as well. In practice, we recommend at least 10 samples for each class and the ratio of sample sizes between different classes should be <5. For example, if class A has a sample size of 10 which is the least among all classes, the sample size of other classes should not exceed 50 for each.

| DATA AU G M E NTATI O N
For pattern recognition tasks, a set of preprocessed, classified, and labeled image data functions as a training dataset, and many training samples usually produce the most robust model. However, the practical circumstances of morphological studies underlie the critical need for a data augmentation step in SVMorph, where the rarity of specific specimens often makes sample collection very challenging.
Data augmentation artificially creates synthetic training data with label-preserving transformations by adding slight modifications such as cropping, flipping, rotation, and color jittering, which effectively deal with limited sample size datasets.
Additionally, data augmentation may potentially involve both prior knowledge about the data and task-specific invariances, each of which can regularize the model (Dao et al., 2019). Therefore, to promote task-specific invariances and generalization of classification models, different augmentation methods are needed for different tasks. Based on data-specific invariances and features, the preprocessed training datasets may undergo one or more augmentation methods (Figure 2c,d) by implementing the augmentation.m script in SVMorph. In our sample datasets, the butterfly wing patterns are less sensitive to parameters such as resolution, brightness, and contrast, so we adopted extensive augmentation methods for butterfly images. As for spider images with texture features more susceptible to the abovementioned image factors, we applied fewer methods to avoid distorting the texture features during the augmentation process. Despite the ubiquity of data augmentation in image classification, its underlying theoretical principles are not well understood.
Therefore, in practice, the choice of data augmentation methods often depends on the actual performance. Basically, cropping and flipping methods are more reliable and commonly used, which may be considered as the primary choices. We used random parameters for data augmentation, which refer to the specific input arguments of data augmentation methods. For example, in the noise addition step, we introduced Gaussian white noise with random mean values and random variance values. Thus, different images were augmented F I G U R E 1 SVMorph framework and workflow. HOG, histogram of oriented gradient (Dalal & Triggs, 2005); LBP, local binary pattern (Baraldi & Panniggiani, 1995); and SVM, support vector machine differently, which could increase the diversity of the image datasets and avoid introducing bias during the augmentation process. We also tested the effects of data augmentation on the two sample datasets, subsequently demonstrating that data augmentation significantly improved model accuracies in both cases ( Figure 3). Therefore, data augmentation is necessary for optimal model performance.

| FE ATURE E X TR AC TI ON
SVMorph offers two descriptors for feature extraction from the augmented training dataset: HOG (Dalal & Triggs, 2005) and LBP (Baraldi & Panniggiani, 1995). To classify organisms that display a combination of various elements and that produce local edge and gradient information, we recommend HOG as the major feature descriptor. HOG feature extraction is implemented by the extractHOG-Features function in the Computer Vision Toolbox in MATLAB 2020b (https://www.mathw orks.com/produ cts/compu ter-vision.html).
Once implemented, HOG divides images into small spatial regions called cells, computes discrete histograms for each cell, and then assembles the cells into larger spatial regions called blocks to normalize histograms of all the cells in a block (Dalal & Triggs, 2005), which helps to maintain better invariance and to illuminate changes or shadowing (Figure 4). For texture classification tasks, we recommend LBP be used together with HOG. LBP calculates neighboring pixel gray values to a center pixel and encodes them as a binary number (Baraldi & Panniggiani, 1995), which also endows LBP the advantage of computational simplicity ( Figure 5).

| CL A SS IFIER TR AINING
To train multiclass SVM models, we implement the fitcecoc function in the Statistics and Machine Learning Toolbox (https://www.mathw orks.com/produ cts/stati stics.html) in MATLAB 2020b. That function adopts a one-versus-one coding design that includes a set of binary SVM classifiers for each possible pair in all classes, so for a model of n classes, n(n -1)/2 binary SVM classifiers are trained. We improve classification accuracy by applying the error-correcting output code multiclass model, which is helpful to deal with multiclass classification problems based on multiple binary classifications (Dietterich & Bakiri, 1994). For the classification tasks using example datasets, all feature extraction and classifier training were performed on a computer with an Intel i7-10700K (3.80G Hz) processor and 32 GB of RAM under a 64-bit Windows operating system.

| CL A SS IFIER TE S TING
During the evaluation step, we used k-fold cross-validation, which partitions all samples into k subsets randomly, to examine classifier performances. While a single subset is reserved as test data, the

| OUTPUT
Our optimally performing, trained and tested classifiers can be used for relevant data classification tasks. Using the prediction.m script in SVMorph, we designed classifiers to output predicted labels directly.
New data that have undergone preprocessing can be classified using the established classifier, and that results in the most relevant labels being assigned to the image data ( Figure 7).

F I G U R E 4
Local histogram of oriented gradient (HOG) feature extractions from the butterfly wing patterns. We chose focal regions (in white boxes) from typical 256 × 256-pixel image data of Kallima inachus Morph 3 (a) and Morph 2 (b), with a cell size of 32 × 32 pixels and a block size of 2 × 2 cells. (c, d) Rose plots of the local HOG features extracted from each specific area in the white boxes in (a) and (b), respectively. Petal length indicates the gradient orientation distribution of a direction within a cell. Petals showing sharp differences between two wing forms are highlighted in red (c) and blue (d) Considering those high-dimensional feature vectors, we selected the SVM model with the linear kernel function to reduce computational complexity and overfitting, which also guarantees that the model has fine generalization properties. To evaluate the butterfly classifier's performance, we evaluated accuracy, precision, recall, and F1 scores (Table 1) by using 20 replicates of 10-fold cross-validation.
Using data augmentation, the butterfly classifier's overall accuracy Butterfly ROC S pider ROC F I G U R E 7 Classifier output that can be used for classification tasks. Examples of the output labels predicted by the trained classifiers for dead-leaf butterflies (a) and jumping spiders (b) using test data. Labels M1-M5 correspond, respectively, to butterfly morphs 1-5 shown in Figure 2a, and labels M1-M3 correspond, respectively, to spider morphs 1-3 shown in Figure 2b  scores. However, Morph 5 had a relatively low recall value (86.8%), most likely due to its small sample size or high within-group variation.
The trained model's macro-averaged ROC curve showed very good predictive ability for butterfly classification (Figure 6a). Now using the trained classifier, we classified the preprocessed test data and labeled it accordingly (Figure 7a). The subsequent computational feature extraction and classification processes for this K. inachus dataset with 1,764-dimensional feature vectors took approximately 0.03 s per image to run. According to our results, the butterfly image data were classified and assigned accurately, efficiently, and appropriately to the abovementioned morphs, thus demonstrating the power of SVMorph to extract and distinguish complex wing patterns.

| Body texture classification in jumping spiders
Toxeus spp. jumping spiders are highly polymorphic mimics that resemble unpalatable Polyrhachis ants and thus have a protective advantage (Yamasaki & Ahmad, 2013), providing an excellent example with which to test SVMorph. Among them, members of the species complex of T. maxillosus, T. magna, and T. globose are closely related and have three mimicry morphs according to the body texture features that we named from Morph 1 to Morph 3 (Figure 2b).
Using a microscope (Nikon SMZ18) with an attached digital camera (Nikon DS-Ri2), we photographed 99 individual spiders. Raw images were captured using SHR Plan Apo 1×, at zoom magnification of 0.75× with Nikon C-FLED2 LED Light Source. We directly transformed the images into eight-bit grayscale format and zoomed into the abdominal areas ( Figure 2b). Finally, we classified and labeled the images manually according to their phenotype information (n Morph 1 = 19, n Morph 2 = 42, and n Morph 3 = 38). Since the spider data were more sensitive than the previous butterfly wing data, we performed data augmentation in the same way as we did for the butterfly data except for noise addition and contrast adjustment (Figure 2d). The images were filtered using an averaging filter with a hsize of [3 3] and the convolution option. During feature extraction, we used the same HOG parameters that we had used for the butterfly wing images and extracted 8,100-dimensional HOG feature vectors from each image.
In addition, we introduced LBP to extract local texture information.
To preserve more local details, we divided the original 512 × 512pixel images into 64 × 64-pixel spatial regions and then calculated uniform LBP histograms for each cell ( Figure 5). Specifically, a oneradius circular pattern was used to compute the LBP of eight neigh- to that of the butterfly classification model performance ( Figure 6).
Finally, the preprocessed spider test data were classified using the trained classifier and labeled accordingly (Figure 7b). The feature extraction and classification process for the spider dataset with 11876-dimensional feature vectors took approximately 0.04 s per image to compute. Our results showed that the spider image data were precisely classified and assigned to the abovementioned morphs, thus suggesting that these spiders could be classified according to their body textures and without any intermediate morph.

| Performance of SVMorph and other methods
To evaluate the performance of SVMorph, we compared the accu- Note: M1-M3 correspond, respectively, to spider morphs 1-3 shown in Figure 2b. (Simonyan & Zisserman, 2014) (Table S1). All the methods were tested using both butterfly and spider datasets on one Intel i7-8700 (3.20G Hz) CPU with 16 GB RAM. We also tested three CNN-based methods on a Quadro P400 GPU, but the VGG-16 did not complete the task owing to memory overflow, indicating a limitation of GPU usage on such sce-

| D ISCUSS I ON
The scales and setae of invertebrates are single-cellular cuticular structures extending from the cuticle, which play an important role in generating phenotypic diversity and can have a profound impact on fitness (Tian et al., 2019). Taken the two sample datasets as an example, the dead-leaf butterflies in K. inachus display a remarkable diversity of leaf wing patterns for masquerade (Protas & Patel, 2008), whereas different morphs of Toxeus jumping spiders mimic different ant species to gain a protective advantage (Yamasaki, 2015). These cuticular structures accommodate not only color variation but also shape variation such as length, width, and diameter of a cross section, and pilosity variation (Matsuoka & Monteiro, 2018

| CON CLUS IONS
SVMorph, a fast and accurate pipeline for handling classification tasks with complex patterns and textures, is particularly helpful for batch processing large image data applications. Also, based on the performance of a trained classifier, it can be used to examine a priori hypotheses of organismal classifications or to extract and investigate images with rare morphological features that cannot be appropriately labeled. Moreover, SVMorph is very modular and easy to schedule, thus giving the researcher the ability to perform each step independently and interactively with other applications. In summary, SVMorph efficiently characterizes and classifies morphological characters of nonmodel organisms.

CO N FLI C T O F I NTE R E S T
None declared.

O PE N R E S E A RCH BA D G E S
This article has been awarded Open Data, Open Materials Badges.
All materials and data are publicly accessible via the Open Science Framework at https://github.com/TDQ23 3/SVMorph.