Nondestructive classification of saffron using color and textural analysis.

Abstract Saffron classification based on machine vision techniques as well as the expert's opinion is an objective and nondestructive method that can increase the accuracy of this process in real applications. The experts in Iran classify saffron into three classes Pushal, Negin, and Sargol based on apparent characteristics. Four hundred and forty color images from saffron for the three different classes were acquired, using a mobile phone camera. Twenty‐one color features and 99 textural features were extracted using image analysis. Twenty‐two classifiers were employed for classification using mentioned features. The support vector machine and Ensemble classifiers were better than other classifiers. Our results showed that the mean classification accuracy was up to 83.9% using the Quadratic support vector machine and Subspace Discriminant classifier.

be avoided using an objective approach such as image processing (Pourreza, Pourreza, Abbaspour-Fard, & Sadrnia, 2012). Advances in machine vision technology make accurate, robust, and low-cost vision machine systems that make it suitable for detection food quality and so this technology can be used to determine the quality of saffron (Kiani & Minaei, 2016). Kiani, Minaei, and Ghasemi-Varnamkhasti (2018) propose the use of E-nose, E-tongue, and CVS systems to evaluate saffron quality and replace sensory recognition by human assessors (Kiani et al., 2018). Minaei, Kiani, Ayyari, and Ghasemi-Varnamkhasti (2017) demonstrated that the combination of computer vision system (CVS) and multilayer perceptron (MLP) is a simple tool for evaluating the quality of saffron samples based on color strength. The performance of the MLP model for saffron color recognition was better than PLS and MLR, and the success rate of classification (CSR) was 96.67%. (Minaei et al., 2017). Today, color computer vision systems are used in various food industries and agricultural products sorting systems because they are reliable, fast, and inexpensive (Donis-González & Guyer, 2016). Color computer vision is used to categorize or recognize the quality of agricultural products and various types of foods, including dates (Muhammad, 2015), pistachios (Omid, Firouz, Nouri-Ahmadabadi, & Mohtasebi, 2017), apple (Paulus & Schrevens, 1999), pizza (Sun, 2016), and Wheat (Pourreza et al., 2012). The computer vision system is trained based on specific patterns extracted from a set of color images provided for different classes, such as texture, geometry, and color properties.
Then, the computer vision system determines which new image belongs to which particular category (Faucitano, Huff, Teuscher, Gariepy, & Wegner, 2005). The first step involves extracting a large number of features from classified images. Then, the features must have the ability to separate the classes correctly, which, by training the system, can automatically categorize the new image. Classification is performed by statistical algorithms and different clustering by assigning each image to the corresponding class (Donis-González & Guyer, 2016). The purpose of this study was to design a visual machine technique for detecting different types of saffron (Sargol, Negin, and Pushal) using images taken with mobile phones from bulk samples. Texture properties, color properties, and the percentage of foreign matter (based on color) of saffron were obtained.

| Saffron samples
A total of 440 samples of different saffron kinds on the market were prepared, without any additives, from various cities of Khorasan Province: Gonabad, Bajestan, Roshtkhar, Sabzevar, Mashhad, Torbat Heydarieh, and Kakhk, without any additives as fraud, and then, the samples were coded. Four experts who had a long history of saffron trading were selected. They divided the specimens into three classes, Sargol, Negin, and Pushal. Samples' information was recorded in a database (Zheng & Lu, 2012) and (Donis-González & Guyer, 2016) and (Zhang, Lee, Lillywhite, & Tippetts, 2017).

| Image acquisition
Image acquisition was done with a cellphone camera (Samsung Galaxy S7 Edge SM-G935FD Dual SIM 32GB Mobile Phone), which was placed on an imaging chamber at a distance of 9 cm from the sample. In the lighting system, SMD LED strip lights (4014 SMD LED Module) have been used in the upper part of the imaging chamber. A diffuser was installed for the uniformity of light under the lamps. The black background color was used to create the best contrast. The shutter speed was 1/500 s without employing flash, and, respectively, lens focal length, Diaphragm value, and ISO were 4/2 mm, F1/7, and 100. Images were captured at their maximum resolution (3024 × 4032 pixels) and were saved in "JPG" format. For imaging, the images were transferred to the laptop, which was equipped with MATLAB software (2017b. ver. 9.3). The images were given to the expert individuals to classify the samples into three classes: Sargol, Negin, and Pushal. Based on the average view of the experts, 440 different samples were F I G U R E 1 Different types of saffron including Negin, Sargol, Pushal, and Daste taken from them, and they were divided into three categories: 195 samples: Pushal, 129 samples: Negin, and 116 samples: Sargol. In this case, the average views of the experts were selected as the criteria for tagging the samples.

| Image preprocessing
Original sample image is presented in Figure 2a. In the first step, in order to remove the noises and smooth it, the image is filtered using a low-pass filter. The result is shown in Figure 2b. Foreground of the image is selected by choosing the pixels having intensity bigger than 20. Results are shown in Figure 2c. Small objects are removed from foreground binary image by morphological opening operation the image where all connected components (objects) that have fewer than 3,000 pixels are removed. Further, the image is eroded and dilated by a morphological structuring element with 5-pixel radius.
The final foreground of the image is shown in Figure 2d. The saffron part of image is cropped by selecting the area, which has nonzero values. For this purpose, the projection of image over vertical and horizontal axis is calculated and the area between minimum and maximum values is cropped. For example, for the sample image, the area between two vertical and horizontal lines shown in Figure 2e is selected. In general, four virtual lines are generated for defining the cropped area. The cropped area image is then used for further processing.

| Textural algorithm
Texture analysis is one of the most important characteristics used in identifying regions of interest in an image and has been widely used in image processing. They are defined as attributes representing spatial arrangement of the gray levels of pixels in a region of a digital image, which provide measures of some properties of a region such as smoothness, coarseness, and regularity (Wang, Zhang, & Wei, 2019). To analyze the textures, the features extracted from the image are local entropy of grayscale image (entropy), local standard deviation of image (STD), local binary patterns (LBP), and gray level co-occurrence matrix (GLCM). Features extracted from GLCM include contrast, homogeneity, correlation, and energy that the mentioned features were extracted from the images. The contrast shows the intensity of the gray variation in the image.
The correlation describes the linearity and dependence of a different two-pixel value. In this case, μ is the mean value of the matrix and σ i σ j of the variance.
The energy represents the order of the image (repetition of the pixel pair) and in fact represents the smoothness and uniformity of the sample surface.
Homogeneity describes the similarity of a pixel with neighboring pixels and reflects the uniformity of the image.
Specifications extracted from entropy, standard deviation, and local binary patterns were calculated according to Table 1. In addition, the histogram is a graphical representation of the number of pixels for each brightness level in the input image. We defined 25 Bin in this study, and in each period, the abundance of things was gathered together and placed there. Finally, 120 features were extracted from each image.

| The local binary patterns (LBPs)
A local binary pattern is a synergistic approach to texture analysis, which can provide a boundary of proximity with a pixel tag and a binary result. The main advantage of LBP in business applications is its ability to maintain independent behavior with grayscale level changes and its computational efficiency, processing images in complex real-time environment. In a basic LBP, each 3 × 3 neighborhood is thresholded by the value of the central pixel. Then, the threshold neighborhood values are multiplied by weights given to

| Classification model
The features outlined in the above sections were used to classify. 22 different calssifiers were used including:

| Decision trees classifiers
Decision tree (DT) is a machine learning algorithm which classifies the training data recursively by each node in order to maximize the separation of data. The decisions in the tree are started from the root node down to a leaf node to predict a response. The leaf node contains the response (Kamiński, Jakubczyk, & Szufel, 2018). Types of models used in this group include Fine Tree, Medium Tree, and Coarse Tree.

| Support vector machine classifiers
Support vector machine (SVM) is an effective modeling tool for classification and was used for regression, pattern classification, prediction, and problem detection (Nasirahmadi et al., 2019). In SVM, data input space is mapped into a high dimensional feature space through a kernel function by using minimal training data (Huang, Tang, Yang, & Zhu, 2016). Types of models used in this group include Linear SVM, Quadratic SVM, Cubic SVM, Fine Gaussian SVM, Medium Gaussian SVM, and Coarse Gaussian SVM.

| Nearest neighbor classifiers
The Nearest neighbor classifiers in the low-precision dimensions is a good predictor. However, they may not have this capability on a large scale. In this classifier, samples that are neighbors or similar to a well-known instance are identified that fall into the set of training, and then, the classification is done based on the training set (Xie, Yang, & He, 2017

| Ensemble classifiers
An ensemble is a supervised learning approach such as bagging, boosting, and variants that use multiple models to improve the predictive performance than could be obtained from any of the constituent models (Dutta et al., 2015). Types of models used in this group include Boosted Trees, Bagged Trees, Subspace Discriminant, Subspace KNN, and RUSBoost Trees.

| Validation and performance evaluation indices
A fivefold stratified cross-validation technique was used to validate the classification. In k-fold cross-validation, the original sample is randomly divided into k equal sized subsamples. Of the k subsamples, a single subsample is remained as the validation data for testing the model, and the remaining/k subsamples are used as training data.
The cross-validation process is then repeated k times, with each of the k subsamples used exactly once as the validation data. The k results can then be averaged to produce a single estimation. The advantage of this method over repeated random subsampling is that all observations are used for both training and validation, and each observation is used for validation exactly once (Siedliska, Baranowski, & Mazurek, 2014). Accuracy, confusion matrix, true-positive rate (TP rate), false-negative rate (FN rate), positive predictive rate (PP rate), and false discovery rate (FD rate) were calculated (Xie et al., 2017).
Also, the receiver operating characteristic (ROC) was computed in MATLAB based on true-positive and false-negative rates. The area under the ROC curve which ranges from 0.5 (no discrimination ability) to 1 (best discrimination ability) was also calculated (Nasirahmadi et al., 2019).
One-way analysis of variance (ANOVA) and Duncan's test were used to determine the significant difference between the accuracy of classifiers. Statistical analysis was performed using SPSS software (IBM Statistics version 23).

| RE SULTS AND D ISCUSS I ON
The 440 color photographs from different samples of saffron including 195 samples of Pushal, 129 Negin, and 116 Sargol were used in this study. The glossary defined for classifiers, including 21 color features and 99 texture features, was extracted from 440 samples.
The classifier was then evaluated using fivefold cross-validation. In the cross-validation, the original samples were randomly partitioned into five groups. Four groups were used as training data for developing the model, and the remaining group was retained as validation data for testing the classifier. The process was repeated for five times, with each of the groups used once as the validation data (Kuo, Chung, Chen, Lin, & Kuo, 2016).

| Classification when features of color were used in the classifiers
The average accuracy of these four classifiers did not differ significantly (p < .05). For Linear SVM classifier, the classification accuracy was 82.23% (±0.66%).   Figure 3 shows the confusion matrix for seven classifiers mentioned. Also, detailed accuracy analysis has been reported in Table 5.

| Classification when combinations of all features were used in the classifier
A high value of TP rate and PP rate, and a low value of FN rate and FD rate, mean the classification model is good. These values for Pushal saffron were better than other classes of saffron. The FN rate and FD rate showed that the classification error of Sargol and Negin is more than Pushal. These errors happen when the values are close to each other, and it is hard to classify them. In terms of appearance, Negin and Sargol are very similar, and the distinction between them is difficult. In the Pushal, three filaments of stigmas are connected, which at the end has a bit of style, but in the Negin and Sargol, three filaments of stigmas are separated.
The receiver operating characteristics (ROC) was an additional method for evaluating the performance of the classification models.
An ROC graph illustrates relative trade-offs between true-positives and false-positives and its x-axis is the false-positive rate, whereas the y-axis is the true-positive rate of the model (Siedliska et al., 2014). The area under the ROC curve (AUC) is an important statistical parameter for evaluating classifier performance. Figure 4 shows the ROC curves, TA B L E 4 Average classification accuracies (%) for 10 times running of fivefold cross-validation using 120 color and texture features for saffron classification

| CON CLUS IONS
In summary, these results showed that the visual texture and color index could be a good index for separating saffron of Pushal, Negin, and Sargol. The saffron samples were collected from the cities of Khorasan Province. A commercially available mobile phone was used to capture the saffron images. The images were given to expert individuals to classify the samples into three classes: Sargol, Negin, and Pushal. A total number of 120 features were extracted from the saffron images.

ACK N OWLED G M ENT
This study was supported by Vice President for Research and Technology, Ferdowsi University of Mashhad, I.R. Iran.

CO N FLI C T O F I NTE R E S T
None declared.

AUTH O R S ' CO NTR I B UTI O N S
The first author was responsible for the accomplishment of most of the works, searching literature data, and write up of the paper.
The second author also contributed in the manuscript preparation and standardized the paper as well as supervision of the whole research works. The third and fourth authors also contributed in the manuscript preparation. All authors approved the final manuscript for publication.

E TH I C A L S TATEM ENT
This study does not involve any human or animal testing.