Performance evaluation of machine learning algorithms to assess soil erosion in Mediterranean farmland: A case‐study in Syria

The development of new techniques, such as machine learning (ML), can provide better insight into the processes and drivers of soil erosion and runoff. However, the performance of these techniques to assess soil erosion in agricultural landscapes is poorly understood. The aim of this study was to evaluate the performance of four machine learning algorithms, generalized linear model (GLM), Random Forest (RF), elastic net regression (EN) and multiple adaptive regression splines (MARS), in predicting soil erosion and runoff in Syria. Soil erosion and runoff were measured on three experimental plots (2.25 m × 1.50 m × 0.50 m, 0.10 m depth in the soil), combined with three different slopes and land use types: RS (8%, olive), SS (12%, citrus), KS (20%, pomegranate). Both erosion and runoff were determined after rainfall events of >10 mm between October 2019 and April 2020. Based on 24 effective rainfall events, the average soil erosion was 0.18 ± 0.14 kg m−2 per event in KS, 0.14 ± 0.11 kg m−2 per event in SS, and 0.12 ± 0.10 kg/m2 per event in RS. Regression analysis indicated strong relationship between the rainfalls and the runoff, the highest connection was recorded in the KS plot (r2 = 0.85; p < 0.05 n = 24). The analysis of covariance indicated that only the runoff had a significant impact on soil erosion (p = 0.02) with a medium effect (ε2p = 0.26). However, the impacts of rainfall events and slope categories on soil erosion were limited (ε2p < 0.01) and not significant (p > 0.05). ML techniques were usually efficient in the prediction, the RF and MARS models were the most accurate: RF had the strongest correlation with the measured values (r = 0.85) with a low estimation error (0.06 kg m−2), but MARS's standard deviation (SD) was closer to the recorded values' SD. GLM and EN were the weakest predictor models. Modeled values of the slightest slope (8%) had the worst accuracies, and the predictions of the 12% slope were the best in all models. This study provides important insights into the usefulness of machine learning techniques and algorithms in predicting the rate of soil erosion and runoff in agricultural dominated landscapes. We highlighted that the RF and MARS algorithms were better predictors of soil erosion and runoff in the coastal region of Syria.

(p = 0.02) with a medium effect (ε 2 p = 0.26). However, the impacts of rainfall events and slope categories on soil erosion were limited (ε 2 p < 0.01) and not significant (p > 0.05). ML techniques were usually efficient in the prediction, the RF and MARS models were the most accurate: RF had the strongest correlation with the measured values (r = 0.85) with a low estimation error (0.06 kg m À2 ), but MARS's standard deviation (SD) was closer to the recorded values' SD. GLM and EN were the weakest predictor models. Modeled values of the slightest slope (8%) had the worst accuracies, and the predictions of the 12% slope were the best in all models. This study provides important insights into the usefulness of machine learning techniques and algorithms in predicting the rate of soil erosion and runoff in agricultural dominated landscapes. We highlighted that the RF and MARS algorithms were better predictors of soil erosion and runoff in the coastal region of Syria.

K E Y W O R D S
Entisols, GLM, ML-algorithms, soil properties, Syria

| INTRODUCTION
Soil erosion is by far the most important type of land degradation, exerting negative effects on the environment and societies worldwide (Nasir & Selvakumar, 2018;van Leeuwen et al., 2019). Soil erosion is recognized as a key environmental issue as water-induced soil erosion remains a dominant agent of degradation globally (Lal et al., 2018). It is a major threat to the sustainable agro-ecosystem, especially in places where high erosion rates alter natural cycles of crops production (Muluneh et al., 2017). This makes soil erosion a critical issue that humanity must address if sustainable food production is desired (García-Ruiz et al., 2015). Soil erosion is characterized by three actions: soil detachment, transport, and deposition. The process of soil relocation is influenced by three main components: (i) rainfall (amount, intensity, duration and erosivity), (ii) soil properties (texture, moisture, organic matter content, aggregate stability), and (iii) local conditions (i.e., land use, land cover, slope) (Dunne et al., 2010;Kulimushi, Bashagaluke, et al., 2021;Mohammed, Al-Ebraheem, et al., 2020;. Since the introduction of the first equation to predict soil erosion (Zingg, 1940), there have been significant advances in soil erosion modeling. These advances have led to the development of numerous models to estimate soil erosion under different ecological and landscape conditions (see Dutta (2016) for a list of common soil erosion models). However, while empirical approaches can be useful for specific cases or for model calibration, they are limited by large uncertainties and high variability between predicted and observed soil erosion values (Alewell et al., 2019).
The Mediterranean region is considered one of the most erosion prone regions in the world. García-Ruiz et al. (2013) noted that the Mediterranean basin is a soil erosion hotspot. This is supported by numerous studies that have reported rapid soil erosion in various countries in the region (Khallouf et al., 2021). Many reasons have been put forward to explain the accelerated soil loss in the Mediterranean basin. These include: (i) poor soil structure that accelerates the mineralization process of organic matter; (ii) shallow soil; and (iii) poor management of vegetation cover, as well as traditional agricultural practices (García-Ruiz, 2010;Keesstra et al., 2019). In addition, climate change (mainly variability in rainfall duration and intensity) and changes in the agricultural system have aggravated soil degradation (especially soil erosion) (Benchettouh et al., 2017;Cerda et al., 2018;de Hipt et al., 2018) in the region.
Soil erosion is a threat to the economic development of Syria, and a large amount of productive soil is lost annually due to natural and anthropogenic factors (Masri et al., 2015;Mohammed et al., 2021). In general, wind erosion is dominant in the eastern and central regions of Syria (Masri et al., 2015), while the western and northern parts, as well as the mountains, are more prone to water-induced soil erosion (Mohammed et al., 2016;Mohammed, Al-Ebraheem, et al., 2020;. Despite the challenges posed by soil erosion in the Mediterranean region, only a few studies have examined the question. In addition, the few studies focusing on soil erosion mostly adopted the revised universal soil loss equation (RUSLE) in a geographic information system (GIS) environment or remote sensing techniques (Abdo, 2021;Abdo & Salloum, 2017;Almohamad, 2020;Alsafadi et al., 2022). However, it is essential to use field-based observations to validate the model outputs.
The development of new techniques, such as machine learning algorithms, can be used for model validation and provide better insight into the processes and drivers of soil erosion and runoff. However, the performance of these techniques for assessing soil erosion and runoff in agriculture dominated landscapes is poorly understood. The aim of this study is to evaluate the performance of four machine learning algorithms, generalized linear model (GLM), Random Forest (RF), elastic net regression (EN), and multiple adaptive regression splines (MARS) in predicting soil erosion and runoff in Syria. This evaluation is important because soil erosion and runoff pose a major threat to agricultural development in many countries including Syria.
The objectives of the study were to: (i) measure soil erosion and runoff in three different slopes in the coastal region of Syria, (ii) predict about 65% of the basin's land, followed by forest land with 21%, urban areas with 5%, water bodies with 5%, and others with 4%. Due to the large variation in temperature with altitude above sea level, there is a discrepancy between the type of crops existing in this study area and the type of natural vegetation cover prevails. In this typical agriculture basin, protected crops such as tomato and cucumber (4%) dominate the downstream region and are mainly irrigated. The middle part of basin has predominantly rain-fed agriculture, mainly olive trees (57%) and crops (25%). In the upstream mountains, fruit plantation such as citrus (7%), apple (7%) is common . The common soil orders are Entisols and Inceptisols according to the American taxonomy of soils, where the parent material is calcareous rocks (Mohammed, Al-Ebraheem, et al., 2020).

| Experimental sites and data collection
An initial soil survey was conducted throughout the study area to determine soil types. Three major soil types were identified in the study area. Based on soil type and slope, three experimental locations were marked for this study. The first location was Kafar-Snief (KS) (x: 36.17, y: 34.812) with a slope of 20%, the second was Snisinia (SS) (x: 36.133, y: 34.776) with a slope of 12%, while the last was Rawas (RS) (x: 36.025, y: 34.748) with a slope of 8% (Table 2). Surface soil samples (0-10 cm) were randomly collected from each study and thoroughly mixed to obtain a homogenous sample before being T A B L E 1 Examples of machine learning algorithms applied for soil erosion and runoff prediction.  10 mm (i.e., effective rain). Thus, in this research, we followed this recommendation, where a 1-L sample was collected from each container.
A total of 72 samples (24 events Â 3 locations) were collected during the monitoring period. Before collecting the samples, the volume of water (i.e., runoff) in each container was accurately measured (mm).
Then, the contents (water + eroded soil) of each container were carefully mixed to ensure sample homogeneity (1 L). Containers were emptied, thoroughly cleaned, and covered in preparation for the next rain event. This process was repeated each time it rained during the study period (between 10/24/2019 and 4/5/2020). The soil from each sample was separated from the water by the sedimentation process, and then dried in a soil drying oven for 24 hr at 105 C. The dried soil was weighed and assigned to the accumulated runoff volume in the tank (kg m À2 ).

| Machine learning models
In this study, the predictive power of four ML models was tested for rainfall, runoff, and the slope factors.
F I G U R E 1 Map of the study area: (a) location of the study area, (b) experimental plots (RS, SS, and KS) within Al-Abrash River basin, (c) box plot of the monthly rainfall in the study area . [Colour figure can be viewed at wileyonlinelibrary.com]

| Generalized linear model (GLM)
The GLM was selected as a linear model with prerequisites on the normal distribution of residuals and the homoscedasticity. The model was developed by McCullagh and Nelder (1989), and the algorithm is used in cases where a target variable is explained by n independent variables and hence creates a linear relationship between the variables (Pourghasemi et al., 2020). In this study, the GLM was the analysis of covariance (ANCOVA model) with an ordinal predictor (slope) and two covariates (rainfall and runoff). We investigated the role of the variables and reported the result of the hypothesis tests, and beside the significance, the effect sizes (ε 2 p) were also calculated.
GLM was also used as a predictor function and as a statistical function known as LOGIT that is expressed as Equation (1): Y is the dependent variable and x 1 , x 2 ,… x n are independent observations.

| Random Forest
The RF is a robust algorithm that relies on bootstrapping (i.e., random sampling) and decision trees (Breiman, 2001). Many decision trees are involved in classification and prediction. The output of RF could be represented as Equation (2):

| Elastic net regression
The elastic net regression (EN) is a type of regularized regression, which overcomes the limitations of LASSO (least absolute shrinkage and selection operator) and ridge regression (Zou & Hastie, 2005).
Ordinary least square (OLS) regression has problems when the independent variables are correlated, and this leads to a decline in the accuracy of the resulting model. The ridge and LASSO methods apply a penalty term (L1-norm and L2-norm, respectively) to the OLS equation to obtain a better model with lower bias and variance. However, both LASSO and ridge have limitations, EN applies a weighted combination of the two regularizations, that is, penalizing the correlating variables, and those that contribute less to the explained variance.
Rainfall and runoff had a correlation of r = 0.79 ( p < 0.001), and thus could bias the results, EN was an effective method to address this problem. Following Zou and Hastie (2005), the EN algorithm is expressed in the Equation 3: α and λ are the turning parameters while α combines the other two regulations alternatives.

| Multiple adaptive regression splines (MARS)
The multiple adaptive regression splines (MARS) was first introduced by Friedman and Roosen (1995). It is as a non-parametric version of multiple linear regression without prerequisites on the variables (Mosavi, Golshanc, et al., 2020;Rotigliano et al., 2018). MARS models are developed on segmented sections of the multivariate space defined by the dependent and independent variables. Linear regression models are then run on all segments. Each segment has a linear function with its own function parameters and the connections of lines are called the knots. All knots have two basic functions (BFs).
Next, BFs will be the independent variables of the final model. MARS model is expressed in Equation 4: All models were built with the caret package (Kuhn et al., 2020) of R 4.04 (Pinheiro et al., 2021) combined with hyper-parameter tuning (fine tuning of model parameters with the built-in procedure of the caret package) aimed at achieving the smallest RMSE.

| Principal component analysis
We applied standardized principal component analysis (PCA) using the correlation matrix to study the effect of slope classes on the multidimensional space defined by erosion, rainfall and runoff. Model fit was assessed using the root mean square residual (RMSR). RMSR<0.05 indicates very good, <0.1 good model fit while higher values were excluded because they do not support the application of PCA (Basto & Pereira, 2012). The principal components (PCs) scores were visualized in a scatterplot diagram and slope classes were delineated.

| Model validation
Model validation was performed using an independent dataset obtained by splitting the dataset into a training and a testing part using a ratio of 70:30 stratified random sampling on slope classes. The training data were used to train the models and the model prediction was based on the test data. This accuracy assessment approach was the independent accuracy testing (IAT). Differences between the predicted and observed data were evaluated using Taylor diagrams. Taylor diagrams show the correlation between observed and predicted values, the squared root of the difference between observed and predicted values (root mean square error, RMSE), and report the standard deviation of observed values and predicted values. We visualized the results by the regression algorithms and slope steepness. Furthermore, Nash-Sutcliffe coefficients (NSE) were also determined by models (Nash & Sutcliffe, 1970).
The models were developed using the repeated k-fold crossvalidation (RKCV) method. Hence, fivefolds and 10 repetitions were applied: data had been randomly split fivefold and four were used to train the model while onefold was kept testing the predicted values with independent data. This procedure was repeated until all folds had become test data. Then, the entire process was repeated 10 times, and accuracy data from 50 models was available. R 2 , and RMSE were reported in the evaluation, and we calculated the basic statistics (LQ: lower quartile, UQ: upper quartile, IQR: inter-quartile range, median, minimum, and maximum) of the 50 models, we also determined the relative RMSE (rRMSE) as the ratio of RMSE and mean erosion.
Although both IAT and RKCV are reliable measures for assessing model accuracy, there are some differences. The IAT evaluates predictions based on different factors (in this case by slope categories), but relies on a single random split, while RKCV uses 50 models to provide prediction accuracy, but only as an overall evaluation. The RKCV can be considered an independent measure of accuracy. However, we examined the model efficacy on a completely independent dataset (testing data) as an extrapolation. In addition, the combined application of the approaches allowed us to exploit the advantages of these approaches.

| Soil erosion and runoff in the coastal region of Syria
Cumulative rainfall during the study period was 946 ± 30.86 mm measured during 24 rainstorm events (rainfall >10 mm) on each plot Although the average amount of rainfall in the study area was similar at all study sites, the average amount of rainfall was slightly higher at KS compared to the other locations ( Figure 4 and Table 3).
A similar pattern was observed for mean runoff and mean soil erosion.
The regression models showed a positive and significant ( p < 0.05) relationship between rainfall and runoff. The relationship was strongest at the KS plot (R 2 = 0.85; p < 0.05 n = 24). In terms of eroded soil, the impact of rainfall was not significant at the RS plot (R 2 = 0.1;

F I G U R E 3 Measured rainfall (mm) (a), runoff (mm) (b), and soil erosion (kg m À2 ) (c) after each rainfall event in the RS, SS, and KS experimental plots. [Colour figure can be viewed at wileyonlinelibrary.com]
p > 0.05 n = 24), however, it was significant at the other locations (Table 4 and Figure 5).
The analysis of covariance (ANCOVA) showed that runoff had a significant effect on soil erosion ( p = 0.02) with a medium effect (ε 2 p = 0.26). However, rainfall events and slope categories were not significant ( p > 0.05) and had a small impact (ε 2 p < 0.01) (

| Performance of machine learning models
According to the RKCV, the MARS algorithm had the best performance, R 2 = 0.406, with respect to the median of the 50 models. The  RMSE was also the smallest for MARS, the mean rRMSE was 52%.
Compared to the other models, upper quartile of RMSE was lower than the medians for GLM, EN, and RF. Based on the 50 models, MARS and EN had the largest IQR of the R 2 (0.563), but the MARS's LQ was the highest (0.097) and EN's LQ was only 0.001 better (i.e., 0.057) than that of GLM. The EN regression was the weakest model, with the median R 2 of 0.167; nevertheless, the largest rRMSE belonged to GLM (74.3%). The RF provided the smallest IQR (0.302) but the UQ was only 0.36 indicating a poor fit with high rRMSE values (82%) (Figure 7).
The IAT provided similar results to the RKCV in terms of model performance, RF and MARS had the strongest correlation with the observed data and had the smallest RMSE. RF had better correlation and lower RMSE than all other models, but MARS's SD was the closest to the real values (Figure 8). NSEs also confirmed that RF and MARS were the most efficient models (GLM: 0.15;EN: 0.19; MARS: 0.31; RF: 0.42).
We also analyzed the effect of slope inclination on the predicted values, which was only possible with the IAT approach. We found that the slightest slope (8%) had the worst predicted values, in some cases this slope category did not provide reasonable outcome and could not be plotted on the Taylor diagrams, the correlation tended towards zero and RMSEs were high (Figure 9). The best predictions were observed with the 12% slope category, and even the steepest slope (20%) category made the prediction efficacy worse.

| DISCUSSION
This research used an experimental approach to validate the results of the implemented machine learning algorithms. Samples from three experimental locations were used to extract soil properties as well as slope inclination and rainfall data, which were then used to calculate soil erosion. It is crucial to define the most significant soil parameters that are most related to soil erosion. Parameters that provide information about soil structure and formation are most relevant, such as texture, moisture, organic matter content, and aggregate stability.
These parameters describe how soil particles aggregate to come together, as the size of aggregated particles affects the ease of soil dispersion, such as dispersion in runoff water (Diaz-Gonzalez et al., 2022). In this sense, soil characteristics, including moisture, organic matter content, aggregate stability, and soil particle size distribution, play a critical role in influencing the rate and pattern of soil erosion, as they control the rate of water infiltration, runoff generation, and erosion processes. For instance, soil structure, soil water holding capacity, and the nature of the landform (slope) in the landscape strongly affect the rate of water infiltration, runoff generation, soil mechanical behavior, ease of soil detachment, and soil erosion.
These processes have been shown to exhibit distinct spatio-temporal characteristics that may determine the nature of erosion at the plot, hillslopes, and catchment scales (Auzet et al., 2004;Mosavi, Sajedi-Hosseini, et al., 2020).

| Soil erosion and runoff in the eastern Mediterranean
The susceptibility to erosion and runoff in the costal part of Syria results from the interaction among the main erosive factors, including soil characteristics, heavy rainfall, and severe inclination (Masri et al., 2015;Mohammed et al., 2021;Mohammed, Al-Ebraheem, et al., 2020;. Although we also confirmed these observations, we found that the triggering (slopes and rainfalls) and modifying (soil characteristics and vegetation cover) factors had different weights. Erosion and rainfall had a positive and significant relationship (R 2 = 0.10-0.47) from the bivariate regression models at the SS and KS sites, but this was not the case for the RS (Table 4, Figure 5). However, the multivariate model (ANCOVA) indicated that neither rainfall nor slope inclination had a significant relationship with soil erosion (Table 5); that is, involving all sites and variables in a single model, it was found that the importance of the main trigger was not significant in this case. The frequency of intense rainfalls events (i.e., erosivity) is significant in the coastal part of Syria (Abdo, 2020), which could influence the sediment transport capacity of runoff. Thus, the fact that rainfall events were equal in quantity, but the intensities were different, resulted in variations in runoff and soil erosion (Table 3 and Figure 4).
With respect to modifying factors, the results agreed with Lal (1994), who reported that regardless of rainfall amount, vegetation coverage dissipates the kinetic energy of raindrops. The identical findings of  and Kulimushi, Maniragaba, et al. (2021) indicated that under constant conditions of rainfall intensity, poor land use practices and land topography are the most important factors that determine the intensity of runoff and soil erosion.
Rainfalls and runoff were highly correlated (r = 0.79 p < 0.001), but the correlation between runoff and soil erosion was moderate (r = 0.52 p < 0.001), accordingly, land cover and soil properties got higher relevance, which justified the findings of Nasir and Selvakumar (2018

| Applicability of machine learning algorithms for soil erosion estimation
Although machine leaning is increasing in popularity in many fields, it has only recently been widely applied for soil erosion estimation and mapping (Dinh et al., 2021;Javidan et al., 2019;Nguyen et al., 2021;Sahour et al., 2021). However, the performance of many algorithms for water-induced soil erosion is mostly poorly understood and requires detailed analysis. We revealed that RF and MARS models are more reliable in predicting soil erosion than GLM and EN. Similar findings were reported by Previously, Garosi et al. (2019) concluded that RF models performed better in predicting soil erosion where soil texture ranged from sandy clay loam to sandy loam and land use types were of orchard rangeland and farmland. However, in contrast to our results which showed that MARS apparently performed better than RF (R 2 RKCV = 40.6%), but the RMSE IAT was more favourable with RF. This result is consistent with the findings of Nguyen et al. (2021): although RF performed better overall, MARS also provided good modeled values. These results draw attention to the importance of using an independent test dataset, as RKCV was efficient in model building, but as a measure of accuracy, IAT provided a more reliable outcome.
According to ANCOVA, the relevance of slope steepness did not directly influence erosion (i.e., slope was not significant and had a small effect size); however, we identified that the predictive ability of the machine learning models was dependent on slopes: an 8% slope had no correlation with observed values, and MARS was not able to predict its values correctly; furthermore, a 20% slope almost doubled the RMSE compared to the well-predicted values of a 12% slope (Figure 9). This means that, in general, the models predicted erosion with a reasonable error, and especially the RF had the performance (Figure 8), but, with respect to the slope categories, the 12% provided the best modeled values. For the 8% and 20% slopes, the role of soil properties and land use biased the overall trend, which also caused the small effect sizes for slopes and rainfalls (Table 5). Another cause of the ambiguous role of slopes was the multivariate similarity in rainfall, runoff, and erosion characteristics of the three types of steepness (Figure 6), which justifies the relevance of soil and land use.
F I G U R E 9 Effect of slope inclination on the performance of the machine learning algorithms, which depicted by using IAT approach (MARS: multiple adaptive regression splines, GLM: generalized liner model, RF: Random Forest , EN: elastic net regression). [Colour figure can be viewed at wileyonlinelibrary.com]

| Research limitation and future outlook
Although the models evaluated in this study produced reasonable results, resource limitations constrained us to limit data collection to three experimental plots. This did not allow for adequate variation in landscape properties and other processes capable of influencing model performance. Therefore, future studies should seek to include more experimental plots that are more representative of landscape features such as land use and topographical features. The plurality of machine learning algorithms implies that future evaluation of machine learning algorithms should include more diverse algorithms applied under different environmental, social, and economic conditions to gain more insight into their performance.

| CONCLUSIONS
We aimed to reveal the effectiveness of four ML algorithms in predicting soil erosion using experimental data from three plots in the Eastern Mediterranean. We made the following observations: • Slope inclination and rainfall were not the most important factors of soil erosion. Although runoff was the most important, its relationship with erosion was only moderate, and the R 2 was only 0.27. Therefore, runoff triggered erosion but rainfall intensity, soil properties (OM and clay content), and vegetation cover may have reduced its erosivity.
• The robust RF and MARS algorithms were superior to the linear GLM and EN models in predicting soil erosion. However, even the MARS, with the best performance, had a median R 2 (of 50 models) of 40.6, with a larger RMSE (52%), which is a consequence of similar erosion rate at the SS and KS sites.
• RKCV and IAT highlighted important features of the models, and we found that using the independent dataset (IAT), the RF model was the best with a high correlation between the modeled and observed data (0.85) and low RMSE (0.06). However, the slope inclination of 8% was not a successful part of the model having unusable results.
• The results draw attention to the fact that, although slopes are important factors in soil erosion, other environmental factors (e.g., land use, soil properties) can overwrite this general rule, and site-specific characteristics require special attention. Soil conservation practices need to be re-evaluated in terms of runoff reduction.
The output of this research calls the attention to the need of careful consideration of the erosion factors, which requires a sitespecific evaluation including the soil properties and land cover beside rainfall and slope inclination.

ACKNOWLEDGMENTS
Authors would like to thank Debrecen University for supporting open access publishing. Additionally, the authors would like to thank the anonymous reviewers and editors for their valuable feedback.