Predicting Debris Yield From Burned Watersheds: Comparison of Statistical and Artificial Neural Network Models1

Authors

  • Jang Hyuk Pak,

    1. Respectively, Research Hydraulic Engineer, U.S. Army Corps of Engineers, Institute for Water Resources, Hydrologic Engineering Center, 609 Second Street, Davis, California 95616-4687 [Formally, Research Associate, Department of Civil and Environmental Engineering, University of Southern California, Los Angeles, California 90089-2531]
    Search for more papers by this author
  • Zhiqing Kou,

    1. Graduate Research Assistant, Department of Civil and Environmental Engineering, University of Southern California, Los Angeles, California 90089-2531
    Search for more papers by this author
  • Hyuk Jae Kwon,

    1. Lecturer, Department of Civil Engineering, Kangwon National University, Chuncheon, Korea 200-701
    Search for more papers by this author
  • Jiin-Jen Lee

    1. Professor, Department of Civil and Environmental Engineering, University of Southern California, Los Angeles, California 90089-2531.
    Search for more papers by this author

  • 1

    Paper No. JAWRA-08-0031-P of the Journal of the American Water Resources Association (JAWRA). Discussions are open until June 1, 2009.

(E-Mail/Kwon: hyukjae68@hotmail.com)

Abstract

Abstract:  Alluvial fans in southern California are continuously being developed for residential, industrial, commercial, and agricultural purposes. Development and alteration of alluvial fans often require consideration of mud and debris flows from burned mountain watersheds. Accurate prediction of sediment (hyper-concentrated sediment or debris) yield is essential for the design, operation, and maintenance of debris basins to safeguard properly the general population. This paper presents results based on a statistical model and Artificial Neural Network (ANN) models. The models predict sediment yield caused by storms following wildfire events in burned mountainous watersheds. Both sediment yield prediction models have been developed for use in relatively small watersheds (50-800 ha) in the greater Los Angeles area. The statistical model was developed using multiple regression analysis on sediment yield data collected from 1938 to 1983. Following the multiple regression analysis, a method for multi-sequence sediment yield prediction under burned watershed conditions was developed. The statistical model was then calibrated based on 17 years of sediment yield, fire, and precipitation data collected between 1984 and 2000. The present study also evaluated ANN models created to predict the sediment yields. The training of the ANN models utilized single storm event data generated for the 17-year period between 1984 and 2000 as the training input data. Training patterns and neural network architectures were varied to further study the ANN performance. Results from these models were compared with the available field data obtained from several debris basins within Los Angeles County. Both predictive models were then applied for hind-casting the sediment prediction of several post 2000 events. Both the statistical and ANN models yield remarkably consistent results when compared with the measured field data. The results show that these models are very useful tools for predicting sediment yield sequences. The results can be used for scheduling cleanout operation of debris basins. It can be of great help in the planning of emergency response for burned areas to minimize the damage to properties and lives.

Introduction

Alluvial fan areas are rapidly being urbanized in arid and semiarid regions of southern California because of their relatively mild terrain and aesthetic views. The mountain areas upslope from the alluvial fans are susceptible to fires, and this can significantly increase the amount of material transported downstream during subsequent major storms. In this situation, the material collected in debris basins is generated through a spectrum of processes, including surface runoff, flooding, and debris flow. Development of these fan areas must consider the possibility of increasing sediment yield from mountain watersheds due to the frequent occurrence of fire events.

An understanding of key surface erosion and watershed geomorphic processes is essential to application of sediment yield prediction techniques. In particular, the variability of these processes in space and time is important in establishing limitation on the accuracy of estimates derived from sediment discharge data and/or predictive models (USACE, 1995). Although typical sediment yield processes are generally familiar, the interplay of factors that influence sediment yield from a watershed is less obvious and much more difficult to estimate quantitatively. Vegetation play essential role in the sediment yield, but knowledge of biological functions is poorly integrated into procedures for prediction of sediment yields especially under burn conditions. On the geological time scale, the surface of the earth is transformed by sediment production in the upper part of a watershed, transportation of sediments in a fluvial system, and deposition in low-lying lakes, alluvial fans, deltas, and in the oceans (USACE, 1995). Even the sediment production and transport system are still extremely complex, involving the interaction of many hydrologic, geomorphic, and geological processes, having a better understanding of their influence on sediment yield should make a better rewarding study by overcoming the corresponding limitations on sediment yield prediction models.

Fires generally cause water repellency in soil to be temporarily hydrophobic, which effect infiltration, runoff, and erosion in burned watersheds (DeBano, 2000). Several previous studies had shown that wildfire had a powerful influence on the erosion of southern California mountain watersheds (Troxell and Peterson, 1937; Cannon et al., 2003; Middleton et al., 2004). Rowe et al. (1949, 1954) estimated that a 100% burned watershed produces 35 times more sediment yield than in the unburned state.

Debris basins have been constructed in many areas to capture debris flows. The amount of solid materials (including boulders, gravel, sand, silt, clay, trees, etc.) accumulated in debris basins is called sediment (or debris) yields. The yield often excludes fine sand, silts, and clays which pass through the debris basin in suspension. Sediment yield prediction is necessary for debris basin design and can also help determine maintenance needs for debris basin management.

The objective of the present study was to develop accurate models to predict the sequential sediment yields resulting from the wildfire and subsequent storm events. In this study, two different sediment yield prediction models were used and compared. The first model is a statistical model, named the Multi-Sequence Debris Prediction Model (MSDPM). A multiple regression analysis was first used to analyze the measured sediment yield data collected from 1938 to 1983. The equation included variables of precipitation, drainage area, relief ratio, and a nondimensional Fire Factor as well as threshold precipitation factors for rainfall-intensity and total rainfall.

The second model employed Artificial Neural Network (ANN) technique for multi-sequence sediment yield prediction based on the same input data as for the MSDPM but the targets were clean-out values of sediment yield counted from trucks. The study varied network architecture and training patterns including the number of calibration data and input parameters to study their effects on neural networks performance.

The MSDPM and ANN models were calibrated or trained using 17 years of measured sediment yields, wildfire data, and rainfall data collected from 1984 to 2000. For verification of the models, the calibrated models were then applied to subsequent events after 2000 and compared with the measured field data.

Model Development

Existing methods for predicting sediment yield have focused on sediment yields for a single design storm event (Ferrell, 1959; Tatum, 1963; McCuen and Hromadka, 1996; Gatwood et al., 2000; Gindi et al., 2006). The authors believe that new models are needed to predict sediment yield accumulation related to multiple rainfall events within one storm season.

The debris basins used in the analysis are located in the San Gabriel Mountains within Los Angeles County, as shown in Figure 1. Debris cleanout data from 1938 to 2002 were obtained for 80 debris basins owned by the Los Angeles County Department of Public Works (LACDPW) and the Los Angeles District of the United States Army Corps of Engineers (USACE) and were used to develop the model. Debris cleanout data were obtained based on the truck count after excavating all material (clay, silt, sand, gravel, boulders, and organic materials) collected in the debris basin.

Figure 1.

 Location of Debris Basins in Study Area.

Multi-Sequence Debris Prediction Model

The MSDPM was developed for sediment prediction of relatively small watersheds (25-800 ha) (Pak, 2005; Pak and Lee, 2008). Development of multiple regression equation was the first step to provide the fundamental statistical equation of MSDPM. The relief ratio (S), drainage area (A), maximum 1-h rainfall intensity (Im) of each storm event generated based on precipitation data, and Fire Factor (F) were finally selected as independent variables among other meteorologic and physiographic parameters through the stepwise multiple linear regression analysis. In the selected stepwise regression routine, independent variables are progressively added by the program in order of decreasing significance. Variables determined to be significant in earlier stages of the computations may be deleted upon introduction of more significant variables at a later stage. This process allows for determination of the effect of an independent variable on the dependent variable as well as the change in the relative value of this variable upon the inclusion of additional variables (Gatwood et al., 2000).

The MSDPM allows the users to determine the sediment yield based on the several parameters. These include rainfall amount, maximum 1-h rainfall intensity, threshold for maximum 1-h rainfall intensity (TMRI), total minimum rainfall amount (TMRA), relief ratio (S), drainage area (A), antecedent precipitation events, and fire condition. The fire condition is defined on the percentage of the basin area burned, the time since the last fire and the number of antecedent effective precipitation events. These effective events include the number of previous events that generated sediment yield and have precipitation values exceeding the threshold TMRI and TMRA. The MSDPM does not consider the spatial variation of effective rainfall within the watershed. Thus, the current MSDPM is applicable primarily for small watersheds and the accuracy will decrease as the watershed area increases.

Regression analysis on the variables above resulted in the MSDPM equation, Equation (1). Equation (1) shows the relationship between the hydrologic processes described above and sediment yields in a watershed in the method known as the MSDPM

image(1)

where P ≠ Pc and Im ≠ Ic; Dy is the sediment yield (m3); Im is the maximum 1-h rainfall intensity per event, (mm/h); Ic is TMRI (mm/h); P is the total rainfall amount per event (mm); Pc is TMRA, (mm); inline image is absolute value; S is relief ratio (m/km) inline image; h2 is highest elevation in the watershed (m); h1 is lowest elevation in the watershed, (m); L is the maximum stream length (km), measured through Geographic Information System processing based on the digital elevation model; A is the size of drainage area (ha); F is the Fire Factor, 3.0 ≤ F ≤ 6.5 (dimensionless):

image(2)

where Bp is the percentage of burn/100 (0 ≤ Bp ≤1); By is the number of year since burn (1 ≤ By ≤10 year); Ap is the number of antecedent effective precipitation events that have enough energy to generate sediment yield.

The values of TMRI (Ic) and TMRA (Pc) are related to the critical condition for sediment entrainment into flows and the transport capacity to allow sediment to flow to the concentration point. When the hydrodynamic force acting on a particle of sediment has reached a critical or threshold condition, sediment particles are entrained into the flows and moved to the concentration point (debris basin) with required energy. Not all rainfall events can generate sediment yields because some minimum energy is needed to entrain sediment particles. Additional energy is needed to move the sediment to the concentration point. Thus, if Im < Ic, or P < Pc, no sediment yield is generated.

The rainfall events were screened to select the effective rainfall that can provide the required energy through Equation (1). The critical maximum 1-h rainfall intensity for entrainment of sediment particles was determined as the TMRI (Ic) based on the relationship between the TMRI and relief ratio shown in Figure 4a and the critical total rainfall amount for the transport capacity to move sediment to the concentration point was determined as TMRA (Pc) based on the relationship between the TMRA and TMRI shown in Figure 4b for each debris basin through calibration processes, which defined the critical conditions used in MSDPM [Detailed discussion of Ic and Pc was given in Pak (2005) and Pak and Lee (2008)].

Figure 4.

 Regression Equations of TMRI and TMRA for Debris Prediction Using MSDPM; (a) Relationship Between TMRI and Relief Ratio; (b) Relationship Between TMRA and TMRI.

The Fire Factor equation, Equation (2), was developed based on the Fire Factor curve for watersheds in the range of 26 to 777 ha (0.1-3.0 mi2) of Los Angeles District Debris Method by adding effects of antecedent precipitation events. Tatum (1963) developed the Fire Factor curve of Los Angeles District Debris Method using a relationship established by Rowe et al. (1949, 1954), to correlate measured sediment yields and computed sediment yields by means of a single fire curve. The Tatum curve relates the percentage increase in the sediment yield attributable to fire to the elapsed time following wildfire occurrence (Tatum, 1963) and was used as the basis for the preliminary Fire Factor curve examined. This curve assumed that watersheds of unequal size and gradient respond at the same rate over a period of time in terms of sediment yield. Therefore, the space and time invariance of Fire Factor is only assumption, not a reality. The spatial variation generally becomes more and more pronounced as the size of watershed increases; thus, the Tatum curve is limited for relatively small size of watersheds (50-800 ha). This technique acknowledges fire and its associated effects as a major component in a sediment yield estimation method (Gatwood et al., 2000). Data (fire maps or burn severity maps) to calculate the Fire Factor were obtained from the Burned Area Emergency Response (BAER) team, the LACDPW or the Los Angeles District of the USACE for the fire areas.

In this study, a Fire Factor (F) was generated using the percentage of the watershed burned, the number of years since the fire, and the number of antecedent precipitation events above a certain threshold value since the fire based on the additional data collected between 1984 and 2000 (Pak, 2005; Pak and Lee, 2008). The impacts of fire are gradually reduced through watershed by re-vegetation and subsequent storms. Robichaud (2000) stated that hydrophobicity in soils is broken up or is washed away within one to two years after fire. The key to understanding soil recovery after fire is how quickly the bare soil can be covered again by vegetation or litter (Pierson et al., 2001).

The final Fire Factor equation, Equation (2), was calibrated in a manner that minimizes the differences between the measured sediment yields and estimated sediment yields within the main sediment yield equation, Equation (1).

Artificial Neural Network Models

Inspired by the functionality of human brains and nerve cells, McCulloch and Pitts (1943) first introduced the concept of artificial neurons. Since the introduction of computers, use of ANNs has been explored in a variety of scientific fields. However, ANNs were not fully developed until Rumelhart and McClelland (1986) proposed the back-propagation (BP) training algorithm for feed-forward ANNs. Research shows that ANNs have potential to recognize the relationship between inputs and outputs, and utilize new data to generate satisfactory results. ANNs are attractive for modeling complex system due to their impressive capability to perform well even when the training data contain noise and measurement errors (ASCE, 2000a). Numerous applications of ANNs in hydrologic field were reported in the last decade in the modeling of rainfall-runoff relationship, streamflow, and water quality (ASCE, 2000b). Raghuwanshi et al. (2006) applied ANN method for simulating runoff and sediment yields for a small agriculture watershed in India based on rainfall intensity and temperature information. They reported that ANN prediction was more accurate than linear regression models. Rai and Mathur (2008) developed a back propagation feedforward ANN model for the computation of runoff hydrograph and sedimentographs for two small watersheds in the United States. The ANN-based runoff hydrograph models used event data of rainfall intensity and runoff from previous lags up to the travel time as inputs. Based on the rainfall intensity, runoff, and sediment flow rate and their lagged data, another set of ANN models was calibrated and validated by observed sediment flow rates. The authors reported that ANN model results show better agreement than a linear transfer function model (i.e., a time-dependent regression model) for the computation of runoff hydrographs and sedimentographs for both the watersheds.

ANN consists of a parallel processing structure that has artificial neurons (or nodes) and many inter-nodal connections, which are the source of the power of ANN. The most commonly used BP neural network is composed of feed-forward pass (where the information flows from input variables to outputs) and backward pass (during which feedbacks between targets and network estimated values flow backwards to the input layer). A three-layer ANN nodal structure is shown in Figure 2. X1, X2…, and Xn are n input parameters, or one input vector. The input vector is connected to m neurons in the hidden layer through connection weights inline image. The superscript 1 indicates the first hidden layer. The subscripts i and j indicate the connection between an input variable (i = 1, 2,…,n) and a hidden layer neuron (j = 1, 2,…, m). Similarly, inline image is the connection weight between a neuron (j = 1, 2,…, m) in the hidden layer and a neuron (k = 1, 2,…, p) in the output layer. Y1Y2, ..., Yp and inline image, inline image are biases or threshold values. The output values, Yk, are calculated based on the following equation:

Figure 2.

 Schematic Diagram of a Three-Layer ANN With n Input Parameters, m Hidden Neurons, and p Output Parameters.

image(3)

where f1 and f2 are transfer or activation functions. The commonly used transfer functions include the sigmoid function, the hyperbolic tangent function, and linear function. The functions can be formulated mathematically as shown in Equations (4-6), respectively.

image(4)
image(5)
image(6)

Generally speaking, the five aspects that may impact ANN model performance are input and output variable selection, separation of calibration and validation datasets, data preprocessing, ANN architecture, and training algorithm selection. Input variable selection is of prime importance to make the “mapping” from the inputs to the outputs efficient and effective (ASCE, 2000a). Therefore, parameters related to sediment yield such as relief ratio, drainage area, Fire Factor, rainfall intensity, and rainfall amount were selected as the inputs. The entire dataset was divided into two sets. The first set was used for calibration or training, the iterative process of adjusting and optimizing the connection weights and biases to produce network outputs that best approximate the training targets. The performance of the calibrated ANN model was then examined by the second dataset, validation data. Before presenting the calibration dataset to train the ANN models, they were normalized first (i.e., they have zero mean and unity standard deviation). The normalized data were then orthogonalized to be uncorrelated with each other and the input vector dimensions were reduced by applying Principal Component Analysis.

The determination of ANN architecture includes the number of layers, the number of neurons in each layer, initial connection weights and bias values, transfer functions, etc. The optimum number of layers and the number of neurons in each layer were determined after running many different ANN geometries. The initial connection weights and biases were small random numbers. ANN models were trained starting from different initial values to escape the local minimum. The BP algorithm, updates the connection weights and biases based on the sum of squared error (SSE) between the target output and the predicted outputs of the neural network (Hagan et al., 1996). The SSE is expressed in the form

image(7)

In Equation (7), q is the number of the dataset provided to train the ANN model every step, p is the number of output neuron, Yrk is the kth network predicted output for the rth input vector, and Trk is the associated target value. The Levenberg-Marquardt (LM) algorithm, a variation of BP algorithm, converges more quickly and is recommended when no memory problem exists (Hagan et al., 1996). ANN models can be overtrained. In that situation the calibration data error is very small but the error of validation data can be very large. In order to reduce the risk of overtraining or overfitting, our ANN models used Bayesian Regulation BP algorithm for training. The connection weights and biases are updated following LM algorithm and the performance function is a linear combination of squared errors and connection weights.

Calibration and Validation

The Santa Anita II Fire of December 27-31, 1999 burned 229 ha (566 acres). The Lannan Debris Basin, which covered 80% of the terrain within the Santa Anita II Fire area, was chosen based on debris cleanout data availability and high percentage of the watershed burn. After analyzing data from four precipitation gages located in the vicinity of the Santa Anita II Fire area, the Santa Anita Dam precipitation gage was selected because its precipitation data (Maximum 1-h Rainfall Intensity and Total Rainfall Amount) were the most reliable in terms of temporal pattern with measured sediment yield data (debris basin cleanout record) for the analysis of Lannan Debris Basin.

On October 27, 1993, the Kinneloa Fire in Altadena caused extensive damage to the surrounding area and destroyed 121 homes in the foothills of the San Gabriel Mountains (Wildfire Safety Panel, 1994). Watersheds above the 13 debris basins were burned somewhere between 20% and 100%. Data from 9 of the 13 debris basins were collected based on debris cleanout data availability and high percentage of the watershed burn. After analyzing data from four precipitation gages located in the vicinity of the Kinneloa Fire area, the Sierra Madre gage was chosen because its precipitation data (Maximum 1-h Rainfall Intensity and Total Rainfall Amount) appeared to be the most consistent in terms of temporal pattern with measured sediment yield data (debris basin cleanout record) for the nine nearby debris basins burned by the Kinneloa Fire.

The Crest Fire of January 26-28, 1984 burned 15 watersheds. The Big Briar and Hay Debris Basins collect runoff from watersheds that were completely burned by the Crest Fire. Five precipitation gages were located in the vicinity of the Crest Fire area. The Briggs Terrace gage was selected for data analysis because it is the only gage with data covering this event.

After calibration, MSDPM and ANN were used to predict sediment yields for the validation purpose. The results were then compared against the field data of sediment yields resulting from the Mountain Fire, a post-2000 fire event. The Mountain Fire in the Glendale area (occurred in September 9, 2002) burned over 303 ha (749 acres) of the watershed, including the watershed for Brand and Childs Debris Basin, as shown in Figure 3 (Storm Report Los Angeles County 2002-2003). The Childs Canyon Debris Basin precipitation gage was chosen for the data analysis because its precipitation data were the most reliable and consistent when compared with the results of other gages.

Figure 3.

 Debris Basin Locations, Watershed Boundaries, and Mountain Fire Map for Childs and Brand Debris Basins.

Multi-Sequence Debris Prediction Model

The TMRI (Ic) and TMRA (Pc) of each debris basin were determined via the model calibration based on data from three fire events that occurred from 1984 to 2000 (Pak, 2005; Pak and Lee, 2008). The TMRI is related to the watershed slope. Higher slopes can entrain the sediment with less intense precipitation. The relationship between the TMRI (Ic) and relief ratio (S) is depicted in Figure 4a. A power regression was developed to obtain the equation of data trend between the relief ratio (S) and TMRI (Ic), Equation (8), with an R2 value of 0.9435.

image(8)

A power regression was also used to obtain the equation of data trend between the TMRI (Ic) and TMRA (Pc), Equation (9), with an R2 value of 0.9673.

image(9)

The relationship is plotted in Figure 4b. Both equations were obtained with the relatively high R2 values. When using the MSDPM to predict the sediment yields, the relief ratio, drainage area, precipitation data, and fire information were needed as well as the TMRI and TMRA values determined from both Equations (8) and (9), respectively.

For validation purpose, the TMRI (Ic) and the TMRA (Pc) for the Brand Debris Basin and Childs Debris Basin were determined through Equations (8) and (9), respectively.

Artificial Neural Network Models

Three sets of ANN models were developed to demonstrate their ability to estimate sediment yield for the present study. The first set of ANN models was trained by 300 data with five input parameters that included the drainage area, watershed relief ratio, maximum 1-h rainfall intensity, total rainfall amount, and fire factor as defined in Equation (2). These are the same parameters as used in the MSDPM except the logarithmic values of the first two input parameters and sediment yield (target) were utilized in ANN models rather than their original values.

For the second set of ANN models used in the present study, data collected from two debris basins (56 data records) obtained before 1986 was removed from the calibration dataset to check their effect on the neural networks performance with the same input parameters. These older data were thought to be less reliable than the more recent data because they were collected during a transition period when the collection agency changed. In addition to the aforementioned five input parameters, the last set of ANN models included three more input parameters which are the percentage of the area that is burned by wildfire within the watershed, years after the last fire event, and the number of the antecedent effective rainfall events, the number of training samples were kept the same as in the second case.

For comparison, the whole datasets were divided into calibration and validation datasets. The sets were the same as those used for the MSDPM. Before training the ANN models, both the input and target values were normalized. The input vectors were made to be uncorrelated by principal component analysis. Using singular value decomposition, those input variables that contributed less than 2% of the total variation, the sum of the square of the singular value divided by the column number of the calibration dataset minus one, were eliminated (Demuth and Beale, 1998).

For all three different cases, ANN models with one or two hidden layers were used for estimating the sediment yields. For ANNs with one hidden layer, the number of hidden neurons varies from 6 to 14; 22 groups of different geometries for four-layer neural networks were examined and they are 3:1, 3:2, 3:3, 3:4, 4:1, 4:2, 4:3, 4:4, 4:5, 5:1, 5:2, 5:3, 5:4, 5:5, 5:6, 6:1, 6:2, 6:3, 6:4, 6:5, 6:6, and 6:7 (the first number indicates the number of the neuron in the first hidden layer, and the second number is the number of the neuron in the second hidden layer). The Bayesian Regulation BP algorithm was chosen as the training algorithm in this study. It is noted that connection weights and biases were changed after the total calibration dataset presented to the neural networks. For all three set of ANN models utilized in this study, the transfer function for the output layer is linear, and the hyperbolic tangent function is applied to hidden layers. Maier and Dandy’s study (1998) showed that results obtained using networks with hyperbolic tangent transfer function is slightly better, 2% difference in terms of root mean squared error (RMSE), than the results using logistic (Equation 4) transfer function. The training process is stopped, whenever one of the following four criteria is reached: (1) the specified epoch size (the number of training sessions), (2) the desired accuracy for the training data, (3) the minimum gradient of the error such as 1e–10, or (4) the specified maximum execution time.

To evaluate the performance of the ANN models, for both the training and the testing dataset, mean squared error (MSE) (Equation 10), a linear regression analysis between the network outputs and the corresponding targets, and the percentage difference between the measured and the ANN predicted sediment yields are used. The MSE is given by

image(10)

Every parameter is same as what is defined in Equation (7). Evaluation of network performance considers three elements: first, the MSEs for both calibration and validation datasets; second, absolute difference between estimated and actual sediment yield. The correlation coefficient and the value of the slope are the best linear regression line. In general, the closer the correlation coefficient and the value of the slope are to unity, the more accurate is the model prediction.

Results and Discussion

Once the model was calibrated and validated, the results obtained from MSDPM are compared with ANN outputs. For the first case, the measured and estimated sediment yields by MSDPM and the best ANN model were compared. Table 1 shows the differences between the models. As seen from Table 1, the MSDPM predicts sediment yields within a range of a difference from −7.5% to 32.8% for the training dataset compared with the field data. The ANN model with the best performance has two hidden layers and six neurons in the first hidden layer and two neurons in the second hidden layer. The predicted sediment yields by this ANN model are within the range of [−4.7%, 10.0%] in terms of training data. It is seen that the ANN model prediction has smaller error bond for the calibration dataset.

Table 1.   Summary of Calibration and Validation Results by Two Models for Case 1.
 Debris Basin S (m/km)Area (ha) Fire Event*% of BurnMeasured Dy (m3)MSDPM Estimated Dy (m3) Difference** (%)ANN Estimated Dy (m3) Difference*** (%)
  1. *Fire event designation: 1-Santa Anita II Fire, 2-Kinneloa Fire, 3-Mountain Fire.

  2. **Difference between measured sediment yield and the estimated valued using MSDPM.

  3. ***Difference between measured sediment yield and the estimated valued using ANN model.

Calibration (Training) DataLannan Case 1405.00463.94318013,57713,480−0.713,340−1.7
Lannan Case 2405.00463.9431805,0475,5269.55,113−1.3
Kinneloa East444.03351.802210023,62722,005−6.923,474−0.6
Kinneloa West475.83852.206210033,26130,751−7.532,383−2.6
Rubio280.060329.0229217,00117,8304.917,0380.2
Bailey337.072153.7929522,94823,5012.422,9650.1
Sunnyside475.8025.21321001,2391,2823.41,181−4.7
Carriage House433.9937.6892901,7102,00717.41,8246.7
Auburn521.71241.2792788,3648,9426.98,6042.9
Fairoaks60.01354.6352411,8471,748−5.41,8630.9
West Ravine286.76163.9432699,33111,27620.89,7064.0
Big Brial509.8695.261310055273332.85886.6
Hay352.74352.20631004,4845,75928.44,93310.0
Validation DataBrand28026739081,35875,935−6.769,401−14.7
Childs3148138022,24920,355−8.521,251−4.5

After calibration, both models were tested by the validation dataset collected from the Brand Debris Basin and the Childs Debris Basin generated with rainfall events between November 8, 2002 and April 2, 2003. For the Brand Debris Basin, the measured sediment yield was 81,358 m3, while the estimated sediment yield by the MSDPM was 75,935 m3 and it was 69,401 m3 by the ANN model. The difference of the estimated and measured values was 5,423 m3 (−6.7%) for the MSDPM, and 11,957 m3 (−14.7%) by the neural network. It is obvious that this sediment yield is much larger than any of the sediment yields used for training. For the Childs Debris Basin, the measured sediment yield was 22,249 m3, while the estimated sediment yield was 20,355 m3 by the MSDPM which underestimated the sediment yield with a magnitude of 1,894 m3 or −8.5%. ANN model prediction was 21,251 m3 which is 998 m3 (−4.5%) lower than the measured value. Both models underestimated sediment yields for the validation data and especially for the Brand Debris Basin by the ANN model. Figure 5 shows the linear regression results for both calibration and validation datasets. The solid line is the best linear regression line based on the data points whose x-value is the measured sediment yield, and the y-value is the predicted values by ANN model. The dashed 45° line is the line for perfect fit.

Figure 5.

 Linear Regression Analysis of Measured and ANN Model (5,6,2,1) Estimated Sediment Yield; (a) Calibration Dataset; (b) Validation Dataset.

For the second set of ANN models, the data at Big Briar Debris Basin and Hay Debris Basin were excluded for training (these data were obtained before 1986). Results of second best-performed ANN model compared with the field data and MSDPM statistical model is shown in Table 2. The optimal ANN model has three and four neurons in the first and second hidden layer, respectively. The difference between the measured and the ANN predicted sediment yields for calibration data ranges from −4.2% to 16.0% while the minimum MSDPM prediction error is −7.5% and the maximum error is 20.8%. The ANN model performance for validation data improved dramatically. The predicted sediment yield for the Brand Debris Basin from November 8, 2002 to April 2, 2003 is 85,694 m3 which is 5.3% greater than the measured value. The ANN model predicted sediment yield was 23,152 m3 for the Childs Debris Basin which is 4.1% greater than the measured value.

Table 2.   Summary of Calibration and Validation Results by Two Models for Case 2.
 Debris Basin S (m/km)Area (ha) Fire Event*% of BurnMeasured Dy (m3)MSDPM Estimated Dy (m3) Difference** (%)ANN Estimated Dy (m3) Difference*** (%)
  1. *Fire event designation: 1-Santa Anita II Fire, 2-Kinneloa Fire, 3-Mountain Fire.

  2. **Difference between measured sediment yield and the estimated valued using MSDPM.

  3. ***Difference between measured sediment yield and the estimated valued using ANN model.

Calibration (Training) DataLannan Case 1405.00463.94318013,57713,480−0.713,559−0.1
Lannan Case 2405.00463.9431805,0475,5269.55,5229.4
Kinneloa East444.03351.802210023,62722,005−6.922,668−4.1
Kinneloa West475.83852.206210033,26130,751−7.531,858−4.2
Rubio280.060329.0229217,00117,8304.917,2101.2
Bailey337.072153.7929522,94823,5012.422,772−0.8
Sunnyside475.8025.212521001,2391,2823.41,214−2.0
Carriage House433.9937.68932901,7102,00717.41,8397.6
Auburn521.71241.2792788,3648,9426.98,5241.9
Fairoaks60.01354.6352411,8471,748−5.41,8661.0
West Ravine286.76163.9432699,33111,27620.810,82016.0
Validation DataBrand28026739081,35875,935−6.785,6945.3
Childs3148138022,24920,355−8.523,1524.1

As presented in Table 3, the MSE for the validation data decreased substantially. This can be explained by a statistical goodness-of-fit index introduced by Gupta and Sorooshian (1985). The index is the ratio of the standard error estimate (Se), shown in Equation (11), to the standard deviation (Sy) of the target values.

Table 3.   Comparison of ANN Models Performance.
Network GeometryCalibration DatasetValidation Dataset
MSERSlopeSe/SyMSERSlopeSe/Sy
1st ANN: (5,6,2,1)0.0001951.0001.0000.012160.0270360.9960.9720.03033
2nd ANN: (5,3,4,1)0.0002431.0001.0000.012690.0003451.0001.0040.00376
3rd ANN: (8,3,2,1)0.0000851.0001.0000.007490.0007421.0000.9890.00550
image(11)

where λ is the degrees of freedom and is defined as the number of calibration data minus the number of connection weights and biases, and the rest of the parameters are the same as before. The smaller the ratio, the more accurate is the model prediction. The ratio of the neural network (5,3,4,1) trained by 244 data is 0.01269 for the calibration data which is very close to the ratio for the ANN model (5,6,2,1) trained by 300 data, 0.01216. However, its ratio for the validation data, 0.00376, is much smaller than the ratio, 0.03033 obtained in the first case.

Results provided by the MSDPM and the best ANN model (5,3,4,1) are compared with the data from field measurement for Brand Debris Basin and Childs Debris Basin. These are presented in Figures 6a and 6b. Results from these two models compared remarkably well with the field data. Included in Figures 6a and 6b are the curves showing the remaining debris basin capacity, which are useful for the planning of clean-out operation of debris basins. The remaining debris basin capacity shown in Figure 6 was calculated based on the MSDPM results and maximum debris basin capacity.

Figure 6.

 Comparison With Estimated Sediment Yields Using MSDPM and ANN and Measured Sediment Yields for (a) Brand Debris Basin and (b) Childs Debris Basin.

The last set of ANN models were trained by eight input parameters including three more input parameters such as the percentage of burned area, years after the last fire event, and the number of the antecedent effective rainfall events. From the results we found that the error bond in calibration data is also reduced but not so much for the validation data (Table 3). In this case, the neural network with three and two neurons in the first and second hidden layer showed the best result. When one considers the overall results for both calibration and validation data, the modeling accuracy of ANN model (8,3,2,1) is similarly good as the second ANN model (5,3,4,1). This is reasonable because the three new input parameters have already been considered in the fire factor (Equation 2) in second ANN model.

The MSEs and the correlation coefficients for all developed ANN models with different number of training patterns and input parameters are summarized in Table 4. For three-layer ANN models, there appears to be an improvement on the modeling accuracy with an increase in the number of hidden neurons up to a certain point. For example, the optimum performance was achieved when 13 neurons (5,13,1) were used in the hidden layer for the first case, 11 neurons (5,11,1) during the second case, and 14 neurons (8,14,1) for the last case. The results indicated that the optimal number of neurons in the hidden layer is not only a function of the number of input parameters but also a function of the number of training samples.

Table 4.   The Summary of Different ANN Models Performance.
1st ANN Geometry (300 data)Calibration DatasetValidation Dataset2nd ANN Geometry (244 data)Calibration DatasetValidation Dataset3rd ANN Geometry (244 data)Calibration DatasetValidation Dataset
MSERMSERMSERMSERMSERMSER
  1. Note: Bold number indicates the optimum ANN geometry for each study case.

5,6,10.2225450.9280.3327500.9525,6,10.1803430.9480.3551650.9568,6,10.1820340.9480.2777460.965
5,7,10.1771200.9430.2537680.9635,7,10.2124440.9390.4243770.9438,7,10.1719340.9510.2769520.961
5,8,10.1228670.9610.2356360.9955,8,10.1007520.9710.2482890.9858,8,10.1638100.9530.2858200.959
5,9,10.1058670.9660.2986320.9645,9,10.1520030.9570.2426720.9688,9,10.1043410.9700.2840180.963
5,10,10.0800440.9750.3880350.9575,10,10.1825560.9480.2838860.9608,10,10.0318440.9910.3707580.963
5,11,10.0948680.9700.4778100.9345,11,10.0374510.9890.2864200.9618,11,10.0924260.9740.2952200.959
5,12,10.0485970.9850.3893830.9465,12,10.0814530.9770.3790430.9448,12,10.1311160.9630.2874470.958
5,13,10.0492940.9840.2177860.9815,13,10.1687600.9520.3839060.9478,13,10.0932330.9740.4806440.943
5,14,10.0550930.9830.3977280.9485,14,10.0974340.9720.2648530.9648,14,10.0196160.9950.3356820.954
5,3,1,10.0699180.9780.1116340.9965,3,1,10.0120970.9970.0887070.9968,3,1,10.0004271.0000.0045011.000
5,3,2,10.0662190.9580.1799670.9925,3,2,10.0529060.9850.0231010.9998,3,2,10.000085171.0000.0007421.000
5,3,3,10.0436840.9860.0471060.9965,3,3,10.0001311.0000.0010831.0008,3,3,10.000080981.0000.0010721.000
5,3,4,10.0275100.9910.0502180.9965,3,4,10.0002431.0000.0003451.0008,3,4,10.000267251.0000.000921.000
5,4,1,10.0058470.9980.0887810.9985,4,1,10.0063830.9980.0851880.9968,4,1,10.001913140.9990.0448940.998
5,4,2,10.0196260.9940.0573990.9975,4,2,10.0017671.0000.0333790.9978,4,2,10.005481750.9980.0732550.996
5,4,3,10.0040730.9990.1070690.9955,4,3,10.0000331.0000.0010821.0008,4,4,10.000003171.0000.0046481.000
5,4,4,10.0011161.0000.0168080.9985,4,4,10.0000691.0000.0016651.0008,4,5,10.000071981.0000.0515530.993
5,4,5,10.0005051.0000.0482750.9935,4,5,10.0000321.0000.0015841.0008,5,1,10.004678470.9990.0705490.996
5,5,1,10.0003701.0000.0083031.0005,5,1,10.0004671.0000.0232230.9988,5,2,10.000055931.0000.0231790.997
5,5,2,10.0013081.0000.0310800.9975,5,2,10.0000981.0000.0008321.0008,5,3,10.000162621.0000.0295160.997
5,5,3,10.0001431.0000.0551980.9955,5,3,10.0000461.0000.0037671.0008,5,4,10.000002731.0000.0392440.997
5,5,4,10.0000031.0000.0120380.9985,5,4,10.0000011.0000.0058370.9998,5,5,10.000023501.0000.0133710.998
5,5,5,10.0000251.0000.0379460.9955,5,5,10.0000011.0000.0094671.0008,5,6,10.000000011.0000.0693860.998
5,5,6,10.0001621.0000.0265780.9985,5,6,10.0000031.0000.0212690.9988,6,1,10.000010671.0000.0216990.997
5,6,1,10.0013851.0000.0339970.9995,6,1,10.0011881.0000.0313960.9998,6,2,10.000021321.0000.0341850.995
5,6,2,10.0001951.0000.0270360.9965,6,2,10.0000111.0000.0072221.0008,6,3,10.000001601.0000.0390460.997
5,6,3,10.0000681.0000.0336230.9955,6,3,10.0000001.0000.0559990.9938,6,4,10.000000011.0000.0467811.000
5,6,4,10.0000021.0000.0408530.9965,6,4,10.0000001.0000.0177180.9998,6,5,10.000000081.0000.0992390.986
5,6,5,10.0000181.0000.1060790.9885,6,5,10.0000001.0000.0131210.9998,6,6,10.000000011.0000.0153790.999
5,6,6,10.0000011.0000.0221220.9975,6,6,10.0000001.0000.0173640.9978,6,7,10.000000011.0000.0252550.999
5,6,7,10.0000001.0000.0266120.9965,6,7,10.0000001.0000.0140850.999     

As shown in Table 4, the ANN models with two hidden layers have better generalization ability than the neural networks with one hidden layer. Although there is some variation in the generalization ability of the networks with two hidden layers, referring to Figure 7, there appears to be such a trend that increasing the number of the neurons in the hidden layers improved the performance for the calibration data. However, there is no such trend for the validation data. In addition it can be seen that as continued training with more neurons in the hidden layers, the prediction accuracy of the validation data is worsened, a sign of overfitting. In the first series of test, the ratio of the number of calibration data to the number of connection weights and biases is 5.66, as compared to 6.26 and 6.42, the ratios for the best fit neural network during the second and third test. It suggests that higher ratio of the number of calibration data to the number of connection weights and biases does not lead to a better simulation.

Figure 7.

 Calibration and Validation Mean Square Error (MSE) for (a) Case 1 (b) Case 3.

As mentioned previously, ANN performance deteriorates when the validation data are out of the range of the calibration data. It is not the case in this study because the sediment yield collected from Brand Debris basin are roughly 2.4 times greater that the maximum sediment yields used for training, the best prediction of sediment yield achieved in the first set of ANN models are not accurate enough, but there is an ANN model within the second case that provided desirable estimations.

Conclusions

The MSDPM and ANN models were developed in the present study for the prediction of sediment yields for subbasins in the San Gabriel Mountains within Los Angeles County. The present research advances sediment yield prediction from a different perspective as most existing methods were developed for a single storm event or average annual yields. The modeling results suggest that both the MSDPM and ANN models can be used to predict the accumulated sediment yields with consistency and reliability for coastal southern California watersheds with an area in the range of 25-800 ha. The developed ANN models simulate sediment yields as a function of watershed area, relief ratio, total rainfall amount and intensity, and fire factors instead of only rainfall intensity and temperature information (Raghuwanshi et al., 2006; Lee et al., 2006). Both MSDPM and ANN models are viable tools for the effective management of existing debris basins, to determine whether they should be excavated immediately to regain storage capacity based on the remaining debris basin capacity before subsequent storms occur. This will enable operators to have more control in scheduling cleanout operation for debris basins.

The MSDPM and ANN models can be used for responding to the emergency situations to protect human lives and to lessen the risks of economic damage by predicting more accurately the postfire sediment yields based on the remaining debris basin capacity based on national weather service forecast information. This will also allow operators to develop a more effective rapid response strategy in emergency situations based on the remaining debris basin capacity.

After the removal of some training samples, an ANN model is capable of predicting sediment yields with higher accuracy when compared with the MSDPM statistical model. In other words, more training samples do not necessarily improve neural network modeling efficiency. Therefore, the selection of calibration and validation data plays a very important role in the neural network modeling success. This study also shows that the addition of more input parameters does not enhance the neural network performance. In addition, the results reinforced the theory that the smaller the ratio between the standard error estimate (Se) and the standard deviation (Sy) of the target values, the better is the prediction accuracy.

From this study, the neural networks with two hidden layers have much better modeling capability than the neural network with a single hidden layer and the training time is acceptable. It seems that the ANN models with two hidden layers achieved higher accuracy for the calibration data with increasing the number of the hidden neurons. However, one must guard against overfitting.

Expanded use of the model to other areas should be verified with additional sediment measurements in order that this model may be applied with confidence for a wider range of conditions in engineering application.

Acknowledgments

The authors are thankful to Kerry Casey and Gregory Peacock of the Hydrology and Hydraulic Branch of the U.S. Army Corps of Engineers for their helpful comments. Dr. Iraj Nasseri, Dr. Youn Sim, Loreto Soriano, Adam Walden, John Burton, and Ben Willardson of the Water Resources Division at the Department of Public Works of Los Angeles County have provided useful data and helpful comments. We are grateful for their help.

Ancillary