Electronic sensors generate valuable streams of forcing and validation data for hydrologic models but are often subject to noise which must be removed as part of model input and testing database development. We developed an automated precipitation correction program (APCP) for weighing bucket precipitation gauge records, which are subject to several types of mechanical and electronic noise and discontinuities, including gauge maintenance, missing data, wind vibration, and sensor drift. Corrected cumulative water year precipitation from APCP did not exhibit an error bias and matched measured water year total precipitation within 2.1% for 58 station years tested. Removal of low-amplitude periodic noise was especially important for developing accurate instantaneous precipitation records at subdaily time steps. Model flexibility for use with other data types is demonstrated through application to time domain reflectometry soil moisture content data, which are also frequently subject to substantial noise.
If you can't find a tool you're looking for, please click the link at the top of the page to "Go to old article view". Alternatively, view our Knowledge Base articles for additional help. Your feedback is important to us, so please let us know if you have comments or ideas for improvement.
 Continuous, accurate data for precipitation and soil moisture are critical as inputs and validation data for hydrological models. Many hydrological processes occur at subdaily time steps. Modeling these processes requires accurate data at the time step of the model [Haddeland et al., 2006]. Manually filtering such data is tedious, subjective, and time-consuming work. In this note we describe an automated program that corrects mechanical errors and noise typical of electronically recorded, weighing bucket-type precipitation gauges. Extension of the program to electronic data from other instruments is also demonstrated using time domain reflectometry (TDR) data.
 The weighing bucket gauge is commonly used in environments receiving precipitation both as rain and snow. Weighing bucket precipitation gauge data are intrinsically cumulative. To process time series precipitation values from cumulative data, it is necessary to derive unbiased instantaneous differences from the cumulative record. Although common, mechanical errors associated with weighing bucket gauges have received only passing attention as part of comparative studies [Nystuen, 1999; Duchon and Essenberg, 2001].
 We classify several types of mechanical errors present in weighing bucket gauge data as either high- or low-amplitude noise. High-amplitude noise can be caused by out of range data values, bucket decanting, bucket recharge, and intermittent noise. Out of range data can arise during periods of instrument or data logger failure and be recorded as null or extreme negative values, resulting in discontinuities in the precipitation record (Figure 1a). During instrument servicing the gauge bucket is decanted and then recharged with mineral oil and antifreeze. Bucket decanting introduces large instantaneous changes in weight that are generally negative (Figure 1b), but can be positive depending on the mass of liquid with which the bucket is recharged (Figure 1c). Occasionally, the gauge record is subject to large instantaneous changes from intermittent electronic noise (Figure 1d). Low-amplitude noise may be periodic (Figure 1e), due to the effects of temperature fluctuations on instrument electronics, or episodic (Figure 1f), due to wind vibration of the gauge [Hanson et al., 2001; Hanson et al., 1979]. Although low-amplitude noise does not affect long-term aggregate precipitation measurements, it is difficult to manually separate from subdaily precipitation data and can obscure the beginning and end of precipitation events.
2. Program Description
 The automated precipitation correction program (APCP) utility was developed in Visual Basic 6.0 to process high-frequency cumulative precipitation records, fill data gaps, remove discontinuous data and filter mechanical and electronic noise (see auxiliary material). APCP compares the difference in consecutive records to user-defined limits in two separate cycles to successively remove high-amplitude then low-amplitude noise. User-defined parameters and value ranges used in the examples are provided in Table 1. Both BucketDecanting and BucketRecharge limits are set smaller than the minimum absolute change in data records because of bucket decanting and bucket recharge, respectively. Similarly, the Noise limit is set smaller than BucketDecanting and BucketRecharge but greater than the maximum low-amplitude noise.
Table 1. User-Defined Parameters Used in APCP Program and Values Used to Process Example Data
Value Range for Precipitation (mm)
Values for TDR (m3 m−3)
Bucket decanting limit
Bucket recharge limit
Threshold of high-magnitude noise
Out of range value
3. Test Data
 The U.S. Department of Agriculture Agricultural Research Service Northwest Watershed Research Center (ARS-NWRC) operates a network of precipitation gauges in Reynolds Creek Experimental Watershed (RCEW). Each measurement location has one unshielded and one shielded Belfort universal recording gauge. The gauges have an orifice diameter of 203 mm, 305 mm capacity [Hanson et al., 2001; Hanson, 2001] and an absolute sensitivity of ±0.25 mm [Kuligowski, 1997]. Precipitation depth in the gauge collection bucket is measured at 15 min intervals by load cell and recorded electronically on a data logger. The APCP correction technique is applied to 22 dual gauge stations in RCEW for water years 1997–2005 (October through September). The gauge sites range in mean annual precipitation from 236 to 1123 mm [Hanson, 2001].
 A long experience with weighing bucket gauge data has led the NWRC to develop extensive quality control protocols and quality analysis techniques. Through 2004, the NWRC used the graphical tool “Rainfall Analyzer” (RA) to manually filter, correct and process raw precipitation data. This method requires 1–3 days per station year and results are subject to operator bias. Nevertheless, to our knowledge RA is the most sophisticated data filter available for weighing type bucket gauges and we used data processed with the RA to evaluate the effectiveness of ACPC. Total gauge catch was added as a supplemental measure of quality analysis. This value is a direct volumetric measurement of gauge catch, recorded during gauge maintenance and summed annually. Corrected data from APCP and RA are compared to total annual gauge catch for eleven dual gauge stations for water years 2002–2004.
 As a demonstration of the capability of APCP to remove random noise from other data, we applied the program to TDR data. The data were collected in the Dry Creek Experimental Watershed near Boise, Idaho, in coarse-loamy, mixed mesic Ultic Haploxerolls [Harkness, 1997; McNamara et al., 2005]. The TDR waveguides were 30 cm in length and logged hourly using TDR100, coaxial multiplexers and CR10X data loggers (Campbell Scientific, Logan, Utah). Data filtering was performed with the first cycle of APCP only, since these data are not cumulative. Contemporaneous data from a proximal water content reflectometer (WCR, Campbell Scientific, Logan, Utah) which was previously calibrated with TDR [Chandler et al., 2004] is provided for comparison.
4. Illustration of Method
 The goal of data processing is noise and error removal without introducing bias. To demonstrate the function of APCP for each defined category of mechanical error, we present example comparisons of raw and APCP processed data at the scale of each error (Figure 1). The error categories “Out of Range Data,” “Bucket Decanting,” “Bucket Recharge” and “Intermittent Noise” occur as instances or constant value shifts from the “true” baseline. These errors are simply corrected relative to local baseline cumulative precipitation by ACPC cycle 1. Establishing a local baseline is more complicated over the temporal scale of “Periodic Noise,” which is by definition a quasi-regular waveform around the expected value, and for “Episodic Noise,” which superimposes random noise onto the diurnal signal of “Periodic Noise” (Figures 1e and 1f). The uncertainty in baseline cumulative precipitation during Periodic and Episodic Noise, as complicated by coincident high-magnitude noise and precipitation, is the greatest potential source of error and bias in data correction for cumulative gauge records.
 In the case of cumulative precipitation, the match between processed and raw cumulative gauge catch data is a qualitative measure of the accuracy of APCP. Such a comparison requires developing a cumulative record from raw data by correction of negative steps in the measured record from bucket decanting, for instance by APCP cycle 1. Figure 2 presents a comparison among: the uncorrected 15 min interval data from a shielded gauge, the cumulative record following manual correction for bucket decanting and recharge, the APCP output, and RA output. All three correction approaches maintain the basic structure of the time series data and match the incremental total catch at the annual time scale. However, at subdaily time scale, the cumulative records for manual correction, RA and APCP often diverge at the millimeter scale (Figure 2, inset) because of differences in the approach to processing periodic and episodic noise.
 We tested for bias between annual total gauge catch and annual cumulative corrected precipitation records from both APCP and RA for 58 station years (Figure 3). APCP corrected annual total precipitation is consistently close to the annual total gauge catch. The mean difference and standard deviation were −0.6 mm (−0.1%) and 3.4 mm (0.6%), respectively, for shielded gauge measurements and −0.2 mm (−0.02%) and 2.1 mm (0.5%), respectively, for unshielded gauge measurements. In contrast, RA results show a mean difference and standard deviation of −2.7 mm (−0.5%) and 11.3 mm (2.1%), respectively, for shielded gauge measurements and −1.0 mm (0.4%) and 12.9 mm (3.4%), respectively, for unshielded gauge measurements. Because APCP is not affected by operator bias, corrected water year precipitation data may be replicated within minutes by different operators. Depending on the extent of noise, manual data processing may require 1–3 days per station year and output records may differ among operators.
 As an example of the temporal-scale dependence of the contribution of low-amplitude noise to error in the instantaneous precipitation record, we compare distributions for daily, hourly, and 15 min time steps for a single station year (Figure 4). Once again, APCP cycle 1 is used to remove all major noise and cycle 2 is then used to filter the low-amplitude noise. In general, reducing the time step increases the number of periods with precipitation, reduces depth per event, and increases the number of periods with noise-induced apparent “negative” precipitation. Excluding the nil instantaneous precipitation data, which were most frequent for all time steps, two clear trends emerge. First, the instances of precipitation represented by cycle 1 exceed those by cycle 2 at all time steps, indicating the importance of low-amplitude noise. Second, the number of representations of negative precipitation increases with decreasing time step for scan 1 (Figure 4). Whereas the positive and negative instantaneous precipitation values balance in the annual record at all time steps, manual correction of erroneous instantaneous precipitation values is clearly impractical for hourly and 15 min records because of the exponential increase in number of errors as time step decreases (Table 2). The absence of negative values following scan 2 at any time step demonstrates that APCP is robust across time steps.
Table 2. Instantaneous Precipitation Occurrence Frequency and Cumulative Annual Depth for Output From APCP Cycle 1 and Cycle 2 at Daily, Hourly, and 15 min Time Steps
 APCP does not eliminate the requirement for careful observation of raw data and field notes. APCP may not remove all noise satisfactorily when the amplitude of noise caused by site maintenance is very small. In this case it is necessary to remove these errors from the raw precipitation record before using APCP. It is also recommended that the processed data generated by APCP be checked by comparison with total annual gauge catch and visual inspection of raw data to verify that all noise and discontinuities have been removed satisfactorily.
 Application of APCP to hourly TDR data is demonstrated in Figure 5. In this case, we assume that instantaneous changes in the hourly water content record greater than 0.025 m3 m−3 in absolute value are associated with noise. This approach identified 459 data records as noise and adjusted them to the local average. Of the records identified as noise, 70 were greater than 0.4 m3 m−3, 83 were less than the apparent residual moisture content of 0.045 m3 m−3 (29 were negative). Of the 306 adjusted data values within the possible range of soil moisture (0.045–0.40 m3 m−3), 128 were greater than the apparent field capacity (0.24 m3 m−3). The remaining 178 identified errors were random changes greater than the assumed noise limit within the expected range of soil moisture, for example the spikes near Julian day 160 (Figure 5). The corrected TDR record is comparable to the calibrated WCR record, which tends not to be subject to similar noise problems at this site.
 The APCP utility was developed to extract continuous, high-quality subdaily time step data from electronic records subject to several types of noise and errors. The method was applied successfully to weighing bucket precipitation records and TDR soil moisture records. The precipitation data processed by APCP has less bias from total gauge catch measurements and requires significantly less time than the graphical RA approach. The program capacity for objective removal of low-amplitude periodic noise is extremely useful for calculating accurate subdaily time step data from cumulative precipitation records. The success of the APCP correction of TDR data indicates further that the model can be used to filter electronic data from other sensors important to hydrologic science.
 The authors would like to thank Steve Van Vactor for graciously providing the precipitation data from RCEW, expert opinion, and guidance in developing the precipitation correction algorithm. This research was supported by USDA-CSREES SRGP award 2005-34552-15828, USDA ARS Northwest Watershed Research Center, and the Utah Agricultural Experiment Station, Utah State University, Logan, Utah, approved as journal paper number 7800. Any reference to specific equipment types or manufacturers is for information purposes and does not represent a product endorsement.