Here we introduce a statistical modeling approach that uses nonparametric regression to estimate smooth trends from time series data. The nonparametric regression uses a set of optimal thin plate splines to represent the trends and can be used to make formal inference (e.g., calculate confidence and prediction intervals). As discussed in the Introduction, the approach adopted here consists of three distinct steps: estimation of individual model trends (IMT), baseline adjustment of the trends, and the weighted combination of the adjusted individual model trends to produce a multimodel trend (MMT) estimate. In this section the development and application of this approach will be illustrated using the time series of column ozone data presented in Figure 1. These data correspond to the CCMVal-1 raw time series analyzed by Eyring et al. [2007, Figure 7]. Several models participating in CCMVal-1 provided two overlapping time series of column ozone to cover the maximum range of the REF-A2 period (1980–2100): one from the REF-A2 experiment and one from the climate-of-the-20th-century experiment REF-A1. This additional complication is accounted for in the TSAM approach by considering these partially overlapping time series as ensemble members.
2.1. Nonparametric Estimation of the Individual Model Trends
 The time series yjk(t) of an ozone-related index, such as one of those displayed in Figure 1, is additively modeled as the sum of a smooth unknown model-dependent trend, hj(t), and irregular normally distributed noise:
where the noise field
is assumed to be an independent normally distributed random variable with zero mean and variance σ2, and the indices j and k, respectively, represent model and ensemble-member number. (Here the ensemble index k extends over both REF-A1 and REF-A2 simulations for some models.) This is a nonparametric regression of the time series on time. The regression is nonparametric because the function of time does not have a fixed functional form with explicit parameters. The noise term (2), representing natural variability about the trend, is considered to be an independent normally distributed random variable: independent between different times, models, and runs. The variance of the noise is assumed to be constant over all models and runs. By fitting the trend to all the data rather than to each model separately one can obtain better estimates of the noise variance (referred to as “borrowing strength”).
 The unknown smooth functions hj(t) are estimated by fitting the data to a finite set of smooth basis functions having optimal interpolating properties. This was done here by using the generalized additive model gam() function in the mgcv library of the R language [R Development Core Team, 2008; Woods, 2006]. The default option was used which fits the data to a set of thin plate regression splines, by maximizing penalized likelihood to find the coefficients multiplying the basis functions. The smoothness of the basis functions is controlled by a smoothing parameter, which is chosen using a leave-one-out generalized cross-validation prediction approach (see Woods  for more details). Unlike iterated 1:2:1 Lanczos filter smoothing, typically used on CCMVal-1 data [e.g., Eyring et al., 2007], the thin plate splines are guaranteed to give smooth trend estimates and do not alter their properties at the ends of the series.
 The first step in the TSAM approach is to apply the nonparametric regression (1) to the raw time series data. This is illustrated in Figures 2a and 2b by the IMT estimates hj(t) of the CCMVal-1 March 60°N–90°N and October 60°S–90°S total column ozone displayed in Figure 1. (Note that, while the smooth trend estimates hj(t) extend over the full period (1950–2100) in Figures 2a and 2b, we have elected to display the hj(t) only over the period where data exists for each model.)
Figure 2. (a and b) Initial estimate of the individual model trends hj(t) for the raw time series displayed in Figure 1. This represents the first step in the TSAM approach. (c and d) The 1980 baseline-adjusted time series data y′jk following from (7) with t0 = 1980. (e and f) The 1980 baseline-adjusted trend estimate h′j(t). This represents the second step in the TSAM approach. The thick gray line in Figures 2c and 2d represents the trend estimate g'(t) for the simpler nonparametric additive model (8). For reference, following Eyring et al. , smooth fits to the observations in these plots have been created by 30 iterations of a 1:2:1 filter (black lines).
Download figure to PowerPoint
2.2. Baseline Adjustment of the Trend Estimates
 The initial IMT estimates hj(t) in Figures 2a and 2b reveal significant differences in the background values of column ozone, particularly in the Arctic (Figure 2a). To facilitate a comparison of the trends across models, anomaly time series are constructed relative to a pre–ozone-hole baseline value of the index. While this is analogous to the procedure employed by Eyring et al. , the smoothness of hj(t) allows a more robust definition of the baseline at a particular time t0 (i.e., hj(t0)), rather than from the average over some period about t0. This results in the anomaly time series:
By construction, the anomaly time series (3) is centered on a baseline value of zero at the time t0. Here we chose to have this baseline changed from zero to the multimodel mean of hj(t0) resulting in the “t0 baseline-adjusted time series”:
where J is the total number of models. Since the multimodel average of the IMT estimates h(t0) is a close approximation to the final multimodel trend estimate (MMT) derived in the third step of the TSAM approach, the baseline adjustment may be viewed simply as forcing the anomaly time series to go roughly through the final MMT estimate at the reference date t0.
 The time series (4) contains all the information of (3) plus the multimodel average h(t0), which can be compared with observations. We will used the baseline t0 = 1980 since a number of the CCMVal-1 models do not have data prior to this date. Following (4), the 1980 baseline-adjusted time series, y′jk for the CCMVal-1 March 60°N–90°N and October 60°S–90°S total column ozone are displayed in Figures 2c and 2d, respectively. The corresponding 1980 baseline-adjusted nonparametric IMT estimates h′j(t) are presented in Figures 2e and 2f. Following (1) and (4) the 1980 baseline-adjusted nonparametric smooth trend in our model is:
Before moving on to the third step in the TSAM, we may ask if the statistical model in (7) is well specified. In other words, are all its model assumptions satisfied in modeling the data, for example, that the noise term εjk (t) is independent from year to year, is normally distributed, and is drawn from the same underlying distribution with zero mean and similar variance. For example, we could have chosen the simpler nonparametric model:
where one trend estimate is made for all time series data instead of individual trend estimates for each model (7). This implicitly defines a different random noise component jk(t). The nonparametric trend estimate g′(t) is displayed as the thick grey line in Figures 2c and 2d. If (8) were a reasonable model for the data then, in addition to being an IMT, g′(t) could also serve as the MMT thereby eliminating the need for the third step of the TSAM. Visual inspection of the smooth estimate g′(t) to the 1980 baseline-adjusted time series y′jk in Figures 2c and 2d would suggest a reasonable fit. However, because we have built the approach on a probabilistic model, the specification of the g′(t) and h′j (t) fits may be tested.
 The year-to-year independence of the model noise term may be tested by calculating its autocorrelation function. In Figure 3 the autocorrelation function for the noise term jk (t) is displayed for each model for the nonparametric fit (7) to the CCMVal-1 October 60°S–90°S column ozone. The dashed blue lines in Figure 3 represent 95% confidence limits. Lines that extend beyond these limits are considered to be sample correlations that are significantly different from zero. Inspection of all the models reveals that the assumption of year-to-year independence is a good one for the model (7). The fits are designed to give “smooth” estimates of the long-term trends; in other words, they capture long-term variations in the trend but do not wiggle up and down annually. Interannual wiggles are suppressed by a roughness penalty when fitting the splines. Since the residuals of (7) are not serially correlated, there is no need to represent the residuals using more complex time series models such as ARMA models or fractional Brownian noise. This would not be the case if one had used simple parametric trends that are unable to follow longer term up and down decadal variations. Unlike model (7), the simpler nonparametric model (8), which has the same trend for all climate models, gives serially correlated residuals (Figure 4) and so is not well specified (i.e., its assumptions are not satisfied when the model is fit to the data).
Figure 3. Individual model autocorrelation functions for the residuals εjk(t) for CCMVal-1 October total column ozone in the latitude band 60°S–90°S. This noise corresponds to the nonparametric model (7) with 1980 baseline trend estimates h′j(t) displayed in Figure 2f. The blue dashed lines represent 95% confidence limits for the sample autocorrelation function. This suggests that the assumption of year-to-year independence is a good one for the (7) model.
Download figure to PowerPoint
Figure 4. Individual model autocorrelation functions for the noise term jk(t) for CCMVal-1 October total column ozone in the latitude band 60°S–90°S. This noise corresponds to the simpler nonparametric model (8) with a 1980 baseline trend estimate g'(t) displayed in Figure 2d. The lines extending past the blue dashed lines for several models indicates that the assumption of year-to-year independence is not well satisfied for the (8) model.
Download figure to PowerPoint
Figure 5. Individual model notched box-and-whisker plots (a) for the noise term jk (t) corresponding to the simpler nonparametric additive model (8) and (b) for the noise term εjk(t) corresponding to the nonparametric additive model (7). These apply to the CCMVal-1 October total column ozone in the latitude band 60°S–90°S. In these plots the central black line represents the median, the extent of the notches away from the median line indicates the 95% confidence interval of the median, the top and bottom of the boxes represent the upper and lower quartiles, respectively, and the top and bottom whiskers extend out to 1.5 times the distance from the first to third quartiles. For the noise term εjk(t) in Figure 5b, the medians of all models fall within the notches and are close to zero. Also, the similar height of the boxes indicates that all models have a similar amount of variance away from the estimated trend h′j(t). For the noise term jk(t), the means are significantly different and the intermodel variance is larger suggesting that (8) is not a suitable model for these data.
Download figure to PowerPoint
 We conclude, therefore, that (7) represents one of the simplest nonparametric additive models that is satisfied by the ozone indices considered in the two examples.
2.3. Multimodel Trend Estimates
 The final step of the TSAM approach involves combining the IMT estimates h′j(t) to arrive at an MMT estimate:
where the weights wj(t) have the properties
If the weights are assumed to be nonrandom, and the errors in the individual trends are assumed to be independent, then the squared standard error of the weighted sum is given by:
where sj(t) is the standard error of the trend estimate h′j(t), which can be calculated using standard expressions from linear regression [Woods, 2006]. The standard error (11) can then be used to estimate the confidence and prediction intervals, respectively, as:
 The 95% confidence interval in the trend gives the uncertainty in the trend estimate, where is the standard deviation of the noise term. In other words, there is 95% chance that this interval will overlap the expected trend predicted by the statistical model. The interval is pointwise (rather than simultaneous) in that it represents the uncertainty in the trend at each year rather than being an interval for all probable trend curves over the whole period. The 95% prediction intervals give an idea of how much uncertainty there might be in a predicted index value for an individual year. In other words, there is 95% chance that a particular index value on a specific year will lie in this interval. This interval is the combination of uncertainty in the trend estimate and the uncertainty due to natural interannual variability about the trend.
 The specific choice of weights in (9) remains open. In general, we decide to base the construction of the weights on a statistical probability model with testable assumptions. Here we have chosen a “random effects” model to determine the weights. This model assumes that the trends for individual models h′j(t) are random samples from the “true trend” j(t):
The quantity λ2 is included to account for additional variance between model trends that cannot be accounted for merely by sampling the uncertainty sj2. Using this random effects model, (11) then generalizes to:
which is used here to calculate intervals. Assuming this model is valid, a least squares estimate of wj(t) may be obtained from (9) employing the weights:
Specification of the weights wj(t) from (17) requires an estimate of the parameter λ2. For this we have used the following iterative approach: An initial estimate of the true trend is obtained by calculating h′λ=0(t). Then an iterative Newton-Raphson algorithm is employed to determine the λ that gives scaled residuals that have unit variance as is expected from (14):
Employing this model for the weights produces the MMT estimate h′(t) for the 1980 baseline CCMVal-1 October 60°S–90°S column ozone displayed in Figure 6c. The associated individual model trend estimates h′j(t) and weights wj(t) are displayed in Figures 6a and 6b, respectively. In Figure 6, the weights are scaled by the number of models so that a scaled weight of 1 implies a proportional contribution of that model to the MMT estimate.
Figure 6. For time series of CCMVal-1 October total column ozone in the latitude band 60°S–90°S are presented the (a, d, and g) individual model fits, (b, e, and h) weights, and (c, f, and i; thick gray line) trend (MMT) estimate for three approaches to determining the weights. Results from the “random effects” model (17) are shown in Figures 6a–6c. One problem with this approach is that models can contribute to the final MMT estimate at times when no data exists of that model (i.e., in regions where h′j(t) represents an extrapolation). The introduction of prior weights (20) can help mitigate this problem. Results from the use of a simple on/off set of prior weights (having a value of one where there are model data and zero where there is none) are presented in Figures 6d–6f. One artifact of this approach is that it causes discontinuities in the final MMT estimate. Finally, results from set of prior weights used for the present study, which employ a smoother quadratic taper from a value of 1 where time series data exist to a value of 0 where it is absent, is displayed in Figures 6g–6i.
Download figure to PowerPoint
 While this formulation of weights provides a smooth final trend estimate h′(t), for this example it highlights a potential problem: the individual model weights wj(t) are very insensitive to the absence of data in the original time series. For example, the time series for the MAECHAM4CHEM model (green) extends only over the period 1980–2019 (see Figure 2). Its scaled weight, however, has a value of roughly 1 over the entire period 1960–2100 suggesting significant contributions of its trend estimate h′j(t) at times when there are no model data. The original idea behind this model for the weights was that the natural increase in standard errors sj2 (t) in the region where h′j(t) is extrapolated beyond the model data would cause the weights to decrease naturally toward zero. While Figure 6b indicates that there is some tendency for the weights to display this behavior, many models retain weight values close to unity out to 2100 where they have provided no data.
 To correct this unphysical behavior, we introduce the concept of prior weights wjp(t) into the formulation such that the final weights now have the form:
(with w′j(t) implicitly replacing wj(t) in expressions (11) and (17)). An example set of prior weights would be the “on/off” set: wjp(t) = 1 at times t when raw time series data exist for model j and wjp(t) = 0 otherwise. This prescription is illustrated in Figures 6d–6f. It corrects the unphysical behavior identified when wj(t) of (17) is used alone. However, this on/off prescription is still problematic in that it causes discontinuities in the MMT estimate Figure 6f. The set of prior weights used for the present chapter employ a smoother quadratic taper from a value of 1 where time series data exists to a value of 0 where it is absent:
and where [tj,min, tj,max] defines the period within which data exist for model j. This scheme is illustrated in Figures 6g–6i.
 Finally, we note that the formulation of prior weights (20) allows a natural entry point for the specification of prior, time-independent, model weights based on performance metrics. Such metric based weights would take on values in the range [0,1] and simply multiply wjp(t) in the numerator and denominator of expression (20).