Impact of data assimilation filtering methods on the mesosphere

Authors


Abstract

[1] Three-dimensional data assimilation schemes typically produce analyses that are not in balance. This is evidenced by the generation of spurious high-frequency waves during the first 2 d of forecasts which start from analyses. To remove these spurious waves, assimilation systems frequently filter analyses before using them in models. This work examines the behavior of various spurious wave filtering methods in the context of a model with a mesosphere. Since gravity waves comprise a significant portion of the mesospheric energy spectrum, it is necessary to retain naturally occurring high-frequency waves while filtering spurious waves. The results show that filtering the full analysis state can remove many important high-frequency oscillations from the mesosphere. On the other hand, filtering analysis increments preserves much more of the natural variability of the model. The incremental analysis updating scheme and the incremental digital filter, which are equivalent for linear models and identical coefficients, are shown to give very similar results in the context of a realistic nonlinear model. Results also show a nonlocal response to the insertion of analysis increments in the troposphere and stratosphere. The global mean temperature in the vicinity of the model lid and the diurnal tidal amplitudes are sensitive to the choice of filtering schemes because the filters reduce the amount of resolved waves available to propagate upward into mesosphere. This sensitivity of the mesosphere to the filtering of the lower atmosphere is exploited to choose an optimal filter for our system using measurements of the mesosphere.

1. Introduction

[2] Data assimilation requires the merging of a background model state with observations to produce an analysis state. The analysis is then used as an initial state for a model forecast, the output of which is used as the background for the next assimilation time step. While analyses have primary importance as initial conditions for launching numerical forecasts, they are also used to provide a priori states for satellite data retrievals [e.g., Riishøjgaard et al., 2000] or correlative measurements for new satellite data [e.g., Swinbank and O'Neill, 1994], to drive chemistry transport models or as proxies for the real atmosphere to aid in understanding physical processes (e.g., Manney et al. [2005] assess the quality of various analyses for polar process studies). Most assimilation methods employ a background error covariance matrix, which not only spreads the influence of the observations in space, but also acts as a filter on the analysis increments [see Daley, 1991]. However, such linear filtering provides only an approximate balance between mass and wind fields, so that the analysis will not be consistent with the nonlinear balance implied by the model equations. Thus spurious energy on short timescales are produced when a nonlinear model is used to produce the forecast.

[3] Various methods exist to filter the analysis, such as nonlinear normal mode initialization (NNMI) [see Daley, 1991], the digital filter (DF) [Lynch and Huang, 1992], the incremental digital filter (IDF), and incremental analysis updates (IAU) [Bloom et al., 1996]. The digital filter has gained widespread acceptance in the assimilation community because of its simplicity, but one issue with the digital filter is that it is indiscriminate, filtering all waves of a certain frequency in the same way. In the past, most major assimilation centers had their model lid in the lower to middle stratosphere, so there was a separation between desirable and undesirable waves. However, as model lids move higher to incorporate the mesosphere, the separation of desirable and undesirable waves become more problematic, since model mesospheres are characterized by high-frequency variations [Koshyk et al., 1999] in accordance with the real atmosphere [e.g., Rapp et al. 2002]. In February 2006, the European Centre for Medium-Range Weather Forecasting (ECMWF) operational weather forecast model lid was raised to 0.01 hPa (80 km). The NASA Goddard's GEOS-DAS has had a lid at this same level since January 2004. The Met Office's operational weather forecast model currently has a lid at 0.1 hPa (63 km), but also plans to raise their lid to 0.01 hPa (M. Keil and D. Jackson, personal communication, 2006). The main motivation for raising the lids of weather forecast models is to better simulate satellite radiances from nadir sounders (such as the Advanced Microwave Sounding Unit, AMSU-A) which are sensitive to the mesosphere. Since operational weather forecast model domains are now encompassing much of the mesosphere, it is important to not filter the naturally occurring high-frequency waves. The challenge is then to separate the desirable high-frequency waves from the spurious ones generated by data insertion.

[4] Initialization applied to analysis increments only (rather than the full analysis) was introduced by Puri et al. [1982] and Ballish et al. [1992] in the context of NNMI. The main motivation was to preserve diurnal signals in the background state (see Seaman et al. [1995] for a demonstration of this). Similarly, the DF can be applied to analysis increments (as in the work by Gauthier and Thépaut [2001]). This method is referred to as the incremental digital filter (IDF). Because the IAU method of Bloom et al. [1996] applies to analysis increments, it also preserves the model's high frequencies. Moreover, Polavarapu et al. [2004] have shown that the IAU can be made identical to the IDF for a linear model by choosing appropriate insertion coefficients. Thus the first goal of this work is to examine the conclusions of Polavarapu et al. [2004] in the context of a nonlinear model. Here, the Canadian Middle Atmosphere Model (CMAM) is used to compare various methods of filtering analyses. The CMAM has a lid that is high enough that we need to pay particular attention to the mesosphere.

[5] Data assimilation techniques designed for the troposphere and lower stratosphere are not always suitable for the mesosphere. For example, Polavarapu et al. [2005b] showed that the large variability (due to gravity waves and tides) in the mesosphere implies forecast error variances that are 2 orders of magnitude larger than at lower levels. These large forecast errors then amplify small uncertainties in the specification of background error correlations, rendering the whole assimilation system extremely sensitive to these uncertainties. As a result, observations from the troposphere and stratosphere can have unrealistically large impacts on the mesosphere. Such large increments can lead rather rapidly to biases, unphysical states in the mesosphere or model failure. The unrealistically large mesospheric increments could be damped by measurements, if available, or simply by reducing the vertical correlations that spread information to the mesosphere.

[6] In a data assimilation cycle, each analysis step is followed by a short-term (typically 6 or 12 h) forecast to obtain a background state for the next analysis. While Polavarapu et al. [2005b] examined the impact of the mesosphere on the analysis step, here we focus on the model's response to the insertion of increments in the troposphere and stratosphere. Rather surprisingly, simple choices made for tropospheric data assimilation have profound impacts on the mesosphere. By examining the response of the CMAM within a data assimilation cycle to different spurious wave filtering schemes, we are able to illustrate the sensitivity of the mesosphere to data assimilation parameters (such as those used in defining filters). This is the second and main goal of this work.

[7] The outline of this paper is as follows. Section 2 briefly describes the Canadian Middle Atmosphere Model used in this study while section 3 describes the filtering methods to be investigated in more detail. Section 4 investigates how the filtering methods affect the CMAM when it is used as a forecast model. This section provides an understanding of the impact of the filters on single forecasts, before considering the more complicated effect of the filtering methods in full assimilation cycles in section 5. The conclusions are presented in section 6.

2. Canadian Middle Atmosphere Model

[8] The Canadian Middle Atmosphere Model (CMAM) is a comprehensive three-dimensional (3-D) chemistry climate model (CCM) incorporating radiation, gravity wave drag, moist processes, interactive chemistry and surface exchanges of heat, moisture and momentum. The winds and chemical transport are calculated using a T47 spectral model, while the chemical reactions and physical processes are calculated on an associated linear Gaussian transform grid of 96 × 48 points. In the vertical the model domain ranges from the surface of the earth to approximately 96 km, having 71 layers of unequal thickness spanning this interval, with approximately a 2 km vertical spacing in the middle atmosphere. 36 different chemical species or families are advected, and 95 gas phase and heterogeneous chemical reactions are calculated. More information on the model can be found in the papers by Beagley et al. [1997] and de Grandpré et al. [1997, 2000], and a recent version of CMAM (at T32 resolution) is assessed as part of the model intercomparison of Eyring et al. [2006].

[9] Recently the CMAM has been coupled with a 3-D variational assimilation (3DVar) scheme to produce a CMAM data assimilation system [Polavarapu et al., 2005a]. Standard meteorological observations from the Canadian Meteorological Centre were used below 1 hPa. Between 10 and 1 hPa, the only measurements used were from the AMSU-A instrument. Vertical correlations were constructed to ensure a negligible impact of the data on the analysis above 1 hPa (to avoid the issues described by Polavarapu et al. [2005b]). No measurements were assimilated above 1 hPa. To reduce the spurious waves produced from inserting the analysis updates, a digital filter along the lines of Lynch and Huang [1992] was inserted into the model. Initially, this was deemed an acceptable solution to the problem, based on the tropospheric and stratospheric parts of the analyses, but as we shall see here, the impact of the digital filter on the mesosphere is significant.

3. Initialization Methods

[10] In this work we consider the initialization methods which were available for our system: the digital filter, the incremental digital filter and the incremental analysis updating scheme. NNMI-based schemes are not considered.

[11] The digital filter [Lynch and Huang, 1992] can be simply described as follows: a time series f(t) can be transformed to frequency space by means of a Fourier transform (ℱ):

equation image

[12] If the function in frequency space, F(ω), is multiplied by a Heaviside step function, H(ω), that is zero for all frequencies to be removed, the result can be transformed back to temporal space. The digital filter, then, is a convolution of the original function and the inverse Fourier transform of the appropriate Heaviside step function in frequency space.

[13] The above works well for an infinite, continuous function f(t), but with a numerical forecast model the time series is neither continuous nor infinite. The methodology still works, but the response function no longer has the sharp cutoff that is desired. Figure 1 shows the ideal response function to remove frequencies higher than 6 h (solid curve), and the response from a digital filter (dashed curve) that has acted upon a 12-h time series. The dashed curve shows Gibbs fringes, the effect of which can be lessened by multiplying the filter coefficients by a Lánczos window (dash-dotted curve), as defined by Fillion et al. [1995].

Figure 1.

Response functions for the digital filter applied over a 6-h period (dashed curve). Multiplying the filter coefficients by a Lánczos window [Fillion et al., 1995] removes most of the Gibbs fringes (dash-dotted curve). The ideal response function is shown as the solid curve. For this plot the cutoff period was chosen to be 6 h, and the time step was chosen to be 7.5 min.

[14] Although the digital filter is simple in its conception, there are problems in that the filtering is indiscriminate: both the spurious high frequencies generated by the unbalanced analysis increments and the high frequencies that naturally occur in the climate model are damped or removed. To overcome this problem, a similar method called the incremental digital filter [e.g., Gauthier and Thépaut, 2001] is often employed instead. This method is computationally more expensive than the standard digital filter, requiring the time series to be filtered twice. First, the time series with the analysis increments included is filtered as before, and then the time series with no analysis increments is filtered. With the assumption that the filtering operation is linear and the model response to the increments is linear, the difference between these two results gives the filtered increments, leaving the natural high frequencies in the model untouched. This difference is then added to the original unfiltered, unincremented time series.

[15] A third method of initializing the forecast considered in this study is that of incremental analysis updates. This method was first introduced by Bloom et al. [1996] and relies on adding the increments over a period of time, rather than at a single time. The increments are added as a forcing to the model equations (almost like a nudging, but note that the forcing added at each time step is independent of the background state). The increments added at each time step are a small fraction of the total analysis increment and thus provide much less of a “shock” to the system. Polavarapu et al. [2004] have shown that a suitable choice of the coefficients for the forcing makes this method identical to the incremental digital filter method for linear models. Thus the response function for the incremental analysis updates method, for a suitable choice of forcing function, can be made identical to that shown in Figure 1.

4. Forecast Version of CMAM

[16] We first consider different initialization methods using a forecast version of the CMAM. This allows us to verify that the different filtering methods are behaving as we expect in a controlled environment. We can also examine if the conclusions of Polavarapu et al. [2004] hold for a nonlinear model. In section 5, we consider the response of the model to data insertion during assimilation cycles.

[17] Analysis fields were used to start a 10-d forecast, and the removal of high-frequency waves was performed only at the start of the forecast. The methods investigated were as follows: (1) no filter, (2) digital filter (DF), (3) incremental digital filter (IDF), (4) incremental analysis updates with constant coefficients (IAUc), and (5) incremental analysis updates with time-varying coefficients (IAUv). The DF and IDF both use a 6-h span, 6-h cutoff period, and 7.5 min time step. The same coefficients are used for the IAU with varying weights to allow comparison with the IDF since the methods would be identical for a linear model. Finally, the IAU with constant weights is tried, since this is the more usual IAU configuration [e.g., Bloom et al., 1996]. In addition, a forecast was performed without the analysis increments. This provided a background comparison for the different initialization methods.

[18] The analysis updates were inserted on 16 January 2002 at 0000 UT. This date allows for 2 weeks of “spin-up” from the initial analysis. (As will be seen in section 5.2, the model climate adjusts to the change in filter within 2 weeks.) With no filter, the increments are inserted into the model at 0000 UT and the model is allowed to run freely in forecast mode, adjusting automatically to the spurious waves. With the digital filter, analysis increments are inserted at 0000 UT and the model is run until 0600 UT with the winds, temperatures and surface pressure being filtered over a 6-h time period from 0000 UT to 0600 UT. This calculates the filtered values at 0300 UT, from which the model is allowed to run freely in forecast mode for the remaining 3 h. For the incremental digital filter, the filtering is done twice: first with the regular digital filter, with the filtered fields saved at 0300 UT; and then again with the filter acting on the background field only, which is also saved at 0300 UT. The difference between these outputs gives the filtered increments at 0300 UT which are then used to start the model forecast. For the two incremental analysis update methods the increments are added over a 6-h window from 2100 UT on 15 January 2002 to 0300 UT on 16 January 2002. In the first instance, equal increments are added at each time step. In the second instance, increments are added according to the digital filter coefficients, which weight the increments to be largest around 0000 UT on 16 January 2002. Despite the different methods of inserting analysis increments, the filtering is complete at 0300 UT for all the DF, IDF and IAU experiments, as explained by Polavarapu et al. [2004].

[19] For all experiments the model was run for 2 days with the fields being output every time step (i.e., every 7.5 min). After that, fields were saved every 6 h to conserve disk space. The time series of the wind and temperature fields were then analyzed in a variety of ways. Simple time plots at various longitudes, latitudes and heights serve as a first indication of the different effects of the filters. More rigorous analysis is achieved through the use of wavelet analysis and consideration of the power spectra of the horizontal winds, temperature and geopotential height. Since the impact of the filters on the mesosphere is of particular interest, we also investigated the amplitudes of the diurnal tides in this region of the atmosphere.

4.1. Time Series

[20] Figure 2a shows the time evolution of temperature at 10 hPa close to 0°E, 48°N for the first day of the forecast period. An arbitrary location in the midlatitudes was chosen since analyses are expected to be better balanced in the midlatitude troposphere and stratosphere. The black line represents the unfiltered free-running CMAM (i.e., with no increments added), and is provided for comparison purposes. Because measurements are inserted at a 6-h frequency, information on timescales less than 6 h will not be resolved by the analysis increments. Thus an assimilation with a perfect filter should not add extra variability relative to the free running model on timescales shorter than 6 h. The cyan curve is the model forecast from the initial analysis field with no filtering. The difference between the free-running CMAM and this curve at time 0000 UT shows the magnitude of the analysis increments: at this particular location they are approximately 1 K. Understandably the sudden introduction of increments lead to the generation of spurious waves, which are clearly seen in the time evolution of the cyan curve which has a high-frequency variation superimposed on the typical CMAM variation. The high-frequency oscillations die out over time.

Figure 2.

Temperature time series in kelvin for a point (a and b) at 10 hPa, 0°E and 47.87°N and (c and d) at 0.005 hPa, 180°E and 17.23°N. The free run is shown along with the assimilation runs with no filter, DF and IDF in Figures 2a and 2c. In Figures 2b and 2d the free run is shown with the incremental filters: IDF, IAUv, and IAUc. Hour 0 corresponds to 16 January 2002 at 0000 UTC.

[21] The yellow curve represents the forecast initialized by applying the full digital filter: the output for this experiment, as noted above, begins at the midpoint of the filter, which in this case is at 0300 UT. The digital filter has removed the high-frequency spurious oscillations seen in the cyan curve, but at the same time the variation is overly damped when compared to the other experiments. This is because the DF filters the background state as well as the analysis increments. In contrast, the incremental digital filter (blue dashed curve) is able to remove some of the spurious high-frequency oscillations, but still preserves the natural frequencies of the model.

[22] Figure 2b repeats the free CMAM and IDF time series and adds the results from the IAU experiments (red for IAUc and green for IAUv). Figure 2b shows that the IAU slowly forces the model state away from the free model as the analysis increments are inserted. The IAUc experiment (red curve) moves away more quickly during hours −3 to 0 than the one with varying coefficients (green curve) since the initial increments are larger than those used with IAUv [see Polavarapu et al., 2004, Figure 2b]. For IAU with constant coefficients, the analysis increment is spread equally in time over 6 h. When varying coefficients are used, the increment is spread smoothly but unequally in time, with maximum amplitude at the middle of the insertion interval (hour 0) and very small amplitude at the ends of the interval (hours −3 or 3).

[23] One aim of this study was to investigate the work of Polavarapu et al. [2004] to see if the IAU with varying coefficients (green curve) is identical to the incremental digital filter (blue dashed curve) even for the case of a nonlinear model. The results here show that although the two curves are not identical they are quite close during hours 3 to 15 and these two forecasts are the closest of all the experiments. Note also that the IAU with constant coefficients (red curve) results in more damping and a slightly smoother time series than the IAU with varying coefficients (green curve).

[24] At later times, all of the forecasts exhibit similar behavior. The high-frequency oscillations tend to die out in the unfiltered forecast, and the amplitude of the digital filter oscillations returns to match that of the other simulations, as the natural frequencies of the model are regenerated.

[25] A second motivation for this study was to investigate the effects that the digital filter might be having on the model mesosphere since gravity waves are a ubiquitous but important component of mesospheric dynamics. Figure 2c shows a similar plot of temperature evolution to that in Figure 2a except for 0.005 hPa (about 85 km) around 17°N. For Figure 2c, we chose a subtropical point at a latitude and height where the meridional component of the migrating diurnal tide is important (see section 5.4). Since no observational data for the mesosphere are used here, the no filter case (cyan curve) is identical to the black free-running CMAM curve at 0000 UT. Since analysis increments are negligible at this height, the impact of filtering the analysis increments in the troposphere and stratosphere may seem somewhat surprising. However, since waves propagate into the mesosphere from below, insertion of information will lead to upward propagation of information. Since analysis increments generate real and spurious waves locally, both real and spurious information propagates into the mesosphere. Apart from the lack of increments at 0000 UT, the features of the curves are identical to those in Figures 2a and 2b. As at 10 hPa, the DF (yellow curve) produces a time series much smoother than that of the free model. However, without any filtering, the cyan curve may be fine at times (as in hours −3 to 14) since the mesosphere is characterized by high-frequency waves, but too noisy at other times (e.g., hours 14 to 24). Filtering increments only (blue dashed curve) produces a time series which is not as smooth as the one obtained by filtering the full analysis but is smoother than that obtained with no filter. Figure 2d shows that in the mesosphere, as at 10 hPa, the IAU with time varying coefficients and the IDF time series (green and blue) are still very similar. Because Figure 2c shows that the DF (yellow curve) acts to damp the natural variation of the CMAM significantly, this method of initialization is clearly questionable for use in the mesosphere.

4.2. Wavelet Analysis

[26] Figure 2 shows that the methods of removing spurious waves are behaving as expected from the analysis of Polavarapu et al. [2004]. For more quantitative detail we consider the evolution of the power spectrum using wavelet analysis. Wavelet analyses decompose time series into the time/frequency space simultaneously in order to obtain the amplitudes of dominant scales (“periodic” signals) and their variation as a function of time. In this section we focus on the first 48 h of the forecasts. On this time interval wavelets will resolve the scales shorter than 24 h. Since the time series step is 7.5 min (the model time step) the minimum scale that can be resolved is 15 min. Thus, using a 48-h time series we expect to resolve timescales of 0.25 to 24 h. The analysis here uses Morlet wavelets as the basis functions. A more detailed explanation of the method is given by Torrence and Compo [1998].

[27] Figure 3 shows the wavelet power spectra for temperature time series at 0.005 hPa and 17°N, the same latitude and height used in Figures 2c and 2d. Because the time series for different longitudes were rather different, it was important to know which features were robust. Thus the spectra obtained for the times series for each of 96 longitudes were averaged. Figure 3 shows strong components at periods of 12 and 24 h, representing the semidiurnal and diurnal cycles, respectively. (Because of the discretization used in the wavelet transform, the resolved scales corresponding to these periods are 11.31 and 22.62 h). Note that the wavelet resolves 2 more scales in between 12 and 24 h, and none of those scales has maxima in the free model case (Figure 3a). Because of edge effects the lines of maxima do not stretch till the ends of the time interval considered. Edge effects are stronger for longer periods. For the 24-h scale, only the middle point of the time interval is not subjected to edge effects. Thus we consider only the middle of the time interval as a local indication of the relative effects of different filters on the 24-h wave. No time evolution of this scale can be seen with a 48-h time series. For small scales, like 6-h waves, only the first and last 6 h are subjected to edge effects, allowing us to see the evolution of power over 6 wave periods.

Figure 3.

Wavelet power spectra, using Morlet wavelets. The x axis is the wavelet location in time. The y axis is the wavelet period in hours. The colors show maxima in energy distribution for timescales of 6, 12, and 24 h. The contour interval is about 3.38°C2. This is one tenth the maximum power of the free CMAM case, i.e., 33.8°C2. The power spectra for 96 time series (one for each longitude) of temperature at 0.005 hPa and 17.23°N were averaged.

[28] Without any filter (Figure 3c), there is more power for time scales of 8 h or less, when compared to all other cases. The large power for the 4-h timescale (Figure 3c) diminishes with time as these waves are dispersed or damped. Applying the digital filter to the full analysis excessively reduces the power for periods of 8 h or less (Figure 3b) though only initially until hour 12. An incremental filter results in a better representation of periods of 8 h or less (compare Figure 3a with Figure 3d, 3e or 3f). Perhaps fortuitously, for small scales, the spectrum that is closest to the free run spectrum is obtained with the IAU with constant coefficients.

[29] The amplitude of the semidiurnal signal is similar in all analyses though the width of the 12-h spectral line varies. The unfiltered case (Figure 3c) is the most dispersive, as expected. Note that the amplitude of the semidiurnal and diurnal waves in the free model is about 5°C. The amplitude of the diurnal signal is strongly affected by the choice of filter. Interestingly, without any filter (Figure 3c), data insertion results in a damping of the diurnal signal. Because only 6 h of measurements were used, they do not help much in capturing the 24-h wave. Another problem is that 3DVar treats all observations over the 6-h window as though valid at the same time, resulting in a damping of the diurnal signal even when no filter is used. Swinbank et al. [1999] have noted that the incorrect time treatment of asynoptic satellite data in 3-D assimilation schemes has a damping effect on tides. Filtering the background state as well as analysis increments further damps the diurnal signal (Figure 3b). Incremental filters are somewhat better at preserving the diurnal signal and the IAU with constant coefficients is the best in this regard (compare Figures 3a and 3e).

4.3. Energy Spectra

[30] The methods to remove spurious high-frequency waves have an effect on the spatial wave number spectra of the model fields, as well as on their frequency spectra. Figure 4 shows divergent kinetic energy spectra for the various forecasts, calculated at 0600 UT on 16 January at four different pressure levels. The color coding of the curves is identical to that of Figure 2. The spectra for the various analyses are remarkably similar. This is in contrast to the case shown later when the filters are applied repeatedly during an assimilation cycle (section 5.3). The spectra from the free run are understandably different from the other spectra because the information from observations is not included. Some differences among analysis spectra are seen in the mesosphere (Figures 4a and 4b) and in particular at the highest and lowest wave numbers. The biggest impact on spectra results from the use of a full digital filter, which results in reduced power when compared to the other initialization methods. The unfiltered increments tend to lead to slightly more power at all wave numbers. Although only the divergent kinetic energy spectra are shown in Figure 4, similar results (although less pronounced) were also found in the rotational kinetic energy spectra, and in the power spectra of temperature and geopotential height.

Figure 4.

Log10 of divergent kinetic energy spectra in J/kg for the various assimilation methods at 0600 UT on 16 January 2002 at four different altitudes. The color coding is identical to that in Figure 2.

[31] To better compare the IDF and the IAU methods, Figure 5 shows the differences in the divergent kinetic energy spectra between the IDF run and the IAU with constant coefficients (dotted curve), and between the IDF run and the IAU with varying coefficients (solid curve). Although differences clearly exist, this again shows some similarity between the IDF and the IAU with suitably chosen time-varying coefficients.

Figure 5.

Difference in the log10 of divergent kinetic energy (J/kg) between the IDF and the IAU with constant coefficients (dotted line) and between the IDF and the IAU with time-varying coefficients (solid line).

4.4. Summary of Forecast Model Results

[32] This section has highlighted the effects that the different initialization methods have on temperature evolution and divergent kinetic energy spectra in CMAM when run in forecast mode. The digital filter, which is sometimes used by models employing a 3-D assimilation scheme, has been shown to excessively damp temperature variations compared to the other methods. These results also provide an extension of the work of Polavarapu et al. [2004] to the case of a nonlinear model. Although the responses from the IAU with time-varying coefficients and the incremental digital filter are not identical, as is the case for linear models, the results are remarkably similar given that the methods were shown to produce different results in theory for even weakly nonlinear models.

[33] The initialization methods have also been shown to affect the kinetic energy spectra of the fields, particularly at the higher wave numbers at the higher altitudes. Again, the digital filter leads to the most damping. However, the effect on the mesospheric diurnal tide in these experiments is minimal since the CMAM regenerates the tide within the 10-d forecast period (not shown).

[34] Even though gravity waves are an important part of mesospheric dynamics, all diagnostics (time series, wavelet analyses, energy spectra) show that some filtering of analysis increments is necessary because analysis increments in the troposphere and stratosphere contain spurious gravity waves which propagate up into the mesosphere. Thus a major difficulty is to separate the real from spurious waves in the mesosphere since their frequencies overlap.

5. Full Assimilation Cycle

[35] Thus far the impact of the different filtering methods on a single forecast were considered, but in an assimilation cycle, the analysis is updated and filtered every 6 h. Here we investigate how repeated application of filters affects the mesosphere. Section 4 showed that the IAU with time-varying coefficients and the IDF produce similar results. Therefore, in this section we only consider the digital filter and IAU.

5.1. Description of Methods

[36] We have compared five different assimilation cycle runs, two initialized with the digital filter, and three initialized by incremental analysis updates. The two digital filter initializations are performed over a 12-h time period, giving the filtered fields at hour 6: the only difference is in the cutoff period (6 h (DF6) versus 12 h (DF12)). For the incremental analysis update methods, the increments are inserted over a 6-h period, centered around the assimilation time. Two of the IAUs use time-varying coefficients (for cutoff periods of 4 (IAU4) and 6 (IAU6) h), and the third uses constant coefficients (IAUc), spreading the increments equally over the 6-h insertion period. Figure 6 shows the response functions for the various filters. The response functions illustrate the impact of each filter on the amplitude of a single wave of a given period. As noted earlier, because of incomplete observations and the lumping of data into 6-h blocks, information on timescales of 6 h or less will not be captured. Thus the filter response should presumably be close to 0 for wave periods less than 6 h and close to 1 for periods greater than this.

Figure 6.

Response functions for the initialization methods used in the full assimilation cycles: thick-dashed (DF with 12-h cutoff), thick solid (DF with 6-h cutoff), dotted (IAU with 4-h cutoff), dot-dashed (IAU with 6-h cutoff), and thin solid (IAU with constant coefficients). The DF12 and DF6 filters use a 12-h span while the IAU cases all use a 6-h span. The time step is 7.5 min in all cases. A Lanczos windowing function [Fillion et al., 1995] is applied to the filter coefficients for DF6, DF12, IAU6, and IAU4.

[37] The DF6 and DF12 schemes are applied to the full analysis and therefore diurnal signals in the background state will be damped. The DF6 is typical of filters used in operational schemes in that the goal is to filter only waves with periods of 6 h or less. However, the use of a windowing function (to avoid Gibbs effects) smooths the response so that the 5-h wave is not completely damped while the 8-h wave is only partially damped (thick solid curve). The DF12 (dashed curve) is an extreme example (not used practically) which damps the 12-h wave by a factor of 2 and even damps the 24-h wave a little. The IAU schemes are applied to analysis increments only, so all are theoretically preferable to the DF schemes. While the IAUc scheme (thin solid curve) may seem overly damping, of the IAU schemes, it alone completely removes the 6-h wave. The IAU6 and IAU4 schemes admit increasingly more fast waves. The IAU4 scheme (dotted curve), in particular, does very little damping considering the fact that CMAM's semi-implicit time scheme and time step choice already filter waves with periods of 3 h or less [Manson et al., 2002].

[38] All five assimilation cycles were started from an analysis field on 1 January 2002 at 0000 UT (obtained from a cycle which used a digital filter). Since the digital filter inserts increments at synoptic times (0000, 0600, 1200, 1800 UT) while the IAU inserts increments in a 6-h window centered on synoptic times, the switch to IAU required an initial 9-h forecast. Thus the IAU analyses start at 1200 UT on 1 January 2002. The data assimilation cycles were run until 28 February 2002.

5.2. Global Mean Temperatures

[39] Figure 7 shows the evolution of global average temperature differences between the runs during January 2002. The differences are plotted relative to the IAU run with a 6-h cutoff period. Positive values correspond to warmer temperatures for IAU6 analyses when compared to other analyses. Below 1 hPa, where data are continually inserted into the cycle, there is very little difference between the cycles (not shown). In this region the different effects of the initialization methods are minimal. In the lower mesosphere (e.g., at 65 km) differences stay within 1 K but in the upper mesosphere, differences grow in time, saturating after about 14 days. The global mean temperature is significantly higher in the IAU6 run than three of the other runs. However, the global mean temperature of the IAU4 run (dash-dotted curve) is even higher. These temperature differences are maintained until the end of February (not shown).

Figure 7.

Evolution of the difference in global mean temperature in kelvin between two sets of analyses during January 2002, at two heights (91 or 65 km). Positive values mean that the IAU6 analysis is warmer than the corresponding analysis from another cycle. A difference of ±1 K is shown by the dotted lines. The four curves depicting the temperature differences at 65 km are all within ±1 K.

[40] Since the global mean temperature equilibrates by the middle of January (Figure 7), comparisons with SABER measurements are examined near the end of this month. SABER [Russell et al., 1999] is an instrument on board the TIMED satellite [Mlynczak, 1997] and has been making measurements of temperature and other fields since January 2002. The SABER data are not used as observations in the assimilation cycles in this study, so provide an independent data set against which to compare.

[41] Figure 8 shows the global average temperature of the various analyses compared to SABER temperature data on 25 January, first for all SABER measurements (Figure 8a), and then divided into different regions (Figures 8b–8d). The Southern Hemisphere represents all locations south of 30°S, the Northern Hemisphere all locations north of 30°N, and the tropics the region in between. Although the analyses all agree with each other at lower altitudes there are marked differences between each of them, and also between them and the SABER data. In particular, in the stratosphere (below 45 km), all analyses are colder than the SABER data. The fact that all analyses agree is not too surprising since they all use the same data which is predominantly from AMSU-A radiances with some sporadic information from radiosondes in the lower stratosphere. Thus the disagreement between the analyses and SABER in this region indicates a relative bias between AMSU-A and SABER data. A comparison for February 2002 of SABER with Met Office analyses, which were also predominantly based on AMSU-A measurements [Remsberg et al., 2003] found SABER data too warm from 100 to 10 hPa (roughly 15 to 30 km) but too cold from 10 to 1 hPa (30 to 45 km). Thus the discrepancy between CMAM and Met Office analyses may be related to the treatment of the AMSU-A data (bias correction) by the two systems, boundary effects due to the placement of the lid at 0.1 hPa in the Met Office model (compared to the CMAM lid of 0.0007 hPa), or the different period of observation.

Figure 8.

Comparison with SABER temperature data (in kelvin) on 25 January 2002 over (a) global SABER measurements, (b) Southern Hemisphere SABER measurements (latitudes 30–90°S), (c) tropical SABER measurements (latitudes 30 N–30°S), and (d) Northern Hemisphere SABER measurements (latitudes 30–90°N). The colors are black (SABER data), cyan (DF with 12-h cutoff), yellow (DF with 6-h cutoff), green (IAU with 6-h cutoff), blue (IAU with 4-h cutoff), and red (IAU with constant coefficients). The CMAM data are sampled at the same locations as the SABER data.

[42] In the lower to middle mesosphere (45–70 km), the agreement of the analyses with the SABER data is quite good. Since no data were assimilated in this region, this agreement indicates that CMAM has the right radiation balance since the global mean temperature is radiatively determined.

[43] In the upper mesosphere and lower thermosphere region (above 70 km), the various analyses differ. The digital filter cases (cyan and yellow curves) damp the full state and they yield the lowest temperatures in this region. The IAU assimilations (red, green and blue curves) damp only analysis increments so they yield higher temperatures. In general, the less damping that is done, the higher the temperatures are in this region.

[44] The response in the mesosphere is largest at altitudes above 80 km, since it is in this region that the model sponge is located. The sponge acts to absorb waves that propagate upward, preventing reflection from the model lid. The sponge is thus a numerical device, which is supposed to mimic the fact that in the absence of a model lid, these waves would propagate higher and eventually break, depositing momentum, creating turbulent dissipation and generating heat. The sponge in the CMAM is a nonzonal sponge, which is more physical than a zonal sponge [Shepherd et al., 1996]. As waves are absorbed by the sponge the temperature in that region of the atmosphere increases from the associated energy deposition, which in turn leads to more radiation to space. Eventually the radiation to space exactly balances the heating from the waves, leading to the maintenance of the temperature structure observed in Figure 7. Although additional heating is generated artificially here, by the enforced dissipation of the waves, in the real atmosphere or in a climate model with a higher lid it would still occur, although higher up, since the sponge mimics the effect of real processes such as molecular viscosity.

[45] The sensitivity of the global mean temperature to the low-pass filters demonstrates the importance of high-frequency waves to the energy budget of the mesopause region. Moreover, the magnitude of the impact seen here (on the order of 10 K) is consistent with observational estimates. Lübken et al. [2002] estimate a frictional heating rate of 10–20 K/d due to breaking gravity waves. Since the radiation timescale at these altitudes is 1–2 d, assuming Newton cooling, this corresponds to a 10–40 K potential heating due to gravity wave dissipation.

[46] Figure 8 also shows different results for different geographic regions. In the tropics (Figure 8c), the average SABER temperature at the model lid is approximately 182 K, lower than that seen in other regions. The fact that the digital filters (cyan and yellow) provide the best agreement with SABER data at this height suggests that either the model forecast has excessive tropical wave activity or the incremental filters do not sufficiently control the noise in tropical analyses. It is difficult to say whether CMAM has excessive tropical wave activity but our tropical analyses are likely excessively noisy. While 3-D assimilation schemes apply a linear mass wind balance in the extratropics, no balance constraint is applied in the tropics. Thus tropical analyses are not expected to be in balance. This then suggests the need to consider improved physically based balance of analysis increments in the tropics, at least for 3-D assimilation schemes.

5.3. Mechanism for Nonlocal Transfer of Information

[47] In section 4, nonlocal responses to data insertion were seen. Specifically, changes to the low-pass filters employed by the data assimilation scheme modified the analysis increments in the troposphere and stratosphere yet the largest model response was in the mesosphere. The mechanism postulated for the impact of analysis increments on the global mean temperature at and above the mesopause is the generation of real and spurious waves which propagate into the mesosphere and break, generating heating. Since the CMAM also parameterizes the effect of subgrid-scale gravity waves on the resolved flow, it is important to know whether this effect is due to resolved waves, or to parameterized waves. To show that the heating is created by the absorption of resolved waves, the variances of time series of analyses were examined. More precisely a time series of 6-h differences of analyses were examined since this difference will remove some of the slowest trends. The standard deviations of the temperature time series for each experiment over January 2002 are shown in Figure 9. The color scheme is the same as that used in Figure 8. Now it is clear that the strongest filters have the least variance, and thus the least wave activity. Moreover, as the filters become increasingly weak, admitting more waves, the variance increases, particularly in the mesosphere. The solid black curve shows the variance of a free CMAM simulation. The digital filters applied to the full analysis reduce wave activity compared with the free-running model simulation. The IAU with constant weights has the closest variance to the free-running CMAM. The dashed curve shows the variance of a set of analyses obtained with no filtering. Clearly, there is excessive variance compared with the free model run. This again shows that the insertion of analysis increments generates spurious waves which propagate upward and contaminate the mesosphere. Thus, even though the CMAM naturally generates gravity waves in the mesosphere, it is still necessary to apply some kind of gravity wave filter to analysis increments in the troposphere and lower stratosphere.

Figure 9.

Global mean temperature standard deviation of time series of 6-h difference fields from the free CMAM run (black solid), and from analyses obtained using a digital filter with a 12-h cutoff (cyan), 6-h cutoff (yellow) and Incremental Analysis Updates with constant weights (red), a 6-h cutoff (green) and a 4-h cutoff (blue). Temperature units are degrees kelvin.

[48] While variances give an overall picture of wave activity, more details are seen in energy spectra. Figure 10 shows the spectra of divergent kinetic energy for 6-h forecasts valid on 31 January 2002, 1800 UT. Also shown is the range (2 standard deviations) of CMAM spectra during January and February over 5 years (a) of a climate run. Because the filters applied to analysis increments try to remove waves with periods of 6 h or less, analyses should be similar to the model forecasts on these scales. If short timescales correspond to short spatial scales, analyses spectra should be reasonably similar to model spectra for smaller scales. Although energy spectra for the various filters were rather similar in Figure 4, the spectra here are now distinctly different. This is because section 4 considered the simpler case of a long forecast following a single assimilation, whereas here in an assimilation cycle, the filter is applied after every 6-h forecast. Figure 10 again shows that DF12 damps waves far too severely. This fact is not so obvious in the troposphere (not shown) or in the lowermost stratosphere (Figure 10, bottom right), but above the midstratosphere, forecast spectra are clearly below the range of model values. Figure 10 also shows that DF6 leads to too little power for wave numbers above 30 at 0.09 hPa and for wave numbers above 20 at 0.01 hPa. Thus, while DF6 is satisfactory for models with lower lids, it is unreasonably detrimental if mesospheric analyses are important. By applying the filters to the full analysis, DF12 and DF6 reduce wave amplitudes in the mesosphere which were generated by the 6-h model forecast since there are no analysis increments in this region. Thus both are detrimental to mesospheric analyses. Finally, note that IAU4, which does very little filtering, has too much power compared to the range of free model values at 0.01 hPa for wave numbers higher than 10. Therefore this filter insufficiently damps the spurious smallest-scale waves.

Figure 10.

Spectra of divergent kinetic energy (J/kg) on 31 January 2002 1800 UT for the five assimilation experiments. The four plots show different heights as labeled at top. The shaded region indicates a climatology from 5 a of a free CMAM run. Spectra were computed 4 times per day during January and February during each of 5 a for a sample size of 1180. The range shaded is the log10 of the mean of the climatology plus or minus two standard deviations.

5.4. Mesospheric Tides

[49] In section 5.3, assimilation schemes were found to alter the generation and the amplitude of waves propagating upward into the mesosphere. Since upward propagating waves affect tides, we consider the impact of the various filtering schemes on the propagating diurnal tide. It was noted earlier that once a filter was applied, the forecast model was able to reestablish the tide during a ten day forecast. However, within an assimilation cycle, a filter is applied after every 6-h forecast. Since the vertical group velocity of the diurnal tide is roughly 0.12 m/s or 10 km/d [Vial, 1991], it takes several days for the tide to reach the upper mesopause. During this time, the mesospheric analysis (which in our system is driven by information from below only, since no mesospheric observations are used) would be filtered many times. Figure 11 shows the amplitude of the meridional wind tide for the data assimilation cycles, along with the tide from the free running CMAM. McLandress [1997] and Beagley et al. [2000] have shown that the free running CMAM captures the diurnal tide reasonably well. The tidal amplitudes are calculated using data from 20 to 30 January 2002.

Figure 11.

Amplitude of the diurnal tide in meridional wind in m/s. (a) Free running CMAM which provides a reference. (c) and (e) Results obtained with a digital filter with a 12-h or 6-h cutoff period, respectively. (b), (d), and (f) Results obtained with an IAU for constant weights, a 6-h cutoff period, and a 4-h cutoff period, respectively.

[50] Figure 11c shows that the digital filter with the 12-h cutoff period has removed virtually all the power in the tidal amplitude. Figure 6 shows that waves with a 24-h period have their amplitudes reduced to about 90% of the original values by the digital filter with a 12-h cutoff period (solid line). However, the 10% power reduction is being repeatedly applied every 6 h, so the cumulative reduction in wave amplitude will lead to a very weak tidal amplitude. This result is consistent with the reduced wave activity in the mesosphere seen in Figure 10.

[51] Figure 11e shows that the amplitude of the diurnal tide is increased when the digital filter with a 6-h cutoff period is used. Although this at first may seem a surprising result, McLandress [2002] has shown that the amplitude of the tide is damped by nonlinear interactions with waves of different frequencies. By removing some of the high-frequency waves, the digital filter reduces the nonlinear damping of the tides thereby increasing its amplitude.

[52] The IAU (Figures 11b, 11d, and 11f) results are very similar to each other, with perhaps the closest to the free running CMAM being the IAU with constant coefficients. The power reduction for the IAU runs is much less than that for the digital filter runs, and acts only on the increments not on the full model fields so the effect on the tides is minimal.

[53] Although Figure 11 only shows the results for the meridional wind tidal amplitude, similar results were found for the zonal wind and temperature signals of the diurnal tide.

6. Conclusion

[54] Because of the current temporal and spatial distribution of observations assimilated at weather centers, high-frequency waves are not completely observed, so they cannot be represented in analyses. Nevertheless, observations contain signals at high frequencies because the real atmosphere being sampled does. Three-dimensional (3-D) assimilation schemes attempt to filter such frequencies using simple linear mass wind balances and background error covariances. However, the filtering is not perfect and a linear balance is not the same as the complex nonlinear balance of a weather forecast model. Thus spurious high-frequency waves are generated when models are integrated from analyses. In addition, 3-D assimilation schemes often lump data into 6-h windows, so the filtering of waves with periods less than 6 h is necessary. This approach of filtering forecasts to remove spurious waves generated by data insertion has worked well when the forecast model domain included only the troposphere and stratosphere. Now that forecast model domains include some or all of the mesosphere, the challenge is to separate naturally occurring gravity waves from spurious ones.

[55] In this work, various filtering methods were applied to analyses or to analysis increments in order to identify their suitability for models with a mesosphere. First, various filters were applied and their impact on 10-d forecasts were examined through time series, wavelet analyses and spatial spectra. When the digital filter was applied to the full model state, the spurious high-frequency energies were significantly reduced but much of the power in the natural high frequencies of the model were reduced at the same time. This effect was seen not only in the mesosphere, which is characterized by high-frequency disturbances, but also at lower levels. If analysis increments only were filtered, the spurious high frequencies were removed, but the natural variability of the CMAM was maintained. Two methods of filtering analysis increments, the IDF and the IAU with time-varying coefficients, were seen to give very similar (but not identical) results, showing that the equivalence between these two methods for the linear case [Polavarapu et al., 2004] also holds to some extent for a realistic nonlinear model.

[56] When filters were applied during assimilation cycles, a nonlocal response to data insertion was seen during the model integration. Analysis increments were inserted into the troposphere and stratosphere, yet the strongest impact was on the mesosphere. Global mean temperatures in the upper mesosphere and lower thermosphere (MLT) varied by roughly 10 degrees depending on which filter was applied. Analysis increments add information in the troposphere and stratosphere which propagates upward. This information is partly real (describing departures of the model forecast from measurements) and partly spurious (due to imperfections in sampling and assimilation schemes). Because some of the information is spurious, filtering of analysis increments remains necessary, even when the forecast model simulates realistic gravity waves. The fact that the mesosphere is so sensitive to changes in the lower atmosphere suggests that measurements of the mesosphere may help constrain uncertain model parameters in the lower atmosphere. Here we use mesospheric observations to constrain assimilation parameters. Specifically, comparisons of analyses obtained using different gravity wave filtering schemes against SABER mesospheric temperature observations were used to help select the best filtering method for our system (IAUc). In the troposphere and stratosphere, the choice of filter is usually made subjectively, based on the smoothness of filtered time series. Using this criterion, there is little to distinguish any filter which is applied to analysis increments (here IDF or IAU). However, looking at the mesosphere does reveal differences between the filters because of the irreversible affect of gravity waves on the mesopause region. Thus the choice of optimal filter for our system was based on the accuracy of the resulting mesospheric analyses. Since the sensitivity of the troposphere and stratosphere to the type of incremental filter was not large, this choice may be suitable for the entire atmosphere.

[57] The mechanism for the transfer of information of observations to the mesosphere was also presented. Although information can be spread by background error covariances during the assimilation step, here it was shown that the information of observations is also spread during the 6-h forecast step by vertically propagating waves. These waves are generated by data insertion in the troposphere and stratosphere and they increase in amplitude as they propagate upward, eventually breaking in the MLT region where they heat the atmosphere through turbulent dissipation. Radiative cooling is increased but the overall impact is one of heating. It is interesting that the change in heating obtained by changing the gravity wave filters is consistent in magnitude with observational studies [Lübken et al., 2002] which estimate the impact of gravity waves on heating of the MLT.

[58] In this work, characteristics of analyses were frequently compared against the free running model. This is because in the absence of measurements analyses equal the background, or model forecast. Since analysis increments do not contain useful information on timescales less than 6 h, analyses should be statistically similar to model forecasts on these timescales. Thus on these short timescales, analyses can do no better than the model forecast. Therefore it would be beneficial if forecast models were realistic in their gravity wave characteristics. Unfortunately, it is difficult to ascertain this for any model given the incomplete observational constraints and the fact that model and measurements sample different parts of the gravity wave spectrum. For CMAM at least, a preliminary comparison with radar observations [Manson et al., 2002] suggests that CMAM amplitudes and spectral slopes at high frequencies are reasonable.

Acknowledgments

[59] This research was supported by the “Canadian Stratospheric Processes and their Role in Climate” project which is funded by the Canadian Foundation of Climate and Atmospheric Sciences and the Canadian Space Agency. Ted Shepherd and three anonymous reviewers provided helpful comments on earlier versions of the manuscript.

Ancillary