Improving regional ozone modeling through systematic evaluation of errors using the aircraft observations during the International Consortium for Atmospheric Research on Transport and Transformation



[1] During the operational phase of the ICARTT field experiment in 2004, the regional air quality model STEM showed a strong positive surface bias and a negative upper troposphere bias (compared to observed DC-8 and WP-3 observations) with respect to ozone. After updating emissions from NEI 1999 to NEI 2001 (with a 2004 large point sources inventory update), and modifying boundary conditions, low-level model bias decreases from 11.21 to 1.45 ppbv for the NASA DC-8 observations and from 8.26 to −0.34 for the NOAA WP-3. Improvements in boundary conditions provided by global models decrease the upper troposphere negative ozone bias, while accounting for biomass burning emissions improved model performance for CO. The covariances of ozone bias were highly correlated to NOz, NOy, and HNO3 biases. Interpolation of bias information through kriging showed that decreasing emissions in SE United States would reduce regional ozone model bias and improve model correlation coefficients. The spatial distribution of forecast errors was analyzed using kriging, which identified distinct features, which when compared to errors in postanalysis simulations, helped document improvements. Changes in dry deposition to crops were shown to reduce substantially high bias in the forecasts in the Midwest, while updated emissions were shown to account for decreases in bias in the eastern United States. Observed and modeled ozone production efficiencies for the DC-8 were calculated and shown to be very similar (7.8) suggesting that recurring ozone bias is due to overestimation of NOx emissions. Sensitivity studies showed that ozone formation in the United States is most sensitive to NOx emissions, followed by VOCs and CO. PAN as a reservoir of NOx can contribute to a significant amount of surface ozone through thermal decomposition.

1. Introduction

[2] Air pollution models have been used to predict air quality during numerous field campaigns [Bates et al., 1998; Jacob et al., 2003; Menut et al., 2000; Ramana et al., 2004; Vautard et al., 2003], with the objective to both place air pollution in a geographical context for experimental design and, as the data is collected, evaluate our current understanding of atmospheric processes [Kiley et al., 2003] and anthropogenic and biogenic emissions [Carmichael et al., 2003]. During the summer of 2004, the International Consortium for Atmospheric Research on Transport and Transformation (ICARTT) field experiment was performed (, which included a NASA experiment called INTEX-A (Intercontinental Chemical Transport Experiment–A), and a NOAA experiment called NEAQS/ITCT-2k4 (New England Air Quality Study–Intercontinental Transport and Chemical Transformation, 2004). During this period NASA DC-8 and NOAA WP-3 aircraft each performed 18 research flights over the continental United States, with a special focus on the northeastern United States. Figure 1 shows the flight tracks of the DC-8 and WP-3 during the mission, and the altitude at which the aircraft flew. The WP-3 aircraft flew at lower altitudes, mostly over the NE United states. More details about the aircraft measurements and main findings are given by Singh et al. [2006]. During the field experiment operations forecasts using the University of Iowa STEM model [Carmichael et al., 1991; Tang et al., 2003, 2004] were used (along with other models) in support for flight planning. Analysis of the forecasts has shown a persistent positive bias (modeled-observed) for ozone [McKeen et al., 2006] in comparison to surface sites in the AIRMAP network (, and in comparison to the aircraft platform observations. The objective of this paper is to show that model performance in relation to ozone and its precursors was improved through systematic analysis of model prediction with the observed data to evaluate where model error persists, and how ozone model error is related to model error of other species. The paper also demonstrates how geospatial interpolation through kriging can be used to provide geographical contextualization to model error, which in turn can be used to improve model performance.

Figure 1.

Flight tracks and altitude range for ICARTT. Red indicates 0–1 km, yellow indicates 1–4 km, green indicates 4–8 km, and blue indicates 8–12 km. (left) DC-8 and (right) WP-3.

2. Methodology

[3] Figure 2 shows the layout of this study, in which systematic data analysis is used to improve model performance by modifying boundary conditions and emissions inventories through an iterative process which included geospatial interpolation of model bias through kriging to provide qualitative support for regional modification of emissions. After selecting the best model run sensitivity studies are carried out to evaluate the contribution of different ozone precursors to ozone formation.

Figure 2.

Study framework for data analysis and sensitivity studies.

[4] In this study we used the STEM-2K3 model [Carmichael et al., 1991]. The model features the lumped species SAPRC99 chemical mechanism [Carter, 2000] with an online photolysis solver, and the SCAPE II aerosol solver (Simulating Composition of Atmospheric Particles in Equilibrium) [Kim et al., 1993a, 1993b; Kim and Seinfeld, 1995]. Meteorological inputs to the model came from MM5 [Grell et al.,1995], using the AVN data [Huang et al., 1997] during forecasting and NCEP FNL (Final Global Data Assimilation System) analyzed data during postanalysis. For this study the model domain was the continental United States, using a 60 km resolution, 62 cells in longitude, and 97 cells in latitude. The model had 21 vertical layers, extending from the surface to 100 hPa. The Grell cumulus parameterization [Grell et al., 1995] and the MRF planetary boundary layer parameterization [Hong and Pan, 1996] were used for the MM5 runs.

[5] During the operational portion of the experiment, anthropogenic emissions were taken from the U.S. EPA National Emissions Inventory for the base year of 1999 (NEI 1999) [U.S. Environmental Protection Agency (U.S. EPA), 2006] while the 2001 update of the same (NEI 2001) was used for the postanalysis stage. It should be noted that NEI 2001 has lower emissions of CO, NOx, and SO2 than NEI 1999 (Table 1). Modifications were still needed since the simulations with NEI 2001 systematically underestimated light alkanes, and overpredicted aromatic species. The large point source emissions (LPS) used were from the updated inventory by Gregory Frost at NOAA Earth Systems Research Laboratory [Frost et al., 2006], which represents emissions for 2004, the year of the campaign. Upper troposphere lightning NOx emissions were added to the model in postanalysis based on the National Lightning Detection Network (NLDN), relating emissions to signal strength and multiplicity of flashes. Further information about the lightning emissions used is given by Tang et al. [2007]. Biogenic emissions were estimated using BEIS 2 (Biogenic Emissions Inventory System) [Geron et al., 1994], which generates time-variable isoprene and monoterpene emissions driven by meteorological variables from MM5 simulations. Forest fires that occurred during the ICARTT period were largely outside our regional model domain (in Alaska and northwestern Canada). Their episodic influence on lateral boundary conditions was incorporated during postanalysis using MOZART NCAR boundary conditions [Pfister et al., 2005] with data assimilated CO concentrations from Measurements of Pollution in the Troposphere instrument on board the TERRA satellite (MOPITT) to constrain the fire emissions influence for the study period, while during the forecast MOZART GFDL [Horowitz et al., 2003] boundary conditions were used, which did not include episodic fire emissions, but climatological emissions.

Table 1. Total Column Emissions in Model Domain, in Tg/year, Including Point Source, Area, and Aviation Emissionsa
 Emissions, Tg/year
CO (as CO)NOx (as N)VOCs (as C)
  • a

    CO emissions reported as Tg C/year, and NOx emissions as Tg N/year.

NEI 199940.77.115.7
NEI 200125.05.213.5
NEI 2001-Frost LPS26.84.813.8
NEI 2001-Frost LPS*

[6] The postanalysis work focused on improving model performance by carefully comparing predictions with observations, and the use of the error information to identify aspects of the model in need of improvement. Model sensitivity studies were done for factors with significant uncertainty including boundary conditions, anthropogenic emissions inventories, sea salt emissions, lightning NOx emissions, and dry deposition rates. From these various runs three were selected for detailed analysis in this paper. These are (1) operational forecasting conditions (Forecast, NEI 1999); (2) the most updated emissions inventory (NEI2001-Frost LPS); and (3) a modification of that emissions inventory (NEI2001-Frost LPS*) that intended to improve model performance by further decreasing the regional NOx bias. Table 2 shows a summary of the model parameters for the different scenarios compared in this study. Table 1 shows that the total column anthropogenic CO emissions for the domain were reduced by ∼40% from NEI 1999 to NEI 2001, and that the NEI2001-Frost LPS estimate increases CO with respect to NEI 2001. The surface NOx emissions were reduced significantly from NEI 1999 to NEI 2001 (∼30%). Figure 3 shows the domain column emissions of NOx for NEI 1999 (Figure 3, left), NEI2001-Frost LPS (Figure 3, middle), and the decrease of emission from NEI 1999 to NEI 2001-Frost LPS (Figure 3, right).

Figure 3.

Total column NOx emissions. (left) NEI 1999. (middle) NEI 2001-Frost LPS emissions. (right) Decrease in NOx emissions from NEI 1999 to NEI 2001-Frost LPS. Scale is tonnes/km2/year.

Table 2. Summary of Model Parameters for Scenarios Studied During Forecast and Postanalysis
National Emissions InventoryBiogenic EmissionsBoundary ConditionsLightning NOxDry DepositionMeteorology
  • a

    Parameters for NEI2001-Frost LPS* are the same but reducing area NOx emissions by 60% for Alabama, Mississippi, Georgia, North Carolina, South Carolina, Tennessee, West Virginia, Indiana, and Ohio.

Forecast1999BEIS 2MOZART-GFDL, climatological firesnodormant agricultural landsMM5 and forecasted AVN global model
NEI2001-Frost LPSa2001, Frost Large Point Sources. VOCs adjusted by increasing alkanes and alkenes, and decreasing aromatics.BEIS 2MOZART NCAR, burned area biomass burning emissions [Pfister et al., 2005]yesnondormant agricultural landsMM5 and NCEP FNL reanalyses

[7] The results from runs with these different conditions for the period 1 July to 18 August 2004, which includes a model spin up period and which spans the times of the DC-8 and WP-3 flights 3–20 for ICARTT, are discussed in this paper. Merged data for both measurement platforms were resampled from a 1 s to a 3 min resolution, and compared to interpolated data from the 3 h, 60 km, and 21-level variable vertical resolution model output.

[8] Kriging is a geospatial interpolation technique [Oliver and Webster, 1990] that assumes that bias at a point without observations can be related to points with observations, by an expression which considers three components: an average term, a spatially correlated term, and a random error term. The spatially correlated component is calculated using a semivariogram, constructed by the semivariance among observations as a function of distance. Kriging has been previously used for interpolating surface measurements of ozone, and particulate matter for health studies, and estimation of exposure [Liao et al., 2006], to generate maps of air pollution based on discrete measurements, such as the AIRNOW network [U.S. EPA, 1999], and to provide a geographical perspective to ozone analyses [Blond et al., 2003]. Kriging produces a surface of predicted values and uncertainty using a semivariogram (in this case exponential), which relates percent bias (bias/mean observed*100) to distance among points. The analysis is limited to altitudes less than 4000 m. This assumes that the vertical variability in this range is smaller than the horizontal variability. The continuous surface output of kriging provides geospatial context to bias, and allows the comparison of related bias of different precursors to ozone.

3. Data Analysis

[9] The surface ozone forecasted during ICARTT has been compared with surface observations and showed a significant high bias for daytime values (∼15 ppbv) [McKeen et al., 2006]. We anticipated such bias would occur because of the fact that the experiment was conducted in 2004, and that the actual emissions would differ from the 1999 values used in the forecast. The summer of 2004 presented significant fires in Alaska and Canada which were misrepresented by climatological fires used in the forecast. Additionally, it was found in postanalysis that the dry deposition velocities for agricultural crops were incorrectly set to low growing season conditions. These factors all contributed to forecast errors.

[10] Below we compare the forecasted values with aircraft observations. We also compare the results from model runs where the dry deposition velocity has been corrected, and where the emissions and boundary conditions have been updated.

3.1. Statistical Performance for All Flights

[11] The predictions of O3, CO and NOy for the various simulation cases are compared with the DC-8 observations in Figure 4 and Table 3. For these comparisons all data from flights 3–20 are combined together and analyzed by altitude. The predicted values are interpolated to the same spatial location of the observations using trilinear interpolation. In general the forecast values show a significant positive bias in predicted ozone at altitudes below ∼4 km, and a high negative bias above this altitude. The mean bias below 1 km is ∼11 ppbv, similar to, but slightly lower than the values found from the analysis of the surface AirMap observations. Comparable patterns are found in the forecast for CO and NOy, with high values at low altitudes and low values at high altitudes. The postanalysis runs show significant improvements in the predictions at altitudes below 4 km (see for example the correlation coefficients for the 1–4 km range). In the case of ozone the NEI2001-Frost LPS case shows that the low altitude bias is reduced to less than 3 ppbv. The bias in the mid troposphere (4–12 km) is also reduced (by ∼25%). Similar improvements are found for CO, where the bias above 4 km is reduced by updated global boundary conditions (MOZART-NCAR) that improve the correlation coefficient for CO because of their better representation of the biomass burning emissions from Alaska and northern Canada. Tang et al. [2007] evaluated the impact of boundary conditions on model performance using results from three different global models and found that they dominate the performance of the regional model at these altitudes. The remaining bias reflects the performance skills of the global models used. For NOy, the bias in the near surface regions is reduced, albeit at a smaller rate, while the negative bias at higher altitudes decreases significantly. The improvements at the higher altitudes reflect the importance of including lightning NOx emissions. The comparison of the ratio of the standard deviation to the mean value of the predictions and those for the observations shows that the model exhibits a variability that is similar to that of the observations.

Figure 4.

Observed and 60 km simulated O3, CO, and NOy vertical profiles and standard deviations for all DC-8 flights. (a) CO, (b) ozone, (c) ethane, (d) NOy, (e) ethyne, and (f) propane. Blue indicates NEI 2001-Frost LPS*, red indicates forecast, NEI 1999, and black indicates observed.

Table 3. Model Performance Statistics for Selected Speciesa
ForecastFrost LPSFrost LPS*ObsForecastFrost LPSFrost LPS*ObsForecastFrost LPSFrost LPS*Obs
  • a

    Modeled versus observed data, DC-8 platform. 0–12 km range. Frost LPS: NEI2001-FrostLPS. Frost LPS*: NEI2001-Frost LPS*. Modified area NOx emissions. SD, standard deviation.

0–1 km
Mean modeled, ppbv60.1251.8250.3648.921481361361384.163.242.951.85
Mean bias, ppbv11.212.901.459.75−2.48−1.842.701.401.11
S.D./mean modeled0.390.340.330.330.360.
1–4 km
Mean modeled, ppbv58.9754.6853.3257.341311271251192.762.262.090.93
Mean bias, ppbv1.63−2.66−4.0211.375.205.821.831.361.19
S.D./mean modeled0.310.
4–12 km
Mean modeled, ppbv63.7167.9767.8980.20798585980.461.431.431.05
Mean bias, ppbv−16.49−12.23−12.31−19.37−12.64−12.60−0.590.430.42
S.D./mean modeled0.

[12] Similar results are found for the WP-3 comparisons (Table 4). As shown by comparing Figures 1 and 2, the flight operations of the WP-3 and DC-8 were different, with the WP-3 focused largely on the northeast United States. This along with the flight altitude differences lead to differences in the statistics in the observed distributions of the DC-8 and WP-3 data. For example, the WP-3 low altitude concentrations on average are higher. In the case of ozone and CO the mean observed values from all flights were 56 and 158 ppbv for the WP-3 and 49 and 138 ppbv, respectively, for the DC-8. The mean bias in the forecast for ozone for the WP-3 was ∼8 ppbv, compared to 11 ppbv for the DC-8 comparison. Correlation coefficients for both aircraft were very similar for the 0–1 km range (0.71 for DC-8, and 0.63 for WP-3). For the postanalysis simulation NEI2001-Frost LPS the bias in predicted ozone at low altitudes was reduced to 1.34 ppbv and the correlation was increased to 0.66. The mean biases for the lower altitude predicted for CO and NOy were also reduced significantly (by ∼90 and 70%, respectively). For the 1–4 km range the ozone correlation improved (R increased from 0.57 to 0.65) and the mean bias decreased from 4.91 to −0.58. CO predictions also improved with R increasing from 0.52 to 0.66, and bias decreasing from 28.37 to 8.79. Note that the CO bias remains significant for the WP-3 flights, from which we can infer that there is still a systematic over prediction of CO for the area that was sampled by WP-3, i.e., the NE United States. For the 4–6 km range emissions and boundary condition improvements significantly enhanced ozone modeling performance, with R increasing from 0.15 to 0.46, and bias decreasing from −16.19 to −7.83. Similarly CO performance increased because of boundary conditions incorporating biomass burning (R increases from 0.11 to 0.36, and the negative bias is improved from −14.57 to −5.28). It is important to note that while the predicted biases in NOy were reduced significantly in the postanalysis runs, they remain quite high (∼1 ppbv overprediction when averaged over all altitudes for the DC-8 and WP-3 observations). The NOy distributions and their comparison with various models used during ICARTT are discussed in detail by Singh et al. [2006]. In Figure 5 we plot the observed and predicted contribution of individual species to NOy for the DC-8 observations and for the NEI2001-Frost LPS simulations. This plot shows that the predicted contributions are similar to those observed. Nitric acid is shown to compose the largest NOy fraction below ∼4 km, above which PAN contributes from 30 to 45% up to about 8 km; the relative contribution of NO in comparison to NO2 increases with altitude in the upper troposphere (up to 45% of total nitrogen) Within the boundary layer PAN and NO2 each contribute ∼20% to NOy. The predicted distributions differ in comparison with the observations in that the relative contributions of HNO3 are lower than those observed, possibly being linked to overprediction of rainout or washout processes upwind of the sampling. In addition the predicted contribution of NO increases with altitude at a slower rate than observed. This fact is probably related to the treatment of the lightning NOx emissions. Lightning NOx emissions at higher altitudes (4–6 km) improved the modeling of reactive nitrogen species, decreasing the negative bias of NOy in the upper troposphere.

Figure 5.

Comparison of mean contributions to NOy along the DC-8 flight tracks as a function of altitude. (left) Observed values and (right) predicted for the NEI2001-Frost-LPS case. Values are plotted as fraction of total NOy, defined as the sum of NO2, NO, HNO3, HNO4, and PAN.

Table 4. Model Performance Statistics for Selected Speciesa
ForecastFrost LPSFrost LPS*ObsForecastFrost LPSFrost LPS*ObsForecastFrost LPSFrost LPS*Obs
  • a

    Modeled versus observed data, WP-3 platform. 0–6 km range. Frost LPS: NEI2001-FrostLPS. Frost LPS*: NEI2001-Frost LPS*. SD, standard deviation.

0–1 km
Mean modeled, ppbv64.5257.6055.9356.27202.81163.56164.49158.607.775.034.793.91
Mean bias, ppbv8.261.34−0.3444.244.965.893.871.120.88
S.D./mean modeled0.380.310.300.350.310.
1–4 km
Mean modeled, ppbv65.0762.0559.5960.16164.04143.34144.46135.665.003.583.342.38
Mean bias, ppbv4.911.89−0.5828.377.678.792.621.200.96
S.D./mean modeled0.340.
4–6 km
Mean modeled, ppbv50.6159.7458.9766.8092.08101.37101.77106.660.821.341.291.13
Mean bias, ppbv−16.19−7.06−7.83−14.57−5.28−4.88−0.310.210.16
S.D./mean modeled0.

[13] Recurring positive bias of NOy and its components suggest that NOx emissions in the model are still higher than the actual emission in the summer of 2004. While the emissions used in this simulation have the large point source sector updated to 2004, emissions from the other sectors are based on 2001 values. The transportation sector is the major emission sector for NOx and transport emissions have shown a downward trend [U.S. Department of Transportation, 2006]. Continuously monitored emissions of NOx from large point sources have also decreased [Frost et al., 2006]. So it is most likely that NOx emissions in 2004 are actually lower than those in 2001 [Kim et al., 2006]. To reflect this case we performed an additional simulation (NEI2001–Frost LPS* case) where the NOx emissions were reduced by an additional 12% with respect to total NOx on a national level(but by 60% for area NOx emissions for selected states, as is discussed in Table 1). The results of this case are also presented in Tables 3 and 4, and Figure 4. The effect of this reduction in NOx emissions is to further reduce by ∼20% the bias in NOy. Since ozone production in the ICARTT region is largely NOx limited, this reduction in NOx emissions also reduced the mean ozone levels by ∼1 ppbv, and further reduced the bias in the lowest layers by 50% (to 1.45 ppbv for the case of the DC-8), compared to the NEI2001-Frost LPS case.

[14] The modeling of volatile organic compounds (VOCs) has always been a challenge because of the uncertainty of volatile organic compound emissions (VOC) emissions inventories in the United States [Parrish, 2006]. At altitudes above 2 km, all model scenarios showed a negative bias for the prediction for ethane, ethene, and propane, largely due to the global model boundary conditions with contributions from errors from imprecise treatment of convective events. In general R values decrease with altitude reflecting the fact that model performance is highly dependent on boundary conditions at higher altitudes.

[15] Figure 6 shows a quantile-quantile plot of observed and modeled O3 for the DC-8 for measurements under 4000 m altitude range. The forecast values show a systematic overprediction across the whole range, while the NEI2001-Frost LPS* case shows great improvement for values over 55 ppbv, which represented the majority of the points sampled.Figure 7 shows that the frequency for which ozone bias (modeled-observed bias divided by average observed value*100) is between −20 to 20% has increased from 51 to 73% of the time with respect to the forecast, and model error between −5 and 5% has increased from 14 to 23% with respect to forecast. Postanalysis work therefore resulted in a nearly unbiased distribution of ozone errors with a significant reduction in the standard deviation of ozone errors.

Figure 6.

Quantile-quantile plot of modeled ozone with observed ozone for DC-8 platform, data points collected at altitude less than 4000 m, STEM-2K3. Forecast is NEI 1999, and postanalysis is NEI2001-Frost LPS*. MOZART-NCAR boundary conditions.

Figure 7.

Probability distribution of % ozone bias for forecast (NEI 1999) and postanalysis runs (NEI2001-FrostLPS and NEI2001-FrostLPS*) for DC-8 measurements under 4000 m.

3.2. Case Studies

[16] The results above provide a mission wide perspective. NEI2001-Frost LPS* ozone bias is shown on a flight by flight basis in Figure 8. Generally the bias in the lowest layers is less than 5 ppbv, while the bias in the 8–12 km range is large, and particularly high in flights 3, 11, 15, 16, and 17. This is a reflection of the boundary conditions from the global models. Tang et al. [2007] shows that boundary conditions varied greatly among global models, and depending on the global models used the ozone bias in the upper troposphere varied from large negative to large positive values, reflecting differences in the global models in placing stratospheric intrusion of ozone. To show how the model predictions changed from the forecast to postanalysis details for specific flights were also analyzed. Figure 9 shows how model performance improved during DC-8 flights 12 and 14 (25 and 31 July 2004). The plots show that at low altitudes (0–1 km) the model bias is reduced from 8 to 2 ppb for flight 14, and from 22 to 12 ppbv for flight 12, when using improved emissions. For the upper troposphere the model showed a large biases (positive for flight 14, and negative for flight 12), which were decreased significantly by using updated boundary conditions from NCAR MOZART. This improvement is reflected in an increase in the correlation coefficients from 0.65 to 0.84 for flight 14, and from 0.01 to 0.78 for flight 12.

Figure 8.

Mean ozone bias for DC-8 flights 3–20, separated by altitude range, for the NEI2001-FrostLPS* case. Error bars represent standard deviation.

Figure 9.

(left) Time series of observed and modeled ozone along DC-8 flight 14 flight track (31 July 2004). (right) Time series of observed and modeled ozone along DC-8 flight 12 track (25 July 2004). Absence of modeled data denotes that flight went beyond model boundaries. Forecast run is NEI 1999, MOZART boundary conditions. Postanalysis is NEI 2001-Frost LPS*, RAQMS boundary conditions.

4. Analysis of Model Error

[17] The relationship between model errors is a key step in understanding model behavior and identifying model deficiencies. This information is also becoming increasingly important as estimates of error covariance are an important aspect of chemical data assimilation [Chai et al., 2007]. The ICARTT experiment produced observations for a large spectrum of species that are involved in the photochemical oxidant cycle. Thus it is possible to use these data to explore the relationships between the calculated and observed species concentrations with respect to ozone. In this section we analyze two relationships: the correlation of observed ozone concentrations with respect to other measured parameters, and the correlation of ozone bias (or error), with respect to other species. Simultaneous analysis of these relationships provides valuable insight on ozone chemistry, and sources of model error.

4.1. Correlation Between Model Biases

[18] Comprehensive field experiments such as ICARTT provide an opportunity to examine the complex relationships between the processes that govern ozone distributions in the troposphere. Toward these ends we examined the correlations between the observed ozone, and ozone bias to various species using the DC-8 data and the results are presented in Table 5. Shown are results for the postanalysis simulation NEI2001-Frost LPS.

Table 5. Correlation Coefficients of Observed Parameters Versus Observed Ozone and Bias (Modeled-Observed) of Parameters to Ozone Bias, at Different Altitude Ranges for DC-8 Platforma
 Observation Correlation CoefficientsBias Correlation Coefficients.
0–1 km1–4 km4–8 km8–12 km0–1 km1–4 km4–8 km8–12 km
  • a

    Bias calculated with respect to NEI2001-FrostLPS. Ranges 0–1 km, 1–4 km, 4–8 km, and 8–12 km.

ARO2− (n = 6)−0.95
Isoprene−0.03−0.03−0.19 −0.48−0.38−0.34 
Propene− −0.25−0.120.13 
Wind speed−0.16−

[19] We first examine the relationships between ozone and other parameters using all the DC-8 flight data binned by altitude. Table 5 shows that the observed ozone concentrations at low altitudes (0–1 km) are most strongly correlated to HNO3 (R = 0.86), ethyne (R = 0.75), CO (R = 0.70), acetone (R = 0.68), MEK (R = 0.65), NOz (R = 0.77), PAN (R = 0.64) and NOy (R = 0.63). These species represent the general features important in photochemical production, i.e., precursor emissions of CO, NOx and nonmethane hydrocarbons. The correlations are small for short-lived species such as NO, propene and isoprene (−0.03, −0.06 and −0.03, respectively), reflecting that the DC8 observations when averaged over 3 min map out large spatial scales (i.e., 60 km) and thus represent air masses that have been photochemically aged for many hours. These relationships change with altitude, and the correlations decrease in value. For the 4–8 km range, where only the nitrogenous species concentrations show the highest correlation to ozone, with NO2 (R = 0.48), NOy (R = 0.48), NOz (R = 0.47), and PAN (R = 0.39). At higher altitudes (8–12 km) correlations HNO3 (R = 0.67), NOz (R = 0.42), NOy(R = 0.27) and with RH (R = −0.35) and CO (R = −0.24) are the highest.

[20] The decrease in correlations with altitude in the aggregate data reflects many factors, including the fact that the ozone relationships are driven by many processes with distinct correlations. For example stratospheric exchange and convection events result in very different and strong correlations, which show up when the data is first sorted by process, but are lost when all data representing all processes are analyzed together. The high correlations near the surface represent the dominance of the sources and photochemical production processes.

[21] The correlations of the ozone bias with respect to the biases in other species are also shown in Table 5. In general the model biases show similar relationships between species as discussed above for the ozone observations. For altitudes <1 km PAN, CO, MEK, and HNO3 are among the most highly correlated biases with respect to the ozone bias (0.6, 0.51, 0.48, and 0.47, respectively). The correlation relationships for the biases tend to extend up to ∼4 km, above which the values tend to decrease. In the upper troposphere the correlation structure is very different than that at the surface, reflecting the different processes and the strong contribution of the boundary conditions to the biases.

[22] The simple correlation analysis discussed above shows the complex relationships between ozone and the ozone biases, with those relationships in the lower troposphere reflecting the emissions and the photochemical production processes. To better understand these relations, factor analysis was performed on the observational data, on the model predicted values, and on the biases. Factor Analysis (FA) is essentially a variable reduction of data sets consisting of large number of intercorrelated variables into a small number of factors, which account for most of the variance in the original variables [Kulkarni, 2004]. This technique is frequently used in air quality studies to unravel the hidden source information from a rich ambient data set. It is particularly very useful for source apportionment studies when there is no prior information available about the nature of the major aerosol sources affecting a particular receptor station [Seinfeld and Pandis, 1998]. Factor Analysis is extremely useful in identifying the relationships among variables that are driven by common processes such as sources, transport, and chemistry. In addition, the Factor Analysis approach is unaffected by errors in modeled emissions, chemistry, or transport [Millet et al., 2006]. In this section, the results obtained by the application of the Factor Analysis to the observed values, the corresponding model predictions and the model errors for the DC-8 at altitudes below 1 km are presented. The main objective of this analysis is to identify the underlying relationships between ozone and its other precursor species. All the analyses presented in this section were performed using SAS 9.0 software [SAS Institute, 2004]. All rows with missing values were deleted prior to performing the analysis, so all analysis uses identical sampling. The factors were extracted using Eigen value one criterion and the extracted factors were subject to the varimax rotation.

[23] Figure 10 depicts the output obtained from FA analysis. We only show the factor that accounts for the highest variance among the retained factors in each analysis here. Figure 10 (top) in gray shows the factor for the observed species. This factor contains a spectrum of species related to ozone and its precursors, and clustering together those photochemical factors identified in Table 5. Figure 10 (middle) in red shows the output from the same analysis conducted using the predicted values (NEI2001-Frost LPS*). In general the factors identified by the model predictions show many similarities to those based on observations suggesting that the modeled processes are capturing many of the ozone relationships in the real atmosphere. Figure 10 (bottom) shows the output of the factor analysis performed for the model errors. The clustering of errors shows a structure similar to that for the species dependencies. Highest factor loadings appear for primary species such as CO, ethyne and ARO1, as well as for secondary species (the largest factor loading is for PAN). These underlying error structures provide guidance into further model improvements, such as continued improvements in emissions, and improvements in the chemical and physical processes controlling key species such as PAN and HCHO. Furthermore this error structure may also be useful in better definition in the error covariance estimates needed by modern data assimilation techniques as discussed by Chai et al. [2007].

Figure 10.

Factor analyses for DC-8, points in 0–1 km range. (top) Observations. (middle) Model. (bottom) Bias of model with respect to observations. Factor criteria are Eigen value one, varimax rotation.

4.2. Bias in a Regional Context

[24] Geographical context is given to the point bias estimations (modeled-observed) by interpolating them through kriging, generating a continuous surface. Data was restricted to the 0–4 km range, for all DC-8 flights. The previous section suggested which variables need to be improved to lower ozone bias. The interpolated bias surface gives guidance toward where model bias needs to be improved, and qualitatively how changes in modeled inputs affect the bias. The surface of ozone bias is shown in Figure 11. For the forecast (Figure 11, left) we observe that during the forecast there was a positive bias in the central and eastern United States with bias in the range of 10–50%. The biases in CO, NOy, NOx, HNO3, and PAN, which showed strong correlation with ozone bias as discussed previously, were also analyzed. Wherever ozone presented a positive bias in the forecast CO (Figure 11, left), NOx, NOy, and HNO3 (Figure 12, left) also presented positive biases. This is particularly clear for NOy (Figure 12, middle left) where the bias in some regions of Ohio, North Carolina and Virginia are as large as 300–400%. When the emission inventories were modified, updating to NEI 2001 and the large point sources (Frost LPS), the ozone bias decreased to ∼5–10% across the domain (Figure 11, top middle). The bias in CO was reduced through the continental United States, along with the bias of NOy. However, a positive systematic bias persisted for ozone and NOy, and its components, in portions of the domain.

Figure 11.

Kriged ozone (n = 1208) and CO (n = 1001) percent bias (modeled-observed) for alt <4000 m, DC-8 platform. (top) Ozone. (bottom) CO. (left) Forecast, NEI 1999. (middle) NEI 2001-Frost LPS. (right) NEI 2001-Frost LPS*.

Figure 12.

Kriged NOx (n = 1028) and NOy (n = 902) and HNO3 (n = 1157) percent bias (modeled-observed) for alt <4000 m, DC-8 platform. (top) NOx. (middle) NOy. (bottom) HNO3. (left) Forecast, NEI 1999. (middle) NEI 2001-Frost LPS. (right) NEI 2001-Frost LPS*.

[25] The effect of changes in the dry deposition velocities is also apparent for the bias plots for ozone and HNO3. In the regions dominated by agricultural crops (i.e., the central United States) the biases of ozone and HNO3 are significantly reduced. For those species with low dry deposition rates the effect on bias reductions are small (CO and NOx). For example, North and South Dakota, regions with negligible changes in NOx emissions between NEI 1999 and 2001 where ozone bias decreased from 20–50% to −5 to 5%.

[26] As a sensitivity study aimed to further reduce the bias, an additional simulation was conducted with a 60% decrease of NOx area emissions (12% reduction of total NOx emissions) for Alabama, Mississippi, Georgia, North Carolina, South Carolina, Tennessee, West Virginia, Indiana, and Ohio, chosen since these states presented the highest bias in NOy, NOx, and HNO3 according to the kriging results. The results of this run showed enhancement of correlation factors for nitrogen species (Table 3 and 4), particularly for HNO3, and NOy, while decreasing their positive bias. As shown in Figure 11 (top right), the ozone bias decreased to a range of −10 to 10% for most of the continent, with large portions showing bias in the −5 to 5% range. The offshore ozone positive bias persists, but to lesser geographical extent, and lower magnitude than the NEI 2001 scenario, and the forecast scenario. NOy for this same scenario decreases its regional bias to less than 100% over large portions of the domain (Figure 12, middle). Figure 12 (bottom right) shows that HNO3 bias also decreases significantly with the updated and modified emissions, with some areas presenting a negative bias. Figure 12 (top right) shows that NOx bias decreased in South Carolina, North Carolina, and Virginia from 200–400% to −30 to 50%, in accordance to locations where the ozone bias was decreased.

[27] Special notice must be taken of Figure 11 (bottom), which shows that forecast CO, using NEI 1999 presented positive biases of 30–50% for the western portion of the United States, which decreased to 20–30% using the NEI 2001. Negative CO bias over Michigan decreased from −30 to −20% to −20 to −10% as a result of improved boundary conditions. The offshore Atlantic positive CO bias decreased from 30–50% to the 10–30% range and from 10–20% to −10 to 10% for the southeastern United States. These changes are due largely to changes in emissions as this region is largely under outflow conditions for the period of study.

5. Discussion

5.1. Sensitivity Analysis

[28] As illustrated in Figure 2 a variety of sensitivity studies were performed to investigate the sensitivity of the model to other important parameters. The most significant parameters studied in terms of reduction of model bias, dry deposition, emissions, and boundary conditions, have been discussed previously. In terms of near surface ozone the largest impacts are due to dry deposition velocities and emissions, with each contributing equally to the bias reduction. In the mid to upper troposphere, the boundary conditions dominated the improvements.

[29] Simulations were also performed for the eastern United States using a 12 km horizontal resolution. The biggest impact was found near the surface for the 12 km resolution. For the 0–1 km range the mean for the WP-3 increased by 3 ppb with a slight increase in correlation to (R = 0.71). For CO the 12 km resolution increased the mean value by 5 ppb and also increased the correlation coefficient. Biogenic emissions represent an additional source of uncertainty. We repeated a simulation using the BIES3 biogenic emissions algorithm, which led to higher biogenic emissions. Under these conditions the near surface ozone increased by 1 ppbv throughout the eastern United States and CO by 10 ppbv.

5.2. Ozone Production Efficiency

[30] Up to now we have shown that the ozone bias was strongly correlated to NOz and NOy bias. We have also discussed that there is geographical concordance of ozone bias with NOz bias. Previous work has related ozone and NOz [Kleinman, 2005; Trainer et al., 1993], in a relationship for ozone production efficiency (OPE), which for the purposes of this analysis is the slope of the plot of odd oxygen (NO2 + O3) to NOz. In Figure 13 the ozone production efficiency for the observed and predicted values for data points less than 4000 m is plotted. For the DC-8 data the observed OPE is 7.8, while the forecast, NEI2001-Frost LPS, and NEI2001-Frost LPS* cases have OPEs of 6.7, and 7.8, respectively. The observed ozone production efficiency for the DC-8 data suggests efficient formation of ozone, typical of NOx limited conditions. The ozone production efficiency in the area sampled by WP-3 is lower than the DC-8 (Observed OPE = 3.49, Modeled OPE = 5.28), which reflects the fact that the area sampled was closer to emission source regions in the northeast.

Figure 13.

Observed and modeled ozone production efficiency (OPE). (left) DC-8. (right) WP-3 for data points with altitude <4000 m, all flights. Red indicates forecast, NEI 1999, and blue indicates NEI 2001-Frost LPS*.

5.3. Model Sensitivity to Emissions

[31] The fact that the modeled and observed production efficiencies are similar suggests that the underlying ozone relationships of the model are reasonably well represented, and therefore that the results of sensitivity studies may be meaningful. Further insights into ozone production can be seen by comparing the change in ozone to changes in emissions. The mean predicted near surface ozone (0–1 km) values for the NEI 2001 Frost LPS* is shown in Figure 14, in which higher values are found in California, Arizona, and the Atlantic states. The sensitivity of O3 to emissions of precursors is also shown in Figure 15. These simulations were done for the period of 21 July to 18 August 2004, in the absence of anthropogenic VOCs, CO, and reduced NOx. The contribution of each precursor is calculated as the difference between the average daytime surface ozone concentration in the presence of a precursors minus the concentration in the absence of the precursor, normalized by the total yearly emissions of the precursor (ppb ozone formed/(Tg precursor/year)). Ozone formation is most sensitive to changes in NOx (as is also shown in Figure 14), especially in the Midwest, where ozone per Tg of NOx/year increases by 10–20 ppbv (note that NOx sensitivity was calculated on the basis of 30% reduction of total NOx). For reference, 1 Tg of NOx/year is roughly equivalent to the emissions of 28 coal fired power plants of 2500 MW (36,000 tonnes NOx/year each) [Miller and Van Atten, 2004]. In the northeast United States, ozone is equally sensitive to NOx and anthropogenic VOCs, while in large parts of the western United States, ozone is more sensitive to VOCs. CO effects are similar to VOCs, but smaller on a per Tg/year basis. Since CO emissions are larger than VOCs, total CO effects on ozone can be in the same order of magnitude. Figure 15 shows that VOCs and CO contribute to a large portion of ozone formation in portions of the northeast United States.

Figure 14.

(top left) Modeled near surface ozone (0–1 km average) for NEI-2001-FrostLPS* case. Modeled net ozone formation (ppbv/(Tg of precursor/year)): (top right) NOx and (bottom left) anthropogenic VOCs emissions. (bottom right) Anthropogenic CO. Study period is 21 July to 18 August 2004.

Figure 15.

Average surface ozone contribution (ppbv) due to (left) anthropogenic VOCs and (right) anthropogenic CO, calculated as the difference between average 0–1 km ozone for NEI 2001-FrostLPS*, and the same in scenario in the absence of VOCs and CO, respectively. Study period is 21 July to 18 August 2004.

[32] As pointed out above, NOx plays an important role in ozone production. Furthermore an appreciable fraction of NOy is composed of PAN (representing ∼20% near the surface and ∼50% at 6–8 km altitudes, Figure 5), and ozone levels and errors were shown to be significantly correlated with those for PAN. PAN plays important roles as both a key photochemical product and as a reservoir for NOx. To assess the role of PAN on ozone production we conducted a simulation where PAN levels within the regional domain (but not in the boundary conditions) were continuously set to zero. In this way the formation of PAN was allowed, but the thermal decomposition source of NOx was blocked, by setting PAN concentrations consistently to zero. The impact of PAN on predicted mean surface ozone for the month of July is shown in Figure 16 (right). This indirect ozone production pathway of PAN via production of NOx is estimated be over 20% throughout the continental United States with large regions with values between 30 and 50%. This impact extends to all altitudes with values exceeding 8% throughout the domain at an altitude of 5.6 km. Differences become small above this height as the PAN levels are dominated by the boundary conditions values, which were not changed, and the low rates of PAN thermal decomposition due to cool temperatures. The results point out the importance of accurately predicting PAN levels, which requires the close coupling between the regional and global models, as PAN sources and sinks reflect process occurring throughout the vertical extent of the atmosphere and over large geographic scales. Figure 16 (left) shows results for a sensitivity analysis where the formation of PAN itself is blocked. As PAN is a reservoir for NOx under these conditions, with its formation pathway is blocked, a greater fraction of NOx is oxidized into nitric acid, with the net effect of more ozone being formed. Mean surface ozone concentration increases up to ∼7 ppb.

Figure 16.

Calculated impact of the thermal decomposition of PAN on ozone. (left) Average difference between surface ozone with the formation of PAN and with the formation of PAN blocked. (right) Average difference between surface ozone concentration with unconstrained PAN and with PAN concentrations set to zero. Study period is 21 to 28 July 2004.

6. Conclusion

[33] The comprehensive ICARTT aircraft observations were used to evaluate and improve ozone prediction for the STEM model by simultaneously analyzing model performance with conventional statistics, mean vertical profiles, and geospatial interpolation to provide a comprehensive three dimensional context on model performance, laying the groundwork for subsequent model improvement.

[34] The STEM model forecasts of ozone were found to have a significant positive high (overprediction) bias near the surface and a large negative bias in the upper troposphere. These biases were due in part to model errors, which included the use of outdated emissions in the forecasts (i.e., the use of NEI 1999 emissions), out of season dry deposition velocities for agricultural crops, and the use of global model boundary conditions (BCs) that were based on climatological biomass burning emissions. Postanalysis simulations were conducted using corrected dry deposition velocities, NEI 2001 inventory with point source emissions updated to 2004, and BCs from global models using biomass burning emissions reflective of 2004 fire activity. These changes resulted in a decrease in low altitude (0–1 km) mean ozone bias from 11.21 to 1.45 ppbv in comparison to DC-8 observations and from 8.26 to −0.34 ppbv for the WP-3 data. The upper troposphere ozone negative bias persisted, but was reduced in magnitude. In addition, the postanalysis simulations resulted in a nearly unbiased distribution of ozone errors with a significant reduction in the standard deviation of ozone errors.

[35] A series of analyses were performed to study the structure of the ozone errors. These included the correlation of ozone errors with errors in other species. In general the model ozone errors show high correlation for altitudes <1 km with the errors in PAN, CO, MEK, and HNO3. These correlation relationships for the biases tended to extend up to ∼4 km, above which the values decreased. To better understand the relationships among these errors, factor analysis was applied. The clustering of errors shows a structure similar to that for the species dependencies, and included contributions from meteorological features of temperature and RH, precursor species such as CO, ethyne and aromatics, as well as photochemical products such as HNO3, PAN, H2O2 and HCHO.

[36] The spatial distribution of the model errors were examined using Kriging. The geospatial distribution of forecast errors identified distinct features, which when compared to the errors in the postanalysis simulations, helped document model improvements. For example, changes in the dry deposition to crops was shown to reduce substantially the high bias in the forecasts in the Midwest, while the updated emissions accounted for the decrease in bias in the eastern United States. In addition, improvements in boundary conditions from global models, which accounted for biomass burning emissions, improved model performance for CO in the upper troposphere, in comparison to the forecast stage.

[37] Nitrogen species, namely NOy, NOz, and HNO3 showed positive bias during forecast stage, which decreased during postanalysis. Reductions in these biases due largely to emission changes resulted in reduction of ozone bias, especially in the 0–4 km range. However, a persistent high NOy bias suggests that the NEI 2001 NOx emissions are still overestimated. Kriging was shown to provide a geospatial analysis of these biases, and to provide information that can help guide further regional modification of emissions.

[38] Predicted ozone production efficiency (OPE) was studied and shown to be similar to the observed OPE (calculated values of OPE of 7.83 in comparison to observed OPE of 7.79 for the DC-8). The OPE for the WP-3 (observed 3.49, modeled 5.28) was lower than for the DC-8, suggesting that the area sampled by the WP-3 was closer to the source regions of NOx.

[39] Simulations with perturbed emissions found ozone formation to be most sensitive to changes in NOx, especially in the Midwest, where ozone per Tg of NOx/year increases by 10–20 ppbv. However, in the northeast United States, ozone was found to be equally sensitive to NOx and anthropogenic VOCs, while in large parts of the western United States, ozone was more sensitive to VOCs. CO effects were found to be similar to those for VOCs, but smaller on a per Tg/year basis. Since CO emissions are larger than VOCs, total CO effects on ozone can be in the same order of magnitude.

[40] The results presented here help demonstrate how more detailed analysis of errors can improve model performance. These results also point out that the underlying error structure is complicated, but that the underlying error structure can provide guidance into further model improvements, such as continued improvements in emissions, and improvements in the chemical and physical processes controlling key species such as PAN and HCHO. Furthermore this error structure may also be useful in better definition in the error covariance estimates needed by modern data assimilation techniques.


[41] This work was supported by grants from NASA GTE and NOAA. We acknowledge the entire ICARTT science team for their contribution in making their valuable data available.