This air quality public information campaign also sought to affect driving behavior to reduce emissions. Assessing drivers' sensitivity to smog alerts requires coupling air quality forecast data with driving behavior data. The driving data come from the Atlanta Regional Commission (ARC). The ARC, the formal metropolitan planning organization for Atlanta, conducted a household travel survey in 2001 and 2002 in order to inform its regional transportation planning. Sampled households were randomly assigned two consecutive days over which each household member was to record their travel information. The data were collected from April 2001 to April 2002, except for July. With a response rate of 66 percent, 8,069 households completed the travel diaries, representing 21,323 persons, 14,449 vehicles, and 126,127 places visited during the two-day sample. See ARC (2003) for further details about the survey administration.
The outcome variable of interest here is daily vehicle-miles-traveled (VMT) by households. The hypothesis tested is that daily VMT falls on alert days, corresponding to a primary goal of the alert program and a prominent metric in transportation planning. Of course, smog alerts might affect other aspects of driving in Atlanta (e.g., frequency, timing and destinations). For brevity and because it is the best proxy for the emitting behavior of policy interest, this analysis focuses on mileage of trips. VMT is aggregated to the household level to control for within-household substitutions and, except where noted, the ARC's sample weights are used. Aggregating driving behavior to the household level and restricting the sample to the 2001 ozone season (May–September) reduces the effective sample to 991.
Using the log of the household's daily miles driven as the outcome variable, the data look noisily distributed across ozone prediction levels. In Figure 3, there does not appear to be an obvious pattern in driving and ozone predictions. More importantly, there does not appear to be a significant discontinuity at the cutoff point, indicated with a vertical line. If anything, perhaps, there is an increase in driving on alert days. Figure 4 shows a similar story with histograms of average outcome variables across forecast levels for a variety of binwidths. The results in Panel A of Figure 4 differ from Figure 3 somewhat because the sample-weighted averages are used in Figure 4. The results do not appear sensitive to binwidth choice.
Several tests for discontinuity are performed. The results of the bootstrapped tests from Nichols (2007), for bandwidths ranging from 3 to 10 and for local mean smoothing or for local linear regression, all fail to reject the hypothesis that VMT is different above the threshold. Depending on the bandwidth and smoothing, (unweighted) household VMTs appear 3–18 percent higher with alerts—though none of these effects are remotely significant. Porter's (2003) nonparametric tests generally reveal similar results, except at small bandwidths (e.g., ≤5). Regardless, the discontinuity is never significant at conventional levels. The insignificant but positive effects at the cutoff appear very sensitive to the sample weighting. Using the sample weighting provided by ARC and examining bandwidths from 3 to 12, the nonparametric smoothing shows 0–17 percent lower household VMTs with smog alerts—though again none of these effects are remotely significant. Figure 5 shows the nonparametric curves for a representative set of bandwidths (3, 6, and 12). The drop-off in VMT evident at high forecasted ozone levels results from few observations in that range. At the least, that drop-off and variation at lower ozone levels are not the result of the smog alert issued at the 85 ppb threshold.
Table 2 summarizes the results across different bandwidth selections. It shows that household miles driven may fall once the 85 ppb threshold is passed, but this effect is very noisy and indistinguishable from zero. This conclusion is not sensitive to the choice of bandwidth. A more focused analysis excludes evening driving, when emissions will not affect that day's ozone levels. This RD analysis is similar, with no significant treatment effects.
Table 2. Estimation Results from RD Analysis of Travel Diary Data
|Variable||# of obs.||Mean at cutoff||Estimated treatment effect (std. err.)|
|N receiving positive weight|
|ln(household miles driven that day), weighted||1,023||3.8204||−0.275||−0.169||−0.088|
|N = 155||N = 183||N = 372|
|ln(household miles driven before 5pm that day), weighted||931||3.0302||−0.247||0.159||0.277|
|N = 139||N = 165||N = 338|
|ln(household miles driven that day), weighted; miles > 500 dropped||1,021||3.8204||−0.121||−0.216||−0.129|
|N = 154||N = 182||N = 371|
|household miles driven that day, weighted||1,200||83.0679|| ||−25.227||−16.254|
|N = 181||N = 211||N = 431|
|household miles driven that day, weighted; miles > 500 dropped||1,198||83.0679||−50.676*||−37.640*||−27.260**|
|N = 180||N = 210||N = 430|
|household miles driven before 5pm that day, weighted||1,145||22.4691||−45.143||12.185||12.117|
|N = 175||N = 203||N = 414|
|household miles driven before 5pm that day, weighted; miles > 500 dropped||1,143||22.4691|| ||−0.565||0.464|
|N = 174||N = 202||N = 413|
|household size, weighted||1,200||2.7439||0.248||−0.194||−0.162|
|N = 181||N = 211||N = 431|
While the smog alerts may not have a significant impact when measuring log-miles, perhaps the effect exists in linear miles. The results across the bandwidth range also fail to reject the hypothesis of no treatment effect. There are a few influential observations, however, and when the two households traveling over 500 miles that day are dropped (perhaps reasonable considering these miles are unlikely to be predominantly in the Atlanta airshed), the evidence of discontinuity becomes much stronger and is now statistically significant at the 10 percent level. The ozone alerts here appear to cause a significant reduction in household VMTs. This significant effect is only found in linear VMT with outliers dropped. It disappears if the analysis is restricted to daytime VMTs, the emissions that might affect peak ozone. The insignificant threshold effects hold for log-VMT regardless of restricting the time of day or dropping the two influential observations.
Table 2 also reports a test for discontinuity in household size. This variable is not expected to exhibit discontinuities around the 85 ppb threshold, and if it did it would call into question the attribution of any treatment effect to the alerts. Household size does not appear discontinuous at the threshold. Other household demographic variables (e.g., age, years of schooling, and income) are also tested for discontinuities around the 85 ppb threshold. The results generally support the use of the alerts as an exogenous treatment as observations on either side of the threshold closely resemble each other. As in the park usage analysis, weather variables can be tested for whether they show a discontinuity at the forecast alert cutoff. Rainfall in particular may affect driving. Here, again, precipitation shows no discontinuity, as it shows no variation whatsoever around the threshold. (In the sample, as is typical in Atlanta, precipitation prevents ozone from reaching high levels.)
The possibility of sorting around the 85 ppb threshold is of some importance here. Sampled households cannot choose which side of the threshold they are on. Yet the sampling design of the ARC travel diary survey leads to observations being dropped if that household took no trips on a given day. This might lead to observations clustering on the lefthand side of the threshold if households responded to smog alerts by taking no trips when ozone forecasts reached 85 ppb. Alternatively, if households tended to respond by taking at least one trip on smog alert days, there might be sorting to the righthand side of the threshold. Given that nontravelers were not sampled, sorting by households onto one side of the cutoff or the other has substantive implications as it suggests the smog alert is affecting driving behavior. This possibility is explored graphically in Figure 6, where the share of observations falling into each ppb “bin” is plotted against local mean-smoothed curves on the right and left of the cutoff (triangle kernel, width of 6). The discontinuity is robust to different constructions (e.g., other width and local linear regression). It appears that some sorting may be occurring where the threshold marks a discontinuous jump in the frequency of observations. A test from Nichols (2007) shows that the density of observations is greater at the threshold. The bootstrap procedure with 1,000 replications to test whether local linear regressions (width of 6, triangle kernel) on the left and right side of the cutoff yield different density values, indicates a statistically significant 0.013 difference. At different bandwidths, the density of observations appears 0.01 higher to the right of the cutoff. This modest sorting is consistent with an increased tendency to take at least one trip on alert days, relative to similar days without an alert. Again, it does not support the hypothesis that the smog alert discourages trip-taking.
In summary, there does not appear to be strong evidence of a negative effect on household VMTs of smog alerts in Atlanta in the summer of 2001. There could be many reasons for this, including offsetting effects of private interests (i.e., driving to avoid exposure) and public interests (i.e., taking transit to contribute to the public good) as described in the theoretical model above.12 Alternatively, the smog alerts may have low salience and low dissemination, or travelers may simply be very inelastic in their short-run travel demand. Perhaps the strongest evidence of a treatment effect comes in the discontinuity in the density at the cutoff. Sorting appears somewhat likely, consistent with households being more likely to take a driving trip on the righthand side of the cutoff than the left.
The results for both the park visitation and the household driving analyses are subject to some important limitations. Both datasets rely heavily on measures of behavior that could include error or even bias. Student observers in the park may mistakenly code some passersby. Travel diary respondents may incorrectly recall their daily activity. In both cases, this approach assumes measurement error in the outcome variables is independent of that day's air quality level. For the travel diary analysis, this dataset—with all its limitations—has been frequently used by Atlanta's regional transportation planners. The park use data, conversely, is a novel observational approach to collecting data inexpensively. Accordingly, it lacks a richer set of demographic control variables (making the RD design even more appealing) and lacks external validity checks. In both cases, as the original datasets had no overt connection or design features related to air quality whatsoever, the threat of air quality issues biasing the data is minimized.