The road test is widely accepted as the criterion standard for licensing new drivers. Given its face validity, it has also been adapted as a tool to monitor older adults for potential declines in driving ability related to physical illness or cognitive impairment.[1] Test characteristics for on-road tests in cognitively healthy older adults show good interrater and test–retest reliability, internal consistency, and correlation between global ratings and performance scores in research settings.[2, 3] Road test performance is also related to office-based cognitive assessment[4-10] and history of motor vehicle crashes.[11]
Despite this, little is known about the relationship between road test and naturalistic driving behavior. Although the road test occurs in a real-world environment, there are environmental differences between the two settings that may affect its ecological validity. For example, the road test may occur in an unfamiliar or more-complex environment. It is well documented that some older adults with and without cognitive impairment restrict their driving space and may reduce the complexity of their driving environments (e.g., avoid nighttime and highway driving).[12-14] As such, placing participants on a new and possibly more-difficult course may put additional cognitive demands on them. Formal test-taking with potential consequences may also lead to anxiety. In a large study of road test performance in healthy and cognitively impaired older adults, road testing was associated with test-taking fear and anxiety in some participants.[15] In a retrospective review of road test performance of older adults, road test failures substantially increased when individuals were no longer allowed to use their own vehicles during the road test.[16] Older adults may therefore be more susceptible to the negative influences of anxiety and course and vehicle unfamiliarity during the road test. These factors may in turn elicit driving difficulties that are not apparent in their typical environments, resulting in inappropriate failures of individuals who are safe to drive. Conversely, some drivers may be on their “best behavior” while being monitored by an in-car instructor. Moreover, older adults may benefit from being accompanied by a driving instructor who is providing navigation instructions and cueing regarding traffic situations or driving rules. Prior research suggests that cueing enhances performance on the road test.[17]
There are no accepted standards for road tests for older drivers[18, 19] in part because knowledge about older adults' driving behavior is largely derived from self- or caregiver report and not direct observation.[20] Based on survey data, older adults limit their driving to well-known streets, reduce their weekly mileage, and reduce the complexity of their driving by avoiding highways and night driving.[14, 21-23] Recent naturalistic data using in-car recording devices confirm these self-report findings and demonstrate that driving behavior declines when older adults are faced with challenging driving situations and are driving in environments further away from home.[24] When driving was monitored over 1 week, older drivers had fewer demerit points for unsafe driving but made different types of errors, including not stopping at stop signs and turning errors, than younger participants.[25]
The validity of the road test in predicting the types of errors and problems detected using naturalistic methods is largely unknown given the paucity of studies directly comparing road tests with naturalistic driving performance. There are many environmental differences between these two settings, and factor analysis comparing a standardized road test with naturalistic driving suggests that the road test requires a more-limited set of skills than naturalistic driving, raising concern that road tests are not optimally designed to estimate driving competence in older adults.[26] Furthermore, it is unclear whether the structured and cued format of the road test enhances driving performance or reveals problems that would not otherwise have been identified. The objective of this study was to compare a road test with 4 hours of video-recorded naturalistic driving in older adults with a range of cognitive impairment.
Results
- Top of page
- Abstract
- Methods
- Results
- Discussion
- Acknowledgments
- References
Demographic characteristics are presented in Table 1 for participants who passed the road test and had cameras installed in their cars (N = 103). The final group consisted of 44 healthy participants, 41 cognitively impaired participants with a CDR rating of 0.5 (questionable to very mild dementia) and 18 with a CDR rating of 1 (mild dementia). Given the small sample size of cognitively impaired participants with a CDR of 1, cognitively impaired participants were combined into one group for all comparisons. Cognitively impaired participants were older (t(101) = −3.44, P < .001) and less educated (t(101) = 3.58, P < .001) and drove fewer miles per week (t(101) = 5.04, P < .001) than the healthy participants. In terms of driving history, both groups reported few crashes during the 3 years before enrollment (n = 14 overall) or violations in the year before enrollment. There were no differences in crash rate per year (t(101) = −1.08, P = .28) or percentage of violations between the two groups (Fisher test, P = .17).
Table 1. Demographic and Driving Characteristics of Study Sample| Characteristic | No Cognitive Impairment, n = 44 | Cognitive Impairment, n = 59 | Total, N = 103 |
|---|
|
| Age, mean ± SD | 71.2 ± 7.6 | 76.0 ± 6.0a | 73.9 ± 7.2 |
| Male (%) | 38.6 | 49.2 | 45.6 |
| Race (Caucasian, %) | 100.0 | 91.5% | 95.2 |
| Education, years, mean ± SD | 16.3 ± 3.8 | 13.8 ± 3.4a | 14.9 ± 3.8 |
| Mini-Mental State Examination, mean ± SD | 29.5 ± 0.7 | 25.2 ± 2.8a | 27.0 ± 3.0 |
| Years driving, mean ± SD | 53.1 ± 6.8 | 55.5 ± 9.9 | 54.5 ± 8.7 |
| Miles driven per week, mean ± SD | 200.8 ± 114.9 | 98.8 ± 90.5a | 142.4 ± 133.1 |
| Crash in past year (%) | 13.6 | 8.5 | 10.7 |
| Crashes per year/10,000 miles driven, mean ± SD | 0.2 ± 0.4 | 1.4 ± 7.5 | 0.8 ± 5.7 |
| History of violations in past year (%) | 13.6 | 5.1 | 8.7 |
To address any potential biases in the cognitively impaired group, refusers (n = 64) were compared with enrolled cognitively impaired participants. Refusers were similar in age (76.5 ± 6.2; t(121) = 0.47, P = .64) and sex (37% male; χ2 = 2.44, P = .12) to enrolled cognitively impaired participants, although refusers were more cognitively impaired as measured according to the MMSE (23.6 ± 3.2; t(121) = −2.95, P < .001).
Table 2 presents group comparisons of driving outcomes. Cognitively impaired participants made more errors and were rated more poorly on global ratings of safety on the road test and naturalistic driving than cognitively healthy participants. The complexity of driving environments was comparable between the groups despite fewer miles driven per week in the cognitively impaired participant group.
Table 2. Group Differences on the Road Test (Rhode Island Road Test; RIRT) and Naturalistic Driving (Composite Driving Assessment Scale; CDAS)| Driving Assessment | No Cognitive Impairment, n = 44 | Cognitive Impairment, n = 59 | t | P-value |
|---|
|
| RIRT |
| Error score, mean ± SD | 0.04 ± 0.03 | 0.08 ± 0.06 | −4.20 | <.001 |
| Global rating (%) |
| Pass, no recommendations | 56.8 | 20.3 | | <.001 |
| Pass with recommendations | 29.5 | 32.2 | |
| Marginal with restrictions | 11.4 | 30.5 | |
| Marginal with training | 2.3 | 15.2 | |
| Fail | 0.00 | 1.7 | |
| CDAS |
| Error score, mean ± SD | 0.10 ± 0.08 | 0.19 ± 0.13 | −4.08 | <.001 |
| Global rating (%) |
| Pass, no recommendations | 61.4 | 35.6 | | .01 |
| Pass with recommendations | 22.7 | 20.3 | |
| Marginal with restrictions | 13.6 | 20.3 | |
| Marginal with training | 2.3 | 16.9 | |
| Fail | 0.00 | 6.8 | |
| Driving course difficulty, mean ± SD | 5.80 ± 1.15 | 5.68 ± 1.32 | 0.47 | .64 |
In the overall group, ratings on the road test and naturalistic driving showed fair agreement, (kappa = .33),[32] reflecting an effect size in the moderate range.[34] Fifty-three percent of participants passed the road test and naturalistic driving. Only four participants (4%) failed naturalistic driving after receiving a pass with recommendations (n = 2), marginal with restrictions (n = 1), or marginal with training (n = 1) rating on the road test. Error scores in the two environments were correlated (r = 0.41, P < .001), but error score severity was higher in the naturalistic environment (0.15 ± 0.12) than on the road test (0.06 ± 0.05) (t(102) = −8.31, P < .001).
Tables 3 and 4 show specific types of driving errors ranked according to severity in both driving environments; only the 10 maneuvers rated most poorly of the entire CDAS and RIRT measures are presented, for brevity. On average, specific maneuvers were rated as mildly impaired in both environments. Although the CDAS and RIRT do not rate identical maneuvers, inspection of the most-severe error types across environments shows overlap of half of the maneuvers, including checking blind spots, making complete stops, lane keeping, traffic awareness, and speed control. Checking blind spots was the most notable error made in both environments. Between-group comparisons for each of these maneuvers showed that the cognitively impaired participants were more impaired than healthy participants on more maneuvers on the road test (6/10 maneuvers) than with naturalistic driving (2/10 maneuvers).
Table 3. Road Test (Rhode Island Road Test) Items Ordered According to Severity in the Overall Group| Item | Overall, N = 103 | No Cognitive Impairment, n = 44 | Cognitive Impairment, n = 59 |
|---|
| Mean Rating |
|---|
|
| Checks blind spotsa | 0.87 | 0.62 | 1.06b |
| Uses mirrors for lane change | 0.25 | 0.13 | 0.34b |
| Uses mirrors | 0.19 | 0.08 | 0.27b |
| Traffic awarenessa | 0.19 | 0.10 | 0.26b |
| Speed controla | 0.18 | 0.14 | 0.21 |
| Approaches intersection at appropriate speed | 0.17 | 0.19 | 0.16 |
| Makes complete stopsa | 0.17 | 0.12 | 0.21 |
| Proceeds timely | 0.17 | 0.09 | 0.23b |
| Brakes smoothly and accurately | 0.14 | 0.13 | 0.15 |
| Lane keepinga | 0.14 | 0.07 | 0.20b |
Table 4. Naturalistic Driving Errors (Composite Driving Assessment Scale) Ordered According to Severity in Overall Group| Item | Overall, N = 103 | No Cognitive Impairment, n = 44 | Cognitive Impairment, n = 59 |
|---|
| Mean Rating |
|---|
|
| Checks blind spotsa | 0.77 | 0.41 | 1.03b |
| Makes complete stopsa | 0.64 | 0.60 | 0.67 |
| Pays attention | 0.51 | 0.48 | 0.54 |
| Awareness of driving of others | 0.51 | 0.32 | 0.66 |
| Responds to signage | 0.48 | 0.36 | 0.56 |
| Lane keepinga | 0.42 | 0.34 | 0.48 |
| Appropriate response to emergency vehicles | 0.38 | 0.25 | 0.47 |
| Traffic awarenessa | 0.38 | 0.25 | 0.47 |
| Right turn | 0.37 | 0.14 | 0.54b |
| Speed controla | 0.36 | 0.32 | 0.39 |
To determine whether road test or naturalistic driving performance is a more-sensitive measure of driving risk, history of crashes, corrected for miles driven per week during the 3 years before enrollment, was entered as a covariate in a linear regression model predicting error scores on the RIRT and CDAS. Two separate models were run using the RIRT and the CDAS as outcomes. Crash history was associated with error scores on the RIRT (F(1,101) = 7.38, P = .01, coefficient of variation (R2) = 0.07, but not on the CDAS (F(1,101) = 0.60, P = .44, R2 = 0.01).
Disease severity and age were then examined to determine whether there were any demographic or disease characteristics that might affect the relationship between road test and naturalistic driving performance. As expected, global cognitive status as measured using the MMSE was correlated with the road test (r = −0.32, P = .001) and naturalistic driving (r = −0.22, P = .02), with poorer performance on the MMSE associated with higher driving error scores in both settings. Age was correlated with the road test (r = 0.26, P = .01) but not naturalistic driving (r = 0.10, P = .30). When the groups were separated, for cognitively impaired participants, there was no relationship between age and performance on the road test (r = 0.06, P = .64) or naturalistic driving (r = 0.13, P = .32). For the healthy older adults, younger age was correlated with lower error severity on the road test (r = 0.40, P = .01) and greater error severity for naturalistic driving (r = −0.32, P = .04).
Discussion
- Top of page
- Abstract
- Methods
- Results
- Discussion
- Acknowledgments
- References
Individuals with cognitive impairment had higher error scores and poorer global ratings of driving competence on the road test and naturalistic driving than cognitively healthy older adults. Cognitively impaired participants had a greater number of more-impaired maneuvers than cognitively intact participants on the road test than with naturalistic driving, suggesting that the road test may be a more cognitively demanding task than naturalistic driving. These results are consistent with several studies demonstrating that mild cognitive impairment or early Alzheimer's disease affects driving skills even if the person is able to pass a road test.[6, 23] Furthermore, global cognitive status (MMSE) was associated with driving errors in both settings. Taken together, the findings suggest that both methods of driving assessment are sensitive to cognitive impairment beyond the effects of normal aging.
Given the limitations of the road test, it was of primary interest to determine whether performance on a standardized road test was related to individuals' driving behavior in their own environment. Results showed fair agreement between global ratings of safety and error scores on the road test and naturalistic environment, suggesting that the road test is a reasonable proxy for driving behavior in the naturalistic setting for older adults with and without cognitive impairment. This is encouraging and suggests that the road test may not necessarily enhance driving performance through structure and cueing[17] or, conversely, cause problems because of anxiety or course or vehicle unfamiliarity that would not be problems in the participants' typical driving environment.[15]
For both groups, more errors were committed in the naturalistic setting than on the road test, but this did not affect global ratings of safety. It is possible that the driving instructor judged the types of errors detected in the naturalistic setting to be less severe than the errors made on the road test. Alternatively, ratings may differ because the driving instructor had more opportunities to make judgments about errors while watching the videos of the naturalistic driving than during the much shorter driving test. Errors may have also been interpreted differently during the in-car experience than the purely observational ratings made using video recordings for naturalistic driving. Qualitatively, the types of errors made on the road test and naturalistic setting were similar, with half of the error types overlapping between the two environments. Failure to check blind spots was the most egregious error in both settings, and awareness of traffic, making complete stops, speed control, and lane keeping were other error types frequently identified in both settings. These types of errors are generally consistent with other studies showing that older adults with cognitive impairment have significant difficulties with lane checking, lane changing, and merging, all of which require checking of blind spots.[6, 7]
It was of interest to determine whether higher error scores in the naturalistic setting were related to a real-world safety measure. To accomplish this, history of crashes 3 years before enrollment was recorded and corrected for miles driven. This correction was made because past studies have shown that individuals with cognitive impairment have higher crash rates when correcting for miles driven.[23, 35] In the current study, crashes per mile driven were positively associated with road test but not naturalistic driving errors, suggesting that the road test may be a more-sensitive measure of crash risk, but it is important to emphasize that, in this group of participants, history of crashes was rare, and the majority had no crashes. Report of crash history was also self-reported, so the reporting could have been biased. Participants with an at-fault crash within 1 year of enrollment were also excluded, so the highest-risk drivers may have been excluded. It is also possible that the few participants who had crashes biased this relationship. These findings should be replicated in a sample with a greater range of crash history.
As expected, disease severity was related to road test and naturalistic driving ratings. Age had a unique effect on driving performance in each of these environments in the healthy older adults. More specifically, younger age was associated with fewer driving errors on the road test and more driving errors in the naturalistic setting, suggesting that the younger participants were able to modify their behavior on the road test to “best behavior” or that older participants modified their driving environment to reduce errors, but individuals with cognitive impairment did not.
This study has limitations. The evaluation of naturalistic driving was restricted by using video technology. It was not possible to observe directional signal or pedal use, and more overlap in error types might have been seen between the two environments if these behaviors had been measured in the naturalistic setting. The composition of the sample also limits the generalizability of these findings. Individuals who refused to participate in the study were more cognitively impaired. As such, the sample largely comprised participants with cognitive impairment in the mild range, making it difficult to apply the findings to individuals with more-advanced dementia. In addition, cognitively impaired participants were slightly older and less educated than the healthy participants. Both groups were driving in comparably difficult settings during their naturalistic driving, despite cognitively impaired participants driving fewer miles per week, suggesting that any differences in the complexity of naturalistic driving environments did not confound the findings. Finally, crash history and miles driven per week were self-reported and could be biased.
Because of safety limitations, only drivers who passed the road test were allowed to be evaluated naturalistically. Consequently, how drivers deemed to be unsafe on the road test might have performed in naturalistic driving is unknown. The cognitively impaired participant who initially failed the road test was judged to be safe based on their video-recorded naturalistic driving. Conversely, four participants who passed the road test ultimately failed naturalistic driving. Overall, the positive predictive value of safe driving for the road test appears to be acceptable, but the negative predictive value cannot be assessed because the worst drivers who failed the road test did not have an assessment of naturalistic driving. The instructor was allowed to provide advice on driving restrictions (e.g., restrict driving to local areas, avoid driving at night or on highways, drive with a copilot), which could have changed their behavior in the naturalistic setting, although this factor would have biased against finding a significant relationship between naturalistic and road test driving, suggesting that the relationship between the two settings may be more robust than the data indicate.
In conclusion, the road test appears to provide a reasonable estimate of driving in the naturalistic setting in older adults with and without cognitive impairment. Although previous work suggests that different skills may be emphasized in the road test than in the naturalistic environment,[26] those differences do not seem to affect the concordance between overall safety ratings and errors in the two environments. These results do not adequately address whether performance in the naturalistic setting or during a road test is a better predictor of driving safety in this population. Future longitudinal naturalistic studies that include safety outcomes are needed to make this determination.
Acknowledgments
- Top of page
- Abstract
- Methods
- Results
- Discussion
- Acknowledgments
- References
This work was supported by the National Institute on Aging at the National Institutes of Health (R01 AG016335 to BRO).
We would like to acknowledge the participation of the following individuals in this research program: Mr. Timothy Souza from the ABC/ACE Driving School, who performed the road test and ratings of video recorded naturalistic driving assessments; Stephanie K. Knott for assistance in data collection; and the participants who graciously offered their time to the study.
Conflict of Interest: None of the authors have any financial or personal conflicts of interest that would affect the presentation or interpretation of the findings presented in this paper.
Author Contributions: All authors contributed significantly to the work presented in this manuscript.
Sponsor's Role: The sponsor had no direct role in design, methods, subject recruitment, data collection, analysis, or preparation of the manuscript.
1532-5415/asset/JGS_left.gif?v=1&s=b47e5065b3ed0ded5f299582b9ee3e73b3fec48b)
1532-5415/asset/JGS_right.gif?v=1&s=2366bc67bf933998b02c33d9b8e69ce2c5c45808)
