Driving performance, sleepiness, fatigue, and mental workload throughout the time course of semi‐automated driving—Experimental data from the driving simulator

Automation offers the potential to mitigate or reduce the risks related to driving. There are some new challenges for drivers related to semi‐automated driving. Some of them are associated with suboptimal mental workload or prolonged need for sustained attention. This paper presents the results of an experiment investigating differences in manual driving before and after the automated phase in the scenario simulating a time‐course of semi‐automated driving. Sample size: 52 participants with two experimental sessions each day and night session. The experiment used a driving simulator to create a semi‐automated driving scenario comprising manual driving, the automated phase, and manual driving. The following questionnaires were collected: Karolinska Sleepiness Scale, Take‐Over Readiness Scale (developed for this research project, included in Appendix), Samn–Perelli Fatigue Scale, and NASA‐TLX. Driving performance significantly decreased after the automated phase (e.g., standard deviation of the steering wheel angle was 255.73 before vs. 287.11 after automation) and the effect was more profound during the night. Participants were sleepier and more fatigued after the automated phase, and assessed mental workload as lower. The results of the questionnaires did not correlate with driving performance. The results of the experiment suggest that manual driving could deteriorate after the automated phase, and that driver might not be able to assess their fitness to drive at the moment of take‐over of manual driving.


| INTRODUCTION
In 2016 car accidents were the primary death reason for people aged 15-29, eighth reason for death overall, and over 1.35 million people died in car accidents worldwide (World Health Organization [WHO], 2018). During the 3rd Global Conference on Road Safety, ministers of over 100 countries joined the WHO in action to halve road-related deaths by 2030 (Ministers to Agree New Global Road Safety Agenda to 2030, n.d.). Based on NHTSA data, 94% of car accidents can be classified as being caused by the human-machine system error (Melnicuk et al., 2016). Automation offers the potential to mitigate or reduce such risks (Stanton & Marsden, 1996). It can also increase efficiency, alleviate the workload, and improve transport capacity has increased over the past three decades and has significantly improved safety (Chialastri, 2012). Therefore, it is anticipated that higher levels of automation can be incorporated into automobiles to reduce safety risks to road users and pedestrians alike (Kyriakidis et al., 2019). At the same time, there are many concerns related to the role of a human driver in partially automated systems. Even though some functions might be automatized, there is an ongoing discussion about who would be responsible for the system's failure.
Similarly, the ability of human drivers to safely interact with automation is questionable (Hancock, 2019). As a result, along with all the benefits, semi-automated driving introduces a specific set of challenges. For example, staying attentive while using automatic driving mode requires sustained attention and vigilance. Vigilance is the task of maintaining an attentive and alert state, which is crucial for effective and safe take-over of manual control. People find it tiring, hard, and stressful to stay vigilant for longer periods, especially during monotonous tasks (Hancock, 2015;Warm et al., 2008). In a perfect world with no automation failures, the driver would have to monitor the driving process, but would not have to intervene. Such a situation would require long periods of attention without stimulating tasks that could help maintain vigilance. As such, while semiautomated driving requires vigilance, it also creates a difficult environment for its maintenance. Even if drivers restrained themselves from activities not related to driving (e.g., texting or reading), they might still experience cognitive distraction (Liang & Lee, 2010), fatigue or increased sleepiness (Schömig et al., 2015;Warm et al., 2008).
However, not all the experimental data confirm concerns related to automated mode, and some show no performance decrease in highautomation (Merat et al., 2014). Until level 5 automation is released, there will always be a requirement for the driver to either monitor the automation or to take back control at some point (Kyriakidis et al., 2019;Warm et al., 2008;Young & Stanton, 2002). This is related to the fact that automation of the variety of driving functions does not necessarily introduce autonomy of the vehicle. Another potential challenge is suboptimal mental workload. The data show an inverted U-shaped association between mental workload and performance. If automation reduces the amount of mental workload, it could lead to an underload (Heikoop et al., 2016;Young & Stanton, 2007), and as a result, decrease the performance. Decreased performance could mean worse or slower intervention as well as worse driving performance after take-over. Paradoxically, a less demanding automated phase could increase the driving risk. Another risk could be related to night-time semi-automated driving. Accident risk significantly increases at night during manual driving (Matthews et al., 2012;Mitler et al., 1988), and from a review of the literature it has been proposed that such risk might be even more pronounced… in semi-automated driving (Kaduk et al., 2020). Possibly because of circadian changes in fatigue and attention. The increased demand for sustained attention might induce fatigue at any time of the day or night, but night-time is generally related to increased fatigue. Likewise, sustained attention is one of the functions highly affected by the circadian rhythmicity and it gets worse during the night (Kaduk et al., 2020). Owing to the monitoring requirement, the need to takeover and the possible need to intervene remains an important part of the semi-automated system not only as a user but also as an active agent. Because of this, proper human-automation interaction remains crucial for driving safety (Kaber, 2018). It is, therefore, important to understand the effect that automation has on manual driving and driver state. The experiment described in this paper measured changes in manual driving performance related to the automated phase as well as the subjective feeling of sleepiness, fatigue, mental workload, and readiness to take-over. The experiment was conducted at two opposite points of the circadian phase: a high point between 9 a.m. and 1 p.m., and a low point between 10 p.m. and 2 a.m. This might be one of the first attempts to experimentally investigate the influence of circadian rhythms on semi-automated driving. This paper presents the results of the experiment and attempts to analyse how driver state and driving performance change when influenced by automation.

| Experimental design
Human interaction: designing autonomy in vehicles is a research project investigating human interactions with an automated vehicle.
It contains a variety of subprojects related to different aspects of the interaction between humans and automated vehicles. This paper is based on an experiment exploring the psychophysiology of the driver in a semi-automated vehicle. Participants underwent an experimental scenario of manual driving, automated driving, and manual driving again while being recorded with a variety of psychophysiological measures. However, for the scope of this paper, only driving performance and the results of the questionnaire were described and analyzed.

| Equipment
Manual and automated driving tasks were conducted in a low feasibility driving simulator with STISIM3 driving simulator software.
The driving simulator consisted of a set of screens, a driver chair, a steering wheel, and pedals. The experimental setup is presented in

| Procedure
The participants were asked to go through an experimental scenario that was the same for both (day and night) sessions. It contained three measurement points, two manual driving tasks, and one automated task. Most of the physiological measurements were continuously collected over the entire experimental session, but some of them (questionnaires, voice, and saliva) were only collected at the three measurement points. The collected questionnaires were the Karolinska Sleepiness Scale (Åkerstedt et al., 2014;Åkerstedt & Gillberg, 1990), Samn-Perelli Fatigue Scale (Samn & Perelli, 1982), NASA-TLX (Hart & Staveland, 1988), and the Take-Over Readiness Scale (TORS) developed for this experiment (the scale is included in the Appendix). Two manual driving tasks and an automated driving task were performed in the STISIM3 driving simulator. The manual driving tasks were a combination of seven challenging driving scenarios with an alternating order of scenarios. Participants were asked to drive as well and, according, to the rules wherever possible. Each participant received a randomized sequence of the same driving scenarios within the manual driving task; however, the duration of the task depended on the speed and driving style and differed between the individuals. The duration of this task was between 13 and 40 min. The automated driving task was a 34-min long scenario when the car was driving autonomously around the area of Southampton. The participants were asked to stay as attentive as possible and monitor the process. They were also asked to complete an attention task based on the detection of red cars on the road. They were not required to intervene at any point in the automated scenario. The duration of the whole experimental session differed between individuals because of the driving speed, differences in the time of questionnaires completion, differences related to the calibration of the equipment, and comfort brakes. Because of that day experiment started at 9 a.m., and the night experiment started at 10 p.m., but the ending time differed between the individuals and sessions.
The first measurement (M1) was conducted before the tasks as baseline level measurements. This was followed by the very short driving training scenario to avoid mistakes coming from the unfamiliarity of the simulator. After the training, the first manual driving task (T1) was followed. After that, there was a second measurement point (M2) expressing the change in the driver state related to manual driving. This was followed by the automated driving task (A), and a third measurement point (M3) expressing the change in the F I G U R E 1 Experimental setup inside the driving simulator. The cables around the hands, neck, and head are parts of the physiological measurements system. The photo was used with written consent from the participant KADUK ET AL.
| 145 driver state related to the automated driving task. The last part of the experiment was the second manual driving task (T2) that expressed the change in the manual driving performance related to the automation. The time elapsed between manual driving and automation was approximately 10 min. The progressions from manual driving to automation and from automation to manual driving were not immediate. There was a measurement point in between when participants were asked to complete questionnaires and some tasks related to the physiological monitoring (the description of the physiological measurements taken is beyond the scope of this paper). All these measurements were conducted while seated in the driver sit inside the driving simulator. Apart from that participants were offered comfort brakes if needed that could increase the time between the tasks. The time course of the experiment is presented in Figure 2.
There was an expectation that the first experimental session will be more influenced by the learning effect between T1 and T2, so the sessions were scheduled in a way that half the participants would have the first experimental session at night and a half during the day.

| Data reduction and analysis
Driving performance was calculated using the following parameters:  Each participant participated in two experimental sessions with the same tasks and measurements. It was expected and confirmed during the pilot experiment that the results in driving tasks during the first experimental session will differ significantly due to the learning effect. The majority of the participants drove the driving simulator for the first time in their lives. The learning effect was smaller during the second session, and as so the difference between the results in the driving tasks could be addressed more to the driver's state than to the familiarity of the tasks. To avoid the confounding effect of the learning curve only the second experimental session was analyzed for each participant. In addition, the order of the scenarios used in the driving tasks was randomized, which allowed sustaining the same level of the driving challenge but reduce familiarity due to the unexpected order of the road scenes.
An additional analysis was conducted to compare day and night semi-automated driving. It was an attempt to initially validate the previous prediction in the literature of the circadian effect on semiautomated driving (Kaduk et al., 2020). As only the second experimental session were analyzed the circadian comparison used 22 participants for the day session and 30 for the night session.

| Driving performance
The normality assumption was not met (tested with the As an additional analysis, the same comparison was conducted separately for the day and night experiments. For both circadian phases, there was a decrease in driving performance after the automated phase; however, during the night experiment, there were more factors that significantly deteriorated. The significantly differing factors for the day session are presented in Table 3 and for the night session in Table 4. A comparison of the day and night changes in the driving performance between T1 and T2 is presented in Figure 5.

| Questionnaires
The questionnaires' results did not meet the normal distribution assumption; hence, a nonparametric Kruskal-Wallis test with multiple comparisons was used to test differences between scores collected at M1 (baseline level), M2 (after manual driving), and M3 (after automation). Before conducting the statistical tests questionnaires' scores were centralized to keep within-participant information during the between-participant analysis. Centralization was applied to each participant and each session separately. Table 5 presents the p values of the multiple comparisons tests and mean values of the questionnaires' scores for the different measurement points.
Sleepiness significantly increased after the automated phase, but not after manual driving. Participants felt less ready to take-over manual driving after they went through the manual driving task, and then even less after the automated phase. Fatigue increased after manual driving and then almost doubled after the automated phase.
Mental workload showed a different tendency. The majority of scales F I G U R E 5 Percentage change in the driving performance between T1 and T2, day and night comparison. The higher increase in scores represents a larger decrease in driving performance from NASA-TLX showed a trend to increase after manual driving and decrease after automation; however, not all the changes were significant. Only the Effort Scale scores increased after automation, but the increase was not significant. Figures 6-8 show changes in sleepiness, readiness to take-over, and fatigue over the time-course of the experimental sessions.

| Driving performance and questionnaires
The driving performance tended to decrease after using the automated mode. Sleepiness tended to increase as well as fatigue. Participants tended to feel less ready to take-over manual control of the car after using the automated mode. At the same time, the mental workload showed a decreasing trend after activating automated mode. To investigate whether driving performance decreases could be addressed by the changes in the subjective state, Spearman correlations were calculated between all the driving performance factors and all the questionnaires' results. The majority of correlations were nonsignificant, and the significant correlations were low. The values of significant correlations are shown in Table 6.

| DISCUSSION
The experiment might provide some data consistent with previous predictions related to semi-automated driving. Several researchers suspected that automation might worsen the performance of drivers due to low mental workload, the demand for sustained attention, going out of the loop (or in other words not being fully aware of the situation), and fatigue (Hancock, 2015;Kyriakidis et al., 2019;Young & Stanton, 2007). Previous experimental data showed an increase of drowsiness both after manual and automated driving (Schömig et al., 2015). The analysis of the experimental data showed a decrease in driving performance after participants went through the automated phase; however, this experiment did not provide enough evidence to fully differentiate it from effects of time on task, especially as manual driving might also increase sleepiness (Schömig et al., 2015). Moreover, the decrease was more profound during  Stanton, 2007). The increase in sleepiness, fatigue, and feeling of not being ready to drive could also be related to the depletion caused by prolonged sustained attention (Hancock, 2015;Warm et al., 2008).
Iinterestingly, the subjective state of the participants was not correlated with the changes in driving performance. This indicates that participants were driving worse after automation and subjectively felt more tired and less ready to drive, but the magnitude and occurrence of these two processes did not significantly coincide. This could be related to several processes. Subjective sleepiness and fatigue could differ from biological sleepiness and fatigue. However, the information that might have even more profound consequences for driving was that participants were not able to predict how ready they were to take-over manual driving with the TORS questionnaire.
This could indicate that drivers were not able to accurately assess their state or the fitness to drive. Such a lack of awareness could lead to take-overs in risky situations and continuation of driving when it should actually be stopped or postponed. This presents a need for additional methods of driver state monitoring, for example, in-car physiological sensors.
The subjective feeling of sleepiness, fatigue, and decreased readiness to drive could potentially lead to results other than safety risks. Apart from the objective consequences of driving safety, users of the vehicles might also find automated driving less comfortable if it leads to higher fatigue. If they felt less ready to drive, they could disuse automation if it appeared unsafe to them. The influence of automation on the subjective state of the driver would then also require the attention of the manufacturers and could be changed with different variations in the design.
There is an ongoing discussion in the scientific literature about whether driving performance decreases as a result of high automation. Many researchers have predicted that automated vehicles could negatively affect manual driving; however, some did not observe any performance decrease (Merat et al., 2014). In this study a decrease in driving performance was observed after the automated phase. It is possible that it was an effect of the automated mode; however, it could also be caused by other factors, like time on task. Full dissociation of the possible causes of the observed phenomenon requires more experimental work. The decrease in the manual driving performance was not restricted only to the period just after takeover, as the manual driving in this experiment took place approximately 10 min after the end of the automated mode. Participants were not accurate in the assessment of their fitness to drive, which makes such a risk more profound, as people could not decide when to stop driving to improve safety.
This study is not without the limitations that should be under- .34 p = .00 Note: Significant correlations still had low values. Abbreviation: TORS, Take-Over Readiness Scale.