Driver behaviour detection using 1D convolutional neural networks

Driver behaviour is an important factor in road safety. Computer vision techniques have been widely used to monitor the driver behaviour. The violation of privacy and the possibility of spooﬁng are two continuing challenges in camera-based systems. To address these challenges, we propose an efﬁcient approach to monitor and detect driver behaviour based on movement characteristics of the vehicle rather than the visual features of the driver. The main goal of this paper is to classify the driver behaviour into ﬁve classes: safe, distracted, aggressive, drunk, and drowsy driving. A lightweight 1D Convolutional Neural Network with high efﬁciency and low computational complexity is suggested to classify the driver behaviour. Experimental results conﬁrm that our method could successfully classify behaviours of a driver with accuracy of 99.999%.

Driver behaviour is an important factor in road safety. Computer vision techniques have been widely used to monitor the driver behaviour. The violation of privacy and the possibility of spoofing are two continuing challenges in camera-based systems. To address these challenges, we propose an efficient approach to monitor and detect driver behaviour based on movement characteristics of the vehicle rather than the visual features of the driver. The main goal of this paper is to classify the driver behaviour into five classes: safe, distracted, aggressive, drunk, and drowsy driving. A lightweight 1D Convolutional Neural Network with high efficiency and low computational complexity is suggested to classify the driver behaviour. Experimental results confirm that our method could successfully classify behaviours of a driver with accuracy of 99.999%.
Introduction: Abnormal driving behaviour (e.g. drowsy, aggressive, drunk, careless, and reckless driving) is defined by the driver behaviour, which increases the risk of the accidents [1]. Most driver behaviour detection systems only identify one type of abnormal driver behaviour [2][3][4], whereas few papers attempt to distinguish among the different types of driver behaviour [5]. Nevertheless, there is still no driver behaviour monitoring system that can efficiently distinguish between different abnormal driver behaviours.
In order to clarify different styles of driver behaviour, we summarize the various characteristics of driving styles in the following.
Aggressive driving behaviour includes unsafe lane change, quick change in car acceleration and speed, tailgating (drive too closely behind another vehicle) [6,7].
Distracted driving style is associated with inattention to necessary activities and task of driving toward other activities such as drinking, eating, and using smartphone or technologies in the car [6]. Distracted driving usually followed by a quick driver reflex to rectify car situation.
Driver fatigue usually leads to the drowsy driving style that is accompanied by observable symptoms such as yawning, closing eyes, slower reactions and steering, rare use of brake, and lower revolutions per minute (RPM) [6].
Using alcohol or drug affects the driver mental ability and leads to the drunk driving style that is accompanied by measurable symptoms such as lower use of brake, abrupt acceleration, and dangerous lane change [5,6]. Finally, safe driving style could be identified by detecting the characteristics of risky driving styles.
The major contributions of our paper are as follows: (1) we propose a three-phase approach based on a lightweight 1D-CNN to detect the driver behaviour from vehicle signals instead of monitoring the driver's face. (2) The many similarities in the characteristics of driving styles have made it harder to identify and classify them, as many papers just identify one driving style. In this paper, by identifying the appropriate and discriminant features of different driving styles, we have classified them into five categories: normal, distracted, aggressive, drunk, and drowsy driving.
Despite many researches in this area, identifying the discriminative features for the driver behaviour analysis and gathering the data are still two challenges. For example, discriminating between drowsy and distracted driving or aggressive and drunk driving is a cumbersome task. Consequently, selecting such feature space to make these styles of driving more separable is very critical. In contrast to our work, The most previous works focused on detecting the abnormal behaviour, instead of five different styles driving. In next section, we propose an approach to detect and classify the driver behaviour into five categories: safe, distracted, aggressive, drunk, and drowsy driving.

FIG. 1
The driver behaviour detection system, including three layers: data gathering, pre-processing, and decision-making. In first layer, data are gathered from vehicle and environment. In pre-processing phase, data are prepared for decision-making by windowing and data standardization. In third layer, driver behaviour is detected and notified the driver, police, insurance and other vehicles Proposed method: Our proposed approach for detecting the driver behaviour comprises three major steps: data gathering, pre-processing, and decision-making. The driving data and environment information are collected in duration of different driving scenarios, then to make these data ready for analysis, several pre-processing approaches are done on these data. Finally, a CNN method is exploited to distinguish the driver behaviour. The proposed approach is illustrated in Figure 1.
Data gathering step is responsible for collecting the required data to interpret the driver behaviour. These data are sensed from the vehicle and environment using different devices: Vehicle acceleration: The acceleration is calculated and collected in three axes (x, y, z) by an accelerometer sensor of a smartphone. The position and orientation of the smartphone are fixed to maintain the experiment in a controlled condition. In addition, we calibrate it to transform the coordinate and realign the axes.
Road slope: We use the GPS of a smartphone to calculate the road slope. Road slope S is calculated based on following equation in time interval of t 1 and t 2 .
Where G t1 and G t2 are GPS altitude at t 1 and t 2 time, respectively. D 1 and D 2 are trip distance at t 1 and t 2 time, respectively.
Vehicle engine RPM: it is related to the number of engine revolution in 60 seconds. An On-Board Diagnostics (OBD) device is used to collect RPM value from Engine Control Unit (ECU).
Vehicle speed: Speed is extracted from the ECU using an OBD adapter.
Throttle Position: it is related to the amount of gas pedal push, which is extracted from the ECU using an OBD adapter.
Steering angle: it is gathered from a smartphone, which is fixed on the centre of steering wheel.
Distance to front vehicle: This parameter is related to how much the driver is close to front vehicle intentionally or unintentionally. This parameter can indicate the braking rate and the car-following behaviour of a driver. Some modern vehicles measure the gap distance between vehicles by sonar or ultrasonic. We use a technique to measure the distance to the front vehicle by using a camera, which estimates this distance by taking the image size of the front vehicle's license plate. Due to the standard size of license plate, we can estimate the distance to front vehicle.
Road curve: This is an important parameter in driver behaviour detection technique because it specifies whether vehicle acceleration and steering angle occurred because of road maze or driver abnormal behaviour. We estimate road curve by variation of GPS latitude and longitude.
Road speed limit: It is used to set the minimum or maximum speed that vehicles may legally travel on particular stretches of road. It can be obtained from Google Maps or OpenStreetMap. Speed limit is not updated or set up in some road therefore, we edit them manually.
Traffic: Traffic condition is extracted as feature from Google Maps. It shows traffic situations in real-time on main street and roads by various colours. Generally, green, yellow and red colour means, no traffic delays, In pre-processing phase, we prepare the dataset for decision-making phase. We use StandardScaler class of Scikit-learn library to standardize features by deleting the mean and scaling to unit variance.
We consider driver behaviours in a time slot. In [4], authors considered a 5 sec time windows for drunk driving detection; in [8], authors considered a 20 s time windows with a 50% overlap for aggressive driving detection; in [9], authors considered a 1.8 s time windows to identify the distracted driver behaviour. Experimentally, we applied the window of 5 sec with 98% overlap. Driving signals were gathered in the time domain and were modelled using a deep learning technique to identify the driver behaviour pattern.
In decision-making phase, we use 1D-CNN technique to distinguish various driver behaviours: safe, distracted, aggressive, drunk, and drowsy.
We use a new trend of CNN model to achieve the best accuracy and computational cost in driver behaviour identification. A major different architecture of CNN architecture used is to avoid a very deep fully connected network, which has a lot of learned parameters. To reduce the number of network parameters, we do not use very deep fully connected layers. Instead, we apply a global-max-pooling layer and a tiny fully connected network. This novel trend of CNN presents low computational complexity, high performance, and less potential to overfitting in applications. Since driving signals have structured patterns and higher temporal properties; compared with conventional natural images and signals, only a small number of convolutional layers are required. We experiment different architectures with various filter sizes and number of filters.
All input signals are matrix 50 × 15 (i.e. 15 is the number of features and 50 is the signal length). Initial layers of the network (i.e. convolutional and maxpooling) extract the features, whereas the final layer (i.e. softmax), classifies samples into one of the five classes. Dropout is a regularization technique to reduce overfitting in ANN by randomly omitting units during the training process. We apply a dropout of 0.25 after the second convolutional layer. The 1D-CNN architecture is illustrated in Figure 2. In experiments, we examine the influence of different numbers of filters and size of filters on the computational complexity and accuracy of our proposed method.

Experiments:
We made a driving dataset in real condition to test our proposed method. We chose a real scenario instead of simulation based scenario to avoid possible deviations from the real driving behaviours. We selected different roads with different conditions such as slope, traffic, and curve to set up the experiment. Three participants drove with a similar car in a predefined scenario. We asked drivers to imitate driving styles, including safe, distracted, aggressive, drunk, and drowsy. To imitate various driving styles, Participants performed the predetermined activities, which are explained in the following.
Aggressive driving was imitated by tailgating, moving with high speed, braking, taking off, and rapidly changing the speed. Distracted driving was imitated by a dummy activity to distract the driver attention from the road for an interval between 1 and 2 second [9]. For this purpose, driver was asked to type a text message with about 30 characters on a smartphone. The driver should carry out some activities to imitate drowsy behaviour such as closing eyes for about 1-2 s, lazy steering, and low change in acceleration or gear.
To imitate drunk driving, participants should carry out some activities such as high speed, abrupt acceleration, and quick steering. Unlike aggressive drivers, drunk drivers have lower awareness of dangers and rarely push the brake pedal. Finally, normal drivers should obey some rules such as normal speed, normal steering, safe acceleration, and safe distance to front vehicle.
In order to identify small oscillation of steering and speed change, frequency of data collection is set to 10 Hz. In other words, each 100 ms one record is collected, including variables that are listed in Table 1. Figure 3 illustrates a 5 s window of driving signals. In data pre-processing phase, statistical operators are applied on these signals to prepare them for the learning phase. Table 2 shows the details of this dataset. Experimental results and discussion: We present the efficiency evaluation of our proposed method in this section. The driver behaviour detection system is evaluated as a multiclass classification problem. The dataset was split into two subsets: 75% for training set and 25% for the validation set. We set epoch size to 25 and batch size to 32.
We use Adadelta as the optimization method with default parameters (i.e., decay of learning rates ρ = 0.95 and initial learning rate = 1.0).
We use the accuracy and confusion matrix to evaluate the efficiency of our proposed method. Accuracy is the ratio of correctly predicted instances to the total instances. The number of parameters in CNN network is a criterion for measuring the computational complexity of the network.
The number of parameters and accuracy of various CNN models are listed in Table 3. According to the trade-off between the number of parameters and accuracy in Figure 4, the 5th network is the most appropriate model for embedded or mobile applications because it has higher accuracy and lower computational complexity in comparison to other models. This model has two convolutional layers, filter size of 11 × 11, and four filters in the 1st layer. The validation accuracy of the 5th model is 99.60% and the validation loss of the experiment is 2.04%. Regardless of the number of parameters, some models in Table 3 achieve higher accuracy. For example, the 10th model with eight filters and filter size of 11 × 11 has the accuracy of 99.999%. It can be deduced  Table 3 that the size of filters is the most effective parameter on the accuracy of models. In addition, the number of filters is the second most effective parameter on the accuracy of models. To showcase the efficiency of the proposed method, confusion matrix of the 5th model is illustrated in Figure 5.
From our standpoint, [5,10,11,4,12,8] are the most closest researches to our paper. However, they cannot classify driver behaviours into five classes as we have proposed in this paper.
Researchers in [12] use Neural Network (NN) and Support Vector Machine (SVM) to detect abnormal behaviour of a driver such as fast turning and braking. The accuracy of their model for 20 different drivers varies from 92.44% to 95.36%. In contrary to our method, this paper just detects the aggressive driver behaviour. [8] proposed a driver behaviour detection system using Gaussian Mixture Model (GMM). They have achieved success rate of 92.65% for identifying only the aggressive driving, while we have successfully detected the five driving styles with the accuracy of 99.999%.
In [4], a drunk driving behaviour detection method was proposed without using alcohol sensor. This research introduced the drunk driving patterns. This paper inspired us to mimic the drunk driving style. They have evaluated the performance of detecting the abnormal speed and curvilinear movement, which in their opinion are the two main symptoms of drunk driving behaviour. False Positive (FP) and False Negative (FN) rate of abnormal curvilinear movement are 0.49% and 0% respectively. FP and FN rate of speed control problem are 2.39% and 0% respectively. However, they just concentrate on drunk driving behaviour, whereas abnormal speed and curvilinear movement might symptoms of aggressive driving behaviour. Therefore, we consider more features of driving pattern such as RPM, throttle, and brake to distinguish between drunk and aggressive driving behaviours.
In [5], a system is proposed, which can detect different driver behaviours, including normal, reckless, fatigued, and drunk driving. The proposed system is based on the environmental information and signals of vehicle and driver. This paper is the closest work to ours. They propose a method to detect four types of driving behaviour. In contrast to our work, they evaluate their system on synthetic data and they have no test bed equipped with the sensors to collect the data. Consequently comparing our method with this approach is not possible.
In [10], the authors use Gated Recurrent Units (GRU), Long Short Term Memory (LSTM), and Simple Recurrent Neural Network (Simple-RNN) to identify the aggressive behaviour of a driver. They achieved an accuracy of 95% for LSTM-and GRU-based approach and an accuracy of 70% for Simple-RNN-based approach. Unlike our approach, their system cannot classify behaviors of a driver into five distinct categories. Therefore, there is no basis to compare the results of our approach with theirs that uses deep learning methods but with quite different architecture for just detecting the aggressive behaviour of a driver.
In [11], authors use LSTM and CNN to classify behaviours of a driver.
They use driving signals, including speed, acceleration, deceleration, lane change, and corner turn. Different CNN and LSTM models were experimented by changing the number of neurons, layers, and epoch. The highest performance was obtained with CNN architecture about 96.1%. However, they have just focused on only aggressive behaviour of driver, while our method classifies the driver behavior into five styles and achieves the accuracy of 99.999%. In addition, our proposed method has less computational complexity than their method.

Conclusion:
In this paper, we proposed a simple yet efficient approach based on CNN to detect different driving styles. It includes three main steps, including data gathering, pre-processing, and decision-making. In data gathering phase, we set up an experiment to collect driving data in different conditions. Collected data are pre-processed by applying time windows. A 1D-CNN-based method is used to classify the driver behaviour, including safe, distracted, aggressive, drunk, and drowsy driving. Finally, driver behaviour detection system can warn the driver, police, and other vehicles. The evaluation results indicate that our proposed method has high efficiency and low computational cost. Future works include extension of the number of participants, investigating the unsupervised learning algorithm, and empowering the model with online learning.