Performance evaluation of IEEE 802.11p, LTE and 5G in connected vehicles for cooperative awareness

In recent years, researchers and scientists are continuously striving to design and develop advance wireless technologies for vehicular communication. Advanced short‐range (IEEE‐802.11p/ITS‐G5) and long‐range (LTE, LTE‐A, and 5G) technologies are indispensable for Vehicular Ad hoc networking (VANETs). Meanwhile, exploiting technologies for cooperative awareness of connected vehicles is important for seamless network connectivity in hazardous situations. For seamless network availability, there is a need to integrate short‐range and long‐range technologies together. In this article, we design a pilot platform based on IEEE‐802.11p, LTE and 5G test network (5GTN) utilizing road traffic and weather information. The use of real‐time and accurate road traffic and weather information together with road condition information is crucial to enhance road safety. To deliver updated road weather and traffic information, we have used a combination of networks. In this article, a comparison of vehicle to vehicle (V2V) and vehicle to infrastructure (V2I) pilot scenarios was carried out considering the mentioned networks. The used IEEE 802.11p acts as a primary vehicular communication channel, assisted by other cellular‐based LTE and 5G networks. Network performance was analyzed by considering latency, packet loss, and throughput in V2V and V2I scenarios. These pilot measurements revealed that the state‐of‐the‐art 5G test network performs better in contrast of IEEE‐802.11p and LTE network by delivering real‐time road weather and road traffic observation data. These pilots filed measurements show the potential of combining short‐ and long‐range technologies for enhanced road safety.


INTRODUCTION
Intelligent Transport Systems (ITS) have facilitated the development of road traffic and weather services utilizing up-to-date real-time observational information collected form vehicles and roadside infrastructures. Road traffic and weather data is one of the ITS service application exploiting road traffic and weather data. These services not only provide convenience in traveling, but they are also crucial traffic safety enhancement 1 The produced real-time weather and traffic data need proper coverage for all road networks, and this is a challenging task in the absence of local road and traffic observation networks. To solve this issue, Road Weather Stations (RWSs) are regularly installed besides main highways and roads. Real-time mobile observations are becoming increasingly popular, that is likely to be advantageous in resolving the absence of road observation issues. 1,2 Moreover, the collected real-time information can be delivered back to vehicles to generate warning alerts related to hazardous situations. For the generation of warning alerts, there is need of a cooperative vehicle platform called Cooperative Intelligent Transport system (C-ITS). The C-ITS system shares information between vehicles and roadside infrastructure in a cooperative fashion. This cooperative architecture certainly helps to reduce road fatalities. It has been shown that the road accidents tendency has also been reduced due to the change in driving strategies, advance roadside infrastructure and vehicles active protection system. 3 To provide road traffic safety using a C-ITS platform, there is need of wireless technology that is compatible with vehicular networks (VN). The vehicular communication uses cooperative facilities for the exchange of information utilizing the short-range networks, such as ITS-G5 (IEEE 802.11p), Wi-Fi, visible light communication (VLC), and long-range cellular networks. As illustrated in Figure 1, there is another solution for an active road traffic management and safety called as next generation V2X (Vehicle to Everything) communications. V2X communication is helpful to enhance the driver's awareness regarding potential hazards as well as the connected and automated (CA) vehicles range. In V2X communication, the information is exchanged between neighboring vehicles, the roadside infrastructure, or pedestrians. VN provide road safety services and applications that relies on exchanges beacon and awareness messages. These messages include the ETSI specified Cooperative Awareness Messages (CAMs) and Society of Automotive Engineers (SAE) specified Basic Safety Messages (BSMs). The messages exchanged between vehicles contain information about the speed, direction, and position of the vehicle, for instance. Currently, there are two leading technologies supporting the first generation V2X communication. The first technology is IEEE 802.11p, and this technology is also a base for the development of ITSG5 by European Telecommunication Standardization Institute (ETSI) and Dedicated Short-Range Communications (DSRC) standards. 2 The IEEE 802.11p is extensively tested and analyzed globally considering V2X communications. Another cellular based competing technology is defined by the Third Generation Partnership Project (3GPP) in Release 14 and 15 named as Long-Term Evolution-V2X (LTE-V2X). 3 3GPP is currently working on 5G-V2X communication platforms to overcome the shortcomings in LTE-V2X. The cellular based technologies can also be referred as Cellular-V2X (C-V2X), and they are built by using a side link LTE/5G radio or a PC5 interface. These interfaces assist to provide a direct communication link without transmitting a data to the cellular network exploiting V2I and V2V communications. Both technologies (IEEE 802.11p and C-V2X) are not compatible, so vehicles are not able to interconnect with each other if they are facilitated with different technologies. This issue in VNs created a new direction of research and debate focused on how these technologies can talk to each other, and which technology should be used. [1][2][3] This involves such as the spectrum allocation at the 5.9 GHz band, and the capability to develop and offer compatibility with the implemented first generation V2X system. The communal and commercial significance as well as the faced challenges in the implementation of non-compatible V2X technologies need an assessment of both technologies in realistic environments. These assessments are essential in order to make decisions related to business and administrative policy making in the interest of public. Several studies have compared IEEE 802.11p and LTEV2X. For instance, the research in Reference 4 introduces a link level evaluation of the IEEE 802.11p, LTE-V2X, and 5G. The IEEE 802.11p technology usually offers good throughput with limited mobility. Although, with few competing nodes using the IEEE 802.11p, the delay in channel access gets high in worst case directly increasing latency. 5 One of the drawbacks in IEEE 802.11p is the limited coverage which can be enhanced by using LTE-V2X and 5G communications. The 3GPP technology combination together with the IEEE 802.11p offers a promising solution for VNs.
The Finnish Meteorological Institute (FMI) is currently developing extensive test case scenarios and environments for ITS safety service applications. One of the developed test environments, called Sod5G, offers a vehicular communication testing site supported by IEEE 802.11p, LTE and 5G cellular networking. As illustrated in Figure 1, a testing environment was developed for operative V2X fleets on main roads underneath severe weather conditions facilitated by IEEE F I G U R E 1 V2X communication alerts using wireless technologies 802.11p, and cellular-based LTE as a part of the Arctic Intelligent Trucks project. 5 Now the FMI is planning to introduce cloud-based services to exchange real-time road weather and traffic data for vehicular networking.
This article presents a novel RWS that has been deployed as a part of the infrastructure aimed to provide testing environments for the road traffic and weather services. The RWS system collects the updated weather data as well as from the different sources, providing forecasts depending on the collected measurement data. By using this data, a novel comparison is carried out of V2V and V2I pilot scenarios by considering IEEE 802.11p, LTE and 5G test networks. This research work is carried out by extending our prior work related to the IEEE 802.11p and cellular enabled road weather and traffic services 6,7 by considering advanced services and prolonged heterogeneous networking. This article also considers an intelligent platform with seamless communication connection for vehicles that can utilize the produced services. The outcomes of this article can be useful for extensive range of stakeholders. For information and communication technology related stakeholders, it describes how the data is collected, processed, and exchanged to enhance road safety. For road administrators, this article can be helpful to understand the possible effects that allows them to prepare for projected changes in road traffic using road weather and traffic information by getting safety alerts as presented in Figure 1. The article is structured as follows: Section 2 describes advanced standardization of vehicular communication. In Section 3 we describe wireless technologies for vehicular communication. In Section 4 we describe vehicular assisted road weather and traffic services. Section 5 provide deployment of a pilot measurement platform. Section 6 provide results evaluation and discussion. Finally, we conclude this article in Section 7.

ADVANCED STANDARDIZATION OF VEHICULAR COMMUNICATION
The standardization of ITS is an important subject for the formal standardization development organizations (SDOs), where vehicular communication is the most dynamic domain of their standardization efforts. The initial standardization was established in Europe in the 1990's with the introduction of electronic fee collection (EFC) and Traffic Message Channel (TMC) by considering the real time traffic information (RTTI). The standardization for vehicular communication using Wi-Fi set out in 2002 (United States) by allocating the spectrum using the 5.9 GHz band. Generally, the vehicular standards have been specified in parallel in Europe and United States. The standardization process was assisted by various research institutions and organizations as well as supported by various (industrial) stakeholders. The standard defined by the US is known as DSRC 5,6 and for Europe it is the C-ITS using ITS-G5 protocol. 6 Both standards are based on the IEEE 802.11p protocol, and that is the reason for several similarities in the standards. Meanwhile, the V2X vehicular communication standards in other regions are different for these two technologies, for example, Japan uses 700 MHz. 7 The International Organization for Standardization (IOS) has developed a standard called CALM3. CALM3 is a system that integrates different wireless technologies into a distinctive system. The 3GPP and International Telecommunication Unit (ITU) have initially commenced the telecommunication and wireless standardization studies. However, we only focus on the DSRC and C-ITS, because both technologies are significant for the intended implementation in the coming years. The most important SDOs in Europe are CEN and ETSI with their several ITS technical agencies. CEN works closely with ISO such as TC 204 and delivers reciprocated standardization. 8 ETSI and CEN work according to the mandate of the European Commission (EC) and they have designed reliable standards for a nominal implementation in Europe. ETSI focuses on a wireless communication system and V2V applications while CEN defines the standards for V2I applications. 9 To certify that the standards have no conflict with the EU standardization activities, CEN and ETSI have developed the European Norms (EN), that were legally certified by the National Standardization Organization (NSO) in EU states. This has been accomplished by an approval released in 2013 by the EC to develop different C-ITS standards. The EU specifications are complemented by different activities that is, consortium for Car-2-Car (C2C). 9,10 This is an industrial consortium of vehicle manufacturers, dealers, and research institutions. ERTICO is an European institute for private and public stakeholders, and CTI, ETSI center for analysis and communication. The C2C consortium has established a draft for the release 1 of C-ITS that limits the huge set of specifications and accompanies the lacking requirements. The automobile industry in C2C consortium (2013) signed an agreement for the establishment of the system. 11 The implementation strategies are developed by the Amsterdam Consortium, 11 this is an alliance of different C-ITS stakeholders in Europe implementing and demonstrating different ITS infrastructure in smart cities and main roads. IEEE 1609 introduces a protocol on top of the PHY and MAC layer of IEEE 802.11. 12 These two standards (1609 and IEEE 802) are generally recognized as Wireless Access for Vehicular Environment (WAVE). On top of this protocol stack, V2X DENM and CAM messages introduced by the by SAE. Generally, the IEEE WAVE specifications, V2X messages, and implementation constraints built a reliable framework of specifications for operational deployment. 5 Similarly, the use cellular technology for vehicular communications is rapidly gaining momentum to provide road traffic safety. There is already a cellular infrastructure providing wide coverage area for ITS applications. DSRC and ITS-G5 could also provide comparable network coverage if they have a similar kind of roadside infrastructure, like cellular networks. The Long-Term Evolution Advanced (LTE-A) is an important initial step towards cloud services for VN provided by different automobile industries. LTE-A has some drawbacks for real-time vehicular applications that is, lacking instantaneous message updates for V2V scenarios. Thus, authors in Reference 13 proposed a possible solution for these issues by integrating DSRC/ITS-G5 together with cellular networks by introducing a heterogeneous communication system. This hybrid communication system is called Cellular Vehicle-to-Everything (C-V2X) that presents the practical methodology, supporting cellular and DSRC/ITS-G5 networking for V2X communication. 13 Hopefully, the next generation mmWave-based 5G technology from 3GPP will deal with these problems and it is likely to offer substantial upgrades for V2X communication that is, more bandwidth with ultra-low latency. 13 The ultra-low latency feature of 5G network can be utilized to exchange the information in V2X by using long-range protocol known as "Uu" and short-range protocol called "PC5" in the 5.9 GHz band (also called C-V2X). The V2X communication can utilize the feature of ultra-low latency by having the Multi-Access Edge Computing (MEC) and using MEC the V2X messages do not go through the base network towards the destination vehicle or infrastructure. 9 The previous studies and field measurements have been performed to analyze the performance and suitability of IEEE 802.11p and LTE in different V2X scenarios. 14 This performance analysis concluded that both technologies are suitable for allowing basic vehicular communication that is, transferring Cooperative Awareness Message (CAM). Nevertheless, these technologies need more advance releases and technologies to meet the latency and reliability requirements of advanced use case scenarios such as automated driving and so forth. However, the advantage of IEEE 802.11p in contrast of LTE is that it has been implemented, analyzed, verified to use this technology for vehicular communication. Though, the benefit of using cellular based LTE is that the similar software and hardware protocols have been utilized. 14 The new technology, for example, 5G, is expected to have an improved performance in terms of latency and reliability as compared to other predecessor technologies.

IEEE 802.11p
The

LTE-V2X
The cellular technology is continuously evolving in near recent years and it has been a good choice for vehicular communication. With the advent of cellular based LTE V2X, the vehicles can exchange accurate positioning information with low latency. 5

5G (mm-Wave)
The use of cellular technology in vehicular communication have been effected by beam selection problems that heavily depends on accurate localization data and complex transceiver chain, resulting in high latency and excessive overhead. To resolve these cellular issues for vehicular communication, the 5G technology is already launched. The 5G cellular systems will also use mm-Wave technology and will play a crucial role in communication. These systems operate in the bands between 30 and 300 GHz, while the carrier frequencies are spread around 60 GHz with a channelization of 2.16 GHz. 10 Through beam forming technologies, mm-Wave will achieve high array gains by implementing large antenna arrays that will help the system to achieve higher data rates typically up to several gigabits/second. IEEE 802.11 is a basic a protocol stack for mm-Wave, that would be able to ensure a data-rate up to 7 Gbps with an end-to-end latency of less than 10 ms. Also, under ideal propagation conditions, mm-Wave systems outperforms IEEE802.11p/DSRC and LTE/LTE-A Pro standards in terms of V2X communication. 11 With the deployment of 5G mmWave networks on large scale would offer wide area coverage for V2X communication. This wide range network would possibly be able to collect and exchange road weather and road observation data between vehicles and roadside infrastructure.

VEHICULAR ASSISTED ROAD WEATHER AND TRAFFIC SERVICES
Establishing an ad hoc connection between driving vehicles in a vicinity has multiple benefits. One of the advantages is that vehicles can broadcast the observation data related to road traffic, weather forecasts, and road hazards (traffic jam, accident etc.), based on in-vehicle deployed sensors. Eventually, a traffic accident notification can be broadcasted to the other approaching vehicles, so that they can avoid more collisions. The road weather and observation data can also be exploited by the ITS service alerts to avoid road traffic accidents. The ITS has assisted in the development of the road traffic and weather services platform collecting real-time observation and forecast information using different Internet of Things (IoT)-based sensors. The FMI has developed a local server and a web site to save the collected road weather and observation data, as illustrated in Figure 2 and Table 1. Likewise, this platform can transfer the collected information for V2X communication by offering instantaneous alerts regarding hazardous situations. In contrast, the number of road fatalities has also been reduced by using advance technologies in vehicles as well as the improved driving arrangements together with advanced roadside infrastructure. 3 For the delivery of information in real-time, different wireless technologies are very much suitable for V2X communication. The V2X communication can work together with the applications and services in C-ITS. So, for the transmission of data, it can use the next generation 5G technology having numerous Radio Access Technologies (RATs). Because the use of conventional vehicular technology, IEEE-802.11p has been analyzed in depth and this have been verified with real-time measurements that IEEE-802.11p performs better in contrast of formerly employed Wi-Fi technology. 6,7 For basic vehicular communication applications and services, the IEEE-802.11p fulfills almost all the basic requirements. Though, having a small number of competing entities in iEEEE-802.11p, the latency to access the channel may possibly increase. 13,14 One of the drawbacks F I G U R E 2 ITS-assisted road weather services (14) TA B L E 1 Road weather and road traffic observation data

DEPLOYMENT OF A PILOT MEASUREMENT PLATFORM
This section discusses the pilot platform deployment in a realistic operational environment to analyze the performance of IEEE 802.11p, LTE and 5G test networks. The deployment of the pilot system is performed by tailoring V2V and V2I use-case scenarios exchanging the road weather and traffic data collected from vehicles and RWS-RSU, as presented in Table 1. The combination of RWS-RSU platform provides real-time updates of weather and road friction observation data collected by various IoT-based sensors installed in RWS-RSU and in-vehicle on the test-track in Sodankylä, Finland. Figure 3 illustrates the test-track facilitated by the IoT sensors assisting two RWS-RSUs and vehicles during pilot measurements. Communications take place between RWS-RSUs and vehicles using IEEE-802.11p, LTE and 5G test network. The communication process is initiated by using a Python program (2.7.3) 16,17 and it has been utilized during all the communication development process in V2V and V2I scenarios. The RWS-RSU basically runs two threads: One thread runs the weather monitoring script, and the other thread runs the message distribution script. The first thread just monitors the weather, and it has been saved in the table in RWS-RSU that can be viewed and written by the messaging script. On the other side, the in-vehicle PC also runs the Python script in parallel with other elements, that collects the information from the nearest RWS at regular intervals. The data is saved with time stamps and the data can be differentiated by the time stamps to exchange the most recent data. For the pilot filed measurements, we have utilized two RWS-RSUs enabled by IEEE-802.11p and LTE together with the 5G test network. Two vehicles have been used for pilot measurements on a 1.7-kilometer-long test-track. The vehicles are driven on the test track in a close loop, as shown in Figure 3, interacting with the other vehicles and the RWS-RSU units. 18 For the deployment of this pilot platform, Cohda MK5 and MK6 radio transceivers have been utilized and a commercial smart phone compatible with 5G test network is used. For in-vehicle user interfacing, a vehicle PC known as SUNIT Fseries is used. As another option, Android-based tablets can also be used as a possible solution for UIs. When starting the vehicle application program, the user chooses the transmission protocol (UDP/TCP), the communication protocol (WiFi/802.11p), the delay between messages and the delay for the program start-up. Figure 4 presents the RWS-RSU connection with vehicle OBU and FMI local server.
The 5G test network base station works in a standalone manner, with no connection with the public wireless network. For road observation field measurements, the sensors like Teconer RCM411 and WCM411 are used together with other in-vehicle sensors. Different IoT sensors are also installed on test-track delivering weather forecast data and road condition data (road temperature, state and condition i.e., friction) information, as presented in Table 1.

F I G U R E 5 UDP packet capture on test track using different wireless technologies
The pilot measurements have been performed by conducting 28 test drives, each for V2V and V2I scenarios. The two vehicles were piloted in a same lane across each other and following each other on a test track by exchanged updated forecast and road friction information. The vehicles were driving in a same lane at an approximate distance of 20 to 200 m. The communication link between vehicles and RWS-RSU was working most of the time during field measurements. Even though there were few spots where the link was weak, and the packet were lost that ultimately affects the latency. Figure 5 presents the UDP packet capture using IEEE 802.11p, LTE and 5G test network. As illustrated in Figure 5, the UDP packet capture has been performed during pilot measurements in V2V and V2I scenarios and analyzed by LabVIEW software. For these pilot measurements, we have used a standard data packet structure for RWS and vehicle data packet generation. This data packet format is sufficient to provide important road traffic and weather data for vehicular communication. The RWS-RSU data packets are having a length of 1348 bytes and the data packet of vehicles are having a length of 790 bytes. The vehicle data packet is small enough to transmit the real-time road weather and friction information for V2V and V2I scenarios. 19,20 Therefore, the network throughput is slightly low when transmitting data packets continuously.

RESULTS EVALUATION AND DISCUSSION
In this section, we discuss the pilot field measurements considering IEEE-802.11p, LTE and 5G test network in V2V and V2I scenarios. We have compared these technologies by exchanging real time road weather and friction data by evaluating the latency, packet loss and throughput, as presented in Figure 6 and Table 2.  Figure 6 shows the latency calculations from the pilot field measurements. The latency is calculated by using the (1).

Transmission latency (Ts) = PS/TR,
where PS is the packet size and at transmission rate TR where NP = number of packets. Figure 5 reveals that the IEEE-802.11p has the highest latency during our field measurements and it is because of restricted mobility and range on test track. The latency was initially slightly high due to the connection initialization process in IEEE-802-11p and LTE. The LTE performed better in contrast of IEEE-802-11p by having low latency in exchange of road weather and observation data in V2V and V2I scenarios. Similarly, Figure 6 and Table 2 illustrate that the performance of 5G test network is better than the other two competing technologies due to ultra-low latency and better network coverage on test track. The short-range IEEE 802.11p and long-range LTE and 5G test network also provides the heterogeneous platform for continuous transmission of real-time road weather and road observation data in V2I and V2V scenarios.
It can be seen that in IEEE 802.11p, LTE and 5G test networks, the window of data transmission is marginally synchronized, and the increase in the speed affects the communication window by shorting the window size. Nonetheless, the communication window size for all the considered field measurements is evidently large enough to support the mutual RWS-RSU infrastructure. Moreover, the window size also affects the packet loss. Figure 7 shows the packet loss during road weather and observation data exchange between V2V and V2I scenarios. The UDP packets for V2V communications are of larger size than those for V2I communications. For pilot filed measurements, UDP data packets are used for a continuous communication and due to its robust nature. Additionally, we have also managed the packet loss by using our Python developed program. It can be observed in Figure 6 and Table 2 that the IEEE 802.11p has a highest packet loss and 5G test network has a lowest packet loss during pilot field measurements. Figure 7 also reveals that the packet loss in three technologies is slightly high in the beginning specially in IEEE 802.11p due to connection initialization. This packet loss is also affected by the network latency, as shown in Figure 6 and Table 2 Figure 8 illustrates the throughput comparison of IEEE-802.11p, LTE and 5G test network. Figure 8 and Table 2 also show that the 5G test network has the highest throughput in contrast of other two used technologies. The performance of 5G test network is the best among the used technologies for pilot field measurements. This is due to the enhanced packet loss robustness at access-layer, improved communication bandwidth as well as the modulation technique with dual carriers. However, the 5GTN still requires time to rollout completely for vehicular applications in contrast of LTE and IEEE-802.11p.

CONCLUSION
Vehicular communication is an important domain of ITS dealing with the exchange of safety messages between vehicles and roadside infrastructure to improve the road safety. Aiming at improving road safety, we have performed a road a system-level comparison of IEEE 802.11p, LTE and 5GTN. The said wireless technologies have been assessed in terms of packet loss, latency, and throughput by exchanging road weather and observation data in V2V and V2I scenarios. For comparative analysis, the pilot filed measurements have been performed on a test track in an Arctic region of Finland having RWS-RSU infrastructure facilitated by the IEEE 802.11p, LTE, and 5GTN cellular networks. The field measurements illustrated that the 5GTN performed well among all used technologies. The performance of IEEE 802.11p is adequate in operational environment with standard transmission frequencies with restricted network coverage. Another factor affecting the performance of IEEE 802.11p is the lack of channel access synchronization and distributed congestion management techniques according to the network situation that is, traffic load on network and vehicle speed and so forth. Moreover, the field measurement results show that the LTE performed well in contrast of IEEE-802.11p but having lower network performance in contrast of 5G. The LTE performance is also affected due to the lack of instantaneous message updates for V2V scenarios. The performance of 5G test network was superior in contrast of IEEE-802.11p and LTE. It is because of low latency and less packet loss that eventually enhances the average throughput. Nevertheless, the IEEE-80.211p, LTE and 5GTN meets the minimum application requirements having latency less than 100 ms to deliver safety-critical information in V2I and V2V scenarios. In this study, we have investigated the advanced features of 5G technology that is, Reliable low-latency communication supported by IEEE 802.11p and LTE networks. This study also offers a platform to consider the opportunity to create a heterogeneous networking platform that offers the best of short/range IEEE 802.11p long/range LTE, 5G networks.

ACKNOWLEDGMENT
I express my sincere thanks to Kari Mäenpää and Timo Sukuvaara for their assistance in the above-mentioned measurements at Finnish Meteorological Institute (FMI), Finland.