Evaluating system architectures for driving range estimation and charge planning for electric vehicles

Due to sparse charging infrastructure and short driving ranges, drivers of battery electric vehicles (BEVs) can experience range anxiety, which is the fear of stranding with an empty battery. To help eliminate range anxiety and make BEVs more attractive for customers, accurate range estimation methods need to be developed. In recent years, many publications have suggested machine learning algorithms as a fitting method to achieve accurate range estimations. However, these algorithms use a large amount of data and have high computational requirements. A traditional placement of the software within a vehicle's electronic control unit could lead to high latencies and thus detrimental to user experience. But since modern vehicles are connected to a backend, where software modules can be implemented, high latencies can be prevented with intelligent distribution of the algorithm parts. On the other hand, communication between vehicle and backend can be slow or expensive. In this article, an intelligent deployment of a range estimation software based on ML is analyzed. We model hardware and software to enable performance evaluation in early stages of the development process. Based on simulations, different system architectures and module placements are then analyzed in terms of latency, network usage, energy usage, and cost. We show that a distributed system with cloud‐based module placement reduces the end‐to‐end latency significantly, when compared with a traditional vehicle‐based placement. Furthermore, we show that network usage is significantly reduced. This intelligent system enables the application of complex, but accurate range estimation with low latencies, resulting in an improved user experience, which enhances the practicality and acceptance of BEVs.


INTRODUCTION
Battery electric vehicles (BEVs) have the potential to solve future problems regarding greenhouse gas emissions from passenger and commercial vehicles and establish independence from depleting fossil fuel resources. To increase driving range of BEVs, choosing a larger battery capacity is one option. However, a large battery corresponds to high vehicle weight, costs, and demand for rare minerals. Therefore, a trade-off solution, where driving range can be fully utilized, needs to be found. With poor charging infrastructure and inaccurate range estimation algorithms, drivers of BEVs experience range anxiety, which is the fear of stranding with an empty battery. 1 As a consequence, drivers reserve battery energy, so that the vehicle's total driving range is not fully utilized. 2 To counteract this problem, accurate range estimation algorithms are required. These algorithms estimate the remaining driving range by predicting the energy consumed on a given, planned route. If the distance to destination is greater than the remaining driving range, charge planning is needed. Charge planning suggests suitable charging stations along the route, considering battery capacity and required energy for the planned trip. Driving range estimation is a problem that can be solved partially or completely with machine learning (ML). It relies on data distributed between vehicles and cloud infrastructures. In the vehicle, information such as vehicle speed and energy consumption are relevant. Information obtained from external infrastructure, such as live traffic and weather forecasts, is also relevant for the range estimation. Unifying and integrating these sources is an issue that currently hinders further development of range estimation and charge planning software. 3 By applying ML algorithms, many different features can be included in the range estimation, without the need of exact mathematical or physical modeling of the influences. Furthermore, an ML algorithm automatically adapts to changes in system behavior, resulting in a robust model. Not all devices and system architectures are ready for using ML software. This is a problem addressed in the Machine Learning and Systems (MLSys) whitepaper, 4 where the authors discuss problems regarding the widespread use of ML systems in commercial applications, such as: • How can ML algorithms and systems be designed for device constraints such as power, latency, and memory limits?
• How should distributed systems be designed to support ML training and serving?
These and similar problems have been analyzed generally in the context of the Internet of Things (IoT) paradigm and how IoT with its cloud, edge, and fog computing can support the increasing amount of data that is transferred and used, for example, for ML-based systems. 5 An important aspect of system design is the user experience (UX) and how applications can be deployed to ensure high Quality of Experience (QoE). 6 Distributed computing in cloud, edge, and fog systems is highly dependent on communication and exhibits high system complexity. 7 Even with the emergence of 5G, lean communications are still essential and resources need to be used efficiently. 8 Despite these challenges, connected vehicles can achieve significant improvements in efficiency, performance, and QoE with ML and IoT applications. Siegel et al. 9 studied the feasibility of different connected vehicle applications and even identified driving range estimation as an application that could be deployed in connected vehicles. In recent years, fog-and cloud-based ML algorithms have received increased attention. Due to the diversity of ML algorithms and their applications, corresponding systems are analyzed individually. Tuli et al. analyzed a system for a fog-cloud-based object detection with deep learning, 10 as well as a deep learning based smart health-care system. 11 Lin et al. 12 proposed and analyzed a deep learning framework for smart manufacturing inspection systems based on fog computing. In current literature about range estimation algorithms, system architecture and the practicability of the proposed methods is rarely investigated. Researchers frequently propose ML algorithms that rely on big data and distributed computing, yet the analysis from a systems aspect is neglected. In this work, we analyze driving range estimation software and evaluate its performance. Our approach models and simulates the software in a distributed computing setting and enables system architecture evaluation in early stages of the development. Our main contributions are: • We show that performance and QoE of range estimation and charge planning software is highly dependent on system architecture. Therefore, software concepts should be evaluated with respect to system architecture.
• We propose estimating resource requirements based on route length, algorithm time complexity, and the number of instructions in the compiled code. In that way, the software can be tested for different use cases to ensure a sufficient QoE for routes of varying length.
• We analyze system-architectures corresponding to modern commercial vehicles, as well as to concepts proposed in related works. Furthermore, we propose promising alternatives to existing concepts.
• We consider both learning and inference of ML-based driving range estimation. The system analysis is divided into inference and learning. Inference is the task of performing prediction or estimation with a learned model. Most research focuses on the learning procedure, whereas in terms of user experience, inference is just as important. 13 Thereby, the focus of our work is on model inference and the performance of the system as a whole.
The article is organized as follows. Section 2 introduces the state of the art of driving range estimation and connected vehicles and gives an overview of different techniques for evaluating the performance of distributed systems. In Section 3, the necessary hardware and software is described and modeled to enable system simulation. In Section 4, the performance of possible systems is evaluated through simulations. Section 5 concludes the article and provides an outlook.

Range estimation, routing, and charge planning
A driving range estimation algorithm is used to determine the distance-to-empty on a given route. Moreover, range estimation is essential in the planning of long-distance trips, where the driving range is shorter than the distance to the destination and charging stops are necessary. The principle of most range estimation algorithms is to first estimate the energy required for the given route. By comparing the required energy with the battery's estimated state of energy (SoE), the driving range and destination attainability can be determined. Figure 1 shows the process of routing, range estimation, and charge planning as a flowchart. Using street maps and the corresponding graphs, routing algorithms calculate the fastest route from the starting point to the destination. A myriad of routing algorithms exist, the most famous one being Dijkstra's algorithm. 14  algorithm is the A-star algorithm. 15 The A-star algorithm can be implemented with different heuristics that speed up the algorithm, such as highway hierarchies, where highway connections are preferred. 16 For BEVs, it is also common to include energy efficiency in the route calculation (eco-routing). In this case, edge weights representing energy consumption could be negative, since BEVs are able to regenerate energy when decelerating or driving downhill. The Bellman-Ford algorithm 17,18 can handle negative edge weights and has been used in eco-routing. 19,20 The estimation of future energy consumption is frequently performed using ML algorithms. [21][22][23] Thereby, information from the vehicle, such as mass or current tractive energy consumption, is used together with predictive information about the selected route. This information comprises the map data on each link of the route (e.g., legal speed limit, curvature, slope) as well as live traffic data from other connected vehicles. The ML algorithm is trained to find the relation between the available predictive information and the resulting tractive energy consumption. Thus, a range estimation algorithm uses both data from the vehicle and from other connected vehicles via the cloud. Depending on the ML algorithm and the amount of data, the range estimation algorithm can demand a lot of resources. The data origin and the computing latencies require a smart system architecture. Future energy consumption can moreover be estimated using mechanistic models based on physical principles and equations. [24][25][26] In addition, a hybrid model combining a mechanistic model and ML can as well be applied. 19,27 Traditionally, automotive software is implemented on electronic control units within the vehicle and uses mostly data from the vehicle itself. This is also true for range estimation and charge planning. Some research articles, however, have proposed distributed systems with software placed partially or completely in the cloud. Table 1 shows a summary of these related works. The articles are classified by the method used for range estimation and by the system architecture which corresponds to the proposed concepts. We differentiate between ML models, mechanistic models, and hybrid models combining the two. Regarding system architecture, we observed whether the software modules are placed mainly in the cloud, in the vehicle or divided uniformly between the two in a hybrid manner.
In the articles, the system architectures are only vaguely described and the feasibility of the proposed concepts regarding system performance is not investigated. In further research articles, proposed algorithms rely on live-information such as traffic or weather, which is normally not available in the vehicle without some sort of connectivity. 21,23,32-35 However, system architecture and performance are not investigated.

Connected vehicles and vehicular networks
In connected vehicles, communication with other vehicles (V2V) and with infrastructure (V2I) is possible. Mobile internet (4G, 5G) and direct short-range communication (DSRC) are two competing communication technologies used in vehicle-to-everything (V2X) settings, which can be seen in a survey conducted by Abboud et al. 36 Different computing principles are possible in connected vehicles, such as cloud, edge, and fog computing. 37 In the vehicles, resources such as computing performance and storage are limited. In turn, cloud resources are ample and a synergistic distribution in a system with vehicles and clouds is reasonable. Deploying software in such systems is a challenge, as the module placement has a significant impact on their performance and latency. Brogi et al. 38 give an overview of existing methodologies for optimally solving module placement problems in fog infrastructures. The vehicle communication platform Cloud-Think was developed to securely enable software functionality divided between vehicle and cloud. Its system architecture consists of vehicle ECUs as well as data and gateway servers that communicate over wireless connections. 39 Cloud-based vehicle functions have been classified into four different models: only cloud, fall-back method, duplicate function, and elastic application. In this article, we focus on the class elastic application, where the software consists of several modules distributed between the vehicle and cloud. 40 One of the identified use cases includes predictive functions relying on predictive cloud information, where the range estimation operates precisely in such a fashion. Examples for elastic applications are functions using computation offloading. In such applications, computational and/or data intensive tasks, such as image processing and ML, are offloaded to the cloud. This offloading can either be permanent or adaptive, but in both cases the module placement is intelligently performed to minimize parameters such as latency, energy, and cost. 41 There are many examples in the literature where cloud or fog computing is used in vehicular technology. Siegel et al. 42 give an overview of the state of the art of connected vehicles and their applications. Lee et al. 31 use ML to analyze driving behavior in the cloud using data from connected BEVs. Wu et al. 43 use cloud computing in electric vehicle charging control and dispatch optimization. Ozatay et al. 44 implemented a velocity profile optimization with dynamic programming where computationally intensive calculations were performed in the cloud. Saini et al. 45 propose a middleware for vehicular infotainment systems where computation tasks are carried out in the cloud and only relevant content is forwarded to the vehicle. Yaseen et al. 46 perform cloud-based video analytics using convolutional neural networks.
Connectivity is necessary for accurate routing, range estimation, and charge planning. Live route and traffic information are already available in many production vehicles. Traditionally, this is transmitted over the traffic message channel (TMC). Another standard, OpenLR, allows more flexibility and a higher resolution. An overview of live traffic related data formats and protocols is given by Henrickson et al. 47 With improved connectivity, application programming interfaces (APIs) with even greater flexibility are available, where additional information such as weather forecasts can be included. In this work, all external information is requested and delivered through APIs.

Evaluation and simulation frameworks for distributed systems
For optimal system design and module placement, performance evaluation is important. In the early stages of development, measurements are rarely possible, but simulations can be performed to approximately evaluate performance of different system architectures and module placements. However, simulating the performance of distributed systems is a challenging task. Cloud, edge, and fog computing concepts introduce increased complexity with high diversity of devices and high numbers of possible system architecture variants. An overview of simulation scenarios in fog and edge computing is given by Svorobej et al. In addition, their review includes a brief comparison of simulation tools. 48 Cloudsim is a widely used toolkit for the modeling and simulation of cloud computing environments. 49 In an analysis of cloud and fog simulation tools, three of six tools were extensions of Cloudsim. 50 The most used tool according to a citation count is iFogSim, which enables the simulation of cloud, edge, and fog computing settings to evaluate the impact of different module placements and resource management techniques on different QoE and QoS (Quality of Service) metrics. 51 Ghosal et al. 52 defined and proposed metrics for the evaluation of system architectures divided into nonfunctional requirements, degree to accommodate changes, customer requirements, and compatibility to legacy designs. iFogSim fulfills all our requirements, allows for variable modeling of resource requirements, evaluates the correct metrics, and is well established in the scientific community. Therefore, we choose iFogSim for our work.

MODELING OF SOFTWARE AND HARDWARE SYSTEMS
To investigate performance through simulations, the range estimation software and the corresponding hardware must be modeled. In the following sections, the modeling of the software and hardware is discussed.

Hardware and connectivity
In hardware modeling, memory, storage, and bandwidth properties are trivial to determine and model. Modeling the computational performance is not as trivial and several different approaches and metrics exist. One option is to measure processor capacity and load or the number of cores or virtual cores. 53 Alternatively, generic and traditional metrics such as FLOPS (FLoating-point Operations Per Second) and MIPS (Million Instructions Per Second) can be used. 54 Several variants and derivative metrics have been defined and a good overview is given by Wang et al. 55 iFogSim defines hardware units with the following indicators: • processor performance (MIPS) • RAM (e.g., GB) • cost rate per MIPS used (e.g., $) • busy and idle power (W) Each hardware unit is thus modeled with these parameters. MIPS is not a perfect performance indicator, its main defect is that it is architecture-dependent. However, for reduced instruction set computing (RISC), it is an acceptable performance indicator. 56 For the range estimation and charge planning, several dedicated hardware units are needed. Figure 2 shows an overview of the hardware units and their topology. The vehicle's central ECU is the main processing unit, whereas a connectivity module establishes a connection with the backend and handles data transfer over wireless or mobile internet. In the cloud, an OEM (Original Equipment Manufacturer) backend is used for all processing except the route calculation, which is performed in a third-party backend from the navigation provider. Table 2 shows these devices and their specified memory size, processor speed in MIPS, power usage, and instruction set of the processor. In contrast to a dedicated cloud server, an ECU needs to perform multiple tasks simultaneously, that is, a single function does not generally have full access to the processor's performance and memory. Therefore, an available processor speed in the magnitude of 10 3 MIPS is assumed for the ECUs. 56 The backend units should be significantly more powerful than the ECUs; therefore an estimated processor speed of 10 5 MIPS is assumed. 49 One difference between processors of a cloud and a vehicle ECU is the architecture and instruction set. Usually, RISC architecture chips are used in ECUs, whereas CISC (complex instruction set computing) architecture processors are more commonly used in servers and backend units. This means that different compilers are required, which can affect the number of instructions of a software module and is therefore considered in the module placement analysis. 57 58,59 The cost of using cloud instances is estimated to be 1$ per instance per hour, based on commercial cloud services. 49,59 Wired and wireless connections between hardware units in iFogSim are defined by bandwidth and average network communication latency. The vehicle is connected to the cloud via 4G mobile internet. The vehicle ECUs can be connected internally via CAN, ethernet, or similar. 60 It is assumed that the cloud units have a high-speed and low-latency wired connection. In Table 3, the estimated connection speeds and communication latencies are shown. We assume that up to 100 vehicles will simultaneously use the service.

Software modules and resource requirements
In this section, the modules of the proposed range estimation and charge planning software are introduced. Figure 3 shows a software block diagram, where modules and their connections are visualized. The software consists of a user-interface, route calculation, energy consumption estimation, charge planning, as well as a vehicle configuration module. In the following, each step of the range estimation is described.
In iFogSim, software modules are modeled by the number of processor instructions required for the computational tasks, measured in million instructions (MI). The running time or latency for a certain task is then calculated using the processor speed in MIPS. The time required for a processor instruction is dependent on the type of instruction, that is, performing some instruction may take a shorter or longer time than another instruction. The instructions are dependent on the processor architecture, such as RISC or CISC and the number of instructions for a software module is therefore dependent on the processor. Simple instructions running in a single clock cycle is a typical characteristic of RISC architecture, whereas many instructions in CISC architectures run over several clock cycles. To determine the number of instructions, iFogSim or CloudSim give no specific directions. We suggest counting the instructions directly in the compiled assembly code. Another possibility is to measure execution time on a certain processor with a known speed in MIPS and then determine the number of instructions. As some modules are dependent on route length, we use the algorithm's time complexity (big-O notation) to describe the number of instructions as a function of route length. In iFogSim, the modules are defined as described, with some definitions being functions of route length or graph size.

User interface
In this module, the driver chooses the destination in the user interface (UI) and the destination is sent to the route calculation module. The drivers input is a string with the destination address or name. Before the communication is initiated, the vehicle must be authenticated through a TLS (transport layer security) handshake. The vehicle and backend exchange messages to establish a secure connection. The duration of the TLS handshake comprises time for cryptoprocessing, network latency and other delays due to message parsing. 61

Routing
After receiving starting position, destination, and routing options, this module calculates the fastest possible route, with regard to actual traffic information. The performance of a routing algorithm is strongly dependent on the graph that represents the road network. The complexity is dependent on the number of edges |E| and vertices |V| in the graph, as well as its sparsity and the branching factor. For the A-star algorithm, the time complexity is between (|V|) and (|E| + |V| log |V|), depending on heuristic and graph type. 62 In our implementation, we estimate the time complexity to be (|E| + |V|). Thereby, the number of instructions needed for a route calculation in a subgraph with |E| edges and |V| vertices is linearly dependent on the sum |E|+ |V|. Based on real graphs for USA 63 and Germany, 64 we assume that for a unit length route and a given heuristic, a subgraph with V(l) = 1600 vertices and E(l) = 4200 edges is required. In our implementation, the number of instructions for one iteration of the A-star algorithm is 345 I. In addition, a baseline computation of 100 MI is assumed. The total number of instructions is thus: where l is the route length in kilometers. For a calculated route, its attributes for each route segment are sent to the energy consumption estimation. On average, the length of a segment is approximately 200 m, that is, there are N = 5 segments per l = 1 km. 65 For each segment, 2 ⋅ k attributes are stored as doubles (8 Bytes). Additionally, a constant 2000B is used to account for HTTP header and other data and overhead not dependent on route length. The total response size for a route is therefore

Energy consumption estimation
For the calculated route, the route specific energy consumption (EC) is estimated with the ML algorithm using attributes for the planned route as input parameters. In this regression problem, we use a neural network (NN). The learning of the NN is a non-event-based process which runs in the background and is not considered in the control loop, but rather analyzed separately in Section 4.3. The time complexity of the inference is (|N|), where N is the number of segments in the route. In our implementation, inference with NNs can be efficiently performed with 200I.
⋅ N(l) By comparing the required energy with the battery's SoE, the destination attainability can be determined. If the current SoE is sufficient to reach the destination, route and energy information can be sent back to the driver. If the destination is not attainable, a charging point search and a charging plan calculation are triggered. The response includes the energy consumption for each route segment, represented by 2 ⋅ N variables. Thereby, the response size is where the 2000 B represent HTTP header and other overhead.

Charging point search and planner
When triggered, the search finds applicable charging points (CPs) in a geographical corridor along the route. These charging points are then sent to the routing module. With an analysis of the charging infrastructure in western Europe, 66 we established that the mean distance between fast chargers (P ≥ 100k W) along major routes is approximately 50 km, that is, the number of charging points along an l km long route is n CPs (l) = 0.02 ⋅ l.
The complexity of the charging point search is dependent on the number of possible charging points, that is, it is dependent on route length. When implemented with linear search, the time complexity of therefore is (l) and the resource requirements are CPU CPs req (l) = (100 + 10 −5 ⋅ l)MI.
The charging points GPS coordinates are given by two variables and the response size is therefore where the 2000 B represents HTTP header and other overhead. Through all reasonable combinations of these charging points, the fastest route is calculated. Depending on battery SoE and maximum driving range, the number of reasonable charging point combinations for a route of length l can be up to n CP-Comb. (l) = 2 n CPs (l) = 2 0.02⋅l .
Subsequently, the energy for each subroute between the charging points is estimated and charging times at each charger are determined. Thereby, the total travel time is calculated. The charging planner sends multiple requests to the routing and energy consumption estimation, dependent on the number of possible and reasonable routes. Ideally, the routes and waypoints overlap to a certain extent, which means that a previously calculated route and energy consumption can be partially used for another route. In the worst case, all possible routes are different, which means that each route and its energy consumption is calculated individually. The number of possible routes is given by Equation (8)

Route selection and sending
The final step is to choose the fastest route and charging plan and to deliver this information to the user interface.  11) and the response size is assumed to be 1 kB. Sending the final response, that is, the route, range estimation, and charging plan, to the user interface requires 50 MI. The route is represented by one 8 B variable for each route segment, so the response size is where the 2000 B represent HTTP header and the display values for range estimation and charge plan, which are independent on route length. In Table 4, we summarize resource requirements of each module. The complete process of the range estimation and charge planning is shown in Figure 4. The figure shows a sequence diagram describing the process beginning from when the driver enters a destination ending with the display of the route, driving range, and charge plan.
In Figure 5, we further visualize the resource requirement of the software modules for different route lengths. Figure 5(A) shows the response size and Figure 5(B) shows the number of instructions of the route-length-dependent software modules as a function of route length. According to Figure 5, it is clear that the routing algorithm and the energy consumption estimation output the most data and also require the most instructions. This gives a hint that the data output from these modules should not be transmitted over mobile internet connection and that placing these in the cloud would result in lower end-to-end latencies.

PERFORMANCE EVALUATION
To evaluate the performance of the systems described in Section 3, we simulate these with iFogSim. The metrics of greatest importance are: timing, network usage, reliability, availability, flexibility, scalability, expandability, security, energy efficiency, and cost. Timing, energy efficiency, network usage, and cost of cloud execution are all measurable with simulations in iFogSim and monetary cost of network communication can be derived from network usage, if the network provider's conditions are known. The other metrics must be assessed subjectively. The control loop shown in Figure 4 is analyzed with different route lengths and module placements. For the control loop, we measure following performance indicators: • Latency • Network usage • Cost of cloud execution

• Energy usage in cloud and vehicle
An optimal module placement maximizes all performance indicators for all route lengths. In this work, the number of possible and reasonable combinations is low. Therefore, all of these combinations can be analyzed to determine the optimal module placement. For each module placement variation, the performance indicators are calculated. Figure 6 shows four module placement variants between the vehicle and cloud. The placement in Figure 6(A) is the classical vehicle-based placement used as a baseline for the evaluation. The vehicle-based placement is commonly applied in current day BEVs. Furthermore, it was suggested by Yi and Bauer, 24 Scheubner et al., 27 and Ferreira et al. 29 In that placement, all software modules are placed in the vehicle except for the routing algorithm, which is based in the cloud. The routing algorithm relies on real-time information on traffic and road conditions and can be seen as an external service. In the cloud-based placement shown in Figure 6(B), all but the UI and vehicle configuration module are placed in the cloud. In terms of inference, this placement corresponds to the systems suggested by Fukushima et al., 22 Grubwinkler et al., 25,28 and Lee et al. 31 The placement shown in Figure 6(C) is a hybrid between the cloud-and vehicle-based placements, where the routing algorithm and the charging point search are implemented in the cloud and the energy consumption estimation module is placed dynamically both in the cloud and in the vehicle. The second hybrid placement shown in Figure 6(D) is the same as Hybrid 1, except the energy consumption estimation module is only implemented in the cloud. This placement corresponds to the systems suggested by Thibault et al. 19 and Jayakumar et al. 26 In the following, the simulations and their results are discussed. In Section 4.1, the setup of our experiments in iFogSim is presented. In Section 4.2, inference with the range estimation algorithm is analyzed. In Section 4.3, the learning of the range estimation model is examined.

Experiment setup
To analyze the driving range estimation and charge planning software, iFogSim is configured for the simulation of the systems presented in Section 3. In the following, we describe which classes of iFogSim we use to set up our experiments.
A FogDevice is created for each of the hardware units in Table 2 with the given specifications. According to the topology shown in Figure 2, a direct hierarchy of FogDevices is defined. The parent-child pair communication in the hierarchy is configured according to the specification in Table 3. In addition, appropriate Sensors are configured in the vehicle to measure velocity, energy consumption, and so on. Finally, one sensor and an actuator are configured to represent the UI. The latency between a sensor/actuator and the vehicle's central ECU is estimated to be 5 ms. An AppModule is created for each of the modules in the driving range estimation and charge planning software shown in Figure 3. For each of the edges between the software blocks in Figure 3, an AppEdge is created. Each edge carries a tuple that defines the function of the edge. The processing requirements of the edges' tuples are specified according to Table 4, where the variable l is used to specify the length of the route. The edges are event-based and their function is triggered by an incoming tuple from a source software module. To monitor and measure the end-to-end latency of the control loop shown in Figure 4, an AppLoop is specified according to the sequence diagram. With the class ModulePlacement, the mapping of the AppModules on the FogDevices is defined. Thereby, the module placement variants shown in Figure 6 can be configured for the simulations. The results of the simulations include the end-to-end latency of the control loop, as well as the network usage. Furthermore, each FogDevice measures the energy used during the simulation and the cost of using cloud instances is calculated. In the following section, we present an analysis of these results.

Inference
For the driving range estimation and charge planning software, we simulated four different module placement variants, each for different route lengths. The route lengths simulated are {10, 50,100, 200,300, 400,500}km. The mean results based on the simulations are shown in Table 5. The table shows the performance indicators control loop latency, cost of cloud execution, energy usage in the cloud, energy usage in the vehicle, and total network usage, in proportion to the baseline vehicle-based placement. As the simulation includes uncertainty, we show the results in proportion to the baseline and not the absolute values. In that way, the absolute system specifications are of lower importance compared with the ratio of the specifications of different system architectures. For all performance indicators, the cloud-based placement is the best. The control loop latency, cost of execution in cloud, and total network usage are significantly lower in the cloud-based placement than in the other three placements. The energy usage, both in the cloud and in the vehicle, is similar for all four module placements. The first hybrid placement shows significantly better results than the vehicle-based placement,

Placement Latency (-) Cost (-) Energy cloud (-) Energy ECU (-) Network usage (-)
Vehicle-based but still fails to attain the performance of the cloud based placement, which achieves more than 10-fold improvement in latency, cost, and network usage. Figure 7 shows the control loop latency and network usage of different module placements in proportion to the vehicle-based placement for different route lengths. For the driver, these performance indicators are most important in terms of user experience. Figure 7(A) shows the latency of the control loop and Figure 7(B) shows the network usage. For both latency and network usage, the cloud-based placement is consistently better than the vehicle-based placement. The improvement in latency is approximately 10-fold for all route lengths. The network usage of the cloud-based placement is approximately 10 times lower than that of the vehicle-based placement for a route length of 500 km and even lower for shorter route lengths. The Hybrid 1 placement's performance is similar to the cloud-based placement for shorter routes, but with increasing route length the performance worsens and becomes similar to that of the vehicle-based placement. The second hybrid placement's performance is similar to that of the vehicle-based placement, apart from an improvement in relative latency with increasing route length. The cloud-based placement achieves clearly the highest over-all performance.
If we analyze the causes for the differences, the most important factor is the time used down-or uploading data needed for the range estimation and charge planning. In the cloud-based placement, the range estimation uses the route, road, and traffic information directly within the cloud and only transmits the final display values to the vehicle, which are considerably smaller in size. Furthermore, computation in the cloud is faster, which also improves the control loop's latency. Of the four module placement variants analyzed, cloud-based inference is clearly superior. Our proposed system architecture can enable driving range estimation concepts such as those of Fukushima et al., 22 Grubwinkler et al. 25,28 and Lee et al. 30,31 to perform efficiently.

Learning
In the previous section, we observed that inference can be done efficiently when the software modules are distributed between vehicle and cloud; however, the learning of models has yet to be considered. In this section, we compare latency and network usage of two different strategies: vehicle-based (on-device) learning and cloud-based learning, utilizing either batch or online learning algorithms. 67 In related works on cloud-based-driving range estimation, cloud-based learning has to date been the preferred choice. 22,30,31 The training data in the range estimation are streaming data that arrive sequentially with a frequency of up to 10 Hz and can be non-IID. 68 For streaming training data, online learning algorithms are a good choice. 69 With this, the learning can be done with a single pass of the observed data, which is then discarded. Optionally, the data may be saved, which lead to a traditional training data set, enabling the application of batch learning algorithms. A significant benefit of online learning algorithms is that the range estimation model is maintained live and can improve the estimation on-the-go. This can also be seen as a requirement for the software, which makes other learning algorithms undesirable.
As discussed in Section 2.1, some of the input parameters for the range estimation algorithms are traffic, navigation, and weather information. This information is also necessary in the learning of the model. However, only a description of the current situation is needed, which can be observed in the vehicle directly. For example, the temperature can be measured with the appropriate sensor and the traffic can be observed with on-board cameras and other sensors. 27 Stochastic gradient descent (SGD) and its variants are widely used learning algorithms. 70 SGD and variance-reduced versions such as stochastic variance reduced gradient (SVRG) and STRSAGA, can be implemented for the online, single-pass setting. 71 Furthermore, it has been shown that the learning can be implemented in linear time. 72 For the performance of the software, the placement of the learning module must be decided. We compare the options of vehicle-based learning and cloud-based learning.
Consider the number of iterations  needed for an algorithm, such as SGD, to converge to an acceptable level. The time needed for each iteration is   and therefore the total time for the learning of an estimation model in a nondistributed setting is In a distributed setting, the communication between vehicle and backend is also important. If c is the latency of the communication, the total time needed for a distributed learning of the model is if communication in each iteration is assumed. Since algorithms, such as SGD, require many fast iterations, even fast communications result in poor performance, as c ≫   . 73 The learning algorithm can be deployed in a vehicle ECU as well as in the cloud. We estimate the communication latency c based on the data size of the training data for cloud-based learning, compared with the model size for vehicle-based learning. In Figure 8, we show our estimation of the communication cost. In the figure, the size of the 10-Hz training data stream (TD) is shown dependent on driving time in minutes. Additionally, the estimated model size of neural networks with five and 10 hidden layers (NN5 and NN10) is visualized.
The preferred placement of the learning module is dependent on the required frequency of the recalculation of the range estimation. If the required frequency is high, the communication cost of the 1-Hz cloud-based setting is lower than that of the vehicle-based setting. For the 10-Hz cloud-based setting, the communication cost is higher than in the vehicle-based setting if the time interval between recalculations is greater than 20 or 36 s for the five and 10 hidden layer networks, respectively.
To minimize total network usage and communication cost c, the recalculation frequency should be as low as possible and as high as necessary for the optimal user experience. From Figure 8 it can be seen that placing the learning algorithm in the vehicle will reduce the communication cost, as the model size is effectively smaller than the size of the training data. In terms of communication and computation cost, vehicle-based learning can be significantly better than cloud-based learning. In this respect, the vehicle-based ML concepts for driving range estimation by Scheubner et al. 27 and Ferreira et al. 29 can therefore be implemented with our proposed system architecture in an efficient way. Related works such as by Fukushima et al. 22 and Lee et al., 30,31 which have favored cloud-based learning, could benefit from placing the learning modules directly in the vehicles.

CONCLUSIONS AND FUTURE WORK
In this work, we evaluated the performance of an electric vehicle driving range estimation software in different system architectures. Our evaluation is intended for the early stages of the development and enables the comparison of different module placements regarding latency, network usage, cost, and energy usage. By modeling the software and the hardware, simulations with iFogSim could be performed. The results show that for inference, a cloud-based module placement is superior to other investigated placements. The cloud-based module placement is significantly better than the current day baseline, the vehicle-based placement. We estimate that the end-to-end latency from the input of the destination to the display of the route and driving range can be improved by a factor of 10. Furthermore, network usage is drastically reduced. Additionally, we analyzed different settings for the learning of the model and found a vehicle-based learning setting to be a more feasible choice than cloud-based learning, which was favored in related works. The result obtained with our analysis can be used in early stages of the development to rate possible solutions, identify which are promising and which are not practicable. To confirm the conclusions drawn from this work, future work will concentrate on further experiments and empirical evaluations. In doing so, the performance of different system architectures shall be tested using a fleet of connected vehicles. In addition, a strategy for deploying the proposed system in large scale shall be devised.
Here, frameworks such as FogBus, 74 iGateLink, 75 or Aneka 76 can be applied to ensure a secure, scalable, and cost efficient deployment. The criteria reliability, availability, flexibility, scalability, expandability, and security cannot be assessed from our simulation results. Our subjective point of view is that the proposed cloud-based module placement offers more flexibility, scalability, and expandability than a traditional vehicle-based placement, as software updates and changes are simpler in the cloud than in the vehicle. As with all connected vehicle functions, the proposed system is dependent on mobile connectivity, which may have negative impact on availability and thus, reliability. With the extension of mobile networks and the development of 5G connectivity, availability, and reliability continues to improve.
Our research shows that the right system architecture for driving range estimation can improve the user experience significantly, by reducing latency and network usage. Hopefully, more accurate range estimation with improved user experience will drive the acceptance of battery electric vehicles.

ACKNOWLEDGEMENT
Open access funding enabled and organized by Projekt DEAL.