SREM: Smart renewable energy management scheme with distributed learning and EV network

In this article, aiming to develop the Green Internet of Vehicles (G‐IoV), we propose a smart energy management system that leverages the intelligence edge clients and the distributed electric vehicles (EVs). The system proposed in this article incorporates the benefits of both software, specifically in terms of the user interface, and hardware, specifically in terms of edge clients. In particular, this system integrates intelligence edge clients with an EV CAN bus network as an electronic control unit. By leveraging the intelligent edge clients recommendation system, EVs can make informed decisions on battery charging or discharging actions. As a result, a virtual‐power‐plant (VPP) can treat the EVs network as a vast intelligent energy storage facility, efficiently managing the battery energy of all distributed EVs connected to the platform and fully utilizing the electricity generated from renewable energy sources. We experimentally verify that using federal learning to train models in EV networks versus training models directly in EVs, using federal learning in EV networks yields better experimental results.

applications.Nonetheless, the number of facilities will thereby have explosive growth in the smart world.2][3] Over the past centuries, fossil energy sources, such as coal and oil, have been widely used for power generation and research purposes due to their high energy content.However, the use of these traditional energy sources leads to the emission of greenhouse gases, including carbon dioxide, which contributes to global warming.With energy demand projected to reach 56% of 2010 levels by 2040, 4 it is imperative to reduce the use of fossil fuels and increase the use of renewable energy sources to proactively address the effects of global warming and reduce carbon emissions.Therefore, the concept of G-IoV 5 is proposed.Normally, there are two directions for achieving the G-IoV.One is to develop energy-efficient IoV facilities or architecture for reducing energy consumption.Another is to utilize the extra electric power produced by renewable clean energy resources (such as solar and wind farms) as much as possible.Since the first solution is highly relied on the innovation of technologies in many fields, it is difficult to satisfy the requirements in the short term.Thus, from the perspective of energy resource, the shift from oil and natural gas to renewable energy (renewables hereafter) is a promising way to develop the future G-IoV.However, the fact is that converting energy systems to renewables and improving energy efficiency are still big challenges for the current energy sector.The reasons are as follows: 1.A large number of distributed units need to be intelligently coordinated to provide a reliable energy supply 6 ; 2. The power producers and consumers are remotely separated, and the technology to store power energy is limited.
Thus, the existing one-way structure cannot be sensitively responsive to power demands; 3. To address the concentrated peak power demand or increase the use of renewable energy, additional power equipment needs to be added outside the current power system, 7 where power demands are concentrated.
But, if the peak load is not reached, then it will lead to energy waste.Hence, a smart energy management system is required to address such pressing needs.Given the explosive growth in the number of shared electric vehicles (EVs), [8][9][10] the continuously increasing EV batteries offer significant potential for energy storage in the coming decades, which may provide a promising solution to alleviate the aforementioned issues.
However, since the current mobile edge computing (MEC) is still at a low level, it is difficult for EVs to make autonomous behaviors.Thus, one of the main challenge for EVs is how to collaboratively schedule the electricity from the distributed batteries to the power grid.Fortunately, leveraging the technologies of cloud computing and Internet-of-Things (IoT), the virtual power plant (VPP) 11 has been acknowledged as a promising digital platform that links geo-distributed energy resources aiming to optimize energy-usage efficiency.The metaverse is characterized by community participation and shared governance. 12A decentralized energy community is created in which residents and participants can jointly manage and make decisions about the allocation and use of energy resources.Not only it allowsaggregationg thousands of units including electricity producers, consumers, and storage facilities, a VPP also enables bidding their power and flexibility into different markets by intelligently controlling their feed-in and consumption.By an interacting with the producers and consumers, the VPP could reduce energy waste much more efficiently than the traditional power management methods.Even though some VPP initiatives have started their businesses on a commercial basis in some counties such as Germany and Japan.However,the current VPP is still facing issues in low flexibility and the limited ability to utilize renewables.This is because it has to build many stationary energy storage units for storing the electricity from renewable resources.
In this article, we exploit the infrastructure of VPP for the energy management of G-IoV networks and utilize artificial intelligence (AI) technologies to enhance the intelligence of mobile EV edge clients.To provide a comprehensive overview, Figure 1 illustrates the holistic framework of the IoV-VPP-based approach proposed in this article.It is evident that IoV-VPP networks rely heavily on AI techniques and data analytics.This AI-empowered autonomous framework incorporates many emerging technologies, such as MEC, real-time telemetry, correlation analysis, and decision-making.Based on real-time status information from EVs and the power grid, the EV fleets will learn how to manage energy demand (ED) allocation Then, the VPP center outputs the ED shortage after deducting the energy supplied by the IoV-VPP system.The final ED shortage information is sent to the power company.Since the ED shortage is far smaller than the total ED, the fossil sources will be consumed as little as possible to meet the rest power load.Therefore, the efficiency of renewable energy sources will be significantly improved under VPP architecture.This article presents a novel approach to enhance the flexibility and scalability of power grids.Our proposed solution is a smart energy management system that utilizes distributed EVs as a large-scale power storage facility.In addition, we aim to improve computational speed and reduce power consumption by integrating neural networks (NN) 13,14 and network-on-chip (NoC) technologies.By doing F I G U R E 1 Illustration of the proposed smart energy management system integrating the IoV networks.so, we enhance the performance and intelligence of MEC in vehicles. 15Viewing EV batteries as distributed intelligent power storage units, the additional power facilities can be avoided to build.In an AI-enabled system, upcoming G-IoV networks will leverage low-cost, environmentally-friendly renewable energy sources and optimize their limited resources and capabilities toward high-yield activities, yielding significant returns on investment.
The major contributions of our study are summarized as follows.
1. We have proposed a smart energy management system for the development of G-IoV networks, leveraging EVs, and the VPP architecture.2.An AI-empowered control system has been devised by combining the merits of software tools and edge clients in terms of efficient management and intelligent MEC. 3. We exploit our system to flexibly control the distributed EV batteries.EVs can autonomously charge or discharge the batteries to sufficiently use renewables for alleviating the energy crisis.
In the rest of this article, we first give an overview of the state-of-the-art work.We then elaborate the proposed smart energy management system in Section 2. Section 3 presents the performance evaluation of the system, in and Section 3.2 as well as we further envision open problems and future research directions in Section 3.3.Finally, Section 4 concludes this article.The nomenclature used in this article are listed in Table 1.

OUR PROPOSAL: SMART ENERGY MANAGEMENT SYSTEM
In this section, we first describe how to achieve data exchange.Next, the implementations of hardware and software are depicted.The flow diagram of the core control system is presented in final.

Data exchange between IoV and VPP
The aggregator, which is an essential component of the VPP system, is designed to facilitate data exchange at the control center.The architecture for data exchange through an aggregator is depicted in the bottom-right corner of Figure 1.On the one hand, the cloud aggregator collects the energy-demand information from main/local grids, for example, a power shortage or a power surplus.On the other hand, all the EVs reports their monitoring signals and reservation information to the aggregator in real-time.The monitoring signals include the battery state of charge (SoC), information of charging or discharging, being idle or busy, temperature, etc.Hence, through a VPP aggregator, real-time information can be efficiently exchanged.Finally, with the help of the edge clients, all the idle EV fleets intelligently make decisions to charge or discharge their batteries.In consequence, the IoV-VPP system not only supplies electricity to deal with energy demands but also stores the surplus portion of the renewable resources, which thereby increase the energy efficiency significantly.

2.2
Implementation of hardware and software for EV decision-making

From EV decision-making to a classification problem
As the aforementioned description, the behaviors of EVs can be generally divided into two types, i.e., charge and discharge.
In this article, with the help of intelligent edge clients, we hope EVs adaptively make self-decision by two behaviors to efficiently manage energy according to the real-time data, for example, energy demand, SoC, etc.Therefore, the EV decision-making issue is then regarded as a classification problem in which two categories are with two EV behaviors.The decision rule of EVs is presented in Figure 2 in detail.
When the input data contain different feature information, an EV responds to the data by making a different decision.The data containing pattern information is inputted into an edge client to enable efficient storage and processing.By analyzing the data features, EV decision-making can be transformed into a classification problem that can be effectively solved using NN techniques.Pre-processing of data features is essential to ensure the accuracy of the decision-making process.

Neural network for classification
Under the proposed architecture, to accurately predict the EV decision, we need to collect much information as the input features, such as date, weather information, and power load from the power grid.
According to the decision rule of Figure 2, a NN model is applicable to the classification of the EV decisions.Typically, the data of the EV state is used as the NN's input, and the output of one layer is used as the input of the following layer to calculate features via nonlinear mapping: where s is the encoding output of the s-th sample, w and b denote the weight and bias of the connection, respectively, and x represents the input features or data.f (⋅) denotes the nonlinear activation function, that is, rectified linear unit(ReLu) in this article.ReLu is defined as follows, with z representing the input: Softmax classifier is employed to estimate the probability of each class in the estimation stage.As mentioned previously, each class has a corresponding reference point in the monitoring area.Cross-entropy loss is employed as the objective function: where y i denotes the ground truth of the samples, y ′ i represent the predicted outcomes, and w and b stand for the parameters of the network.The Eq. ( 3) evaluates the error between the estimation and the ground truth.The prediction accuracy can be improved by optimizing the parameters of the network and minimizing the loss of objective function using stochastic gradient descent (SGD) and back propagation algorithms.
There are at least four output features, including the energy demand of the power grid (EDPG), the remaining power of EV batteries, the predicted energy consumption (PEC) of an EV in future use, and the EV state, respectively.EDPG and PEC can be effectively predicted by some typical NNs, such as the Recurrent Neural Network (RNN), the long short-term memory (LSTM) or the gated recurrent unit (GRU).A multi-NN joined the prediction model and the classification model will be an effective way for EV to make the accurate decision.In this article, since the goal is to propose a smart management system for VPP, we mainly focus on exploiting the core components in terms of hardware edge clients and software user interface (UI).In our future work, we plan to utilize the multi-NN model mechanism to achieve our objectives.
Hence, four known features are used as inputs to evaluate the proposed system.A NN is therefore designed for classification.The parameters of the NN are optimized by minimizing the loss function via the stochastic gradient decent algorithm and backpropagation algorithm.A classifier, for example, Softmax, will finally estimate the probability that the input belongs to individual classes, that is, charge or discharge.

2.2.3
Software implementation of the UI For effective operation of the whole control system, except the hardware edge clients, we also need a software tool for real-time management and monitoring in the control center.Technical problems.In order to develop a real-time system, several technical challenges need to be addressed.These include.

Ensuring real-time communication between the server and edge clients.
2. Establishing a unified system for managing and coordinating communication between multiple EVs and the control center.3. Developing an optimal strategy for allocating power demand to distributed EVs.
Solutions.Correspondingly, we propose to devise software, that is, graphical UI, for well addressing the aforementioned problems.Figure 3 illustrates the main flowcharts of our UI in detail.Especially, the following methods are investigated in our UI.
1. TCP/IP socket technique for data exchange between the server and a client.The server and clients are on the same Ethernet communicating by TCP/IP protocol.Due to differences in programming environments, the server is equipped with a Python socket, while each client is equipped with a C socket.This configuration allows the server to establish real-time remote communication with a client, even if the client is located at a significant distance from the server.By this way, the server is able to remotely real-time communicate with a client even if the client locates at a long distance.2. Multi-thread method for managing EV groups.To efficiently manage all the distributed EVs, we have exploited a multiple-thread technique in our software (See the UI flowchart in Figure 3).That is, thread 1 is created for initializing the layout, fetching data from the aggregator, and updating parameters to display UI.Thread 2 is used to execute the energy-demand allocation algorithm, and communicate with edge clients for data collection.In particular, thread 2 contains a structure of multiple sub-threads.Each sub-thread is in charge of the communication of one server-client channel.By the socket technique, the server can exchange request/response with the distributed clients.

Flow diagram of the entire control system
Based on the aforementioned functions of hardware and software, we then integrate them together to build out the entire control system which is illustrated in the bottom part of Figure 3.In this system, we have devised a graphical UI as shown in the top part of Figure 3.The UI is installed on a server.It sends data to edge clients and meanwhile receives feedback from them for monitoring.In consequence, our smart management system combines both the advantages of hardware and software in intelligent controlling and efficient communication.Such an AI-empowered system enables VPP to real-time manage all the distributed EVs for improving energy efficiency.
Figure 4 illustrates the basic architecture of our proposal.The training procedure will be performed multiple times until convergence.The steps of one round training are as follows:

Local models training
Each client train its local model separately using its unique set of data in accordance with Equations ( 1) and ( 2).The gradient g L , which is derived as follows, is employed to update the weight parameters in each local model.
where  is the learning rate, W represents the parameters of weight before updating, and W ′ denotes the updated weights.

Local models uploading
All the local clients upload the weight parameters of their models to the server after updating their specific parameters.

Global model updating
The global model on the server is created or updated by integrating the weight parameters of the local models, which are uploaded by all the clients.In general, average summation is adopted for the parameters integration, as described below: where W s denotes the weight of the global model on server, W ′ i represents weight of the ith local model of client, and n denotes the number of clients.

Local models updating
The weight parameters of the global model is transmitted to each client, which will be employed to replace the local ones of all the clients.The updating of local parameters is as follows: where W c denotes the updated weight parameters of the local models for the clients.Since there is no raw data exchange under such a FL architecture, data privacy will be preserved.

Evaluation methodology
We focus on the performance metric in terms of classification accuracy that is defined as the percentage of EV status correctly classified.To evaluate the proposed algorithm, we build a dataset consisted of the following features, Remaining Power of EVs, EV state, energy demand of power grid and energy consumption prediction of EVs for future use.
For each feature of the dataset, (a) ED used in our dataset refers to the total ED which is calculated by energy generation minus energy consumption.Here, the energy generation includes all the energy sources, for example., fossil and renewables; renewable sources refer to wind energy and solar in this article; local ED means fossil energy subtracts the total consumption of power grid.The value ranges are set as wind energy (2-20 kW), solar (0-60 kW) and local ED (20-70 kW) in the daytime, which is referenced to. 16(b) The remaining power and energy consumption are randomly generated according to the maximum capacity of EV batteries.Considering the general conditions in Japan, we take a typical EV Nissan leaf as an example.According to, 17 the maximum capacity of Nissan leaf is 30 kWh.Thus, assuming that the operating time is 6 hours per day, the future power consumption is about 0-5 kWh per hour.(c) for the last feature EV state, we set it by 0 and 1, where 0 denotes EV is idle while 1 denotes busy or unavailable, for example, EV is running or there is no enough power.The dataset is partitioned into two parts, that is, 2400 samples (about 80%) for training, 700 ones (20%) for validation.

Performance evaluation
In this part, we present the NN implementation and system performance.The difference of data distribution is shown in Figure 5. Figure 5a-d shows the probability density function (PDF) of labels data.Among them, the label data type of Client 1 is discharge and charge; the label data type of Client 2 is neutral and discharge and the label data type of Client 3 is charge and neutral.As shown in Figure 5a, the ratio is 4:4.In Figure 5b, the ratio is 5:3.In Figure 5c, the ratio is 6:2.In Figure5d, the ratio is 7:1.Since a concise NN structure will save memory cost for hardware implementation as much as possible.Based on the experimental results, the classification accuracy achieves almost 100% with the optimal parameters.The main settings in this work include the NN structure of 4 × 3 × 3, cross-entropy as a loss function, Adam as an optimizer, and the epoch number being set to 100.
As we discuss in the end of Section 2, distributed machine learning will improve the training efficiency.Moreover, the user privacy of sharing EVs must be protected, including the location, identification, and other personal information. 18hus, FL is a promising solution that is based on the architecture of distributed machine learning for privacy-preserving.Some initial experiment results of the online FL is presented in Figure 6.The experimental results which like Figure 6a-d shows that the F could achieve higher accuracy than the isolated training on edge clients.

Discussion and future works
In addition to predicting and controlling EV behaviors for energy efficiency, the proposed smart energy-management system can contribute to many other open issues and topics for the future research.In the following, we summarize several potential directions.
1. Decentralized technology for enhancing network security.Since the current IoV-VPP is of a centralized architecture, it faces various potential issues including network security, reliability, etc.0][21][22] Especially, when existing a malicious client that shares poisonous data or trained models, the entire system will be unstable and vulnerable.A promising approach is using Blockchain networks to replace the VPP center.2. Automatic fault diagnosis for IoV system based on AI units-empowered edge computing.Fault diagnosis is of great importance to stakeholders of enterprises.A large amount of monetary cost has been spent on monitoring and maintaining numerous facilities.As a representative scenario, the smart IoV or IIoT networks particularly face such severe challenges.For example, the traditional manual fault diagnosis under IIoT environments is usually unaccessible, inefficient, and non-real-time. 23This is because the environment is usually featured by high temperature, high pressure, being mobile, or being poisonous.Therefore, efficient automatic fault diagnosis that utilizes high-performance AI technology is mandatory for the future intelligent edges of IoV and industrial IoT.

CONCLUSION
We proposed an AI-empowered smart energy management system for the development of G-IoV networks, leveraging EVs, the VPP architecture.The proposed system combines the merits of both software and hardware.Thus, distributed EVs can collaborate as a huge smart power-storage facility by autonomously charge or discharge their batteries.Moreover, EV batteries have the ability to store surplus electricity generated from renewable energy resources.This approach will bring greater flexibility and scalability to the future energy management of green IoV networks and power grids.Finally, we also envision open issues to shed new light on future studies.We expect that this article can motivate the successive studies on the related topics of intelligent G-IoV and VPP systems.

E 2
Decision rule of the EV behaviors.Note that the ED denotes energy demand, EC is short for energy consumption.The upcoming EC of EV fleets needs to be predicted by NNs.The energy generation from renewable sources and user consumption from local power grid need to be separately predicted via other NNs according to the history curves, weather condition, etc.

F I G U R E 3 F I G U R E 4
Illustration of EV client and NN.Illustration of the federated learning (FL) architecture integrating with our proposed smart control system.Using the software UI, we can real-time manage the data exchange and monitor the status of all EVs.(a) Real-time information interaction between the VPP control center and the remote EV clients.(b) Data exchange between aggregator and edge clients.(c) EV or EV battery fault detection in time according to the monitoring signals.(d) Coordination of all the EV batteries to work as a smart power-storage system.(e) Efficient management of distributed EVs.(f) Quickly respond to some urgent needs, for example, through the software, we can select a related group of EVs to supply power for a local grid in some emergency situations.

F I G U R E 5
Probability density function of different label quantity ratios.(a) The PDF of Labels (4:4), (b) the PDF of Labels (5:3), (c) the PDF of Labels (6:2), and (d) the PDF of Labels (7:1).

F I G U R E 6
Accuracy comparison of the online FL (i.e., FLFEV in this figure), and the isolate training on three edge clients.(a) Labels distribution is 4:4, (b) labels distribution is 5:3, (c) labels distribution is 6:2, and (d) labels distribution is 7:1.