Nodal dynamic charging price formulation for electric vehicle through the Stackelberg game considering grid congestion

National Natural Science Foundation of China, Grant/Award Number: 51507022 Abstract The disordered charging of electric vehicles (EVs) may result in transmission congestion during the peak load period; therefore, a nodal dynamic charging price (NDCP) strategy is proposed to schedule the charging activity of regional electric vehicle agents (REVAs) to manage congestion. REVA is responsible for meeting the charging demand of users at a minimum charging cost, while the grid aims at minimising the congestion management cost and regulates the charging price of different nodes. The Stackelberg game theory is used to model the negotiation process of charging price, and the congestion contribution index is proposed to determine the congestion cost that the user side should be allocated. The proposed approach is implemented on amodified IEEE30bus system, and the results show that the congestion caused by EVs is reduced effectively, while the charging cost and congestion cost are also optimised.


| INTRODUCTION
With the deterioration of energy and environment issues, the number of EV users has been growing continually due to the prominent advantage of being environment-friendly and energysaving. By 2019, the number of electric vehicles in the world exceeded five million [1]. However, the uncontrolled charging activity of many EV fleets will result in adverse effects on the safety operation of the power system, such as increasing the peakto-valley difference of the load curve, causing the overloading of transmission lines.
The studies about congestion management in a power system have made some progress. In general, there are two kinds of congestion management methods, that is, direct control and indirect control [2,3]. The former has a better effect on congestion management, but the profits and willingness of the user side are not usually considered. Therefore, the market-based method, namely indirect control, received wide attention. The idea of the indirect control method is to guide users through the price, including line shadow price [4], subsidy-based method [5][6][7], probabilistic congestion management methods [8][9][10], dynamic tariff (DT) [2,11,12], and locational marginal price [13,14]. Reference [4] utilises the flexibility of consumers to balance the energy trading, and the shadow prices are used to resolve the grid congestion. In [5], the incentive-based demand response (CIDR) is proposed to reduce the system operational cost and guide users to participate in demand response (DR). Reference [8] uses a chance-constrained method to solve the congestion management problem considering system uncertainties. However, these earlier studies do not consider the EVs' effect on congestion management.
Since the flexibility and controllability of EVs could make them accept the regulation of the power system, much research has been undertaken to study how to utilise this resource to participate in congestion management. Reference [11] uses the DT method to reduce the congestions that occur in the distribution network with high penetration of distributed energy resources (DERs), and the chance-constrained method is used to increase the robustness of the system. On the basis of [11], references [2,12] improve the control parameter and optimisation method of the DT method, respectively, which reduces the management cost. In [13], the author proposes a market-clearing model for the distribution network, including the interaction between direct and indirect methods, then the decomposed distribution locational marginal price (DLMP) is used to encourage DERs to improve the result of congestion management, and reference [14] optimises the DERs through the iterative DLMP, and then alleviates the line congestion. References [6,7] use demand side management to change users' consumption behaviour to relieve the pressure of system operation. Reference [9] solves the uncertainty of DER through a probabilistic power flow method, and regulates the charging and discharging power to reduce the probability of line congestion, while reference [10] also uses the chance-constrained method based on the mixed-integer programming to describe the stochastic characteristics of EVs, and combines with the DLMP to manage congestion. In [15], a bi-level model is proposed between the distribution system and EV operator, and the charging demand of EV is analysed in detail. The above-mentioned references have studied the congestion management in a distribution network, while the congestion caused by REVAs in the transmission network is not considered. Considering that the concentrated charging process of numerous EV fleets in some areas, combined with many compulsory power transactions, may influence the operation of transmission grids, the effect of EVs in transmission congestion needs to be studied.
REVA could integrate the EV charging load with large quantities and wide distribution, in order to easily control for the powergridandimprovesomeexisting problems,suchasachieving accommodation of renewable energy [16,17], improving economy and reliability of the power system [18,19], and providing voltage support [20], etc., which plays an important role in connecting EV users to the power grid. Therefore, the study of the interaction among REVAs and the power grid is of importance to the EV users and grid, and the game theory provides an effective way to solve such a multi-agent optimisation problem. Traditionally, game theory could be divided in two ways, a cooperative game [21] and a non-cooperation game [22][23][24][25][26], and the status of each subject in the two models is equal. However, with the interaction problems such as EV agents and grid, and EV users and agents, the status of each subject is unequal [27], so we model this kind of problem through a special non-cooperation game, that is Stackelberg game theory. Reference [28] proposes a robust optimisation method based on the Stackelberg game to study the charging game among EVs under the leadership of an EV aggregator, and develops distributed energy scheduling algorithms under energy demand uncertainty. Reference [29] considers the impactofotherelasticloadsonEVs'chargingbyDRmanagement. In [30], the author analyzes the effects on the strategy of charging, which is caused by different weights of utility function. Reference [31] proposed a multileader-multifollower Stackelberg game model with EVs as the leader and microgrids as the follower to study the charging strategy of EVs. In [32], the two-stage Stackelberg gameisproposedtostudytheproblemofdifferentcharging stationpricing.Inthefirststage,thechargingstationwillannounce the charging price,and EVs will determine the charging strategy in second stage, then proving the only equilibrium depends on the price difference among charging stations. The research reviewed above show that the game theory is suitable for charging and discharging pricing of EVs, and could attract EV users to participate in congestion management effectively.
Focussing on the above problems, a nodal dynamic charging price (NDCP) method based on the Stackelberg game model to solve the congestion caused by disordered charging of EVs is proposed. The power flow tracing is used to calculate the congestion contribution of EVs, then REVAs share the congestion cost based on the calculated contributed index, which could regulate the charging price of each node dynamically. Finally, EV users could charge at a comparatively lower price, while the grid has the minimal congestion cost. The simulation results show that the proposed methods are feasible and effective. The main contributions are as follows: � The nodal dynamic charging price (NDCP) is proposed to guide the REVAs charging schedule to reduce the congestion caused by EVs. � The congestion contribution index is proposed to measure the impact caused by EVs on the system congestion, which is also the basis of the congestion cost sharing. � The Stackelberg game model is used to describe the negotiation process of the charging price among the grid and REVAs. The result shows that the proposed method could effectively alleviate the congestion, and gain minimal charging and congestion management costs.
The charging load model of regional EV agents is established in Section 2. The calculation and allocation model of grid congestion cost is provided in Section 3. In Section 4, the one-to-many game model between grid and regional EV agents based on the Stackelberg game theory is described. In Section 5, a case study of the proposed model is presented and discussed. Conclusions are drawn in Section 6.

| Travel demand model for various types of EVs
The charging load of an EV is closely related to the type, charging characteristics, and travel distance. The type of EV directly determines its battery characteristics and travel needs. Depending on different uses, EVs are classified into three types in this section: electric private cars, electric buses, and electric taxis. These three types of EVs are the main focus of this research.
The travel time directly determines the charging time of an EV. According to the results of the Department of Transportation's travel statistics for household vehicles in the United States [33], the private vehicle's departure time and arrival time approximately obey a normal distribution, while the daily travel distance approximately obeys a lognormal distribution. In fact, EV owners will send the arrival and departure time to REVAs in advance, herein the authors use probability distribution instead, due to the lack of real data. For private EVs, it is assumed that they only charge once a day, for example, from 19:00 today to 8:00 the next day. Their probability density functions of the departure time, arrival time, and the daily travel distance are as shown in Equations (1)-(3), respectively: where f dt ðtÞ, f at ðtÞ and f d ðdÞ represent the distribution function of departure time, arrival time, and daily travel distance, and t and d are time and travel distance. Considering the operation mode of electric buses is usually fixed, and for convenience, the authors assume their departure time and arrival time also follow (1) and (2), while the daily travel distance is assumed to follow a normal distribution as shown in Equation (4): The electric taxi drivers usually charged during the afternoon and night, and their operation mode is comparatively stochastic compared with the other two types of vehicle, and it is assumed that their arrival times follow a uniform distribution, and each period of charging time is set to a fixed 3 h considering their job requirements. The distribution function of arrival time is as follow: where t dt and t at are the departure and arrival time, respectively.

| Operation model of regional EV agents
The current threshold for allowing users to participate in electricity market transactions usually exceeds 1 MW. However, the charging power of a single EV cannot reach the required level. Therefore, it is unrealistic for the power grid to schedule a single EV and complete the transaction. Also, if the power grid directly schedules a large number of EVs, it will greatly increase the computational burden. Usually, the distribution of EVs in large areas is relatively concentrated, and it is possible to establish an REVA for each EV charged in the area. REVA could consider the travel and charging needs of each EV, and also collect the charging demand of EVs participating in the electricity market during each period. Therefore, it is necessary to set up regional EV agents in each area.
In general, EVs that are frequently charged in this area need to sign a contract with the REVA agent for a period of time. If someone breaks the contract, there will be penalties. So we assume that EV owners won't change their charging plan after signing a contract. As an intermediary between the power grid and EV owners, the REVA collects electricity price and users' charging information to negotiate with the grid on behalf of the EV owners, and herein, the profits of the REVA are not considered. EVs in the area access the power grid through intelligent terminals, which are under the control of the REVA, and intelligent terminals automatically take optimal decisions and execute them according to the price information collected by the REVA. The operation process of the REVA is as follows, and the charging service framework is shown in Figure 1: � EV owner submits the arrival time, departure time, and charging demand to the intelligent terminal. � REVA inputs the charging price information to the intelligent terminal. Combining with the information that the EV owner submitted and grid constraints, the intelligent terminal obtains the optimal charging strategy and cost. Finally, the REVA signs a charging contract with the EV owner. � According to the charging decision and cost information calculated by intelligent terminals, the REVA and power grid sign the electricity purchase contract in the day-ahead market to determine the amount of power purchased during each time period.
Based on the process above, the charging load model of each REVA can be established. According to the charging price published by the grid, the charging strategy is to minimise the total charging cost of a REVA. The objective function is shown as in Equation (6): where i represents the node number and t represents the time period number, n is the total node number. P i;t and p i;t are the charging power of EV fleets and the charging price under node i during period t, respectively. The necessary charging constraints are as follows: where m represents the m-th EV, N is the total EV number. Equations (7) and (8)  and P i;m;max are the maximum charging power and minimum charging power of the m-th EV under node i, respectively. Equation (10) represents that the charging capacity of the EV is equal to the amount of power consumed by driving, η c is the charging efficiency, and Cap i;m;total is the total amount of electricity consumed by the m-th EV under node i in 1 day. The total charging power constraint in each period is shown in Equation (11), N i;t is the number of schedulable EVs under node i. Equation (12) is the charging capacity constraint of each node, Cap i;max is the maximum charging capacity of node i. Equation (13) is the constraint of the electricity price of each node. p i;max is the maximum electricity price of node i.

| MODEL OF GRID CONGESTION COST CALCULATION AND ALLOCATION
In the future, the REVA will play an increasingly important role as the EV penetration rapidly increases. Without appropriate charging control, EVs may start charging once they access the grid, and the new load peaks may cause the transmission lines and transformers to be overloaded. In order to ensure the safe and stable operation of the system and market efficiency, the power grid will adjust the power generation schedule. Compared with normal conditions, the additional cost is the system's congestion cost, and REVAs need to share the corresponding part according to the influence of EVs charging power on grid congestion. In electricity market transactions, the congestion cost shall be fairly shared by market participants in accordance with the portion causing congestion [34], and this usually is obtained by power flow calculations. The DC optimal power flow method is widely used in power system economic schedules, congestion management, security check, etc. due to its linear expression and rapidity. Based on the considerations above, the DC optimal power flow method is used herein. It is assumed that the congestion cost is allocated to the load side. After calculating the DC power flow, the power flow tracking method [35] will be used to find the power flow in each line caused by individual loads. Since the load in a system consists of the basic load and the charging load of EVs, on the basis of proportional principle, the congestion costs will be shared by REVAs according to the ratio of EVs charging load to the base load. The calculation and allocation process of the congestion cost is shown in Figure 2.

| The calculation of grid congestion
First, DC power flow is used to calculate the power flow system, and its equation is as follows: where P ij , x ij are the power flow and reactance of each branch, θ is the phase angle. Then power generation costs with and without the power flow limitation in lines, C 1 and C 0 , can be obtained. According to C 0 and C 1 , the system congestion cost C T can be calculated, as shown in Equation (15) The cost function of generator (16) and necessary constraints (17)- (19) are as follows: where G is the set of generators, and D is the set of loads. P 1 Gi is the active output power with safety constraints, P 0 Gi is active output power without the safety constraints, and a, b, c are the cost coefficients of the i-th generator. Equations (17), (18), and (19) are the power flow balance constraint, generator active power constraint, and line power constraint, respectively. P Di is the active load of i-th node, P Gimin and P Gimax represent the maximum active and the minimum active output of i-th generator, P ijmin and P ijmax represent the minimum and maximum active power limits of line i − j. When calculating C 0 , only constraints (17) and (18) should be considered, and constraint (19) should also be considered while calculating C 1 .
According to the above model, the optimisation solution can obtain the generation power plan with and without safety constraints and the congestion cost C T .

| The allocation model of congestion cost
After DC optimal power flow calculation, the congestion cost C T can be obtained. If C T ≠ 0, it needs to be distributed to the loads which contributed to the congestion. Considering the impact of the load under different nodes on the transmission power of each line, the influence of each load should be quantified, so as to make the congestion cost allocation. For node i, the power flow balance equation for this node is shown in Equation (20): where P i is the output power from the node i, a d i is the set of nodes directly supplied active power from node i. To form a matrix, change the above equation to Equation (22): where P is the node load vector; P D is the node load demand vector; ½A d �is the downstream allocation matrix, and the elements of ½A d � are: Then the vector P can be obtained from the Equations (22) and (23), its i-th element is equal to Equation (24): Because P i is equal to the sum of generators and the inflows in lines entering the node, so the inflow to node i from line i − j, that is P ij , can be calculated using Equation (25): where a u i is the set of nodes directly supplying node i. When this makes k in Equation (25) a specific node number, the power flow in line i − j of k-th load could be obtained by Equation (26): According to this method, it can track the loads flow on each line in the transmission network. Setting a n � n line  active constraint matrix P flow;max , its elements are defined as follows: According to the power flow results, set matrix of power flowP flow , its elements are defined as follows: Then Equations (27) and (28) can be used to obtain the congestion power matrix, its elements are: where½P con � ij is the overloaded power on line i − j. According to the overloaded power of each line, congestion cost is initially distributed to each branch, as shown in the following equation: where C ij is the congestion cost caused by overloaded power on line i − j. From Equation (30), congestion cost allocated by each line can be obtained, then the cost is distributed to the load side: where C ij;Dk represents the congestion cost of the k-th load, which caused congestion power on line i − j, and the k-th load contains both base load and EV loads. Superimposing the congestion cost of all lines could obtain the total congestion cost C Dk which the k-th load should share. The congestion cost of the REVA at the node in which the k-th load resided is: where P EV k is the charging power of EV fleets and P Dk is the total load at the node in which the k-th load resided, τ ij;EV k is the congestion contribution index of the EV loads to the power flow of line i − j. The congestion cost allocated to the load side is equivalent to the node electricity price in the form of additional charging price, and the additional congestion electricity price allocated to the REVA under the k-th load is: According to Equation (38), the nodal dynamic charging price of REVAs is as follows: where p EV i is the final charging price under node i, which is equal to the sum of the price under node i and additional congestion charging price in period t, that is the NDCP. When the REVAs and grid reached a charging agreement, the charging price is settled according to p EV i .

| THE ESTABLISHMENT OF GRID-EV ONE-TO-MANY GAME MODEL BASED ON THE STACKELBERG GAME
The Stackelberg game is a dynamic non-cooperative game and is applicable to the asymmetric game situation in which the states of game players are inconsistent. In the scheduling problem described herein, the power grid, as the publisher of the electricity price, is the leader of the game, and REVAs are responders to the electricity price, that is, game followers. As the leader of the game, the power grid makes the decision γ grid , and REVAs makes the best response R EV ðγ grid Þ to the decision of the power grid. Finally, the grid makes the most favourable decision γ * grid based on followers' decisions. At this time, the leader's objective J grid is the smallest, and the REVAs choose their own optimal strategy according to the optimal strategy response set of the leader grid, that is γ * EV ∈ R EV ðγ * grid Þ. Considering the existence of multiple REVAs, the equilibrium point of the Stackelberg game is ðγ * grid ; γ * EV 1 ; γ * EV 2 ; ⋯; γ * EV Na Þ.

| The structure of the Stackelberg game
The decision variable of grid γ grid is defined as the time-of-use (TOU) price p i;t , the decision variable of the regional EV agent γ EV i is the total charging power of EVs P EV i . In the game, both sides make decisions in the most favourable direction for themselves. The objective function of the REVA is to minimise the total charging cost of EVs, that is: Constraints are as shown by the Equations (7)- (13).
The objective function of the power grid is to minimise the grid congestion cost of the system, that is: The constraints include the system security constraint, as shown by Equations (17)-(19) the lower limit of the charging price of each node, which means the node marginal cost of the node, is as shown in Equation (38): where p i;t;LMC is the locational marginal cost of the i-node in t period. The grid will get profits only if the charging price p i;t is higher than p i;t;LMC , then it will be willing to provide power supports to the REVAs. The Stackelberg game structure of the grid-region EV agents is shown in Figure 3.

F I G U R E 3
The Stackelberg game structure of grid-REVA

F I G U R E 4
The solving process of the Stackelberg game

| Model solution of the Stackelberg game
The Stackelberg game model of the grid-REVAs is a non-linear model, which can be divided into two parts: the internal part and the external part. In the inner layer model, each REVA makes decisions, aiming to obtain the response plan set of TOU price of node R EV ðγ grid Þ. The outer layer is designed to optimise the TOU price to obtain the optimal TOU price of corresponding node γ * grid . The complete solution flow of the game model is shown in Figure 4.
At the beginning of the Stackelberg game, the power grid gives its initial decision, that is, the initial TOU price of the node. The regional EV agent in each area makes charging decisions based on the initial TOU price of the node, and starts to solve the inner layer model. For each REVA, the charging price has been given when making decisions, considering the constraints of charging power, total charging capacity, etc. The linear programme is used to perform optimisation, takes the total charging power of EVs as the optimisation variable, then obtains the optimal response set of the TOU price of node R EV ðγ grid Þ, and finally feeds it back to the grid. The grid records the information of response sets and starts to solve the outer layer model. The power grid uses the particle swarm optimisation (PSO) algorithm to optimise the unit output and power flow of system according to the feedback information of the REVAs. Here, the optimisation variable is TOU price. The model also considers all charging schemes of the charging scheme response set in the current round of the game, calculates the optimal power flow, and obtains the minimum grid congestion cost of system. The charging price which minimising the objective function is the optimal price of the current round, and the charging scheme corresponding to the minimum cost is the optimal scheme of the current round.
The above process is a round of the Stackelberg game process. After multiple rounds of the game until the convergence condition is reached, the equilibrium solution ðγ * grid ; γ * EV 1 ; γ * EV 2 ; …; γ * EV Na Þ will be found.

| Simulation parameters
The proposed method is implemented on the modified IEEE 30 bus system and the game model is solved by the PSO algorithm which is programmed in MATLAB, and the power flow is calculated by Matpower. The system structure diagram and generator parameters are shown in Figure 5 and Table 1. Four REVAs in the area participate in congestion management, and the parameters of EVs are shown in Table 2 Under fixed price 100. 4 24,858 Under NDCP 66. 34 12,004 Abbreviation: NDCP, nodal dynamic charging price.

| Impact on system operation
The number of EVs connected to the grid in each period is shown in Figure 6. The authors mainly compare the disordered charging under fixed charging price (p f ixed = 0.7 ¥/kWh) and the ordered charging under NDCP. The result of load curves is shown in Figure 7. In the disordered charging mode, the EV will start to charge at once when it connects with the grid, which makes the charging load overlap on the base load, and leads to a higher peak-to-valley difference of the load curve. While with the charging load shifts to the period after 22:00 under the guide of the NDCP, the system operating pressure in peak load period can be effectively alleviated, and the peak-tovalley difference reduced from 100.4 to 66.34 MW, and load variance also declined from 24,858 to 12,004 MW 2 , as shown in Table 3 NDCP could also improve the congestion in transmission lines. As the Figure 8 shows, when the charge mode is fixed price, the overload occurred in lines 9-10 and 12-15 during about 16:00-20:00 and 12:00-21:00 respectively, and the maximal load rate of lines 12-15 is close to 1.1. While under NDCP, the congestion in lines 12-15 only happened during about 17:00-19:00 with an obvious decline, and in lines 9-10 it is almost eliminated. This is because the congestion caused by EVs has been eliminated totally combined with the Figure 7 (b), which shows the existing congestion is caused by the base load.

| Impact on Economics
The local marginal cost p i;t;LMC is calculated by Matpower, and variation of the value can be ignored, and is taken as 0.138 yuan/kWh. The charging price of the nodes where the four REVAs resided is shown in Figure 9. In the case of fixed price, EVs of REVA 1, 3, and 4 have contributions to transmission congestion, their charging price includes both the congestion additional price and the fixed price, while the EVs of REVA 2 have no contribution to the congestion, so its congestion additional price is 0, which is shown in Figure 9a. In Figure 9b, during 14:00-22:00, the NDCP is comparatively higher than other periods, so there is no REVA choosing to charge considering the aim of reducing charging costs. After 22:00, the NDCP goes down gradually, and the REVAs also start to charge accordingly, and the charging cost is lower than the case under fixed price, which dropped from 137,178 to 42,928 yuan. For the power grid, the system congestion cost dropped significantly from 4527 to 213 yuan, the generation cost also reduced from 703,087 to 697,688 yuan, as shown in Table 4. Moreover, the NDCP could effectively avoid a new load peak and congestion caused by centralised charging of EVs during the peak load period.
It can be seen from Figures 8 and 9a that the lines 12-15, which supplied power to REVAs 3 and 4 but not REVAs 1 and 2, have a much heavier congestion during peak load period, and the charging load of REVA 3 is higher than REVA 4, therefore the congestion cost of REVA 3 is the highest. Meanwhile in lines 9-10, which supplied power to REVAs 1 and 4 but not REVAs 2 and 3, have a lower congestion, hence the congestion cost of REVA 1 is lower than REVA 3. And the charging load of REVA 1 is also higher than REVA 4, so the congestion costs of both REVAs 1 and 3 are higher than REVA 4. It can be concluded that the congestion cost has a positive correlation with the congestion contribution of EV loads.

| CONCLUSION
An attractive price-based method could encourage EV users to participate in congestion management, and not only improve the system security, but also benefit both users and the power grid. An NDCP method for REVAs based on the Stackelberg game has been proposed to manage transmission congestion, and was simulated on a modified IEEE 30 bus system. Through the simulation results, the following conclusions can be drawn: � The proposed NDCP strategy could guide REVA charging schedules to improve the operation of system stability during the peak load period. The scheduling result shows that, compared with EV disordered charging, the NDCP could reduce system congestion, peak-to-valley differences, and load variance obviously.  � The NDCP also improves the economics of the grid and EV users effectively, which avoids an increase of new load peaks and makes REVAs charge in periods of lower charging price. The EV users' charging cost, system power generation costs, and congestion costs are all reduced to different degrees. � According to the congestion contribution index of the EVs proposed herein, the total congestion cost will be shared by different REVAs during each time period. The bigger the congestion contribution from EVs, the higher the charging price will be, which can further guide REVAs to charge after the peak load time.
An adjustment of the EVs charging plan has a significant impact on the system's congestion management and charging cost of EVs, but it cannot solve the congestion problem caused by other loads. In future work, the authors will consider the impact of charging and discharging of EVs on system congestion management, especially the discharging behaviour of EVs. In addition, they will also study the profits and competitions of different REVAs, as well as the detailed charging schedule of each EV user.