A pricing strategy based on potential game and bargaining theory in smart grid

In this paper, a two-stage pricing framework is proposed for the electricity market which is consisted of a generation company (GC), multiple electric utility companies (EUC) and consumers. In the electricity wholesale market, the EUCs will choose an agent to negotiate the wholesale price with GC. An appropriate wholesale price plays an important role in the stable operation of the electricity wholesale market. However, it is challenging to ﬁnd the optimal wholesale price. Therefore, the Raiffa-Kalai-Smorodinsky bargaining solution (RBS) is applied to realize the pricing equilibrium which is 0 . 3$ ∕ KWh. In the electricity retail market, this study designs a retail pricing strategy based on the potential game, which can optimize both social welfare and the beneﬁt of the EUCs. Moreover, the impact of demand disturbance on the beneﬁt of the EUCs and GC is studied in the electricity retail market. Then an iterative pricing algorithm is proposed for the two-stage pricing model. The simulation results reveal that the demand disturbance has little effect on the beneﬁt of the EUCs and GC, indicating the reliable/promising robustness of the algorithm.


INTRODUCTION
Smart grid is a typical energy-based networked system that integrates network and physical components, which introduces advanced information communication technologies into the grid. Smart grid has many advantages which can improve the operating efficiency of the grid and enhancing the energy utilization rate. Energy pricing strategy is the focus of smart grid research, which is a key research content in smart grid. An appropriate pricing strategy is of great significance to GC. By formulating an appropriate pricing strategy, the optimal energy scheduling of GC can be realized and increase its benefit. In [1], an energy scheduling strategy for GC based on mixed integer programming method is proposed, which improves the benefit of power companies. In [2], A multi-objective bidding strategy framework was proposed to improve the expected benefit of GC and reduce the amount of pollution generated by thermal units. A novel energy self-dispatching strategy for GC is proposed in literature [3]. At the same time, the pricing strategy is also very important to power system. In [4], the authors proposed a proportional differential pricing strategy to reduce the total cost of the power system. A real variable pricing strategy based on the inverse optimization method was proposed to achieve the control goal of the power system in literature [5]. In [6], a two-stage energy dispatching pricing model was proposed to improve the individual benefit in the power market.
Recently, the application of game theory to analyze the problems in smart grid has received widely attention. In [7], the noncooperative game was applied to solve the interaction problem between power companies and users. At the same time in [8], the authors further applied the non-cooperative game to construct the energy consumption model of the Plug-in Hybrid Electric Vehicles (PHEVs), so as to reduce its charging cost. However, the results cannot be Pareto optimal when solving the problem through the non-cooperative game. Therefore, the cooperative game is often used by many scholars to solve some problems in smart grid. In [9], the cooperative game was used in microdistribution systems to detect non-technical losses and power outages. In [10], a cooperation mechanism was introduced to solve the cooperative demand-side management problem in low-voltage power grids. Compared with the non-cooperative game and the cooperative game, the stackelberg game can better solve the multilevel hierarchical decision problem, so it has been widely used. In [11], the stackelberg game was applied to micro-grid with commercial buildings to solve the energy management problem in it. At the same time, game theory also has important applications in the field of energy pricing strategy in the electricity market. In literature [12], the author proposed a new decisionmaking framework based on game theory, which optimized the social welfare. In [13], the authors proposed a pricing mechanism based on game theory and revenue sharing contract to optimize the social welfare. In [14], a pricing strategy based on stackelberger game was proposed to maximize the benefit of EUCs in the micro-grid. Meanwhile, a stackelberger game pricing model that considers both retail and wholesale markets was proposed in [15], which solved the optimal wholesale and retail prices. In [16], a Bertrand-based game pricing model is proposed while considering the close-loop interaction of the wholesale market and retail market, which solved the optimal retail pricing issues. The differences between this work and the existing work are shown in Table 1.
In the above literature, the authors only studied the retail or wholesale process of electricity. The wholesale and retail processes are not integrated, which is not conducive to the efficient operation of the electricity market. At the same time, the pricing models in the above-mentioned literature are based on the non-cooperative game theory, which cannot make the social welfare reach the optimal. Therefore, this paper proposed a novel electricity pricing strategy based on bargaining theory and potential game theory, which considered the wholesale and retail processes of electricity. Moreover, the pricing model can optimize the benefit of EUCs while optimizing the social welfare. The contributions of this study are given as below: • In the electricity wholesale market, the RBS bargaining theory was applied to solve the problem of formulating the wholesale electricity price. It proved that the wholesale price negotiation problem between GC and agent is a bargaining problem. Then, the optimal equilibrium solution of the bargaining model was obtained, and the effectiveness of the bargaining model was illustrated by simulation case. • In the electricity retail market, the potential game was applied to formulate the retail electricity price. It was proved that the retail pricing model among multiple EUCs is a potential game model. Then, the NE of the pricing game model is obtained by a price iteration algorithm. Moreover, the feasibility of the pricing model was illustrated by simulation case. • The potential game pricing model proposed in this paper can increase social welfare by 6%. Meanwhile, the proposed pricing iterative algorithm has a better robustness and the convergence error can be bounded by 2% under additive disturbance.
The rest of this paper is organized as follows. Some preliminaries and relevant definitions are introduced in Section 2. In Section 3, we establish the system model. A distributed algorithm is developed in Section 4. The simulation results are carried out in Section 5, and conclusions are summarized in Section 6.

PRELIMINARIES
Definition 1. (Potential Game [17]) In a non-cooperative game G = {N, (S i ) i∈N , (U i (p)) i∈N }, if the game satisfies the following conditions: then the game is defined as a potential game, where N = {1, 2, … , n} is the set of players, x and z are the possible strategies that consumer i can take, p −i is the possible strategies adopted by others except for consumer i, U i (p) is the payoff function, u i (p) is the potential function, and is the set of possible strategies. If the payoff function is continuous and differentiable, then the necessary and sufficient condition for the existence of potential function is: The potential game has some good properties [16]: • The NE solution of the potential function is consistent with the potential game model. • Each ordinal potential game has an NE solution.
• Each ordinal potential game is capable of finite improvement.

Definition 2.
(Finitely increasing property [17]) If the incremental path of a game is finite length or the subject of the game can reach the NE after a finite number of iterations, then the game has the property of finite increasing. Definition 3. (Nash equilibrium [18]) In a non-cooperative game G = {N, (S i ) i∈N , (U i (p)) i∈N }, the strategy vector for any p i ∈ S i . Definition 4. (Bargaining Problem [19], [20]) Let { j | j = 1, 2, … , n} denote the set of participants and F represent the set of feasible payoff allocation, where F is a closed and convex subset on the R n .
, where V min j denotes the minimum payoff of participant j . Then, (F, V min ) denotes a n-person bargaining problem, and f (F, V min ) is the outcome of the bargaining problem.
RBS and NBS are the main methods for solving bargaining models, and each method has its own characteristics. The NBS only considers the minimum utility needs of participants. In the process of bargaining, if the maximum utility is not considered, it will lead to a decrease in the fairness of participants' utility distribution. The RBS considers the profit of participant i relative to the minimum utility demand and the expenditure of other participants relative to the maximum utility, thereby increasing the participation of all participants. Therefore, this paper will use the RBS bargaining strategy to achieve the optimal solution of the bargaining model.
Then, we can obtain the bargaining solution: where j is player j 's payoff function, which is denoted by where V max k and V min j are the maximum and minimum payoffs of the participant k and j , respectively.
The RBS are usually described by some axioms, such as independence, feasibility, linear axiom, pareto optimality, symmetry and monotonicity. The electricity market consists of EUCs, agent, GC and users. GC is responsible for formulating the wholesale price, and EUCs are responsible for formulating the retail price. The agent is an important participant in the electricity market and plays a very important role in the pricing process. For example, in [22], the agent is applied to the pricing process in the microgrid. In [23], the agent is applied to the real-time pricing process in the electricity market. The operation process of the proposed two-stage pricing model is shown in Figure 2.
In the process of formulating retail price, multiple EUCs first choose an agent which on behalf of EUCs negotiates with the GC about the wholesale price. The pricing process of wholesale price is uniform pricing. Then, the EUCs interact with each other to determine the retail price based on the wholesale price and announce it to the users. The pricing process of wholesale price is pay-as-bid.
Remark 1. The proposed pricing model is more in line with the trading environment of the open electricity market, which has a large market size and a high degree of coupling between the trading parties. Therefore, the calculation amount of the pricing process is very complicated. This work constructs a two-stage pricing model. In the process of electricity retail, the distributed optimization method can reduce the computation overhead. Therefore, the pricing model constructed in this work is more suitable for the open environment of electricity market. Compared with the existing industry benchmark, the two-stage pricing model proposed in this paper takes into account the transaction costs of the electricity wholesale process and the retail process, and further maximizes the transaction revenue in each stage.

Wholesale pricing based on bargaining
According to Definition 4, the negotiation between the agent and the GC can be formulated as a bargaining problem, and the agent's payoff function is defined as the sum of utilities of all the EUCs: and the utility function of the GC is denoted by where c s denotes the cost of the generation, w is the wholesale price and its interval is w min ≤ w ≤ w max .

Theorem 1. The wholesale price negotiation between agent and GC can be formulated as a bargaining problem.
Proof. We define the set of feasible profit as: where w min and w max are the lowest wholesale price and the highest wholesale price, respectively. From (9), (10) and (11), we can conclude that the feasible profit set F consists of the profits of the agent and GC, and the set F is closed on R n . Now we take two elements {u A c , u A g } ∈ F and {u B c , u B g } ∈ F at random: and Then, adding the weights of the two elements, we have where 0 < r < 1. Comparing Equations (12) and (14), we define: Obviously we can prove that w A − w max ≤ 0 and w B − w max ≤ 0 and obtain at the same time, we can also prove that w A − w min ≥ 0 and w A − w min ≥ 0. Then, we have Therefore, the set F is closed and convex on R 2 . According to Definition 4, the bargaining problem between the agent and the GC was established. □ Thus the cooperative strategy based on RBS is defined as: Then, we can calculate a globally optimal solution: where w * is the optimal wholesale price.

Retail pricing based on potential game
According to the wholesale price announced by GC, the EUCs will game with each other to maximize the total profits of all EUCs. Therefore, the game between EUCs can be viewed as a potential game.

Definition 6. The EUCs game can be formulated as
, n} is the set of EUCs, S i is the strategies set of the EUC i, and U i is the payoff function of the total profits of all EUCs.
In electricity markets, the GC is mainly responsible for producing electricity and selling electricity to EUCs in the wholesale market to achieve the purpose of maximizing their profits. The EUCs will predict the electricity demand of its consumers and then purchase electricity in the wholesale market. After that, they optimize the retail price of electricity and sell it to users in the retail market. In [13], the market demand for EUC i is denoted as where a i is the maximum possible demand of EUC i, p i is the retail price of the EUC i, p j is the retail price of other EUCs j ∈ {1, 2, … , n}, j ≠ i, and d is the conversion coefficient between the price and the demand. The conversion coefficient is a measure of the sensitivity of the change of ith EUC's price on the other EUCs' sales.
In the electricity markets, the total profits of all EUCs can be expressed as The function in (23) represents the total net income of all EUCs.
In the game process of EUCs, considering that not only social welfare should be maximized, but also each EUC should be able to optimize its own benefits. We assume that the benefit of each EUC is the potential function of function (24), i.e.
Next, we prove the game among EUCs (EUC game) is a potential game.

Theorem 2. The EUC game is a potential game if
Proof. We take the first derivative of u i (p) with respect to p i : Then, taking the first derivative of U (p i ) with respect to p i , we can obtain Based on constraint (25), comparing Equations (26) and (27), we find that then Hence, according to (3), we can know the function (24) is the potential function of the EUC game. Therefore, the game is a potential game. □ Next, we will prove the existence and uniqueness of NE of the potential game.

Existence and uniqueness of NE
Now we will discuss the existence and uniqueness of NE of the EUC game. At the NE, the vector p = (p * 1 , p * 2 , … , p * n ) satisfies the following conditions: Based on the equation (26), we make u i (p)∕ p i = 0. After that we can obtain the optimal response function: Next, we take the second derivative of u i (p) with respect to p i : Since the second derivative of u i (p) with respect to p i is always negative, u i (p) is concave in p i . Therefore, we can prove that the proposed EUC game has at least one NE. Then, we will give the uniqueness condition of the NE in the EUC game. First of all, we make the following definition.  In Definition 7, the symbol ′′ > ′′ or ′′ ≥ ′′ is an element form, if the optimal response function is standard, the EUC game has a unique NE.

Theorem 3. The EUC game exits a unique NE if
which is equivalent to Proof. Positivity: From the constraint (34) and the equation (31) we can derive Therefore, the positivity is proved. Monotonicity: Given two different price vectors p and p ′ , now we assume that p ≥ p ′ . It means that p ≥ p ′ , ∀i ∈ N . If and then we can prove that monotonicity is established. Therefore, the problem transforms to proving J i ∕ p i ≥ 0 and J j ∕ p i ≥ 0. Then taking the first-order partial derivative of J j with respect to p i , we have In the same way, we can also conclude that Thus, the monotonicity is proved. Scalability: Comparing J(p) to J( p), we can obtain According to (34), we know that J(p) > J( p) for all > 1. Thus we can prove the scalability. Therefore, we can prove that J(p) is standard and the NE is unique. □

System implementation
The implementation process of the system is shown in Figure 2. Firstly, the EUCs choose an agent to represent them by resolution. Secondly, in the wholesale market, the GC and the agent negotiate the wholesale price based on the RBS bargaining strategy. Furthermore, in the retail market, the EUCs formulate their optimal retail price based on the wholesale price.
The decision-making process of retail price is constructed as a potential game model. Finally, the EUCs announce the optimal retail price to users. Through the proposed two-stage pricing model, the optimal wholesale and retail prices in the electricity market can be Algorithm 1 Price Iteration Algorithm Initialization: Initial wholesale price w 0 , retail price p 0 .
Set the iteration count t=0.
Set the maximum iteration number k max .
Set the maximum and minimum wholesale price.

Output:
The optimal wholesale price w * .
The optimal retail price p * .

4: end for.
The optimal retail price p * .
The optimal wholesale price w * . obtained. Compared with the existing pricing strategy, the twostage pricing strategy constructed in this paper considers more comprehensive factors, which can make the pricing process of the electricity market more complete, thereby improving the equilibrium and economy of the electricity trading process.

Optimization algorithm
In this section, a distributed algorithm is developed to obtain the wholesale price and retail price. Through this algorithm, the GC can obtain the RBS equilibrium solution and the EUC's strategy profile can be converged to the NE. The retail price p i in (26) can be calculated by the following algorithm where h i (p) = u i (p)∕ p i and p = (p 1 , … , p n ). In the distributed algorithm, k is the iteration number, p i (k) denotes the strategy profiles for EUC in kth iteration, and q i (k) represents the demand of the EUC. The distributed algorithm is summarized in Algorithm 1. The GC obtains the wholesale price through the RBS bargaining theory and announces it to EUCs. Then the EUCs interact with each other to determine the optimal retail price. In each iteration, the EUCs be update the price and their demand.
Remark 2. Iterative algorithm is a important optimization algorithm. In the process of pricing, each EUC will protect the privacy of its decision information as much as possible. However, due to the huge amount of information in the decision-making process of electricity price, it is difficult to realize the absolute privacy of the decision information of EUCs. The distributed algorithm can perfectly solve the game pricing problem among multiple EUCs and avoid the leakage of decision information of EUCs to the greatest extent. The iterative algorithm in this The profit ($) 10 5

FIGURE 3 RBS simulation diagram
paper has excellent convergence, which can reach convergence after 10 iterations.

SIMULATION RESULT ANALYSIS
In this section, the simulation analysis of the two-stage pricing model is given to analyze the effectiveness of pricing iterative algorithm and the superiority of the pricing model, respectively. Then the influence of demand disturbance on GC and EUC profit is analyzed and the relationship between bargaining model and game model is also studied.
In the simulation process, it is assumed that there is one GC and three EUCs to participate in the electricity trading process. Assuming the range of wholesale price is [0.1$/KWh, 0.5$/KWh], then the wholesale price can be obtained by formula (22). As shown in Figure 3, the optimal wholesale price is w * = 0.3 $/KWh. To further study the effect of w min and w max on RBS solution, letting the w min vary from 0.8 to 0.2 when w max = 0.9 and w max vary from 0.2 to 0.8 when w min = 0.1. Then the result is shown in Figure 4. The simulation results show that the RBS solution is linearly related to w min and w max . Some other parameter settings are given in Table 2.
Based on the wholesale price, EUCs will formulate the optimal retail price to optimize their profits. The simulation results of the pricing iterative algorithm are shown in Figures 5 and 6. From Figures 5 and 6, it is concluded that the retail price and electricity demand can converge to the equilibrium within 10 iterations. Hence, the pricing iterative algorithm has excellent convergence speed and convergence performance, and it can well solve the established two-stage pricing model. The optimal retail price and electricity demand of the EUCs are given in Table 3, Table 4.
The simulation result of EUCs' profits is shown in Figure 7. In Figure 7, the yellow bars and the blue bars  represent the EUCs' profits under the optimization model and EUC game model, respectively. The profits of the EUCs are given in Table 5. The data in Table 5 show that there is an error in social welfare between the EUC game model and the optimization model. Comparing equation (26) and (27), the effect of d ∑ n j ≠i [(p j − w j )] on retail price is not considered in the EUC game model. So the retail price occurs the error, which affects the social welfare. The error of the social welfare is 6%. The electricity demand (MWh) Demand of the first EUC by EUC game Demand of the first EUC by optimizing Demand of the second EUC by EUC game Demand of the second EUC by optimizing Demand of the third EUC by EUC game Demand of the third EUC by optimizing

FIGURE 6
The demand obtained by EUC game In Figures 8 and 9, the convergence of the pricing iterative algorithm with additive disturbance be analyzed. Figures 8 and  9 show the convergence process of retail price without disturbance and with the additive disturbance, respectively. The convergence process illustrate that the pricing iterative algorithm without additive disturbance can reach convergence within 10 steps, and the error of the pricing iterative algorithm with additive disturbance can be bounded by 2%. Thus, the proposed pricing iterative algorithm has a better robustness.
Next, the effect of different wholesale prices on NE retail price is studied in Figure 10. The results show that the retail price of EUCs will increases with the wholesale price, which is an approximate linear relationship. Furthermore, the impact of demand distribution on the profits of the EUCs and the GC is analyzed. The demand distributions of three EUCs are assumed to be [8MWh, −8MWh], [−2MWh, 7MWh], [−0.8MWh, 8MWh], then the simulation results are shown in Table 6, Figure 11 and Figure 12. From Figure 11, it is observed that the profits of the second and third EUC will increase with the demand distribution, however the profit of the first EUC is decreased The profit of EUC ($) 10 5 Profit of EUC by EUC game Profit of EUC by optimizing

FIGURE 7
The profits of EUC in optimization model and EUC game   The retail price convergence process with disturbance with the demand distribution. According to Figure 12, it is concluded that the profits of the GC from the second and third EUCs are improved with the demand distribution and the profit from the first EUC is decreased.

CONCLUSION
In order to improve the equilibrium of the electric energy trading process and the benefit of both parties, this paper proposes a two-stage pricing strategy based on bargaining theory and potential game theory. Through the two-stage pricing strategy, the problem of formulating wholesale price and retail price is solved. In the pricing process, it is proved that the wholesale price negotiation problem between GC and EUCs is a The profit of the EUC ($) 10 5 The profit of the first EUC The profit of the second EUC The profit of the third EUC

FIGURE 11
Profit of EUC versus demand disruption bargaining problem, and the optimal outcome of bargaining problem is realized by RBS. Moreover,the EUCs design their own optimal retail price based on the wholesale price issued by GC in order to maximize their benefit. Then the interaction relationship among EUCs is formulated into a potential game model, and a price iteration algorithm is designed to obtain retail price. Simulation results show that the potential game pricing model can simultaneously optimize the social welfare and the benefit of EUCs. Furthermore, the case simulation shows that the price iteration algorithm has good convergence and Robustness. In the future, we will study the electricity pricing process involving multiple GCs and select agents for GCs and EUCs. Then, the RBS negotiation process can be conducted between the two agents to determine the optimal wholesale price. The profit from the GC ($) 10 5 The profit from the first EUC The profit from the second EUC The profit from the third EUC The profit from the EUC The minimum benefit of the jth bargaining player. c s The cost of the generation. w max The maximum wholesale price. w min The minimum wholesale price. u A c , u A