Developing a new framework for transactive peer-to-peer thermal energy market

The rapid deployment of district heating systems in local energy markets and increasing the number of small-scale heat producers along with the expansion of local electricity markets increase the need for a transaction framework to manage the transactions between local participants in both heat and electricity markets. This paper presents a peer-to-peer thermal energy transaction framework to manage the transactions between small-scale heat prosumers. This framework enables small-scale thermal energy producers and consumers to participate in the market as price maker agents. Moreover, the optimal strategy of heat market participants is determined by proposing a linear proﬁt function for each agent. This optimization problem enables the agents to determine their optimal participation strategy in electricity and heat markets by addressing the interdependencies of electricity market, gas price, and heat market. The numerical results successfully demonstrate the beneﬁts and applicability of the proposed framework.


INTRODUCTION
With the increasing growth of distributed energy resources (DERs) in the distribution sector of energy systems, passive consumers will be transformed into active ones who can manage their consumption in response to fluctuations of electricity and gas prices [1]. In addition, the expansion of district heating systems and widespread utilization of solar water heaters along with small-scale natural gas boilers and combined heat and power units (CHP) will result in more interdependency between electricity and gas infrastructures [2]. These events call for new energy management procedures in energy infrastructures. In this regard, heat and electricity integrated energy systems (HE-IES) have attracted significant attention in recent years [3][4][5][6][7]. Authors in [5] have introduced a decentralized solution to identify the optimum scheduling of integrated heating and electrical systems in urban areas. In [6], a novel framework is proposed for stochastic planning of integrated energy systems, considering the uncertainty of demand and resources such as wind turbine generations. A linear model is proposed in [7] for the centralized dispatch for integrated heat and power systems by incorporating thorough modelling of heat storage tanks' charging processes. A dual-decomposition-based distributed algorithm is proposed in [8] to estimate the optimal dispatching of micro-integrated energy systems. Also, the main advantages of the proposed model over the traditional ADMM method in terms of dealing with convergence issues is investigated. Although the above-mentioned works have mainly focused on the coordination of power and heating systems, employing a market mechanism to facilitate energy transactions between producers and customers is missed in these studies.
Encouraging new investments in district heating systems necessitates the introduction of a heat market and also requires approaches that enable local prosumers to simultaneously determine their bidding strategies in both heat and electricity markets. In this regard, many scholars consider a similar structure for both heat and electricity markets. For instance, in [9], strategic bidding of an energy hub in electricity and heat markets is investigated based on a mathematic program with equilibrium constraints (MPEC) model. In [10], a two-stage gametheoretic model, considering thermal comfort, is introduced to trade both heat and power between integrated energy suppliers and distributed consumers. In [11], an incentive-compatible mechanism based on Vickrey-Clarke-Groves auction is applied to enhance the efficiency of integrated energy market (IEM) operations in real-time and day-ahead markets, considering high penetration of renewable energy sources. In [12], by considering demand elasticity and strategic providers, an integrated heatpower distribution system's market equilibrium is obtained, running the optimal thermal flow and the optimal power flow problems for clearing heat and electricity markets, respectively. In [13], a tri-layer market structure enabling simultaneous electricity, gas, and heat transactions at the wholesale level is proposed. Moreover, in the proposed model, demand response programs are addressed by defining load serving entities and maximizing their profit from participating in these programs. However, the proposed model is unable to properly consider the generation capacities of prosumers, especially renewable generations. Moreover, the market players are considered as price taker agents in the heat market. Although all of the reviewed studies have tried to propose market-based mechanisms for modelling the independencies between thermal and electrical energies, impractical assumptions overshadow their practicality.
In these studies, it has been assumed that the market is settled by a central authority who supervises the system. As a result, privacy issues arise when market participants tend to retain their privacy and are unwilling to cede full information of their configurations and preferences that are necessary to settle centralized market frameworks [14]. Moreover, when instead of selfdispatch by each market participant, central dispatch is utilized by central authorities, the uplift payment issue arises. It causes the market participants to transmit overstated cost information to market operators to compensate for the probable uplift costs [15]. Another disadvantage of central market mechanisms is the concerns related to the failure of the central market operator or any other core agent that may bring about the resilience issues [16].
In order to overcome the limitation of centralized market schemes, decentralized market mechanisms for integrated heat and electricity markets have been considered by a few researchers. In this context, a decentralized market framework, enabling energy transactions between district heating and power networks, is proposed in [17]. However, this reference only addresses the energy transaction between the district heating and power distribution networks and fails to consider transactions between different small-scale prosumers. Moreover, it does not consider competition in heat market since the contract price for thermal energy transactions is assumed to be fixed. In [18], a bilateral market framework is proposed to enable a coupled gas-electricity market in the distribution sector trading at locational marginal prices. However, this framework only enables the transaction between gas and electricity networks and does not consider the transactions between small-scale market participants and thermal energy transactions.
In brief, most of the existing decentralized frameworks fail to consider the market-based competitions in the heat market as well as the electricity market. On the other hand, the privacy issues and other inherent drawbacks of central market operation are an obstacle to consider them as an applicable market framework. Meanwhile, for electricity markets, peer-to-peer frameworks have been introduced to address the requisites of a well-formatted decentralized market framework.
Peer-to-peer markets enable the formation of mutually beneficial trading contracts between different agents based on agentto-agent negotiation [19]. For instance, in [20], bilateral contracts for peer-to-peer energy trading are proposed, considering agent-to-agent negotiations and real-time and forward market contracts. In [21], peer-to-peer trades are considered between the distribution system operator and prosumers participating in the local flexibility market. Authors in [22] have presented a market framework based on peer-to-peer negotiations to manage electricity transactions at the distribution level. This market framework considers electricity as a heterogeneous product concerning attributes of its generation source, i.e. green energy, fossil fuel-based generation, which are valued by each prosumer individually. These papers successfully model peer-to-peer electricity trading or ancillary services related to electrical energy in distribution networks. However, to the best of our knowledge, a peer-to-peer heat trading framework has not been introduced yet. The framework, presented in this paper, would address the aforementioned gaps in the reviewed scholarly works to manage the transactions of local participants in both heat and electricity markets.
Compared with the existing studies, the proposed framework does not face the aforementioned concerns that the centralbased market frameworks do. It would also implement market competitions between small-scale agents at the distributions level that the reviewed decentralized market frameworks fail to consider. Besides, the proposed framework results in a linear optimization problem that would guarantee the optimal solution for each market participant's strategy. However, the peer-to-peer market frameworks, presented in the literature, are modelled as some nonlinear optimization problems that means no guarantee for reaching the optimal solution and impose a more computational burden. In contrast to the reviewed peer-to-peer market frameworks, the proposed peer-to-peer framework can address the interdependencies and existence of both the heat and electricity markets. It would result in a better market strategy for agents, participating in both heat and electricity markets.
In response, this paper, for the first time, proposes a framework for peer-to-peer heat trading. Besides, the optimal strategies of small-scale agents who can simultaneously participate in local heat and electricity markets are determined. The main contributions of this paper are: • Design a decentralized market framework that enables peerto-peer thermal energy trading. • Determine the optimal strategies of agents participating in both electricity and heat markets by a linear model. • Address the interdependencies between electricity, gas, and district heating networks by considering the optimal behaviour of market participants in these markets.
The rest of the paper is organized as follows. Section 2 presents an introduction to the proposed market. Section 3 elaborates on the profit optimization problem of all the market participants. Section 4 describes the methodology of the proposed heat market. In Section 5, demonstration of the proposed heat market framework is attributed by presenting the results of the

GENERAL STRUCTURE OF THE PROPOSED PEER-TO-PEER HEAT MARKET
In the proposed framework, it has been assumed that market participants can purchase natural gas, trade electricity in the dayahead local electricity market, and also participate in the dayahead peer-to-peer heat market to conduct transactions. The agents may have combinations of the following installations and appliances, depicted in Figure 1. Since this paper is concentrated on the peer-to-peer heat market, a simple conventional transactive market structure is assumed for the electricity market, in which the agents are assumed to be price takers. The conceptual figure representing the peer-to-peer heat market for N market participants is illustrated in Figure 2. As it can be inferred from this figure, each agent negotiates directly with other agents to broadcast its contracts. The results, accepted/rejected contracts, are also broadcasted between agents by direct negotiations. Besides, the conceptual figure of the agents' participation in the markets is illustrated in Figure 3.
The proposed peer-to-peer heat market is operated based on an iterative algorithm, as depicted in Figure 4. In each market iteration, agents run their profit optimization problem, explained in Section 4, to identify their price-quantity contracts to propose for all 24 h of the day-ahead heat market and their optimum operation schedule for the next 24 h. Then agents  The proposed framework for the participation of agents in the markets broadcast their selling and purchasing contracts to the other heat market participants. The agents with approximately the same selling and purchasing contract prices will be matched, and the associated contract will be accepted. Besides, if the mutual contracts' prices are not close enough to settle the contracts, the agents would inform each other that their proposing prices are rejected.
Subsequently, agents modify and resubmit their contract prices to the upcoming market iteration to make their rejected contracts be accepted by other agents. In this regard, producers would reduce their rejected contract prices, and consumers would increase their rejected contract prices by adding a constant step. This procedure will continue in the next iterations of the market until there is no remaining offer in the heat market, or the agents are not willing to modify their contract prices in two consecutive iterations anymore. Therefore, the heat market is finalized, and agents realize the final thermal energy contracts to fulfil in the next day. Then agents re-run their profit maximization problems to determine the required natural gas to purchase from the gas distribution system and associated electricity that has to be traded in the electricity market to reach the optimum operation of the next day.

THE PROFIT OPTIMIZATION PROBLEM
As explained in the proposed framework for the heat market, each agent in each heat market iteration needs to run an optimization problem to determine its selling/purchasing contracts. This linear optimization problem is as follows: The first term of the objective function indicates the agent's net revenue of electricity exchanges, with the underlying assumption that the agent can participate in the electricity market right after the current heat market iteration. The second term indicates the agent's net revenue achieved as a result of participating in the heat market if all its proposed contracts are accepted in the upcoming market iteration. The next term quantifies the cost of consumed natural gas. Carbon emission penalties and operating costs of micro CHP are also considered in the function's two last terms. It should be noted that the price of carbon emission penalty, CEP, is considered 20 $ ton of carbon emission [23]. Also, the total carbon emission penalty costs are calculated based on the boiler and CHP output thermal, electrical energy, respectively [24][25][26]. It is worth mentioning that the proposed formulation is general, and this term can be omitted if there is no regulation on environmental pollutions. The associated constraints of the linear optimization problem are: Equations (2) and (3) satisfy the loading limit of the transformer and electricity-to-heat appliances. Equation (4) determines the net exchanged electricity of the agent with the grid. Power and heat balance constraints are enforced in Equations (5) and (6), respectively. The natural gas input rate limit in each time interval t is determined by Equations (7)- (9). The state of charge of electrical (e) and heat (h) storage units is evaluated by Equation (10), and Equation (11) ensures that the energy stored in the first time interval of the time horizon to be equal to the energy stored at the first time interval of its next day. Finally, input and output power rates and the maximum capacities of storage units are restricted in Equations (12)- (14).
The abovementioned optimization problem determines the optimal strategy of a typical agent with micro CHP, boiler, solar water heater (SWH), photovoltaic arrays, air conditioners, gas to heat, and electricity to heat appliances. However, if an agent does not have either of these appliances, the associated constraints and variables of the specified item would be ignored for that agent.

OPERATION OF THE PROPOSED PEER-TO-PEER HEAT MARKET
As discussed earlier, the proposed framework is based on an iterative algorithm for the heat market. As it can be inferred from Figure 4., the agent's first iteration contract prices are considered as input data in the algorithm and are obtained before the first market iteration. To do so, an agent for each time interval, according to the operating costs of its energy converters and sources, defines a maximum and minimum price that each 1 kWh of thermal energy is valued to him. Therefore, the agent can choose a price within this price interval to propose in the heat market to sell or purchase every unit, 1 kWh, of thermal energy. The initial maximum and minimum prices, which are defined before the first iteration for the selling and purchasing contracts, are denoted as h Meanwhile, for the selling contracts, if an agent can sell its generated thermal energy at the highest possible price, it will be willing to offer the maximum quantity of thermal energy to gain maximum benefit from energy exchange. Whilst for a buyer agent, if the agent can purchase thermal energy at the lowest possible price, it will be willing to purchase the highest amount of thermal energy. Therefore, as agents selling (purchasing) thermal energy try to maximize their profit, they are willing to propose their defined maximum (minimum) prices as contract prices to gain more benefit from thermal energy transactions. This means that the contract prices for each agent to propose for 1 kWh of thermal energy at each time interval for the first market iteration is as follows: When these prices are obtained, the heat market can be started. To do so, in each iteration, the following steps are taken.

Forming negotiable contracts
As the first step of each market iteration, the purchasing and selling contracts of agents in each time interval are obtained running the optimization problem addressed in Equations (1)- (14), which are denoted as In the proposed market framework, these contracts are called negotiable selling/purchasing contracts. Negotiable contracts are contracts that their associated prices can be changed in the next iterations. On the other hand, there is another category of contracts whose prices are fixed in different iterations. As a note, the concept of negotiable and non-negotiable contracts is further discussed in the following subsection.

Forming non-negotiable contracts
Non-negotiable contracts are those contracts whose prices are defined once and cannot be changed any further during market iterations. As mentioned earlier, in each iteration, agents may reduce (increase) their selling (purchasing) contract prices to sell (purchase) their available (required) thermal energy. However, reducing (increasing) the proposed thermal energy price would also reduce the proposed quantity since the agent is less willing to trade energy at lower (higher) prices. Thus an agent can propose the amount of reduced proposed quantity at higher (lower) prices of the previous iteration while reduces (increases) the proposed price for the remaining available production capacity (required demand). This means that in each iteration, there could be some newly identified non-negotiable contracts that can be proposed to the market for the current and subsequent iterations, while the remaining portion of the production capacity (required demand) can be proposed with lower (higher) prices. The contracts with reduced (increased) prices that can be reduced (increased) further in next iterations are called negotiable selling (purchasing) contracts. As an example, for the selling contracts, this concept is illustrated in Figure  5, representing hypothetical sample participation of a prosumer agent in the heat market. As depicted in Figure 5, after each market iteration, the agent reduces its selling contract prices to reduce the gap between its offered prices and other agents' purchasing contract prices for the mutual contract. Consequently, the agent's offered quantity for thermal energy with negotiable contracts might be reduced in each iteration in comparison with its previous one, which creates a gap between the iterations' proposed quantities. The sources of this gap are illustrated in Figure 5, sold thermal energy in previous iterations, and possibly identified nonnegotiable selling contracts in each iteration. It can be inferred from the figure that the agent would offer all its generating capacity in the first iteration when the offered price is high (P 1 ).
However, it fails to sell any quantity of thermal energy in the first iteration. Consequently, it reduces its offered price by a constant step, which results in the price of P 2 that leads to offering 0.8 Q m of negotiable selling contract, 0.2 Q m less than the previous iteration. This reduced quantity is due to the reduction of the contract price that brings about the non-negotiable contract with P 1 price. This contract can be offered in this iteration and the following ones with this fixed price. Subsequently, the agent reduces its offer price in the third iteration, which leads to forming a negotiable contract with P 3 price and quantity of 0.7 Q m . The reduced quantity, 0.1 Q m , is the result of sold thermal energy in the second iteration at P 2 price. In the next iteration, the agent reduces its offer price again that leads to an offer of 0.4 Q m at P 4 price. The reduced quantity, 0.3 Q m , is the result of 0.2 Q m sold thermal energy in the third iteration and 0.1 Q m of the reduced quantity that can be proposed with a nonnegotiable selling contract with P 3 price in this iteration and the subsequent ones. As an example, the identified non-negotiable selling contract with P 3 price is As a result, total selling (purchasing) contracts of an agent, for each time interval, and in each market iteration is the union of the negotiable selling (purchasing) contract in that iteration, with the union of the non-negotiable selling (purchasing) contracts from the previous iterations until the current one as

Contract broadcasting and selection
When the potential contracts are determined by the agents, each agent broadcasts its available contracts. Subsequently, agents compare their contract price for every unit of thermal energy, 1 kWh, in each time interval with the broadcasted contract prices, which they have received from other agents. For each mutual contract proposed for the same time intervals, if the prices are close enough to satisfy the below constraint, the contract is finalized between the agent and the other agent, proposing the contract. P s i,t,k − P P j,t,k < P s j,t,k − P P i,t,k < It is worth mentioning that the transacted quantity for accepted mutual contracts between every two agents is the minimum proposed quantity of the two agents proposing that contract. Also, if a purchasing contract price of a buyer agent matches the selling contract price offered by more than one seller agent simultaneously, the quantity of the contract to trade with the qualified sellers is allocated based on the weighted average of their contracts' proposed quantities. As a result, at the end of this step, agents identify their accepted and rejected contracts.

Modifying contract prices for rejected contracts
For rejected contracts, agents realize that their broadcasted prices are not acceptable for other agents. Therefore, for purchasing (selling) contracts, they increase (decrease) their rejected contracts' prices for the next market iteration by a constant step. This increase (decrease) is done if the modified prices do not violate the maximum (minimum) predefined limits for agents in each time intervals, which were identified before the first market iteration. This concept is shown in Equations (24) and (25)

Energy contracts broadcasting
Finally, for each time interval, agents realize their successful selling and purchasing quantity of thermal energy from the beginning iteration until the current one. The total purchasing quantity of negotiable contracts for each agent are in which the PN i,t,l is the quantity that an agent succeeds in purchasing in each specific iteration and time interval of negotiable purchasing contracts. Also, the total purchased thermal energy of the non-negotiable purchasing contracts of an agent equals: Total sold heat, from the first market iteration until the current one, of negotiable selling contracts for an agent, TSN i,t,k , is equal to: Similarly, the total sold thermal energy of the non-negotiable selling contracts of an agent, TSNN i,t,k , equals: As a result, each agent's total sold and purchased quantities of thermal energy is obtained from the following equations, respectively: The total net revenue associated with the sold and purchased thermal energy quantities from the beginning iteration until the current one can be obtained for each agent as follows: As discussed earlier, these iterative processes are done until there is no remaining offer in the heat market, or the agents are not willing to modify their contract prices in two consecutive iterations anymore. It is worth mentioning that the iterative market framework is guaranteed to converge. Since, as mentioned before, the seller (buyer) agents would decrease (increase) their proposing prices step by step in consecutive market iterations until these prices equal the defined minimum (maximum) prices, which the agents determine individually for each time interval before the first market iteration. These maximum and minimum prices are limited values, constrained by each agent's heat generation costs. So, after running some market iterations, the seller agents' contract prices would reach their defined minimum prices. On the other hand, the buyer agents' contract prices would also reach their defined maximum prices. So the agents would not be willing to adjust their proposing prices anymore, which results in the finalizing of the heat market.
When the heat market is finalized, the agents, based on their accepted contracts, re-run their profit optimization problem to participate in the electricity market and purchase gas from the gas distribution system. In order to summarize the aforementioned processes, the flowchart of Figure 6 illustrates the

RESULTS AND DISCUSSION
In this section, the simulation results in a case study are presented to demonstrate the feasibility of the proposed algorithm. The simulations are done using the Cplex solver in General Algebraic Modelling System (GAMS) on an Intel Core i5 with a 2.4 GHz processing clock and 4 GB of RAM.

System under study and main assumptions
In this section, the numerical results of implementing the proposed peer-to-peer thermal energy market are illustrated. It is worth mentioning that the required market iterations equal 40, and the computational time of the whole market's clearing is equal to 15 min since all the computations are done through a single computational system. However, in real applications, the aforementioned processes are done individually by each agent, and this would decrease the computational time significantly. The value of ε is considered as 0.003 $. The price adjustment parameter, ∆b, is also considered as 0.0015 $ kWh . It should be noted that any other values can be considered for these two parameters, resulting in different computational time and contracts' prices accuracy. The case study consists of 50 market participants, including 20 prosumers who can sell or purchase thermal energy and 30 consumers who only consume thermal energy. The electrical load data and PV units' generations are obtained from the UK Customer-Led Network Revolution project's data [27].
The solar water heater generation profile, thermal and cooling energy demands are also assumed based on the modification of data obtained from [24]. The agents' installations parameters are listed in Table 1 [28]. For a better comparison between different prosumers' optimum strategy and their profit objective values, the load profiles are assumed to be similar for all prosumers.

FIGURE 7
Forecasted electricity price [30] The prosumers are categorized into three groups, AS 1 , AS 2 , AS 3 . Each agent in the first category of prosumers has a micro CHP to supply their thermal and electrical demand. The second category's agents have a boiler to supply their thermal demand. The last category's agents have SWH to supply their thermal demand, yet due to the uncertainty and lack of enough supplied energy of SWH, this category's agents are also assumed to be equipped with gas-to-heat appliances. Moreover, all of these categories are assumed to have PV units with a similar generation profile. The consumers are also categorized into two different categories, AB 1 , AB 2 . The first category's agents have only electricity-to-heat appliances. The second category's agents have only gas-to-heat appliances. None of the consumers have PV units. The gas price is considered as 3.3 ¢$/kWh constant during the whole day [29]. As mentioned earlier, the agents are considered price takers in the electricity market with an accurate price forecasting unit. The forecasted electricity price for the next day is considered as depicted in Figure 7 [30].
In the following, the proposed market is implemented, and the results are elaborated.

The optimum strategy of market participants
In this part, the heat market is executed for the next day's 24 h, and the optimum strategy of different agents is obtained after market closure. One agent from each category of market participants is selected, and its optimum strategy is analysed. The result of a sample agent from the first category of prosumers, P 1 , is illustrated in Figure 8. As can be inferred from Figure 8, in most time intervals, P 1 not only satisfies its own demand but also sells thermal energy to the market. This is because of the high electricity price, which justifies the operation of the micro CHP and enables the agent to offer lower selling contract prices for thermal energy. On the other hand, for time intervals 2 AM and 3 AM, P 1 decides not to operate its micro CHP as low electricity price and low thermal demand in the market leads to utilizing the energy stored in the storage unit to satisfy its thermal energy demand. The next agent is a sample agent from the second category of prosumers, P 2 . As can be inferred from Figure 9, in most cases, P 2 succeeds in selling thermal energy to other agents and also uses its own generation to supply its demand. Although the whole market's thermal demand is low at 2 AM and 3 AM, the agent sells a high quantity of thermal energy in these time intervals. This is due to the fact that the prosumers who have micro CHP, AS 1 , and prosumers who have a solar water heater, AS 3 , are not willing to generate thermal energy in these time intervals due to the low electricity price or lack of solar generation. Hence, P 2 and other AS 2 agents, benefiting from high gas-to-heat efficiency, can sell a high amount of thermal energy in these time intervals. On the other hand, at midnight and 5 AM, due to the low thermal energy demand of other agents and a medium electricity price that causes the AS 1 prosumers to offer low selling contract prices, P 2 fails to sell thermal energy in the heat market and utilizes its whole generation to satisfy its demand. It is worth mentioning that due to the constant gas price throughout the day, P 2 does not utilize its heat storage.
The next agent is a sample agent from the third group of prosumers, P 3 . In Figures 10 and 11, the generation profile of the solar water heater and P 3 thermal energy transactions are illustrated. Due to the low thermal energy generation by the SWH, P 3 in most time intervals purchases thermal energy from the market to supply its own demand yet succeeds in selling its excess energy in the heat market at some hours. Despite the high generation at 3 PM, P 3 is not willing to propose its full capacity in the heat market and stores its excess thermal energy in the heat storage and utilizes the stored energy at 5 PM and 6 PM. This is because of the fact that P 3 can sell thermal energy at 5 PM and 6 PM at a higher price due to the higher electricity price, which leads to high purchasing contract prices by AB 1 consumers. One of the inherent benefits of the proposed mar-FIGURE 10 Solar water heater generation profile [22] FIGURE 11 Thermal energy trade of P 3 ket for each prosumer is the ability to sell and purchase thermal energy for the same time interval in different iterations. This means that an agent like P 3 can sell and purchase thermal energy for the same hour at different market iterations. For instance, at 6 PM, P 3 uses its heat storage to sell 0.528 kWh of thermal energy at 5.8 ¢$/kWh in the 21 st market iteration. On the other hand, it purchases 1.34 kWh of thermal energy at 4.45 ¢$/kWh in the 28 th market iteration to satisfy its thermal energy demand.
The next agents are selected from consumers' categories, C 1 from AB 1 and C 2 from AB 2 . These agents succeed in purchasing thermal energy to supply their demands for the whole day. This is because the prosumers can generate and propose thermal energy at lower prices than the consumers' generation costs, which seems reasonable for the consumers to accept the prosumers' selling contracts' prices. The results for C 1 and C 2 trades are illustrated in Figures 12 and 13.

Comparison between the proposed market framework and other strategies
In this part, the pros and cons of the proposed method and two other common strategies for settling the heat market are put under investigation. Since, in most cases [31][32][33], a central pool-based heat market framework is proposed, this market structure is applied to the system under study. To do so, each market participant submits quantity and price bids for each hour of the next day along with their technical operational limits. Subsequently, the market operator collects all the demand and supply bids and clears the market for each time interval. As a result, the final heat transactions along with their prices, which each agent should fulfil in the day-ahead heat market, will be realized.
Meanwhile, another common strategy to settle the heat market transactions is applied as the second method for the sake of comparative studies. Based on this method, a constant price for each 1 kWh of heat in the heat market is taken into consideration [17]. Thus the agents would only submit quantities of heat as bids in the heat market. In order to implement this strategy in the same case study, heat price is considered as 6.5 ¢$/kWh and constant during the whole next day [34]. In the following, the proposed peer-to-peer market framework and the other two strategies are employed, and the results are investigated.
To do so, the total net costs of different agents are reported in Figure 14. It is worth mentioning that all consumers' net costs represent the summation of the total net costs of AB 1 and AB 2 agents. Also, all prosumers' net costs indicate the summation of total net costs of AS 1 , AS 2 , and AS 3 agents. Similarly, all agents' As can be inferred from Figure 14, although the prosumers' total net cost in the proposed method has the highest amount among other strategies, the consumers' total net cost has decreased in a manner that results in the lowest total agents' net cost among other two strategies. Thus, it can be concluded that the proposed method has settled the market more fairly in comparison with the other two strategies.
Moreover, the total transacted thermal energy under the aforementioned market schemes is investigated, as depicted in Figure 15. As can be inferred from this figure, the total transacted thermal energy in the peer-to-peer heat market has the highest amount among two other strategies. This is due to the price adjustment process in the peer-to-peer market, which enables the agents to resubmit their proposing prices in the market if they are not acceptable by other agents. Thus, more trades have been made in the proposed peer-to-peer heat market. Besides, as mentioned before, the proposed market framework would eliminate the concerns related to central-based market frameworks while the other two common strategies are dealing with. In this regard, the privacy concerns, computational burden, and core agent's failure issues are eliminated in the proposed peer-to-peer market framework.

CONCLUSION
In this paper, a peer-to-peer framework is designed to manage thermal energy transactions in district heating systems. In this regard, at first, agents' bidding strategy to participate in the heat market is investigated by a linear optimization problem. Introducing different strategies for selling/purchasing contracts of agents, the main steps of running the proposed heat market are thoroughly discussed. The proposed framework also addresses the interdependencies between the electricity market and the heat market as the electricity price effects on agents' optimum behaviour in the heat market clearly illustrate this concept. The numerical results demonstrate the efficiency of the proposed framework that not only all of the market participants gain benefits from participating in the market but also even small-scale heat producers can be regarded as price maker agents. The linearity of the proposed optimization problem makes the applicability of the proposed framework easier due to the reduction of the required computational burdens. Therefore, it can be applied by smallscale domestic prosumers, lacking the high-performance computational tools.
For future studies, the proposed market framework can be extended to address the peer-to-peer electricity and heat transactions between the agents at the distribution level. This market framework would not only obtain the optimal strategy and operation of the agents but also would manage the electricity and heat transactions between the agents by such a previously explained iterative market framework.

Nomenclature
Set T Time periods, T = {1,2,…,24} Indices i,j Index for market participants t Index for time periods, t ∈ T k,l Index for heat market iterations Parameters e t , g t Electricity, gas price in each time interval PV t Output power of photovoltaic system in t (kW) Psw t Output power of solar water heater (SWH) in t (kW) PT Transformer power limit (kW) EL i,t Electrical Load of participant i at time t HL i,t , CL i,t Heat, cooling loads of participant i at time t T Transformer efficiency AC Air conditioner efficiency gte , gth Electrical, thermal efficiencies of micro CHP b Boiler efficiency gtha Gas-to-heat appliances efficiency etha Electricity-to-heat appliances efficiency E st Loss of standby stored energy + , − Storage charging, discharging efficiencies Q,Ē Storage maximum output power and capacity P c , P b Maximum micro CHP, boiler output power (kW) P gtha , P etha Maximum input power of gas-to-heat, electricityto-heat appliances (kW) CEP Carbon emission penalty OC Operation cost of micro CHP unit Variables NNC s i,t,k (NNC p i,t,k ) Non-negotiable selling (purchasing) contract TS i,t,k Total sold thermal energy TP i,t,k Total purchased thermal energy QNN s i,t,k (QNN p i,t,k ) Proposed quantity of non-negotiable selling (purchasing) contract TTHR i,K Total transacted heat revenue of agent i P i Profit objective value of agent i Pe i,t Net electricity traded with the grid Pe eth,i,t Purchased electricity from grid to supply heat Pg gtha,i,t Purchased gas from grid to use in gas-toheat appliances Pg c,i,t Purchased gas from grid to use in CHP Pg b,i,t Purchased gas from grid to use in boiler Q i,t Storage charging or discharging power E i,t Storage energy capacity CE c (CE b ) Micro CHP (Boiler) carbon dioxide emission NC s i,t,k (NC p i,t,k ) Negotiable selling (purchasing) contract C s i,t,k (C p i,t,k ) Selling (purchasing) contract P s i,t,k (P p i,t,k ) Selling (purchasing) contract price SN i,t,k (PN i,t,k ) Sold (purchased) thermal energy with negotiable prices in each iteration SNN i,t,l (PNN i,t,l ) Sold (purchased) thermal energy with non-negotiable prices in each iteration QN s i,t,k (QN Min. and max allowable selling contract price of agent i in time interval t P i,t (P i,t ) Sold (purchased) electricity to (from) grid