A bang‐bang control based water‐loop heat pump load aggregation method for power levelling dispatch

The National Nature Science Foundation of China, Grant/Award Number: 51837004 Abstract In this article, a bang‐bang control based water‐loop heat pump (WLHP) load aggregation method is proposed to facilitate its attending demand response service for the purpose of power levelling dispatch. A WLHP aggregation model is first established to determine its power limit, in which a priority list based on the state of temperature (SOT) index for each heat pump unit (HPU) is developed to measure the HPU's power regulation capability. Users' comfort is also well guaranteed by introducing a forced switching rule in the model. The proposed model is then applied for a power levelling dispatch problem with WLHP load and a bi‐section based bang‐bang control algorithm is developed to determine HPUs' ON/OFF states. A microgrid test system including wind and solar generation, WLHP load and a battery storage is used to validate the effectiveness of the proposed method. To further test its robustness, an influence analysis of initial operating conditions such as input power profiles and initial ON/OFF states as well as multiple operating‐condition simulations are both conducted.


| INTRODUCTION
The intermittency and fluctuation of the distributed renewable generation and new types of loads can cause power quality and stability issues in the power system, which bring serious challenges to dispatch and perform operation [1][2][3]. To deal with such issues, demand response (DR) has been widely studied to alleviate the imbalance between the supply and demand, smooth power fluctuations, improve energy efficiency, etc. [4,5].
Air conditioning loads (ACLs) are considered as one of the most effective and controllable DR resources, for the reason that they can store electrical energy as thermal energy and shift electricity consumption from one period to another [6]. Usually, split air conditioning (SAC) loads and central air conditioning (CAC) loads are the two main types of ACLs. Compared with the former, a CAC system has a higher power capacity and thermal energy storage capacity, which is composed of a chiller, a circulating water system and some terminal devices such as heat pump units (HPUs) equipped in the user's side. In particular, the water-loop heat pump (WLHP) load is becoming a widely-used CAC load in large commercial buildings, which can bring about potential benefits for energy consumption [7], operating costs and environment [8]. However, from the perspective of the power system, their potential as controllable electrical loads has not been deeply explored yet. The lack of the related work is due to the special characteristics of WLHPs. On the one hand, they have some features similar to SAC (such as room temperature can be controlled individually); on the other hand, their electricthermal conversion process, which contains more variables and even non-linear relationships, is much more similar to the CAC's. Therefore, a literature review of the existing aggregated modelling and control for both SACs and CACs is presented as the basis for the WLHP participating in load dispatch.
In order to realise a centralised dispatch and control of SACs, many aggregated load modelling and control methods have been proposed. There are two common input control signals for SACs: one is the binary ON/OFF state sent by the bang-bang controller and the other is the adjustment of temperature setpoint [9]. According to the different input signals, the key idea of the aggregated load modelling is to analyse the ON/OFF state or power change of the SACs, and establish the relationship between the switching probability distribution or the aggregate power over time [10]. Lu et al. [11] proposed the state-queueing (SQ) model based on a temperature priority list to evaluate the aggregated ACLs' potential for providing load balancing service. Inspired by the above SQ model, Wang et al. [12,13] further proposed a coordinated control strategy of SACs to smooth power flow fluctuations. Reference [14,15] established a battery model of the aggregated ACLs to provide more adjustment information (i.e. the power limit and energy capacity of SACs) for operator. A coordinated state-estimation method for ACLs was proposed in [16] to provide primary frequency regulation service, which can reduce the coordination burden. To summarise, the conditions of application for the above methods are given as: 1) each terminal device can be controlled individually; 2) the electric power and the thermal power of SAC have a linear relationship; 3) the working modes (heating or cooling) for all SACs are the same. However, for the WLHP load, only the first condition is met and none of the above methods can be directly applied.
In comparison with SACs, CACs in large commercial buildings have more potential for DR due to their larger capacity and higher flexibility. However, they have complex structures and various control variables so that they cannot be aggregated using traditional SAC models. Some studies for CACs focus on optimising their operation modes to achieve energy saving and consumption reduction, and the control objects are usually the main device and the circulating water system. Such homogeneous regulation methods can affect the operating modes of all terminal devices, which reduce the flexibility of the user's choice [17,18]. Recently, some modelling and control methods designed for certain types of CACs participating in DR have been proposed. In [19], an aggregation strategy of CAC loads was proposed to smooth wind power fluctuation, in which the electric power of a single CAC was assumed to have a linear relationship with its thermal capacity for simplification. Gao et al. [20] proposed three CAC load aggregation strategies based on the CAC load characteristics index, comfort index and economic index. However, all of them were established on the basis of homogeneous control methods for a single CAC. Wang et al. [21] presented a control strategy for the load rate CAC based on temperature adjustment of partial terminal devices, which can effectively guarantee user's comfort. Reference [22] conducted a thoroughly comparative investigation on the performance of the three existing methods, including one open-loop and two closedloop DR control structures, which indicate ACLs can provide an effective frequency regulation service. By using a constant ratio control for CAC regulation, [23] proposed a virtual peaking unit emulated by CACs for day-ahead power dispatch based on short-term CAC baseline load forecasting.
Inspired by the related work above, an aggregation modelling method for WLHP loads in two full working modes is established based on the bang-bang control for the power levelling purpose. Based on the framework of our previous work in [24], this paper first develops a temperature state index SOT to measure user's comfort degree. Then, an aggregated WLHP modelling method is proposed to determine WLHP's power limits, in which a priority list based on SOTs is developed to coordinate the bang-bang control of each HPU in WLHP. Furthermore, the proposed model is applied for a power levelling optimal dispatch problem with the WLHP load. The main contributions of this paper are listed as follows: 1) An aggregated WLHP model with the forced switching rule for HPUs is proposed for the power levelling dispatch and a bi-section-based bang-bang control algorithm is developed to determine each HPU's ON/OFF state; 2) A user's comfort index SOT is developed to unify HPUs' different working modes and measure their power regulation capability so that the dispatchable power range for each WLHP can be determined.
The rest of this article is organised as follows: a WLHP aggregation model is first established in Section 2; then the power levelling optimal dispatch problem for WLHP load is investigated in Section 3; further, simulation studies based on initial operating conditions analyses and multiple scenarios are presented in Section 4; finally, the conclusion and future work are summarised in Section 5.

| WLHP LOAD AGGREGATION MODELLING METHOD
WLHP systems are typically installed in buildings with distinguished inner and outer zones [8]. As is shown in Figure 1, the WLHP system is composed of many water source HPUs linked by a closed water-piping loop. Each HPU can be regarded as a terminal device, which is independently controlled by its user. The water in the loop, which is represented by arrows, can be regarded as a heat source/sink for each HPU. It can store the heat inside a building and meet the different requirements of cooling and heating of each HPU during a certain period of time [7].
In this section, thermal dynamics of HPUs are described first as the basis of load aggregation and a temperature state index is defined considering the full working modes of the WLHP load. The aggregation model is established in three steps and finally determines the dispatchable power limits for the optimal dispatch.

| Thermal dynamics of HPUs
Every HPU in the WLHP load can be described by a simplified first-order equivalent thermal parameter (ETP) model to simulate its thermal dynamics behaviour [13]. HPU has the following two full working modes, and for the kth HPU its corresponding indoor temperature at time t can be described as: a) Heating mode: b) Cooling mode: where τ i,k and τ o,k are the indoor and outdoor temperature; C k is the indoor air heat capacity; R k is the mean envelope thermal resistance; Q H,k /Q C,k is the operational heating/cooling rate; q k is a 0-1 variable representing the OFF or ON state of HPU; Δt is the time interval (Δt = 1 min is set in the study).

| Definitions for SOT
Assuming that τ set,k is the setpoint and τ set,k ± δ k is the acceptable temperature range for users. For the kth HPU working in heating mode, its SOT is: For the kth HPU working in cooling mode, its SOT is: where τ max,k and τ min,k are calculated by τ set,k +δ k and τ set,k −δ k . The SOT index reflects the relative position of the current temperature in the acceptable temperature range, which is between 0 and 1 when the user's comfort is satisfied. Different calculation formulas for different operation modes are designed for uniform sorting in the following procedure. Load diversity is well considered in calculating SOTs for HPUs with different thermodynamic parameters.

| Aggregation implementation
There are three steps for aggregation of HPUs, which are explained as follows. Without ambiguity in the context, the superscript t of the variables for time-interval indication hereinafter is omitted for simplification.

| Grouping and sorting
HPUs of WLHP are divided into two 'ON' and 'OFF' groups according to their ON/OFF states. To achieve the maximum regulation capacity, the following selection principle should be respected: 1) the HPU in the 'ON' group with the highest SOT is to be turned off with the highest priority; 2) the HPU in the 'OFF' group with the lowest SOT is to be turned on with the highest priority. To this end, the HPUs in the 'ON' group are sorted in descending order based on their SOTs. Similarly, the HPUs in the 'OFF' group are sorted in ascending order based on their SOTs.

| Forced switching rule
The purpose of adding this step in the original algorithm is to further ensure the user's comfort. For HPUs that exceed their indoor temperature limit, they are forced to change the ON/ OFF states under the constraint of forced switching SOT threshold. For 'ON' group, the index set of HPUs which have to be turned off at the next time step is denoted as: where index(⋅) is selecting HPUs which satisfy the SOT requirement and α 1 is the forced turning off threshold. Also, for 'OFF' group, the index set of HPUs which have to be turned on at the next time step is denoted as: where β 1 is the forced turning on threshold. The value of α 1 and β 1 are determined by the contract between the user and management centre.

| Candidate selection
For the 'ON' group, a threshold value α 2 for SOT is set to choose the controllable HPUs to be turned off, the index set of which can be denoted as: where descend(⋅) is sorting the selected HPUs in descending order according to their SOTs.
Similarly, for 'OFF' group, a threshold value β 2 for SOT is set to choose the controllable HPUs to be turned on, the index set of which can be denoted as follows: where ascend(⋅) is sorting the selected HPUs in ascending order according to their SOTs. By setting threshold values, we can avoid selecting the HPU that is prone to state reversal after control, which can effectively avoid frequent switching of the state. In order to make full use of WLHP load resources, choose α 2 ∈(0,0.5) and β 2 ∈(0.5,1).

| Maximum/minimum power limits evaluation
Suppose there are total N HPUs in one WLHP, M of them are working in the heating mode and the others are working in the cooling mode, then ON/OFF states of all HPUs for WLHP are denoted as: There are three power consumption modes of WLHP according to the water loop temperature τ W . As is deduced in [7], τ W can be calculated by: where ρ is the density of water in the loop; V is the volume of water in the loop and the water tank; C W is the specific heat of water; ε h /ε c is the average heating/cooling coefficient of performance (COP) of HPUs; Q H and Q C are the total heating/cooling loads, which are defined as: Suppose τ Wmin and τ Wmax are the lower and upper limit of the range of temperature inside the water loop, respectively, the power consumption of a single WLHP can be calculated as: where ε CT and ε b are the COPs of the cooling tower (including water pumps) and the electric boiler, respectively. All these COPs are given by manufacturers of WLHP.
By turning off all HPUs in A f set and turning on all HPUs in the B f set, the forced ON/OFF states setΦ and power consumptionP WLHP for the next time step is obtained.P WLHP will be used hereinafter to determine the status of increase of the power consumption.
By turning off all HPUs in A c set, we can have ON/OFF states set Φ min corresponding to minimum WLHP power for next time step, which are defined as: Similarly, by turning on all HPUs in B c set, the Φ max value corresponding to the maximum WLHP power is: Therefore, P WLHP,min and P WLHP,max can be added as constraints for WLHP load in the optimal dispatch model below.

| System dispatch modelling
The modelling and control method for WLHP loads can be used in active distribution networks. For simplification, the microgrid is taken as a typical application scenario, the configuration of which is shown in Figure 2. On the supply side, P W and P V represent the power generated by wind farm and solar plant. The power released by the battery storage is P BT and the tie-line power flow provided by the main grid is P TL . On the demand side, there are two types of loads: WLHP load P WLHP , and other uncontrollable loads P NL .
With increasing renewable energy penetration in microgrid, the intermittency of wind and solar generation will cause significant fluctuations in tie-line power flow. The optimization target is minimising the difference of P TL between adjacent time intervals, that is, During load dispatching process, the following constraints should be satisfied.

| Power balance constraint
The power balance inside community should always be satisfied without considering power loss, which can be written as: P t UN is defined as the power to be balanced in microgrid.

| WLHP constraint
The lower and upper power limits for WLHP are calculated by (15) and (17), so the power range for WLHP can be written as:

| Battery storage constraints
Charge-discharge power constraint for battery storage is: where P BT,max is the maximum discharge power.Capacity constraint for battery storage is: where E t−1 and E t are battery storage capacity at time t-1 and t. E max and E min represent the maximum and the minimum capacity, respectively. Equation (19)-(25) is a constrained linear least-squares problem, which can be solved by the optimization function lsqlin in MATLAB. Therefore, the power reference values for WLHP load and battery storage, that is, P t * WLHP and P t * BT are determined.

| HPU bang-bang control implementation
The central controller is designed to track P t * WLHP by sending bang-bang control signal Φ t to WLHP, which satisfies: For this one-dimension optimization problem, the following bi-section method is used to determine the ON/ OFF states of HPUs.

| Procedure of the proposed algorithm
The flowchart of the proposed algorithm for WLHP load dispatch is presented in Figure 3 and the procedure is summarised as follows: Step 1 All data above are collected by the microgrid measurement system.
Step 2 Based on the data collected, the following WLHP load aggregation modelling process is executed: -325

2.4) Evaluate the maximum and minimum WLHP power limits (15)-(18).
Step 3 Solve the power levelling optimal dispatch model to get the power reference values for WLHP load and battery storage according to (19)-(25).
Step 4 Use the bi-section method in Section 3.2 to determine each HPU's ON/OFF state.
Step 5 Repeat Step 1-4 over time with respect to the updated information until t = T sim .

| System description
A community-based microgrid as is shown in Figure 2 is studied in this paper, in which a WLHP load system including 2000 HPUs is used. In order to model the load diversity, thermo dynamic parameters for HPUs are randomized by the normal distribution. The main parameters are listed in Table 1. Figure 4 shows 24-h outdoor temperatures and real-time wind, PV and uncontrollable load power profiles for some typical working day in spring or autumn. Here are some assumptions for the case study: 1) 1500 HPUs in inner zones are working in cooling mode and others in outer zones are working in heating mode. 2) For outer zones, only heat exchange with outdoor environment is considered and outdoor temperatures in Figure 4a are taken as τ o,outer ; for inner zones, only heat exchange with neighbouring outer zones is considered and outer zone indoor temperatures τ i,outer are taken as τ o,inner .

| Base cases
The dispatch results of the following three cases are investigated and compared according to the five aspects: 1) the smoothing effect of the tie-line power; 2) the user's comfort; 3) the regulation capacity of WLHP; 4) the number of HPU ON/ OFF switching and 5) the charge-discharge frequency of battery storage. All HPUs are set to OFF initially for the following three cases.
Case A Only the battery storage operates as the dispatchable resource, while the WLHP is controlled by the commonly used thermostats according to the following logics [11]: a) Heating mode Case B Both the WLHP and battery storage operate as dispatchable resources, and the WLHP are controlled without the forced switching rule, that is, no temperature limitation is enforced.
Case C Both the WLHP and battery storage operate as the dispatchable resources, and the WLHP are controlled with the forced switching rule.
To quantify the performance of the method, two indices are defined as follows: (1) Variance of the smoothed tie-line power where Var% is the relative variance of the tie-line power, and δP TL is the power ramping rate. The smaller the Var% value, the better the smoothing effect.
(2) Up/down regulation capacity The up-and down-regulation capacity of WLHP at time t are defined as follows, which can which reflect the regulation margin for WLHP: The larger the value of P t up and P t down , the higher the regulation capacity of WLHP.

| Tie-line power
As is shown in Figure 5, power fluctuations of the tie-line, especially high-frequency components, are well smoothed after WLHP participating in the dispatch. The Var% values in case A, B and C are 5.26%, 0.12% and 0.11%, respectively. By comparing the latter two cases, we can find that adding forced switching rule to the original control algorithm does not cause any negative effects on the smoothing purpose.

| Indoor temperature
The comfortable temperature range set by users is 15-21°C for the inner zones and 23-29°C for the outer zones, both of which are outlined with thick black lines in Figure 6. When the WLHP is controlled by the thermostat in case A, indoor temperatures are strictly limited to the comfortable range (see Figure 6a). In case B, since there is no direct temperature limitation enforced, severe fluctuation in WLHP power occurs during 00:00-08:00. The forced switching rule in case C can effectively help to guarantee user's comfort as well as smooth power fluctuations (see Figure 6c).

| Regulation capacity of WLHP
Comparing Figure 7a with Figure 7b, it is observed that the regulation capacity of the controlled WLHP load has more obvious time-sequence characteristics than the thermostatcontrolled WLHP. For peak shaving and valley filling of P NL presented in Figure 4c, it can be observed that the upregulation margin of the WLHP is comparatively large in the F I G U R E 5 The tie-line power in case A, B and C SI ET AL.
-327 peak-load time (08:00-10:00 and 14:00-22:00) and the down-regulation margin is relatively small. Since case C has similar performance to case B, case B is taken as an example here.

| Number of HPU ON/OFF switching
The increase of HPU ON/OFF switching frequency reflects the loss of HPU unit caused by load regulation. As is shown in Figure 8, since the duration of the switching cycle (T on + T off ) for outer zones is longer than that for inner zones (see Appendices), HPUs in outer zones have much less ON/OFF switching than those in the inner zones.

| Power output of battery storage
When there is not enough regulation capacity for the WLHP, battery storage will act as a supplement by additional charging and discharging (see Figure 9b,c). Compared with Figure 9a, the charge-discharge frequency is obviously reduced after the WLHP participates in the load dispatch, which can prolong the service life of battery storage.

| Influence of power profiles
Here we will discuss the influence of the frequency bands of the power signals on the dispatch result. As is shown in Figure 10a, the high-frequency and low-frequency power signals filtered from P UN time-series signal in (21) are used to replace the original P UN signal respectively.
Case D1 Keep the high-frequency components of P UN only: where HPF(x) means removing the low-frequency components of the input signal x using a high pass filter. The passband frequency of the filter is set to 10 −4 Hz.
Case D2 Keep the low-frequency components of P UN only: where LPF(x) means removing the high-frequency components of the input signal x using a low pass filter. The passband frequency of the filter is also set to 10 −4 Hz. In the study, a comparison study among cases D1, D2 and C is conducted, in which case C is regarded as a base case for comparison seeing that only differences among them are their signal replacement schemes.
As is shown in Figure 10b, the Var% values of P TL in cases D1 and D2 are 0.081% and 0.12% respectively. Thus, the WLHP load has a better smoothing effect on high-frequency power profile (case D1) than the low-frequency counterpart (case D2). It can be seen from Figure 10c that the highfrequency power smoothing causes less switching loss for the WLHP. As is shown in Figure 11, although the WLHP regulation capacity in case D1 has small fluctuations, it is more reliable for DR during the total simulation time than case D2. Moreover, since the ON duration is shorter than the OFF duration when the WLHP works in steady state (see Appendices), P up becomes the main limit for the WLHP load to participate in DR during 11:00-12:00.
As a result, it is verified that WLHP load is more appropriate for smoothing high-frequency power fluctuations, and the up-regulation capacity is more limited.

| Influence of initial ON/OFF states
This subsection discusses the effect of the initial HPU ON/ OFF state setting on the control result, in which the following two cases are studied.
Case E1 All HPUs are set to ON initially.
Case E2 HPUs are set to half ON and half OFF initially.
As is shown in Figure 12a, both case C and case E1 show worse performance than case E2 in the early stage. It is due to the candidate selection rule in Section 2.3, which reduces both the up and down regulation capacity. The more difference between the number of initial ON and OFF states is, the more obvious weakening effect will be.
Moreover, since the OFF duration is much longer than the ON duration (see Appendices), there will be more HPUs in OFF states when reaching the steady state. Thus, the final states of HPUs are mostly off. This can explain why in Figure 12b the number of ON/OFF switching in case C and E1 fluctuates like a sawtooth. Also, for the reason that the difference between the number of the two states becomes larger, the regulation capacity for WLHP decreases and the smoothing effect in case E2 becomes worse after 18:00 (see Figure 12a). In summary, the initial ON/OFF states of HPUs will affect the resultant short-term load dispatch. It is suggested to select equal numbers of HPUs in ON and OFF states initially to achieve a better control effect in this case.

| Multiple time-scale simulation
To verify the robustness of the proposed method, the scenarios in hourly, daily and weekly time horizon are explored in this section. For the purpose of multiple time-scale simulation, the historical area control error (ACE) data between 8/4/2020 and 8/11/2020 from USA PJM system [25] are selected as the balanced power P UN data in (21). All data are scaled to 1-min signals and normalized to ±0.5 MW for all HPUs in the WLHP system.
The input unbalanced power P UN in microgrid and the resultant smoothed tie-line power P TL are shown in Figure 13.
The Var% values of case A, B and C in hourly, daily and weekly time horizon are compared in Table 2. All the values of case C are small enough for power levelling dispatch, which indicates that the proposed method is robust for multiple time horizons.

| Multiple seasons and weather conditions
Since outdoor temperature and uncontrollable load profile change a lot according to different seasons and can have significant influence on the dispatch result, typical days in four seasons are simulated in Figure 14. As is assumed in (1) and (2) in Section 4.1, 500 HPUs in the outer zones work in heating mode and they are all in OFF states because of over Other weather features such as sunlight and wind also have direct influence on P V and P W . Simulation results for cloudy, sunny, windy and windless days are presented in Figure 15 and Figure 16. Severe sunlight changes for solar plants will result in the WLHP staying in the peak-power status and unsatisfactory power smoothing effect (see Figure 15). Also, when there is surplus wind power which the microgrid can inject it into the main grid (see Figure 16).

| Comparison with the existing aggregation modelling method
To further justify the performance of the proposed SOTindex-based modelling method, we compare it with another temperature-based modelling method used in [13]. Considering the actual use of WLHP, in both methods the temperature value of each HPU is set at random according to a uniform distribution. The comparison results are shown in Figure 17.
The proposed method performs better than [13] in smoothing the fluctuation of tie line with a Var% value of 0.11% compared with 0.18% in [13] (see Figure 17a). Also, according to Figure 17b, the number of ON/OFF switching are less under the proposed method, which can prolong the service life of the WLHP system.

| CONCLUSION
This article proposes an aggregated WLHP model with the forced switching rule for HPUs and develops a bi-section-based bang-bang control algorithm to determine each HPU's ON/ OFF state. Besides, in order to unify HPUs' different working modes and measure their power regulation capability, a user's comfort index SOT is developed, by which the dispatchable power range for each WLHP can be determined. Several test cases are examined, and the results confirm that the proposed method is able to effectively smooth power fluctuations of tieline as well as guarantee the user's comfort. Initial operating condition effects including input power profiles and the initial ON/OFF states for HPUs are discussed. Further, scenarios of multiple time horizons and typical seasons and weather conditions can verify the robustness of the proposed method.
Our future work will focus on establishing the ON/OFF switching loss model to quantify the effect of load regulation on WLHP's lifetime thus achieving economic benefits by coordinating the WLHP load with other DR resources for the power levelling dispatch. Moreover, since in the paper it is assumed that all input data is measured or forecasted accurately, the method to consider uncertainties from both the supply and demand sides can be another potential direction.  -331

R k
The mean envelope thermal resistance of the kth HPU, ℃/W T sim Total simulation time, min V Volume of water in the loop and water tank, m 3 α 1 /β 1 SOT threshold for HPUs to be turned off/on by force α 2 /β 2 SOT threshold for controllable HPUs to be turned off/on for candidate selection τ max,k /τ min,k The maximum/minimum comfortable temperature for the kth HPU, ℃ τ set,k The temperature setpoint of the kth HPU, ℃ τ t W The water loop temperature at time t, ℃ τ 0 W The initial water loop temperature, ℃ τ Wmax /τ Wmin The maximum/minimum value of the range of temperature inside the water loop, ℃ δ k Temperature dead band of the kth HPU, ℃