The use of dynamic programming and golden section search for the optimal load-shedding strategy of HEMS participating in demand response program

This paper presents the optimal load-shedding strategy of home energy management sys-tems (HEMSs). The HEMS uses the dynamic programming (DP) method to plan the day ahead, schedule, and connects to smart plugs to perform load-shedding. HEMS solves the shedding schedule of smart plugs considering the prediction of consumption of each smart plug and the consumer’s demand response (DR) setting. In addition, the maximum DR capacity was determined by using golden section search (GSS). Some DR experiments were conducted by the proposed method in laboratory and these comprised three different DR durations. The results show that the estimated DR capacities and shedding schedules were feasible. Thus, it helps both the DR participants and aggregator to seize the optimum performance and beneﬁts.


INTRODUCTION
Traditionally, the participants in the demand response are both high voltage users and high consumption users. However, low voltage users account for about half of the total power consumption at peak hour, including both residential and commercial sections, and might be the significant contributors to the demand response. The residential customers receive less attention from the DR program providers due to the unpredictable load profile and the inconsistent reduction than industrial and commercial counterparts do. Hence, at the current stage, demand side management (DSM) ability in low voltage demand side has not been explored in depth. Recently, load aggregators (LA) and curtailment service provider (CSP) are going to integrate the flexible resources of demand side and then provide for market buyers. The electric grid can realize the energy demand reduction of low voltage users with the help of LA [1][2][3][4].
Among the DR programs, the peak time rebate (PTR) program is more attractive to the residential customers. The important idea of PTR is to offer a higher rebate price to customers who can bring about high and consistent reductions. The reduction is voluntary and the rebate price is fixed for any reduction This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. © 2021 The Authors. The Journal of Engineering published by John Wiley & Sons Ltd on behalf of The Institution of Engineering and Technology in PTR. As result, the actual reductions are often volatile and low, making it difficult for the LA or CSP to earn profits in the market [1].
An alternative to PTR is direct load control (DLC) programs. DLC programs for residential load management are based on an agreement between the utility, or an aggregator, and the customers. The utility, or an aggregator, can remotely control the operations and energy consumption of certain appliances [5]. However, the DLC will cause much inconvenience to residential participants who need to use appliances in shedding schedule [6].
This paper suggests that residential customers participate in the PTR program to avoid the penalties and use home energy management system (HEMS) to intelligently and automatically perform the load-shedding. HEMS executes monitoring and controlling of smart household appliances and information devices [7][8][9][10][11][12][13][14][15][16]. User-friendly HEMS can meet the electricity demands based on a user's setting [11]. In case of insufficient power supply, HEMS executes the load-shedding or shifting immediately using the direct load scheduling (DLS) method. The DLS can be set according to the interruptible load chosen by the consumer in advance [6,14]. The real research gap in the field of demand response includes not only user engagement and reward profit but also cost expansion. Hence, the cost reduction of HEMS including smart appliances is one of the most important issues [2]. This paper focuses on improving the flexibility of HEMSs coordinating with the LA to enhance the efficiency of demand management. If the HEMS can provide more information about demand response in different time interval, the LA can achieve maximum and stable demand responses for utility. This paper presented the evaluating of the feasible load-shedding for each HEMS by using dynamic programming and evaluating the maximum demand response in different time intervals by using golden section search.
In the following sections, the concept of DR and its calculation method has been explained, followed by an explanation of the DR program, dynamic programming, the HEMS architecture, and the demonstration. Subsequently, the calculation of the real-time control and the load-shedding achievement are discussed in addition to the different conditions being considered in the laboratory.

Definition of PTR demand response
The performance of the PTR program is made on the customer basic line (CBL) which should ideally be included in a user's contract and utility. In common, CBL is the average consumption observed at the same time on five working days, excluding the event days. Thus, the performance is dependent on the reduction of actual consumption and CBL [6]. Figure 1 depicts the relationship between the actual DR and CBL. The solid red line depicts the actual load power consumption, and the dotted blue line depicts the CBL, respectively. The effective DR is the CBL minus highest demand, rather than the CBL minus lowest demand in the DR duration. The effective DR can decrease owing to the suddenly high demand within basic interval of DR. The difference between highest demand and lowest demand causes the effort in vain. Thus, to control the consumption for each period is necessary, and it can help in avoiding waste of effort.   Figure 2 denotes the process of flow of the DR energy management. It can be divided into two parts, namely, the day ahead and the DR event day. The day ahead shedding plan and the load forecasting is carried out at the end of the day. If a user participates in the DR program, the command of the load-shedding from the previously planned schedule shall be executed on the event day [12]. In order to effectively achieve the target of load-shedding during demand response, the second degree polynomial of curve fitting method is used to predict the electricity consumption in total load and individual load. The curve fitting model is solved by the least square method.

Demand response objective function
To ensure that the load-shedding contributes to the effective DR, the amount of planned load-shedding (DR SCH ) must be higher than or equal to the target value (DR * ) at each period. However, it must be of concern that the user does not face any inconvenience at the time of over shedding and the user receives benefits from the DR event. Thus, the objective function can be set as the minimum magnitude of difference between DR SCH and DR * at each period. The DR SCH value is defined by the difference between the CBL and the scheduled load consumptions (P SCH ), as shown in Equations (1)- (5), and the process of calculation for each time period must conform to constraints [13]. The solution of the objective function can be calculated by adjusting the total power consumption of the controllable loads(P c DR ), which is affected by the scheduling commands (SCH) of smart plugs. On the event day, the smart plugs perform the action on the basis of the scheduled commands. If the load of the smart plug is as per the expectation, then the target set for DR limit can be achieved. The total load (W) of all non-controllable load at period t; SCH (n, t ) : The DR scheduling command of the nth controllable load at period t, 1 is turn on and 0 is turn off. N : The total number of controllable loads and is set to 12.
The constraints of the objective function are expressed by Equations (6)- (10). As per Equation (6), the planned DR should be greater than the target DR at each period to prevent under performance of the DR. Equation (7) depicts the total shedding times per day, which should be lower than the limit of smart plug. Equation (8) denotes the shedding of plug (n) in the time period t, and it is only allowed if the DR permission is enabled simultaneously. Equation (9) denotes the successive shedding of plug (n), which should not be longer than the maximum shedding period of plug (T max o f f ). Equation (10) represents the subsequent shedding, which should occur subsequent to the minimum operating period of plug (T min on ).
where N DR (n): The maximum shedding times per day of plug n.
where DR enable (n, t ): The user-defined DR permission of plug n at period t. (10) where n sh : The number of shaded of plug n. This objective function is non-linear programming which can be used to determine the optimal scheduling of appliances taking the shedding limitations of different appliances into consideration [13].
Since the smart plugs are able to control the switching on and off the electric power only, linear programming is not suitable for solving the DR schedule. Even though the performance of the genetic algorithm or mixed-integer linear programming (MILP) is better than that of the other algorithms, only 8-16 DR periods are available which are not sufficient to exhibit optimum performance. Thus, we used the dynamic programming method to achieve suitable combinations of the load-shedding.

Dynamic programming
In the dynamic programming methods, a problem is divided into front and back steps. Each step is characterized by a number of possible starting states which must perform the function of decision-making. At the beginning of the current step, all possible starting states corresponding to the next step are obtained in the process. The process of decision-making of the previous step does not affect the subsequent processes and it is subsequently transferred to the next step. In the end, the best solution can be obtained through the process of decision-making of each step which is derived in an order. The main purpose of this process is to determine the optimal (maximum or minimum) value of the objective function. We have used the backward dynamic programming method (backward DP) for the same [17]. ... ...

FIGURE 3
Schematic of the dynamic programming method. Figure 3 represents a schematic diagram of the dynamic programming method mentioned here [18]. It comprises T stages that represent the time period starting from 1, 2…, till T. Each stage exhibit 2 12 = 4096 nodes that represents the operating states, 0 and 1, for 12 smart plugs. For example, the path value f (S 1 2 ) from the previous node (S 1 1 ) at stage 1 to the node (S 1 2 ) at the next stage 2 is calculated through Equation (5) usingSCH (n, t ) = S 1 2 . The number of nodes present at each stage can be reduced by removal of the non-compliant conditions as mentioned in Equations (6)-(10). For example, as seen in Figure 3, the path value from node S to node S 1 2 does not satisfy Equation (6), thus, it was removed and was not extended to any node in the following stages. At each step, the best path for each possible starting state is recorded, and the optimal value of the objective function can be obtained using the back tracing path.

On-line management of load-shedding
The execution of DR on the event day is based on the schedule that is determined by the load forecast of the day ahead. However, the load forecast is inevitably different from the actual consumption observed on the event day; thus, the actual consumption probably exceeds the expected limit value during the DR period. Therefore, real-time control of load-shedding is vital for preventing the failure of the DR. Figure 4 denotes the state chart diagram of the real-time control of an HEMS. The system collects electric consumption data from the smart meters and the smart plugs at intervals of 1 min each. Meanwhile, if the time reached the duration of the DR, the system will execute the DR command for this period based on the DR schedule of the first minute of this period.
It is observed that the performance of the DR is reduced as the forecasted total consumption is different from the actual consumption. The algorithms can help in increasing the accuracy of load forecasting; however, zero error can never be achieved. Hence, it is important to prepare a reserve capacity to reduce the effect of the forecasting error.

2.6
Real-time control of demand response A real-time control system has been proposed to facilitate reserve shedding, as seen on the right hand side of Figure 4.

FIGURE 4 State chart diagram of DR program
Since the actual DR is dependent on the readings of the smart meter, the real-time control system checks the total consumption every 15 min. Thus, the real-time system can determine whether the quantity of load-shedding is sufficient at the 6th and the 11th minute within this period. The HEMS is capable of searching for an alternate loadshedding equipment that can compensate for insufficient loadshedding during the remaining time of DR on the basis of the accumulated shortage in the given period at the 6th or the 11th minute. This enables the system to prioritize the selected ones which are below the value of the insufficient load-shedding.
The alternate controllable loads are prioritized from small to large on the basis of the capacity of load-shedding. It is not known whether more than one load-shedding state co-exist at the same time. On the basis of over-load-shedding accumulated during the DR period, an alternate load-shedding equipment is identified and restored to be used as an alternate load-shedding equipment.

DEMONSTRATION SYSTEM
The proposed method has been set up in the laboratory to verify this system. The system is described as follows: Figure 5 represents the design of a HEMS. It comprises of 1 smart meter and 12 smart plugs. The smart meter measures the total consumption of whole loads after an interval of 1 min, whereas the smart plug measures its own load after an interval of 1 s. The computer automatically converts the data into a form which is based on an interval of 1 min. The total consumption includes 12 smart plugs as well as the non-monitored loads. Each smart plug supplies power to the uninterruptible power system (UPS) which is connected to the user's appliances. The UPS not only provides power supply to the appliance but also gives sufficient time for a user to shut down the PC, which in turn reduces the effect of DR event. A UPS can supply power for 30 min only.

Smart metering data of whole consumption
The smart meter can measure the voltage, current, and power factor, as well as the average timing demand; furthermore, it has functionality for two-way measurement of the power. The smart meter used in this study is PA3000. It is connected to HEMS via built-in RS-485 communication interfaces. The data considered in this study is the total electricity consumption (in kWh) with an accuracy of 0.5%.

Controllable loads with smart plugs
Electrical appliances can be divided into two types based on their connection to the smart plugs, namely, controllable load appliances and uncontrollable load appliances. The capacity and the average consumption of the appliances during the DR is shown in Table 1. A total of 12 loads are connected to the smart plugs, represented by loads P1-P12, and the load P13 is equivalent to the sum of the uncontrollable loads. The load devices mainly comprise computers, screens and lighting based on the environment of the laboratory. The smart plug measures the power parameters which includes voltage, current, power factor, and frequency. The smart plug which has been used in this set up is IM-1232 and the communication interface used is RS-485 over protocol Modbus RTU. Table 2 depicts the constraints experienced by the controllable appliances at the time of the load-shedding. DR enable indicates the time at which the smart plug is allowed to shed its load, as seen in Equation (8). For instance, the load P3 is not allowed to shut down whereas load P9 is permitted to shut down at any point of time. T on,min is the minimum operation time between two DR events, as Equation (10). T o f f ,max is the maximum time for power off, as Equation (9). The shorter the DR time, the lesser is its influence on a user. A lower value of T o f f ,max indicated a better performance of the DR. N DR is the maximum shedding time, as Equation (7)

EXPERIMENT AND ANALYSIS
In the experiment, three different durations of the DR were simulated. In Case 1, the DR is run for 2 h from 1:00 PM to 3:00 PM (53th to 60th time period), in Case 2, the DR is run for 1 h from 1:00 PM to 2:00 PM (53th to 56th time period) and in Case 3, the DR is run for 1 h from 2:00 PM to 3:00 PM (57th to 60th time period), respectively. It can show the probable variety of DR that is the selectable resources for the aggregator.
The load profile of each case is the same and the average of CBL is calculated to be 1750 W using Equation (3). The actual load reduction of the DR is determined by the difference between the actual power consumption and the CBL during the entire DR periods, as seen in Equation (10). The DR execution rate is defined as the average of DR performance divided by the DR target and is represented by Equation (11). If the value of the DR execution rate is lesser than that of the DR command, the incentive might get cancelled on the basis of the contract of the utility.
It may be noted that the performance of the DR is dependent on the CBL, hence, it is different from the actual loadshedding. The actual load-shedding can be determined from the consumption prior to the load-shedding.
where P actual (t ): Total consumption at period t from smart plug. ST : The staring time period of DR.

Case 1: Executing DR in 2 h
In Case 1, the DR capacity of the day ahead plan is calculated as 172 W. If the DR command exceeds 172 W, the target is not achievable even with the help of spare shedding load. Figure 6 and Table 3 show various demand curves through simulation of the DR duration. The CBL represents the demand which is predicted on the basis of the previous five working days and its average is calculated to be 1739 W. The curve depicting P SCH , determined by subtracting DR command from the CBL, has an average of 1547 W. The original demand (OD) is the real consumption which has been measured by a smart meter and its average is 1720 W. The values of OD at T53 and T54 are higher than that of the CBL and are contrary at other periods of time. The difference between the CBL and the OD is usually small. The curve "OD" becomes lower as compared to the curve "OD with shedding" if the plugs assigned to shed are turned off.
The curve "OD with shedding" is lower than the curve P SCH at each period of time. It may be concluded that the result of the DR is either equal or better than the predicted value. The average of DR is 244 W but the effective DR is only 186 W, the minimum in whole DR duration depending on DR contract. Thus, the DR ratio is found to be 108% on the basis of the DR capacity. The actual shed load by smart plugs was observed to be 225 W. This difference arises due to higher prediction of CBL in comparison to the original demand. In Table 3, the item, DR without shedding, represents the DR based on non-controllable devices. Notice that the DR failure will happen when the DR without shedding is of negative value, such as in T53, T54 and T57, if the controllable load was not available. Table 4 shows the shedding schedule of the plugs for each time period. Three plugs are shed at each time period except at T54, T56 and T59, which are connected to high consumptions devices. Most of the smart plugs have shed load more than two

Case 2: Executing DR for 1 h from T53 to T56
In Case 2, the DR capacity for the day ahead plan is 172 W. When the DR command exceeds 172 W, the simulation results are similar to the one observed in the first half of Case 1. This is due to the shedding time limitations of the remaining smart plugs.
The average of DR is 247 W but the effective DR is only 186 W. Thus, the DR ratio is also 108% on the basis of the predictive goal. The actual shedding by smart plugs is observed at 246 W. The result indicates that a consumer's flexibility is reduced from T53 to T56.

Case 3: Executing DR for 1 h from T57 to T60
In Case 3, the DR capacity of the day ahead plan is 195 W which is more than that of the second half of Case 1. Figure 7 and Table 5 shows various demand curves through simulation of the DR duration. It is observed that the curve of "OD with shedding" is closer to the curve P SCH . It means that the effect of over shedding can be reduced. The less the duration of the DR, the more is the selection of shedding devices to achieve the goal using lower DR cost.
The average of DR is 251 W but the effective DR is 217 W, higher than the value of Case 1. Thus, the DR ratio is calculated to be 111% on the basis of the predictive goal. The actual shedding by the smart plugs is calculated to be 214 W. This dif-ference arises due to higher prediction of CBL as compared to the original demand. Table 6 shows the shedding schedule of the plugs at each time period. Three plugs are shed at each time period except at T58 and T60 which are connected to heavy consumption devices, such as P1. It is observed that most of the smart plugs shed their load twice.

Estimating DR capacity
When the utility announces a DR request, the aggregators consider their users to determine the DR capacity. Each user, in turn, reports its own DR capacity [14]. The aggregator analyses the optimal DR capacity from all the generated reports and sends it to the utility to facilitate DR bidding.

FIGURE 7
The profiles in the simulation of DR command 195 W with 1 h DR from T57 to T60  The DR capacity is defined as the maximum shed load at which the smart plugs can be shed under the given constraints. The golden section search, which is an algorithm for finding the extremum (minimum or maximum) of a strictly unimodal function by successively narrowing the range of values, is used to determine the maximum DR capacity. [19] The algorithm of the golden section search (GSS) is similar to that of the half-division method, however, the difference is that the division of the segment is calculated on the basis of the golden ratio with a value of 0.5(−1 + √ 5) ≅ 0.618. The limitations of the GSS method are the unimodal problem and linear convergence rate. It is acceptable because the achievable range of DR can be easily represented and is useful to LA. Figure 8 presents the flow chart of the proposed GSS algorithm. The range of search of the feasible DR capacity is represented by [A, B], where A is set to 0 and B is set to the total consumption (W) of all smart plugs in DR time period. The objective function is unconstrained, any value among A and B is reasonable. If the unique maximum happens at X * , for first iteration, the GSS points are chosen as 1-G and G relative to [A, B]. Suppose for instance that tDR(x1) > tDR(x2). Therefore, the X * must be in the interval [A, x2]. Iterating on the same principle, the length of the interval of uncertainty is reduced by a factor of G. The iteration will break until the error lower than the acceptable error tolerance (Accu).
As shown in Equation (12), tDR(x) is the evaluation of the DR capacity of x, the output is x if DP(x) has a solution of shedding schedule of smart plugs for the DR target x, otherwise the output is 0.
x i f DP (x) has a solution 0 i f DP (x) does not have a solution (12) In Figure 9, the maximum DR observed in 2 h is less than or equal to the value determined in 1 h, which can be veri-  The assessment of DR capacities in 30 min, 1, 2, and 4 h fied using Cases 1 and 3. This arises due to the constraints that are experienced at the time of load-shedding. The difference between Cases 2 and 3 indicates that the constraints due to a user's behaviour affects the value of DR.
It is observed that the duration of the DR affects the DR capacity. The shorter the duration of the DR the easier it is to achieve the goal at a lower cost. From an aggregators point of view, if more information can be taken from a customer then a higher capacity can be arranged. The estimated DR capacities are 370, 186, 166, and 76 W for duration of the DR at 30 min, 1, 2, and 4 hours, respectively. The data obtained enables the customer to carry out a self-assessment of their DR capacity. The aggregator collects this data and analyses it to provide optimal DR insurance to the utility.

CONCLUSION
A reliable and flexible auto-DR method is proposed for HEMSs to enable low voltage users to participate in DR aggregation. When the HEMS receives a DR request from the aggregator, it estimates several feasible DR capacities for different durations. The aggregator could have more solutions that are possible for optimizing DR planning. The auto-DR method has been emulated based on the environment of the laboratory and comprises smart meters and 12 smart plugs. The smart plugs execute the function of measuring consumption as well as shedding the load. The golden search method was used to evaluate the feasible DR capacities at 30 min, and 1, 2, and 4 h. Moreover, a dynamic programming algorithm was utilised to solve the issue of the shedding schedule of each smart plug. At the time of the execution of the DR, the reserve shedding attempts to attain the targeted DR.
The results show that the feasible DR capacities are 370, 186, 166, and 76 W for the respective durations. In the cases of the durations of 30 min and 1 h, the execution ratio was found to be greater than 100%. The proposed method can help in easing the low voltage and residential consumers. In the near future, this method may be used for application in smart grids and smart homes.