Machine learning algorithm for activity-aware demand response considering energy savings and comfort requirements

: Due to the high cost of peak hour power generation and a push towards sustainability, the need for demand response (DR) is increasing. Compared to commercial-level DR, residential-level DR is more challenging. Residents are reluctant to participate, and DR controllers lack sufficient real-time activity information to balance energy savings with residents' need for comfort and convenience. To address the above challenges, we propose a sensor data-driven activity-based controller for heating, ventilation, and air conditioning devices. Using our proposed novel strategy, resident activities are recognized in real-time through a random forest machine learning approach. Integrating activity information and forecasted electricity pricing, the proposed controller can simultaneously reduce energy consumption for sustainability and maintain resident constraints for comfort based on recognized activities. Results demonstrate the superiority of the proposed approach.


Introduction
Total residential energy consumption reached 20,413 trillion Btu in 2016, which comprises approximately 10% of the US total electricity demand [1]. The US Energy Information Administration (EIA) estimates that 17% of residential electricity consumption is targeted for air conditioning and 15% is targeted for space heating [1]. Because demands for space cooling and heating contribute significantly to peak electricity demand and require associated peak generators, an extra 20% generation capacity is required to meet these needs. Less-efficient generators are typically used to meet these occasional peak demands that occur approximately 5% of each year [2]. Such peak demands lead to additional losses and overload in transmission and distribution systems. To mitigate these concerns and improve the efficiency and sustainability of the power system, demand response (DR) that can reduce or shift the load offers a promising option [3]. Both developed and developing countries have thus adopted such DRs [4][5][6][7]. There are two common DR schemes. The incentive-based DR scheme will establish an agreement between customers and utilities, which authorizes operators to directly control a portion of the load to reduce operation cost [8]. The price-based scheme will provide customers with time-varying prices and encourage them to reduce or shift their load to low-price times [9].
In the literature, DR implementations have been well reviewed at the commercial level. A preemptive demand response management (DRM) system for commercial buildings is proposed in [10] to ensure contracted capacities or demand limits are met as well as to reduce overall electricity consumption. Zhou et al. [11] explore the impact of building-level DR on electricity markets with varying competitive levels. In these settings, DR actions can reduce peak load and lower the volatility of electricity demand. Higher employment rates can also reduce the overall electricity price and volatility. In [12], Mathieu et al. assess the impact of DR on three different facilities. A tested bakery facility required around 15 to 30 min to provide load shedding, while the county building and the furniture store needed less than 15 min. These authors found that changing the temperature set-points of heating, ventilation and air conditioning (HVAC) caused more intra-shed variability than changing the light schedule. In [13], Tascikaraoglu et al. propose a bi-level optimization algorithm to maintain end-user comfort and satisfy operational target as part of a HVAC unit and energy storage system.
Compared to commercial-level DR, residential-level DR faces even more challenges. These challenges include the unwillingness of residents to respond to pricing information and the risk of leaking private activity information [14]. Gyamfi et al. [15] investigate responsiveness to time-varying electricity prices for 1800 Californian households and found that 44% of the sample were unresponsive to such incentives. Similarly, the US EIA discovers that 87% of US homes use air-conditioners, and 41% of them have programmable thermostats, but only 12% of customers are actually programming their air-conditioner. To understand the potential impact of resource control strategies, Hamidi et al. [16] investigate the impact of tariffs on domestic user responsiveness, while Farahani et al. [17] studies the impact of incentive offers on customer responsiveness and peak load reduction. Hansen et al. [18] propose a partially observable Markov decision process approach for residential home energy management to minimize the household electricity bill. Ahmed et al. [19] utilize a hybrid lightning search based ANN scheduling algorithm to reduce the peak-hour energy consumption during the DR event. As noted in these studies, a high DR participation rate can contribute to system reliability by reducing the probability of curtailment. In addition to individual bill savings, all customers benefit from wholesale market price decreases through demand shifting [20]. Therefore, it is important to improve residents' participation rates to maximize these benefits.
For residential-level DR, addressing user need for bill savings and comfort must be combined with an easy-use feature to improve participation rates. In the literature, Erdinc et al. [21] propose an HVAC control strategy to minimize violation of average comfort constraints. Alternatively, Babar et al. [22] utilize a demand reduction-bidding incentive strategy to maximize user convenience. However, due to the lack of real-time users' activity information, resident comfort requirements cannot be fully addressed. The temperature setpoint of existing HVAC systems is set based on a rough estimation of resident routines (e.g. one setpoint for daytime and another for nighttime). In actuality, residents may have different temperature preferences within these time periods. For example, a customer may wish their home temperature to be lower when they take a nap, higher when they work, and use minimum resources when they leave the home. To address their needs, HVAC controllers need to obtain real-time activity data and be activity-aware when they adjust temperature settings. The emerging machine learning-based activity recognition (AR) methods have been gaining both popularity and usability over the past decade. In the market, there are some available commercial smart controllers such as Google Nest. Nest can learn from the previous HVAC usage information and user prompts to provide energy savings and comfort. Nest does not interact with the grid and, hence, does not provide any demand response. Ecobee can adjust the on/off mode based on motion and proximity sensors [23] and can also be used to participate in demand response, but only within manual mode and at set times in automated mode. These controllers do not recognize the resident's specific activities and therefore do not correspondingly adjust the temperature settings to maximize energy savings and comfort while simultaneously participating in demand response and helping the grid as necessary. In the literature, researchers have started to investigate more methods for recognizing activities from many types of sensor sources, including video and audio data [24,25], wearable data [26,27], and ambient data [28,29]. A large range of machine learning methods have been employed for activity recognition, including random forests, support vector machines (SVM), hidden Markov models (HMMs), and deep networks [30]. Foerster et. al. [31] utilize k-NN classifier based template matching techniques to detect posture and motion. Bao et. al. [32] use decision tree classifiers to recognize everyday activities using user-annotated acceleration data. Ravi. et al. [33] combine metaclassifiers with plurality voting to do activity recognition and find energy is the least significant extracted feature. Kasteren et. al. [34] use both static Bayesian model and dynamic Bayesian network to do activity recognition. Lester et. al. [35] utilize boosted decision stumps to select the right features and feed them into HMMs to recognize different activities. Singla et. al. [36] use HMMs to recognize independent and joint activities among multiple residents in smart environments. Aminikhanghahi et. al. [37] uses a random forest collection of decision trees to provide activity labels from ambient sensor data in real-time, as the activities occur, which exhibits strong performance in actual smart homes, even with multiple residents.
From the above AR review, we postulate that AR can provide necessary activity information to DRs. The goal of this paper is to investigate how to utilize activity information in residential-level DR controllers to provide maximum comfort and convenience with a minimum possible cost. In this work, a data-driven residentiallevel AR-based HVAC controller is proposed. This paper offers several unique contributions: • A machine-learning driven automated HVAC controller has been proposed for demand response considering comfort zone and energy savings. The proposed controller can map recognized activities to different comfort zones in autonomous way based on resident preference activity mapping rules. • Also, RTP has been integrated with AR-driven DR. RTP not only provides residents with another control dimension for their HVAC controller, but it also provides the system operators with a way to interact with each HVAC controller. Resident preference activity mapping rules give users the freedom to adjust all settings to gain either more comfort or more energy savings. • Combining the robust look-ahead HVAC controller, the random forest-driven activity recognition engine, and the comfort zone controller, the proposed HVAC control will provide an automated control scheme that meets both comfort and energy savings targets with demand response. In our understanding, this is a first paper to integrate machine learning based activity recognition (AR) with the demand response while maximizing comfort and energy savings. • A detailed analysis and simulation results for the HVAC controller demand response performance has also been presented.

Methodology
The proposed AR-based HVAC controller has four components: an AR engine, a comfort zone controller, a temperature predictor, and an HVAC controller, as shown in Fig. 1. The AR engine takes activity sensor data as input and generates an activity label at each timestamp. The comfort zone controller processes residents' preferences and the RTP to generate comfort zone temperature constraints for each time interval. The temperature predictor accepts historical outdoor temperature data as input and generates forecasted outside temperatures. The HVAC controller takes comfort settings and predicted outdoor temperature as input and generates the control signals.

AR engine
The goal of the AR engine is to map sensor events to a label that indicates the corresponding activity that the individual, in this case the resident of a controlled home, is performing. Let A = a 1 , a 2 , …, a K represent a set of K activities where a j corresponds to the jth activity class. Our training data consists of a sequence of raw sensor events Λ = λ 1 , λ 2 , …, λ N , where event λ i corresponds to a sensor reading or sensor value with an associated timestamp t i . Given features x ∈ ℜ d extracted from the raw sensor data as input, an activity recognizer uses a supervised learning algorithm to construct a mapping from Λ to the corresponding activity label, A. The AR activity recognition algorithm learns a random forest model to perform real-time activity labeling on streaming data [38]. This supervised learning algorithm constructs an ensemble of decision tree structures from training data in which each tree node queries a particular feature value. The feature is selected for querying that provides the greatest information gain, or weighted reduction in entropy. In detail, AR extracts features from a sliding window containing w consecutive sensor events, λ i − w , …, λ i , and learns a function that maps the feature vector Λ i onto an activity label indicating the activity that is performed at the time of the last event in the window, or time t i . By generating a label for the most recent sensor event in the window, AR can provide real-time activity recognition as sensor events occur within a home environment.
The features that are used for the activity models in this paper include time of day (in hours, minutes, and seconds past midnight) for the last sensor event in the sliding window, the window duration, the counts of sensor events in the window for each sensor, the elapsed time since the previous event for each sensor, the location of the most recent sensor event, the dominant sensor (sensor generating the most events) in the previous two windows, the activity level in the window, and the entropy of the events in the window. While the size of the window can be adjusted based on the most likely current activities, for our experiments, we set the window size w to be 30. For houses with multiple residents, sensors may have conflicting influences on the AR engine. This situation is dealt with using sensor event based windowing. Here, the AR engine will divide the sequence into windows with equal numbers of sensor events to capture the relation between the sensor events in a window as well as between multiple windows.
For each of our activity models, training data includes one month of sensor data that is labeled with the corresponding activity categories by human annotators. Based on interviews with the residents and photographs of the apartments, the annotators assign likely activity labels to each sensor event, which represents ground truth labels. If sensor events do not belong to the defined core activity classes, they will be labeled as 'Other Activity'. To maximize the consistency of those ground truth labels, several annotators will label sensor data together and resolve the disagreement by discussion. The annotators demonstrate an overall inter-annotator agreement of 85%. The AR accuracy is evaluated based on the 3-fold cross-validation testing method. The 3-fold cross-validation accuracy for each of 30 homes yielded an average accuracy of 97.5% across all homes [37]. Cross-validation accuracy for the combined 30 homes is 98.5%, and leave-onehome-out validation results in 88.0% accuracy. The leave-onehome-out accuracy reflects the performance that could be expected when training a model on a large set of homes and using the model to label activities in a new home.

Comfort zone controller
The comfort zone controller provides temperature set-points for each comfort zone based on the resident's preference and RTP and feeds them to the HVAC controller. The controller will map all recognized activities to either a high or low-level comfort zone. Users have the freedom to adjust all settings to gain either more comfort or more energy savings. Assuming outdoor temperature is lower than the comfortable indoor temperature, the high-level comfort one will require more electricity than the low-level comfort zone.
The default mapping rules are summarized in Fig. 2, which will be utilized for our simulation. The sleep and bathe activities are mapped to a high-level comfort zone regardless of RTP. The cook and relax activities will be mapped to a high-level comfort zone if the RTP is lower than a certain threshold. Otherwise, they will be mapped to the low-level comfort zone. All other activities will be mapped to a low-level comfort zone. For our experiments, we specify the high-level comfort zone temperature range as 74°F − 76°F and the low-level comfort zone temperature range as 73°F − 77°F. Thus, the controller maps activity set A to comfort zone set ΔT = [74 76], [73 77] based on resident-supplied mappings from each activity to a corresponding comfort zone.

Temperature predictor
The temperature predictor provides the forecast outside temperature for the DR controller, which will be further utilized by the look-ahead HVAC controller. An autoregressive integrated moving average (ARIMA) model is utilized inside the temperature predictor due to its stable performance and computational efficiency [39,40].
ARIMA was initially introduced by Box and Jenkins for timeseries forecasting and exhibits strong performance at capturing diurnal cycle characteristics [41]. Since temperature data has clear seasonal features, our algorithm employs a seasonal ARIMA (p, d, q)(P, D, Q) s using the ARIMA toolbox from Matlab. Here, d indicates the degree of nonseasonal integration of time series; p, q indicates the degree of nonseasonal autoregressive and moving average operator; D indicates the degree of seasonal integration of time series. P, D indicates the degree of seasonal autoregressive and moving average operator, and s represents a seasonality index.
This seasonal model is built through three standard steps: model identification, model estimation, and model diagnostic check. Tuning parameters require expert knowledge of the system or can be found by trial and error. The detailed tuning process is explained in [42]. After the tuning process, ARIMA (1, 0, 1)(1, 0, 0) 720 is selected, and training data is captured for 30 previous days. Since the temperature data interval is 2 min, s is equal to 60 × 24 2 = 720.

Look-Ahead H × VAC controller
The look-ahead HVAC controller will generate control signals based on the predicted outdoor temperature and the comfort zone setting. The robust controller is derived from a deterministic lookahead controller, and the HVAC load is modeled with an equivalent thermal parameter (ETP) model.

ETP model:
ETP models have exhibited a good balance of model accuracy and computing efficiency. Therefore, our strategy utilizes ETP to model the HVAC load. As shown in Fig. 3, the model is represented with an equivalent circuit. The inside air temperature T a and mass temperature T m are coupled in the (1) and (2) In these equations, ξ a and ξ m represent the thermal mass of air and building materials in (Btu/°F), η a represents the conductance between inner mass and air in (Btu/h/°F), η m represents the conductance for the apartment envelope in (Btu/h/°F), and Q represents the heat flux from the HVAC system in (Btu/h). The thermal parameters vary for each house [44]. However, these parameters usually vary within a certain range. The typical ranges and corresponding chosen values are summarized in Table 1 [43,45]. Given the value of Q for each time step, the air temperature can be calculated. Generally, the goal of the HVAC controller is to minimize the total Q over a certain time period.

Deterministic look-ahead controller:
The deterministic look-ahead controller aims to maintain the temperature within the pre-specified activity comfort zone while simultaneously minimizing electricity consumption. The objective function and constraints for the deterministic controller are modeled in (3) and (4): Objective: Constraints: At each time step t, the objective function is designed to minimize electricity consumption during the period from t + 1 to t + N and ensure that the forecasted indoor temperature T^a , t + h satisfies the comfort zone limits T t + h L , T t + h H for every horizon h. In the objective function, N represents the look ahead steps, K represents the Btu conversion coefficient to convert Btu/sec to kW which equals to 1.0551. Δt represents the time step for the controller, which is set to 2 min for our experiments. s t + h represents the operation mode of HVAC at t + h. β(T^o , t + h ) and α(T^o , t + h ) represent the HVAC Btu rating and cooling coefficients when the outdoor temperature is T^o , t + h , which are defined as shown in (5) and (6) [46]: Here, β 0 = 11.6667Btu/sec is the nominal Btu rating of the HVAC system at 95°F and α 0 = 3.8 is the nominal cooling coefficient of performance for the HVAC system at 95°F. β 1 = 1.4892, β 2 = 0.0052, α 1 = − 0.01364, α 2 = 0.01067 are the constants in the above formula.

Robust look-ahead controller:
The robust look-ahead controller aims to minimize the performance variability of HVAC due to the variations in the forecasted outdoor temperature. μ f is the objective function derived from the deterministic look-ahead controller. σ f is used as a measure of the variation of the objective function to achieve objective robustness [47]. They are derived from a first-order Taylor expansion. The objective function and constraints are given in 7 through 12: Objective: Here, ω represents the relative weight for μ f and σ f . Using the weighted sum of them is one approach to deal the conflict in the robust optimization objective function. μ f and σ f are formulated as follows: Constraints: In (10), k r represents the weight constant. Parameter σ(T^a , t + h ) represents the standard deviation vector of the derived future indoor air temperature defined in (11): To calculate the derivative of above formulation, first and second terms are neglected and it can be simplified to convexity the constraints as follows: The constraint is formulated through a feasible region reduction, which reduces the feasible region by increasing the weight constant k r . The robust algorithm thus represents a generalization of the deterministic algorithm. When ω = 1 and k r = 0, the above robust algorithm reverts back to the deterministic algorithm.

Simulation environment and simulation results for energy savings
In this paper, real-world sensor data obtained from four apartments in Seattle were utilized. Each apartment is occupied by 1-2 residents and includes at least one bedroom, a kitchen, a dining area, and one bathroom [48]. For HVAC control, ambient sensor data is a reasonable choice because ambient sensors are unobtrusive, can sense human behavior for all settings and lighting conditions, and do not require any changes to resident daily routine. Activity data are extracted from ambient sensors installed in those apartments including passive infrared motion sensors and door sensors. Approximately three sensors are installed in each room, while door sensors are installed on each exterior door as well as cabinets that contain critical objects such as medicine. The frequency distribution of recognized activities for each apartment is presented in Table 2. AR has been used to recognize the most basic and instrumental activities of daily living. The types of activities that are difficult to differentiate using PIR motion sensors and magnetic door sensors are those that involve fine motor movement. For example, it is difficult to distinguish getting dressed in the morning from brushing hair, if both are done at the same time and place. However, it is unlikely that HVAC levels would need to be different for activities that are performed at similar times and places. For our experiments, weather data is collected from the National Renewable Energy Laboratory. To examine the impact of the proposed controller, the apartment thermal parameters and HVAC parameters are assumed to be the same for all four apartments using the chosen values summarized in Table 1. The RTP is extracted from a 12-Bus distribution system in [49] with a threshold of 5.5 cents/kWh.

Energy savings summary
During the testing day, the proposed controller allows the temperature to vary within the specified comfort zone. During the times that the residents prefer a warmer environment, the inside temperature varies within the high-level comfort zone [74°F . .76°F]. To illustrate the impact of the controller, Fig. 4 shows the inside temperature for apartment HH101 shaped by the look-ahead controller and the on/off controller. An on/off controller is utilized as a benchmark to validate the performance of the proposed controller. For a cooling mode, it will turn on the unit when the inside air temperature is higher than the set point and turn off the unit when the inside air temperature is lower than the setpoint. In this graph, the black line indicates the comfort-zone variation. With the help of a comfort-zone dynamic adjustment that is driven by the AR engine, the controller has more room to save energy and satisfy user temperature requirements at the same time.
Noting that the comfort zone is automatically adjusted by real-time AR information, residents only need to preset the comfort choice for each activity once.
The performance of the deterministic and robust controllers is compared to the performance of a on/off controller, as summarized in Table 3. Since the thermal parameters are assumed to be the same for all four apartments, the on/off controller consistently consumes 22.49 kWh energy during the testing day. Both lookahead controllers look one step ahead. The parameter ω equals 0.85 for the robust controller. The detailed tuning process will be explained in the following section. Seen from Table 3, the deterministic controller offers a potential savings of 5.14% energy and the robust controller offers a potential savings of 4.23% energy.
Due to the impact of the high-frequency control signal rate and our simulated stationary outdoor temperature, the deterministic look-ahead controller potentially yields greater energy savings.

Impact of tuning parameter
Mathematically, the proposed look-ahead controller could look ahead to an infinite horizon. However, due to the accumulated forecasting error and increasing computation time, looking ahead too many steps will harm the performance. As shown in Fig. 5, the best performance for all four apartments is achieved at one step ahead for the robust controller compared to the on/off controller. This feature is especially favored by the AR engine. Once an activity is recognized, the resident stays on this activity for at least two minutes (in over 99% of the cases) and therefore need a certain corresponding comfort level for at least two minutes. Extending the look-ahead steps will harm the performance of the AR engine.
In this section, we present the impact of the mean (ω) on the objective function for HVAC energy consumption. A higher value of ω means the objective function focuses more on energy savings, and a lower value of ω means the objective function focuses more on forecasting error. As shown in Fig. 6, the highest actual energy savings is achieved when ω = 1.

Impact of comfort zone temperature setting
The impact of the comfort zone temperature setting on HVAC energy consumption is presented in Fig. 7. If the temperature range  is 1 o F wider, the energy savings are almost doubled. Since the residents are seniors, a conservative ΔT is utilized. If residents prefer energy savings, they will earn 5% more energy savings when the temperature range is 1 o F wider. In extreme cases, the proposed DR can also provide the system with a predictable peak shave support without entirely turning off the HVAC load.

Impact of RTP integration
The comfort zone controller shapes the comfort zones utilizing AR input, RTP and user preference. The impacts of RTP are presented as follows. AR labels determine the initial comfort zone for each time slot. The RTP will further adjust the comfort zone settings. Fig. 8 shows the simulated RTP and RTP threshold during the testing day. If the RTP is higher than the threshold, cooking and relaxing will be mapped to a low-level comfort zone (Default mapping rule). Basically, a lower proportion of high comfort zone will lead to more energy savings. The impact of RTP on energy savings and high-level comfort zone portion is presented in Table 4. RTP integration will reduce the proportion of the high-level comfort zone and allow more room for energy savings. RTP integration not only provides residents with another control dimension for their HVAC controller, but it also provides the system operators with a way to interact with each HVAC controller.

Simulation results for comfort level
Aside from energy savings, comfort level satisfaction is also important. In the following, components that have an impact on comfort level satisfaction are analyzed.

Impact of AR engine
AR accuracy is evaluated by comparing the activity labels recognized by the AR engine and the manual annotation labels which can be assumed to be the ground truth activity labels. In Table 5, the AR accuracy for apartment HH101 is presented for three months of data. As shown in the table, the AR accuracy is over 99.5%. In fact, the accuracy of AR is fairly stable for a new home with a distinct setting [50]. As a result, we can conclude the proposed controller will meet resident comfort needs 99.5% of the time in this home.
For large-scale applications, users have the freedom to choose to obtain labeled activity data for their home or utilize a pre-trained generic model. If the user prefers a high AR accuracy rate, time and effort are required to collect training data for model training for the best possible performance. If the user prefers to use the proposed controller immediately, the pre-trained model can be used to get reasonable performance regarding comfort level and energy savings. Moreover, the user's comfort zone settings will also impact the final conflict rate for comfort level. This is because all activities will eventually be mapped to two zones. The conflict rate will be generally less than AR errors

Impact of temperature predictor
The impact of temperature forecasting error of is analyzed in this section. The forecasted and measured outdoor temperature are plotted in Fig. 9. For this test day, the root-mean-square error (RMSE) is 1.96%.
To evaluate the impact of the forecasting error on the indoor temperature, the difference between derived future inside temperature and actual inside temperature is plotted in Fig. 10. The temperature difference is very low with a maximum difference of 0.06°F. This phenomenon results from a relatively high control signal frequency. The control signal is at a rate of 1 signal/2 min so that the forecasting error can be well compromised. Thus it is better to use our proposed high rate temperature forecasting engine instead of temperature forecasting from a public weather institute which usually updates the temperature forecast every few hours.

Conclusions
In this paper, a novel data-driven AR-based look-ahead controller is proposed to address energy savings for sustainability and comfort requirements simultaneously. With the help of the machine learning method, the activity recognized accuracy is over 99.5%, which can address resident comfort needs in real-time. After evaluating the proposed method on four selected apartments, the energy consumption of HVAC from the proposed controller yields 5.14% savings over the on/off controller. Our experimental results provide evidence that adjusting the comfort zone controller based on preferred comfort and utilizing sensor-driven activityrecognition can lead to more energy savings by modifying the  mapping rules, temperature setting ranges, and the RTP threshold. Through simulation and parameter tuning, the deterministic approach out-performs the robust approach due to the impact of a high-resolution control signal rate and the relative stationary feature of outdoor temperature. The accuracy impact of the AR engine and temperature predictor on the comfort level satisfaction are also evaluated. The impact of the proposed temperature predictor on actual indoor temperature is less dramatic. With 1.96% RMSE outdoor temperature forecasting error, the maximum indoor temperature difference is only 0.06°F. In the future, this work will be extended for different possible activity-driven demand response mechanisms and additional machine learning algorithms for AR.  hygiene  7466  0  0  0  0  1  2  5  0  0  0  0  leave  1  827  0  20  0  0  0  0  0  0  0  0  enter  0  0  636  0  0  0  0  0  0  0  0  0  relax  3  16  14  18,619  83  1  17  0  8  0  0  1  eat  0  0  1  18  2307  0  3  0  3  0  0  0  sleep  1  0  0  0  0  2451  0  0  0  0  0  0