Applications of thermostatically controlled loads for demand response with the proliferation of variable renewable energy

Abstract

More flexibility is desirable with the proliferation of variable renewable resources for balancing supply and demand in power systems.Thermostatically controlled loads (TCLs) attract tremendous attentions because of their specific thermal inertia capability in demand response (DR) programs. To effectively manage numerous and distributed TCLs, intermediate coordinators, e.g., aggregators, as a bridge between end users and dispatch operators are required to model and control TCLs for serving the grid. Specifically, intermediate coordinators get the access to fundamental models and response modes of TCLs, make control strategies, and distribute control signals to TCLs according the requirements of dispatch operators. On the other hand, intermediate coordinators also provide dispatch models that characterize the external characteristics of TCLs to dispatch operators for scheduling different resources. In this paper, the bottom-up key technologies of TCLs in DR programs based on the current research have been reviewed and compared, including fundamental models, response modes, control strategies, dispatch models and dispatch strategies of TCLs, as well as challenges and opportunities in future work.

Graphical abstract

Keywords

thermostatically controlled load / demand response / renewable energy / power system operation

1 Introduction

The proliferation of renewable energy enhances the sustainability of power systems, but the inherent variability also poses great challenges to the planning and operation of large power grids [1]. The corresponding electric power deficiencies can be compensated by fast ramping generators and energy storage devices [2]. However, frequent ramp up/down power adjustments can increase the operation and the maintenance cost of generators. Moreover, storage devices are regarded as costly alternatives. Demand response (DR) can address this problem owing to its attractive and versatile capability for balancing the supply-demand, improving energy efficiency, and enhancing system resilience. The flexible load can be adjusted rather quickly without imposing any negative impacts on participating end users by providing financial opportunities to compensate power deficiencies caused by variable renewable energy.

Thermostatically controlled loads (TCLs) (i.e., air conditioners, water heaters, and refrigerators) represent an example of flexible loads, the ratio of which to the total power consumption in developed countries is up to 30%–40% [3,4]. Providing tremendous potentials in adjustable power consumption, TCLs have attracted major interests in DR opportunities. It has highlighted the advantages of TCLs in responding to uncertainties in power systems. In general, there are two categories for DR programs, i.e., price-based control and direct load control (DLC). This paper is only focused on DLC of TCLs.

Due to the small capacity and large quantity of TCLs, individual TCLs cannot have a significant impact on power systems, which makes it necessary to manipulate TCLs in a population. Accordingly, load aggregators which serve as intermediate coordinators will manage TCL operating behaviors and determine strategies to effectively control TCL populations based on the requirements of dispatch operators. On the other hand, load aggregators should be responsible for characterizing the performance and providing the aggregated TCL characteristics to dispatch operators. Dispatch operators will schedule aggregated TCL operations with other resources, e.g., renewable energy [5] for satisfying operational objectives.

This paper is a state-of-the-art review of TCLs in DR, including fundamental models and response modes of individual TCLs, control strategies, dispatch models, and dispatch strategies of TCLs. Specifically, fundamental models describe the basic characteristics of TCLs, which bridges the electricity and the surroundings constrained by the requirements of customers. It is widely known that TCLs can be regulated in different ways, including on/off switches, temperature set-point adjustment, and the combination way which are characterized by response modes. TCLs have various performances in different response modes, which determine the types of ancillary services provided by TCLs. As described above, load aggregators as intermediate coordinators not only provide aggregated characteristics of TCL populations at a higher level but also make control strategies for TCLs in terms of dispatch signals. In this paper, dispatch models are utilized to describe the collective performances of TCLs.

2 Fundamental models of TCLs

The physical model of TCL in DR programs includes the energy conversion process which converts electric power to cooling/heating, and the energy exchange process which corresponds the internal TCL temperature with cooling/heating power. These two processes enable TCLs to adjust its power consumption while keeping the internal temperature within a certain range as illustrated in Fig. 1.

2.1 Energy conversion

For residential TCLs, the major power is consumed for compressor operation. In literature, the highlights are TCLs with compressors of fixed speed. In other words, the ratio of electric power to cooling/heating power is fixed [6]. However, TCL compressors are evolving from fixed speed to variable speed for higher efficiency. Accordingly, the ratio of electric power to cooling/heating power is not fixed, but depends on their compressor frequency [7,8]. For large TCLs, e.g., the commercial air conditioning system, power consumption refers to several components, including fan, chiller, etc [9], which makes the energy conversion more complicated.

2.2 Energy exchange

There are mainly two methods to model the energy exchange between the internal surroundings of TCLs and the external surroundings, as summarized below.

1) The equivalent electric circuit method [10] utilizes common electrical components, e.g., electric resistance, capacitance, and power source, to model the external and internal surrounding and cooling/heating power of TCLs, as well as the heat transfer process [10–12]. A third-order thermodynamic air conditioning models is proposed in Ref. [12] considering the temperature differences among internal air, and external and internal walls of dwellings. Although higher-order thermodynamic models provide a better accuracy, complicated structures with heavy computational burden limit the application. Therefore, the simplified second-order [13] and first-order [14] thermodynamic models are usually employed for TCL aggregation and control.

2) The cooling load calculation method [11] derives the thermodynamic model of TCLs according to the energy conservation law, i.e., the energy increment of objects equals the difference between the cooling/heating power and the energy obtained from surrounding environment. Many object parameters, e.g., size and material, need to be measured for the calculations of energy increment.

The comparisons of different methods are summarized as follows [11]. The second and higher order equivalent electric circuit method has a high modeling accuracy but it has complicated structures and computations. The structure of the first-order equivalent electric circuit method is simple. However, it cannot accurately describe the actual operation behaviors of TCLs. Moreover, parameter identification is required for the models captured by the equivalent electric circuit method. Although the cooling load calculation method does not need parameter identification, complicated cooling load calculation is required whose accuracy is low.

The energy conversion and exchange models characterize the operating behaviors of TCLs, which provide the basis for TCL aggregation, control, and scheduling. The physical parameters in TCL fundamental models are critical to modeling TCLs in DR programs. However, parameter heterogeneity, including device types, object structures, external surrounding, and customers preferences poses a great challenge. Although physical parameters can be calculated in terms of individual measurements of each TCL, it may result in privacy disclosure of end users as well as huge cost. Thus, the evaluation of physical parameters of TCL populations while protecting information of customers and reducing measurement investment remains a pivotal challenge.

3 Response modes of TCLs

Response modes are defined as the action of TCLs upon receiving control signals. Based on the response modes, aggregators will determine control strategies and generate control signals to end users according to system operating requirements, which will be introduced in Section 4.

Response modes are closely related to the types of the TCL compressor. In literature, the highlight of TCLs is fixed-speed, which is regulated by changing the switch status [15] and the temperature preferences [3]. As the power consumption of fixed-speed TCLs is restricted to the constant frequency of compressors, the TCLs only have two operation status, including the rated and zero power output. This results in periodic changes of internal temperature within a certain range [T_min, T_max] [16]. If the internal temperature is out of the range [T_min, T_max], TCLs will work at rated power or turn off to either reduce or increase the internal temperature, respectively. Based on specific characteristics, TCLs have different operating characteristics on response speed and duration at different control signals.

It is shown that changing switch status can instantaneously modify TCL consumptions for adjusting power output, but the adjustment cannot be kept for a long time due to the temperature preference constraints of customers [17]. Changing switches are usually employed to quickly modify the power consumption of TCLs, such as critical peak shaving [18] and regulation service [19]. On the contrary, the TCL power consumption can be regulated for a longer period by modifying the thermostat set-point, but with a delayed effect. The adjustments to temperature set-points in TCL energy management are applied to minimize energy costs [6] and provide load following service [20]. Moreover, on/off switches can be combined with temperature set-point adjustment to provide fast response speed, large capacity, and long duration in different applications [17,18], e.g. providing power pulses [16]. The comparisons of switches, temperature set-point adjustment, and their combination are summarized in Table 1 [11].

Compared to fixed-frequency TCLs, inverter TCLs can be controlled by temperature set-point change, as well as continuously manipulated by frequency adjustment [7,21]. Therefore, inverter TCLs can be regulated with more flexibility in response time, capacity, and duration, which are generally applied in minute-based DR programs [8,22].

4 Control strategies of TCLs

There are two control categories for TCLs, centralized control and decentralized/distributed control.

4.1 Centralized control

Aggregated control models for homogeneous TCLs: When residential TCLs are regulated in a centralized way, a centralized controller is required to characterize the TCL aggregated characteristics and generate control signals for each TCL. To provide guidance for the centralized controller, it is critical to build aggregated control models that characterize the distribution of TCL internal temperatures and switch states. Initially, TCL dynamic behaviors are characterized by Fokker-Planck equations [23], which is solved in a non-closed-form analytic way with heavy complexity. To reduce the complexity of aggregated TCL models, some simplifications are developed by using discrete temperature bins and time settings [24].

The common aggregated control models of TCLs include the state queuing model [17], the transport model [25], the Markov chain [26], and the state space/bin transition model [27]. As shown in Fig. 2, different state bins are employed to describe the collective behaviors of TCLs according to their internal temperature and switch states. It is shown that more states with a shorter discretization time slot [25], state bin with non-uniform temperature ranges [28], two-dimensional state-bin model based on second-order thermodynamic model of TCLs [29] are helpful to improve the modeling accuracy.

Aggregated control models for heterogeneous TCLs: TCL parameters could also be heterogeneous as a result of different object structures, comfort requirements of end-users, device types, and geographic locations. Most aggregated control models are initially developed for homogeneous TCLs and then modified to heterogeneous ones [28,30]. The heterogeneous TCLs can be classified into several clusters using k-means algorithms [28]. Within each cluster, TCLs are regarded as homogeneous, which can be aggregated using aggregation techniques, e.g., state space/bin transition models. Then the sum of different aggregated models of homogeneous TCLs is seen as the model of heterogeneous TCLs.

There are other methods for heterogeneous TCLs. An improved aggregated model with the lockout time is developed in Ref. [31] to extract the collective dynamics of heterogeneous TCLs by analytical derivations. A similar modeling methodology is proposed in Ref. [32] to describe the behaviors of aggregated TCLs.

Control methods: Control signals are generated by the centralized controller to regulate TCLs according to the reference signals from dispatch requirement. Priority stack [28,33] and probability control [27] are often utilized to determine the control signals. Besides, some other methods are also applied to control TCLs [30,31,34–38]. For example, the model predictive control with on/off switching is developed [36] for the sequential temperature set-point adjustment for power adjustment. A hierarchical centralized control algorithm is presented in Ref. [30] to provide load following without affecting the comfort of end-users.

One major disadvantage of these model-based control approaches is that aggregated models of TCLs are required to forecast and optimize the behavior of TCLs. However, it is difficult to obtain the accurate models of aggregated TCLs as a result of parameter heterogeneity and extremely large number of TCLs [13]. Therefore, model-free control methodologies have been developed recently to regulate TCLs without aggregated models. Advanced solution algorithms are developed to make sequential decisions according to TCL dynamics. The queueing DLC method is utilized in Ref. [39] to continuously select proper TCLs for regulation services based on the power and energy limits of TCLs. A three-step approach is proposed in Ref. [13] to regulate both steady-state and transient thermal dynamics of TCLs by identifying tracer devices. The reinforcement learning is studied in Ref. [40] to improve the scalability of TCL centralized control.

4.2 Decentralized/distributed control

The decentralized/distributed control enables end-users to optimize their own behaviors for providing services to the grid, based on local information or the exchanged information from other end-users. For example, the stochastic switching of refrigerators for frequency control is developed in Ref. [41], considering the lockout constraints of devices and the incremental power consumption. A stochastic control approach is developed in Ref. [24] to accurately model the collective power for TCL control. To eliminate the two-way communication, a decentralized approach similar to the droop control of generators is suggested in Ref. [42] for frequency regulation. A decentralized cooperative algorithm is proposed in Ref. [43] to make the plan of energy usage based on the hybrid system model of TCLs and electric vehicles. Recently, transactive control strategy [19] has attracted much attention, in which self-interested entities are encouraged using market mechanisms to serve the grid. The advantage is that the price and power consumption quantity are the only required information, which well protect privacy of entities. For example, a transactive control method is developed in Ref. [19] to regulate commercial building for peak load shaving.

Comparing two control methods, the centralized control has a good controllability but with a high communication burden and time delay. In contrast, the decentralized control can respond faster but with less predictability. The characteristics of both methods have been compared in the literature. A hierarchical control scheme is developed in Ref. [44], in which the TCLs in a community are centrally controlled by an aggregator and different aggregators are controlled by decentralized controllers.

5 Modeling and control issues of TCLs

5.1 Cold load pickup

Mechanism: When controlled by on-off switches or temperature set-point adjustment, the TCL operating state heterogeneity is destroyed and becomes homogeneous. The aggregated power may experience oscillation [18] and power system operations appear more variability. For example, the evolution of TCL collective operating states in the operation of raising temperature set-point is depicted in Fig. 3. In Fig. 3(a), TCLs are working at temperature set-points and their operating states are uniformly distributed in terms of temperature, resulting in the steady power. When the temperature set-point increases by DT (i.e., the internal temperature of TCLs varies between [T_min + DT, T_max + DT]), as demonstrated in Fig. 3(b), the TCLs with the switch state of on and the internal temperature lower than T_min + DT will be immediately turned off while the other TCLs with the switch state of on will be turned off once their internal temperature reaches T_min + DT. For TCLs with the switch state of off, they will remain off until their internal temperature reaches T_max + DT. Gradually, the number of TCLs in Box I keeps increasing and there are no TCLs in Box II, as exhibited in Fig. 3(c). In other words, the aggregated power consumption of TCLs decreases to cause a power valley. With more TCLs being turned off, as displayed in Fig. 3(d), the collective TCL power continues decreasing to achieve the desired load shedding. However, moving forward, more TCLs will be turned on while less TCLs are turned off, as described in Fig. 3(e). At this stage, the collective power of TCLs presents a peak which is larger than the original one.

In summary, as the temperature set-point is adjusted, TCL collective power will be oscillatory with the evolution of TCL operating states, as a result of the loss of operating state heterogeneity. This is the cold load pickup phenomenon, which is usually characterized by the peak duration and magnitude [45].

Solutions: The cold load pickup phenomenon is influenced by many variables. A sensitivity analysis to identify the important factors that have great impacts on the aggregated dynamics of TCLs is proposed in Ref. [46], which is helpful to facilitate the control design for restraining the cold load pickup. A methodology is provided in Ref. [47] to estimate the peak magnitude of the cold load pickup, considering the outdoor temperature, the TCL concentration level and the outage duration. Moreover, the cold load pickup of TCLs can cause undesired power fluctuations. To address this issue, it is shown that the operating state heterogeneity of TCLs should be restored as soon as possible [18,22]. For example, several safety protocols are proposed in Ref. [18] to restrain the undesired power oscillations by imbedding instructions and memory to the TCL controllers, which relieve the communication burden and protect the privacy of end-users.

5.2 Impact factors of TCL control strategies

Among different impact factors on the operating behaviors of TCLs, two main factors, i.e., the lockout phenomenon and the outdoor temperature have great impacts on the control strategies.

Lockout phenomenon: The durability of the TCL compressor can be destroyed by too frequent switches [3,27]. There are two main methods to address this problem. The first one is to utilize a minimum time constraint when TCLs are off [31]. In other words, TCL compressors are kept off for a certain period, during which TCLs will be locked and not respond to any external signals. When the predefined time is passed, TCLs will be unlocked and begin to react in terms of the external control signals. The second one is the priority-stack method [33,39], in which TCLs are ranked and given different priorities to change their switch states. When selecting TCLs to be turned on, those TCLs whose switch status is off will be sorted in the descending order according to their internal temperatures. Those TCLs whose internal temperatures are closer to T_max will be turned on with higher priorities. Similarity, when selecting TCLs to be turned off, the TCLs whose internal temperatures are closer to T_min will be turned off with higher priorities. In summary, TCLs close to the end of their operating cycles will be given higher priorities to change their switch states for reducing the forced switch times.

Outdoor temperature: The power consumption of TCLs is sensitive to external temperature. It is shown that a lower external temperature will cause more power consumption due to the longer and more frequent operation of heat pumps to maintain the internal temperature at the set-point [48]. Similarly, as demonstrated in Ref. [8], a higher outdoor temperature will increase the discharge power limit, which can reduce the charge power limit of thermal batteries for inverter air conditioners. Therefore, the time-varying external temperature is a critical factor in the design of TCL control strategies. In Ref. [33], a forecast-and- update method is developed to consider the changing outdoor temperature in TCL control algorithms, in which the external temperature is first fixed during a time slot and then updated by measurements. The impact of external temperature on the aggregated behaviors of TCLs is modeled in a mathematical formulation for TCL control in DR programs [49]. Real-time outdoor temperature deviations, the ramping rate of temperature set-points, and the related deviation are considered in Ref. [30] as three inputs of the aggregated models, which provide the frequency capacity and ramping rate of TCLs under time-varying surroundings.

6 Dispatch models of TCLs

Aggregation is critical to integrate TCLs into power system operations. The majority of studies conducted on aggregated TCL models are focused on characterizing the detailed evolutions of internal temperatures and switch states at control signals of TCLs from the perspective of control (see Section 4) [22].

To dispatch TCLs in power system operations, a concise aggregated model at system level is required to accurately characterize TCL primary behaviors. TCL system models can be formulated as common components (e.g., batteries), which offer additional compatibility for power system operation [8]. In this way, TCL populations with complicated structures and alien parameters are invisible. In other words, TCLs are modeled as common batteries or generators connected with the grid from the perspective of operation and dispatch.

It is shown in Section 3 that TCLs have different response performances, in terms of response speed and duration, using diverse control methods. Accordingly, there are three categories for the dispatch models of TCLs in terms of response time scales, which will provide various ancillary services for the grid.

Hour-time scale: When scheduled in an hour-time scale, TCLs are usually utilized for peak load shaving. Thus, TCLs should have a longer duration but with less need for a fast response speed. The temperature adjustment is more suitable to regulate TCL power profiles for the hour-time scale dispatch. A virtual generator model is developed in Ref. [22] to describe the TCL response capacity with temperature control, considering a control strategy for cold load pickup. In Ref. [50], the virtual power plant that consists of a large TCL population is developed to manage distributed energy resources.

Minute-time scale: At this time scale, TCLs are generally employed to provide load following service. Thus, TCLs are required to respond faster but with less need for long duration time compared to that in hour-time scale. It is widely known that TCLs can make thermal energy stored in the related objects (e.g., buildings). For example, a leak storage unit is proposed in Refs. [51,52] for security-constrained economic dispatch and multi- service allocation. A thermal battery model of inverter air conditioners is developed in Refs. [8,53] to reduce the economic loss caused by renewable energy uncertainty in the real-time market. The capacity of TCLs at the 10-min time scale is estimated in Ref. [54] to dispatch generators in terms of the day-ahead temperature forecast of the market.

Second-time scale: TCLs are able to provide the second-time scale frequency regulation service to the grid. Accordingly, TCLs should act very quickly and tend to be controlled by on-off switches. Battery models are the most common and popular dispatch models for TCLs [22,55]. A battery model that describes the external characteristics of TCLs under dispatch signals is modeled in Ref. [22]. A stochastic battery model with dissipation is developed in Ref. [39] by aggregating the flexibility of TCL populations, considering the corresponding power limits and energy capacity. The optimization-based algorithms to build a virtual battery model is proposed in Ref. [55] for representing the TCL flexibility.

In summary, dispatch models of TCLs are closely related to control methods, in order to provide different ancillary services with various response characteristics. When scheduling aggregated TCLs with other resources, accurate dispatch model parameters are critical. However, there are only a few parameter estimation methods for TCL dispatch models. In Ref. [22], a high dimensional model representative method is developed to identify the aggregated parameters of multi-time-scale TCL system models, in which the probability distribution of the TCL parameters is utilized. An alternative approach is presented in Ref. [56] to identify the aggregated parameters without the information of individual TCL parameters.

7 Dispatch strategies of TCLs

It is an optimization problem when TCLs are dispatched in power system operations. The objective functions are to minimize load [50,57], cost [9,56,58], or the uncomfortableness of end users [59]. Decision variables can be power adjustment [8,9,51] and off-shift time [60]. Constrains are related with system operating states and TCL adjustment capacity, including power balancing [58], power capacity limit [51,59], and load payback [59]. Solution algorithms are usually based on linear programming [9,50,57] and dynamic programming [59].

As TCLs can be aggregated as common system components (e.g., batteries), the dispatch strategies of these components can be applied to the integration of TCL aggregated models into system operation. For example, there has been extensive literature focusing on the dispatch strategies of unit commitment [61], peak load shifting [62], output smoothing of wind generation [63], and economic optimization in electricity market [64]. Aggregated battery models of TCLs can substitute batteries to provide similar grid services, which shows the compatibility of TCL battery models with current dispatch strategies. Besides similarity between battery and aggregated TCLs models, there are also some differences in operating behaviors [8]. The optimal coordination of virtual storage (i.e., TCL battery models) and real storage (e.g., conventional batteries) is an interesting topic [9].

8 Discussion and future research

Many studies have been conducted on TCL modeling and control strategies for ancillary services. The remaining challenges and new opportunities with smart grid technology are discussed in this section.

8.1 Non-invasive parameter estimation

Non-invasive parameter estimation is a method that employs the external information of power, current or voltage to analyze and obtain the related internal parameters without installing the devices into the objects. This method is effective in protecting the privacy of objects without causing widespread concerns. For example, a non-invasive method is developed in Ref. [65] to identify the parameters of turbine, generator, and exciter using real network data and the stochastic optimization algorithm. However, there are few studies of non-invasive parameter estimation in the field of TCLs.

TCL physical parameters are relevant to structures, locations, device types, and behaviors of customers. Moreover, TCLs have large a quantity and are widely distributed. Thus, it is difficult to obtain TCL parameters because of the limited data available, measurement errors, and costly device installations. TCL parameters are generally assumed to be distributed in the form of normal [38], lognormal [66], or uniform distribution [67]. Few studies have been conducted on the parameter estimation of TCLs, considering privacy protection of end users for control and dispatch. Besides, the actual parameter distributions of TCL populations or accurate physical parameters of individual TCL remain a problem for the practical applications of TCLs. The identification of TCL parameters in a non-invasive way is an important topic.

8.2 Reward allocation mechanism

TCL parameters are heterogeneous with different operating characteristics. In other words, heterogeneous TCLs have diverse contributions in service provision with different monetary reward. To encourage the participation of end users in DR programs, a fair reward allocation mechanism is required to evaluate the response performances and distribute the compensation to participating TCLs, in terms of their contributions in ancillary services. However, few studies have been conducted on the fair reward allocation mechanism. Only the authors in Ref. [68] develop a specific index to evaluate the diverse capacity of TCLs and make a priority list to manage them accordingly. Besides, the compensation for TCLs is also determined according to a reward curve. When TCLs participate in energy service, the index of incremental average power is used to quantify the contribution of TCLs during the given time interval. As the time scale of ancillary service becomes shorter (e.g., load following), a minute-to-minute power measurement will be given to monitor the TCL capability. The power adjustment capability will affect the reward allocation, and intermediate coordinators may even receive penalty if the actual power deviates from the required amount. When the regulation service is provided, the on/off times of TCLs are important factors to be considered in contribution evaluation. It can be seen that not only the system requirement but also the specific performances of TCLs in different control methods need to be taken into account in the reward allocation mechanism. The design of a fair and reasonable reward allocation mechanism is important for TCLs to actively participate in DR programs.

8.3 Market-based control and dispatch mechanism

TCLs as important DR resources usually participate in the whole-sale market in a hierarchical way under load serving entities or aggregators. Therefore, TCLs have no access to market prices, which makes them insensitive to market signals and cannot be scheduled with full capability. Recently, transactive energy market has been proposed with intensive studies. The authors in Ref. [69] present a transactive framework in day-ahead market. To optimize the market operation, transactive markets of both the transmission and distribution levels are simultaneously cleared. Reference [70] also studies the trading mechanism and clearing process of the transactive energy market using Lagrangian relaxation. When the transactive energy market is utilized to manage TCLs, it will allow participants to interact with each other in a distributed scheduling way without a central controller. Then, the market information can be fully employed to effectively manage distributed resources with optimized objectives.

Moreover, transactive energy market will provide a distributed control method to manipulate TCL populations by market prices. This cannot only protect the privacy of customers but also relieve the communication burden between aggregator and customers. A transactive market to control electric vehicles is developed in Ref. [71], considering the cost of utilizing the charging flexibility of electric vehicles. The authors in Ref. [14] propose a transactive control method to manipulate commercial buildings for the ancillary service.

In summary, the transactive energy market mechanism is a powerful tool to manage TCLs in an effective and economical way, which will attract more attentions from researchers.

8.4 Optimal coordination of virtual and real energy storage

Energy storage devices are critical to smooth out the output of renewable generation and keep the supply-demand balance in real-time. Energy storage devices have been utilized in unit commitment [61], shifting peak [62], smoothing wind generation [63], and economic optimization in electricity market [64]. TCLs can store thermal energy in related objects, with similar energy storage characteristics as conventional energy storage devices. Thus, TCLs can be aggregated as virtual energy storage models to be compatible with existing dispatch strategies. The coordination of TCLs and real energy storage devices is helpful to improve power system operation. Virtual and real energy storage are optimally coordinated by the algorithms proposed in Ref. [9] for both grid and customers’ services. Moreover, the locations and scheduling cost between virtual and real energy storage devices should also be taken into account in developing coordination strategies, which could be another challenging topic.

9 Conclusions

TCLs as important DR resources have been intensively studied. This paper summarizes current research on fundamental models, response modes, control strategies, dispatch models, and dispatch strategies of TCLs in DR programs from a bottom-up perspective. Comparisons of different research methods in each aspect are presented. Besides abundant studies conducted on the modeling and control of TCLs, there are still great potentials of TCLs in DR programs. Four future research topics are discussed, including non-invasive parameter estimation, reward allocation mechanism, novel market control and dispatch mechanism, and optimization of virtual and real energy storage.

References

[1] Wu H, Shahidehpour M, Alabdulwahab A, . Demand response exchange in the stochastic day-ahead scheduling with variable renewable generation. IEEE Transactions on Sustainable Energy, 2015, 6(2): 516–525

[2] Sajjad I, Chicco G, Napoli R. Definitions of demand flexibility for aggregate residential loads. IEEE Transactions on Smart Grid, 2016, 7(6): 2633–2643

[3] Radaideh A, Vaidya U, Ajjarapu V. Sequential set-point control for heterogeneous thermostatically controlled loads through an extended Markov chain abstraction. IEEE Transactions on Smart Grid, 2019, 10(1): 116–127

[4] Kohlhepp P, Harb H, Wolisz H, . Large-scale grid integration of residential thermal energy storages as demand-side flexibility resource: a review of international field studies. Renewable & Sustainable Energy Reviews, 2019, 101: 527–547

[5] Xu X, Yan Z, Shahidehpour M, . Data-driven risk-averse two-stage optimal stochastic scheduling of energy and reserve with correlated wind power. IEEE Transactions on Sustainable Energy, 2020, 11(1): 436–447

[6] Erdinc O, Tascikaraoglu A, Paterakis N, . End-user comfort oriented day-ahead planning for responsive residential HVAC demand aggregation considering weather forecasts. IEEE Transactions on Smart Grid, 2017, 8(1): 362–372

[7] Kim Y J, Norford L K, Kirtley J L. Modeling and analysis of a variable speed heat pump for frequency regulation through direct load control. IEEE Transactions on Power Systems, 2015, 30(1): 397–408

[8] Song M, Gao C, Yan H, . Thermal battery modeling of inverter air conditioning for demand response. IEEE Transactions on Smart Grid, 2018, 9(6): 5522–5534

[9] Hao H, Wu D, Lian J, . Optimal coordination of building loads and energy storage for power grid and end user services. IEEE Transactions on Smart Grid, 2018, 9(5): 4335–4345

[10] Pahwa A, Brice C W. Modeling and system identification of residential air conditionning load. IEEE Transactions on Power Apparatus and Systems, 1985, 104(6): 1418–1425

[11] Song M, Gao C, Su W. Modeling and controlling of air-conditioning load for demand response applications. Automation of Electric Power Systems, 2016, 40(14): 158–167 (in Chinese)

[12] Molina A, Gabaldon A, Fuentes J A, . Implementation and assessment of physically based electrical load models: application to direct load control residential programmes. IEE Proceedings. Generation, Transmission and Distribution, 2003, 150(1): 61–66

[13] Iacovella S, Ruelens F, Vingerhoets P, . Cluster control of heterogeneous thermostatically controlled loads using tracer devices. IEEE Transactions on Smart Grid, 2017, 8(2): 528–536

[14] Hao H, Corbin C D, Kalsi K, . Transactive control of commercial buildings for demand response. IEEE Transactions on Power Systems, 2017, 32(1): 774–783

[15] Wu X, He J, Xu Y, . Hierarchical control of residential HVAC units for primary frequency regulation. IEEE Transactions on Smart Grid, 2018, 9(4): 3844–3856

[16] Song M, Gao C, Shahidehpour M, . Impact of uncertain parameters on TCL power capacity calculation via HDMR for generating power pulses. IEEE Transactions on Smart Grid, 2019, 10(3): 3112–3124

[17] Wai C H, Beaudin M, Zareipour H, . Cooling devices in demand response: a comparison of control methods. IEEE Transactions on Smart Grid, 2015, 6(1): 249–260

[18] Sinitsyn N A, Kundu S, Backhaus S. Safe protocols for generating power pulses with heterogeneous populations of thermostatically controlled loads. Energy Conversion and Management, 2013, 67: 297–308

[19] Lu N, Zhang Y. Design considerations of a centralized load controller using thermostatically controlled appliances for continuous regulation reserves. IEEE Transactions on Smart Grid, 2013, 4(2): 914–921

[20] Hu J, Cao J, Yong T, . Demand response load following of source and load systems. IEEE Transactions on Control Systems Technology, 2017, 25(5): 1586–1598

[21] Kim Y J, Fuentes E, Norford L K. Experimental study of grid frequency regulation ancillary service of a variable speed heat pump. IEEE Transactions on Power Systems, 2016, 31(4): 3090–3099

[22] Song M, Cao C W, Shahidehpour M, . Multi-time-scale modeling and parameter estimation of TCLs for smoothing out wind power generation variability. IEEE Transactions on Sustainable Energy, 2019, 10(1): 105–118

[23] Malhame R, Chong C Y. Electric load model synthesis by diffusion approximation of a high-order hybrid-state stochastic system. IEEE Transactions on Automatic Control, 1985, 30(9): 854–860

[24] Tindemans S H, Trovato V, Strbac G. Decentralized control of thermostatic loads for flexible demand response. IEEE Transactions on Control Systems Technology, 2015, 23(5): 1685–1700

[25] Bashash S, Fathy H K. Modeling and control of aggregate air conditioning loads for robust renewable power management. IEEE Transactions on Control Systems Technology, 2013, 21(4): 1318–1327

[26] Kamgarpour M, Ellen C, Soudjani S E Z, . Modeling options for demand side participation of thermostatically controlled loads. In: Bulk Power System Dynamics and Control- IX Optimization, Security and Control of the Emerging Power Grid (IREP), Rethymno, Greece, 2013

[27] Song M, Gao C, Shahidehpour M, . State space modeling and control of aggregated TCLs for regulation services in power grids. IEEE Transactions on Smart Grid, 2019, 10(4): 4095–4106

[28] Song M, Cao C, Yang J, . Novel aggregate control model of air conditioning loads for fast regulation service. IET Generation, Transmission & Distribution, 2017, 11(17): 4391–4401

[29] Liu M, Shi Y. Model predictive control of aggregated heterogeneous second-order thermostatically controlled loads for ancillary services. IEEE Transactions on Power Systems, 2016, 31(3): 1963–1971

[30] Hu J, Cao J, Chen M Z Q, . Load following of multiple heterogeneous TCL aggregators by centralized control. IEEE Transactions on Power Systems, 2017, 32(4): 3157–3167

[31] Liu M, Shi Y, Liu X. Distributed MPC of aggregated heterogeneous thermostatically controlled loads in smart grid. IEEE Transactions on Industrial Electronics, 2016, 63(2): 1120–1129

[32] Mathieu J L, Koch S, Callaway D S. State estimation and control of electric loads to manage real-time energy imbalance. IEEE Transactions on Power Systems, 2013, 28(1): 430–440

[33] Lu N. An evaluation of the HVAC load potential for providing load balancing service. IEEE Transactions on Smart Grid, 2012, 3(3): 1263–1270

[34] Esmaeil Zadeh Soudjani S, Abate A. Aggregation and control of populations of thermostatically controlled loads by formal abstractions. IEEE Transactions on Control Systems Technology, 2015, 23(3): 975–990

[35] Li C, Chen Y, Luo F, . Real-time decision making model for thermostatically controlled load aggregators by natural aggregation algorithm. In: IEEE International Conference on Energy Internet, Beijing, China, 2017

[36] Radaideh A, Vaidya U, Ajjarapu V. Sequential set-point control for heterogeneous thermostatically controlled loads through an extended markov chain abstraction. IEEE Transactions on Smart Grid, 2019, 10(1): 116–127

[37] Bhattarai B P, de Cerio Mendaza I D, Myers K S, . Optimum aggregation and control of spatially distributed flexible resources in smart grid. IEEE Transactions on Smart Grid, 2018, 9(5): 5311–5322

[38] Bashash S, Fathy H K. Modeling and control insights into demand-side energy management through setpoint control of thermostatic loads. In: Proceedings of the 2011 American Control Conference, San Francisco, CA, USA, 2011

[39] Hao H, Sanandaji B M, Poolla K, . Aggregate flexibility of thermostatically controlled loads. IEEE Transactions on Power Systems, 2015, 30(1): 189–198

[40] Ruelens F, Claessens B J, Vandael S, . Residential demand response applications using batch reinforcement learning. IEEE Transactions on Smart Grid, 2017, 8(5): 2149–2159

[41] Vrettos E, Ziras C, Andersson G. Fast and reliable primary frequency reserves from refrigerators with decentralized stochastic control. IEEE Transactions on Power Systems, 2017, 32(4): 2924–2941

[42] Molina-Garcia A, Bouffard F, Kirschen D S. Decentralized demand-side contribution to primary frequency control. IEEE Transactions on Power Systems, 2011, 26(1): 411–419

[43] Guo D, Zhang W, Yan G, . Decentralized control of aggregated loads for demand response. In: 2013 American Control Conference (ACC), Washington D.C., USA, 2013

[44] Meng K, Wang D, Dong Z Y, . Distributed control of thermostatically controlled loads in distribution network with high penetration of solar PV. CSEE Journal of Power and Energy Systems., 2017, 3(1): 53–62

[45] Saker N, Petit M, Vannier J C, . Demand side management of electrical water heaters and evaluation of the cold load pick-up characteristics. In: 2011 IEEE Trondheim PowerTech, Trondheim, Norway, 2011

[46] Perfumo C, Braslavsky J, Ward J K. A sensitivity analysis of the dynamics of a population of thermostatically-controlled loads. In: 2013 Australasian Universities Power Engineering Conference (AUPEC), Hobart, TAS, Australia, 2013

[47] Gilvanejad M, Askarian Abyaneh H, Mazlumi K. Estimation of cold-load pickup occurrence rate in distribution systems. IEEE Transactions on Power Delivery, 2013, 28(2): 1138–1147

[48] Wang D, Lu N, Miao W, . Performance evaluation of controlling thermostatically controlled appliances as virtual generators using comfort-constrained state-queueing models. IET Generation Transmission & Distribution, 2014, 8(4): 591–599

[49] Mahdavi N, Braslavsky J H, Perfumo C. Mapping the effect of ambient temperature on the power demand of populations of air conditioners. IEEE Transactions on Smart Grid, 2018, 9(3): 1540–1550

[50] Ruiz N, Cobelo I, Oyarzabal J. A direct load control model for virtual power plant management. IEEE Transactions on Power Systems, 2009, 24(2): 959–966

[51] Trovato V, Tindemans S H, Strbac G. The leaky storage model for optimal multi-service allocation of thermostatic loads. IET Generation, Transmission & Distribution, 2016, 10(3): 585–593

[52] Trovato V, Tindemans S H, Strbac G. Security constrained economic dispatch with flexible thermostatically controlled loads. In: IEEE PES Innovative Smart Grid Technologies, Istanbul, Turkey, 2014

[53] Song M, Gao C, Yang J, . Energy storage modeling of inverter air conditioning for output optimizing of wind generation in the electricity market. CSEE Journal of Power & Energy Systems, 2018, 4(3): 305–315

[54] Luo F, Dong Z Y, Meng K, . An operational planning framework for large-scale thermostatically controlled load dispatch. IEEE Transactions on Industrial Informatics, 2017, 13(1): 217–227

[55] Zhao L, Zhang W, Hao H, . A geometric approach to aggregate flexibility modeling of thermostatically controlled loads. IEEE Transactions on Power Systems, 2017, 32(6): 4721–4731

[56] Mathieu J L, Kamgarpour M, Lygeros J, . Arbitraging intraday wholesale energy market prices with aggregations of thermostatic loads. IEEE Transactions on Power Systems, 2015, 30(2): 763–772

[57] Kurucz C N, Brandt D, Sim S. A linear programming model for reducing system peak through customer load control programs. IEEE Transactions on Power Systems, 1996, 11(4): 1817–1824

[58] Yang H T, Huang K Y. Direct load control using fuzzy dynamic programming. IEE Proceedings–Generation, Transmission and Distribution, 1999, 146(3): 294–300

[59] Lee T F, Cho M Y, Hsiao Y C, . Optimization and implementation of a load control scheduler using relaxed dynamic programming for large air conditioner loads. IEEE Transactions on Power Systems, 2008, 23(2): 691–702

[60] Chu C M, Jong T L, Huang Y W. Mitigating DLC constraints of air-conditioning loads using a group-DLC method. In: 2007 IEEE Power Engineering Society General Meeting, Tampa, FL, USA, 2007

[61] Wen Y, Li W, Huang G, . Frequency dynamics constrained unit commitment with battery energy storage. IEEE Transactions on Power Systems, 2016, 31(6): 5115–5125

[62] Ning Y, Li X, Ma X, . Optimal schedule strategy of battery energy storage systems for peak load shifting based on interior point method. In: 2016 12th World Congress on Intelligent Control and Automation (WCICA), Guilin, China, 2016

[63] Li N, Uckun C, Constantinescu E M, . Flexible operation of batteries in power system scheduling with renewable energy. IEEE Transactions on Sustainable Energy, 2016, 7(2): 685–696

[64] Lujano-Rojas J M, Dufo-Lopez R, Bernal-Agustin J L, . Optimizing daily operation of battery energy storage systems under real-time pricing schemes. IEEE Transactions on Smart Grid, 2017, 8(1): 316–330

[65] Hutchison G, Giaouris D, Gadoue S, . Non-invasive identification of turbo-generator parameters from actual transient network data. IET-Generation Transmission & Distribution, 2015, 9(11): 1129–1136

[66] Mahdavi N, Braslavsky J H, Seron M M, . Model predictive control of distributed air-conditioning loads to compensate fluctuations in solar powe. IEEE Transactions on Smart Grid, 2017, 8(6): 3055–3065

[67] Mathieu J L, Callaway D S. State estimation and control of heterogeneous thermostatically controlled loads for load following. In: 2012 45th Hawaii International Conference on System Sciences, Maui, HI, USA, 2012

[68] Vanouni M, Lu N. A reward allocation mechanism for thermostatically controlled loads participating in intra-hour ancillary services. IEEE Transactions on Smart Grid, 2018, 9(5): 4209–4219

[69] Renani Y K, Ehsan M, Shahidehpour M. Optimal transactive market operations with distribution system operators. IEEE Transactions on Smart Grid, 2018, 9(6): 6692–6701

[70] Gregoratti D, Matamoros J. Distributed energy trading: the multiple-microgrid case. Industrial Electronics IEEE Transactions on Industrial Electronics, 2015, 62(4): 2551–2559

[71] Liu Z, Wu Q, Shahidehpour M, . Transactive real-time electric vehicle charging management for commercial buildings with PV on-site generation. IEEE Transactions on Smart Grid, 2019, 10(5): 4939–4950

RIGHTS & PERMISSIONS

Higher Education Press