State identification of home appliance with transient features in residential buildings

Abstract

Nonintrusive load monitoring (NILM) is crucial for extracting patterns of electricity consumption of household appliance that can guide users’ behavior in using electricity while their privacy is respected. This study proposes an online method based on the transient behavior of individual appliances as well as system steady-state characteristics to estimate the operating states of the appliances. It determines the number of states for each appliance using the density-based spatial clustering of applications with noise (DBSCAN) method and models the transition relationship among different states. The states of the working appliances are identified from aggregated power signals using the Kalman filtering method in the factorial hidden Markov model (FHMM). Thereafter, the identified states are confirmed by the verification of system states, which are the combination of the working states of individual appliances. The verification step involves comparing the total measured power consumption with the total estimated power consumption. The use of transient features can achieve fast state inference and it is suitable for online load disaggregation. The proposed method was tested on a high-resolution data set such as Labeled hIgh-Frequency daTaset for Electricity Disaggregation (LIFTED) and it outperformed other related methods in the literature.

Graphical abstract

Keywords

nonintrusive load monitoring (NILM) / load disaggregation / online load disaggregation / Kalman filtering / factorial hidden Markov model (FHMM) / Labeled hIgh-Frequency daTaset for Electricity Disaggregation (LIFTED)

1 Introduction

Over the past two decades, load disaggregation has been used in different engineering areas such as smart grids, sustainable cities, smart buildings, and photovoltaic (PV) generation [1]. As an essential step for effective and economic energy management in a smart city, load disaggregation plays a critical role in cultivating the energy consumption behavior of customers to achieve a sustainable society [2]. Customers can achieve energy savings of up to 12% when the energy consumption is regulated at the appliance level [3]. Moreover, increasing installation of smart meters in buildings has improved the development of load disaggregation. Therefore, nonintrusive load monitoring (NILM) for disaggregating energy consumption measurements and identifying appliance patterns has been widely developed. The nonintrusive appliance load monitoring proposed by Hart [4] provides a forward-looking introduction to NILM that focuses on the decomposition of household load and the determination of when appliances are being used.

Appliance modeling is a crucial step of the NILM, which can be described by the hidden Markov model (HMM) with observed variables and hidden state variables. Factional hidden Markov model (FHMM) is one of the HMM variants. When FHMM is applied in NILM, each appliance can be represented as an HMM. Kong et al. [5] combined the FHMM and integer quadratic constraints to address load disaggregation. However, this time-consuming method cannot identify an appliance with similar features. Guo et al. proposed an explicit-duration HMM that characterizes each state’s duration using different observations [6]. Makonin et al. used the FHMM model to reduce the number of available states using a sparse algorithm and decoding the correlation of each appliance load in a household [7]. However, the primary limitation of this method lies in its assumption that each load is a type I appliance, which only has two states: on and off, whereas many household appliances may have multiple states.

Additionally, feature selection helps with the characterization of appliance behaviors and improves accuracy. The features can be divided into steady-state and transient features [8]. Steady-state features can be extracted from a low sampling rate of 1 Hz. When the appliances switch states, transient features can be extracted from higher frequency data in the range of 1 Hz to 1000 Hz. Compared with the classic load signatures, a novel type of two-dimensional load signature [9] based on the instantaneous reactive power theory is proposed, which is different from the V-I based load signature features. Brito et al. proposed a fuzzy rule-based method to identify different loads in which the harmonic impedances of different loads are determined [10]. However, these two methods are not suitable for practical applications because they require an accurate electrical operating time. In a real situation, multiple appliances may be turned on simultaneously. Therefore, the signals of the individual appliances may overlap into an aggregated signal. A single feature is not sufficient to identify the appliance state in an overlapped situation. Thus, the steady-state and transient features are combined to overcome the misidentification of single-device features [11]. In addition, most disaggregation methods take a long time to disaggregate composite load because of the complexity of the algorithm, which does not work efficiently when real-time load disaggregation is needed [12,13]. The Viterbi algorithm is widely used in the FHMM model [14]. Similarly, this method is time consuming as it requires forward and backward steps to infer the electrical states, in which the forward step considers the transition relationship among states. Two variants of the Bayesian filter, deformable particle filter [15], and Kalman filter [16], have fast computation speed and reliable accuracy, which are the basic requirements for an online algorithm. However, existing methods have a high computational complexity. Herein, we focus on reducing the complexity of disaggregation by replacing the observed power with the transient feature in the real-time disaggregation method.

To address the above-mentioned problems, we propose a transient-based NILM framework that implements a Kalman filter-based method into the FHMM model using the transient and steady-state features to infer each appliance state. The contributions of this study are as follows:

• The proposed method integrates the transient features, regardless of the resolution of the measurements, in the Kalman filter with FHMM to identify working appliances using a sliding window rather than point-by-point inference, as in Ref. [5]. Additionally, it reduces the computation time; therefore, it can be applied in real time.

• The proposed method encodes system states as super states that represent the state combination of different appliances with multiple states. This can be used to verify whether the estimated results are consistent with the measurements, guaranteeing the accuracy of load disaggregation.

• The proposed method applies the density-based spatial clustering of applications with noise (DBSCAN) algorithm to determine the number of states for individual appliances in an unsupervised manner, which can be used in practice.

The remainder of this paper is organized as follows: Section 2 models individual appliances and constructs a feature library. Section 3 introduces the proposed real-time NILM method, which includes the FHMM and Kalman filtering. A case study and discussion are presented in Section 4. In Section 5, we conclude and propose some opportunities for future work.

2 Modeling of individual appliance and super state

NILM aims to identify the working appliances from the composite load, as shown in Eq. (1),

(1)y_t=\sum _{n=1}^N y_t^n+e_t,

where y_t is the composite load at time t, y_t^n is the load consumption of appliance n at time t, and e_t is the noise at time t. In NILM, the known information is the total measured load y_t and the objective is to estimate the detailed load y_t^n.

The proposed NILM framework is shown in Fig. 1. It starts from the ground truth, that is, labeled individual appliance reading, followed by modeling of individual appliances. All the appliances are modeled as HMMs, where all the steady-states transit from one to another along the Markov chains. Moreover, transient features are required to determine the states of the working appliances. Both the transient and steady-state features are stored in the generalized appliance model library and they are used to identify appliances from aggregated loads (e.g., high-resolution smart meter readings) and estimate the associated power consumption. The next steps are event detection, feature extraction, and clustering steps to model each appliance as Markov chains, and they use the feature information of each appliance to disaggregate the composite load.

The gray square in Fig. 1 represents the modeling of the Markov chains of individual appliances that need to be trained. The overall flowchart of the training steps is illustrated in Fig. 2. The proposed method is an event-based NILM method that requires the features extraction from segments divided by events. The feature extraction step is to extract transient features from transition segments and extract steady-state features from steady segments, and the DBSCAN clustering method determines the number of states with the steady-state features. Finally, the training models of different appliances were stored in the model library.

2.1 Modeling of individual appliances as Markov chains

HMM can be used to represent the working states and power consumption of individual appliances, as shown in Fig. 3. In each time step, each appliance emits an observation of its power consumption. For a period of observations with T time steps, the time series of observations can be represented as y=\{y_1, y_2,\cdots , y_T\}, which are shaded circles in Fig. 3. Correspondingly, the working states are represented as x =\{x_1,x_2,\cdots ,x_T\}, which are shown as squares. Each appliance has a limited number of states and it transfers from one state to another under working conditions. Assuming that one appliance has states X=\{0, 1,\cdots , K-1\} with K operating states, its operating cycle from x_1 to x_t can be considered as a series of state transitions among K operating states, as shown in Fig. 4.

In addition to state transition, the appliance emits a series of observations y=\{y_1,y_2,\cdots ,y_T\}. The observations of each state are typically modeled as a Gaussian distribution. The typical HMM model for each electrical appliance includes three elements: the prior probability of the initial state, transition matrix, and emission vector, as represented in Eqs. (2)–(4).

(2)p(x_1^n=k )={\pi }_k,

(3)p(x_t^n=j x_{t-1}^n=i)=A_{ij}^n,

(4)p(y^n{x}^n=i)=B_{n,i}=N({\mu }_{n,i}, {\sigma }_{n,i}) ,

where x_t^n is the state of appliance n at time t, {\pi }_k is the probability of state k as the initial state, and A_{i,j}^n is the state transition probability of appliance n from state i at time t-1 to state j at time t. p(y_t^n{x}_t^n) is governed by mean value {\mu }_{n,i} and standard deviation {\sigma }_{n,i} of appliance n at state i, representing the Gaussian distribution of the emission vector.

2.2 Event detection

Event detection is the first and most basic step in an event-based NILM. It determines the switching on and off actions of an appliance from the measurements. The transition process of an appliance contains abundant information that can help track its operating cycles.

Difficulties in event detection include long transients, high fluctuations, and simultaneous events. Long transition refers to the long duration of a transition process; high fluctuation refers to large power oscillation during the steady-state period; simultaneous events mean that two or more events occur consecutively in a short period. Long transitions and simultaneous events are contradictory pairs that require different time window parameters to detect events accurately. Conversely, high fluctuations can easily mask the behaviors of small appliance. The difference in current intensity is used in Ref. [17] to determine whether an appliance-switching event occurs in the circuit. The hybrid event detection method [18] from our previous work can be used to detect events.

Figure 5 shows the feature points that can be used to uniquely locate the waveform and measure the transition process. Although the transition process of different appliances significantly varies, it involves a transient spike, which decreases slowly to a steady-state. The start, peak, and end points are three feature points that can be used to label the entire transient, which can be used for event detection.

2.3 Feature extraction

After capturing the transition process, significant features can be extracted and used as appliance signatures to help identify the corresponding appliances. The method proposed in this study takes advantage of transients to extract the power difference between transient spike power and steady state average power, namely DTS.

Transient spike power exists widely in the measurements of motor-driven electrical appliances. Such appliances usually generate a high power consumption transient that goes far beyond steady-state power. Transient feature is the deterministic feature to decide which type of appliance it belongs to. Figure 6 shows multiple cycles of vacuum at a sample rate of 50 Hz. The figure shows that there is significant and stable transient spike power consumption. Once the transient spike power is determined, the DTS can be easily calculated.

Beyond motor-driven appliances, most common appliances such as kettles, rice cookers, and steamers in residential buildings are mostly resistive. These appliances do not generate a transient spike power that is significantly larger than the steady-state power. Additionally, they reach steady-state power directly, instead of experiencing transient spike power. Figure 7 shows sample power measurements of a kettle at a sample rate of 50 Hz. Therefore, it is not necessary to consider transient features for appliances that are mostly resistive.

Generally, the difference in the transient waveform between motor- and resistance-based appliances can be used as a feature to distinguish these two types of appliances. It not only increases the identification accuracy of individual appliances, but it can also save significant computational time because it only needs to infer the possible type of working appliances based on whether or not there is a transient feature.

After establishing that the transient feature is deterministic enough to classify different types of appliances, it is important to extract the transient features effectively. As shown in Fig. 5, feature points can be detected using various event detection methods. After the feature points are identified, they can be used to calculate the feature parameters such as DTS. In addition, the difference between the average of datapoint values within a certain range before and after the start and end points is an important feature that indicates the difference between two steady-state powers, namely DSS. DSS and DTS can be used to infer the appliance that is in a changing state.

In the training process, feature parameters such as DTS and DSS have many instances because the same transition is repeated. These feature parameters are modeled as a Gaussian distribution that increases the accuracy and robustness of the test process, even if there are large variations in the parameters.

2.4 Determining the number of states using DBSCAN

In this study, DBSCAN was performed to cluster the steady-state measurements to the corresponding state. DBSCAN is a typical density-based clustering algorithm [19]. This method has the following advantages:

• Unlike k-means [20] and Fuzzy C-means (FCM) clustering algorithms [21], DBSCAN does not require prior appliance-state information and it does not need to specify the number of clusters.

• Although DBSCAN does not work well on selecting parameters with high-dimensional data, it is suitable for clustering the appliance state because the NILM data set is low-dimensional data.

• Owing to the fluctuation and complexity of the load, noise interference and fluctuation occurs during data acquisition. DBSCAN can divide the region with a sufficiently high density into clusters and find clusters of arbitrary shapes in the spatial database with noise.

Eps and MinPts are two essential DBSCAN parameters. Eps specifies the radius of a neighborhood with respect to the core points in a cluster. MinPts is the minimum number of points in a cluster. The number of cores is reduced if MinPts is too large. Conversely, if MinPts is too small, the number of cores will produce many sub-clusters that should have been in the same cluster. In this study, a KNN-DBSCAN based algorithm was used to determine the value of Eps and to generate MinPts [22]. The algorithm does not need to input Eps manually; instead, Eps is determined by the training data.

The average distance value, which is the average Euclidean distance between all the data points, was selected as the candidate Eps parameter. The MinPts parameter was generated using the mathematical expectation method. The specific process is as follows: first, the number of neighboring data points of each datapoint within the Eps distance was calculated, and then all the numbers were averaged as MinPts.

When Eps and MinPts are determined, the data set can be grouped into three types of data points: core, border, and noise. The core point is a point whose number of neighboring data points within Eps is larger than MinPts. Border points are data points on the edge of a cluster. The algorithm starts from one random datapoint and continues until the border points occur, and all the data points in this procedure belong to the same cluster.

After each appliance is classified into limited states, the emission vector of the HMM parameter can be quickly calculated according to the number of data points in each cluster.

2.5 System super state

Appliance states can be identified with event detection (to locate the occurrence of events) and feature extraction (to extract features, specifically DTS, which can be used to identify appliance states). The estimated states can be further verified by comparing the measured aggregated load with the total estimated power consumption that needs to be retrieved from the corresponding power consumption of each appliance state from the trained appliance models. However, it is time-consuming to enumerate each appliance for real-time applications. To solve this problem, the power consumption of all the possible combinations of appliance states can be calculated in advance and stored in the trained model library, where the system super state links each appliance’s state and the total estimated power consumption.

A system super state refers to a state combination of different appliances. Different appliances may work in different states and have different power consumptions; therefore, it might be complicated to store the many system states of all appliances. The system super state provides a straightforward and direct way to illustrate the state of the system and the amount of power consumed.

To simplify the expression of multiple appliances with multiple states, in addition to the on and off modes, we propose a modified encoding method to serialize the system state number. In addition, the storage burden declines significantly when many appliances are recorded as a single number. The super state of the system is expressed according to Eq. (5),

(5)x_t=( x_t^1,x_t^2,\cdots ,x_t^n) .

The system state index in terms of the state of each appliance can be described as follows:

(6)I=\sum _{n=1}^{N-1}(x^n\prod _{k=n+1}^N{m}^k)+x^N,

where I represents the system state index, x^nis the current state of the nth appliance, x^Nrepresents the current state of the Nth appliance, m^k is the number of appliance k’s states, and N is the total number of appliances.

A simple example of a 5-appliance-state encoding/decoding process is summarized in Table 1. All the appliances (kettle, vacuum, steamer, hairdryer, hotpot, and mixer) should be arranged in an unchanged sequence. According to Eq. (6), the state index of the current system is 1 \times (2 \times 4 \times 2 \times 3) + 0 \times (4 \times 2 \times3) + 1 \times (2 \times 3) + 1 \times 3 + 2 = 59. Thus, the internal states of the five appliances are replaced by a system state number.

To obtain the current state of the appliances, the single appliance state x^n can be decoded from system state I. The process calculates the remainder of the division of the total available states of each appliance from the last device to the first device. To illustrate, system state index 59 divided by 3 (the appliance dryer’s total number of states) leads to 19 (quotient) with a remainder of 2, which is the dryer’s current state. The last quotient, i.e.,19, divided by the appliance mixer’s total number of states 2 leads to 9 (new quotient) with a remainder of 1, which is the mixer’s current state. This process was continued until the last appliance, kettle. Finally, system state 59 can be decoded into a set of states [1,0,1,1,2] that correspond to the appliance states listed in the last row of Table 1.

2.6 Construction of the model library

A model library is used to store the models of individual appliances. The construction of a model library starts with the training of labeled appliance data and extends as the connection of new appliances. In the load disaggregation stage, the models are called to identify working appliances. It is also helpful to construct the library in such a way that it can be retrieved quickly.

In this section, individual appliances are modeled as Markov chains, including the prior probability of the initial state, transition matrix, and emission vector after clustering the measurements into several states. The transients are mapped accurately through three key feature points: the start, peak, and end points. Additionally, the associated features are extracted from the transients. Furthermore, the super state is used to represent the system state in an abstract manner, which is convenient for probability calculation and other operations.

The representation of each appliance model is shown in Eq. (7), where the elements are the prior probabilities of the initial state, transition matrix, emission vector, transient features, and super state information, where {\phi }^n is the transient feature of nth appliance, and X_I is the Ith super state,

(7)\theta =\left\{{\pi }_n,A^n,B^n,{\phi }^n,X_I\right\}.

As more models of appliances are stored in the library, the library develops into a generalized model library. A more powerful library uses more general features, such as waveform and periodicity, to identify the appliance or the type of appliance that is working.

3 Composite load disaggregation methodology

Appliance-state inference plays a crucial role in NILM. The NILM estimates the states of each appliance given the composite load. Various techniques are used to analyze the appliance combination of the given observations. These methods have a large computational complexity; thus, they cannot perform load disaggregation in real time. This study proposes a FHMM with a Kalman filtering solver (FHMM-KF) to overcome these limitations. This method estimates the working appliances in the transition process and confirms the working states in the subsequent steady periods.

3.1 Framework of FHMM-KF

During the modeling of individual appliances, an individual appliance is modeled as a Markov chain. Suppose there are N appliances in a building. The overall appliances can be formulated as the FHMM, as shown in Fig. 8, where x_t^n represents the state of appliance n at time t.

Unlike the conventional FHMM method, the proposed FHMM-KF method depends on the transient feature to infer the appliance that is changing mode in the transition process. The Kalman filter performed well in the continuous system state evaluation. It reduces the memory requirement because it only requires the previous state value. Therefore, it is suitable for real-time applications. It estimates a specific variable posteriorly in two phases: prediction and updating. The prediction step uses the previous state to estimate the current state. In the updating phase, the filter optimizes the predicted value obtained in the prediction step using the observed value of the current state. Based on the above principles, the Kalman filter method is viable for solving the event-based NILM because it enables posterior inference with observation and state prediction. The prediction step selects the following possible state based on the transition matrix in the Markov chains. Thereafter, the posterior probability of the chosen state is returned based on the transient feature variable, {\phi }_t, that is extracted from observation y.

3.2 State inference in transients using FHMM-KF

The transient characteristics of appliances provide valuable information that is not available in a known low-frequency data set. The proposed method relies on transient features for real-time identification. It overcomes the feature similarity problem that arises when we depend only on the steady-state features. Accordingly, the FHMM-KF is divided into two steps: prediction and updating. The prediction step can select the following possible states at time t according to the transition matrix in the Markov chains. The probability of state transition is different for different end states, even in the same state.

Thereafter, the posterior probability of the chosen state is returned based on {\phi }_t that is extracted from observation y. The updating step of FHMM-KF aims to match the unknown feature {\phi }_t to an appliance with the maximum probability, as shown in Eq. (8). It calculates the probability along each Markov chain of each appliance iteratively.

The most likely state from all appliances is estimated according to Eq. (8). The second line in Eq. (8) is based on Bayes’ theorem. Because {\phi }_t is the feature variable from measurements that is a constant, it can be ignored, as shown in the third line. Finally, the equation can be represented as the state transition probability, p( x_t^n x_{t-1}^n,\theta ), and the emission vector probability of each appliance, p({\phi }_t{x}_{t-1}^n,x_t^n,\theta ). It estimates the states according to probability multiplication, as shown in Eq. (8),

(8)\begin{array}{rl}& \underset{n=1:N}{\text{argmax}}p(x_t^n{x}_{t-1}^n,\theta ,{\phi }_t)\\ =& \underset{n=1:N}{\text{argmax}}\frac{p(x_t^n,{\phi }_t{x}_{t-1}^n,\theta )}{p({\phi }_t{x}_{t-1}^n,\theta )}\propto \underset{n=1:N}{\text{argmax}}p(x_t^n,{\phi }_t{x}_{t-1}^n,\theta )\\ =& \underset{n=1:N}{\text{argmax}}p(x_t^n{x}_{t-1}^n,\theta )\times p({\phi }_t{x}_{t-1}^n,x_t^n,{\phi }_t).\end{array}

3.3 Steady-state confirmation with super state

In the state inference step using the FHMM-KF, the transition matrix and transient features are used to identify the changing state of the appliance in the current transition process. It guarantees and enables fast identification of working appliances in real time. However, there might be some misidentification owing to high fluctuation and feature similarity for some appliances. The proposed method checks whether the identification results are correct by comparing the estimated total load and the power emission in the current super state, which is generated immediately after the inference in transients.

The emission vector is modeled as a Gaussian distribution for each appliance; therefore, the emission value of the super state should follow a Gaussian distribution, based on the property of Gaussian distribution, as shown in Eq. (9),

(9)X_I\sim N(\sum _{n=1}^N {\mu }_n ,\sum _{n=1}^N{\sigma }_n),

where X_I is the Ith super state and N is the total number of appliances. If the difference between y_t and \sum _{n=1}^N{\mu }_n is less than three times that of \sum _{n=1}^N{\sigma }_n, the inference result is correct at a probability of 99.7% according to the property of Gaussian distribution. Otherwise, the NILM algorithm rolls back to infer another possible state.

The false estimated state rectification was obtained from the results of Eq. (8). This method iteratively checks the state with the next highest probability and performs a steady-state confirmation. The iteration ends once it meets the requirement that the difference between y_t and \sum _{n=1}^N{\mu }_n is less than three times that of \sum _{n=1}^N{\sigma }_n. Subsequently, the state is determined.

4 Case study

In this section, the proposed FHMM-KF method is tested on synthetic data based on the Labeled hIgh-Frequency daTaset for Electricity Disaggregation (LIFTED) data set [23] and compared with other methods. All tests were run in Python 3.6 on a laptop with a 3.7 GHz Intel I7-7700 K CPU and 8 G memory.

4.1 Synthetic data and evaluation metric

LIFTED is a one-week NILM data set at a sampling rate of 50 Hz. It includes appliance-level details that can be used to train an individual appliance model. Testing on such a high-resolution data set can take advantage of the complete transient information to verify the effectiveness of the proposed FHMM-KF method. In this study, the overall synthetic data are the measurements of approximately 11 h, as shown in Fig. 9, where 30% of the data are used for training and 70% is used for testing. Synthetic data can be downloaded from the website of GitHub.

In the synthetic data, ten appliances were chosen to test the proposed FHMM-KF method. Six of the appliances are motor-driven appliances, including a vacuum, hair dryer, refrigerator, mixer, blender, and washing machine. Other appliances, including a kettle, steamer, toaster, and hotpot are electric resistance heating appliances.

To evaluate the performance of the proposed FHMM-KF method, the accuracy metric and f₁-score used in Ref. [21] were used in this study, as shown in Eqs. (10)‒(11), respectively,

(10)Accur acy=(TP+ TN)/(TP+TN+FP+FN),

(11)f_1=2\times Preci sion\times Recal l/ (Pre cisio n+Rec all) ,

where TP, TN, FP, and FN represent true positive, true negative, false positive, and false negative, respectively. Precision = TP/(TP + FP), Recall = TP/(TP + FN).

4.2 Extracted transient feature validation

Features are the abstract and complete representation of continuous data streaming from electrical appliances. It should be effective and unique enough to distinguish different appliances. This study uses transient features, DTS, which depicts the overall transition process and reflects the characteristics of measurement in the power and time domains.

The proposed FHMM-KF method was tested on synthetic data and compared with FHMM-KF without using transient features (FHMM-KFNT). The test results in terms of accuracy and f₁-score for the two methods are presented in Fig. 10. The detailed accuracy data and f₁-score are summarized in Table A1 in the Appendix. The test results indicate that the performance is poor if transient features are not used in the inference step.

Compared with other methods that only use steady-state features, transient-based methods can identify appliances with similar steady-state features. As shown in Fig. 10(a), the accuracy of the kettle and vacuum decreases significantly when the transients are not completely considered. This is because the vacuum is a motor-driven appliance with significant transient. It is easy to identify these two appliances when using the transient features of the DTS. y only depending on the parameters in Markov chains, such as transition probability A_{ij}^n and emission vector B_i^n, the two appliances cannot be identified with high accuracy. Because these two appliances only have OFF and ON states, the transition probability is one. The kettle has an average power consumption of 1025 W and its standard deviation is 5.2 W; however, the vacuum has a close average power consumption of 1010 W with a higher standard deviation of 22 W. This means that the distributions of the average power consumption of these two appliances easily generate a close emission value. It is difficult to identify the appliance that is turning on using only the probability from the modeled Gaussian distribution in the emission vector.

Figure 10(b) shows the results of the f₁-score that indicates the overall performance of the test case. The f₁-score focuses more on the FP and FN, which refer to the misidentification of appliances with similar features. In the test case, there were 10 appliances with 28 different states. The average power consumption of the mixer and blender are close to some states of the washing machine and refrigerator.

4.3 Validation of the system super state

When using the transient features to infer the appliance state, the appliance and its associated working state are returned according to the sorted inference probability. However, it is difficult to distinguish the appliance that is changing mode for appliances without clear transition processes, specifically for small appliances changing state during high fluctuations of large appliances such as washing machines. Notably, although some appliances do not have a significant transition process, the proposed method extracts transient features, which are close to DTS, for the inference step.

The super state is used to verify whether the inference result in the transient is correct by comparing the measured power consumption with that of the corresponding super state. After obtaining individual appliance states from the previous inference step, the corresponding super state is expressed according to Eq. (5), and its index is calculated according to Eq. (6). Thus, the stored information (e.g., mean and variance) is retrieved based on the index from the appliance model database. When the difference between the retrieved mean of the super state and the measured power consumption is larger than the preset threshold (the threshold is 20 W in this study), the algorithm returns to the transient feature inference step to return the appliance with the second highest probability as the inference result.

The test results in terms of accuracy and f₁-score for the proposed FHMM-KF method and the comparison of FHMM-KF method not using super state validation (FHMM-KFNS) are presented in Fig. 11. Detailed data are listed in Table A2 in the Appendix.

If steady-state confirmation with the system super state is not used, the refrigerator and hair dryer could not be identified with high accuracy. For instance, the f₁-score of the refrigerator and hair dryer was 72.79% and 87.41%, respectively. The super state is used to correct the misidentification after state inference in the transition process. In particular, for the turning-off or power decreasing transition process, some appliances might be misidentified because of the lack of transient information. Because the refrigerator and hair dryer have some states with similar power consumption, they could not be identified correctly. Moreover, because the refrigerator operates for a long time, misidentification continues until its next action. Therefore, its accuracy and f₁-score were the worst. State inference is used to identify working appliances from total power consumption, whereas the super state verifies whether the identification is correct. Super state verification ensures misidentification rectification when there is a large gap between the given total power consumption and the estimated total power consumption. Accordingly, the accuracy and f₁-score are increased through prompt misidentification rectification.

Additionally, Tables A1 and A2 show that the accuracy of the kettle and vacuum of FHMM-KFNS is higher than that of the FHMM-KFNT. This indicates that the transient features in FHMM-KFNS are significantly important for distinguishing appliances with similar steady-state features.

4.4 Comparison with other published methods

A comparison with other published methods could better illustrate the performance of the proposed method. Notably, the segment-wise integer quadratic constraint programming (SIQCP) method [5] is the best among the NILM methods using the FHMM framework. The SIQCP method does not take advantage of transient features and it can be used as a benchmark for evaluating the FHMM-KF method.

As shown in Fig. 12, the FHMM-KF method outperformed the SIQCP method on the LIFTED data set. Detailed data on accuracy and f₁-score comparison between FHMM-KF and SIQCP are listed in Table A3 in the Appendix. Similar to the FHMM-KFNT method, the SIQCP method does not consider the transient feature; therefore, its performance trend is highly similar to that of the FHMM-KFNT method. As shown in Fig. 12(a), the low accuracy of the kettle, vacuum, mixer, and blender is mainly owing to the feature similarity and lack of transient features, which are the advantages of the proposed FHMM-KF method. When transient features are not considered, the kettle and vacuum do not have a significant difference because they have similar steady-state power consumptitablkeon and they do not have too much fluctuation. It might be easy to distinguish these appliances when only one is working. However, if such appliances are working in complex practical scenarios, the SIQCP method cannot distinguish them.

Moreover, the SIQCP method requires a batch of data to infer the appliance that is working; thus, it has a high latency. However, the proposed FHMM-KF method extracts the features from transients and infers the corresponding working appliances 2 s after they are turned on. Therefore, the FHMM-KF method can acquire the inference results in a real-time manner. In comparison, the SIQCP method infers the working states of appliances when the steady-state finishes. Considering the kettle as an example, the steady-state lasts approximately 10 min; thus, it takes the SIQCP method 10 min after the kettle turns on to infer the working states. This is another advantage of the FHMM-KF method.

4.5 Power consumption estimation

Additionally, state inference acquires the power consumption information of individual appliances and decreases the waste of electricity. Therefore, it is crucial to estimate the power consumption of appliances accurately for a qualified NILM method. A sample of the ground truth (using the proposed FHMM-KF method) and estimated power of the kettle, vacuum, steamer, and hair dryer is illustrated in Fig. 13. The estimated power is the corresponding power of the inferred state in the trained model, whereas the ground truth is the collected measurement. Figure 13 shows the operating cycles of these appliances using power consumption information.

It can be observed from Fig. 13 that the power curves in these two plots are nearly the same, except that the ground-truth version has transient information. Because the proposed FHMM-KF method can determine the correct state number of individual appliances and the corresponding steady-state power, there is no significant difference between the ground truth and estimated power. Thereafter, the performance of the FHMM-KF method was compared with those of the FHMM-KFNT, FHMM-KFNS, and SIQCP methods in terms of power consumption.

(12)Ratio \text{=}\frac{\sum _{i=1}^N{\hat{y}}_i}{\sum _{i=1}^N{y}_i}.

The normalized power consumption, which is the ratio of the estimated power consumption of each appliance to the ground-truth power consumption in Eq. (12), using different methods on LIFTED is shown in Fig. 14, and the detailed data are listed in Table 2. In Eq. (12), {\hat{y}}_i and y_i are the estimated and ground-truth power consumptions of ith appliance, respectively. For instance, the ratio for kettle with the FHMM-KFNT method is 0.76, which means that the estimated power consumption of the kettle is approximately 76% of the actual power consumption. As discussed before, the kettle and vacuum are easily mistaken when transient features are not considered. In the same case, the ratio for vacuum is approximately 1.19, which means that the estimated power consumption of the vacuum is larger than the actual consumption.

The ratio of the power consumption is similar to that of the accuracy and f₁-score. For each appliance, the power consumption ratio is closer to 1.0 if the accuracy is higher. Among all the methods, only the FHMM-KF method considers transient and steady-state features; thus, it should achieve the best result. Figure 14 shows that the ratio of the power consumption of the FHMM-KF is closer to 1.0. Therefore, we can conclude that the FHMM-KF method is superior to the other methods.

The ratios of the FHMM-KFNT and SIQCP methods are similar because they consider Markov chains and steady-state features rather than transient features. The ideas of these two methods are almost the same. Some ratios of the FHMM-KFNT, FHMM-KFNS, and SIQCP methods deviate far from 1.0, which verifies the effectiveness of considering transient and steady-state features in NILM.

4.6 Computational time validation

The proposed method includes two steps: the first step infers the state-changing appliance and verifies the results in the second step. The inference in the first step is triggered only when an event occurs. If there is no event, the proposed method checks whether the difference between the composite load and the total estimated power consumption is larger than a threshold. The computation time of the first step is relatively low because appliances do not frequently change their states. The computation in the second step is very efficient; therefore, it does not take much time. As a result, the overall computational time of this method is limited.

However, for other methods, specifically the non-event-based method such as SIQCP, it continuously infers the appliance that is changing states even if there is a small power change, which significantly increases the computational time. A comparison of the computational times between the FHMM-KF and SIQCP methods is shown in Fig. 15. It can be observed that as the appliance number increases, the computational time of the SIQCP method increases exponentially. Notably, the order in which the appliances are added is kettle, vacuum, steamer, hair dryer, refrigerator, toaster, hotpot, mixer, blender, and washing machine. This is because more appliances cause more fluctuations and increase the computational time. As shown in Fig. 15, the tenth appliance is a washing machine with high power fluctuations, which significantly increases the computational time. The proposed FHMM-KF method only infers changing states when an event occurs; thus, the computational time is almost linear to the number of events.

A closer view of the computational time of the FHMM-KF method is shown in Fig. 16. It can be observed that when the number of appliances increases, the computational time of the FHMM-KF method increases linearly, and it is much shorter than that of the SIQCP method. This proves the effectiveness of the event-based NILM method that takes advantage of transients.

4.7 Comparison with nonintrusive load monitoring toolkit (NILMTK)

NILMTK provides an open-source toolkit for comparing different types of energy disaggregation algorithms in a reproducible manner [24]. It can be used to compare multiple disaggregation approaches on different data sets. Here, the LIFTED data set is converted to the format of NILMTK, and we compared the proposed method with two methods in NILMTK, including the combinatorial optimization (CO) [4] method and the Exact FHMM (FHMM_EXACT) method.

The CO method involves finding the combination of appliance states that minimizes the difference between the sum of the estimated appliance power and the ground-truth aggregate power. The FHMM_EXACT method models each appliance using HMM and models a household as FHMM that is similar to the proposed FHMM-KF method in this study. However, the proposed FHMM-KF method integrates the transient features and Kalman filter into the mathematical model; thus, it can identify each appliance state and estimate their power consumption with more precision.

Table 3 compares the root mean square error (RMSE), one of the error metrics in NILMTK, CO, FHMM_EXACT, and FHMM-KF methods. The RMSE of CO and FHMM_EXACT for vacuum is slightly better than that of the FHMM-KF method, which could because the FHMM-KF method calculates the mean of steady-state power only using steady-state observations, but the other two methods use the entire data including transients, which reduces the effect of high fluctuation in the case of the vacuum. The FHMM-KF method outperforms the other two methods for refrigerators, washing machines, and blenders. This is because the state inferred using the FHMM-KF method is closer to the ground truth as it decides the state number based on the observations, whereas the other two methods have to preset the state number before training on the observations.

As summarized in Table 3, the average RMSE on 10 appliances is 15.9 W, 103.9 W, and 112.7 W using the FHMM-KF, CO, and FHMM_EXACT methods, respectively. Thus, the FHMM-KF method is more robust than the CO and FHMM_EXACT methods, although the performance of the latter on some appliances such as vacuum is better than that of the proposed FHMM-KF method.

5 Conclusion and future work

In this study, FHMM-KF was proposed to identify the appliance that is changing state(s) in the transition process. Thereafter, the system super state was used to ensure that the inference results were correct. To achieve the FHMM-KF model, the study covers event detection, feature extraction, and modeling of each appliance to build an appliance model library.

Transient feature is an important factor in identifying different types of appliances, such as heating and motor-driven appliances. More features would make it easier to identify similar appliances with more accurate results. Therefore, in future work, feature extraction or feature engineering is one of the most important tasks to improve the performance of the proposed method further.

Although the FHMM-KF method achieves high accuracy and an f₁-score that is nearly 100% in the test case and it outperforms the FHMM-KFNT, FHMM-KFNS, and SIQCP methods, it may not be suitable for low sampling rate-data sets without a significant transient process. When multiple appliances change states at the same time, it is difficult to identify each appliance at a high accuracy. These should be the subjects of future research.

6 Appendix

Tables A1, A2, and A3 in Section 4 are listed as follows:

References

[1] Ghosh S, Chatterjee A, Chatterjee D. Improved non-intrusive identification technique of electrical appliances for a smart residential system. IET Generation, Transmission & Distribution, 2019, 13( 5): 695– 702

[2] Luo F, Ranzi G, Kong W. . Nonintrusive energy saving appliance recommender system for smart grid residential users. IET Generation, Transmission & Distribution, 2017, 11( 7): 1786– 1793

[3] Carrie Armel K, Gupta A, Shrimali G. . Is disaggregation the holy grail of energy efficiency? The case of electricity. Energy Policy, 2013, 52 : 213– 234

[4] Hart G W. Nonintrusive appliance load monitoring. Proceedings of the IEEE, 1992, 80( 12): 1870– 1891

[5] Kong W, Dong Z Y, Ma J. . An extensible approach for non-intrusive load disaggregation with smart meter data. IEEE Transactions on Smart Grid, 2018, 9( 4): 3362– 3372

[6] Guo Z, Wang Z J, Kashani A. Home appliance load modeling from aggregated smart meter data. IEEE Transactions on Power Systems, 2015, 30( 1): 254– 262

[7] Makonin S, Popowich F, Bajić I V. . Exploiting HMM sparsity to perform online real-time nonintrusive load monitoring. IEEE Transactions on Smart Grid, 2016, 7( 6): 2575– 2585

[8] Sadeghianpourhamami N, Ruyssinck J, Deschrijver D. . Comprehensive feature selection for appliance classification in NILM. Energy and Building, 2017, 151 : 98– 106

[9] Pulipaka S, Ramesh A, Kumar R. . Non-intrusive real-time monitoring of PV generation at inverters using Internet of photovoltaics. Electronics Letters, 2017, 53( 16): 1137– 1138

[10] BritoR, Wong M C, ZhangH CInstantaneous active and reactive load signature applied in non-intrusive load monitoring systems. IET Smart Grid, 2021, 4(1): 121–133

[11] YanL, HanJ, LiZAn online transient-based electrical appliance state tracking method via Markov chain Monte Carlo sampling. In: 2020 IEEE Power and Energy Society General Meeting (PESGM), Montreal, Canada, 2020

[12] KellyJ, Knottenbelt W.Neural NILM: deep neural networks applied to energy disaggregation. In: Proceedings of the 2nd ACM International Conference on Embedded Systems for Energy-efficient Built Environments, Seoul, South Korea, 2015

[13] KaselimiM, Doulamis N, DoulamisABayesian-optimized bidirectional LSTM regression model for non-intrusive load monitoring. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, 2019

[14] ZeifmanM, Roth K. Viterbi algorithm with sparse transitions (VAST) for nonintrusive load monitoring. In: 2011 IEEE Symposium on Computational Intelligence Applications in Smart Grid (CIASG), Paris, France, 2011

[15] Gustafsson F. Particle filter theory and practice with positioning applications. IEEE Aerospace and Electronic Systems Magazine, 2010, 25( 7): 53– 82

[16] Breschi V, Piga D, Bemporad A. Kalman filtering for energy disaggregation. IFAC-PapersOnLine, 2018, 51( 5): 108– 113

[17] Wu X, Han X, Liang K X. Event-based non-intrusive load identification algorithm for residential loads combined with underdetermined decomposition and characteristic filtering. IET Generation, Transmission & Distribution, 2019, 13( 1): 99– 107

[18] Lu M, Li Z. A hybrid event detection approach for non-intrusive load monitoring. IEEE Transactions on Smart Grid, 2020, 11( 1): 528– 540

[19] Li Z, Luo K. Research on adaptive parameters determination in DBSCAN algorithm. Computer Engineering and Applications, 2016, 52( 3): 70– 73

[20] Krishna K, Narasimha Murty M. Genetic K-means algorithm. IEEE Transactions on Systems, Man, and Cybernetics. Part B, Cybernetics, 1999, 29( 3): 433– 439

[21] LinY H, Tsai M S, ChenC S.Applications of fuzzy classification with fuzzy c-means clustering and optimization strategies for load identification in NILM systems. In: 2011 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011), Taipei, China, 2011

[22] SharmaA, Sharma A.KNN-DBSCAN: Using k-nearest neighbor information for parameter-free density based clustering. In: 2017 International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT), Kerala, India, 2017

[23] YanL, HanJ, LiZLIFTED: household appliance-level load dataset and data compression with lossless coding considering precision. In: 2020 IEEE Power and Energy Society General Meeting (PESGM), Montreal, Canada, 2020

[24] BatraN, Kelly J, ParsonONILMTK: an open-source toolkit for non-intrusive load monitoring. In: Proceedings of the 5th International Conference on Future Energy Systems, Cambridge, UK, 2014

RIGHTS & PERMISSIONS

Higher Education Press 2022