Consistency of weighted feature set and polyspectral kernels in individual communication transmitter identification

Abstract

This paper presents a method using support vector machine with polyspectral kernels for classification of individual transmitters. Then, the neighborhood-rough-set-based weighted feature set is proposed. The experiments of the algorithms mentioned above indicate that they have consistency, which raises a new weighted kernel. The experiment shows that better classification rate can be achieved.

Keywords

polyspectral kernel / support vector machine (SVM) / neighborhood rough set / weighted feature set / weighted kernel

Introduction

On the difficulty for finding an optimal features’ set of individual communication transmitters because of the faint discrepancies, it is extremely hard to highly classify the transmitters especially when they are with the same model and manufacturing lot. The features of “turn-on” transient signals are usually used for classifying the transmitters [1-3] because they have unique characteristics. These have achieved good recognition rate, however, the features are raw and unstable to be widely applicable.

Then, researchers paid more attention to using stable characteristics of transmitters instead of transient features. There is a consensus that higher order spectra (polyspectra) of signal plays an important role in the analysis of subtle characteristics of transmitters. The major problem of polyspectra is the computation complexity caused by the increase of order. Even the calculus of the lowest order polyspectra (bispectra) is computationally expensive. Researchers are working at transforming the two-dimensional bispectrum into a one-dimensional template [4-6]. In nature, these algorithms are for feature extraction, which is, including linear or nonlinear mapping technique, high-dimensional data are projected toward an output space with dimension equal to the intrinsic dimension. Via feature extraction, the original physical significance is destroyed, and it is disadvantageous to the classification. Feature selection is another method. It chooses a subset of input variables by eliminating features with little or no predictive information, therefore physical significance can remain. The major problem of feature selection is how to evaluate the remaining variables in subset.

In this paper, we construct polyspectral kernels which are applied in support vector machines (SVMs). Meanwhile, the neighborhood-rough-set-based weighted feature subset is proposed. We design a simulation for analyzing the consistency of weighted feature set and polyspectrum kernels.

The rest of this paper is organized as follows: We define polyspectral kernels in support vector machines in Sect. 2, and the performance of the polyspectral kernels is investigated. In Sect. 3, the method of evaluating the features using the parameter “significances” is proposed. Then in Sect. 4, the performance of polyspectral kernels is compared with the performance of neighborhood-rough-set-based weighted feature subset. In Sect. 5, a weighted kernel is constructed from the algorithms mentioned above. Finally, the conclusions are made in Sect. 6.

Polyspectral kernels in SVMs

Definition of polyspectra

Let x(t) be a real stationary signal. The kth order moment of x(t) is defined as

(1)$$\begin{gathered} m_{kx}({\tau }_{\mathrm{1}},{\tau }_{\mathrm{2}},\cdots ,{\tau }_{k-\mathrm{1}}) \\ =\mathrm{E}\{x(t)x(t+{\tau }_{\mathrm{1}})\cdots x(t+{\tau }_{k-\mathrm{1}})\} \\ ={\displaystyle \underset{-\infty }{\overset{\infty }{\int }}x(t)x(t+{\tau }_{\mathrm{1}})\cdots x(t+{\tau }_{k-\mathrm{1}})\mathrm{d}t}. \end{gathered}$$

m_{kx}({\tau }_{\mathrm{1}},{\tau }_{\mathrm{2}},\cdots ,{\tau }_{k-\mathrm{1}}) is the kth order correlation function of x(t) where k\ge \mathrm{2}.

The kth order moment spectra of x(t) is the discrete multi-dimensional Fourier transform of the kth order moment. It is defined as

(2)$$\begin{gathered} M_{kx}({\omega }_{\mathrm{1}},{\omega }_{\mathrm{2}},\cdots ,{\omega }_{k-\mathrm{1}}) \\ ={\displaystyle \sum _{{\tau }_{\mathrm{1}}=-\infty }^{\infty }\cdots }{\displaystyle \sum _{{\tau }_{k-\mathrm{1}}=-\infty }^{\infty }{m}_{kx}({\tau }_{\mathrm{1}},{\tau }_{\mathrm{2}},\cdots ,{\tau }_{k-\mathrm{1}})}{\mathrm{e}}^{-\mathrm{j}{\displaystyle \sum _{i=\mathrm{1}}^{k-\mathrm{1}}{\omega }_i{\tau }_i}} \\ ={\displaystyle \sum _{{\tau }_{\mathrm{1}}=-\infty }^{\infty }\cdots }{\displaystyle \sum _{{\tau }_{k-\mathrm{1}}=-\infty }^{\infty }{\displaystyle {\int }_{-\infty }^{\infty }x(t){\displaystyle \prod _{n=\mathrm{1}}^{k-\mathrm{1}}x(t+{\tau }_n)}{\mathrm{e}}^{-\mathrm{j}{\displaystyle \sum _{i=\mathrm{1}}^{k-\mathrm{1}}{\omega }_i{\tau }_i}}\mathrm{d}t.}} \end{gathered}$$

The polyspectrum for k=2 is the power spectrum, for k=3 it is called bispectrum and for k=4 it is called trispectrum.

In the paper, the kth order moment spectrum is used as polyspectrum, although the higher-order moment spectra and higher-order cumulants spectra constitute the higher-order spectra (polyspectra).

Polyspectral kernels

The n-order polyspectral characteristics reflect n-1 order frequencies components; therefore, they take more information of the transmitters. However, n-order polyspectra is much more computationally expensive than n-1 order polyspectra. Moreover, the computation of higher dimensionality is much more expensive. It is difficult to be utilized due to the computational complexity of polyspectra and high dimensionality of the feature space caused by the orders of the polyspectra growing higher. To address this problem we present an algorithm of using polyspectral kernels in SVMs. The computation of polyspectra can be avoided; the inner product of polyspectra will be used, which reduces the computational complexity.

SVM is a popular technique for pattern recognition and is considered as the state-of-the-art tool for linear and nonlinear classification [7]. Some conventional kernels are usually mentioned, such as the radial-basis function (RBF) kernel and polynomial kernel [8]. The kernel function is defined as

(3)K(\mathit{x},\mathit{y})={\displaystyle \sum _i\mathrm{\Phi }{(\mathit{x})}_i}\mathrm{\Phi }{(\mathit{y})}_i,

where \mathrm{\Phi } is a mapping function. Computation of K(\mathit{x},\mathit{y}) is usually much less than evaluating \mathrm{\Phi }, thus it allows us to construct hyperplanes more easily in very high dimensional spaces. In this paper, the new polyspectral kernel is defined as a simple mathematical form; moreover, the computational complexity is reduced. In the same experimental environment, experiments show that calculation speed increases at least one order of magnitude when the order of polyspectra is low. The higher the order grows, the more obvious the advantage of polyspectral kernel takes on.

By defining {\mathbf{\Omega }}_{k-\mathrm{1}}={({\mathit{\omega }}_{\mathrm{1}}^{\mathrm{T}}{\mathit{\omega }}_{\mathrm{2}}^{\mathrm{T}}\cdots {\omega }_{k-\mathrm{1}}^{\mathrm{T}})}^{\mathrm{T}}, {\mathit{T}}_{k-\mathrm{1}}={({\tau }_{\mathrm{1}}^{\mathrm{T}}{\tau }_{\mathrm{2}}^{\mathrm{T}}\cdots {\tau }_{k-\mathrm{1}}^{\mathrm{T}})}^{\mathrm{T}}, the kth polyspectral kernel is defined as the following:

(4)$$\begin{gathered} K(\mathit{x},\mathit{y})=\langle M_{kx}({\mathbf{\Omega }}_{k-\mathrm{1}}),M_{ky}({\mathbf{\Omega }}_{k-\mathrm{1}})\rangle \\ ={\displaystyle \int \cdots }{\displaystyle \int M_{kx}({\mathbf{\Omega }}_{k-\mathrm{1}})\times M_{ky}^{*}({\mathbf{\Omega }}_{k-\mathrm{1}})\mathrm{d}{\mathit{\omega }}_{\mathrm{1}}\cdots d{\mathit{\omega }}_{k-\mathrm{1}}} \\ =\langle {\displaystyle \sum _{{\tau }_{\mathrm{1}}=-\infty }^{\infty }\cdots }{\displaystyle \sum _{{\tau }_{k-\mathrm{1}}=-\infty }^{\infty }{\displaystyle {\int }_{-\infty }^{\infty }x(t){\displaystyle \prod _{n=\mathrm{1}}^{k-\mathrm{1}}x(t+{\tau }_n)}{\mathrm{e}}^{-\mathrm{j}{\mathbf{\Omega }}_{k-\mathrm{1}}^{\mathrm{T}}{\mathit{T}}_{k-\mathrm{1}}}\mathrm{d}t}}, \\ {\displaystyle \sum _{{\tau }_{\mathrm{1}}=-\infty }^{\infty }\cdots }{\displaystyle \sum _{{\tau }_{k-\mathrm{1}}=-\infty }^{\infty }{\displaystyle {\int }_{-\infty }^{\infty }y(t){\displaystyle \prod _{n=\mathrm{1}}^{k-\mathrm{1}}y(t+{\tau }_n)}{\mathrm{e}}^{-\mathrm{j}{\mathbf{\Omega }}_{k-\mathrm{1}}^{\mathrm{T}}{\mathit{T}}_{k-\mathrm{1}}}\mathrm{d}t}}\rangle \\ ={\displaystyle \sum _{{\omega }_{\mathrm{1}}=-\infty }^{\infty }\cdots }{\displaystyle \sum _{{\omega }_{k-\mathrm{1}}=-\infty }^{\infty }[{\displaystyle \sum _{{\tau }_{\mathrm{1}}=-\infty }^{\infty }\cdots }{\displaystyle \sum _{{\tau }_{k-\mathrm{1}}=-\infty }^{\infty }{\displaystyle {\int }_{-\infty }^{\infty }{\displaystyle {\int }_{-\infty }^{\infty }x(t_{\mathrm{1}})y(t_{\mathrm{2}})}}}} \\ \cdot {\displaystyle \prod _{n=\mathrm{1}}^{k-\mathrm{1}}x(t_{\mathrm{1}}+{\tau }_n)}{\displaystyle \prod _{n=\mathrm{1}}^{k-\mathrm{1}}y(t_{\mathrm{2}}+{\tau }_n)}\cdot {\mathrm{e}}^{-\mathrm{j}{\mathbf{\Omega }}_{k-\mathrm{1}}^{\mathrm{T}}{\mathit{T}}_{k-\mathrm{1}}}\mathrm{d}{t}_{\mathrm{1}}\mathrm{d}{t}_{\mathrm{2}}] \\ ={(\mathrm{2}\mathrm{\pi })}^{\mathrm{2}(k-\mathrm{1})}{{\displaystyle {\int }_{-\infty }^{\infty }\left[{\displaystyle \underset{-\infty }{\overset{\infty }{\int }}x(t)y(t+\tau )\mathrm{d}t}\right]}}^k\mathrm{d}\tau . \end{gathered}$$

It is clear that the polyspectral kernel K(\mathit{x},\mathit{y}) is a function of the correlation of BoldItalic and BoldItalic.

Experiment 1

In order to illustrate the performance of polyspectral kernel, four sets of data are provided, amplitude modulation (AM), frequency modulation (FM), frequency shift keying (FSK), and frequency-hopping spread spectrum (FHSS) samples included. Figure 1 shows that the recognition rates vary with the order of polyspectra for two classes of patterns. It seems that the performance for classification of AM and FM signals are much better than that of FSK and FHSS signals. The result also represents that higher order of polyspectra cannot always bring about higher classification effect. Certainly the recognition rates will decline as the number of classes increases in terms of multi-classification.

Neighborhood-rough-set-based weighted feature set

Neighborhood rough set

Neighborhood rough set (RST) is an approach proposed in this paper for not only feature selection, but also advanced feature evaluation. In RST, a set of all similar objects is called an elementary set, which makes a fundamental atom of knowledge. Any union of elementary sets is called a crisp set and other sets are referred to as rough set [9]. Reference [10] introduced neighborhood relation into rough set methodology.

Let an information system be defined as a function f:U\times A\to V, where U is the set of all objects, A is the set of all attributes, V is the set of all attribute values. A neighborhood decision system is denoted as a 6-tuple \mathrm{N}\mathrm{D}\mathrm{S}=\{U,A,V,f,\delta ,\theta \}, where \delta is the neighborhood measure function and \theta is the threshold. The lower and upper approximations of the decision D with respect to attributes B are denoted as [10]

(5)\underset{¯}{N_B}D={\cup }_{i=\mathrm{1}}^N\underset{¯}{N_B}{X}_i,\overline{N_B}D={\cup }_{i=\mathrm{1}}^N\overline{N_B}{X}_i,

where

(6)\underset{¯}{N_B}X=\{x_i|{\delta }_B(x_i)\subseteq X,x_i\in U\},\overline{N_B}X=\{x_i|{\delta }_B(x_i)\cap X\ne \varnothing ,x_i\in U\}.

Then the decision boundary region of D with respect to attribute B defined as [10]

(7)\mathrm{B}\mathrm{N}(D)=\overline{N_B}X-\underset{¯}{N_B}X.

The dependency degree of D to B [10] is defined as the ratio of consistent objects:

(8){\gamma }_B(D)={\displaystyle \frac{\left|\mathrm{P}\mathrm{O}{\mathrm{S}}_B(D)\right|}{\left|U\right|}},

in which \mathrm{P}\mathrm{O}{\mathrm{S}}_B(D) is called positive region of decision [10].

Given a decision system \langle U,C,D\rangle, B\subseteq C, a\in B, the significance of an attribute is defined as [10]

(9)\mathrm{S}\mathrm{I}\mathrm{G}(a,B,D)={\gamma }_B(D)-{\gamma }_{B-a}(D).

Given a decision system \langle U,C,D\rangle, B\subseteq C, a\notin B, the significance of an attribute is defined as [10]

(10)\mathrm{S}\mathrm{I}\mathrm{G}(a,B,D)={\gamma }_{B\cup a}(D)-{\gamma }_B(D).

In Ref. [11], a forward greedy search algorithm for attribute reduction is proposed, and the minimal feature subset is achieved. More details of how to get the minimal feature subset can be found in Ref. [11].

Neighborhood-rough-set-based weighted feature set

Generally, the method of feature selection aims at how to evaluate the subsets and then grade a subset of features to remove most redundant features from the data. The disadvantage is that after the process of reducing, \mathrm{S}\mathrm{I}\mathrm{G}(a,B,D) will usually be abandoned. We think that it could be useful for constructing a more effective subset, that is, it could be utilized to construct a “weighted” feature set which will be used in SVM.

The process will be described as follows:

Step 1 Let the raw feature dataset be a matrix {\mathit{X}}^{\mathrm{T}}=\{{\mathit{x}}_{\mathrm{1}}^{\mathrm{T}},{\mathit{x}}_{\mathrm{2}}^{\mathrm{T}},\cdots ,{\mathit{x}}_N^{\mathrm{T}}\}. BoldItalic is matrix of size N\times M_{\mathrm{1}}, where {\mathit{x}}_i is a feature vector and N is the number of samples.

Step 2 Using the forward greedy search algorithm defined in Ref. [11], the matrix BoldItalic is transformed into a new matrix, which is denoted as {\mathit{R}}_X^{\mathrm{T}}=\{{\mathit{r}}_{\mathrm{1}}^{\mathrm{T}},{\mathit{r}}_{\mathrm{2}}^{\mathrm{T}},\cdots ,{\mathit{r}}_N^{\mathrm{T}}\}. {\mathit{R}}_X is a matrix of size N\times M_{\mathrm{2}}, in which M_{\mathrm{2}}\le M_{\mathrm{1}}, means that the feature set has been reduced after being transformed by the algorithm.

Step 3 In Step 2, the significance of an attribute has been calculated, and r_{i\mathrm{1}}, r_{i\mathrm{2}}, \cdots, r_{i{M}_{\mathrm{2}}}, i=\mathrm{1},\mathrm{2},\cdots ,N, are descending orderly sequences ordered by their significance. Take the record of the significances of every attribute which is selected into the feature subset. Denote the significances as r_{\mathrm{1}}, r_{\mathrm{2}}, \cdots, r_{M_{\mathrm{2}}}, {\gamma }_{\mathrm{1}}\ge {\gamma }_{\mathrm{2}}\ge \cdots \ge {\gamma }_{M_{\mathrm{2}}}.

Step 4 Normalize the reduced dataset {\mathit{R}}_X. Let the normalization of {\mathit{R}}_X be

(11)z_{i,j}={\displaystyle \frac{r_{ij}-{\overline{r}}_j}{s_j}},i=\mathrm{1},\mathrm{2},\cdots ,N,j=\mathrm{1},\mathrm{2},\cdots ,M_{\mathrm{2}},

where

(12){\overline{r}}_j={\displaystyle \frac{\mathrm{1}}{N}}{\displaystyle \sum _{i=\mathrm{1}}^N{r}_{ij}},s_j^{\mathrm{2}}={\displaystyle \frac{\mathrm{1}}{N}}{\displaystyle \sum _{i=\mathrm{1}}^N(r_{ij}-{\overline{r}}_j{)}^2}.

Step 5 In standard SVMs [8], the decision function is obtained as

(13)f(\mathit{x})=\mathrm{sgn}\left[{\displaystyle \sum _{i=\mathrm{1}}^l{y}_i{\alpha }_{i}K({\mathit{x}}_i,\mathit{x})}+b\right],

where K is the kernel function.

Define \mathit{A}=diag({\gamma }_{\mathrm{1}},{\gamma }_{\mathrm{2}},\cdots ,{\gamma }_{M_{\mathrm{2}}}), then the decision function is rewritten as

(14)f(\mathit{x})=\mathrm{sgn}\left[{\displaystyle \sum _{i=\mathrm{1}}^l{y}_i{\alpha }_{i}K(\mathit{A}{\mathit{x}}_i,\mathit{A}\mathit{x})}+b\right].

Experiments for consistency of significances and polyspectral kernels

Experiment 2

The data used in Experiment 1 are also applied to the experiment of weighted feature set in this section. The polyspectra with different order of the data are chosen as the attributes and Gaussian kernel is selected for SVM in simulation. A series of neighborhood measure function \mathit{\delta } is adopted for comparison. Take AM samples for example, given that the order of polyspectra are selected from 2 to 20, Table 1 shows the significance of every attribute when different \mathit{\delta } is selected.

Table 1 seems complex and no rules to follow. So we observe the classification rate.

Experiment 3

When weighted features are used, for multi-classifier (class=5), it is shown in Fig. 2 that the AM classification rate can be better and stable if \mathrm{0.2}\le \delta \le \mathrm{0.5}. The result is what we would like, that is, an appropriate value of \mathit{\delta } can make stable and satisfactory performance, rather than best performance. Experiments of FM and FHSS signals are similar, whereas performance of FSK is unsatisfactory just as Experiment 1.

Figure 3 shows the significances of attributes which are the nth polyspectra of AM sample as δ=0.4. Compared to Fig. 1, it can be found that the fluctuations of significances are, in general, consistent with the curve in Fig. 1. The result reveals two points:

1) The method of neighborhood-rough-set-based weighted feature set and the method of polyspectral kernels are both effective to some degree. As Experiment 1 shows, higher order does not always bring about higher performance of recognition.

2) In case the two methods are combined, it will be easier to choose the parameter of polyspectra and neighborhood measure function δ. At least the performance of recognition would not too bad.

To address the issue, an improved algorithm, which can be called weighted kernel, will be proposed in the next section.

Weighted polyspectral kernel

Combined kernel function has been proposed in Ref. [12], which means the combination of different kernel functions by addition and multiplication operators. In this section, we make some improvements on the algorithm of neighborhood-rough-set-based weighted feature set, using combined kernel function, called weighted kernel, which can be described as follows:

1) Assuming that the nth polyspectra are chosen to be features, \mathrm{2}\le n\le N, denote corresponding significances as {\lambda }_{\mathrm{1}},{\lambda }_{\mathrm{2}},\cdots ,{\lambda }_N.

2) The combined kernel function is represented as [13]

(15)K_{\mathrm{c}\mathrm{o}\mathrm{m}\mathrm{b}}={\alpha }_{\mathrm{1}}{K}_{\mathrm{1}}\circ {\alpha }_{\mathrm{2}}{K}_{\mathrm{2}}\circ \cdots \circ {\alpha }_m{K}_m,

where K_i, i=\mathrm{1},\mathrm{2},\cdots ,m, denotes the kernel function, \circ represents the operator between the two kernel functions, which can be addition and multiplication operators. {\alpha }_i is the weighted parameter of the kernel function K_i, {\displaystyle {\sum }_{i=\mathrm{1}}^N{\alpha }_i}=\mathrm{1}.

3) It can be seen that {\displaystyle {\sum }_{i=\mathrm{1}}^N{\lambda }_i}=\mathrm{1}; moreover, the significances have been evaluated. Thus, we would like to construct a weighted kernel as

(16)K_{w\mathrm{1}}={\lambda }_{\mathrm{1}}{K}_{\mathrm{1}}+{\lambda }_{\mathrm{2}}{K}_{\mathrm{2}}+\cdots +{\lambda }_N{K}_N.

Experiment 4

The dataset used in Experiment 1 are also applied. Figure 4 shows the classification rates for AM, FM, FHSS and FSK signals when weighted kernel is utilized. It is implicated that the performances are stable, and the result indicates that the polyspectra feature is inapplicable for recognition of FHSS and FSK signals, especially FSK signals. Besides, in terms of training time, the speed of using weighted polyspectral kernels is much higher than conventional kernels, although the weighted polyspectral kernels look more complex.

Conclusions

In this paper, we construct polyspectral kernels applied in SVMs and discuss the performances of this algorithm for different signals. To address the problem of fluctuating classification rates, we propose a novel method of neighborhood-rough-set-based weighted feature set. The experiment shows how to choose a suitable neighborhood measure function \delta, and it also gives a suggestion of significance of every feature. However, the calculation of conventional kernel in SVMs is harder than polyspectral kernels. Thus, the weighted polyspectral kernel is developed. Although the combined kernel function is more computationally expensive than single polyspectral kernel, it takes better and stable performance of recognition. The work is imperfect with respect to that this method seems inapplicable for FHSS and FSK signals.

References

[1] Toonstra J, Kinsner W. Transient analysis and genetic algorithms for classification. In: Proceedings of IEEE WESCANEX 95: Communications, Power, and Computing. 1995, 2: 432-437

[2] Shaw D, Kinsner W. Multifractal modelling of radio transmitter transients for classification. In: Proceedings of IEEE WESCANEX 97: Communications, Power and Computing. 1997, 306-312

[3] Sun L, Kinsner W, Serinken N. Characterization and feature extraction of transient signals using multifractal measures. In: Proceedings of 1999 IEEE Canadian Conference on Electrical and Computer Engineering. 1999, 2: 781-785

[4] Tugnait J K. Detection of non-Gaussian signals using integrated polyspectrum. IEEE Transactions on Signal Processing, 1994, 42(11) : 3137-3149

[5] Liao X J, Bao Z. Circularly integrated bispectra: novel shift invariant features for high resolution radar target recognition. Electronics Letters, 1998, 34 (19): 1879-1880

[6] Xu S H, Huang B X, Huang Y C, Xu Z G. Identification of individual radio transmitters based on selected surrounding-line integral bispectra. In: Proceedings of the 9th International Conference on Advanced Communication Technology.2007, 2: 1147-1150

[7] Burges C J C. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 1998, 2(2): 121-167

[8] Vapnik V. The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995

[9] Pawlak Z. Rough Sets: Theoretical Aspects of Reasoning about Data. Boston: Kluwer Academic Publishers, 1991

[10] Hu Q H, Yu D R, Xie Z X. Neighborhood classifiers. Expert Systems with Applications, 2008, 34(2): 866-876

[11] Hu Q H, Yu D R, Liu J F, Wu C X. Neighborhood rough set based heterogeneous feature subset selection. Information Sciences, 2008, 178(18): 3577-3594

[12] Nguyen H N, Ohn S Y, Park J, Park K S. Combined kernel function approach in SVM for diagnosis of cancer. Lecture Notes in Computer Science, 2005, 3610: 1017-1026

[13] Tan Y, Wang J. A support vector machine with a hybrid kernel and minimal Vapnik-Chervonenkis dimension. IEEE Transactions on Knowledge and Data Engineering, 2004, 16(4): 385-395

RIGHTS & PERMISSIONS

Higher Education Press and Springer-Verlag Berlin Heidelberg