Least squares weighted twin support vector machines with local information

Abstract

A least squares version of the recently proposed weighted twin support vector machine with local information (WLTSVM) for binary classification is formulated. This formulation leads to an extremely simple and fast algorithm, called least squares weighted twin support vector machine with local information (LSWLTSVM), for generating binary classifiers based on two non-parallel hyperplanes. Two modified primal problems of WLTSVM are attempted to solve, instead of two dual problems usually solved. The solution of the two modified problems reduces to solving just two systems of linear equations as opposed to solving two quadratic programming problems along with two systems of linear equations in WLTSVM. Moreover, two extra modifications were proposed in LSWLTSVM to improve the generalization capability. One is that a hot kernel function, not the simple-minded definition in WLTSVM, is used to define the weight matrix of adjacency graph, which ensures that the underlying similarity information between any pair of data points in the same class can be fully reflected. The other is that the weight for each point in the contrary class is considered in constructing equality constraints, which makes LSWLTSVM less sensitive to noise points than WLTSVM. Experimental results indicate that LSWLTSVM has comparable classification accuracy to that of WLTSVM but with remarkably less computational time.

Keywords

least squares / similarity information / hot kernel function / noise points

References

[1] MangasarianO L, WildE. Multisur face proximal support vector machine classification via generalized eigenvalues [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(1): 69-74

[2] FungG, MangasarianO L. Proximal support vector machine classifiers [C]. Proceedings KDD-2001: Knowledge Discovery and Data Mining, 2001New YorkACM Press77-86

[3] Jayadeva, KhemchandaiR, ChandraS. Twin support vector machines for pattern classification [J]. IEEE Transaction on Pattern Analysis and Machine Intelligence, 2007, 29(5): 905-910

[4] KumarM A, GopalM. Least squares twin support vector machines for pattern classification [J]. Expert Systems with Applications, 2009, 36(4): 7535-7543

[5] YeQ-l, ZhaoC-x, YeN, ChenX-bo. Localized twin SVM via convex minimization [J]. Neurocomputing, 2011, 74(4): 580-587

[6] PengX-jun. TPMSVM: A novel twin parametric-margin support vector machine for pattern recognition [J]. Pattern Recognition, 2011, 44(10/11): 2678-2692

[7] QiZ-q, TianY-j, ShiYong. Structural twin support vector machine for classification [J]. Knowledge-Based Systems, 2013, 43(5): 74-81

[8] PengX-j, XuDong. Twin mahalanobis distance-based support vector machines for pattern recognition [J]. Information Science, 2012, 200(10): 22-37

[9] ShaoY-h, ZhangC-h, WangX-b, DengN-yang. Improvements on twin support vector machines [J]. IEEE Transactions on Neural Networks, 2011, 22(6): 962-968

[10] YeQ-l, ZhaoC-x, YeN, ChenY-nan. Multi-weight vector projection support vector machines [J]. Pattern Recognition Letters, 2010, 31(13): 2006-2011

[11] ChenX-b, YangJ, YeQ-l, LiangJun. Recursive projection twin support vector machine via within-class variance minimization [J]. Pattern Recognition, 2011, 44(10): 2643-2655

[12] ShaoY-h, WangZ, ChenW-j, DengN-yang. A regularization for the projection twin support vector machine [J]. Knowledge-Based Systems, 2013, 37: 203-210

[13] YeQ-l, ZhaoC-x, GaoS-b, ZhengHao. Weighted twin support vector machines with local information and its application [J]. Neural Networks, 2012, 35(11): 31-39

[14] HeX-f, NiyogiPLocality preserving projection [OL], 2012

[15] WangX-m, ChungF-l, WangS-tong. On minimum class locality preserving variance support vector machine [J]. Patter Recognition, 2010, 43(8): 2753-2762

[16] HuaX-p, DingS-fei. Locality preserving twin support vector machines [J]. Journal of Computer Research and Development, 2014, 51(3): 590-597

[17] BlakeC, MerzCUCI repository of machine learning databases [OL], 1998

[18] MusicantD RNDC: Normally distributed clustered datasets [OL], 1998

[19] ShaoY-h, DengN-y, YangZ-m, ChenW-j, WangZhen. Probabilistic outputs for twin support vector machines [J]. Knowledge-Based Systems, 2012, 33(9): 145-151

[20] XueH, ChenS-can. Glocalization pursuit support vector machine [J]. Neural Computing and Applications, 2011, 20(7): 1043-1053

[21] DingL-z, LiaoS-zhong. KMA-a: A kernel matrix approximation algorithm for support vector machines [J]. Journal of Computer Research and Development, 2012, 49(4): 746-753

[22] PengX-j, XuDong. Norm-mixed twin support vector machine classifier and its geometric algorithm [J]. Neurocomputing, 2013, 99(1): 486-495