Optimal locality preserving least square support vector machine

Abstract

In this paper, a novel least square support vector machine (LSSVM), termed as optimal locality preserving LSSVM (OLP-LSSVM) is proposed. By integrating structural risk minimization and locality preserving criterion in a unified framework, the resulting separating hyperplane is not only in accordance with the structural risk minimization principle but also be sensitive to the manifold structure of data points. The proposed model can be solved efficiently by alternating optimization method. Experimental results on several public available benchmark datasets show the feasibility and effectiveness of the proposed method.

Key words： structural risk minimization; locality preserving criterion; support vector machine (SVM); least square classifier

References

1 Burges C J C. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 1998, 2(2): 121-167

2 Cristianini N, Shawe-Taylor J. An Introduction to Support Vector Machines. Cambridge: Cambridge University Press, 2000

3 Vapnik V N. The Nature of Statistical Theory. New York: Springer-Verlag, 1995

4 Shawe-Taylor J, Cristianini N. Kernel Methods for Pattern Analysis. Cambridge: Cambridge University Press, 2004

5 Osuna E, Freund R, Girosi F. An improved training algorithm for support vector machines. In: Proceedings of Neural Networks for Signal Processing VII. 1997, 276-285

6 Platt J C. Fast training of support vector machines using sequential minimal optimization. Advances in Kernel Methods: Support Vector Machines, 1999, 185-208

7 Joachims T. Making large-scale SVM learning practical. Advances in Kernel Methods: Support Vector Machine, 1999, 169-184

8 Chang C C, Lin C J. LIBSVM: a library for support vector machines, 2001. http://www.csie.ntu.edu.tw/∼cjlin

9 Suykens J A K, Vandewalle J. Least squares support vector machine classifiers. Neural Processing Letters, 1999, 9(3): 293-300

10 Suykens J A K, Van Gestel T, De Brabanter J, De Moor B, Vandewall J. Least Squares Support Vector Machines. Singapore: World Scientific, 2002

11 Yeung D S, Wang D, Ng W W Y, Tsang E C C, Wang X. Structured large margin machines: sensitive to data distributions. Machine Learning, 2007, 68(2): 171-200

12 Xue H, Chen S C, Yang Q. Structural support vector machine. In: Proceedings of the 5th International Symposium on Neural Networks. 2007, 501-511

13 Tenenbaum J B, de Silva V, Langford J C. A global geometric framework for nonlinear dimensionality reduction. Science, 2000, 290(5500): 2319-2323

14 Saul L, Roweis S. Think globally, fit locally: unsupervised learning of low dimensional manifolds. Journal of Machine Learning Research, 2003, 4(2): 119-155

15 Belkin M, Niyogi P. Laplacian eigenmaps and spectral techniques for embedding and clustering. Advances in Neural Information Processing Systems, 2001, 585-591

16 Yang J, Yang J Y. Why can LDA be performed in PCA transformed space? Pattern Recognition, 2003, 36(2): 563-566

17 He X F, Niyogi P. Locality preserving projections. Advances on Neural Information Processing Systems, 2003, 135-160

18 Liu W, Chang S F. Robust multi-class transductive learning with graphs. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2009, 381-388

19 Zhang L M, Qiao L S, Chen S C. Graph-optimized locality preserving projections. Pattern Recognition, 2010, 43(6): 1993-2002

20 Mangasarian O L. Nonlinear programming. SIAM, 1998

21 Chung F R K. Spectral Graph Theory. Regional Conference Series inMathematics. AmericanMathematical Society, 1997

22 Muphy P M, Aha D W. UCI Repository of machine learning databases, 1992

23 Michie D D, Spiegelhalter D J, Taylor C C. Machine Learning, Neural and Statistical Classification. Upper Saddle River: Ellis Horwood, 1994. http://archive.ics.uci.edu/ml/machine-learning-databases/statlog/

24 The MOSEK Optimization Tools Version 5.0, Denmark, 2008. http://www.mosek.com