Optimal locality preserving least square support vector machine

Abstract

In this paper, a novel least square support vector machine (LSSVM), termed as optimal locality preserving LSSVM (OLP-LSSVM) is proposed. By integrating structural risk minimization and locality preserving criterion in a unified framework, the resulting separating hyperplane is not only in accordance with the structural risk minimization principle but also be sensitive to the manifold structure of data points. The proposed model can be solved efficiently by alternating optimization method. Experimental results on several public available benchmark datasets show the feasibility and effectiveness of the proposed method.

Keywords

structural risk minimization / locality preserving criterion / support vector machine (SVM) / least square classifier

References

[1] Burges C J C. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 1998, 2(2): 121-167

[2] Cristianini N, Shawe-Taylor J. An Introduction to Support Vector Machines. Cambridge: Cambridge University Press, 2000

[3] Vapnik V N. The Nature of Statistical Theory. New York: Springer-Verlag, 1995

[4] Shawe-Taylor J, Cristianini N. Kernel Methods for Pattern Analysis. Cambridge: Cambridge University Press, 2004

[5] Osuna E, Freund R, Girosi F. An improved training algorithm for support vector machines. In: Proceedings of Neural Networks for Signal Processing VII. 1997, 276-285

[6] Platt J C. Fast training of support vector machines using sequential minimal optimization. Advances in Kernel Methods: Support Vector Machines, 1999, 185-208

[7] Joachims T. Making large-scale SVM learning practical. Advances in Kernel Methods: Support Vector Machine, 1999, 169-184

[8] Chang C C, Lin C J. LIBSVM: a library for support vector machines, 2001. http://www.csie.ntu.edu.tw/∼cjlin

[9] Suykens J A K, Vandewalle J. Least squares support vector machine classifiers. Neural Processing Letters, 1999, 9(3): 293-300

[10] Suykens J A K, Van Gestel T, De Brabanter J, De Moor B, Vandewall J. Least Squares Support Vector Machines. Singapore: World Scientific, 2002

[11] Yeung D S, Wang D, Ng W W Y, Tsang E C C, Wang X. Structured large margin machines: sensitive to data distributions. Machine Learning, 2007, 68(2): 171-200

[12] Xue H, Chen S C, Yang Q. Structural support vector machine. In: Proceedings of the 5th International Symposium on Neural Networks. 2007, 501-511

[13] Tenenbaum J B, de Silva V, Langford J C. A global geometric framework for nonlinear dimensionality reduction. Science, 2000, 290(5500): 2319-2323

[14] Saul L, Roweis S. Think globally, fit locally: unsupervised learning of low dimensional manifolds. Journal of Machine Learning Research, 2003, 4(2): 119-155

[15] Belkin M, Niyogi P. Laplacian eigenmaps and spectral techniques for embedding and clustering. Advances in Neural Information Processing Systems, 2001, 585-591

[16] Yang J, Yang J Y. Why can LDA be performed in PCA transformed space? Pattern Recognition, 2003, 36(2): 563-566

[17] He X F, Niyogi P. Locality preserving projections. Advances on Neural Information Processing Systems, 2003, 135-160

[18] Liu W, Chang S F. Robust multi-class transductive learning with graphs. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2009, 381-388

[19] Zhang L M, Qiao L S, Chen S C. Graph-optimized locality preserving projections. Pattern Recognition, 2010, 43(6): 1993-2002

[20] Mangasarian O L. Nonlinear programming. SIAM, 1998

[21] Chung F R K. Spectral Graph Theory. Regional Conference Series inMathematics. AmericanMathematical Society, 1997

[22] Muphy P M, Aha D W. UCI Repository of machine learning databases, 1992

[23] Michie D D, Spiegelhalter D J, Taylor C C. Machine Learning, Neural and Statistical Classification. Upper Saddle River: Ellis Horwood, 1994. http://archive.ics.uci.edu/ml/machine-learning-databases/statlog/

[24] The MOSEK Optimization Tools Version 5.0, Denmark, 2008. http://www.mosek.com

RIGHTS & PERMISSIONS

2014 Higher Education Press and Springer-Verlag Berlin Heidelberg