Multi-view dimensionality reduction via canonical random correlation analysis

Abstract

Canonical correlation analysis (CCA) is one of the most well-known methods to extract features from multiview data and has attracted much attention in recent years. However, classical CCA is unsupervised and does not take discriminant information into account. In this paper, we add discriminant information into CCA by using random cross view correlations between within-class samples and propose a new method for multi-view dimensionality reduction called canonical random correlation analysis (RCA). In RCA, two approaches for randomly generating cross-view correlation samples are developed on the basis of bootstrap technique. Furthermore, kernel RCA (KRCA) is proposed to extract nonlinear correlations between different views. Experiments on several multi-view data sets show the effectiveness of the proposed methods.

Keywords

canonical correlation analysis / discriminant / multi-view / dimensionality reduction

References

[1] Duda R O, Hart P E, Stork D G, Pattern Classification. 2nd ed. New York: Wiley-Interscience, 2000.

[2] Yarowsky D. Unsupervised word sense disambiguation rivaling supervised methods. In: Proceedings of the 33rd Annual Meeting on Association for Computational Lingustics. 1995, 189–196

[3] Xia T, Tao D, Mei T, Zhang Y. Multiview spectral embedding. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2010, 40(6): 1438–1446

[4] Zheng H, Wang M, Li Z. Audio-visual speaker identification with multi-view distance metric learning. In: Proceedings of 17th IEEE International Conference on Image Processing. 2010, 4561–4564

[5] Wang M, Li H, Tao D, Lu K,Wu X. Multimodal graph-based reranking for Web image search. IEEE Transactions on Image Processing, 2012, 21(11): 4649–4661

[6] Yu J, Wang M, Tao D. Semisupervised multiview distance metric learning for cartoon synthesis. IEEE Transactions on Image Processing, 2012, 21(11): 4636–4648

[7] Long B, Philip S Y, Zhang Z. A general model for multiple view unsupervised learning. In: Proceedings of the SIAM International Conference on Data Mining. 2008, 822–833

[8] Han Y, Wu F, Tao D, Zhuang Y, Jiang J. Sparse unsupervised dimensionality reduction for multiple view data. IEEE Transactions on Circuits and Systems for Video Technology. 2012, 22(10): 1485–1496

[9] Xie B, Mu Y, Tao D, Huang K. m-SNE: multiview stochastic neighbor embedding. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2011, 41(4): 1088–1096

[10] Hotelling H. Relation between two sets of variates. Biometrica, 1936, 28: 321–377

[11] Diethe T, Hardoon D R, Shawe-Taylor J. Multiview fisher discriminant analysis. In: Proceedings of NIPS Workshop on Learning from Multiple Sources. 2008

[12] Akaho S. A kernel method for canonical correlation analysis. In: Proceedings of the International Meeting of the Psychometric Society. 2001

[13] Vía J, Santamaría I, Pérez J. A learning algorithm for adaptive canonical correlation analysis of several data sets. Neural Networks. 2007, 20(1): 139–152

[14] Hardoon D R, Szedmak S, Shawe-Taylor J. Canonical correlation analysis: an overview with application to learning methods. Neural Computation, 2004, 16(12): 2639–2664

[15] Yang C, Wang L, Feng J. On feature extraction via kernels. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2008, 38(2): 553–557

[16] Sun T, Chen S. Locality preserving CCA with applications to data visualization and pose estimation. Image and Vision Computing, 2007, 25(5): 531–543

[17] Blaschko M B, Jacquelyn J A, Bartels A, Lampert C H, Gretton A. Semi-supervised kernel canonical correlation analysis with application to human fMRI. Pattern Recognition Letters, 2011, 32(11): 1572–1583

[18] Blaschko M B, Lampert C H, Gretton A. Semi-supervised laplacian regularization of kernel canonical correlation analysis. Lecture Notes in Computer Science, 2008, 5211: 133–145

[19] Golugula A, Lee G, Master S R, Feldman M D, Tomaszewski J E, Speicher D W, Madabhushi A. Supervised regularized canonical correlation analysis: integrating histologic and proteomic measurements for predicting biochemical recurrence following prostate surgery. BMC Bioinformatics, 2011, 12(1): 483

[20] Thum A, Mönchgesang S, Westphal L, Lübken T, Rosahl S, Neumann S, Posch S. Supervised Penalized Canonical Correlation Analysis. 2014, arXiv preprint arXiv:1405.1534

[21] Jing X Y, Hu R M, Zhu Y P, Wu S S, Liang C, Yang J Y. Intra-view and inter-view supervised correlation analysis for multi-view feature learning. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence. 2014

[22] Jing X, Sun J, Yao Y, Sui Z. Supervised and unsupervised face recognition method base on 3CCA. In: Proceedings of International Conference on Automatic Control and Artificial Intelligence. 2012, 2009–2012

[23] Guo S, Ruan Q, Wang Z, Liu S. Facial expression recognition using spectral supervised canonical correlation analysis. Journal of Information Science and Engineering, 2013, 29(5): 907–924

[24] Shelton J A. Semi-supervised subspace learning and application to human functional magnetic brain resonance imaging data. Dissertation for the Doctoral Degree. Oxford: University of Oxford, 2010

[25] Sun T, Chen S, Yang J, Shi P. A novel method of combined feature extraction for recognition. In: Proceedings of the 8th IEEE International Conference on Data Mining. 2008, 1043–1048

[26] Majumdar A, Ward R. Robust classifiers for data reduced via random projections. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2010, 40(5): 1359–1371

[27] Wegelin J A. A survey of partial least squares (PLS) methods, with emphasis on the two-block case. Department of Statistics, University of Washington, Technical Report. 2000, 371

[28] Bourke P. Cross correlation. Auto Correlation–2D Pattern Identification, 1996

[29] Theodoridis S, Koutroumbas K. Pattern Recognition. 3rd ed. New York: Academic Press, 2006

[30] Sun Q, Zeng S, Liu Y, Heng P, Xia D. A new method of feature fusion and its application in image recognition. Journal of Pattern Recognition, 2005, 38(12): 2437–2448

[31] Melzer T, Reiter M, Bischof H. Appearance models based on kernel canonical correlation analysis. Journal of Pattern Recognition, 2003, 36(9): 1961–1971

[32] Shawe-Taylor J, Williams C K I, Cristianini N, Kandola J S. On the eigenspectrum of the gram matrix and the generalization error of kernel-PCA. IEEE Transactions on Information Theory, 2005, 51(7): 2510–2522

[33] Bach F R, Jordan M I. Kernel independent component analysis. Journal of Machine Learning Research, 2002, 3: 1–48

[34] Turk M, Pentland A. Eigenfaces for recognition. Journal of Cognitive Neuro Science, 1991, 3(1): 71–86

[35] Belhumeur P N, Hespanha J P, Kriegman D J. Eigenfaces vs. fisherfaces: recognition using class-specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, 19(7): 711–720

[36] He X, Cai D, Niyogi P. Lplacian score for feature selection. Advances in Neural Information Processing Systems. 2005, 18: 507–514

[37] Cai D, He X, Hu Y, Han J, Huang T. Learning a spatially smooth subspace for face recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2007, 1–7

[38] Ahonen T, Hadid A, Pietikainen M. Face recognition with local binary patterns. In: Proceedings of the 8th European Conference on Computer Vision. 2004, 469–481

[39] Zhang J, Zhang D. A novel ensemble construction method for multiview data using random cross-view correlation between within-class examples. Pattern Recognition, 2011, 44(6): 1162–1171

RIGHTS & PERMISSIONS

Higher Education Press and Springer-Verlag Berlin Heidelberg