Hypergraph regularized multi-view subspace clustering with dual tensor log-determinant

Abstract

The existing multi-view subspace clustering algorithms based on tensor singular value decomposition (t-SVD) predominantly utilize tensor nuclear norm to explore the intra view correlation between views of the same samples, while neglecting the correlation among the samples within different views. Moreover, the tensor nuclear norm is not fully considered as a convex approximation of the tensor rank function. Treating different singular values equally may result in suboptimal tensor representation. A hypergraph regularized multi-view subspace clustering algorithm with dual tensor log-determinant (HRMSC-DTL) was proposed. The algorithm used subspace learning in each view to learn a specific set of affinity matrices, and introduced a non-convex tensor log-determinant function to replace the tensor nuclear norm to better improve global low-rankness. It also introduced hyper-Laplacian regularization to preserve the local geometric structure embedded in the high-dimensional space. Furthermore, it rotated the original tensor and incorporated a dual tensor mechanism to fully exploit the intra view correlation of the original tensor and the inter view correlation of the rotated tensor. At the same time, an alternating direction of multipliers method (ADMM) was also designed to solve non-convex optimization model. Experimental evaluations on seven widely used datasets, along with comparisons to several state-of-the-art algorithms, demonstrated the superiority and effectiveness of the HRMSC-DTL algorithm in terms of clustering performance.

Keywords

multi-view clustering / tensor log-determinant function / subspace learning / hypergraph regularization

References

[1] MURTAGH F, CONTRERAS P. Algorithms for hierarchical clustering: an overview. WIRES Data Mining and Knowledge Discovery, 2012, 2(1): 86-97.

[2] RAM A, SHARMA A, JALAL A S, et al. An enhanced density based spatial clustering of applications with noise//2009 IEEE International Advance Computing Conference, March 6-7, 2009, Patiala, India. New York: IEEE, 2009: 1475-1478.

[3] YANG J C, ZHAO C. Survey on k-means clustering algorithm. Computer Engineering and Applications, 2019, 55(23): 7-14.

[4] YANG S J, LI L, WANG S H, et al. SkeletonNet: a hybrid network with a skeleton-embedding process for multi-view image representation learning. IEEE Transactions on Multimedia, 2019, 21(11): 2916-2929.

[5] YE Y K, LIU X W, YIN J P, et al. Co-regularized kernel k-means for multi-view clustering//2016 23rd International Conference on Pattern Recognition, December 4-8, 2016, Cancun, Mexico. New York: IEEE, 2016: 1583-1588.

[6] LIU X W, DOU Y, YIN J P, et al. Multiple kernel k-means clustering with matrix-induced regularization//Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona, USA. California: AAAI Press, 2016: 1888-1894.

[7] ZHAO Y J, YUN Y, ZHANG X D, et al. Multi-view spectral clustering with adaptive graph learning and tensor schatten p-norm. Neurocomputing, 2022, 468: 257-264.

[8] LI X F, REN Z W, SUN Q S, et al. Auto-weighted tensor schatten p-norm for robust multi-view graph clustering. Pattern Recognition, 2023, 134: 109083.

[9] ZHANG X Q, WANG J, XUE X Q, et al. Confidence level auto-weighting robust multi-view subspace clustering. Neurocomputing, 2022, 475: 38-52.

[10] ZHANG X T, ZHANG X C, LIU H, et al. Multi-task multi-view clustering. IEEE Transactions on Knowledge and Data Engineering, 2016, 28(12): 3324-3338.

[11] CAI B, LU G F, YAO L, et al. High-order manifold regularized multi-view subspace clustering with robust affinity matrices and weighted TNN. Pattern Recognition, 2023, 134: 109067.

[12] YANG Y, WANG H. Multi-view clustering: a survey. Big Data Mining and Analytics, 2018, 1(2): 83-107.

[13] XU C, TAO D C, XU C. A survey on multi-view learning. 2013: 1304.5634.

[14] ELHAMIFAR E, VIDAL R. Sparse subspace clustering: algorithm, theory, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(11): 2765-2781.

[15] LIU G C, LIN Z C, YAN S C, et al. Robust recovery of subspace structures by low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 35(1): 171-184.

[16] BRBIĆ M, KOPRIVA I. Multi-view low-rank sparse subspace clustering. Pattern Recognition, 2018, 73: 247-258.

[17] TANG K W, CAO L Y, ZHANG N, et al. Consistent auto-weighted multi-view subspace clustering. Pattern Analysis and Applications, 2022, 25(4): 879-890.

[18] CAO X C, ZHANG C Q, FU H Z, et al. Diversity-induced multi-view subspace clustering//2015 IEEE Conference on Computer Vision and Pattern Recognition, June 7-12, 2015, Boston, MA. New York: IEEE, 2015: 586-594.

[19] ZHANG C Q, FU H Z, LIU S, et al. Low-rank tensor constrained multiview subspace clustering//2015 IEEE International Conference on Computer Vision, December 7-13, 2015, Santiago, Chile. New York: IEEE, 2015: 1582-1590.

[20] WU J L, LIN Z C, ZHA H B. Essential tensor learning for multi-view spectral clustering. IEEE Transactions on Image Processing, 2019, 28(12): 5910-5922.

[21] XIE Y, TAO D C, ZHANG W S, et al. On unifying multi-view self-representations for clustering by tensor multi-rank minimization. International Journal of Computer Vision, 2018, 126(11): 1157-1179.

[22] KILMER M E, BRAMAN K, HAO N, et al. Third-order tensors as operators on matrices: a theoretical and computational framework with applications in imaging. SIAM Journal on Matrix Analysis and Applications, 2013, 34(1): 148-172.

[23] LU C Y, FENG J S, CHEN Y D, et al. Tensor robust principal component analysis: exact recovery of corrupted low-rank tensors via convex optimization//2016 IEEE Conference on Computer Vision and Pattern Recognition, June 27-30, 2016, Las Vegas, NV, USA. New York: IEEE, 2016: 5249-5257.

[24] YIN M, GAO J B, LIN Z C. Laplacian regularized low-rank representation and its applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(3): 504-517.

[25] KANG Z, PENG C, CHENG Q. Robust subspace clustering via smoothed rank approximation. IEEE Signal Processing Letters, 2015, 22(11): 2088-2092.

[26] YANG M, LUO Q L, LI W, et al. Multiview clustering of images with tensor rank minimization via nonconvex approach. SIAM Journal on Imaging Sciences, 2020, 13(4): 2361-2392.

[27] ZHANG Z M, ELY G, AERON S, et al. Novel methods for multilinear data completion and de-noising based on tensor-SVD//2014 IEEE Conference on Computer Vision and Pattern Recognition, June 23-28, 2014, Columbus, OH, USA. New York: IEEE, 2014: 3842-3849.

[28] ZHANG Y, SUN X, CAI H M, et al. Collaborative embedding learning via tensor integration for multi-view clustering. IEEE Transactions on Emerging Topics in Computational Intelligence, 2024, 8(2): 1841-1852.

[29] XIE Y, ZHANG W S, QU Y Y, et al. Hyper-Laplacian regularized multilinear multiview self-representations for clustering and semisupervised learning. IEEE Transactions on Cybernetics, 2020, 50(2): 572-586.

[30] WANG S Q, CHEN Y Y, ZHANG L N, et al. Hyper-Laplacian regularized nonconvex low-rank representation for multi-view subspace clustering. IEEE Transactions on Signal and Information Processing Over Networks, 2022, 8: 376-388.

[31] NG A Y, JORDAN M I, WEISS Y. On spectral clustering: analysis and an algorithm//Advances in Neural Information Processing Systems 14 (NIPS), December 3-8, 2001, Vancouver, British Columbia, Canada. Cambridge, MA: MIT Press, 2001: 849-856.

[32] LIU J Y, LIU X W, YANG Y X, et al. Multiview subspace clustering via co-training robust data representation. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(10): 5177-5189.

[33] YANG Z Y, XU Q Q, ZHANG W G, et al. Split multiplicative multi-view subspace clustering. IEEE Transactions on Image Processing, 2019, 28(10): 5147-5160.