Quasi-likelihood Ratio Test for Structural Changes in Vector–Tensor Regression Model

Abstract

In this paper, we investigate a test for parameter instability with an unknown change point in the low-rank tensor regression model. Assuming the tensor coefficient admits the CP decomposition, we embed the low-rank structure into the tensor regression, ensuring structural preservation and computational feasibility. To estimate the parameters, we construct the full-sample and partial-sample OLS estimators and prove their consistency under the null hypothesis. Based on these, we propose the quasi-likelihood ratio test statistic for detecting non-stationarity and derive its asymptotic distribution under the null hypothesis as the supremum of the square of a standardized tied-down Bessel process. Additionally, we explore the asymptotic local power under local alternatives, demonstrating that our test is a powerful tool for structural change detection. The results of simulation studies and empirical applications confirm the efficiency of the proposed method.

Keywords

Tensor regression / Quasi-Likelihood ratio test / Change-point detection / CP decomposition / Bessel process / 62F03 / 62F05

References

[1] Andrews, D.W.: Tests for parameter instability and structural change with unknown change point. Econom. J. Econom. Soc. 821–856 (1993)

[2] Andrews DW, Fair RC. Inference in nonlinear econometric models with structural change. Rev. Econ. Stud., 1988, 55(4): 615-640

[3] Bai J, Lumsdaine RL, Stock JH. Testing for and dating common breaks in multivariate time series. Rev. Econ. Stud., 1998, 65(3): 395-432

[4] Bi X, Qu A, Shen X. Multilayer tensor factorization with applications to recommender systems. Ann. Stat., 2018, 46(6B): 3308-3333

[5] Bulat, A., Kossaifi, J., Tzimiropoulos, G., Pantic, M.: Incremental multi-domain learning with network latent tensor factorization. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 10470–10477 (2020)

[6] Chen R, Yang D, Zhang C-H. Factor models for high-dimensional tensor time series. J. Am. Stat. Assoc., 2022, 117(537): 94-116

[7] Cho H. Change-point detection in panel data via double cusum statistic. Electron. J. Stat., 2016, 10: 2000-2038

[8] Chow, G.C.: Tests of equality between sets of coefficients in two linear regressions. Econom. J. Econom. Soc. 591–605 (1960)

[9] Cichocki A, Phan A-H, Zhao Q, Lee N, Oseledets I, Sugiyama M, Mandic DP, et al.. Tensor networks for dimensionality reduction and large-scale optimization: Part 2 applications and future perspectives. Foundations and Trends in Machine Learning, 2017, 9(6): 431-673

[10] Cui X, Geng H, Wang Z, Zou C. Robust estimation of high-dimensional linear regression with changepoints. IEEE Trans. Inf. Theory, 2024, 70: 7297-7319

[11] Davies RB. Hypothesis testing when a nuisance parameter is present only under the alternative. Biometrika, 1977, 64(2): 247-254

[12] Donsker MD. An invariance principle for certain probability limit theorems. Ann. Probab., 1951, 6: 1-10

[13] Fanelli G, Dantone M, Gall J, Fossati A, Van Gool L. Random forests for real time 3d face analysis. Int. J. Comput. Vis., 2013, 101: 437-458

[14] Fernandes S, Fanaee-T H, Gama J. Tensor decomposition for analysing time-evolving social networks: an overview. Artif. Intell. Rev., 2021, 54(4): 2891-2916

[15] Guo W, Kotsia I, Patras I. Tensor learning for regression. IEEE Trans. Image Process., 2011, 21(2): 816-827

[16] Hitchcock FL. The expression of a tensor or a polyadic as a sum of products. J. Math. Phys., 1927, 6(1–4): 164-189

[17] Horváth L, Shao Q-M. Limit theorems for the union-intersection test. J. Stat. Plan. Inference, 1995, 44(2): 133-148

[18] Horváth L, Pouliot W, Wang S. Detecting at-most-m changes in linear regression models. J. Time Ser. Anal., 2017, 38(4): 552-590

[19] Joseph L, Wolfson DB. Estimation in multi-path change-point problems. Commun. Stat. Theory Methods, 1992, 21(4): 897-913

[20] Kaandorp ML, Dwight RP. Data-driven modelling of the Reynolds stress tensor using random forests with invariance. Comput. Fluids, 2020, 202 104497

[21] Kanagawa, H., Suzuki, T., Kobayashi, H., Shimizu, N., Tagami, Y.: Gaussian process nonparametric tensor estimator and its minimax optimality. In: International Conference on Machine Learning, pp. 1632–1641 (2016)

[22] Kolda TG, Bader BW. Tensor decompositions and applications. SIAM Rev., 2009, 51(3): 455-500

[23] Kumar Sen P. Asymptotic theory of some tests for a possible change in the regression slope occurring at an unknown time point. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete, 1980, 52(2): 203-218

[24] Lee S, Seo MH, Shin Y. The lasso for high dimensional regression with a possible change point. J. R. Stat. Soc. Ser. B Stat Methodol., 2016, 78(1): 193-210

[25] Li, Q., Jiang, L., Li, P., Chen, H.: Tensor-based learning for predicting stock movements. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 29 (2015)

[26] Li L, Zhang X. Parsimonious tensor response regression. J. Am. Stat. Assoc., 2017, 112(519): 1131-1146

[27] Li X, Xu D, Zhou H, Li L. Tucker tensor regression and neuroimaging analysis. Stat. Biosci., 2018, 10(3): 520-545

[28] Li D, Teng Y, Zhou X, Zhang J, Luo W, Zhao B, Yu Z, Yuan L. A tensor-based approach to unify organization and operation of data for irregular spatio-temporal fields. Int. J. Geogr. Inf. Sci., 2022, 36(9): 1885-1904

[29] Liu, B., Zhang, X., Liu, Y.: Simultaneous change point detection and identification for high dimensional linear models. arXiv preprint (2024) arXiv:2401.08173

[30] Liu Y, Liu J, Zhu C. Low-rank tensor train coefficient array estimation for tensor-on-tensor regression. IEEE Trans. Neural Netw. Learn. Syst., 2020, 31(12): 5402-5411

[31] Luo Y, Zhang AR. Tensor clustering with planted structures: statistical optimality and computational limits. Ann. Stat., 2022, 50(1): 584-613

[32] Mamonov AV, Olshanskii MA. Tensorial parametric model order reduction of nonlinear dynamical systems. SIAM J. Sci. Comput., 2024, 46(3): 1850-1878

[33] Matteson DS, James NA. A nonparametric approach for multiple change point analysis of multivariate data. J. Am. Stat. Assoc., 2014, 109(505): 334-345

[34] Osipov, D., Sun, K.: Tensor decomposition based adaptive model reduction for power system simulation. In: 2020 IEEE Power & Energy Society General Meeting (PESGM), pp. 1–5 (2020)

[35] Page ES. Continuous inspection schemes. Biometrika, 1954, 41(2): 100-115

[36] Qu Z, Perron P. Estimating and testing structural changes in multivariate regressions. Econometrica, 2007, 75(2): 459-502

[37] Quandt RE. The estimation of the parameters of a linear regression system obeying two separate regimes. J. Am. Stat. Assoc., 1958, 53(284): 873-880

[38] Quandt RE. Tests of the hypothesis that a linear regression system obeys two separate regimes. J. Am. Stat. Assoc., 1960, 55(290): 324-330

[39] Shao, R., Zheng, Z., Tu, H., Liu, B., Zhang, H., Liu, Y.: Tensor4d: Efficient neural 4d decomposition for high-fidelity dynamic reconstruction and rendering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16632–16642 (2023)

[40] Shi Q, Cheung Y-M, Zhao Q, Lu H. Feature extraction for incomplete data via low-rank tensor decomposition with feature regularization. IEEE Trans. Neural Netw. Learn. Syst., 2019, 30(6): 1803-1817

[41] Shimoda K, Nagasaka Y, Chao ZC, Fujii N. Decoding continuous three-dimensional hand trajectories from epidural electrocorticographic signals in Japanese macaques. J. Neural Eng., 2012, 9(3 036015

[42] Si, Y., Zhang, Y., Cai, Y., Liu, C., Li, G.: An efficient tensor regression for high-dimensional data. arXiv preprint (2024). arXiv:2205.13734

[43] Sidiropoulos ND, Bro R. On the uniqueness of multilinear decomposition of n-way arrays. J. Chemom. A J. Chemom. Soc., 2000, 14(3): 229-239

[44] Sidiropoulos ND, De Lathauwer L, Fu X, Huang K, Papalexakis EE, Faloutsos C. Tensor decomposition for signal processing and machine learning. IEEE Trans. Signal Process., 2017, 65(13): 3551-3582

[45] Sun WW, Li L. Store: sparse tensor response regression and neuroimaging analysis. J. Mach. Learn. Res., 2017, 18(135): 1-37

[46] Wang, D., Zhao, Z.: Optimal change-point testing for high-dimensional linear models with temporal dependence. arXiv preprint (2022) arXiv:2205.03880

[47] Xie, Y., Siegmund, D.: Sequential multi-sensor change-point detection. In: 2013 Information Theory and Applications Workshop (ITA), pp. 1–20 (2013)

[48] Xu X, Cao X, Yu L. Carbon emissions forecasting based on tensor decomposition with multi-source data fusion. Inf. Sci., 2024, 681 121235

[49] Yu, R., Liu, Y.: Learning from multiway data: Simple and efficient tensor regression. In: International Conference on Machine Learning, pp. 373–381 (2016)

[50] Zhang, B., Geng, J., Lai, L.: Change-point estimation in high dimensional linear regression models via sparse group lasso. In: 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 815–821 (2015)

[51] Zhao Q, Caiafa CF, Mandic DP, Chao ZC, Nagasaka Y, Fujii N, Zhang L, Cichocki A. Higher order partial least squares (HOPLS): a generalized multilinear regression method. IEEE Trans. Pattern Anal. Mach. Intell., 2012, 35(7): 1660-1673

[52] Zhou H, Li L, Zhu H. Tensor regression with applications in neuroimaging data analysis. J. Am. Stat. Assoc., 2013, 108(502): 540-552

Funding

National Natural Science Foundation of China(No. 12171391)

Shaanxi Fundamental Science Research Project for Mathematics and Physics(No. 23JSY043)

RIGHTS & PERMISSIONS

School of Mathematical Sciences, University of Science and Technology of China and Springer-Verlag GmbH Germany, part of Springer Nature

Just Accepted

This article has successfully passed peer review and final editorial review, and will soon enter typesetting, proofreading and other publishing processes. The currently displayed version is the accepted final manuscript. The officially published version will be updated with format, DOI and citation information upon launch. We recommend that you pay attention to subsequent journal notifications and preferentially cite the officially published version. Thank you for your support and cooperation.