A Robust Projection-Based Test for Goodness-of-Fit in Linear Models with a Divergent Number of Covariates

Abstract

We propose a robust projection-based test to check linear regression models when the dimension may be divergent. The proposed test can maintain the type I error rate effectively when outliers are present, achieve dimension reduction as if only a single covariate was present, and inherited the robustness of the M-estimation. The test is shown to be consistent and can detect root-n local alternative hypotheses. We further derive asymptotic distributions of the proposed test under the null hypothesis and analyze asymptotic properties under the local and global alternatives. We evaluate the finite-sample performance via simulation studies and apply the proposed method to analyze a real dataset as an illustration.

Keywords

Consistent test / Curse of dimensionality / Divergent dimension / Empirical process / Integrated condition moment / M-estimation / Projection / Uncountable moments restriction. / 62F35 / 62J20

References

[1] Bai Z, Wu Y. Limiting behavior of M-estimators of regression coefficients in high dimensional linear models I. scale dependent case. J. Multivar. Anal.. 1994, 51(2): 211-239.

[2] Bierens HJ. Consistent model specification tests. J. Econom.. 1982, 20: 105-134.

[3] Chen X, Zhao L. M-methods in Linear Model. 1996, Shanghai, Shanghai Scientific and Technical Publishers

[4] Chiang AP, Beck JS, Yen H-J, Tayeh MK, Scheetz TE, Swiderski RE, Nishimura DY, Braun TA, Kim K-YA, Huang J. Homozygosity mapping with SNP arrays identifies TRIM32, an E3 ubiquitin ligase, as a Bardet–Biedl syndrome gene (BBS11). Proc. Natl. Acad. Sci.. 2006, 103(16): 6287-6292.

[5] Conde-Amboage M, Sánchez-Sellero C, González-Manteiga W. A lack-of-fit test for quantile regression models with high-dimensional covariates. Comput. Stat. Data Anal.. 2015, 88: 128-138.

[6] Dong C, Li G, Feng X. Lack-of-fit tests for quantile regression models. J. R. Stat. Soc.: Ser. B (Stat. Methodol.). 2019, 81(3): 629-648.

[7] Escanciano JC. A consistent diagnostic test for regression models using projections. Economet. Theor.. 2006, 22: 1030-1051.

[8] Escanciano JC. Goodness-of-fit tests for linear and nonlinear time series models. J. Am. Stat. Assoc.. 2006, 101(474): 531-541.

[9] Härdle W, Mammen E. Testing parametric versus nonparametric regression. Ann. Stat.. 1993, 21: 1926-1947.

[10] He X, Shao Q-M. On parameters of increasing dimensions. J. Multivar. Anal.. 2000, 73(1): 120-135.

[11] He X, Zhu L-X. A lack-of-fit test for quantile regression. J. Am. Stat. Assoc.. 2003, 98(464): 1013-1022.

[12] Huang J, Ma SG, Zhang C. Adaptive lasso for sparse high-dimensional regression models. Stat. Sin.. 2008, 18: 1603-1618

[13] Huber PJ. Robust regression: asymptotics, conjectures and Monte Carlo. Ann. Stat.. 1973, 1(5): 799-821.

[14] Ichimura H. Semiparametric least squares (sls) and weighted sls estimation of single-index models. J. Econom.. 1993, 58: 71-120.

[15] Kosorok M. Introduction to Empirical Processes and Semiparametric Inference. 2008, New York, Springer-Verlag.

[16] Liu J, Zhong W, Li R. A selective overview of feature screening for ultrahigh-dimensional data. Sci. China Math.. 2015, 58(10): 1-22.

[17] Ma S, Zhang J, Sun Z, Liang H. Integrated conditional moment test for partially linear single index models incorporating dimension-reduction. Electron. J. Stat.. 2014, 8: 523-542.

[18] Mammen E. Asymptotics with increasing dimension for robust regression with applications to the bootstrap. Ann. Stat.. 1989, 17: 382-400.

[19] Mammen, E.: Bootstrap and wild bootstrap for high dimensional linear models, Annals of Statistics pp. 255–285 (1993)

[20] Maronna RA, Martin RD, Yohai VJ. Robust Statistics, Theory and Methods, Wiley Series in Probility and Statistics. 2006, UK, Wiley

[21] Portnoy S. Asymptotic behavior of M-estimators of p\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$p$$\end{document} regression parameters when p2/n\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$p^2/n$$\end{document} is large. I. consistency. Ann. Stat.. 1984, 12(4): 1298-1309.

[22] Portnoy S. Asymptotic behavior of M-estimators of p regression parameters when p2/n\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$p^2/n$$\end{document} is large; II. Normal approximation. Ann. Stat.. 1985, 13: 1403-1417.

[23] Portnoy S. Asymptotic behavior of likelihood methods for exponential families when the number of parameters tends to infinity. Ann. Stat.. 1988, 16: 356-366.

[24] Scheetz TE, Kim K-YA, Swiderski RE, Philp AR, Braun TA, Knudtson KL, Dorrance AM, DiBona GF, Huang J, Casavant TL, Sheffield VC, Stone EM. Regulation of gene expression in the mammalian eye and its relevance to eye disease. Proc. Natl. Acad. Sci.. 2006, 103(39): 14429-14434.

[25] Shah RD, Bühlmann P. Goodness-of-fit tests for high dimensional linear models. J. R. Stat. Soc.: Ser. B (Stat. Methodol.). 2018, 80(1): 113-135.

[26] Sun Z, Chen F, Liang H, Ruppert D. A projection-based consistent test incorporating dimension-reduction for partially linear models. Stat. Sin.. 2021, 31: 1489-1508

[27] van der Vaart, A.W., Wellner, J.A.: Weak Convergence and Empirical Processes, Springer Series in Statistics, Springer-Verlag, New York (1996)

[28] Welsh AH. On M-processes and M-estimation. Ann. Stat.. 1989, 17(1): 337-361.

[29] Wu CFJ. Jackknife, bootstrap and other resampling methods in regression analysis. Ann. Stat.. 1986, 14: 1261-1343

[30] Zhang Q, Chen F, Wu S, Liang H. A simple yet powerful test for assessing goodness-of-fit of high-dimensional linear models. Stat. Med.. 2021, 40: 3153-3166.

[31] Zhu L. Nonparametric Monte Carlo Tests and Their Applications. 2005, New York, Springer182

[32] Zhu L-P, Li L, Li R, Zhu L-X. Model-free feature screening for ultrahigh-dimensional data. J. Am. Stat. Assoc.. 2011, 106(496): 1464-1475.

Funding

NNSFC(12101346)

Natural Science Foundation of Shandong Province(ZR2021QA044)

Science and Technology Support Plan for Youth Innovation of Colleges in Shandong Province(DC2000000891)

RIGHTS & PERMISSIONS

School of Mathematical Sciences, University of Science and Technology of China and Springer-Verlag GmbH Germany, part of Springer Nature