Robust Model Structure Recovery for Ultra-High-Dimensional Varying-Coefficient Models

Abstract

As an important extension of the varying-coefficient model, the partially linear varying-coefficient model has been widely studied in the literature. It is vital that how to simultaneously eliminate the redundant covariates and separate the varying and nonzero constant coefficients for varying-coefficient models. In this paper, we consider the penalized composite quantile regression to explore the model structure of ultra-high-dimensional varying-coefficient models. Under some regularity conditions, we study the convergence rate and asymptotic normality of the oracle estimator and prove that, with probability approaching one, the oracle estimator is a local solution of the nonconvex penalized composite quantile regression. Simulation studies indicate that the novel method as well as the oracle method performs in both low dimension and high dimension cases. An environmental data application is also analyzed by utilizing the proposed procedure.

Keywords

Asymptotic properties / Composite quantile regression / Nonconvex penalty / Robust structure recovery / Ultra-high dimension

References

[1] AhmadI, LeelahanonS, LiQ. Efficient estimation of a semiparametric partially linear varying coefficient model. Ann. Statist., 2005, 33: 258-283

[2] ChenY, BaiY, FungW. Structural identification and variable selection in high-dimensional varying-coefficient models. J. Nonparametr. Stat., 2017, 29: 258-279

[3] ChenJ, ChenZ. Extended Bayesian information criteria for model selection with large model spaces. Biometrika, 2008, 95: 759-771

[4] ChengM, HondaT, LiJ, PengH. Nonparametric independence screening and structure identification for ultra-high dimensional longitudinal data. Ann. Statist., 2014, 42: 1819-1849

[5] De BoorCA Practical Guide to Splines, 2001New YorkSpringer

[6] EubankRL, HuangCF, MaldonadoYM, WangN, WangS, BuchananRJ. Smoothing spline estimation in varying-coefficient models. J. R. Stat. Soc. Ser. B Stat. Methodol., 2004, 66: 653-667

[7] FanJ, LiR. Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Statist. Assoc., 2001, 96: 1348-1360

[8] FanJ, LvJ. Nonconcave penalized likelihood with NP-dimensionality. IEEE Trans. Inform. Theory, 2011, 57: 5467-5484

[9] FanJ, HuangT. Profile likelihood inferences on semiparametric varying-coefficient partially linear models. Bernoulli, 2005, 11: 1031-1057

[10] FanJ, ZhangW. Simultaneous confidence bands and hypothesis testing in varying-coefficient models. Scand. J. Stat., 2000, 27: 715-731

[11] HastieT, TibshiraniR. Varying-coefficient models. J. R. Stat. Soc. Ser. B Stat. Methodol., 1993, 55: 757-796

[12] HuT, XiaY. Adaptive semi-varying coefficient model selection. Statist. Sinica, 2012, 22: 575-599

[13] HuangJ, WeiF, MaS. Semiparametric regression pursuit. Statist. Sinica, 2012, 22: 1403-1426

[14] HunterD, LangeK. Quantile regression via an MM algorithm. J. Comput. Graph. Statist., 2000, 9: 60-77

[15] JiangQ, WangH, XiaY, JiangG. On a principal varying coefficient model. J. Am. Statist. Assoc., 2013, 108: 228-236

[16] KaiB, LiR, ZouH. Local composite quantile regression smoothing: an efficient and safe alternative to local polynomial regression. J. R. Stat. Soc. Ser. B Stat. Methodol., 2010, 72: 49-69

[17] KaiB, LiR, ZouH. New efficient estimation and variable selection methods for semiparametric varying-coefficient partially linear models. Ann. Statist., 2011, 39: 305-332

[18] KimMO. Quantile regression with varying coefficients. Ann. Statist., 2007, 35: 92-108

[19] KimY, ChoiH, OhH. Smoothly clipped absolute deviation on high dimensions. J. Am. Statist. Assoc., 2008, 103: 1665-1673

[20] KoenkerRQuantile Regression, 2005New YorkCambridge University Press

[21] LengC. A simple approach for varying-coefficient model selection. J. Statist. Plann. Infer., 2009, 139: 2138-2146

[22] LiD, KeY, ZhangW. Model selection and structure specification in ultra-high dimensional generalised semi-varying coefficient models. Ann. Statist., 2015, 43: 2676-2705

[23] LiG, PengH, ZhangJ, ZhuL. Robust rank correlation based screening. Ann. Statist., 2012, 40: 1846-1877

[24] LianH, LaiP, LiangH. Partially linear structure selection in cox models with varying coefficients. Biometrics, 2013, 69: 348-357

[25] LianH. Variable selection for high-dimensional generalized varying-coefficient models. Statist. Sinica, 2012, 22: 1563-1588

[26] LianH, LiangH, RuppertD. Separation of covariates into nonparametric and parametric parts in high-dimensional partially linear additive models. Statist. Sinica, 2015, 25: 591-607

[27] MaX, ZhangJ. A new variable selection approach for varying coefficient models. Metrika, 2016, 79: 59-72

[28] NohH, Van KeilegomI. Efficient model selection in semivarying coefficient models. Electron. J. Stat., 2012, 6: 2519-2534

[29] ParkBU, MammenE, LeeYK, LeeER. Varying coefficient regression models, a review and new developments. Intern. Statist. Rev., 2015, 83: 36-64

[30] QinG, MaoJ, ZhuZ. Joint mean-covariance model in generalized partially linear varying coefficient models for longitudinal data. J. Statist. Comput. Simulat., 2016, 86: 1166-1182

[31] QuA, LiR. Quadratic inference functions for varying-coefficient models with longitudinal data. Biometrics, 2006, 62: 379-391

[32] SherwoodB, WangL. Partially linear additive quantile regression in ultra-high dimension. Ann. Statist., 2016, 44: 288-317

[33] StoneCJ. Additive regression and other nonparametric models. Ann. Statist., 1985, 13: 689-705

[34] TangY, WangHJ, ZhuZ, SongX. A unified variable selection approach for varying coefficient models. Statist. Sinica, 2012, 22: 601-628

[35] Tibshirani, R.: Regression shrinkage and selection via the Lasso. J. R. Stat. Soc. Ser. B Stat. Methodol. 58, 267–288 (1996)

[36] WangL, KaiB, LiR. Local rank inference for varying coefficient models. J. Amer. Statist. Assoc., 2009, 104: 1631-1645

[37] WangD, KulasekeraKB. Parametric component detection and variable selection in varying-coefficient partially linear models. J. Multiv. Anal., 2012, 112: 117-129

[38] WangK, LinL. Robust structure identification and variable selection in partial linear varying coefficient models. J. Statist. Plann. Infer., 2016, 174: 153-168

[39] WangK, LinL. Robust and efficient estimator for simultaneous model structure identification and variable selection in generalized partial linear varying coefficient models with longitudinal data. Statist. Pap., 2019, 60: 1649-1676

[40] WangM, ZhaoP, KangX. Structure identification for varying coefficient models with measurement errors based on kernel smoothing. Statist. Pap., 2020, 61: 1841-1857

[41] WangHJ, ZhuZ, ZhouJ. Quantile regression in partially linear varying coefficient models. Ann. Statist., 2009, 37: 3841-3866

[42] WeiY, HeX. Conditional growth charts (with discussion). Ann. Statist., 2006, 34: 2069-2097

[43] Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B Stat. Methodol. 68, 49–67 (2006)

[44] ZhangCH. Nearly unbiased variable selection under minimax concave penalty. Ann. Statist., 2010, 38: 894-942

[45] ZhangHH, ChengG, LiuY. Linear or nonlinear? Automatic structure discovery for partially linear models. J. Am. Statist. Assoc., 2011, 106: 1099-1112

[46] ZhouY, LiangH. Statistical inference for semiparametric varying-coefficient partially linear models with error-prone linear covariates. Ann. Statist., 2009, 37: 427-458

[47] ZouH, YuanM. Composite quantile regression and the oracle model selection theory. Ann. Statist., 2008, 36: 1108-1126

Funding

National Natural Science Foundation of China(11801168)

Natural Science Foundation of Hunan Province(2018JJ3322)

Scientific Research Fund of Hunan Provincial Education Department(18B024)

Discovery Grants(RGPIN-2018-06466)

National Statistical Science Research Project of China(2020LZ26)

Natural Science Foundation of Shandong Province(ZR2019MA002)

RIGHTS & PERMISSIONS

School of Mathematical Sciences, University of Science and Technology of China and Springer-Verlag GmbH Germany, part of Springer Nature