Variable selection for single-index varying-coefficient model

Abstract

We consider the problem of variable selection for single-index varying-coefficient model, and present a regularized variable selection procedure by combining basis function approximations with SCAD penalty. The proposed procedure simultaneously selects significant covariates with functional coefficients and local significant variables with parametric coefficients. With appropriate selection of the tuning parameters, the consistency of the variable selection procedure and the oracle property of the estimators are established. The proposed method can naturally be applied to deal with pure single-index model and varying-coefficient model. Finite sample performances of the proposed method are illustrated by a simulation study and the real data analysis.

Keywords

Single-index varying-coefficient model / variable selection / SCAD / oracle property

References

[1.] Bunea F. Consistent covariate selection and post model selection inference in semiparametric regression. Ann Statist, 2004, 32: 898-927

[2.] Cai Z W, Fan J Q, Li R Z. Efficient estimation and inferences for varying coefficient models. J Amer Statist Assoc, 2000, 95: 888-902

[3.] Fan J Q, Li R Z. Variable selection via nonconcave penalized likelihood and its oracle properties. J Amer Statist Assoc, 2001, 96: 1348-1360

[4.] Fan J Q, Li R Z. New estimation and model selection procedures for semiparametric modeling in longitudinal data analysis. J Amer Statist Assoc, 2004, 99: 710-723

[5.] Fan J Q, Lv J C. A selective overview of variable selection in high dimensional feature space. Statist Sinica, 2010, 20: 101-148

[6.] Fan J Q, Yao Q W, Cai Z W. Adaptive varying-coefficient linear models. J R Stat Soc Ser B Stat Methodol, 2003, 65: 57-80

[7.] Fan J Q, Zhang W Y. Statistical estimation in varying-coefficient models. Ann Statist, 1999, 27: 1491-1518

[8.] Frank I E, Friedman J H. A statistical view of some chemometrics regression tools. Technometrics, 1993, 35: 109-148

[9.] Härdle W, Hall P, Ichimura H. Optimal smoothing in single-index models. Ann Statist, 1993, 21: 157-178

[10.] Hastie T J, Tibshirani R. Varying-coefficient models. J R Stat Soc Ser B Stat Methodol, 1993, 55: 757-796

[11.] He X M, Zhu Z Y, Fung W K. Estimation in a semiparametric model for longitudinal data with unspecified dependence structure. Biometrika, 2002, 89: 579-590

[12.] Hoerl A, Kennard R. Ridge regression. In: Encyclopedia of Statistical Sciences, Vol 8, 1988, New York: Wiley 129 136

[13.] Huang J Z, Shen H. Functional coefficient regression models for non-linear time series: a polynomial spline approach. Scand J Stat, 2004, 31: 515-534

[14.] Li G R, Peng H, Zhang J, Zhu L X. Robust rank correlation based screening. Ann Statist, 2012, 40: 1846-1877

[15.] Li G R, Peng H, Zhu L X. Nonconcave penalized M-estimation with a diverging number of parameters. Statist Sinica, 2011, 21: 391-419

[16.] Li R Z, Liang H. Variable selection in semiparametric regression modeling. Ann Statist, 2008, 36: 261-286

[17.] Lu Z D, Tjøstheim D, Yao Q W. Adaptive varying-coefficient linear models for stochastic processes: Asymptotic theory. Statist Sinica, 2007, 17: 177-197

[18.] Potter S M. A nonlinear approach to US GNP. J Appl Econometrics, 1995, 10: 109-125

[19.] Schumaker L L. Spline Functions, 1981, New York: Wiley Press

[20.] Stone C J. Optimal global rates of convergence for nonparametric regression. Ann Statist, 1982, 10: 1348-1360

[21.] Stute W, Zhu L X. Nonparametric checks for single-index models. Ann Statist, 2005, 33: 1048-1083

[22.] Tiao G C, Tsay R S. Some advances in non-linear and adaptive modelling in timeseries. J Forecast, 1994, 13: 109-131

[23.] Tibshirani R. Regression shrinkage and selection via the Lasso. J R Stat Soc Ser B Stat Methodol, 1996, 58: 267-288

[24.] Wang L, Li H, Huang J Z. Variable selection in nonparametric varying-coefficient models for analysis of repeated measurements. J Amer Statist Assoc, 2008, 103: 1556-1569

[25.] Wu T Z, Lin H Q, Yu Y. Single-index coefficient models for nonlinear time series. J Nonparametr Stat, 2011, 23: 37-58

[26.] Xia Y C, Li W K. On single-index coefficient regression models. J Amer Statist Assoc, 1999, 94: 1275-1285

[27.] Xue L G, Pang Z. Statistical inference for a single-index varying-coefficient model. Stat Comput, online, doi: 10.1007/s11222-012-9332-x

[28.] Xue L G, Wang Q H. Empirical likelihood for single-index varying-coefficient models. Bernoulli, 2012, 18(3): 836-856

[29.] Xue L G, Zhu L X. Empirical likelihood for single-index model. J Multivariate Anal, 2006, 97: 1295-1312

[30.] Xue L G, Zhu L X. Empirical Likelihood for a varying coefficient model with longitudinal data. J Amer Statist Assoc, 2007, 102: 642-654

[31.] Yu Y, Ruppert D. Penalized spline estimation for partially linear single-index models. J Amer Statist Assoc, 2002, 97: 1042-1054

[32.] Yuan M, Lin Y. Model selection and estimation in regression with grouped variables. J R Stat Soc Ser B Stat Methodol, 2006, 68: 49-68

[33.] Zhao P X, Xue L G. Variable selection for semiparametric varying coefficient partially linear model. Statist Probab Lett, 2009, 79: 2148-2157

[34.] Zhao P X, Xue L G. Variable selection for semiparametric varying coefficient partially linear errors-in-variables models. J Multivariate Anal, 2010, 101: 1872-1883

[35.] Zou H. The adaptive Lasso and its oracle properties. J Amer Statist Assoc, 2006, 101: 1418-1429