Frequentist model averaging for linear mixed-effects models

Abstract

Linear mixed-effects models are a powerful tool for the analysis of longitudinal data. The aim of this paper is to study model averaging for linear mixed-effects models. The asymptotic distribution of the frequentist model average estimator is derived, and a confidence interval procedure with an actual coverage probability that tends to the nominal level in large samples is developed. The two confidence intervals based on the model averaging and based on the full model are shown to be asymptotically equivalent. A simulation study shows good finite sample performance of the model average estimators.

Keywords

Asymptotic equivalence / asymptotic normality / mixed-effects models / model averaging

References

[1] Buckland S T, Burnham K P, Augustin N H. Model selection: An integral part of inference. Biometrics, 1997, 53: 603-618

[2] Claeskens G, Hjort N L. Model Selection and Model Averaging, 2008, New York: Cambridge University Press

[3] Danilov D, Magnus J R. On the harm that ignoring pretesting can cause. J Econometrics, 2004, 122: 27-46

[4] Di C, Crainiceanu C, Caffo B, Punjabi N. Multilevel functional principal component analysis. Ann Appl Stat, 2008, 3: 458-488

[5] Dimova R B, Markatou M, Talal A H. Information methods for model selection in linear mixed effects models with application to HCV data. Comput Statist Data Anal, 2011, 55: 2677-2697

[6] Draper D. Assessment and propagation of model uncertainty. J Roy Statist Soc Ser B, 1995, 57: 45-97

[7] Goldenshluger A. A universal procedure for aggregating estimators. Ann Statist, 2009, 37: 542-568

[8] Greven S, Kneib T. On the behaviour of marginal and conditional AIC in linear mixed models. Biometrika, 2010, 97: 773-789

[9] Hansen B E. Least squares model averaging. Econometrica, 2007, 75: 1175-1189

[10] Hansen B E. Least squares forecast averaging. J Econometrics, 2008, 146: 342-350

[11] Hansen B E. Averaging estimators for autoregressions with a near unit root. J Econometrics, 2010, 158: 142-155

[12] Hansen B E, Racine J. Jackknife model averaging estimators. J Econometrics, 2012, 167: 38-46

[13] Hjort N L, Claeskens G. Frequentist model average estimators. J Amer Statist Assoc, 2003, 98: 879-899

[14] Hjort N L, Claeskens G. Focussed information criteria and model averaging for Cox’s hazard regression model. J Amer Statist Assoc, 2006, 101: 1449-1464

[15] Hodges J S. Some algebra and geometry for hierarchical models, applied to diagnostics (with Discussion). J Roy Statist Soc Ser B, 1998, 60: 497-536

[16] Hoeting J A, Madigan D, Raftery A E, Volinsky C T. Bayesian model averaging: A tutorial. Statist Sci, 1999, 14: 382-417

[17] Hodegs J S, Sargent D J. Counting degrees of freedom in hierarchical and other parameterized models. Biometrika, 2001, 88: 367-379

[18] Kabaila P, Leeb H. On the large-sample minimal coverage probability of confidence intervals after model selection. J Amer Statist Assoc, 2006, 101: 619-629

[19] Laird N M, Ware J H. Random-effects models for longitudinal data. Biometrics, 1982, 38: 963-974

[20] Lee Y, Nelder J A. Hierarchical generalized linear models: A synthesis of generalised linear models, random effect models and structured dispersions. Biometrika, 2001, 88: 987-1006

[21] Li Y, Baron J. Behavioral Research Data Analysis with R, 2012, New York: Springer

[22] Liang H, Wu H, Zou G. A note on conditional AIC for linear mixed-effects models. Biometrika, 2008, 95: 773-778

[23] Liang H, Zhang X, Liu A, Ruppert D, Zou G. Selection strategy for covariance structure of random effects in linear mixed-effects models, 2011, Mimeo: University of Rochester

[24] Liang H, Zou G, Wan A T K, Zhang X. Optimal weight choice for frequentist model average estimators. J Amer Statist Assoc, 2011, 106: 1053-1066

[25] Liu S, Yang Y. Combining models in longitudinal data analysis. Ann Inst Statist Math, 2012, 64: 233-254

[26] Ngo L, Brand R. Model selection in linear mixed effects models using SAS Proc Mixed. SAS Global Forum 22, 2002

[27] Pinheiro J C, Bates D M. Mixed Effects Models in S and S-plus, 2000, New York: Springer

[28] Raftery A, Madigan D, Hoeting J. Bayesian model averaging for linear regression models. J Amer Statist Assoc, 1997, 92: 179-191

[29] Rao J N K. Small Area Estimation, 2003, New York: John Wiley

[30] Schomaker M, Wan A T K, Heumann C. Frequentist model averaging with missing observations. Comput Statist Data Anal, 2010, 54: 3336-3347

[31] Staiger D, Stock J H. Instrumental variables regression with weak instruments. Econometrica, 1997, 65: 557-586

[32] Tarpey T, Petkova E, Lu Y, Govindarajulu U. Optimal partitioning for linear mixed effects models: Applications to identifying placebo responders. J Amer Statist Assoc, 2010, 105: 968-977

[33] Vaida F, Blanchard S. Conditional Akaike information for mixed-effects models. Biometrika, 2005, 92: 351-370

[34] Wan A T K, Zhang X, Zou G. Least squares model averaging by Mallows criterion. J Econometrics, 2010, 156: 277-283

[35] Wang H, Zhang X, Zou G. Frequentist model averaging estimation: A review. J Syst Sci Complex, 2009, 22: 732-748

[36] Wang H, Zou G. Frequentist model average estimation for linear errors-in-variables models. J Syst Sci Math Sci, 2012, 32: 1-14

[37] Yang Y. Adaptive regression by mixing. J Amer Statist Assoc, 2001, 96: 574-586

[38] Yuan Z, Yang Y. Combining linear regression models: When and how?. J Amer Statist Assoc, 2005, 100: 1202-1214

[39] Zhang X, Liang H. Focused information criterion and model averaging for generalized additive partial linear models. Ann Statist, 2011, 39: 174-200

[40] Zhang X, Wan A T K, Zhou Z. Focused information criteria, model selection and model averaging in a Tobit model with a non-zero threshold. J Bus Econom Statist, 2012, 30: 132-142