Frequentist model averaging for linear mixed-effects models

Abstract

Linear mixed-effects models are a powerful tool for the analysis of longitudinal data. The aim of this paper is to study model averaging for linear mixed-effects models. The asymptotic distribution of the frequentist model average estimator is derived, and a confidence interval procedure with an actual coverage probability that tends to the nominal level in large samples is developed. The two confidence intervals based on the model averaging and based on the full model are shown to be asymptotically equivalent. A simulation study shows good finite sample performance of the model average estimators.

Key words： Asymptotic equivalence; asymptotic normality; mixed-effects models; model averaging

References

1 Buckland S T, Burnham K P, Augustin N H. Model selection: An integral part of inference. Biometrics, 1997, 53: 603-618

2 Claeskens G, Hjort N L. Model Selection and Model Averaging. New York: Cambridge University Press, 2008

3 Danilov D, Magnus J R. On the harm that ignoring pretesting can cause. J Econometrics, 2004, 122: 27-46

4 Di C, Crainiceanu C, Caffo B, Punjabi N. Multilevel functional principal component analysis. Ann Appl Stat, 2008, 3: 458-488

5 Dimova R B, Markatou M, Talal A H. Information methods for model selection in linear mixed effects models with application to HCV data. Comput Statist Data Anal, 2011, 55: 2677-2697

6 Draper D. Assessment and propagation of model uncertainty. J Roy Statist Soc Ser B, 1995, 57: 45-97

7 Goldenshluger A. A universal procedure for aggregating estimators. Ann Statist, 2009, 37: 542-568

8 Greven S, Kneib T. On the behaviour of marginal and conditional AIC in linear mixed models. Biometrika, 2010, 97: 773-789

9 Hansen B E. Least squares model averaging. Econometrica, 2007, 75: 1175-1189

10 Hansen B E. Least squares forecast averaging. J Econometrics, 2008, 146: 342-350

11 Hansen B E. Averaging estimators for autoregressions with a near unit root. J Econometrics, 2010, 158: 142-155

12 Hansen B E, Racine J. Jackknife model averaging estimators. J Econometrics, 2012, 167: 38-46

13 Hjort N L, Claeskens G. Frequentist model average estimators. J Amer Statist Assoc, 2003, 98: 879-899

14 Hjort N L, Claeskens G. Focussed information criteria and model averaging for Cox’s hazard regression model. J Amer Statist Assoc, 2006, 101: 1449-1464

15 Hodges J S. Some algebra and geometry for hierarchical models, applied to diagnostics (with Discussion). J Roy Statist Soc Ser B, 1998, 60: 497-536

16 Hoeting J A, Madigan D, Raftery A E, Volinsky C T. Bayesian model averaging: A tutorial. Statist Sci, 1999, 14: 382-417

17 Hodegs J S, Sargent D J. Counting degrees of freedom in hierarchical and other parameterized models. Biometrika, 2001, 88: 367-379

18 Kabaila P, Leeb H. On the large-sample minimal coverage probability of confidence intervals after model selection. J Amer Statist Assoc, 2006, 101: 619-629

19 Laird N M, Ware J H. Random-effects models for longitudinal data. Biometrics, 1982, 38: 963-974

20 Lee Y, Nelder J A. Hierarchical generalized linear models: A synthesis of generalized linear models, random effect models and structured dispersions. Biometrika, 2001, 88: 987-1006

21 Li Y, Baron J. Behavioral Research Data Analysis with R. New York: Springer, 2012

22 Liang H, Wu H, Zou G. A note on conditional AIC for linear mixed-effects models. Biometrika, 2008, 95: 773-778

23 Liang H, Zhang X, Liu A, Ruppert D, Zou G. Selection strategy for covariance structure of random effects in linear mixed-effects models. University of Rochester, Mimeo, 2011

24 Liang H, Zou G, Wan A T K, Zhang X. Optimal weight choice for frequentist model average estimators. J Amer Statist Assoc, 2011, 106: 1053-1066

25 Liu S, Yang Y. Combining models in longitudinal data analysis. Ann Inst Statist Math, 2012, 64: 233-254

26 Ngo L, Brand R. Model selection in linear mixed effects models using SAS Proc Mixed. SAS Global Forum 22, 2002

27 Pinheiro J C, Bates D M. Mixed Effects Models in S and S-plus. New York: Springer, 2000

28 Raftery A, Madigan D, Hoeting J. Bayesian model averaging for linear regression models. J Amer Statist Assoc, 1997, 92: 179-191

29 Rao J N K. Small Area Estimation. New York: John Wiley, 2003

30 Schomaker M, Wan A T K, Heumann C. Frequentist model averaging with missing observations. Comput Statist Data Anal, 2010, 54: 3336-3347

31 Staiger D, Stock J H. Instrumental variables regression with weak instruments. Econometrica, 1997, 65: 557-586

32 Tarpey T, Petkova E, Lu Y, Govindarajulu U. Optimal partitioning for linear mixed effects models: Applications to identifying placebo responders. J Amer Statist Assoc, 2010, 105: 968-977

33 Vaida F, Blanchard S. Conditional Akaike information for mixed-effects models. Biometrika, 2005, 92: 351-370

34 Wan A T K, Zhang X, Zou G. Least squares model averaging by Mallows criterion. J Econometrics, 2010, 156: 277-283

35 Wang H, Zhang X, Zou G. Frequentist model averaging estimation: A review. J Syst Sci Complex, 2009, 22: 732-748

36 Wang H, Zou G. Frequentist model average estimation for linear errors-in-variables models. J Syst Sci Math Sci, 2012, 32: 1-14

37 Yang Y. Adaptive regression by mixing. J Amer Statist Assoc, 2001, 96: 574-586

38 Yuan Z, Yang Y. Combining linear regression models: When and how? J Amer Statist Assoc, 2005, 100: 1202-1214

39 Zhang X, Liang H. Focused information criterion and model averaging for generalized additive partial linear models. Ann Statist, 2011, 39: 174-200

40 Zhang X, Wan A T K, Zhou Z. Focused information criteria, model selection and model averaging in a Tobit model with a non-zero threshold. J Bus Econom Statist, 2012, 30: 132-142