Composite T-Process Regression Models

Abstract

Process regression models, such as Gaussian process regression model (GPR), have been widely applied to analyze kinds of functional data. This paper introduces a composite of two T-process (CT), where the first one captures the smooth global trend and the second one models local details. The CT has an advantage in the local variability compared to general T-process. Furthermore, a composite T-process regression (CTP) model is developed, based on the composite T-process. It inherits many nice properties as GPR, while it is more robust against outliers than GPR. Numerical studies including simulation and real data application show that CTP performs well in prediction.

Keywords

Composite Gaussian process regression / Composite T-process regression / Extended T-process regression / Functional data

References

[1] Álvarez, M., Luengo, D., Titsias, M., Lawrence, N.D.: Efficient multioutput Gaussian processes through variational inducing kernels. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 25–32 (2010)

[2] Archambeau, C., Bach, F.: Multiple Gaussian process models. arXiv:11105238 (2010)

[3] Aronszajn N. Theory of reproducing kernels. Trans. Am. Math. Soc.. 1950, 68 3 337-404

[4] Ba S, Joseph VR. Composite Gaussian process models for emulating expensive functions. Ann. Appl. Stat.. 2012, 6 1838-1860

[5] Boyd S, Boyd SP, Vandenberghe L. Convex Optimization. 2004 Cambridge: Cambridge University Press

[6] Dai J, Krems RV. Interpolation and extrapolation of global potential energy surfaces for polyatomic systems by Gaussian processes with composite kernels. J. Chem. Theory Comput.. 2020, 16 3 1386-1395

[7] Gramacy RB, Lee HKH. Bayesian treed Gaussian process models with an application to computer modeling. J. Am. Stat. Assoc.. 2008, 103 483 1119-1130

[8] Hampel FR. Robust Statistics: The Approach Based on Influence Functions. 1986 New York: Wiley

[9] Higdon D, Swall J, Kern J. Non-stationary spatial modeling. Bayesian Stat.. 1999, 6 1 761-768

[10] Jiao J, Hengjian C. Parametric estimation based on robust scatter matrix in semiparametric regression model. J. Beijing Normal Univ. Nat. Sci. Ed.. 2006, 42 3 224

[11] Kuß, M.: Gaussian process models for robust regression, classification, and reinforcement learning. PhD thesis, Technische Universität (2006)

[12] Liu H, Cai J, Ong YS. Remarks on multi-output Gaussian process regression. Knowl. Based Syst.. 2018, 144 102-121

[13] Naish-Guzman, A., Holden, S.: Robust regression with twinned Gaussian processes. In: Advances in Neural Information Processing Systems, 20, 1065–1072 (2007)

[14] Neal, R.M.: Monte Carlo implementation of Gaussian process models for Bayesian regression and classification. arXiv preprint physics (1997)

[15] Paciorek, C.J., Schervish, M.J.: Nonstationary covariance functions for Gaussian process regression. In: Advances in Neural Information Processing Systems, 16, 273–280 (2003)

[16] Paciorek CJ, Schervish MJ. Spatial modelling using a new class of nonstationary covariance functions. Environmetrics. 2006, 17 5 483-506

[17] Qin G, Zhu Z, Fung WK. Robust estimation of covariance parameters in partial linear model for longitudinal data. J. Stat. Plan. Inference. 2009, 139 2 558-570

[18] Ranjan P, Haynes R, Karsten R. A computationally stable approach to Gaussian process interpolation of deterministic computer simulation data. Technometrics. 2011, 53 4 366-378

[19] Rockafellar RT. Convex Analysis. 1970 Princeton: Princeton University Press

[20] Sampson PD, Guttorp P. Nonparametric estimation of nonstationary spatial covariance structure. J. Am. Stat. Assoc.. 1992, 87 417 108-119

[21] Seeger MW, Kakade SM, Foster DP. Information consistency of nonparametric Gaussian process methods. IEEE Trans. Inf. Theory. 2008, 54 2376-2382

[22] Shah, A., Wilson, A.G., Ghahramani, Z.: Student-t processes as alternatives to Gaussian processes. In: Proceedings of the 17th International Conference on Artificial Intelligence and Statistics, AISTATS, pp. 877–885 (2014)

[23] Shi JQ, Choi T. Gaussian Process Regression Analysis for Functional Data. 2011 London: Chapman and Hall/CRC

[24] Vanhatalo, J., Jylänki, P., Vehtari, A.: Gaussian process regression with student-t likelihood. In: Advances in Neural Information Processing Systems, 22, 1910–1918 (2009)

[25] Wahba G. Spline Models for Observational Data. 1990 Philadelphia: SIAM

[26] Wang X, Jiang Y, Huang M, Zhang H. Robust variable selection with exponential squared loss. J. Am. Stat. Assoc.. 2013, 108 632-643

[27] Wang Z, Shi JQ, Lee Y. Extended t-process regression models. J. Stat. Plan. Inference. 2017, 189 38-60

[28] Wang Z, Li K, Shi JQ. A robust estimation for the extended t-process regression model. Stat. Probab. Lett.. 2020, 157 108626

[29] Wauthier, F.L., Jordan, M.I.: Heavy-tailed process priors for selective shrinkage. In: Advances in Neural Information Processing Systems, 23, 2406–2414 (2010)

[30] Williams CK, Rasmussen CE. Gaussian Processes for Machine Learning. 2006 Cambridge: The MIT Press

[31] Xiong Y, Chen W, Apley D, Ding X. A non-stationary covariance-based kriging method for metamodelling in engineering design. Int. J. Numer. Methods Eng.. 2007, 71 6 733-756

[32] Xu P, Lee Y, Shi JQ, Eyre J. Automatic detection of significant areas for functional data with directional error control. Stat. Med.. 2019, 38 3 376-397

[33] Xu, Z., Yan, F., Qi, Y.: Sparse matrix-variate t process blockmodels. In: Proceedings of the 25th AAAI Conference on Artificial Intelligence, pp. 543–548 (2011)

[34] Yu, S., Tresp, V., Yu, K.: Robust multi-task learning with t-processes. In: Proceedings of the 24th International Conference on Machine Learning, pp. 1103–1110 (2007)

[35] Zang, Q., Klette, R.: Evaluation of an adaptive composite Gaussian model in video surveillance. In: International Conference on Computer Analysis of Images and Patterns, Springer, pp. 165–172 (2003)

[36] Zhang, Y., Yeung, D.Y.: Multi-task learning using generalized t process. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 964–971 (2010)

Funding

National Natural Science Foundation of China(11971457)

Natural Science Foundation of Anhui Province(1908085MA06)