A Proposal on Machine Learning via Dynamical Systems

Abstract

We discuss the idea of using continuous dynamical systems to model general high-dimensional nonlinear functions used in machine learning. We also discuss the connection with deep learning.

Keywords

Deep learning / Machine learning / Dynamical systems

References

[1] Fan J, Gijbels I. Local Polynomial Modeling and Its Applications. 1996 London: Chapman & Hall

[2] Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction a, Springer Series in Statistics, second edition, (2013)

[3] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015, 521 7553 436-444

[4] Han, J., E, W.: in preparation

[5] Li, Q., Tai, C., E, W.: in preparation

[6] Almeida, L.B.: A learning rule for asynchronous perceptrons with feedback in a combinatorial environment. In: Proceedings ICNN 87. San Diego, IEEE (1987)

[7] LeCun, Y.: A theoretical framework for back propagation. In: Touretzky, D., Hinton, G., Sejnouski, T. (eds.) Proceedings of the 1988 connectionist models summer school, Carnegie-Mellon University, Morgan Kaufmann, (1989)

[8] Pineda, F.J.: Generalization of back propagation to recurrent and higher order neural networks. In: Proceedings of IEEE conference on neural information processing systems, Denver, November, IEEE (1987)

[9] Recht, B.: http://www.argmin.net/2016/05/18/mates-of-costate/

[10] E, W., Ming, P.: Calculus of Variations and Differential Equations, lecture notes, to appear

[11] He, K., Zhang, X., Ren, S., Sun, J.: Identity mapping in deep residual networks. (July, 2016) arXiv:1603.05027v3

[12] Lambert JD. Numerical Methods for Ordinary Differential Systems: The Initial Value Problem. 1992 New York: Wiley

[13] Stroock DW, Varadhan SRS. Multi-Dimensional Diffusion Processes. 2006 Berlin: Springer

[14] Wang, C., Li, Q., E, W., Chazelle, B.: Noisy Hegselmann–Krause systems: phase transition and the 2R-conjecture. In: Proceedings of 55th IEEE Conference on Decision and Control, Las Vegas, (2016) (Full paper at arXiv:1511.02975v3, 2015)

[15] Tabak EG, Vanden-Eijnden E. Density estimation by dual ascent of the log-likelihood. Commun. Math. Sci.. 2010, 8 1 217-233