Dynamical behaviors of recurrently connected neural networks and linearly coupled networks with discontinuous right-hand sides

Abstract

The aim of this paper is to provide a systematic review on the framework to analyze dynamics in recurrently connected neural networks with discontinuous right-hand sides with a focus on the authors’ works in the past three years. The concept of the Filippov solution is employed to define the solution of the neural network systems by transforming them to differential inclusions. The theory of viability provides a tool to study the existence and uniqueness of the solution and the Lyapunov function (functional) approach is used to investigate the global stability and synchronization. More precisely, we prove that the diagonal-dominant conditions guarantee the existence, uniqueness, and stability of a general class of integro-differential equations with (almost) periodic self-inhibitions, interconnection weights, inputs, and delays. This model is rather general and includes the well-known Hopfield neural networks, Cohen-Grossberg neural networks, and cellular neural networks as special cases. We extend the absolute stability analysis of gradient-like neural network model by relaxing the analytic constraints so that they can be employed to solve optimization problem with non-smooth cost functions. Furthermore, we study the global synchronization problem of a class of linearly coupled neural network with discontinuous right-hand sides.

Keywords

delayed integro-differential system / discontinuous activation / almost periodic function / nonsmooth cost function / complete synchronization

References

[1] Cohen M A, Grossberg S. Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Transactions on Systems, Man, and Cybernetics, 1983, 13(5): 815-826

[2] Hopfield J J. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the United States of America, 1982, 79(8): 2554-2558

[3] Hopfield J J. Neurons with graded response have collective computational properties like those of two-stage neurons. Proceedings of the National Academy of Sciences of the United States of America, 1984, 81(10): 3088-3092

[4] Chua L O, Yang L. Cellular neural networks: Theory. IEEE Transactions on Circuits and Systems, 1988, 35(10): 1257-1272

[5] Chua L O, Yang L. Cellular neural networks: Applications. IEEE Transactions on Circuits and Systems, 1988, 35(10): 1273-1290

[6] Forti M, Tesi A. Absolute stability of analytic neural networks: An approach based on finite trajectory length. IEEE Transactions on Circuits and Systems I: Regular Papers, 2004, 51(12): 2460-2469

[7] Civalleri P P, Gilli L M, Pandolfi L. On stability of cellular neural networks with delay. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 1993, 40(3): 157-165

[8] Chen T P, Lu W L, Chen G R. Dynamical behaviors of a large class of general delayed neural networks. Neural Computation, 2005, 17(4): 949-968

[9] Lu W L, Chen T P. Global convergent dynamics of delayed neural networks. In: Atay F M, ed. Complex Time-Delay Systems, Understanding Complex Systems, Chapter 7. Springer-Verlag, 2010

[10] Forti M, Nistri P. Global convergence of neural networks with discontinuous neuron activations. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 2003, 50(11): 1421-1435

[11] Harrer H, Nossek J A, Stelzl R. An analog implementation of discrete-time cellular neural networks. IEEE Transactions on Neural Networks, 1992, 3(3): 466-476

[12] Kennedy M P, Chua L O. Neural networks for nonlinear programming. IEEE Transactions on Circuits and Systems, 1988, 35(5): 554-562

[13] Utkin V I. Variable structure systems with sliding modes. IEEE Transactions on Automatic Control, 1977, 22(2): 212-222

[14] Lojasiewicz S. Une propriet′e topologique des sous-ensembles analytiques r′eels, Colloques internationaux du C.N.R.S. Les ′Equations aux D′erive′es Partielles, 1963, 117: 87-89

[15] Lojasiewicz S. Sur la g′eom′etrie semi- et sous-analytique. Annales de L’Institut Fourier, 1993, 43(5): 1575-1595

[16] Lu W L, Chen T P. Almost periodic dynamics of a class of delayed neural networks with discontinuous activations. Neural Computation, 2008, 20(4): 1065-1090

[17] Lu W L, Wang J. Convergence analysis of a class of nonsmooth gradient systems. IEEE Transactions on Circuits and Systems I: Regular Papers, 2008, 55(11): 3514-3527

[18] He X N, Lu W L, Chen T P. Nonnegative periodic dynamics of delayed Cohen-Grossberg neural networks with discontinuous activations. Neurocomputing, 2010, 73(13-15): 2765-2772

[20] Liu B, Lu WL, Chen T P. New conditions on synchronization of networks of linearly coupled dynamical systems with non-Lipschitz right-hand sides. Neural Networks, 2012, 25(1): 5-13

[21] Filippov A F. Classical solution of differential equations with multivalued right-hand side. SIAM Journal on Control, 1967, 5(4): 609-621

[22] Aubin J P, Cellina A. Differential Inclusions. Berlin: Springer-Verlag, 1984

[23] Aubin J P, Frankowska H. Set-Valued Analysis. Boston: Birkhauser, 1990

[24] Aubin J P. Viability Theory. Boston: Birhauser, 1991

[25] Haddad G. Monotine viable trajectories for functional differential inclusions. Journal of Differential Equations, 1981, 42(1): 1-24

[26] Hale J. Theory of Functional Differential Equations. New York: Springer-Verlag, 1977

[27] Chen T P, Rong L B. Robust global exponential stability of Cohen-Grossberg neural networks with time delays. IEEE Transactions on Neural Networks, 2004, 15(1): 203-206

[28] Lu W L, Chen T P. New conditions on global stability of Cohen-Grossberg neural networks. Neural Computation, 2003, 15(5): 1173-1189

[29] Lu WL, Chen T P. Dynamical behaviors of Cohen-Grossberg neural networks with discontinuous activation functions. Neural Networks, 2005, 18(3): 231-242

[30] Grossberg S. Biological competition: Decision rules, pattern formation, and oscillations. Proceedings of the National Academy of Sciences of the United States of America, 1980, 77(4): 2338-2342

[31] Grossberg S. Nonlinear neural networks: Principles, mechanisms, and architectures. Neural Networks, 1988, 1(1): 17-61

[32] Forti M, Tesi A. The _Lojasiewicz exponent at equilibrium point of a standard CNN is 1/2. International Journal of Bifurcation and Chaos in Applied Sciences and Engineering, 2006, 16(8): 2191-2205

[33] Bolte J, Daniilidis A, Lewis A. The _Lojasiewicz inequality for nonsmooth functions with applications to subgradient dynamical systems. SIAM Journal on Optimization, 2007, 17(4): 1205-1223

[34] Krantz S G, Parks H R. A Primer of Real Analytic Functions. 2nd ed. Boston: Birkhäuser, 2002

[35] Vidyasagar M. Minimum-seeking properties of analog neural networks with multilinear objective functions. IEEE Transactions on Automatic Control, 1995, 40(8): 1359-1375

[36] Wu C W, Chua L O. Synchronization in an array of linearly coupled dynamical systems. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 1995, 42(8): 430-447

[38] Forti M, Nistri P, Papini D. Global exponential stability and global convergence in finite time of delayed neural networks with infinite gain. IEEE Transactions on Neural Networks, 2005, 16(6): 1449-1463

[40] Lu W L, Chen T P. Dynamical behaviors of delayed neural network systems with discontinuous activation functions. Neural Computation, 2006, 18(3): 683-708

[41] Papini D, Taddei V. Global exponential stability of the periodic solution of the delayed neural networks with discontinuous activations. Physics Letters A, 2005, 343(1-3): 117-128

[42] Wang Y, Zuo Y, Huang L, Li C. Global robust stability of delayed neural networks with discontinuous activation functions. IET Control Theory & Applications, 2008, 2(7): 543-553

[43] Liu X, Cao J. On periodic solutions of neural networks via differential inclusions. Neural Networks, 2009, 22(4): 329-334

[44] Wang L. Multistability of almost periodic solutions of neural networks with discontinuous activation functions. In: Proceedings of the Third International Workshop on Advanced Computational Intelligence. 2010, 16-20

[45] Lin W, Chen T P. Positive periodic solutions of delayed periodic Lotka-Volterra systems. Physics Letters A, 2005, 334(4): 273-287

[46] Lu WL, Chen T P. R+nRn+-global stability of a Cohen-Grossberg neural network system with nonnegative equilibria. Neural Networks, 2007, 20(6): 714-722

[47] Forti M, Nistri P, Quincampoix M. Convergence of neural networks for programming problems via a nonsmooth Lojasiewicz inequality. IEEE Transactions on Neural Networks, 2006, 17(6): 1471-1486

[49] Bolte J, Daniilidis A, Lewis A. A nonsmooth Morse-Sard theorem for subanalytic functions. Journal of Mathematical Analysis and Applications, 2006, 321(2): 729-740

[50] Liu Q, Wang J. A one-layer recurrent neural network with a discontinuous hard-limiting activation function for quadratic programming. IEEE Transactions on Neural Networks, 2008, 19(4): 558-570

[51] Liu Q, Wang J. A one-layer recurrent neural network with a discontinuous activation function for linear programming. Neural Computation, 2008, 20(5): 1366-1383

[52] Belykh I, Belykh V, Hasler M. Blinking model and synchronization in small-world networks with a time-varying coupling. Physica D: Nonlinear Phenomena, 2004, 195(1-2): 188-206

[54] Pavlov A, Pogromsky A, van de Wouw N, Nijmeijer H. On convergence properties of piecewise affine systems. International Journal of Control, 2007, 80(8): 1233-1247

[55] van de Wouw N, Pavlov A. Tracking and synchronisation for a class of PWA systems. Automatica, 2008, 44(11): 2909-2915

RIGHTS & PERMISSIONS

2014 Higher Education Press and Springer-Verlag Berlin Heidelberg