Dynamical behaviors of recurrently connected neural networks and linearly coupled networks with discontinuous right-hand sides

Abstract

The aim of this paper is to provide a systematic review on the framework to analyze dynamics in recurrently connected neural networks with discontinuous right-hand sides with a focus on the authors’ works in the past three years. The concept of the Filippov solution is employed to define the solution of the neural network systems by transforming them to differential inclusions. The theory of viability provides a tool to study the existence and uniqueness of the solution and the Lyapunov function (functional) approach is used to investigate the global stability and synchronization. More precisely, we prove that the diagonal-dominant conditions guarantee the existence, uniqueness, and stability of a general class of integro-differential equations with (almost) periodic self-inhibitions, interconnection weights, inputs, and delays. This model is rather general and includes the well-known Hopfield neural networks, Cohen-Grossberg neural networks, and cellular neural networks as special cases. We extend the absolute stability analysis of gradient-like neural network model by relaxing the analytic constraints so that they can be employed to solve optimization problem with non-smooth cost functions. Furthermore, we study the global synchronization problem of a class of linearly coupled neural network with discontinuous right-hand sides.

Key words： delayed integro-differential system; discontinuous activation; almost periodic function; nonsmooth cost function; complete synchronization

References

1 Cohen M A, Grossberg S. Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Transactions on Systems, Man, and Cybernetics, 1983, 13(5): 815-826

2 Hopfield J J. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the United States of America, 1982, 79(8): 2554-2558

3 Hopfield J J. Neurons with graded response have collective computational properties like those of two-stage neurons. Proceedings of the National Academy of Sciences of the United States of America, 1984, 81(10): 3088-3092

4 Chua L O, Yang L. Cellular neural networks: Theory. IEEE Transactions on Circuits and Systems, 1988, 35(10): 1257-1272

5 Chua L O, Yang L. Cellular neural networks: Applications. IEEE Transactions on Circuits and Systems, 1988, 35(10): 1273-1290

6 Forti M, Tesi A. Absolute stability of analytic neural networks: An approach based on finite trajectory length. IEEE Transactions on Circuits and Systems I: Regular Papers, 2004, 51(12): 2460-2469

7 Civalleri P P, Gilli L M, Pandolfi L. On stability of cellular neural networks with delay. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 1993, 40(3): 157-165

8 Chen T P, Lu W L, Chen G R. Dynamical behaviors of a large class of general delayed neural networks. Neural Computation, 2005, 17(4): 949-968

9 Lu W L, Chen T P. Global convergent dynamics of delayed neural networks. In: Atay F M, ed. Complex Time-Delay Systems, Understanding Complex Systems, Chapter 7. Springer-Verlag, 2010

10 Forti M, Nistri P. Global convergence of neural networks with discontinuous neuron activations. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 2003, 50(11): 1421-1435

11 Harrer H, Nossek J A, Stelzl R. An analog implementation of discrete-time cellular neural networks. IEEE Transactions on Neural Networks, 1992, 3(3): 466-476

12 Kennedy M P, Chua L O. Neural networks for nonlinear programming. IEEE Transactions on Circuits and Systems, 1988, 35(5): 554-562

13 Utkin V I. Variable structure systems with sliding modes. IEEE Transactions on Automatic Control, 1977, 22(2): 212-222

14 Lojasiewicz S. Une propriet′e topologique des sous-ensembles analytiques r′eels, Colloques internationaux du C.N.R.S. Les ′Equations aux D′erive′es Partielles, 1963, 117: 87-89

15 Lojasiewicz S. Sur la g′eom′etrie semi- et sous-analytique. Annales de L’Institut Fourier, 1993, 43(5): 1575-1595

16 Lu W L, Chen T P. Almost periodic dynamics of a class of delayed neural networks with discontinuous activations. Neural Computation, 2008, 20(4): 1065-1090

17 Lu W L, Wang J. Convergence analysis of a class of nonsmooth gradient systems. IEEE Transactions on Circuits and Systems I: Regular Papers, 2008, 55(11): 3514-3527

18 He X N, Lu W L, Chen T P. Nonnegative periodic dynamics of delayed Cohen-Grossberg neural networks with discontinuous activations. Neurocomputing, 2010, 73(13-15): 2765-2772

20 Liu B, Lu WL, Chen T P. New conditions on synchronization of networks of linearly coupled dynamical systems with non-Lipschitz right-hand sides. Neural Networks, 2012, 25(1): 5-13

21 Filippov A F. Classical solution of differential equations with multivalued right-hand side. SIAM Journal on Control, 1967, 5(4): 609-621

22 Aubin J P, Cellina A. Differential Inclusions. Berlin: Springer-Verlag, 1984

23 Aubin J P, Frankowska H. Set-Valued Analysis. Boston: Birkhauser, 1990

24 Aubin J P. Viability Theory. Boston: Birhauser, 1991

25 Haddad G. Monotine viable trajectories for functional differential inclusions. Journal of Differential Equations, 1981, 42(1): 1-24

26 Hale J. Theory of Functional Differential Equations. New York: Springer-Verlag, 1977

27 Chen T P, Rong L B. Robust global exponential stability of Cohen-Grossberg neural networks with time delays. IEEE Transactions on Neural Networks, 2004, 15(1): 203-206

28 Lu W L, Chen T P. New conditions on global stability of Cohen-Grossberg neural networks. Neural Computation, 2003, 15(5): 1173-1189

29 Lu WL, Chen T P. Dynamical behaviors of Cohen-Grossberg neural networks with discontinuous activation functions. Neural Networks, 2005, 18(3): 231-242

30 Grossberg S. Biological competition: Decision rules, pattern formation, and oscillations. Proceedings of the National Academy of Sciences of the United States of America, 1980, 77(4): 2338-2342

31 Grossberg S. Nonlinear neural networks: Principles, mechanisms, and architectures. Neural Networks, 1988, 1(1): 17-61

32 Forti M, Tesi A. The _Lojasiewicz exponent at equilibrium point of a standard CNN is 1/2. International Journal of Bifurcation and Chaos in Applied Sciences and Engineering, 2006, 16(8): 2191-2205

33 Bolte J, Daniilidis A, Lewis A. The _Lojasiewicz inequality for nonsmooth functions with applications to subgradient dynamical systems. SIAM Journal on Optimization, 2007, 17(4): 1205-1223

34 Krantz S G, Parks H R. A Primer of Real Analytic Functions. 2nd ed. Boston: Birkhäuser, 2002

35 Vidyasagar M. Minimum-seeking properties of analog neural networks with multilinear objective functions. IEEE Transactions on Automatic Control, 1995, 40(8): 1359-1375

36 Wu C W, Chua L O. Synchronization in an array of linearly coupled dynamical systems. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 1995, 42(8): 430-447

38 Forti M, Nistri P, Papini D. Global exponential stability and global convergence in finite time of delayed neural networks with infinite gain. IEEE Transactions on Neural Networks, 2005, 16(6): 1449-1463

40 Lu W L, Chen T P. Dynamical behaviors of delayed neural network systems with discontinuous activation functions. Neural Computation, 2006, 18(3): 683-708

41 Papini D, Taddei V. Global exponential stability of the periodic solution of the delayed neural networks with discontinuous activations. Physics Letters A, 2005, 343(1-3): 117-128

42 Wang Y, Zuo Y, Huang L, Li C. Global robust stability of delayed neural networks with discontinuous activation functions. IET Control Theory & Applications, 2008, 2(7): 543-553

43 Liu X, Cao J. On periodic solutions of neural networks via differential inclusions. Neural Networks, 2009, 22(4): 329-334

44 Wang L. Multistability of almost periodic solutions of neural networks with discontinuous activation functions. In: Proceedings of the Third International Workshop on Advanced Computational Intelligence. 2010, 16-20

45 Lin W, Chen T P. Positive periodic solutions of delayed periodic Lotka-Volterra systems. Physics Letters A, 2005, 334(4): 273-287

46 Lu WL, Chen T P. R+nRn+-global stability of a Cohen-Grossberg neural network system with nonnegative equilibria. Neural Networks, 2007, 20(6): 714-722

47 Forti M, Nistri P, Quincampoix M. Convergence of neural networks for programming problems via a nonsmooth Lojasiewicz inequality. IEEE Transactions on Neural Networks, 2006, 17(6): 1471-1486

49 Bolte J, Daniilidis A, Lewis A. A nonsmooth Morse-Sard theorem for subanalytic functions. Journal of Mathematical Analysis and Applications, 2006, 321(2): 729-740

50 Liu Q, Wang J. A one-layer recurrent neural network with a discontinuous hard-limiting activation function for quadratic programming. IEEE Transactions on Neural Networks, 2008, 19(4): 558-570

51 Liu Q, Wang J. A one-layer recurrent neural network with a discontinuous activation function for linear programming. Neural Computation, 2008, 20(5): 1366-1383

52 Belykh I, Belykh V, Hasler M. Blinking model and synchronization in small-world networks with a time-varying coupling. Physica D: Nonlinear Phenomena, 2004, 195(1-2): 188-206

54 Pavlov A, Pogromsky A, van de Wouw N, Nijmeijer H. On convergence properties of piecewise affine systems. International Journal of Control, 2007, 80(8): 1233-1247

55 van de Wouw N, Pavlov A. Tracking and synchronisation for a class of PWA systems. Automatica, 2008, 44(11): 2909-2915