Galerkin approximationwith Legendre polynomials for a continuous-time nonlinear optimal control problem

Abstract

We investigate the use of an approximation method for obtaining near-optimal solutions to a kind of nonlinear continuous-time (CT) system. The approach derived from the Galerkin approximation is used to solve the generalized Hamilton-Jacobi-Bellman (GHJB) equations. The Galerkin approximation with Legendre polynomials (GALP) for GHJB equations has not been applied to nonlinear CT systems. The proposed GALP method solves the GHJB equations in CT systems on some well-defined region of attraction. The integrals that need to be computed are much fewer due to the orthogonal properties of Legendre polynomials, which is a significant advantage of this approach. The stabilization and convergence properties with regard to the iterative variable have been proved. Numerical examples show that the update control laws converge to the optimal control for nonlinear CT systems.

Keywords

Generalized Hamilton-Jacobi-Bellman equation / Nonlinear optimal control / Galerkin approximation / Legendre polynomials

References

[1] Aguilar , C.O., Krener , A.J., 2014. Numerical solutions to the Bellman equation of optimal control.J. Optim. Theory Appl., 160(2):527–552. https://doi.org/10.1007/s10957-013-0403-8

[2] Bardi , M., Capuzzo-Dolcetta , I., 1997. Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkhäuser Boston, Inc., Boston, MA (with appendices by Maurizio Falcone and Pierpaolo Soravia). https://doi.org/10.1007/978-0-8176-4755-1

[3] Beard , R.W., Saridis , G.N., Wen , J.T., 1996. Improving the performance of stabilizing controls for nonlinear systems. IEEE Contr. Syst. Mag. , 16(5):27–35. https://doi.org/10.1109/37.537206

[4] Beard , R.W., Saridis , G.N., Wen , J.T., 1997. Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation. Automatica, 33(12):2159–2177. https://doi.org/10.1016/S0005-1098(97)00128-3

[5] Bellman , R., 1957. Dynamic Programming. Princeton University Press, New Jersey, USA.

[6] Cacace , S., Cristiani , E., Falcone , M., , 2012. A patchy dynamic programming scheme for a class of Hamilton-Jacobi-Bellman equations. SIAM J. Sci. Comput. , 34(5):A2625–A2649. https://doi.org/10.1137/110841576

[7] Canuto , C., Hussaini , M.Y., Quarteroni , A., , 1988. Spectral Methods in Fluid Dynamics. Springer-Verlag, New York, USA.

[8] Gong , Q., Kang , W., Ross , I.M., 2006. A pseudospectral method for the optimal control of constrained feedback linearizable systems. IEEE Trans. Autom. Contr. , 51(7):1115–1129. https://doi.org/10.1109/TAC.2006.878570

[9] Govindarajan , N., de Visser , C.C., Krishnakumar , K., 2014. A sparse collocation method for solving time-dependent HJB equations using multivariate B-splines. Automatica, 50(9):2234–2244. https://doi.org/10.1016/j.automatica.2014.07.012

[10] Isidori , A., 2013. Nonlinear Control Systems. Springer Science & Business Media.

[11] Kirk , D.E., 2012. Optimal Control Theory: an Introduction. Courier Corporation.

[12] Kleinman , D., 1968. On an iterative technique for Riccati equation computations. IEEE Trans. Autom. Contr. , 13(1):114–115. https://doi.org/10.1109/TAC.1968.1098829

[13] Lews , F., Syrmos , V., 1995. Optimal Control. Wiley, New Jersey, USA.

[14] Luo , B., Wu , H.N., Huang , T., , 2014. Databased approximate policy iteration for affine nonlinear continuous-time optimal control design. Automatica, 50(12):3281–3290. https://doi.org/10.1016/j.automatica.2014.10.056

[15] Luo , B., Huang , T., Wu , H.N., , 2015a. Data-driven H_∞ control for nonlinear distributed parameter systems. IEEE Trans. Neur. Netw. Learn. Syst. , 26(11):2949–2961. https://doi.org/10.1109/TNNLS.2015.2461023

[16] Luo , B., Wu , H.N., Huang , T., 2015b. Off-policy reinforcement learning for H_∞ control design. IEEE Trans. Cybern. , 45(1):65–76. https://doi.org/10.1109/TCYB.2014.2319577

[17] Luo , B., Wu , H.N., Li , H.X., 2015c. Adaptive optimal control of highly dissipative nonlinear spatially distributed processes with neuro-dynamic programming. IEEE Trans. Neur. Netw. Learn. Syst. , 26(4):684–696. https://doi.org/10.1109/TNNLS.2014.2320744

[18] Markman , J., Katz , I.N., 2000. An iterative algorithm for solving Hamilton-Jacobi type equations. SIAM J. Sci. Comput. , 22(1):312–329. https://doi.org/10.1137/S1064827598344315

[19] Sakamoto , N., van der Schaft , A.J., 2008. Analytical approximation methods for the stabilizing solution of the Hamilton-Jacobi equation. IEEE Trans. Autom. Contr. , 53(10):2335–2350. https://doi.org/10.1109/TAC.2008.2006113

[20] Saridis , G.N., Lee , C.S.G., 1979. An approximation theory of optimal control for trainable manipulators. IEEE Trans. Syst. Man Cybern. , 9(3):152–159. https://doi.org/10.1109/TSMC.1979.4310171

[21] Smears , I., Süli , E., 2014. Discontinuous Galerkin finite element approximation of Hamilton-Jacobi-Bellman equations with Cordes coefficients. SIAM J. Numer. Anal. , 52(2):993–1016. https://doi.org/10.1137/130909536

[22] Wu , H.N., Luo , B., 2012. Neural network based online simultaneous policy update algorithm for solving the HJI equation in nonlinear control. IEEE Trans. Neur. Netw. Learn. Syst. , 23(12):1884–1895. https://doi.org/10.1109/TNNLS.2012.2217349

[23] Yu , J., Jiang , Z.P., 2015. Global adaptive dynamic programming for continuous-time nonlinear systems. IEEE Trans. Autom. Contr. , 60(11):2917–2929. https://doi.org/10.1109/TAC.2015.2414811

RIGHTS & PERMISSIONS

2017 Zhejiang University and Springer-Verlag GmbH Germany