×

Multi-event-triggered adaptive critic control with guaranteed cost for discrete-time nonlinear nonzero-sum games. (English) Zbl 1530.93286

Summary: In this article, a new event-based adaptive critic algorithm with multiple triggering conditions is investigated to address multi-player nonzero-sum game problems for discrete-time nonlinear dynamics. In order to improve resource utilization while ensure mutual independence among players, the corresponding novel triggering conditions are designed for each player. The corresponding control input is updated only when the relevant triggering condition is violated. It is emphasized that these triggering conditions are established based on the iteration of the time-triggered mechanism. Then, according to the setting triggering conditions, we prove that the real cost function possesses a predetermined upper bound, which realizes the cost guarantee of the controlled system. Additionally, the multi-player closed-loop system is proved to be asymptotically stable and the multi-event-triggered control method is implemented by constructing three kinds of neural networks. Finally, the effectiveness of the developed multi-event-triggered control approach is verified through conducting two simulation examples.
{© 2022 John Wiley & Sons Ltd.}

MSC:

93C65 Discrete event control/observation systems
93C55 Discrete-time control/observation systems
93C10 Nonlinear systems in control theory
91A06 \(n\)-person games, \(n>2\)
Full Text: DOI

References:

[1] NiuB, LiuJ, WangD, ZhaoX, WangH. Adaptive decentralized asymptotic tracking control for large‐scale nonlinear systems with unknown strong interconnections. IEEE/CAA J Automat Sin. 2022;9(1):173‐186.
[2] ZhangH, ZhangJ, YangG‐H, LuoY. Leader‐based optimal coordination control for the consensus problem of multiagent differential games via fuzzy adaptive dynamic programming. IEEE Trans Fuzzy Syst. 2015;23(1):152‐163.
[3] SongR, WeiQ, ZhangH, LewisFL. Discrete‐time non‐zero‐sum games with completely unknown dynamics. IEEE Trans Cybern. 2021;51(6):2929‐2943.
[4] WangG, YuZ. A Pontryagin’s maximum principle for nonzero sum differential games of BSDEs with applications. IEEE Trans Automat Contr. 2010;55(7):1742‐1747. · Zbl 1368.91035
[5] MathiyalaganK, SuH, ShiP, SakthivelR. Exponential
[( {H}_{\infty } \]\) filtering for discrete‐time switched neural networks with random delays. IEEE Trans Cybern. 2015;45(4):676‐687.
[6] BasarT, BernhardP. H∞ optimal control and related minimax design problems: a dynamic game approach. IEEE Trans Automat Contr. 1996;41(9):1397‐1399.
[7] DongJ, HouQ, RenM. Control synthesis for discrete‐time T‐S fuzzy systems based on membership function‐dependent
[( {H}_{\infty } \]\) performance. IEEE Trans Fuzzy Syst. 2020;28(12):3360‐3366.
[8] LewisFL, VrabieD. Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits Syst Mag. 2009;9(3):32‐50.
[9] ZhaoD, ZhuY. MEC-A near‐optimal online reinforcement learning algorithm for continuous deterministic systems. IEEE Trans Neural Netw Learn Syst. 2015;26(2):346‐356.
[10] WerbosPJ. Approximate dynamic programming for real‐time control and neural modeling. In: WhiteDA (ed.), SofgeDA (ed.), eds. Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches. Vol ch. 13. Van Nostrand Reinhold; 1992.
[11] WangD, LiuD, WeiQ, ZhaoD, JinN. Optimal control of unknown nonaffine nonlinear discrete‐time systems based on adaptive dynamic programming. Automatica. 2012;48(8):1825‐1832. · Zbl 1269.49042
[12] ProkhorovDV, WunschDC. Adaptive critic designs. IEEE Trans Neural Netw. 1997;8(5):997‐1007.
[13] WangD, HaM, ZhaoM. The intelligent critic framework for advanced optimal control. Artif Intell Rev. 2022;55(1):1‐22.
[14] KiumarsiB, VamvoudakisKG, ModaresH, LewisFL. Optimal and autonomous control using reinforcement learning: a aurvey. IEEE Trans Neural Netw Learn Syst. 2018;29(6):2042‐2062.
[15] HeydariA. Stability analysis of optimal adaptive control under value iteration using a stabilizing initial policy. IEEE Trans Neural Netw Learn Syst. 2018;29(9):4522‐4527.
[16] Al‐TamimiA, LewisFL, Abu‐KhalafM. Discrete‐time nonlinear HJB solution using approximate dynamic programming: convergence proof. IEEE Trans Syst Man Cybern B (Cybern). 2008;38(4):943‐949.
[17] LiuD, WeiQ. Generalized policy iteration adaptive dynamic programming for discrete‐time nonlinear systems. IEEE Trans Syst Man Cybern Syst. 2015;45(12):1577‐1591.
[18] GuoW, SiJ, LiuF, MeiS. Policy approximation in policy iteration approximate dynamic programming for discrete‐time nonlinear systems. IEEE Trans Neural Netw Learn Syst. 2018;29(7):2794‐2807.
[19] ModaresH, LewisFL, Naghibi‐SistaniM‐B. Adaptive optimal control of unknown constrained‐input systems using policy iteration and neural networks. IEEE Trans Neural Netw Learn Syst. 2013;24(10):1513‐1525.
[20] WangD, ZhaoM, QiaoJ. Intelligent optimal tracking with asymmetric constraints of a nonlinear wastewater treatment system. Int J Robust Nonlinear Control. 2021;31(14):6773‐6787. · Zbl 1527.93223
[21] WangD, ZhaoM, HaM, RenJ. Neural optimal tracking control of constrained nonaffine systems with a wastewater treatment application. Neural Netw. 2021;143:121‐132. · Zbl 1526.93141
[22] KiumarsiB, LewisFL. Actor‐critic‐based optimal tracking for partially unknown nonlinear discrete‐time systems. IEEE Trans Neural Netw Learn Syst. 2015;26(1):140‐151.
[23] HaM, WangD, LiuD. Data‐based nonaffine optimal tracking control using iterative DHP approach. Proceedings of 21st IFAC World Congress; July 2020:4246‐4251.
[24] ModaresH, LewisFL, JiangZ‐P. H∞ tracking control of completely unknown continuous‐time systems via off‐policy reinforcement learning. IEEE Trans Neural Netw Learn Syst. 2015;26(10):2550‐2562.
[25] XuH, JagannathanS, LewisFL. Stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses. Automatica. 2012;48(6):1017‐1030. · Zbl 1244.93177
[26] WangD, XuX, ZhaoM. Neural critic learning toward robust dynamic stabilization. Int J Robust Nonlinear Control. 2020;30(5):2020‐2032. · Zbl 1465.93174
[27] YangY, GaoW, ModaresH, XuCZ. Robust actor‐critic learning for continuous‐time nonlinear systems with unmodeled dynamics. IEEE Trans Fuzzy Syst. 2022;30(6):2101‐2112.
[28] WangD, HuL, ZhaoM, QiaoJ. Adaptive critic for event‐triggered unknown nonlinear optimal tracking design with wastewater treatment applications. IEEE Trans Neural Netw Learn Syst. 2021. doi:10.1109/TNNLS.2021.3135405
[29] FanQ, YangG. Event‐based fuzzy adaptive fault‐tolerant control for a class of nonlinear systems. IEEE Trans Fuzzy Syst. 2018;26(5):2686‐2698.
[30] ZhaoF, GaoW, LiuT, JiangZ‐P. Adaptive optimal output regulation of linear discrete‐time systems based on event‐triggered output‐feedback. Automatica. 2022;137:110103:1‐110103:10. · Zbl 1482.93334
[31] WangD, QiaoJ, ChengL. An approximate neuro‐optimal solution of discounted guaranteed cost control design. IEEE Trans Cybern. 2022;52(1):77‐86.
[32] TallapragadaP, ChopraN. On event triggered tracking for nonlinear systems. IEEE Trans Automat Contr. 2013;58(9):2343‐2348. · Zbl 1369.93256
[33] VamvoudakisKG, MojoodiA, FerrazH. Event‐triggered optimal tracking control of nonlinear systems. Int J Robust Nonlinear Control. 2017;14(4):598‐619. · Zbl 1356.93056
[34] WangX, LemmonMD. Event‐triggering in distributed networked control systems. IEEE Trans Automat Contr. 2011;56(3):586‐601. · Zbl 1368.93211
[35] HeemelsWPMH, DonkersMCF, TeelAR. Periodic event‐triggered control for linear systems. IEEE Trans Automat Contr. 2013;58(4):847‐861. · Zbl 1369.93363
[36] WangD, HaM, QiaoJ. Self‐learning optimal regulation for discrete‐time nonlinear systems under event‐driven formulation. IEEE Trans Automat Contr. 2020;65(3):1272‐1279. · Zbl 1533.93653
[37] HaM, WangD, LiuD. Event‐triggered adaptive critic control design for discrete‐time constrained nonlinear systems. IEEE Trans Syst Man Cybern Syst. 2020;50(9):3158‐3168.
[38] SahooA, XuH, JagannathanS. Near optimal event‐triggered control of nonlinear discrete‐time systems using neurodynamic programming. IEEE Trans Neural Netw Learn Syst. 2016;27(9):1801‐1815.
[39] XueS, LuoB, LiuD, YangY. Constrained event‐triggered
[( {H}_{\infty } \]\) control based on adaptive dynamic programming with concurrent learning. IEEE Trans Syst Man Cybern Syst. 2022;52(1):357‐369.
[40] ZhangQ, ZhaoD, ZhuY. Event‐triggered
[( {H}_{\infty } \]\) control for continuous‐time nonlinear system via concurrent learning. IEEE Trans Syst Man Cybern Syst. 2017;47(7):1071‐1081.
[41] BatmaniY, DavoodiM, MeskinN. Event‐triggered suboptimal tracking controller design for a class of nonlinear discrete‐time systems. IEEE Trans Ind Electron. 2017;64(10):8079‐8087.
[42] LuoB, YangY, LiuD, WuH‐N. Event‐triggered optimal control with performance guarantees using adaptive dynamic programming. IEEE Trans Neural Netw Learn Syst. 2020;31(1):76‐88.
[43] LuJ, WeiQ, LiuY, ZhouT, WangF‐Y. Event‐triggered optimal parallel tracking control for discrete‐time nonlinear systems. IEEE Trans Syst Man Cybern Syst. 2022;52(6):3772‐3784.
[44] ZhangY, ZhaoB, LiuD. Deterministic policy gradient adaptive dynamic programming for model‐free optimal control. Neurocomputing. 2020;387:40‐50.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.