×
Monti, Andrea; Nortmann, Benita; Mylvaganam, Thulasi; Sassano, Mario

Feedback and open-loop Nash equilibria for LQ infinite-horizon discrete-time dynamic games. (English) Zbl 1537.91064

SIAM J. Control Optim. 62, No. 3, 1417-1436 (2024).
Summary: We consider dynamic games defined over an infinite horizon, characterized by linear, discrete-time dynamics and quadratic cost functionals. Considering such linear-quadratic dynamic games, we focus on their solutions in terms of Nash equilibrium strategies. Both feedback (F-NE) and open-loop (OL-NE) Nash equilibrium solutions are considered. The contributions of the paper are threefold. First, our detailed study reveals some interesting structural insights in relation to F-NE solutions. Second, as a stepping stone toward our consideration of OL-NE strategies, we consider a specific infinite-horizon discrete-time (single-player) optimal control problem, wherein the dynamics are influenced by a known exogenous input and draw connections between its solution obtained via dynamic programming and Pontryagin’s minimum principle. Finally, we exploit the latter result to provide a characterization of OL-NE strategies of the class of infinite-horizon dynamic games. The results and key observations made throughout the paper are illustrated via a numerical example.

MSC:

91A50 Discrete-time games
91A25 Dynamic games
90C39 Dynamic programming
49N10 Linear-quadratic optimal control problems

Keywords:

infinite-horizon dynamic games; dynamic programming; Pontryagin’s minimum principle
Cite Review PDF
Full Text: DOI arXiv

References:

[1] Başar, T. and Bernhard, P., H-infinity Optimal Control and Related Minimax Design Problems: A Dynamic Hame Approach, Springer, New York, 2008. · Zbl 1130.93002
[2] Başar, T. and Olsder, G. J., Dynamic Noncooperative Game Theory, SIAM, Philadelphia, 1998.
[3] Bertsekas, D., Dynamic Programming and Optimal Control: Volume II, 4th ed., Athena Scientific, Nashua, NH, 2012. · Zbl 1298.90001
[4] Cappello, D., Garcin, S., Mao, Z., Sassano, M., Paranjape, A., and Mylvaganam, T., A hybrid controller for multi-agent collision avoidance via a differential game formulation, IEEE Trans. Control Syst. Technol., 29 (2021), pp. 1750-1757.
[5] Cappello, D. and Mylvaganam, T., Distributed differential games for control of multi-agent systems, IEEE Trans. Control Netw. Syst., 9 (2022), pp. 635-646.
[6] Chen, F. and Ren, W., On the control of multi-agent systems: A survey, Found. Trends Syst. Control, 6 (2019), pp. 339-499.
[7] Engwerda, J., The solution of the infinite horizon tracking problem for discrete time systems possessing an exogenous component, J. Econ. Dyn. Control, 14 (1990), pp. 741-762. · Zbl 0722.90014
[8] Engwerda, J., LQ Dynamic Optimization and Differential Games, John Wiley & Sons, New York, 2005.
[9] Engwerda, J., Algorithms for computing Nash equilibria in deterministic LQ games, Comput. Manag. Sci., 4 (2007), pp. 113-140. · Zbl 1134.91333
[10] Etesami, S. R. and Başar, T., Dynamic games in cyber-physical security: An overview, Dyn. Games Appl., 9 (2019), pp. 884-913. · Zbl 1431.91052
[11] Freiling, G., Jank, G., and Abou-Kandil, H., Discrete-time Riccati equations in open-loop Nash and Stackelberg games, Eur. J. Control, 5 (1999), pp. 56-66. · Zbl 1093.91505
[12] Golub, G. H. and Van Loan, C. F., Matrix Computations, 3rd ed., John Hopkins University Press, Baltimore, 1996. · Zbl 0865.65009
[13] Jaśkiewicz, A. and Nowak, A. S., A note on topological aspects in dynamic games of resource extraction and economic growth theory, Games Econ. Behav., 131 (2022), pp. 264-274. · Zbl 1483.91046
[14] Jaśkiewicz, A. and Nowak, A. S., Stochastic games with unbounded payoffs: Applications to robust control in economics, Dyn. Games Appl., 1 (2011), pp. 253-279. · Zbl 1263.91008
[15] Jørgensen, S., Martín-Herrán, G., and Zaccour, G., Dynamic games in the economics and management of pollution, Environ. Model. Assess., 15 (2010), pp. 433-467.
[16] Kordonis, I., Lagos, A.-R., and Papavassilopoulos, G. P., Dynamic games of social distancing during an epidemic: Analysis of asymmetric solutions, Dyn. Games Appl., 12 (2022), pp. 214-236. · Zbl 1489.91043
[17] Kremer, D., Non-Symmetric Riccati Theory and Noncooperative Games, Ph.D. thesis, RWTH Aachen, , 2003. · Zbl 1111.91302
[18] Kučera, V., The discrete Riccati equation of optimal control, Kybernetika, 8 (1972), pp. 430-447. · Zbl 0245.93033
[19] Lewis, F. L., Vrabie, D., and Syrmos, V. L., Optimal Control, 3rd ed., John Wiley & Sons, New York, 2012. · Zbl 1284.49001
[20] Li, Y., Carboni, G., Gonzalez, F., Campolo, D., and Burdet, E., Differential game theory for versatile physical human-robot interaction, Nat. Mach. Intell., 1 (2019), pp. 36-43.
[21] Limebeer, D., Anderson, B., and Hendel, B., Mixed \(H_2/H_\infty\) filtering by the theory of Nash games, in Robust Control, Springer, New York, 1992, pp. 9-15. · Zbl 0795.93096
[22] Linan, H. and Quanyan, Z., A dynamic games approach to proactive defense strategies against advanced persistent threats in cyber-physical systems, Comput. Security, 89 (2020), 101660.
[23] Long, N. V., Dynamic games in the economics of natural resources: A survey, Dyn. Games Appl., 1 (2011), pp. 115-148. · Zbl 1214.91085
[24] Mylvaganam, T., Sassano, M., and Astolfi, A., A differential game approach to multi-agent collision avoidance, IEEE Trans. Automat. Control, 62 (2017), pp. 4229-4235. · Zbl 1375.91034
[25] Nadini, M., Zino, L., Rizzo, A., and Porfiri, M., A multi-agent model to study epidemic spreading and vaccination strategies in an urban-like environment, Appl. Network Sci., 5 (2020), pp. 68-97.
[26] Nagata, T. and Sasaki, H., A multi-agent approach to power system restoration, IEEE Trans. Power Syst., 17 (2002), pp. 457-462.
[27] Nortmann, B. and Mylvaganam, T., Data-driven cost representation for optimal control and its relevance to a class of asymmetric linear quadratic dynamic games, in Proceedings of the European Control Conference, , 2022, pp. 2185-2190.
[28] Sassano, M., Mylvaganam, T., and Astolfi, A., On the analysis of open-loop Nash equilibria admitting a feedback synthesis in nonlinear differential games, Automatica, 142 (2022), 110389. · Zbl 1525.91047
[29] Singh, V. P., Kishor, N., and Samuel, P., Distributed multi-agent system-based load frequency control for multi-area power system in smart grid, IEEE Trans. Ind. Electron., 64 (2017), pp. 5151-5160.
[30] Starr, A. W. and Ho, Y.-C., Further properties of nonzero-sum differential games, J. Optim. Theory Appl., 3 (1969), pp. 207-219. · Zbl 0169.12303
[31] Starr, A. W. and Ho, Y.-C., Nonzero-sum differential games, J. Optim. Theory Appl., 3 (1969), pp. 184-206. · Zbl 0169.12301
[32] Sweriduk, G. and Calise, A., A differential game approach to the mixed \(H_2/H_\infty\) problem, in Proceedings of the Guidance, Navigation, and Control Conference, , 1994, pp. 1072-1082.
[33] Wrzos-Kaminska, M., Mylvaganam, T., Pettersen, K. Y., and Tommy Gravdahl, J., Collision avoidance using mixed \(H_2/H_\infty\) control for an articulated intervention-AUV, in Proceedings of the European Control Conference, , 2020, pp. 881-888.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.