Linear-quadratic stochastic leader-follower differential games for Markov jump-diffusion models. (English) Zbl 1505.91056
Summary: This paper considers the linear-quadratic (LQ) leader-follower Stackelberg differential game for Markov jump-diffusion stochastic differential equations (SDEs). We first obtain the open-loop type optimal solutions for the leader and the follower by establishing the general stochastic maximum principle for (indefinite) LQ control with Markovian jumps. Then we obtain the state-feedback representation of the open-loop type optimal solutions for the leader and the follower in terms of the coupled Riccati differential equations (CRDEs) by generalizing the classical four-step scheme to the case with Markovian jumps. Unlike the existing literature, the four-step scheme in our paper is not symmetric due to the presence of the nonsymmetric quadratic variation induced by the Markov chain. We develop the decoupling approach to find the simplified expression of the corresponding quadratic variation. Under the well-posedness of the CRDEs, the state-feedback type optimal solutions for the leader and the follower constitute the Stackelberg equilibrium. We also show the well-posedness of the CRDEs for the follower under suitable assumptions of the coefficients. Finally, we demonstrate the well-posedness of the CRDEs for the leader and the follower via numerical simulations for both indefinite and definite cost parameter cases.
MSC:
| 91A15 | Stochastic games, stochastic differential games |
| 91A65 | Hierarchical games (including Stackelberg games) |
| 93E20 | Optimal stochastic control |
| 49N10 | Linear-quadratic optimal control problems |
| 60J74 | Jump processes on discrete state spaces |
Keywords:
leader-follower Stackelberg game; Markov jump systems; linear-quadratic control; forward-backward stochastic differential equation; Riccati differential equationReferences:
| [1] | Applebaum, D., Lévy processes and stochastic calculus (2009), Cambridge University Press: Cambridge University Press Cambridge · Zbl 1200.60001 |
| [2] | Barles, G.; Buckdahn, R.; Pardoux, E., Backward stochastic differential equations and integral-partial differential equations, Stochastic Processes and their Applications, 60, 57-83 (1997) · Zbl 0878.60036 |
| [3] | Başar, T.; Olsder, G. J., Dynamic noncooperative game theory (1999), SIAM: SIAM Philadelphia · Zbl 0946.91001 |
| [4] | Başar, T.; Srikant, R., A Stackelberg network game with a large number of followers, Journal of Optimization Theorey and Applications, 115, 3, 479-490 (2002) · Zbl 1031.91016 |
| [5] | Bensoussan, A.; Chen, S.; Sethi, S. P., The maximum principle for global solutions of stochastic Stackelberg differential games, SIAM Journal on Control and Optimization, 53, 4, 1956-1981 (2015) · Zbl 1320.91042 |
| [6] | Chen, C. I.; Cruz, J. B., Stackelberg solution for two-person games with biased information patterns, IEEE Transactions on Automatic Control, 17, 6, 791-798 (1972) · Zbl 0262.90095 |
| [7] | Chizeck, H. J., & Ji, Y. (1998). Optimal Quadratic Control of Jump Linear Systems with Gaussian Noise in Discrete-time. In Proceedings of the 27th conference on decision and control (pp. 1989-1993). Austin, Texas. |
| [8] | Costa, L. V.O.; Fragoso, M. D.; Marques, R. P., Discrete-time Markov jump linear systems (2005), Springer · Zbl 1081.93001 |
| [9] | Costa, L. V.O.; Fragoso, M. D.; Todorov, M. G., Continuous-time Markov jump linear systems (2013), Springer · Zbl 1277.60003 |
| [10] | Dragan, V.; Morozan, T., Stability and robust stabilization to linear stochastic systems described by differential equations with Markovian jumping and multiplicative white noise, Stochastic Analysis and Applications, 20, 1, 33-92 (2002) · Zbl 1136.60335 |
| [11] | Dragan, V.; Morozan, T., Systems of matrix rational differential equations arising in connection with linear stochastic systems with Markovian jumping, Journal of Differential Equations, 194, 1-38 (2003) · Zbl 1039.34052 |
| [12] | Elliot, R. J.; Aggoun, L.; Moore, J. B., Hidden Markov models (1994), Springer |
| [13] | Farias, D. P.; Geromel, J. C.; Val, J. B.R.; Costa, L. V.O., Output feedback control of Markov jump linear systems in continuous-time, IEEE Transactions on Automatic Control, 45, 5, 944-949 (2000) · Zbl 0972.93074 |
| [14] | Feng, X.; Loparo, K. A.; Ji, Y.; Chizeck, H. J., Stochastic stability properties of jump linear systems, IEEE Transactions on Automatic Control, 37, 1, 38-53 (1992) · Zbl 0747.93079 |
| [15] | Freiling, G.; Jank, G.; Lee, R., Existence and uniqueness of open-loop Stackelberg equilibria in linear-quadratic differential games, Journal of Optimization Theory and Applications, 110, 3, 515-544 (2001) · Zbl 0989.49027 |
| [16] | Hanson, F. B., Applied stochastic processes and control for jump-diffusions (2007), SIAM · Zbl 1145.60003 |
| [17] | Horn, R. A.; Johnson, C. R., Matrix analysis (2013), Cambridge University Press: Cambridge University Press New York · Zbl 1267.15001 |
| [18] | Ji, Y.; Chizeck, H. J., Controllability, stabilizability, and continuous-time Markovian jump linear quadratic control, IEEE Transactions on Automatic Control, 35, 7, 777-788 (1990) · Zbl 0714.93060 |
| [19] | Li, L.; Ugrinovskii, V. A., On necessary and sufficient conditions for \(H{}_\infty\) output feedback control of Markov jump linear systems, IEEE Transactions on Automatic Control, 52, 7, 1287-1292 (2007) · Zbl 1366.93158 |
| [20] | Li, N.; Yu, Z., Forward-backward stochastic differential equations and linear-quadratic generalized Stackelberg games, SIAM Journal on Control and Optimization, 56, 6, 4148-4180 (2018) · Zbl 1417.91150 |
| [21] | Lin, Y.; Jiang, X.; Zhang, W., An open-loop Stackelberg strategy for the linear quadratic mean-field stochastic differential game, IEEE Transactions on Automatic Control, 64, 1, 97-110 (2019) · Zbl 1425.91042 |
| [22] | Lv, S., Two-player zero-sum stochastic differential games with regime switching, Automaica, 114, 108819, 1-8 (2020) · Zbl 1441.91010 |
| [23] | Maharjan, S.; Zhu, Q.; Zhang, Y.; Gjessing, S.; Başar, T., Dependable demand response management in the smart grid: A Stackelberg game approach, IEEE Transactions on Smart Grid, 4, 1, 120-132 (2013) |
| [24] | Martion, M., Jump linear systems in automatic control (1990), Marcel Dekker |
| [25] | Moon, J., A sufficient condition for linear-quadratic stochastic zero-sum differential games for Markov jump systems, IEEE Transactions on Automatic Control, 64, 4, 1619-1626 (2019) · Zbl 1482.91018 |
| [26] | Moon, J., Linear-quadratic stochastic Stackelberg differential games for jump-diffusion systems, SIAM Journal on Control and Optimization, 59, 2, 954-976 (2021) · Zbl 1460.91028 |
| [27] | Moon, J., Risk-sensitive maximum principle for stochastic optimal control of mean-field type Markov regime-switching jump-diffusion systems, International Journal of Robust and Nonlinear Control, 31, 2141-2167 (2021) · Zbl 1526.93280 |
| [28] | Moon, J.; Başar, T., Risk-sensitive control of Markov jump linear systems: Caveats and difficulties, International Journal of Control, Automation and Systems, 15, 1, 462-467 (2017) |
| [29] | Moon, J.; Başar, T., Linear quadratic mean field Stackelberg differential games, Automatica, 97, 200-213 (2018) · Zbl 1420.91021 |
| [30] | Moon, J.; Duncan, T. E., A simple proof of indefinite linear-quadratic stochastic optimal control with random coefficients, IEEE Transactions on Automatic Control, 65, 12, 5422-5428 (2020) · Zbl 1536.93987 |
| [31] | Moon, J.; Yang, H. J., Linear-quadratic time-inconsistent mean-field type Stackelberg differential games: Time-consistent open-loop solutions, IEEE Transactions on Automatic Control, 66, 1, 375-382 (2021) · Zbl 1536.91052 |
| [32] | Norris, J. R., Markov chains (2009), Cambridge University Press · Zbl 0873.60043 |
| [33] | Papavassilopoulos, G. P.; Cruz, J. B., Nonclassical control problems and Stackelberg games, IEEE Transactions on Automatic Control, 24, 2, 155-166 (1979) · Zbl 0406.49008 |
| [34] | Shen, Y.; Siu, T. K., The maximum principle for a jump-diffusion mean-field model and its application to the mean-variance problem, Nonlinear Analysis, 86, 58-73 (2013) · Zbl 1279.49015 |
| [35] | Shi, J.; Wang, G.; Xiong, J., Leader-follower stochastic differential game with asymmetric information and applications, Automatica, 63, 60-73 (2016) · Zbl 1329.93157 |
| [36] | Stoica, A.; Yaesh, I., Jump Markovian-based control of wing development for an uncrewed air vehicle, Journal of Guidance, Control, and Dynamics, 25, 2, 407-411 (2002) |
| [37] | Sun, J.; Li, X.; Yong, J., Open-loop and closed-loop solvabilities for stochastic linear-quadratic optimal control problems, SIAM Journal on Control and Optimization, 5, 54, 2274-2308 (2016) · Zbl 1347.49033 |
| [38] | Sun, H.; Yan, L.; Li, L., Linear-quadratic stochastic differential games with Markov jumps and multiplicative noise: Infinite-time case, International Journal of Innovative Computing and Applications, 11, 1, 349-361 (2015) |
| [39] | Tang, S.; Li, X., Necessary conditions for optimal control of stochastic systems with random jumps, SIAM Journal on Control and Optimization, 32, 5, 1447-1475 (1994) · Zbl 0922.49021 |
| [40] | Wonham, W. M., Random differential equations in control theory, (Bharucha-Reid, A., Probabilistic methods in applied mathematics, Vol. 2 (1970), Academic Press: Academic Press San Diego), 131-212 · Zbl 0235.93025 |
| [41] | Xu, J.; Shi, J.; Zhang, H., A leader-follower stochastic linear quadratic differential game with time delay, Science China. Information Sciences, 61, 1-13 (2018) |
| [42] | Yong, J., A leader-follower stochastic linear quadratic differential game, SIAM Journal on Control and Optimization, 41, 4, 1015-1041 (2002) · Zbl 1039.93070 |
| [43] | Yong, J., Linear-quadratic optimal control problems for mean-field stochastic differential equations, SIAM Journal on Control and Optimization, 51, 4, 2809-2838 (2013) · Zbl 1275.49060 |
| [44] | Yong, J.; Zhou, X. Y., Stochastic controls: hamiltonian systems and HJB equations (1999), Springer · Zbl 0943.93002 |
| [45] | Zhang, F.; Dong, Y.; Meng, Q., Backward stochastic Riccati equation with jumps associated with stochastic linear quadratic optimal control with jump and random coefficients, SIAM Journal on Control and Optimization, 58, 1, 393-424 (2020) · Zbl 1435.93181 |
| [46] | Zhang, X.; Elliott, R. J.; Siu, T. K., A stochastic maximum principle of a Markov regime-switching jump-diffusion model and its application to finance, SIAM Journal on Control and Optimization, 50, 2, 964-990 (2012) · Zbl 1244.93180 |
| [47] | Zhang, C.; Li, F., Non-zero sum differential game for stochastic Markovian jump systems with partially unknown transition probabilities, Journal of the Franklin Institute, 358, 15, 7528-7558 (2021) · Zbl 1472.93203 |
| [48] | Zhang, X.; Li, X.; Xiong, J., Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems of Markovian regime switching system, ESAIM. Control, Optimisation and Calculus of Variations, 69, 1-35 (2021) · Zbl 1473.49040 |
| [49] | Zheng, Y.; Shi, J., A Stackelberg game of backward stochastic differential equations with applications, Dynamic Games and Applications, 10, 968-992 (2020) · Zbl 1457.91121 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
