Adaptive T-S fuzzy controller using reinforcement learning based on Lyapunov stability. (English) Zbl 1398.93191
Summary: In this paper, an adaptive Takagi-Sugeno (T-S) fuzzy controller based on reinforcement learning for controlling the nonlinear dynamical systems is proposed. The parameters of the T-S fuzzy system are learned using the reinforcement learning based on the actor-critic method. This on-line learning algorithm improves the controller performance over the time, which it learns from its own faults through the reinforcement signal from the external environment and tries to reinforce the T-S fuzzy system parameters to converge. The updating parameters are developed using the Lyapunov stability criterion. The proposed controller is faster in learning than the T-S fuzzy that parameters learned using the gradient descent method under the same conditions. Moreover, it is able to handle the load changes and the system uncertainties. The test is carried out based on two mathematical models. In addition, the proposed controller is applied practically for controlling a direct current (DC) shunt machine. The results indicate that the response of the proposed controller has a good performance compared with other controllers.
MSC:
| 93C42 | Fuzzy control/observation systems |
| 93C40 | Adaptive control/observation systems |
| 93D05 | Lyapunov and other classical stabilities (Lagrange, Poisson, \(L^p, l^p\), etc.) in control theory |
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 93C10 | Nonlinear systems in control theory |
| 93C20 | Control/observation systems governed by partial differential equations |
Keywords:
adaptive T-S fuzzy controller; reinforcement learning; Lyapunov stability; nonlinear dynamical systemsReferences:
| [1] | Wang, H.; Wang, Z.; Liu, Y. J.; Tong, S., Fuzzy tracking adaptive control of discrete-time switched nonlinear systems, Fuzzy Sets Syst., 316, 35-48, (2017) · Zbl 1392.93028 |
| [2] | Kumar, R.; Srivastava, S.; Gupta, J. R.P., Diagonal recurrent neural network based adaptive control of nonlinear dynamical systems using Lyapunov stability criterion, ISA Trans., 67, 407-427, (2017) |
| [3] | Singh, M.; Srivastava, S.; Gupta, J. R.P.; Handmandlu, M., Identification and control of a nonlinear system using neural networks by extracting the system dynamics, IETE J. Res., 53, 43-50, (2007) |
| [4] | Chemachema, M., Output feedback direct adaptive neural network control for uncertain SISO nonlinear systems using a fuzzy estimator of the control error, Neural Netw., 36, 25-34, (2012) · Zbl 1258.93067 |
| [5] | Vaidyanathan, S., Adaptive integral sliding mode control of a chemical chaotic reactor system, 371-392, (2017), Springer International Publishing |
| [6] | Liu, Y. J.; Gao, Y.; Tong, S.; Li, Y., Fuzzy approximation-based adaptive backstepping optimal control for a class of nonlinear discrete-time systems with dead-zone, IEEE Trans. Fuzzy Syst., 24, 16-28, (2016) |
| [7] | Sabzi, H. Z.; Humberson, D.; Abudu, S.; King, J. P., Optimization of adaptive fuzzy logic controller using novel combined evolutionary algorithms, and its application in diez Lagos flood controlling system, Exp. Syst. Appl., 43, 154-164, (2016) |
| [8] | Martínez-Soto, R.; Castillo, O.; Castro, J. R., Genetic algorithm optimization for type-2 non-singleton fuzzy logic controllers, Recent Advances on Hybrid Approaches For Designing Intelligent Systems, 3-18, (2014), Springer International Publishing |
| [9] | Hu, M.; Wu, T.; Weir, J. D., An adaptive particle swarm optimization with multiple adaptive methods, IEEE Trans. Evol. Comput., 17, 705-720, (2013) |
| [10] | Zhang, C.; Zou, W.; Cheng, N.; Gao, J., Trajectory tracking control for rotary steerable systems using interval type-2 fuzzy logic and reinforcement learning, J. Frankl. Inst., 355, 803-826, (2018) · Zbl 1384.93074 |
| [11] | Jiang, H.; Zhang, H.; Luo, Y.; Wang, J., Optimal tracking control for completely unknown nonlinear discrete-time Markov jump systems using data-based reinforcement learning method, Neurocomputing, 194, 176-182, (2016) |
| [12] | Murray, J. J.; Cox, C. J.; Lendaris, G. G.; Saeks, R., Adaptive dynamic programming, IEEE Trans. Syst. Man Cybern., 32, 140-153, (2002) |
| [13] | Radac, M. B.; Precup, R. E.; Roman, R. C., Model-free control performance improvement using virtual reference feedback tuning and reinforcement Q-learning, J. Syst. Sci., 48, 1071-1083, (2017) · Zbl 1362.93062 |
| [14] | Syafiie, S.; Tadeo, F.; Martinez, E., Model-free learning control of neutralization processes using reinforcement learning, Eng. Appl. Artif. Intell., 20, 767-782, (2007) |
| [15] | Boubertakh, H.; Tadjine, M.; Glorennec, P. Y.; Labiod, S., Tuning fuzzy PD and PI controllers using reinforcement learning, ISA Trans., 49, 543-551, (2010) |
| [16] | Khater, A. A.; El-Bardini, M.; El-Rabaie, N. M., Embedded adaptive fuzzy controller based on reinforcement learning for dc motor with flexible shaft, Arab. J. Sci. Eng., 40, 2389-2406, (2015) · Zbl 1390.93475 |
| [17] | Wang, F. Y.; Jin, N.; Liu, D.; Wei, Q., Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with ε-error bound, IEEE Trans. Neural Netw., 22, 24-36, (2011) |
| [18] | Lewis, L. F.; Liu, D., Reinforcement Learning and Approximate Dynamic Programming for Feedback Control, 17, (2013), John Wiley & Sons |
| [19] | Hendzel, Z.; Szuster, M., Discrete neural dynamic programming in wheeled mobile robot control, Commun. Nonlinear Sci. Numer. Simul., 16, 2355-2362, (2011) · Zbl 1221.70043 |
| [20] | Zhang, J.; Zhang, H.; Luo, Y.; Feng, T., Model-free optimal control design for a class of linear discrete-time systems with multiple delays using adaptive dynamic programming, Neurocomputing, 135, 163-170, (2014) |
| [21] | Wang, D.; Liu, D.; Wei, Q.; Zhao, D., Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming, Automatica, 48, 1825-1832, (2012) · Zbl 1269.49042 |
| [22] | Zhong, X.; He, H.; Zhang, H.; Wang, Z., A neural network based online learning and control approach for Markov jump systems, Neurocomputing, 149, 116-123, (2015) |
| [23] | Song, R.; Xiao, W.; Zhang, H.; Sun, C., Adaptive dynamic programming for a class of complex-valued nonlinear systems, IEEE Trans. Neural Netw. Learn. Syst., 25, 1733-1739, (2014) · Zbl 1313.94017 |
| [24] | Si, J.; Wang, Y. T., Online learning control by association and reinforcement, IEEE Trans. Neural Netw., 12, 264-276, (2001) |
| [25] | Huang, X.; Naghdy, F.; Du, H.; Naghdy, G.; Todd, C., Reinforcement learning neural network (RLNN) based adaptive control of fine hand motion rehabilitation robot, (Proceedings of the IEEE Conference on Control Applications (CCA), (2015)), 941-946 |
| [26] | Shen, H.; Guo, C., Path-following control of underactuated ships using actor-critic reinforcement learning with MLP neural networks, (Proceedings of the Information Science and Technology (ICIST) IEEE Conference, (2016)), 317-321 |
| [27] | Niedzwiedz, C.; Elhanany, I.; Liu, Z.; Livingston, S., A consolidated actor-critic model with function approximation for high-dimensional pomdps, (Proceedings of the AAAI Workshop for Advancement in POMDP Solvers (part of the AAAI Conference), (2008)), 37-42 |
| [28] | He, H.; Ni, Z.; Fu, J., A three-network architecture for on-line learning and optimization based on adaptive dynamic programming, Neurocomputing, 78, 3-13, (2012) |
| [29] | Ni, Z.; Tang, Y.; Sui, X.; He, H.; Wen, J., An adaptive neuro-control approach for multi-machine power systems, Int. J. Electr. Power Energy Syst., 75, 108-116, (2016) |
| [30] | Lv, Y.; Na, J.; Ren, X., Online H∞ control for completely unknown nonlinear systems via an identifier-critic-based ADP structure, Int. J. Control, 1-12, (2017) |
| [31] | Wang, X. S.; Cheng, Y. H.; Wei, S., A proposal of adaptive PID controller based on reinforcement learning, J. China Univ. Min. Technol., 17, 40-44, (2007) |
| [32] | Sedighizadeh, M.; Rezazadeh, A., Adaptive PID controller based on reinforcement learning for wind turbine control, (Proceedings of World Academy of Science, Engineering and Technology, 27, (2008)), 257-262 |
| [33] | Rezazadeh, A.; Sedighizadeh, M., A modified adaptive wavelet PID control based on reinforcement learning for wind energy conversion system control, Adv. Electr. Comput. Eng., 10, 153-159, (2010) |
| [34] | Chen, C. H., Design of TSK-type fuzzy controllers using differential evolution with adaptive mutation strategy for nonlinear system control, Appl. Math. Comput., 219, 8277-8294, (2013) · Zbl 1287.93048 |
| [35] | Sugeno, M., On stability of fuzzy systems expressed by fuzzy rules with singleton consequents, IEEE Trans. Fuzzy Syst., 7, 201-224, (1999) |
| [36] | Hoffmann, F.; Nelles, O., Genetic programming for model selection of TSK-fuzzy systems, Inf. Sci., 136, 7-28, (2001) · Zbl 0996.68155 |
| [37] | Damousis, I. G.; Alexiadis, M. C.; Theocharis, J. B.; Dokopoulos, P. S., A fuzzy model for wind speed prediction and power generation in wind parks using spatial correlation, IEEE Trans. Energy Convers., 19, 352-361, (2004) |
| [38] | Kabziński, J.; Kacerka, J., TSK fuzzy modeling with nonlinear consequences, (Proceedings of the IFIP International Conference on Artificial Intelligence Applications and Innovations, (2014), Springer), 498-507 |
| [39] | Feng, G., A survey on analysis and design of model-based fuzzy control systems, IEEE Trans. Fuzzy Syst., 14, 676-697, (2006) |
| [40] | Masmoudi, M. S.; Krichen, N.; Masmoudi, M., A hardware- based implementation of a TSK fuzzy controller for a wheelchair system, Int. J. Robot. Autom., 33, (2018) |
| [41] | Wang, S. Y.; Tseng, C. L.; Chiu, C. J., Online speed controller scheme using adaptive supervisory TSK fuzzy CMAC for vector controlled induction motor drive, Asian J. Control, 17, 569-581, (2015) · Zbl 1332.93218 |
| [42] | Choi, O. K.; Kim, J.; Kim, J.; Lee, J. S., BIBO stability analysis of TSK fuzzy PI/PD control systems, Intell. Autom. Soft Comput., 21, 645-658, (2015) |
| [43] | Begian, M. B.; Melek, W. W.; Mendel, J. M., Parametric design of stable type-2 TSK fuzzy systems, (Proceedings of the Fuzzy Information Processing Society, IEEE, (2008)), 1-6 |
| [44] | Zheng, G.; Wang, J.; Jiang, L., Research on type-2 TSK fuzzy logic systems, Fuzzy Inf. Eng., 2, 491-500, (2009) · Zbl 1188.68294 |
| [45] | Deng, Z.; Choi, K. S.; Cao, L.; Wang, S., T2fela: type-2 fuzzy extreme learning algorithm for fast training of interval type-2 TSK fuzzy logic system, IEEE Trans. Neural Netw. Learn. Syst., 25, 664-676, (2014) |
| [46] | Fan, Q. F.; Wang, T.; Chen, Y.; Zhang, Z. F., Design and application of interval type-2 TSK fuzzy logic system based on QPSO algorithm, Int. J. Fuzzy Syst., 20, 835-846, (2018) |
| [47] | Hyun, C. H.; Park, C. W.; Kim, S., Takagi-sugeno fuzzy model based indirect adaptive fuzzy observer and controller design, Inf. Sci., 180, 2314-2327, (2010) · Zbl 1214.93063 |
| [48] | Smoczek, J.; Szpytko, J., Evolutionary algorithm-based design of a fuzzy TBF predictive model and TSK fuzzy anti-sway crane control system, Eng. Appl. Artif. Intell., 28, 190-200, (2014) |
| [49] | Malhotra, R.; Singh, N.; Singh, Y., Genetic algorithms: concepts, design for optimization of process controllers, Comput. Inf. Sci., 4, 39-54, (2011) |
| [50] | Juang, C. F.; Chang, Y. C., Evolutionary-group-based particle-swarm-optimized fuzzy controller with application to mobile-robot navigation in unknown environments, IEEE Trans. Fuzzy Syst., 19, 379-392, (2011) |
| [51] | Juang, C. F.; Chang, P. H., Designing fuzzy-rule-based systems using continuous ant-colony optimization, IEEE Trans. Fuzzy Syst., 18, 138-149, (2010) |
| [52] | Lin, C. M.; Li, H. Y., TSK fuzzy CMAC-based robust adaptive backstepping control for uncertain nonlinear systems, IEEE Trans. Fuzzy Syst., 20, 1147-1154, (2012) |
| [53] | Bessa, W. M.; Dutra, M. S.; Kreuzer, E., An adaptive fuzzy sliding mode controller for remotely operated underwater vehicles, Robot. Auton. Syst., 58, 16-26, (2010) · Zbl 1203.93106 |
| [54] | Balochian, S.; Ebrahimi, E., Parameter optimization via cuckoo optimization algorithm of fuzzy controller for liquid level control, J. Eng., (2013) |
| [55] | Karaboga, D.; Kaya, E., An adaptive and hybrid artificial bee colony algorithm (aabc) for ANFIS training, Appl. Soft Comput., 49, 423-436, (2016) |
| [56] | Tseng, C. L.; Wang, S. Y.; Chien, S. C.; Chang, C. Y., Development of a self-tuning TSK-fuzzy speed control strategy for switched reluctance motor, IEEE Trans. Power Electron., 27, 2141-2152, (2012) |
| [57] | Lin, F. J.; Lu, K. C.; Ke, T. H.; Yang, B. H.; Chang, Y. R., Reactive power control of three-phase grid-connected PV system during grid faults using Takagi-sugeno-kang probabilistic fuzzy neural network control, IEEE Trans. Ind. Electron., 62, 5516-5528, (2015) |
| [58] | Sun, Y. L.; Er, M. J., Hybrid fuzzy control of robotics systems, IEEE Trans. Fuzzy Syst., 12, 755-765, (2004) |
| [59] | Xie, S. Q.; Jamwal, P. K., An iterative fuzzy controller for pneumatic muscle driven rehabilitation robot, Expert Syst. Appl., 38, 8128-8137, (2011) |
| [60] | Wang, H.; Shi, P.; Lim, C. C., Notice of violation of IEEE publication principles decentralized event-triggering passive control of networked TS fuzzy systems, IEEE Trans. Fuzzy Syst., (2015) |
| [61] | Chang, Y. C.; Chen, C. H.; Zhu, Z. C.; Huang, Y. W., Speed control of the surface-mounted permanent-magnet synchronous motor based on Takagi-sugeno fuzzy models, IEEE Trans. Power Electron., 31, 6504-6510, (2016) |
| [62] | Azar, A. T.; Hassanien, A. E., Dimensionality reduction of medical big data using neural-fuzzy classifier, Soft Comput., 19, 1115-1127, (2015) |
| [63] | Lin, D.; Wang, X.; Nian, F.; Zhang, Y., Dynamic fuzzy neural networks modeling and adaptive backstepping tracking control of uncertain chaotic systems, Neurocomputing, 73, 2873-2881, (2010) |
| [64] | Precup, R. E.; Hellendoorn, H., A survey on industrial applications of fuzzy control, Comput. Ind., 62, 213-226, (2011) |
| [65] | Soliman, M.; Elshafei, A. L.; Bendary, F.; Mansour, W., LMI static output-feedback design of fuzzy power system stabilizers, Expert Syst. Appl., 36, 6817-6825, (2009) |
| [66] | Lilly, J. H., Fuzzy control and identification, (2011), John Wiley & Sons. |
| [67] | Wang, D.; Ding, F., Least squares based and gradient based iterative identification for Wiener nonlinear systems, Signal Process., 91, 1182-1189, (2011) · Zbl 1219.94052 |
| [68] | Passino, K. M.; Yurkovich, S.; Reinfrank, M., Fuzzy control, 20, (1998), Addison-wesley Menlo Park, CA |
| [69] | Denaï, M. A.; Palis, F.; Zeghbib, A., Modeling and control of non-linear systems using soft computing techniques, Appl. Soft Comput., 7, 728-738, (2007) |
| [70] | Chen, C. W., Stability analysis and robustness design of nonlinear systems: an NN-based approach, Appl. Soft Comput., 11, 2735-2742, (2011) |
| [71] | Kumar, R.; Srivastava, S.; Gupta, J. R.P.; Mohindru, A., Diagonal recurrent neural network based identification of nonlinear dynamical systems with Lyapunov stability based adaptive learning rates, Neurocomputing, 287, 102-117, (2018) |
| [72] | Narendra, K. S.; Parthasarathy, K., Identification and control of dynamical systems using neural networks, IEEE Trans. Neural Netw., 1, 4-27, (1990) |
| [73] | Kayacan, E.; Kayacan, E.; Khanesar, M. A., Identification of nonlinear dynamic systems using type-2 fuzzy neural networks—a novel learning algorithm and a comparative study, IEEE Trans. Ind. Electron., 62, 1716-1724, (2015) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
