Stochastic Optimal Control of Unknown Linear Networked Control System using Q-learning Methodology
Abstract
In this paper, the Bellman equation is utilized forward-in-time for the stochastic optimal control of Networked Control System (NCS) with unknown system dynamics in the presence of random delays and packet losses which are unknown. the proposed stochastic optimal control approach, referred normally as adaptive dynamic programming, uses an adaptive estimator (AE) and ideas from Q-learning to solve the infinite horizon optimal regulation control of NCS with unknown system dynamics. Update laws for tuning the unknown parameters of the adaptive estimator (AE) online to obtain the time-Based Q-function are derived. Lyapunov theory is used to show that all signals are asymptotically stable (AS) and that the approximated control signals converge to optimal control inputs. Simulation results are included to show the effectiveness of the proposed scheme. © 2011 AACC American Automatic Control Council.
Recommended Citation
H. Xu and S. Jagannathan, "Stochastic Optimal Control of Unknown Linear Networked Control System using Q-learning Methodology," Proceedings of the American Control Conference, pp. 2819 - 2824, article no. 5991278, Institute of Electrical and Electronics Engineers, Sep 2011.
Department(s)
Electrical and Computer Engineering
Second Department
Computer Science
Keywords and Phrases
Adaptive Estimator (AE); Networked Control System (NCS); Optimal Control; Q-function
International Standard Book Number (ISBN)
978-145770080-4
International Standard Serial Number (ISSN)
0743-1619
Document Type
Article - Conference proceedings
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2024 Institute of Electrical and Electronics Engineers, All rights reserved.
Publication Date
29 Sep 2011
