Stochastic Optimal Control of Unknown Linear Networked Control System in the Presence of Random Delays and Packet Losses

Abstract

In this paper, the stochastic optimal control of linear networked control system (NCS) with uncertain system dynamics and in the presence of network imperfections such as random delays and packet losses is derived. the proposed stochastic optimal control method uses an adaptive estimator (AE) and ideas from Q-learning to solve the infinite horizon optimal regulation of unknown NCS with time-varying system matrices. Next, a stochastic suboptimal control scheme which uses AE and Q-learning is introduced for the regulation of unknown linear time-invariant NCS that is derived using certainty equivalence property. Update laws for online tuning the unknown parameters of the AE to obtain the Q-function are derived. Lyapunov theory is used to show that all signals are asymptotically stable (AS) and that the estimated control signals converge to optimal or suboptimal control inputs. Simulation results are included to show the effectiveness of the proposed schemes. the result is an optimal control scheme that operates forward-in-time manner for unknown linear systems in contrast with standard Riccati equation-Based schemes which function backward-in-time. © 2012 Elsevier Ltd. All rights reserved.

Department(s)

Electrical and Computer Engineering

Second Department

Computer Science

Comments

Intelligent Systems Center, Grant 1128281

Keywords and Phrases

Adaptive estimator; Networked control system (NCS); Optimal control; Q-function

International Standard Serial Number (ISSN)

0005-1098

Document Type

Article - Journal

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2024 Elsevier; International Federation of Automatic Control (IFAC), All rights reserved.

Publication Date

01 Jun 2012

Link to Full Text

Share

 
COinS