Abstract
This paper presents an explainable deep-reinforcement learning (DRL)-based safety-aware optimal adaptive tracking (SOAT) scheme for a class of nonlinear discrete-time (DT) affine systems subject to state inequality constraints. The DRL-based SOAT utilizes a multilayer neural network (MNN)-based actor-critic to estimate the cost function and optimal policy while the MNN update laws are tuned both using the singular value decomposition (SVD) of activation function gradient in order to mitigate the vanishing gradient issue and safety-aware Bellman error at each layer. An approximate safety-aware optimal policy is developed using Karush–Kuhn–Tucker (KKT) conditions by incorporating the higher-order control barrier function (HOCBF) into the Hamiltonian through the Lagrangian multiplier. The resulting safety-aware Bellman error helps with safe exploration both during online learning phase and at steady state without any explicit actor-critic MNN update law changes. To study the explainability and gain insights, we employ the Shapley Additive Explanations (SHAP) method to construct an explainer model for the DRL-based SOAT scheme in order to identify the important features in determining the optimal policy. The overall stability is established. Finally, the effectiveness of the proposed method is demonstrated on Shipboard Power Systems (SPS), achieving over a 35% reduction in cumulative cost compared to the existing actor-critic MNN optimal control policy.
Recommended Citation
B. Farzanegan and S. Jagannathan, "Explainable and Safety Aware Deep Reinforcement Learning-based Control of Nonlinear Discrete-Time Systems using Neural Network Gradient Decomposition," IEEE Transactions on Automation Science and Engineering, Institute of Electrical and Electronics Engineers, Jan 2025.
The definitive version is available at https://doi.org/10.1109/TASE.2025.3554431
Department(s)
Electrical and Computer Engineering
Second Department
Computer Science
Keywords and Phrases
Approximate dynamic programming; Deep reinforcement learning; Explainability; Online safety-aware control; Singular value decomposition-based weight tuning
International Standard Serial Number (ISSN)
1558-3783; 1545-5955
Document Type
Article - Journal
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2025 Institute of Electrical and Electronics Engineers, All rights reserved.
Publication Date
01 Jan 2025
Included in
Computer Sciences Commons, Electrical and Computer Engineering Commons, Medicine and Health Sciences Commons
