Adaptive Critic Design-Based Reinforcement Learning Approach in Controlling Virtual Inertia-Based Grid-Connected Inverters

Abstract

In this paper, an adaptive critic design (ACD) approach is proposed to control the phase and voltage of a grid-connected virtual synchronous generator (VSG). The penetration of fast responding inertia-less power converters significantly affect the stability of the power system, especially weak systems such as micro grids. The concept of virtual inertia addresses this concern by virtually emulating the behavior of a synchronous generator. However, the conventional VSG is designed based on two conditions: (i) fixed operating point and (ii) inductive grid connections. The performance of VSGs in low-voltage semi-resistive microgrids is far from optimal. To overcome the aforementioned concerns, a heuristic dynamic programing (HDP) approach is proposed to optimally control grid-connected VSGs. The neural-network-based inherence of the HDP enables the proposed technique to adapt to any impedance angle. The HDP controller includes two subnetworks: (i) the action network that controls the system optimally and (ii) the critic network, which evaluates the effectiveness of the action network. The simulation and experimental results are provided to evaluate the effectiveness of the proposed technique. As shown, the HDP-based approach illustrates a better performance in comparison with the conventional PI-based VSG in various operating conditions.

Department(s)

Electrical and Computer Engineering

Research Center/Lab(s)

Intelligent Systems Center

Comments

U.S. Department of Energy, Grant DE-EE0008449

Keywords and Phrases

Adaptive Critic Design; Approximate Dynamic Programming; DC/AC Converters; Heuristic Dynamic Programming; Machine Learning; Reinforcement Learning; Synchronverter; Virtual Inertia

International Standard Serial Number (ISSN)

0142-0615

Document Type

Article - Journal

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2020 Elsevier, All rights reserved.

Publication Date

01 May 2021

Link to Full Text

Share

 
COinS