Approximate Dynamic Programming and Backpropagation on Timescales
Lewis, F. L. and Liu, D.
Timescales calculus allows signals to have both continuous-time and discrete-time properties. It has become an emerging key topic due to many multidisciplinary applications. the timescales calculus is an increasingly relevant and developed area of mathematics with wide-ranging opportunities for application. in addition to showing its applicability to backpropagation, this chapter shows that the dynamic programming algorithm, derived from Bellman's principle of optimality, obtains on timescales. Also derived is the HJB equation on timescales. Furthermore, ordered derivatives and the backpropagation update rule have been established using the emerging mathematical field of timescales calculus. This calculus unifies the discrete and continuous domains, so our results provide a complete theoretical framework for discussing learning in connectionist systems, which can admit input signals of any type.
J. E. Seiffertt and D. C. Wunsch, "Approximate Dynamic Programming and Backpropagation on Timescales," Reinforcement Learning and Approximate Dynamic Programming for Feedback Control, pp. 474-493, Wiley-IEEE Press, Jan 2012.
The definitive version is available at https://doi.org/10.1002/9781118453988.ch21
Electrical and Computer Engineering
International Standard Book Number (ISBN)
Book - Chapter
© 2012 Wiley-IEEE Press, All rights reserved.