The time scales calculus, which includes the study of the nabla derivatives, is an emerging key topic due to many multidisciplinary applications. We extend this calculus to Approximate Dynamic Programming. In particular, we investigate application of the nabla derivative, one of the fundamental dynamic derivatives of time scales. We present a nabla-derivative based derivation and proof of the Hamilton-Jacobi-Bellman equation, the solution of which is the fundamental problem in the field of dynamic programming. By drawing together the calculus of time scales and the applied area of stochastic control via Approximate Dynamic Programming, we connect two major fields of research.
J. E. Seiffertt et al., "Decision Theory on Dynamic Domains Nabla Derivatives and the Hamilton-Jacobi-Bellman Equation," Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, 2008. SMC 2008, Institute of Electrical and Electronics Engineers (IEEE), Oct 2008.
The definitive version is available at https://doi.org/10.1109/ICSMC.2008.4811761
IEEE International Conference on Systems, Man and Cybernetics, 2008. SMC 2008
Electrical and Computer Engineering
National Science Foundation (U.S.)
Keywords and Phrases
Hamilton-Jacobi-Bellman Equation; Approximate Dynamic Programming; Reinforcement Learning; Time Scales
Article - Conference proceedings
© 2008 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.
01 Oct 2008