Reinforcement Learning and Approximate Dynamic Programming for Feedback Control

Copyright © 2013 The Institute of Electrical and Electronics Engineers, Inc.

Reinforcement Learning and Approximate Dynamic Programming for Feedback Control

Editor(s): Frank L. Lewis, Derong Liu

Published Online: 7 FEB 2013 02:18PM EST

Print ISBN: 9781118104200

Online ISBN: 9781118453988

DOI: 10.1002/9781118453988

About this Book

Product Information

About The Product

Reinforcement learning (RL) and adaptive dynamic programming (ADP) has been one of the most critical research fields in science and engineering for modern complex systems. This book describes the latest RL and ADP techniques for decision and control in human engineered systems, covering both single player decision and control and multi-player games. Edited by the pioneers of RL and ADP research, the book brings together ideas and methods from many fields and provides an important and timely guidance on controlling a wide variety of systems, such as robots, industrial processes, and economic decision-making.

Table of contents

    1. You have free access to this content
      Frontmatter (pages i–xxvi)

  2. Part I: Feedback Control Using RL and ADP

    1. Chapter 1

      Reinforcement Learning and Approximate Dynamic Programming (RLADP)—Foundations, Common Misconceptions, and the Challenges Ahead (pages 1–30)

      Paul J. Werbos

    2. Chapter 2

      Stable Adaptive Neural Control of Partially Observable Dynamic Systems (pages 31–51)

      J. Nate Knight and Charles W. Anderson

    3. Chapter 3

      Optimal Control of Unknown Nonlinear Discrete-Time Systems Using the Iterative Globalized Dual Heuristic Programming Algorithm (pages 52–77)

      Derong Liu and Ding Wang

    4. Chapter 4

      Learning and Optimization in Hierarchical Adaptive Critic Design (pages 78–97)

      Haibo He, Zhen Ni and Dongbin Zhao

    5. Chapter 5

      Single Network Adaptive Critics Networks—Development, Analysis, and Applications (pages 98–118)

      Jie Ding, Ali Heydari and S. N. Balakrishnan

    6. Chapter 6

      Linearly Solvable Optimal Control (pages 119–141)

      K. Dvijotham and E. Todorov

    7. Chapter 7

      Approximating Optimal Control with Value Gradient Learning (pages 142–161)

      Michael Fairbank, Danil Prokhorov and Eduardo Alonso

    8. Chapter 8

      A Constrained Backpropagation Approach to Function Approximation and Approximate Dynamic Programming (pages 162–181)

      Silvia Ferrari, Keith Rudd and Gianluca Di Muro

    9. Chapter 9

      Toward Design of Nonlinear ADP Learning Controllers with Performance Assurance (pages 182–202)

      Jennie Si, Lei Yang, Chao Lu, Kostas S. Tsakalis and Armando A. Rodriguez

    10. Chapter 10

      Reinforcement Learning Control with Time-Dependent Agent Dynamics (pages 203–220)

      Kenton Kirkpatrick and John Valasek

    11. Chapter 11

      Online Optimal Control of Nonaffine Nonlinear Discrete-Time Systems without Using Value and Policy Iterations (pages 221–257)

      Hassan Zargarzadeh, Qinmin Yang and S. Jagannathan

    12. Chapter 12

      An Actor–Critic–Identifier Architecture for Adaptive Approximate Optimal Control (pages 258–280)

      S. Bhasin, R. Kamalapurkar, M. Johnson, K. G. Vamvoudakis, F. L. Lewis and W. E. Dixon

    13. Chapter 13

      Robust Adaptive Dynamic Programming (pages 281–302)

      Yu Jiang and Zhong-Ping Jiang

  3. Part II: Learning and Control in Multiagent Games

    1. Chapter 14

      Hybrid Learning in Stochastic Games and Its Application in Network Security (pages 303–329)

      Quanyan Zhu, Hamidou Tembine and Tamer Başar

    2. Chapter 15

      Integral Reinforcement Learning for Online Computation of Nash Strategies of Nonzero-Sum Differential Games (pages 330–349)

      Draguna Vrabie and F. L. Lewis

    3. Chapter 16

      Online Learning Algorithms for Optimal Control and Dynamic Games (pages 350–377)

      Kyriakos G. Vamvoudakis and Frank L. Lewis

  4. Part III: Foundations in MDP and RL

    1. Chapter 17

      Lambda-Policy Iteration: A Review and a New Implementation (pages 379–409)

      Dimitri P. Bertsekas

    2. Chapter 18

      Optimal Learning and Approximate Dynamic Programming (pages 410–431)

      Warren B. Powell and Ilya O. Ryzhov

    3. Chapter 19

      An Introduction to Event-Based Optimization: Theory and Applications (pages 432–451)

      Xi-Ren Cao, Yanjia Zhao, Qing-Shan Jia and Qianchuan Zhao

    4. Chapter 20

      Bounds for Markov Decision Processes (pages 452–473)

      Vijay V. Desai, Vivek F. Farias and Ciamac C. Moallemi

    5. Chapter 21

      Approximate Dynamic Programming and Backpropagation on Timescales (pages 474–493)

      John Seiffertt and Donald Wunsch

    6. Chapter 22

      A Survey of Optimistic Planning in Markov Decision Processes (pages 494–516)

      Lucian Buşoniu, Rémi Munos and Robert Babuška

    7. Chapter 23

      Adaptive Feature Pursuit: Online Adaptation of Features in Reinforcement Learning (pages 517–534)

      Shalabh Bhatnagar, Vivek S. Borkar and L. A. Prashanth

    8. Chapter 24

      Feature Selection for Neuro-Dynamic Programming (pages 535–559)

      Dayu Huang, W. Chen, P. Mehta, S. Meyn and A. Surana

    9. Chapter 25

      Approximate Dynamic Programming for Optimizing Oil Production (pages 560–581)

      Zheng Wen, Louis J. Durlofsky, Benjamin van Roy and Khalid Aziz

    10. Chapter 26

      A Learning Strategy for Source Tracking in Unstructured Environments (pages 582–600)

      Titus Appel, Rafael Fierro, Brandon Rohrer, Ron Lumia and John Wood

    1. You have free access to this content
      Index (pages 601–613)

    1. You have free access to this content
      IEEE Press Series on Computational Intelligence (page 614)

SEARCH