Adaptive Dynamic Programming for Control (eBook)
XVI, 424 Seiten
Springer London (Verlag)
978-1-4471-4757-2 (ISBN)
• infinite-horizon control for which the difficulty of solving partial differential Hamilton-Jacobi-Bellman equations directly is overcome, and proof provided that the iterative value function updating sequence converges to the infimum of all the value functions obtained by admissible control law sequences;
• finite-horizon control, implemented in discrete-time nonlinear systems showing the reader how to obtain suboptimal control solutions within a fixed number of control steps and with results more easily applied in real systems than those usually gained from infinite-horizon control;
• nonlinear games for which a pair of mixed optimal policies are derived for solving games both when the saddle point does not exist, and, when it does, avoiding the existence conditions of the saddle point.
Non-zero-sum games are studied in the context of a single network scheme in which policies are obtained guaranteeing system stability and minimizing the individual performance function yielding a Nash equilibrium.
In order to make the coverage suitable for the student as well as for the expert reader, Adaptive Dynamic Programming in Discrete Time:
• establishes the fundamental theory involved clearly with each chapter devoted to a clearly identifiable control paradigm;
• demonstrates convergence proofs of the ADP algorithms to deepen understanding of the derivation of stability and convergence with the iterative computational methods used; and
• shows how ADP methods can be put to use both in simulation and in real applications.
This text will be of considerable interest to researchers interested in optimal control and its applications in operations research, applied mathematics computational intelligence and engineering. Graduate students working in control and operations research will also find the ideas presented here to be a source of powerful methods for furthering their study.
There are many methods of stable controller design for nonlinear systems. In seeking to go beyond the minimum requirement of stability, Adaptive Dynamic Programming in Discrete Time approaches the challenging topic of optimal control for nonlinear systems using the tools of adaptive dynamic programming (ADP). The range of systems treated is extensive; affine, switched, singularly perturbed and time-delay nonlinear systems are discussed as are the uses of neural networks and techniques of value and policy iteration. The text features three main aspects of ADP in which the methods proposed for stabilization and for tracking and games benefit from the incorporation of optimal control methods: * infinite-horizon control for which the difficulty of solving partial differential Hamilton-Jacobi-Bellman equations directly is overcome, and proof provided that the iterative value function updating sequence converges to the infimum of all the value functions obtained by admissible control law sequences; * finite-horizon control, implemented in discrete-time nonlinear systems showing the reader how to obtain suboptimal control solutions within a fixed number of control steps and with results more easily applied in real systems than those usually gained from infinite-horizon control; * nonlinear games for which a pair of mixed optimal policies are derived for solving games both when the saddle point does not exist, and, when it does, avoiding the existence conditions of the saddle point. Non-zero-sum games are studied in the context of a single network scheme in which policies are obtained guaranteeing system stability and minimizing the individual performance function yielding a Nash equilibrium. In order to make the coverage suitable for the student as well as for the expert reader, Adaptive Dynamic Programming in Discrete Time: * establishes the fundamental theory involved clearly with each chapter devoted to aclearly identifiable control paradigm; * demonstrates convergence proofs of the ADP algorithms to deepen understanding of the derivation of stability and convergence with the iterative computational methods used; and * shows how ADP methods can be put to use both in simulation and in real applications. This text will be of considerable interest to researchers interested in optimal control and its applications in operations research, applied mathematics computational intelligence and engineering. Graduate students working in control and operations research will also find the ideas presented here to be a source of powerful methods for furthering their study.
Adaptive Dynamic Programming for Control 2
Preface 4
Background of This Book 4
Why This Book? 5
The Content of This Book 5
Acknowledgments 8
Contents 10
Chapter 1: Overview 15
1.1 Challenges of Dynamic Programming 15
1.2 Background and Development of Adaptive Dynamic Programming 17
1.2.1 Basic Structures of ADP 18
1.2.1.1 Heuristic Dynamic Programming (HDP) 18
1.2.1.2 Dual Heuristic Programming (DHP) 19
1.2.2 Recent Developments of ADP 20
1.2.2.1 Development of ADP Structures 20
1.2.2.2 Development of Algorithms and Convergence Analysis 23
1.2.2.3 Applications of ADP Algorithms 24
1.3 Feedback Control Based on Adaptive Dynamic Programming 25
1.4 Non-linear Games Based on Adaptive Dynamic Programming 31
1.5 Summary 33
References 33
Chapter 2: Optimal State Feedback Control for Discrete-Time Systems 40
2.1 Introduction 40
2.2 In nite-Horizon Optimal State Feedback Control Based on DHP 40
2.2.1 Problem Formulation 41
2.2.2 In nite-Horizon Optimal State Feedback Control via DHP 43
2.2.3 Simulations 57
2.3 In nite-Horizon Optimal State Feedback Control Based on GDHP 65
2.3.1 Problem Formulation 65
2.3.2 In nite-Horizon Optimal State Feedback Control Based on GDHP 67
2.3.2.1 NN Identi cation of the Unknown Nonlinear System 67
2.3.2.2 Derivation of the Iterative ADP Algorithm 70
2.3.2.3 Convergence Analysis of the Iterative ADP Algorithm 71
2.3.2.4 NN Implementation of the Iterative ADP Algorithm Using GDHP Technique 77
2.3.3 Simulations 80
2.4 In nite-Horizon Optimal State Feedback Control Based on GHJB Algorithm 84
2.4.1 Problem Formulation 84
2.4.2 Constrained Optimal Control Based on GHJB Equation 86
2.4.3 Simulations 91
2.5 Finite-Horizon Optimal State Feedback Control Based on HDP 93
2.5.1 Problem Formulation 95
2.5.2 Finite-Horizon Optimal State Feedback Control Based on HDP 97
2.5.2.1 Derivation and Properties of the Iterative ADP Algorithm 97
2.5.2.2 The epsilon-Optimal Control Algorithm 104
2.5.3 Simulations 115
2.6 Summary 119
References 119
Chapter 3: Optimal Tracking Control for Discrete-Time Systems 121
3.1 Introduction 121
3.2 In nite-Horizon Optimal Tracking Control Based on HDP 121
3.2.1 Problem Formulation 122
3.2.2 In nite-Horizon Optimal Tracking Control Based on HDP 123
3.2.2.1 System Transformation 123
3.2.2.2 Derivation of the Iterative HDP Algorithm 124
3.2.2.3 Summary of the Algorithm 129
3.2.2.4 Neural-Network Implementation for the Tracking Control Scheme 130
3.2.3 Simulations 130
3.3 In nite-Horizon Optimal Tracking Control Based on GDHP 132
3.3.1 Problem Formulation 135
3.3.2 In nite-Horizon Optimal Tracking Control Based on GDHP 138
3.3.2.1 Design and Implementation of Feedforward Controller 138
3.3.2.2 Design and Implementation of Optimal Feedback Controller 139
3.3.2.3 Convergence Characteristics of the Neural-Network Approximation Process 147
3.3.3 Simulations 149
3.4 Finite-Horizon Optimal Tracking Control Based on ADP 150
3.4.1 Problem Formulation 153
3.4.2 Finite-Horizon Optimal Tracking Control Based on ADP 156
3.4.2.1 Derivation of the Iterative ADP Algorithm 156
3.4.2.2 Convergence Analysis of the Iterative ADP Algorithm 158
3.4.2.3 The epsilon-Optimal Control Algorithm 162
3.4.2.4 Summary of the Algorithm 163
3.4.2.5 Neural-Network Implementation of the Iterative ADP Algorithm via HDP Technique 163
3.4.3 Simulations 166
3.5 Summary 170
References 171
Chapter 4: Optimal State Feedback Control of Nonlinear Systems with Time Delays 173
4.1 Introduction 173
4.2 In nite-Horizon Optimal State Feedback Control via Delay Matrix 174
4.2.1 Problem Formulation 174
4.2.2 Optimal State Feedback Control Using Delay Matrix 175
4.2.2.1 Model Network 184
4.2.2.2 The M Network 185
4.2.2.3 Critic Network 185
4.2.2.4 Action Network 186
4.2.3 Simulations 187
4.3 In nite-Horizon Optimal State Feedback Control via HDP 189
4.3.1 Problem Formulation 189
4.3.2 Optimal Control Based on Iterative HDP 192
4.3.3 Simulations 198
4.4 Finite-Horizon Optimal State Feedback Control for a Class of Nonlinear Systems with Time Delays 200
4.4.1 Problem Formulation 200
4.4.2 Optimal Control Based on Improved Iterative ADP 202
4.4.3 Simulations 208
4.5 Summary 209
References 210
Chapter 5: Optimal Tracking Control of Nonlinear Systems with Time Delays 212
5.1 Introduction 212
5.2 Problem Formulation 212
5.3 Optimal Tracking Control Based on Improved Iterative ADP Algorithm 213
5.4 Simulations 224
5.5 Summary 231
References 231
Chapter 6: Optimal Feedback Control for Continuous-Time Systems via ADP 233
6.1 Introduction 233
6.2 Optimal Robust Feedback Control for Unknown General Nonlinear Systems 233
6.2.1 Problem Formulation 234
6.2.2 Data-Based Robust Approximate Optimal Tracking Control 234
6.2.3 Simulations 246
6.3 Optimal Feedback Control for Nonaf ne Nonlinear Systems 252
6.3.1 Problem Formulation 252
6.3.2 Robust Approximate Optimal Control Based on ADP Algorithm 253
6.3.3 Simulations 260
6.4 Summary 263
References 264
Chapter 7: Several Special Optimal Feedback Control Designs Based on ADP 266
7.1 Introduction 266
7.2 Optimal Feedback Control for a Class of Switched Systems 267
7.2.1 Problem Description 267
7.2.2 Optimal Feedback Control Based on Two-Stage ADP Algorithm 268
7.2.3 Simulations 277
7.3 Optimal Feedback Control for a Class of Descriptor Systems 280
7.3.1 Problem Formulation 280
7.3.2 Optimal Controller Design for a Class of Descriptor Systems 282
7.3.3 Simulations 288
7.4 Optimal Feedback Control for a Class of Singularly Perturbed Systems 290
7.4.1 Problem Formulation 290
7.4.2 Optimal Controller Design for Singularly Perturbed Systems 292
7.4.2.1 Algorithm Design 292
7.4.2.2 Neural Network Approximation 295
7.4.3 Simulations 297
7.5 Optimal Feedback Control for a Class of Constrained Systems Via SNAC 297
7.5.1 Problem Formulation 297
7.5.2 Optimal Controller Design for Constrained Systems via SNAC 301
7.5.3 Simulations 308
7.6 Summary 315
References 315
Chapter 8: Zero-Sum Games for Discrete-Time Systems Based on Model-Free ADP 317
8.1 Introduction 317
8.2 Zero-Sum Differential Games for a Class of Discrete-Time 2-D Systems 317
8.2.1 Problem Formulation 318
8.2.2 Data-Based Optimal Control via Iterative ADP Algorithm 325
8.2.2.1 The Derivation of Data-Based Iterative ADP Algorithm 326
8.2.2.2 Properties of Data-Based Iterative ADP Algorithm 327
8.2.2.3 Neural Network Implementation 334
8.2.2.4 Critic Network 334
8.2.2.5 Action Networks 335
8.2.3 Simulations 336
8.3 Zero-Sum Games for a Class of Discrete-Time Systems via Model-Free ADP 339
8.3.1 Problem Formulation 340
8.3.2 Data-Based Optimal Output Feedback Control via ADP Algorithm 342
8.3.3 Simulations 349
8.4 Summary 351
References 351
Chapter 9: Nonlinear Games for a Class of Continuous-Time Systems Based on ADP 353
9.1 Introduction 353
9.2 In nite Horizon Zero-Sum Games for a Class of Af ne Nonlinear Systems 354
9.2.1 Problem Formulation 354
9.2.2 Zero-Sum Differential Games Based on Iterative ADP Algorithm 355
9.2.2.1 Derivation of the Iterative ADP Method 355
9.2.2.2 The Iterative ADP Algorithm 357
9.2.2.3 Properties of the Iterative ADP Algorithm 358
9.2.3 Simulations 363
9.3 Finite Horizon Zero-Sum Games for a Class of Nonlinear Systems 366
9.3.1 Problem Formulation 368
9.3.2 Finite Horizon Optimal Control of Nonaf ne Nonlinear Zero-Sum Games 370
9.3.3 Simulations 378
9.4 Non-Zero-Sum Games for a Class of Nonlinear Systems Based on ADP 380
9.4.1 Problem Formulation of Non-Zero-Sum Games 381
9.4.2 Optimal Control of Nonlinear Non-Zero-Sum Games Based on ADP 384
9.4.3 Simulations 395
9.5 Summary 399
References 400
Chapter 10: Other Applications of ADP 402
10.1 Introduction 402
10.2 Self-Learning Call Admission Control for CDMA Cellular Networks Using ADP 403
10.2.1 Problem Formulation 403
10.2.2 A Self-Learning Call Admission Control Scheme for CDMA Cellular Networks 405
10.2.2.1 Adaptive Critic Designs for Problems with Finite Action Space 405
10.2.2.2 Self-learning Call Admission Control for CDMA Cellular Networks 409
10.2.3 Simulations 413
10.3 Engine Torque and Air-Fuel Ratio Control Based on ADP 419
10.3.1 Problem Formulation 419
10.3.2 Self-learning Neural Network Control for Both Engine Torque and Exhaust Air-Fuel Ratio 420
10.3.3 Simulations 422
10.3.3.1 Critic Network 422
10.3.3.2 Controller/Action Network 424
10.3.3.3 Simulation Results 424
10.4 Summary 426
References 427
Index 430
| Erscheint lt. Verlag | 14.12.2012 |
|---|---|
| Reihe/Serie | Communications and Control Engineering |
| Zusatzinfo | XVI, 424 p. |
| Verlagsort | London |
| Sprache | englisch |
| Themenwelt | Informatik ► Theorie / Studium ► Künstliche Intelligenz / Robotik |
| Mathematik / Informatik ► Mathematik ► Angewandte Mathematik | |
| Mathematik / Informatik ► Mathematik ► Finanz- / Wirtschaftsmathematik | |
| Naturwissenschaften | |
| Technik ► Elektrotechnik / Energietechnik | |
| Schlagworte | Adaptive Dynamic Programming • Finite-horizon Control • Infinite-horizon Control • Reinforcement Learning • Zero-sum Game |
| ISBN-10 | 1-4471-4757-X / 144714757X |
| ISBN-13 | 978-1-4471-4757-2 / 9781447147572 |
| Haben Sie eine Frage zum Produkt? |
PDF (Wasserzeichen)Größe: 7,2 MB
DRM: Digitales Wasserzeichen
Dieses eBook enthält ein digitales Wasserzeichen und ist damit für Sie personalisiert. Bei einer missbräuchlichen Weitergabe des eBooks an Dritte ist eine Rückverfolgung an die Quelle möglich.
Dateiformat: PDF (Portable Document Format)
Mit einem festen Seitenlayout eignet sich die PDF besonders für Fachbücher mit Spalten, Tabellen und Abbildungen. Eine PDF kann auf fast allen Geräten angezeigt werden, ist aber für kleine Displays (Smartphone, eReader) nur eingeschränkt geeignet.
Systemvoraussetzungen:
PC/Mac: Mit einem PC oder Mac können Sie dieses eBook lesen. Sie benötigen dafür einen PDF-Viewer - z.B. den Adobe Reader oder Adobe Digital Editions.
eReader: Dieses eBook kann mit (fast) allen eBook-Readern gelesen werden. Mit dem amazon-Kindle ist es aber nicht kompatibel.
Smartphone/Tablet: Egal ob Apple oder Android, dieses eBook können Sie lesen. Sie benötigen dafür einen PDF-Viewer - z.B. die kostenlose Adobe Digital Editions-App.
Zusätzliches Feature: Online Lesen
Dieses eBook können Sie zusätzlich zum Download auch online im Webbrowser lesen.
Buying eBooks from abroad
For tax law reasons we can sell eBooks just within Germany and Switzerland. Regrettably we cannot fulfill eBook-orders from other countries.
aus dem Bereich
