Human-Robot Interaction Control Using Reinforcement Learning (eBook)

WEN YU, PhD, is Professor and Head of the Departamento de Control Automático with the Centro de Investigación y de Estudios Avanzados, Instituto Politécnico Nacional (CINVESTAV-IPN), Mexico City, Mexico. He is a co-author of Modeling and Control of Uncertain Nonlinear Systems with Fuzzy Equations and Z-Number. ADOLFO PERRUSQUÍA, PhD, is a Research Fellow in the School of Aerospace, Transport, and Manufacturing at Cranfield University in Bedford, UK.

Author Biographies xi

List of Figures xiii

List of Tables xvii

Preface xix

Part I Human-robot Interaction Control 1

1 Introduction 3

1.1 Human-Robot Interaction Control 3

1.2 Reinforcement Learning for Control 6

1.3 Structure of the Book 7

References 10

2 Environment Model of Human-Robot Interaction 17

2.1 Impedance and Admittance 17

2.2 Impedance Model for Human-Robot Interaction 21

2.3 Identification of Human-Robot Interaction Model 24

2.4 Conclusions 30

References 30

3 Model Based Human-Robot Interaction Control 33

3.1 Task Space Impedance/Admittance Control 33

3.2 Joint Space Impedance Control 36

3.3 Accuracy and Robustness 37

3.4 Simulations 39

3.5 Conclusions 42

References 44

4 Model Free Human-Robot Interaction Control 45

4.1 Task-Space Control Using Joint-Space Dynamics 45

4.2 Task-Space Control Using Task-Space Dynamics 52

4.3 Joint Space Control 53

4.4 Simulations 54

4.5 Experiments 55

4.6 Conclusions 68

References 71

5 Human-in-the-loop Control Using Euler Angles 73

5.1 Introduction 73

5.2 Joint-Space Control 74

5.3 Task-Space Control 79

5.4 Experiments 83

5.5 Conclusions 92

References 94

Part II Reinforcement Learning for Robot Interaction Control 97

6 Reinforcement Learning for Robot Position/Force Control 99

6.1 Introduction 99

6.2 Position/Force Control Using an Impedance Model 100

6.3 Reinforcement Learning Based Position/Force Control 103

6.4 Simulations and Experiments 110

6.5 Conclusions 117

References 117

7 Continuous-Time Reinforcement Learning for Force Control 119

7.1 Introduction 119

7.2 K-means Clustering for Reinforcement Learning 120

7.3 Position/Force Control Using Reinforcement Learning 124

7.4 Experiments 130

7.5 Conclusions 136

References 136

8 Robot Control in Worst-Case Uncertainty Using Reinforcement Learning 139

8.1 Introduction 139

8.2 Robust Control Using Discrete-Time Reinforcement Learning 141

8.3 Double Q-Learning with k-Nearest Neighbors 144

8.4 Robust Control Using Continuous-Time Reinforcement Learning 150

8.5 Simulations and Experiments: Discrete-Time Case 154

8.6 Simulations and Experiments: Continuous-Time Case 161

8.7 Conclusions 170

References 170

9 Redundant Robots Control Using Multi-Agent Reinforcement Learning 173

9.1 Introduction 173

9.2 Redundant Robot Control 175

9.3 Multi-Agent Reinforcement Learning for Redundant Robot Control 179

9.4 Simulations and experiments 183

9.5 Conclusions 187

References 189

10 Robot H2 Neural Control Using Reinforcement Learning 193

10.1 Introduction 193

10.2 H2 Neural Control Using Discrete-Time Reinforcement Learning 194

10.3 H2 Neural Control in Continuous Time 207

10.4 Examples 219

10.5 Conclusion 229

References 229

11 Conclusions 233

A Robot Kinematics and Dynamics 235

A.1 Kinematics 235

A.2 Dynamics 237

A.3 Examples 240

References 246

B Reinforcement Learning for Control 247

B.1 Markov decision processes 247

B.2 Value functions 248

B.3 Iterations 250

B.4 TD learning 251

Reference 258

Index 259