Haptic Feedback Teleoperation of Optical Tweezers

Zhenjiang Ni is a postdoctoral researcher at Institut des Systèmes Intelligents et de Robotique (ISIR) at University Pierre and Marie Curie in Paris, France. Céline Pacoret is a postdoctoral researcher at Institut des Systèmes Intelligents et de Robotique (ISIR) at University Pierre and Marie Curie in Paris, France. Ryad Benosman is a lecturer at University Pierre and Marie Curie as well as at Institut de la Vision in Paris, France. Stéphane Régnier is Professor at Institut des Systèmes Intelligents et de Robotique (ISIR) at University Pierre and Marie Curie in Paris, France.

PREFACE  ix

INTRODUCTION  xi

CHAPTER 1. INTRODUCTION TO HAPTIC OPTICAL TWEEZERS  1

1.1. Introduction  1

1.2. A dexterous experimental platform 3

1.2.1. A dexterous micromanipulation technique  3

1.2.2. A dexterous user interaction for micromanipulation  5

1.2.3. Pioneering works  8

1.3. Interactive optical tweezers  10

1.3.1. Displacement techniques  10

1.3.2. Impact of the laser deflection 14

1.3.3. Measurement techniques 16

1.4. Specific designs for haptic interactions 21

1.4.1. Temporal sharing  22

1.4.2. Spatial sharing  24

1.5. Discussion  26

1.6. Conclusion  29

1.7. Bibliography   30

CHAPTER 2. HIGH-SPEED VISION: FROM FRAME-BASED TO EVENT-BASED 45

2.1. High-speed cameras 45

2.1.1. Image data acquisition 46

2.1.2. Image data transmission  48

2.1.3. Image data processing 51

2.2. Silicon retinas  52

2.2.1. Neuromorphic engineering  52

2.2.2. Dynamic vision sensor (DVS) 54

2.2.3. Asynchronous time-based image sensor   57

2.3. The advantages of asynchronous event-based vision  59

2.3.1. Frame-based methodology 59

2.3.2. Event-based acquisition  60

2.3.3. Event-based processing 62

2.4. The fundamentals of event-based computation  64

2.5. State of the art of silicon retina applications  67

2.6. High-speed vision in robotics 70

2.6.1. Examples   71

2.6.2. Difficulties   74

2.7. Necessity of high-speed vision in microrobotics  76

2.7.1. Automatic control of a microrobot  76

2.7.2. Teleoperated micromanipulation 77

2.7.3. Two concrete applications 80

2.8. Bibliography   85

CHAPTER 3. ASYNCHRONOUS EVENT-BASED 2D MICROSPHERE TRACKING  93

3.1. Reliable haptic optical tweezers 93

3.2. State of the art of high-speed microparticle tracking  95

3.2.1. Position detection devices 96

3.2.2. Candidate algorithms  98

3.3. Microsphere tracking using DVS 101

3.3.1. Event-based continuous Hough transform  101

3.3.2. Multiple microsphere tracking  103

3.3.3. Brownian motion detection  108

3.4. 2D haptic feedback micromanipulation with optical tweezers  112

3.4.1. Strategy of haptic coupling with optical tweezer  113

3.4.2. Haptic feedback optical tweezer system setup  114

3.4.3. First experiments on force sensing in the microworld   117

3.4.4. A comparison of frame-based and event-based vision in micromanipulation  121

3.5. Conclusions  124

3.6. Bibliography   125

CHAPTER 4. ASYNCHRONOUS EVENT-BASED 3D MICROSPHERE TRACKING 129

4.1. 3D sphere tracking methods  130

4.1.1. Defocus  131

4.1.2. Intensity average on frame-based images   133

4.1.3. Polarity integration 135

4.1.4. Extension of continuous Hough transform  137

4.1.5. Robust circle fitting 139

4.1.6. Summary of different methods  143

4.2. 3D haptic feedback teleoperation of optical tweezers   144

4.2.1. Configuration and method 144

4.2.2. Z-axis force feedback  147

4.3. Haptic feedback on multitrap optical tweezers   149

4.3.1. Time multiplexing multitrapping by galvanometer 149

4.3.2. Events-trap correspondence  152

4.3.3. Multitrap experimental results  154

4.3.4. Marketability 158

4.4. Piezoelectric microgripper tracking for stable haptic feedback  160

4.4.1. System setup 161

4.4.2. Vision system 164

4.4.3. Haptic coupling strategy  167

4.4.4. Experimental results  170

4.4.5. Interest to industry 177

4.5. Conclusions  177

4.6. Bibliography   178

CONCLUSIONS AND PERSPECTIVES  181

INDEX  187