Matchmoving

Tim Dobbert is a layout lead at Industrial Light & Magic in San Francisco and has been in the visual effects industry for over ten years. He has worked as a matchmover and digital artist on over 25 feature films, including Avatar, Pirates of the Caribbean: On Stranger Tides, and Harry Potter and the Goblet of Fire. In addition to teaching university classes in matchmoving and image-based content creation, he has lectured all over the world on the subject.

Introduction xxi Chapter 1 The Basics of Matchmoving 1

Exploring a Typical Matchmove  2

Moving from 3D to 2D and Back Again 4

Understanding the Matchmoving Process  5

Evaluating the Footage 5

Applying Information 6

Defining the Camera 6

Fitting the Set 7

Testing the Matchmove 9

Delivering the Scene 9

Matchmoving in the Production Pipeline  9

Perspective Matching Tutorial  10

Gathering the Data 11

Setting Up the Camera 13

Adding Rough Geometry and Refining the Camera 15

Creating a Camera Rig 16

Evaluating and Adjusting the Camera  18

Chapter 2 Matchmoving Software 23

Using Matchmoving Programs  24

Introducing Photogrammetry   24

Calibration 25

The Optics of Photography 26

Projection 28

Building a Better Camera 31

Parallax: A Matchmover’s Best Friend 38

Understanding How Matchmoving Programs Work        40

Requirements for Matchmoving Programs 41

Chapter 3 2D Tracking 43

Understanding the 2D Tracking Process  44

Exploring the Anatomy of a 2D Track  45

Track Placement: Making Every Track Count             47

Sample the 3D Space 47

Maintain the Minimum Number of Tracks 48

Track Stationary Objects 49

Track “True” 3D Positions 50

Getting Tracks to Stick        51

Following Hints for Effective 2D Tracking               52

Handling Plate Issues         54

Motion Blur 55

Soft Focus or Rack Focus 56

Occlusion 56

Noise 58

Low-Contrast Features 58

Lack of Trackable Features 59

Optimizing the Plate for 2D Tracking  61

2D Tracking Tutorial         62

Evaluating the Shot 63

Placing the First Tracks in SynthEyes 63

Choosing Your Tracks 66

Adding the Remaining Tracks 66

Chapter 4 3D Calibration 69

Understanding “Good” Calibration    70

Evaluating Error 70

Calibrating Your Cameras     72

Finding the Right Fit 72

Calibration Tutorial 73

Evaluating the Solution        77

Evaluating the 3D Nulls 77

Evaluating the 3D Space 78

Rendering the Matchmove 79

When Good Solutions Go Bad 80

Determining Camera Moves    81

Dolly/Crane/Moving Camera 81

Pan/Tilt Shots 82

Slight Dollies/Translation 82

Lock-offs 83

Zoom Shots 83

Shaky Footage/Handheld Footage 84

Setting Up a Coordinate System       84

Coordinate System Tutorial    85

Exporting the Scene 88

Chapter 5 Automatic Tracking 91

Tracking on Autopilot         92

Editing Automatic Tracks      93

Using Masks                96

Reaping the Benefits of Automatic Tracking               97

Noise Reduction 97

Environment Definition 98

Implementing the Automatic Tracking Process             99

Automatic Tracking Tutorial 99

Chapter 6 Cameras 107

How Film Cameras Work     108

Lenses   110

Focal Length 111

Focus 113

Nodality 113

Film Backs                 115

Film Back and Focal Length 115

Digital Cameras 117

CMOS, CCD, and Rolling Shutter 119

Camera Information 121

Format  122

Lens Distortion             123

Anamorphic Distortion 127

Capture and Digitization     128

Video 128

Film 129

HD 130

Keeping It All Straight       130

Chapter 7 Set Fitting 133

Fitting the Camera          134

Checking the Matchmove     135

Proxy Geometry 135

Checkerboards 137

Fitting the Set              138

Moving the Camera 138

Scaling the Camera 139

Creating Camera Rigs 141

Set-Fitting Tutorial          141

Examining the Exported Scene 142

Adding Test Geometry 143

Moving the Camera 144

Texturing the Objects 149

Performing the First Tests 150

Placing the Matchmove in the Set 152

Altering the Coordinate System      156

Delivering the Scene         156

Tutorial Continued 157

Chapter 8 Being on Set 159

Getting the Right Information       160

Getting Camera Information  162

Before the Shoot 162

During the Shoot 162

After the Shoot 165

Marking Up the Set          165

Tracking Markers for a Green Screen or Blue Screen 166

Tracking Markers for Set Pieces 167

Tracking Markers for Exterior Scenes 168

Working with Others on Set   168

Building a Matchmoving Kit  172

Shooting Distortion Grids     173

Chapter 9 Matchamation 175

Understanding the Basic Technique   176

Establishing the Distance to the Camera 176

Setting the Initial Pose 177

Using Nonlinear Animation Techniques 178

Analyzing the Movement 178

Making the First Pass 179

Making Additional Passes/Tweaks 180

Making Model Changes      181

Working with Imperfect Data  181

Matchamation Tutorial       182

Setting Up the Camera 182

Creating the Proxy Object 184

Creating a Matchamation Rig 185

Analyzing the Motion 186

Making the First Pass 188

Making the Second Pass 191

Making Additional Passes 194

Chapter 10 Troubleshooting and Advanced Techniques 197

Performing Effective Troubleshooting 198

Following a Troubleshooting Checklist 199

Check the Camera’s Motion 199

Check Set Placement and Fitting 201

Scrutinize 2D Tracking Closely 202

Check for Plate Issues 202

Diagnosing the Problem      203

No nulls are visible after calibration 204

Nulls are all in a line or a single point 204

The camera path jumps suddenly to another position or is extremely jagged and chaotic 204

Everything lines up on the fi rst frame but quickly slips off 205

Noise or jitter appears in the camera’s motion path 205

Objects appear to tear away from the plate feature during areas of heavy motion blur 206

Using Advanced Tools and Tricks     206

Using Advanced 2D Tracking Tools 206

Using Constraints 207

Working with Reference Photos 209

Forcing a Calibration 210

Faking the Focal Length 210

Using Survey Data 211

Chapter 11 Object Tracking 215

The Object-Tracking Process  216

Understanding Object Tracking vs Matchamation 216

Following the Object-Tracking Workfl ow 216

Converting a Camera Track to an Object Track 217

Object-Tracking Tutorial     219

Common Problems with Object Tracking               229

2D Tracking Problems 229

Difficult Object Motion 230

Object Not Truly Rigid 231

Tracking Human Faces 232

Geometry Tracking          233

Geometry-Tracking Tutorial   234

Common Problems with Geometry Tracking             237

Advanced Object Tracking    237

Moving Camera/Moving Object 238

Tracking Deforming Objects 239

Chapter 12 Multipurposing Matchmoving Data 241

Creating Special Cameras     242

Extending or Enhancing Matchmoves 242

Blending Matchmoves 243

Blending Cameras in Maya 245

Object Track as a Camera Track 251

Matchmoves for Paint Work: “Patchmoves”             252

Matchmoves for Compositors  253

Matchmoves for TD Work    254

Not the End of the Road     256

Chapter 13 Stereo 257

Understanding Stereoscopy    258

Interocular Distance 259

Convergence 261

How IO and Convergence Affect Stereo Depth 264

Adjusting Stereo Parameters in Postproduction 265

Stereo Postconversion 266

Filming Stereo Footage       266

Side-By-Side Rigs 266

Beam-Splitter Rigs 267

Dual-Lens Cameras 269

Viewing Stereo Footage      269

Anaglyphs 270

Polarized Lens Systems 271

Modern Stereo Projection 271

Active Shutter Systems 272

Matchmoving Stereo Footage  272

2D Tracking 273

Solving 273

Set Fitting 275

Evaluating the Matchmove 275

Overcoming Common Problems      276

Problems with 2D Tracking 276

Problems with 3D Tracking 277

Focal-Length Issues 277

Appendix A Resources 279

Appendix B Matchmoving Software and Useful Links 281

Matchmoving Software       282

Other Useful Programs       283

Books   283

Websites  284

Appendix C Film Formats 285

16 mm Formats             286

35 mm Formats             286

Large Formats              287

Digital Formats             287

Calculating the Film Back for Cropped Images           288

Glossary 289

Index 299