Computer Vision for Visual Effects

Book Preface

Neo fends off dozens of Agent Smith clones in a city park. Kevin Flynn confronts a thirty-years-younger avatar of himself in the Grid. Captain America’s sidekick rolls under a speeding truck in the nick of time to plant a bomb. Nightcrawler “bamfs” in and out of rooms, leaving behind a puff of smoke. James Bond skydives at high speed out of a burning airplane. Harry Potter grapples with Nagini in a ramshackle cottage. Robert Neville stalks a deer in an overgrown, abandoned Times Square. Autobots and Decepticons battle it out in the streets of Chicago. Today’s blockbuster movies so seamlessly introduce impossible characters and action into real-world settings that it’s easy for the audience to suspend its disbelief. These compelling action scenes are made possible by modern visual effects.

Visual effects, the manipulation and fusion of live and synthetic images, have been a part of moviemaking since the first short films were made in the 1900s. For example, beginning in the 1920s, fantastic sets and environments were created using huge, detailed paintings on panes of glass placed between the camera and the actors. Miniature buildings or monsters were combined with footage of live actors using forced perspective to create photo-realistic composites. Superheroes flew across the screen using rear-projection and blue-screen replacement technology.

These days, almost all visual effects involve the manipulation of digital and computer-generated images instead of in-camera, practical effects. Filmgoers over the past forty years have experienced the transition from the mostly analog effects of movies like The Empire Strikes Back to the early days of computer-generated imagery in movies like Terminator 2: Judgment Day to the almost entirely digital effects of movies like Avatar. While they’re often associated with action and science fiction movies, visual effects are now so common that they’re imperceptibly incorporated into virtually all TV series and movies—even medical shows like Grey’s Anatomy and period dramas like Changeling.