Digital puppetry
Encyclopedia
Digital puppetry is the manipulation and performance of digitally animated 2D or 3D figures and objects in a virtual environment that are rendered in real-time by computers. It is most commonly used in filmmaking
and television production, but has also been utilized in interactive theme park attractions and live theatre
.
The exact definition of what is and is not digital puppetry is subject to debate among puppeteers and computer graphics
designers, but it is generally agreed that digital puppetry differs from conventional computer animation
in that it involves performing characters in real time, rather than animating them frame by frame.
Digital puppetry is closely associated with motion capture
technologies and 3D animation. Digital puppetry is also known as Virtual Puppetry, Performance Animation, Living Animation, Live Animation or Realtime Animation (although the latter also refers to the animation generated by computer game engines). Machinima
is another form of digital puppetry and Machinima performers are increasingly being identified as puppeteers.
. He conducted experiments in the early 1960s that animated figures using analog circuits and a cathode ray tube. Harrison rigged up a body suit with potentiometers and created the first working motion capture rig, animating 3D figures in real-time on his CRT screen. He made several short films with this system, which he called ANIMAC.
, a character created in 1988 by Jim Henson
and Pacific Data Images for the Muppet television series The Jim Henson Hour
. Henson had been trying to create computer generated puppets as early as 1985 and Waldo grew out of experiments Henson conducted to create a computer generated version of his character Kermit the Frog
.
Waldo's strength as a computer generated puppet was that he could be controlled by a single puppeteer (Steve Whitmire
) in real-time in concert with conventional puppets. The computer image of Waldo was mixed with the video feed of the camera focused on physical puppets so that all of the puppeteers in a scene could perform together. (It was already standard Muppeteering practice to use monitors while performing, so the use of a virtual puppet did not significantly increase the complexity of the system.) Afterwards, in post production, PDI re-rendered Waldo in full resolution, adding a few dynamic elements on top of the performed motion.
Waldo C. Graphic can be seen today in Jim Henson's Muppet*Vision 3D at the Disney's Hollywood Studios
and Disney's California Adventure
theme parks.
' then-new 4D series workstations. Unveiled at the 1988 SIGGRAPH
convention, it was the first live performance of a digital character. Mike was a sophisticated talking head driven by a specially built controller that allowed a single puppeteer to control many parameters of the character's face, including mouth, eyes, expression, and head position.
The system developed by deGraf/Wahrman to perform Mike Normal was later used to create a representation of the villain Cain in the motion picture RoboCop 2
, which is believed to be the first example of digital puppetry being used to create a character in a full-length motion picture.
Trey Stokes
was the puppeteer for both Mike Normal's SIGGRAPH debut and Robocop II.
trilogy. The motion capture technology was created by Biomechanics Inc in Atlanta (now Giant Studios), Ramon's ideas contributed to enhancements to the technology, directly related to markering systems; virtual feedback of footage and computerised versions of the film sets; as well as the retargeting software called CharMapper (short for Character Mapper). Although the final footage was made with keyframe animation, a few seconds of Ramon's original performance can still be appreciated in the film. The character Gollum, tested by Ramon but performed by Andy Serkis, was also made with the same technology and is still considered the epitome of a virtual character in the film industry. Contrary to the Cave Troll, most of the animation of Gollum made it to the final footage using the original motion captured performance.
created by Phillip Reay for Warner Brothers Pictures. The puppet was created using hand drawn frames of animation that were puppeteered by Bruce Lanoil and David Barclay. The Bugs Live puppet was used to create nearly 900 minutes of live, fully interactive interviews of 2D animated Bugs character about his role in the movie Looney Tunes: Back in Action
in English and Spanish. Bugs Live also appeared at the 2004 SIGGRAPH Digital Puppetry Special Session with the Muppet puppet Gonzo
.
has also been an important innovator in the field of digital puppetry, developing new technologies as part of its "Living Character Initiative" in Disney theme parks. In 2004 they used digital puppetry techniques to create the Turtle Talk with Crush
attractions at Epcot
and Disney's California Adventure
. In the attraction, a hidden puppeteer performs and voices a digital puppet of Crush, the laid-back sea turtle from Finding Nemo
, on a large rear-projection screen. To the audience Crush appears to be swimming inside an aquarium and engages in unscripted, real-time conversations with theme park guests.
Disney Imagineering continued its use of digital puppetry with the Monsters Inc. Laugh Floor, a new attraction in Tomorrowland
at Walt Disney World's Magic Kingdom
, which opened in the spring of 2007. Guests temporarily enter the "monster world" introduced in Disney and Pixar's
2001 film, Monsters, Inc.
, where they are entertained by Mike Wazowski and other monster comedians who are attempting to capture laughter, which they convert to energy. Much like Turtle Talk, the puppeteers interact with guests in real-time, just as a real-life comedian would interact with his/her audience.
Disney also uses digital puppetry techniques in Stitch Encounter
, which opened in 2006 at the Hong Kong Disneyland
park. Disney has another version of the same attraction in Disneyland Resort Paris
called Stitch Live!
(CGI) using the low-end 3D engines in video games. Players act out scenes in real-time using characters and settings within a game and the resulting footage is recorded and later edited in to a finished film.
Filmmaking
Filmmaking is the process of making a film, from an initial story, idea, or commission, through scriptwriting, casting, shooting, directing, editing, and screening the finished product before an audience that may result in a theatrical release or television program...
and television production, but has also been utilized in interactive theme park attractions and live theatre
Theatre
Theatre is a collaborative form of fine art that uses live performers to present the experience of a real or imagined event before a live audience in a specific place. The performers may communicate this experience to the audience through combinations of gesture, speech, song, music or dance...
.
The exact definition of what is and is not digital puppetry is subject to debate among puppeteers and computer graphics
Computer graphics
Computer graphics are graphics created using computers and, more generally, the representation and manipulation of image data by a computer with help from specialized software and hardware....
designers, but it is generally agreed that digital puppetry differs from conventional computer animation
Computer animation
Computer animation is the process used for generating animated images by using computer graphics. The more general term computer generated imagery encompasses both static scenes and dynamic images, while computer animation only refers to moving images....
in that it involves performing characters in real time, rather than animating them frame by frame.
Digital puppetry is closely associated with motion capture
Motion capture
Motion capture, motion tracking, or mocap are terms used to describe the process of recording movement and translating that movement on to a digital model. It is used in military, entertainment, sports, and medical applications, and for validation of computer vision and robotics...
technologies and 3D animation. Digital puppetry is also known as Virtual Puppetry, Performance Animation, Living Animation, Live Animation or Realtime Animation (although the latter also refers to the animation generated by computer game engines). Machinima
Machinima
Machinima is the use of real-time 3D computer graphics rendering engines to create a cinematic production. Most often, video games are used to generate the computer animation...
is another form of digital puppetry and Machinima performers are increasingly being identified as puppeteers.
Early experiments
One of the earliest pioneers of digital puppetry was Lee Harrison IIILee Harrison III
Lee Harrison III was a pioneer in analog electronic animation. He is best known as the inventor of Scanimate and the ANIMAC. He received an Emmy Award in 1972 for his work.-External links:...
. He conducted experiments in the early 1960s that animated figures using analog circuits and a cathode ray tube. Harrison rigged up a body suit with potentiometers and created the first working motion capture rig, animating 3D figures in real-time on his CRT screen. He made several short films with this system, which he called ANIMAC.
Waldo C. Graphic
Perhaps the first truly commercially successful example of a digitally animated figure being performed and rendered in real-time is Waldo C. GraphicWaldo C. Graphic
Waldo C. Graphic is a computer-generated puppet character who appeared in the movie Jim Henson's Muppet*Vision 3D and the television series The Jim Henson Hour. He was performed by Steve Whitmire...
, a character created in 1988 by Jim Henson
Jim Henson
James Maury "Jim" Henson was an American puppeteer best known as the creator of The Muppets. As a puppeteer, Henson performed in various television programs, such as Sesame Street and The Muppet Show, films such as The Muppet Movie and The Great Muppet Caper, and created advanced puppets for...
and Pacific Data Images for the Muppet television series The Jim Henson Hour
The Jim Henson Hour
The Jim Henson Hour was a short-lived television series that aired on NBC in 1989. It was developed as a showcase for The Jim Henson Company's various puppet creations, including the popular Muppet characters. Only nine of the twelve episodes produced managed to air on NBC before the low-rated...
. Henson had been trying to create computer generated puppets as early as 1985 and Waldo grew out of experiments Henson conducted to create a computer generated version of his character Kermit the Frog
Kermit the Frog
Kermit the Frog is puppeteer Jim Henson's most famous Muppet creation, first introduced in 1955. He is the protagonist of many Muppet projects, most notably as the host of The Muppet Show, and has appeared in various sketches on Sesame Street, in commercials and in public service announcements over...
.
Waldo's strength as a computer generated puppet was that he could be controlled by a single puppeteer (Steve Whitmire
Steve Whitmire
Steven Whitmire is an American puppeteer who works for The Jim Henson Company, Sesame Workshop and Disney's The Muppets Studio. He has been the performer of two signature Muppets - Kermit the Frog and Sesame Street's Ernie - since the death of their creator and original performer, Jim Henson, in...
) in real-time in concert with conventional puppets. The computer image of Waldo was mixed with the video feed of the camera focused on physical puppets so that all of the puppeteers in a scene could perform together. (It was already standard Muppeteering practice to use monitors while performing, so the use of a virtual puppet did not significantly increase the complexity of the system.) Afterwards, in post production, PDI re-rendered Waldo in full resolution, adding a few dynamic elements on top of the performed motion.
Waldo C. Graphic can be seen today in Jim Henson's Muppet*Vision 3D at the Disney's Hollywood Studios
Disney's Hollywood Studios
Disney's Hollywood Studios is a theme park at the Walt Disney World Resort. Spanning 135 acres in size, its theme is show business, drawing inspiration from the heyday of Hollywood in the 1930s and 1940s...
and Disney's California Adventure
Disney's California Adventure
Disney California Adventure, or simply California Adventure, is a theme park in Anaheim, California, adjacent to Disneyland Park and part of the larger Disneyland Resort. It opened on February 8, 2001 as Disney's California Adventure Park. The park is owned and operated by the Walt Disney Parks and...
theme parks.
Mike Normal
Another significant development in digital puppetry in 1988 was Mike Normal, which Brad DeGraf and partner Michael Wahrman developed to show off the real-time capabilities of Silicon GraphicsSilicon Graphics
Silicon Graphics, Inc. was a manufacturer of high-performance computing solutions, including computer hardware and software, founded in 1981 by Jim Clark...
' then-new 4D series workstations. Unveiled at the 1988 SIGGRAPH
SIGGRAPH
SIGGRAPH is the name of the annual conference on computer graphics convened by the ACM SIGGRAPH organization. The first SIGGRAPH conference was in 1974. The conference is attended by tens of thousands of computer professionals...
convention, it was the first live performance of a digital character. Mike was a sophisticated talking head driven by a specially built controller that allowed a single puppeteer to control many parameters of the character's face, including mouth, eyes, expression, and head position.
The system developed by deGraf/Wahrman to perform Mike Normal was later used to create a representation of the villain Cain in the motion picture RoboCop 2
RoboCop 2
RoboCop 2 is a 1990 science fiction action film directed by Irvin Kershner and starring Peter Weller, Nancy Allen, Dan O'Herlihy, Belinda Bayer, Tom Noonan and Gabriel Damon. Set in the near future in a dystopian metropolitan Detroit, Michigan...
, which is believed to be the first example of digital puppetry being used to create a character in a full-length motion picture.
Trey Stokes
Trey Stokes
Trey Stokes is an American filmmaker and puppeteer, best known for his Star Wars parody series Pink Five, and his puppeteering work on various movie, TV, and motion-ride projects.-Early career:...
was the puppeteer for both Mike Normal's SIGGRAPH debut and Robocop II.
Cave Troll and Gollum on "The Lord of the Rings, The Fellowship of The Ring" (2001)
In 2000, Ramon Rivero was the first person to perform a digital puppet using Optical Motion Capture against pre-recorded action footage of a feature film. The character was the Cave Troll in the first episode of the The Lord of the RingsThe Lord of the Rings film trilogy
The Lord of the Rings is an epic film trilogy consisting of three fantasy adventure films based on the three-volume book of the same name by English author J. R. R. Tolkien. The films are The Fellowship of the Ring , The Two Towers and The Return of the King .The films were directed by Peter...
trilogy. The motion capture technology was created by Biomechanics Inc in Atlanta (now Giant Studios), Ramon's ideas contributed to enhancements to the technology, directly related to markering systems; virtual feedback of footage and computerised versions of the film sets; as well as the retargeting software called CharMapper (short for Character Mapper). Although the final footage was made with keyframe animation, a few seconds of Ramon's original performance can still be appreciated in the film. The character Gollum, tested by Ramon but performed by Andy Serkis, was also made with the same technology and is still considered the epitome of a virtual character in the film industry. Contrary to the Cave Troll, most of the animation of Gollum made it to the final footage using the original motion captured performance.
Bugs Live
"Bugs Live" was a digital puppet of Bugs BunnyBugs Bunny
Bugs Bunny is a animated character created in 1938 at Leon Schlesinger Productions, later Warner Bros. Cartoons. Bugs is an anthropomorphic gray rabbit and is famous for his flippant, insouciant personality and his portrayal as a trickster. He has primarily appeared in animated cartoons, most...
created by Phillip Reay for Warner Brothers Pictures. The puppet was created using hand drawn frames of animation that were puppeteered by Bruce Lanoil and David Barclay. The Bugs Live puppet was used to create nearly 900 minutes of live, fully interactive interviews of 2D animated Bugs character about his role in the movie Looney Tunes: Back in Action
Looney Tunes: Back in Action
Looney Tunes: Back in Action is a 2003 American live action/animated adventure comedy film directed by Joe Dante and starring Brendan Fraser, Jenna Elfman, Timothy Dalton, and Steve Martin. The film is essentially a feature-length Looney Tunes cartoon, with all the wackiness and surrealism typical...
in English and Spanish. Bugs Live also appeared at the 2004 SIGGRAPH Digital Puppetry Special Session with the Muppet puppet Gonzo
Gonzo (Muppet)
Gonzo the Great is a puppet character, one of Jim Henson's Muppets. He was developed and performed by Dave Goelz. The character made his first appearance in a 1970 Christmas special entitled "The Great Santa Claus Switch". Known as a "Whatever" , he is considered one of The Frackles...
.
Disney theme parks
Walt Disney ImagineeringWalt Disney Imagineering
Walt Disney Imagineering is the design and development arm of the Walt Disney Company, responsible for the creation and construction of Disney theme parks worldwide...
has also been an important innovator in the field of digital puppetry, developing new technologies as part of its "Living Character Initiative" in Disney theme parks. In 2004 they used digital puppetry techniques to create the Turtle Talk with Crush
Turtle Talk with Crush
Turtle Talk with Crush is an interactive attraction that has appeared at several of the Disney theme parks. It first opened on November 16, 2004 at "The Seas with Nemo & Friends" pavilion at Epcot, and was duplicated at Disney California Adventure in July 2005. The attraction was open in Hong Kong...
attractions at Epcot
Epcot
Epcot is a theme park in the Walt Disney World Resort, located near Orlando, Florida. The park is dedicated to the celebration of human achievement, namely international culture and technological innovation. The second park built at the resort, it opened on October 1, 1982 and was initially named...
and Disney's California Adventure
Disney's California Adventure
Disney California Adventure, or simply California Adventure, is a theme park in Anaheim, California, adjacent to Disneyland Park and part of the larger Disneyland Resort. It opened on February 8, 2001 as Disney's California Adventure Park. The park is owned and operated by the Walt Disney Parks and...
. In the attraction, a hidden puppeteer performs and voices a digital puppet of Crush, the laid-back sea turtle from Finding Nemo
Finding Nemo
Finding Nemo is a 2003 American comi-drama animated film written by Andrew Stanton, directed by Andrew Stanton and Lee Unkrich and produced by Pixar. It tells the story of the overly protective clownfish Marlin who, along with a regal tang called Dory , searches for his abducted son Nemo...
, on a large rear-projection screen. To the audience Crush appears to be swimming inside an aquarium and engages in unscripted, real-time conversations with theme park guests.
Disney Imagineering continued its use of digital puppetry with the Monsters Inc. Laugh Floor, a new attraction in Tomorrowland
Tomorrowland
- Tomorrowland 1955–1967 :The first Tomorrowland opened at Disneyland on July 18, 1955, with only several of its planned attractions open, due to budget cuts. The construction of the park was rushed, so Tomorrowland was the last land to be finished. It became something of a corporate showcase,...
at Walt Disney World's Magic Kingdom
Magic Kingdom
Magic Kingdom Park is one of four theme parks at the Walt Disney World Resort located near Orlando, Florida. The first park built at the resort, Magic Kingdom opened Oct. 1, 1971. Designed and built by WED Enterprises, the park's layout and attractions are similar to Disneyland in Anaheim, California...
, which opened in the spring of 2007. Guests temporarily enter the "monster world" introduced in Disney and Pixar's
Pixar
Pixar Animation Studios, pronounced , is an American computer animation film studio based in Emeryville, California. The studio has earned 26 Academy Awards, seven Golden Globes, and three Grammy Awards, among many other awards and acknowledgments. Its films have made over $6.3 billion worldwide...
2001 film, Monsters, Inc.
Monsters, Inc.
Monsters, Inc. is a 2001 American computer-animated film and the fourth feature-length film produced by Pixar Animation Studios. It was directed by Pete Docter, co-directed by Lee Unkrich and David Silverman, and written by Jill Culton, Peter Docter, Ralph Eggleston, Dan Gerson, Jeff Pidgeon, Rhett...
, where they are entertained by Mike Wazowski and other monster comedians who are attempting to capture laughter, which they convert to energy. Much like Turtle Talk, the puppeteers interact with guests in real-time, just as a real-life comedian would interact with his/her audience.
Disney also uses digital puppetry techniques in Stitch Encounter
Stitch Encounter
Stitch Encounter is an interactive show located in Tomorrowland at Hong Kong Disneyland. A similar attraction, under the name "Stitch Live!", is located inside Walt Disney Studios Park at Disneyland Paris....
, which opened in 2006 at the Hong Kong Disneyland
Hong Kong Disneyland
Hong Kong Disneyland is located on reclaimed land in Penny's Bay, Lantau Island. It is the first theme park located inside the Hong Kong Disneyland Resort and is owned and managed by the Hong Kong International Theme Parks. The park opened to visitors on 12 September 2005...
park. Disney has another version of the same attraction in Disneyland Resort Paris
Disneyland Resort Paris
Disneyland Paris is a holiday and recreation resort in Marne-la-Vallée, a new town in the eastern suburbs of Paris, France. The complex is located from the centre of Paris and lies for the most part within the commune of Chessy, Seine-et-Marne....
called Stitch Live!
Waldo puppetry
A digital puppet is controlled onscreen by a puppeteer who uses a telemetric input device connected to the computer. The X-Y-Z axis movement of the input device causes the digital puppet to move correspondingly. A keyboard, mouse or joystick-like device is sometimes used in place of a telemetric control.Motion capture puppetry (mocap puppetry) or Performance Animation
An object (puppet) or human body is used as a physical representation of a digital puppet and manipulated by a puppeteer. The movements of the object or body are matched correspondingly by the digital puppet in real-time.Machinima
A production technique that can be used to perform digital puppets. Machinima involves creating computer-generated imageryComputer-generated imagery
Computer-generated imagery is the application of the field of computer graphics or, more specifically, 3D computer graphics to special effects in art, video games, films, television programs, commercials, simulators and simulation generally, and printed media...
(CGI) using the low-end 3D engines in video games. Players act out scenes in real-time using characters and settings within a game and the resulting footage is recorded and later edited in to a finished film.
External links
- Machin-X: Digital Puppetry - Discussion of theories, tools and applications of digital puppetry as well as news from the digital puppetry community.
- The Henson Digital Puppetry Wiki - Wiki for Henson Digital Puppetry projects, people, characters, and technology.
- Animata - Free, open source real-time animation software commonly used to create digital puppets.
- Mike the talking head - Web page about Mike Normal, one of the earliest examples of digital puppetry.
- Digital Puppet - Taiwanese blog about puppetry applications of interactive technologies.