Compositing
Encyclopedia
Compositing is the combining of visual elements from separate sources into single images, often to create the illusion that all those elements are parts of the same scene. Live-action
shooting for compositing is variously called "chroma key
", "blue screen", "green screen" and other names. Today, most, though not all, compositing is achieved through digital image
manipulation. Pre-digital compositing
techniques, however, go back as far as the trick films of Georges Méliès
in the late 19th century; and some are still in use.
.
programs. In sophisticated installations, subjects, cameras, or both can move about freely while the computer-generated imagery
(CGI) environment changes in real time to maintain correct relationships between the camera angle
s, subjects, and virtual “backgrounds.”
Virtual sets are also used in motion pictures filmmaking
, some of which are photographed entirely in blue or green screen environments; as for example in Sky Captain and the World of Tomorrow
. More commonly, composited backgrounds are combined with sets – both full-size and models – and vehicles, furniture, and other physical objects that enhance the “reality” of the composited visuals. “Sets” of almost unlimited size can be created digitally because compositing software can take the blue or green color at the edges of a backing screen and extend it to fill the rest of the frame outside it. That way, subjects recorded in modest areas can be placed in large virtual vistas. Most common of all, perhaps, are set extensions: digital additions to actual performing environments. In the film, Gladiator
, for example, the arena and first tier seats of the Roman Coliseum
were actually built, while the upper galleries (complete with moving spectators) were computer graphics, composited onto the image above the physical set. For motion pictures originally recorded on film, high-quality video conversions called “digital intermediate
s” are created to enable compositing and the other operations of computerized post production
. Digital compositing is a form of matting, one of four basic compositing methods. The others are physical compositing, multiple exposure
, and background projection.
Partial models are typically used as set extensions such as ceilings or the upper stories of buildings. The model, built to match the actual set but on a much smaller scale, is hung in front of the camera, aligned so that it appears to be part of the set.
Models are often quite large because they must be placed far enough from the camera so that both they and the set far beyond them are in sharp focus.
Glass shots are made by positioning a large pane of glass so that it fills the camera frame, and is far enough away to be held in focus along with the background visible through it. The entire scene is painted on the glass, except for the area revealing the background where action is to take place. This area is left clear. Photographed through the glass, the live action is composited with the painted area. A typical glass shot is the approach to Ashley Wilkes’ plantation in Gone with the Wind
. The plantation and fields are all painted, while the road and the moving figures on it are photographed through the glass area left clear.
A variant uses the opposite technique: most of the area is clear, except for individual elements (photo cutouts or paintings) affixed to the glass. For example, a ranch house
could be added to an empty valley by placing an appropriately scaled and positioned picture of it between the valley and the camera.
An in-camera multiple exposure is made by recording on only one part of each film frame
, rewinding the film to exactly the same start point, exposing a second part, and repeating the process as needed. The resulting negative is a composite of all the individual exposures. (By contrast, a “double exposure” records multiple images on the entire frame area, so that all are partially visible through one another.) Exposing one section at a time is made possible by enclosing the camera lens
(or the whole camera) in a light-tight box fitted with maskable openings, each one corresponding to one of the action areas. Only one opening is revealed per exposure, to record just the action positioned in front of it.
Multiple exposure is difficult because the action in each recording must match that of the others; so multiple exposure composites typically contain only two or three elements. However, as early as 1900 Georges Méliès
used seven-fold exposure in L'homme-orchestre/The One-man Band; and in the 1921 film The Playhouse
, Buster Keaton
used multiple exposures to appear simultaneously as nine different actors on a stage, perfectly synchronizing all nine performances.
the background image on a screen
behind the subjects in the foreground while the camera makes a composite by photographing both at once. The foreground elements conceal the parts of the background image behind them. Sometimes, the background is projected from the front, reflecting off the screen but not the foreground subjects because the screen is made of highly directional, exceptionally reflective material. (The prehistoric opening of 2001: A Space Odyssey
uses front projection.) However, rear projection has been a far more common technique.
In rear projection
, background images (called “plates”, whether they are still pictures or moving) are photographed first. For example, a camera car may drive along streets or roads while photographing the changing scene behind it. In the studio, the resulting “background plate” is loaded into a projector with the film "flipped" (reversed), because it will be projected onto (and through) the back of a translucent screen. A car containing the performers is aligned in front of the screen so that the scenery appears through its rear and/or side windows. A camera in front of the car records both the foreground action and the projected scenery, as the performers pretend to drive.
Like multiple exposure, rear projection is technically difficult. The projector and camera motors must be synchronized to avoid flicker and perfectly aligned behind and before the screen. The foreground must be lit to prevent light spill onto the screen behind it. (For night driving scenes, the foreground lights are usually varied as the car “moves” along.) The projector must use a very strong light source
so that the projected background is as bright as the foreground. Color filming presents additional difficulties, but can be quite convincing, as in the famous crop duster
sequence in Alfred Hitchcock
’s North by Northwest
. Because of its complexity, rear projection has been largely replaced by digital compositing with, for example, the car positioned in front of a blue or green screen.
Traditional matting is the process of compositing two different film elements by printing them, one at a time, onto a duplicate strip of film. After one component is printed on the duplicate, the film is re-wound and the other component is added. Since the film cannot be exposed twice without creating a double exposure
, the blank second area must be masked while the first is printed; then the freshly exposed first area must be masked while the second area is printed. Each masking is performed by a “traveling matte:” a specially altered duplicate shot which lies on top of the copy film stock
.
Like its digital successor, traditional matte photography uses a uniformly colored backing – usually, but not always a special blue or green (fig. 1). Because a matching filter on the camera lens screens out only the backing color, the background area records as black, which, on the camera’s negative film
, will develop clear (fig. 2).
First, a print from the original negative is made on high-contrast film, which records the backing as opaque and the foreground subject as clear (fig. 3). A second high-contrast copy is then made from the first, rendering the backing clear and the foreground opaque (fig. 4).
Next, a three-layer sandwich of film is run through an optical printer
. On the bottom is the unexposed copy film. Above it is the first matte, whose opaque backing color masks the background. On top is the negative of the foreground action. On this pass, the foreground is copied while the background is shielded from exposure by the matte (fig. 5).
Then the process is repeated; but this time, the copy film is masked by the reverse matte, which excludes light from the foreground area already exposed (fig. 6). The top layer contains the background scene (fig. 7), which is now exposed only in the areas protected during the previous pass. The result is a positive print of the combined background and foreground (fig. 8). A copy of this composite print yields a “dupe negative” (fig. 9) that will replace the original foreground shot in the film’s edited negatives.
This means that multi-layer digital composites can easily be made. For example, models of a space station
, a space ship
, and a second space ship could be shot separately against blue screen, each "moving" differently. (In such shots, it is the camera that moves, not the model). The individual shots could then be composited with one another, and finally with a star background. With pre-digital matting, the several extra passes through the optical printer would degrade the film quality and increase the probability of edge artifacts. Elements crossing behind or before one another would pose additional problems.
Live action
In filmmaking, video production, and other media, the term live action refers to cinematography, videography not produced using animation...
shooting for compositing is variously called "chroma key
Chroma key
Chroma key compositing is a technique for compositing two images together. A color range in the top layer is made transparent, revealing another image behind. The chroma keying technique is commonly used in video production and post-production...
", "blue screen", "green screen" and other names. Today, most, though not all, compositing is achieved through digital image
Digital image
A digital image is a numeric representation of a two-dimensional image. Depending on whether or not the image resolution is fixed, it may be of vector or raster type...
manipulation. Pre-digital compositing
Digital compositing
Digital compositing is the process of digitally assembling multiple images to make a final image, typically for print, motion pictures or screen display...
techniques, however, go back as far as the trick films of Georges Méliès
Georges Méliès
Georges Méliès , full name Marie-Georges-Jean Méliès, was a French filmmaker famous for leading many technical and narrative developments in the earliest cinema. He was very innovative in the use of special effects...
in the late 19th century; and some are still in use.
Basic procedure
All compositing involves the replacement of selected parts of an image with other material, usually, but not always, from another image. In the digital method of compositing, software commands designate a narrowly defined color as the part of an image to be replaced. Then every pixel within the designated color range is replaced by the software with a pixel from another image, aligned to appear as part of the original. For example, a TV weather person is recorded in front of a plain blue or green screen, while compositing software replaces only the designated blue or green color with weather mapsSurface weather analysis
Surface weather analysis is a special type of weather map that provides a view of weather elements over a geographical area at a specified time based on information from ground-based weather stations...
.
Typical applications
In television studios, blue or green screens may back news-readers to allow the compositing of stories behind them, before being switched to full-screen display. In other cases, presenters may be completely within compositing backgrounds that are replaced with entire “virtual sets” executed in computer graphicsComputer graphics
Computer graphics are graphics created using computers and, more generally, the representation and manipulation of image data by a computer with help from specialized software and hardware....
programs. In sophisticated installations, subjects, cameras, or both can move about freely while the computer-generated imagery
Computer-generated imagery
Computer-generated imagery is the application of the field of computer graphics or, more specifically, 3D computer graphics to special effects in art, video games, films, television programs, commercials, simulators and simulation generally, and printed media...
(CGI) environment changes in real time to maintain correct relationships between the camera angle
Camera angle
The camera angle marks the specific location at which a camera is placed to take a shot. A scene may be shot from several camera angles. This will give different experience and sometimes emotion. the different camera angles will have different effects on the viewer and how they perceive the scene...
s, subjects, and virtual “backgrounds.”
Virtual sets are also used in motion pictures filmmaking
Filmmaking
Filmmaking is the process of making a film, from an initial story, idea, or commission, through scriptwriting, casting, shooting, directing, editing, and screening the finished product before an audience that may result in a theatrical release or television program...
, some of which are photographed entirely in blue or green screen environments; as for example in Sky Captain and the World of Tomorrow
Sky Captain and the World of Tomorrow
Sky Captain and the World of Tomorrow is a 2004 American pulp adventure science-fiction film written and directed by Kerry Conran in his directorial debut. The film is set in an alternative 1939 and follows the adventures of Polly Perkins , a newspaper reporter, and Harry Joseph "Joe" Sullivan ,...
. More commonly, composited backgrounds are combined with sets – both full-size and models – and vehicles, furniture, and other physical objects that enhance the “reality” of the composited visuals. “Sets” of almost unlimited size can be created digitally because compositing software can take the blue or green color at the edges of a backing screen and extend it to fill the rest of the frame outside it. That way, subjects recorded in modest areas can be placed in large virtual vistas. Most common of all, perhaps, are set extensions: digital additions to actual performing environments. In the film, Gladiator
Gladiator (2000 film)
Gladiator is a 2000 historical epic film directed by Ridley Scott, starring Russell Crowe, Joaquin Phoenix, Connie Nielsen, Ralf Möller, Oliver Reed, Djimon Hounsou, Derek Jacobi, John Shrapnel and Richard Harris. Crowe portrays the loyal Roman General Maximus Decimus Meridius, who is betrayed...
, for example, the arena and first tier seats of the Roman Coliseum
Colosseum
The Colosseum, or the Coliseum, originally the Flavian Amphitheatre , is an elliptical amphitheatre in the centre of the city of Rome, Italy, the largest ever built in the Roman Empire...
were actually built, while the upper galleries (complete with moving spectators) were computer graphics, composited onto the image above the physical set. For motion pictures originally recorded on film, high-quality video conversions called “digital intermediate
Digital intermediate
Digital intermediate is a motion picture finishing process which classically involves digitizing a motion picture and manipulating the color and other image characteristics. It often replaces or augments the photochemical timing process and is usually the final creative adjustment to a movie...
s” are created to enable compositing and the other operations of computerized post production
Post-production
Post-production is part of filmmaking and the video production process. It occurs in the making of motion pictures, television programs, radio programs, advertising, audio recordings, photography, and digital art...
. Digital compositing is a form of matting, one of four basic compositing methods. The others are physical compositing, multiple exposure
Multiple exposure
In photography, a multiple exposure is the superimposition of two or more individual exposures to create a single photograph. The exposure values may or may not be identical to each other.-Overview:...
, and background projection.
Physical compositing
In physical compositing the separate parts of the image are placed together in the photographic frame and recorded in a single exposure. The components are aligned so that they give the appearance of a single image. The most common physical compositing elements are partial models and glass paintings.Partial models are typically used as set extensions such as ceilings or the upper stories of buildings. The model, built to match the actual set but on a much smaller scale, is hung in front of the camera, aligned so that it appears to be part of the set.
Models are often quite large because they must be placed far enough from the camera so that both they and the set far beyond them are in sharp focus.
Glass shots are made by positioning a large pane of glass so that it fills the camera frame, and is far enough away to be held in focus along with the background visible through it. The entire scene is painted on the glass, except for the area revealing the background where action is to take place. This area is left clear. Photographed through the glass, the live action is composited with the painted area. A typical glass shot is the approach to Ashley Wilkes’ plantation in Gone with the Wind
Gone with the Wind (film)
Gone with the Wind is a 1939 American historical epic film adapted from Margaret Mitchell's Pulitzer-winning 1936 novel of the same name. It was produced by David O. Selznick and directed by Victor Fleming from a screenplay by Sidney Howard...
. The plantation and fields are all painted, while the road and the moving figures on it are photographed through the glass area left clear.
A variant uses the opposite technique: most of the area is clear, except for individual elements (photo cutouts or paintings) affixed to the glass. For example, a ranch house
Ranch-style house
Ranch-style houses is a domestic architectural style originating in the United States. First built in the 1920s, the ranch style was extremely popular amongst the booming post-war middle class of the 1940s to 1970s...
could be added to an empty valley by placing an appropriately scaled and positioned picture of it between the valley and the camera.
Multiple exposure
Film frame
In filmmaking, video production, animation, and related fields, a film frame or video frame is one of the many still images which compose the complete moving picture...
, rewinding the film to exactly the same start point, exposing a second part, and repeating the process as needed. The resulting negative is a composite of all the individual exposures. (By contrast, a “double exposure” records multiple images on the entire frame area, so that all are partially visible through one another.) Exposing one section at a time is made possible by enclosing the camera lens
Photographic lens
A camera lens is an optical lens or assembly of lenses used in conjunction with a camera body and mechanism to make images of objects either on photographic film or on other media capable of storing an image chemically or electronically.While in principle a simple convex lens will suffice, in...
(or the whole camera) in a light-tight box fitted with maskable openings, each one corresponding to one of the action areas. Only one opening is revealed per exposure, to record just the action positioned in front of it.
Multiple exposure is difficult because the action in each recording must match that of the others; so multiple exposure composites typically contain only two or three elements. However, as early as 1900 Georges Méliès
Georges Méliès
Georges Méliès , full name Marie-Georges-Jean Méliès, was a French filmmaker famous for leading many technical and narrative developments in the earliest cinema. He was very innovative in the use of special effects...
used seven-fold exposure in L'homme-orchestre/The One-man Band; and in the 1921 film The Playhouse
The Playhouse (film)
The Playhouse is a 1921 silent short film written, directed by, and starring Buster Keaton. The movie runs for 22 minutes, and is most famous for an opening sequence in which Keaton plays every role.- Plot :...
, Buster Keaton
Buster Keaton
Joseph Frank "Buster" Keaton was an American comic actor, filmmaker, producer and writer. He was best known for his silent films, in which his trademark was physical comedy with a consistently stoic, deadpan expression, earning him the nickname "The Great Stone Face".Keaton was recognized as the...
used multiple exposures to appear simultaneously as nine different actors on a stage, perfectly synchronizing all nine performances.
Background projection
Background projection throwsProjection
Projection, projector, or projective may refer to:* The display of an image by devices such as:** Movie projector** Video projector** Overhead projector** Slide projector** Camera obscura** Projection screen...
the background image on a screen
Screen
- Separation or partitioning :* Window screen, a wire mesh that covers a window opening* Fire screen, a device to put in front of a fireplace* Windbreak of trees or shrubs* Windshield , protects the driver of a vehicle...
behind the subjects in the foreground while the camera makes a composite by photographing both at once. The foreground elements conceal the parts of the background image behind them. Sometimes, the background is projected from the front, reflecting off the screen but not the foreground subjects because the screen is made of highly directional, exceptionally reflective material. (The prehistoric opening of 2001: A Space Odyssey
2001: A Space Odyssey (film)
2001: A Space Odyssey is a 1968 epic science fiction film produced and directed by Stanley Kubrick, and co-written by Kubrick and Arthur C. Clarke, partially inspired by Clarke's short story The Sentinel...
uses front projection.) However, rear projection has been a far more common technique.
In rear projection
Rear projection effect
Rear projection is part of many in-camera effects cinematic techniquesin film production for combining foreground performances with pre-filmed backgrounds. It was widely used for many years in driving scenes, or to show other forms of "distant" background motion...
, background images (called “plates”, whether they are still pictures or moving) are photographed first. For example, a camera car may drive along streets or roads while photographing the changing scene behind it. In the studio, the resulting “background plate” is loaded into a projector with the film "flipped" (reversed), because it will be projected onto (and through) the back of a translucent screen. A car containing the performers is aligned in front of the screen so that the scenery appears through its rear and/or side windows. A camera in front of the car records both the foreground action and the projected scenery, as the performers pretend to drive.
Like multiple exposure, rear projection is technically difficult. The projector and camera motors must be synchronized to avoid flicker and perfectly aligned behind and before the screen. The foreground must be lit to prevent light spill onto the screen behind it. (For night driving scenes, the foreground lights are usually varied as the car “moves” along.) The projector must use a very strong light source
Light
Light or visible light is electromagnetic radiation that is visible to the human eye, and is responsible for the sense of sight. Visible light has wavelength in a range from about 380 nanometres to about 740 nm, with a frequency range of about 405 THz to 790 THz...
so that the projected background is as bright as the foreground. Color filming presents additional difficulties, but can be quite convincing, as in the famous crop duster
Agricultural aircraft
An agricultural aircraft is an aircraft that has been built or converted for agricultural use - usually aerial application of pesticides or fertilizer ; in these roles they are referred to as "crop dusters" or "top dressers"...
sequence in Alfred Hitchcock
Alfred Hitchcock
Sir Alfred Joseph Hitchcock, KBE was a British film director and producer. He pioneered many techniques in the suspense and psychological thriller genres. After a successful career in British cinema in both silent films and early talkies, Hitchcock moved to Hollywood...
’s North by Northwest
North by Northwest
North by Northwest is a 1959 American thriller film directed by Alfred Hitchcock, starring Cary Grant, Eva Marie Saint and James Mason, and featuring Leo G. Carroll and Martin Landau...
. Because of its complexity, rear projection has been largely replaced by digital compositing with, for example, the car positioned in front of a blue or green screen.
Matting
Double Exposure
Double exposure is a photographic technique in which two images are captured and combined into a single image.Double exposure may also refer to:* Double patterning, a technique for improving the resolution of patterning semiconductors...
, the blank second area must be masked while the first is printed; then the freshly exposed first area must be masked while the second area is printed. Each masking is performed by a “traveling matte:” a specially altered duplicate shot which lies on top of the copy film stock
Film stock
Film stock is photographic film on which filmmaking of motion pictures are shot and reproduced. The equivalent in television production is video tape.-1889–1899:...
.
Like its digital successor, traditional matte photography uses a uniformly colored backing – usually, but not always a special blue or green (fig. 1). Because a matching filter on the camera lens screens out only the backing color, the background area records as black, which, on the camera’s negative film
Negative (photography)
In photography, a negative may refer to three different things, although they are all related.-A negative:Film for 35 mm cameras comes in long narrow strips of chemical-coated plastic or cellulose acetate. As each image is captured by the camera onto the film strip, the film strip advances so that...
, will develop clear (fig. 2).
First, a print from the original negative is made on high-contrast film, which records the backing as opaque and the foreground subject as clear (fig. 3). A second high-contrast copy is then made from the first, rendering the backing clear and the foreground opaque (fig. 4).
Next, a three-layer sandwich of film is run through an optical printer
Optical printer
An optical printer is a device consisting of one or more film projectors mechanically linked to a movie camera. It allows filmmakers to re-photograph one or more strips of film...
. On the bottom is the unexposed copy film. Above it is the first matte, whose opaque backing color masks the background. On top is the negative of the foreground action. On this pass, the foreground is copied while the background is shielded from exposure by the matte (fig. 5).
Then the process is repeated; but this time, the copy film is masked by the reverse matte, which excludes light from the foreground area already exposed (fig. 6). The top layer contains the background scene (fig. 7), which is now exposed only in the areas protected during the previous pass. The result is a positive print of the combined background and foreground (fig. 8). A copy of this composite print yields a “dupe negative” (fig. 9) that will replace the original foreground shot in the film’s edited negatives.
Advantages of digital mattes
Digital matting has replaced the traditional approach for two reasons. In the old system, the five separate strips of film (foreground and background originals, positive and negative mattes, and copy stock) could drift slightly out of registration, resulting in halos and other edge artifacts in the result. Done correctly, digital matting is perfect, down to the single-pixel level. Also, the final dupe negative was a “third generation” copy, and film loses quality each time it is copied. Digital images can be copied without quality loss.This means that multi-layer digital composites can easily be made. For example, models of a space station
Space station
A space station is a spacecraft capable of supporting a crew which is designed to remain in space for an extended period of time, and to which other spacecraft can dock. A space station is distinguished from other spacecraft used for human spaceflight by its lack of major propulsion or landing...
, a space ship
Spacecraft
A spacecraft or spaceship is a craft or machine designed for spaceflight. Spacecraft are used for a variety of purposes, including communications, earth observation, meteorology, navigation, planetary exploration and transportation of humans and cargo....
, and a second space ship could be shot separately against blue screen, each "moving" differently. (In such shots, it is the camera that moves, not the model). The individual shots could then be composited with one another, and finally with a star background. With pre-digital matting, the several extra passes through the optical printer would degrade the film quality and increase the probability of edge artifacts. Elements crossing behind or before one another would pose additional problems.
See also
- Alpha compositingAlpha compositingIn computer graphics, alpha compositing is the process of combining an image with a background to create the appearance of partial or full transparency. It is often useful to render image elements in separate passes, and then combine the resulting multiple 2D images into a single, final image in a...
- Broadcast designerBroadcast designerA broadcast designer is a person involved with creating graphic designs and electronic media incorporated in television productions that are used by character generator operators...
- Character generatorCharacter generatorA character generator, often abbreviated as CG, is a device or software that produces static or animated text for keying into a video stream. Modern character generators are computer-based, and can generate graphics as well as text...
- Clean feed (TV)Clean feed (TV)In television technology, clean feed is a term that describes a signal which has not come from the main output of the Video switcher, such as the output of a vision mixer before the downstream keyer stage - the clean feed is identical to the main program output but without any captions keyed into...
- Compositing window managerCompositing window managerA compositing window manager is a type of window manager. A window manager is software that draws a graphical user interface on a computer display – it positions windows, draws additional elements on windows , and controls how windows interact with each other, and with the rest of the desktop...
- Digital assetDigital assetA digital asset is any item of text or media that has been formatted into a binary source that includes the right to use it. A digital file without the right to use it is not an asset. Digital assets are categorised in three major groups which may be defined as textual content , images and...
- Digital compositingDigital compositingDigital compositing is the process of digitally assembling multiple images to make a final image, typically for print, motion pictures or screen display...
- Digital on-screen graphicDigital on-screen graphicA digital on-screen graphic is a watermark-like station logo that many television broadcasters overlay over a portion of the screen-area of their programs to identify the channel...
(Bug or DOG) - Graphics coordinatorGraphics coordinatorA graphics coordinator, GC, or font assist is an individual who works, usually on a television show, as a producer of on-air still and motion graphics. The graphics coordinator decides what content should be displayed on-air - such as on a fullpage or a lower third...
- Motion graphic design
- Primatte chromakey technologyPrimatte chromakey technologyPrimatte is a high-end chroma key technology used in motion picture, television and photographic host applications to remove solid colored backgrounds and replace them with transparency to facilitate ‘background replacement’...
Further reading
- T. Porter and T. Duff, "Compositing Digital Images", Proceedings of SIGGRAPHSIGGRAPHSIGGRAPH is the name of the annual conference on computer graphics convened by the ACM SIGGRAPH organization. The first SIGGRAPH conference was in 1974. The conference is attended by tens of thousands of computer professionals...
'84, 18 (1984). - Ron Brinkmann, The Art and Science of Digital Compositing (ISBN 0-12-133960-2)
- Steve Wright, Digital Compositing for Film and Video, Second Edition (ISBN 0-240-80760-X)
- American CinematographerAmerican CinematographerAmerican Cinematographer is a monthly magazine published by the American Society of Cinematographers.American Cinematographer focuses on the art and craft of cinematography, going behind the scenes on domestic and international productions of all shapes and sizes...
Manual, 2nd ed., Mascelli, Joseph V., A.S.C. and Miller, Arthur, A.S.C, eds. Los Angeles, 1966, p. 500 ff.