Primatte chromakey technology
Encyclopedia
Primatte is a high-end chroma key
technology used in motion picture, television and photographic host applications to remove solid colored backgrounds (greenscreen or bluescreen usually) and replace them with transparency to facilitate ‘background replacement’. It uses a unique algorithm
based on three multi-faceted polyhedron
s floating in RGB
colorspace that are used to isolate color regions in the foreground image. Primatte is often referred to as a compositing
technology and is usually used as a plug-in for host products such as Adobe After Effects
, Adobe Photoshop
, Autodesk Media and Entertainment
Inferno or Flame, Eyeon Fusion
and several other compositing and editing software packages.
If the foreground object was filmed close to the backing screen or with less than ideal lighting conditions, the foreground object will usually have ‘spill’ somewhere on it. This is most common when filming blonde people against a blue or greenscreen as their translucent hair will absorb the backing screen color. Using the Primatte tool set, this ‘colorspill’ can be removed and replaced with several options to achieve a realistic result.
Another problem is chromakeying images that were created on film stock
is that some film stocks have a distinct ‘film grain
’ to them. A large amount of film grain will often cause rough edges around the foreground object. Again, using the Primatte tool set, these rough edges can be minimized to achieve a smooth transition between the edges of the ‘chroma keyed’ foreground object and the user selected background image.
It was initially released as a stand-alone product on Silicon Graphics
workstations but it was later determined to be more useful as a ‘plug-in’ actuated from within a host application. This alleviated the need to save the images, exit the host compositing application, start Primatte, load the images, create the chroma key, save the images, start the host application, load the images and continue creating the composition. Since then it has been ported to almost every major compositing and editing software application on the market today. The current version is the fourth generation of the Primatte technology and has enhanced features such as an ‘Auto-Compute’ feature that automatically detects the backing screen color, eliminates it and does clean-up on the foreground and backing screen area noise making Primatte almost a 'one step' operation. It is now available on the Microsoft Windows
, Red Hat Linux
, SGI IRIX
and the Apple Macintosh
platforms.
that can be easily explained to help a user achieve maximum effectiveness with the tool. Basically Primatte segments all the colors in the foreground image into one of four separate categories. The result is a 'spill suppressed' foreground image and a matte
which is used to apply the modified foreground to a suitable background.
Primatte works in 3D RGB color space. Here is a visual representation of the Primatte algorithm after an image has been processed.
By operating the Primatte interface, the user essentially creates three concentric
, multi-faceted polyhedrons. These can be pictured as three globes (or polyhedrons or polys), one within the other, which share a common center point. The creation of these polyhedrons separates all possible foreground colors into one of four regions; inside the small polyhedron (1), between the small and medium polyhedrons (2), between the medium and the large polyhedrons (3) and outside the large polyhedron (4).
The four regions created are described as follows:
Region 1 (inside the small polyhedron) - This region contains all of the foreground image colors that are considered 100% background. These are the green or blue or whatever colors that were used as the backing color of the foreground image.
Region 2 (between the small and medium polyhedrons) - This region contains all the foreground colors that are at the edges of the foreground object(s), in glass, glass reflections, shadows, sheets of water and other transparent and semi-transparent color regions. These color regions also have spill suppression applied to them to remove color spill from the backing screen.
Region 3 (between the medium and large polyhedrons) - This region contains all the foreground image colors that are 100% foreground but have spill suppression applied to them to remove color spill from the backing screen. Otherwise they are 100% solid foreground colors.
Region 4 (outside the large polyhedron) - This region contains all the 100% foreground image colors that are not modified from the original foreground image. There is no spill suppression applied to these colors.
In the first step in using Primatte, Select BG Color, the user is asked to indicate the backing color on the original foreground image. The sample should usually be taken from a 'medium shaded' area near the foreground object. By 'medium shaded' area, it is meant that if green is the backing color and the green area of the foreground image has many shades of green ranging from very pale green to almost black, a shade of green in-between these extreme ranges should be chosen. If good results are not obtained using this sample, Primatte should be reset and another sample taken using a slightly darker or lighter shade of green. The first sample of Primatte often determines the final result as the center point of all three polyhedrons is created based on this first sample.
NOTE: An alternative method of keying in the Primatte Version 4 release is an Auto-Compute button the algorithmically detects the background color, eliminates it and does foreground and background noise removal. All the user might have to do is clean up some spill in the image.
A single pixel
may be selected or a range of pixels (snail trail or rectangular sample). If a range of pixels is taken, the sample will be averaged to get a single color sample. This single pixel or averaged color sample then becomes the center of the small polyhedron. A few other shades around that color are included in the original small polyhedron.
After making a sample of the backing screen color in the first step, the result is a small golf ball-shaped poly as shown in the following image.
The second step in using Primatte is to clean up the backing color area by adding additional shades of green or blue to the small poly. This second step, Clean BG Noise, is usually executed while viewing the black and white Matte View.
While in the Clean BG Noise sampling mode, the user samples the white milky regions as shown in the left-hand image. As the user samples these regions, they turn to black as shown in the right-hand image above.
What is happening in the Primatte algorithm is that these new shades of green (the white milky areas) are added to the small poly where all the shades of green or blue are moved. The next two images show the new pixels sampled (white dots) in relation to the small poly and the image next to it shows how the small poly extends outward to encompass the newly sampled colors into the small poly.
The advantage of this technique is that the polyhedron distorts to enclose only the shades of green that are in the backing screen. Other shades of green around these colors are left undisturbed in the foreground. Other chromakeyers expand from a golf ball-sized shape to a baseball to a basketball to a beach ball. Since it expands in all directions, many shades of green are relegated to 100% background making it hard to get good edges around the foreground objects.
Now that the user has created a small polyhedron, he must shape the medium and large polys. A default medium and large poly are both automatically created and are then modified based on the next couple of Primatte operations. The third Primatte step, Clean FG Noise, is to sample and eliminate gray areas in the 100% foreground area of the image.
Again, the user makes several samples on the dark, grayish areas on the foreground object until it is solid white in color. Primatte is shaping the large polyhedron with each color region that is sampled. Care should be taken in both this and the previous steps to not sample too close to the edges of the foreground object. Getting too close to the foreground object's edges will result in hard edges around the foreground object. Primatte uses these samples to modify and shape the medium and large polys to the desired shape. At this point, the matte or key has been created and would allow the foreground objects to be composited into a new background image.
If the user changes the display mode from the black and white Matte View to the color Composite View, there is usually 'color spill' on the edges (and sometimes the center) of the foreground objects. When on the edges of the foreground object, this spill comes from where the edges of the foreground object blended into the backing color. If it is on the center of the foreground object, it usually results from reflected color from the backing screen. The next Primatte step, either Spill Sponge, Fine Tuning or Spill(-), can now be used to eliminate this spill color.
What is happening in the Primatte algorithm while this next step is performed? Here is what the various tools in Primatte do to the Polyhedrons when they are used:
The Spill Sponge bulges the large polyhedron in the color region specified. A color region is specified by clicking on the image in a particular area with spill present. For example, if the user clicks on some spill on the cheek of a foreground person, Primatte goes to the section of the large polyhedron closest to that particular flesh tone and bulges the polyhedron there. As a result, the flesh tones move from outside the large poly to in-between the medium and large polys. This is Region 3 and is 100% foreground with spill suppression. As a result of the suppression, the spill is removed from that cheek color and all other shades of that color on the foreground. The user would then continue to sample areas of the image where spill exists and each sample would remove spill from another color region.
When all spill has been removed, the user should have a final composite. As a last step, go back to the Matte View and make sure that gray, transparent areas have not appeared in the foreground area. If there are any, the Matte Sponge operation mode should be selected and those gray pixels are sampled until they have all turned white again.
The Matte Sponge and Spill Sponge tools bulge or dent the polyhedrons a pre-selected amount. If the desired results are not achieved or the results are too extreme for the image, a manual method can be applied. The user should choose the Fine Tuning sliders, select a color region of interest and then move the appropriate slider to get the desired results.
For example, to remove spill, select a region of the composite image with spill on it. Move the spill or large poly slider to the right a little bit, the large poly will bulge and the spill should disappear. Move it a little more, if necessary. Moving this slider to the right removes spill (moves the colors from outside the large poly to between the medium and large polyhedrons) and moving it to the left, dents the large poly and moves that color region to outside the large poly.
If the user samples a foreground object shadow and then moves the Matte or medium poly slider to the right, the shadow will become more transparent. This is useful for matching composited shadows to shadows on the plate photography. It can also be used to make clouds or smoke more transparent.
If some foreground detail disappears during the composite, the user can select where the detail should be and then move the detail or small poly slider to the left. This dents the small poly in that color region and releases the detail pixels from the small poly into the visible region between the small and medium polyhedrons.
The Spill Sponge and Matte Sponge tools are 'shortcut tools' that automatically move the sliders a pre-selected amount as a timesaving step for the user. Other 'shortcut tools' include the Make FG Trans. tool and the Restore Detail tool.
These 'shortcut tools' are one-step operations where the user clicks on a color region of interest and Primatte performs a pre-calculated operation. Most operations using Primatte would only require these tools, but the manual operation of the sliders is always an option.
The Spill(-) tool bulges the large poly a small amount incrementally in the color region that is clicked on and the Spill(+) tool dents it a small amount with each click. The Matte(-) and Matte(+) tools do the same to the medium poly and the Detail(-) and Detail(+) do it to the small poly.
Chroma key
Chroma key compositing is a technique for compositing two images together. A color range in the top layer is made transparent, revealing another image behind. The chroma keying technique is commonly used in video production and post-production...
technology used in motion picture, television and photographic host applications to remove solid colored backgrounds (greenscreen or bluescreen usually) and replace them with transparency to facilitate ‘background replacement’. It uses a unique algorithm
Algorithm
In mathematics and computer science, an algorithm is an effective method expressed as a finite list of well-defined instructions for calculating a function. Algorithms are used for calculation, data processing, and automated reasoning...
based on three multi-faceted polyhedron
Polyhedron
In elementary geometry a polyhedron is a geometric solid in three dimensions with flat faces and straight edges...
s floating in RGB
RGB color model
The RGB color model is an additive color model in which red, green, and blue light is added together in various ways to reproduce a broad array of colors...
colorspace that are used to isolate color regions in the foreground image. Primatte is often referred to as a compositing
Compositing
Compositing is the combining of visual elements from separate sources into single images, often to create the illusion that all those elements are parts of the same scene. Live-action shooting for compositing is variously called "chroma key", "blue screen", "green screen" and other names. Today,...
technology and is usually used as a plug-in for host products such as Adobe After Effects
Adobe After Effects
Adobe After Effects is a digital motion graphics and compositing software published by Adobe Systems, used in the post-production process of filmmaking and television production. Its main uses are the origination of 2D and 2.5D animation, visual effects compositing and finishing...
, Adobe Photoshop
Adobe Photoshop
Adobe Photoshop is a graphics editing program developed and published by Adobe Systems Incorporated.Adobe's 2003 "Creative Suite" rebranding led to Adobe Photoshop 8's renaming to Adobe Photoshop CS. Thus, Adobe Photoshop CS5 is the 12th major release of Adobe Photoshop...
, Autodesk Media and Entertainment
Autodesk Media and Entertainment
Autodesk Media and Entertainment, formerly Discreet, is based in Montreal, Quebec as the entertainment division of Autodesk. This division produces software used in feature films, television commercials and computer games. It also provides products for management and distribution to complement its...
Inferno or Flame, Eyeon Fusion
Eyeon Fusion
Fusion is an image compositing software program created by eyeon Software Inc. It is typically used to create visual effects and digital compositing for film, HD and commercials...
and several other compositing and editing software packages.
Workflow
Primatte is usually activated on a foreground image with a person or other foreground object filmed or digitized against a solid colored background or backing screen; usually a bluescreen or a greenscreen. Using the Primatte tool set, the solid colored background area is removed and replaced with transparency. This allows the user to replace the solid colored background with a background image of his choice.If the foreground object was filmed close to the backing screen or with less than ideal lighting conditions, the foreground object will usually have ‘spill’ somewhere on it. This is most common when filming blonde people against a blue or greenscreen as their translucent hair will absorb the backing screen color. Using the Primatte tool set, this ‘colorspill’ can be removed and replaced with several options to achieve a realistic result.
Another problem is chromakeying images that were created on film stock
Film stock
Film stock is photographic film on which filmmaking of motion pictures are shot and reproduced. The equivalent in television production is video tape.-1889–1899:...
is that some film stocks have a distinct ‘film grain
Film grain
Film grain or granularity is the random optical texture of processed photographic film due to the presence of small particles of a metallic silver, or dye clouds, developed from silver halide that have received enough photons. While film grain is a function of such particles it is not the same...
’ to them. A large amount of film grain will often cause rough edges around the foreground object. Again, using the Primatte tool set, these rough edges can be minimized to achieve a smooth transition between the edges of the ‘chroma keyed’ foreground object and the user selected background image.
History
The Primatte algorithm was created by Yasushi Mishima while working at IMAGICA Corporation in Tokyo, Japan in 1992. The basic algorithm utilized in Primatte was originally presented at the 8th NICOGRAPH Conference and the 23rd Imaging Technology Conference and a U.S. patent was granted in 1994.It was initially released as a stand-alone product on Silicon Graphics
Silicon Graphics
Silicon Graphics, Inc. was a manufacturer of high-performance computing solutions, including computer hardware and software, founded in 1981 by Jim Clark...
workstations but it was later determined to be more useful as a ‘plug-in’ actuated from within a host application. This alleviated the need to save the images, exit the host compositing application, start Primatte, load the images, create the chroma key, save the images, start the host application, load the images and continue creating the composition. Since then it has been ported to almost every major compositing and editing software application on the market today. The current version is the fourth generation of the Primatte technology and has enhanced features such as an ‘Auto-Compute’ feature that automatically detects the backing screen color, eliminates it and does clean-up on the foreground and backing screen area noise making Primatte almost a 'one step' operation. It is now available on the Microsoft Windows
Microsoft Windows
Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
, Red Hat Linux
Red Hat Linux
Red Hat Linux, assembled by the company Red Hat, was a popular Linux based operating system until its discontinuation in 2004.Red Hat Linux 1.0 was released on November 3, 1994...
, SGI IRIX
IRIX
IRIX is a computer operating system developed by Silicon Graphics, Inc. to run natively on their 32- and 64-bit MIPS architecture workstations and servers. It was based on UNIX System V with BSD extensions. IRIX was the first operating system to include the XFS file system.The last major version...
and the Apple Macintosh
Macintosh
The Macintosh , or Mac, is a series of several lines of personal computers designed, developed, and marketed by Apple Inc. The first Macintosh was introduced by Apple's then-chairman Steve Jobs on January 24, 1984; it was the first commercially successful personal computer to feature a mouse and a...
platforms.
The Algorithm
The Primatte chromakey algorithm is a sophisticated method of color space segmentationSegmentation (image processing)
In computer vision, segmentation refers to the process of partitioning a digital image into multiple segments . The goal of segmentation is to simplify and/or change the representation of an image into something that is more meaningful and easier to analyze...
that can be easily explained to help a user achieve maximum effectiveness with the tool. Basically Primatte segments all the colors in the foreground image into one of four separate categories. The result is a 'spill suppressed' foreground image and a matte
Matte (filmmaking)
Mattes are used in photography and special effects filmmaking to combine two or more image elements into a single, final image. Usually, mattes are used to combine a foreground image with a background image . In this case, the matte is the background painting...
which is used to apply the modified foreground to a suitable background.
Primatte works in 3D RGB color space. Here is a visual representation of the Primatte algorithm after an image has been processed.
By operating the Primatte interface, the user essentially creates three concentric
Concentric
Concentric objects share the same center, axis or origin with one inside the other. Circles, tubes, cylindrical shafts, disks, and spheres may be concentric to one another...
, multi-faceted polyhedrons. These can be pictured as three globes (or polyhedrons or polys), one within the other, which share a common center point. The creation of these polyhedrons separates all possible foreground colors into one of four regions; inside the small polyhedron (1), between the small and medium polyhedrons (2), between the medium and the large polyhedrons (3) and outside the large polyhedron (4).
The four regions created are described as follows:
Region 1 (inside the small polyhedron) - This region contains all of the foreground image colors that are considered 100% background. These are the green or blue or whatever colors that were used as the backing color of the foreground image.
Region 2 (between the small and medium polyhedrons) - This region contains all the foreground colors that are at the edges of the foreground object(s), in glass, glass reflections, shadows, sheets of water and other transparent and semi-transparent color regions. These color regions also have spill suppression applied to them to remove color spill from the backing screen.
Region 3 (between the medium and large polyhedrons) - This region contains all the foreground image colors that are 100% foreground but have spill suppression applied to them to remove color spill from the backing screen. Otherwise they are 100% solid foreground colors.
Region 4 (outside the large polyhedron) - This region contains all the 100% foreground image colors that are not modified from the original foreground image. There is no spill suppression applied to these colors.
In the first step in using Primatte, Select BG Color, the user is asked to indicate the backing color on the original foreground image. The sample should usually be taken from a 'medium shaded' area near the foreground object. By 'medium shaded' area, it is meant that if green is the backing color and the green area of the foreground image has many shades of green ranging from very pale green to almost black, a shade of green in-between these extreme ranges should be chosen. If good results are not obtained using this sample, Primatte should be reset and another sample taken using a slightly darker or lighter shade of green. The first sample of Primatte often determines the final result as the center point of all three polyhedrons is created based on this first sample.
NOTE: An alternative method of keying in the Primatte Version 4 release is an Auto-Compute button the algorithmically detects the background color, eliminates it and does foreground and background noise removal. All the user might have to do is clean up some spill in the image.
A single pixel
Pixel
In digital imaging, a pixel, or pel, is a single point in a raster image, or the smallest addressable screen element in a display device; it is the smallest unit of picture that can be represented or controlled....
may be selected or a range of pixels (snail trail or rectangular sample). If a range of pixels is taken, the sample will be averaged to get a single color sample. This single pixel or averaged color sample then becomes the center of the small polyhedron. A few other shades around that color are included in the original small polyhedron.
After making a sample of the backing screen color in the first step, the result is a small golf ball-shaped poly as shown in the following image.
The second step in using Primatte is to clean up the backing color area by adding additional shades of green or blue to the small poly. This second step, Clean BG Noise, is usually executed while viewing the black and white Matte View.
While in the Clean BG Noise sampling mode, the user samples the white milky regions as shown in the left-hand image. As the user samples these regions, they turn to black as shown in the right-hand image above.
What is happening in the Primatte algorithm is that these new shades of green (the white milky areas) are added to the small poly where all the shades of green or blue are moved. The next two images show the new pixels sampled (white dots) in relation to the small poly and the image next to it shows how the small poly extends outward to encompass the newly sampled colors into the small poly.
The advantage of this technique is that the polyhedron distorts to enclose only the shades of green that are in the backing screen. Other shades of green around these colors are left undisturbed in the foreground. Other chromakeyers expand from a golf ball-sized shape to a baseball to a basketball to a beach ball. Since it expands in all directions, many shades of green are relegated to 100% background making it hard to get good edges around the foreground objects.
Now that the user has created a small polyhedron, he must shape the medium and large polys. A default medium and large poly are both automatically created and are then modified based on the next couple of Primatte operations. The third Primatte step, Clean FG Noise, is to sample and eliminate gray areas in the 100% foreground area of the image.
Again, the user makes several samples on the dark, grayish areas on the foreground object until it is solid white in color. Primatte is shaping the large polyhedron with each color region that is sampled. Care should be taken in both this and the previous steps to not sample too close to the edges of the foreground object. Getting too close to the foreground object's edges will result in hard edges around the foreground object. Primatte uses these samples to modify and shape the medium and large polys to the desired shape. At this point, the matte or key has been created and would allow the foreground objects to be composited into a new background image.
If the user changes the display mode from the black and white Matte View to the color Composite View, there is usually 'color spill' on the edges (and sometimes the center) of the foreground objects. When on the edges of the foreground object, this spill comes from where the edges of the foreground object blended into the backing color. If it is on the center of the foreground object, it usually results from reflected color from the backing screen. The next Primatte step, either Spill Sponge, Fine Tuning or Spill(-), can now be used to eliminate this spill color.
What is happening in the Primatte algorithm while this next step is performed? Here is what the various tools in Primatte do to the Polyhedrons when they are used:
The Spill Sponge bulges the large polyhedron in the color region specified. A color region is specified by clicking on the image in a particular area with spill present. For example, if the user clicks on some spill on the cheek of a foreground person, Primatte goes to the section of the large polyhedron closest to that particular flesh tone and bulges the polyhedron there. As a result, the flesh tones move from outside the large poly to in-between the medium and large polys. This is Region 3 and is 100% foreground with spill suppression. As a result of the suppression, the spill is removed from that cheek color and all other shades of that color on the foreground. The user would then continue to sample areas of the image where spill exists and each sample would remove spill from another color region.
When all spill has been removed, the user should have a final composite. As a last step, go back to the Matte View and make sure that gray, transparent areas have not appeared in the foreground area. If there are any, the Matte Sponge operation mode should be selected and those gray pixels are sampled until they have all turned white again.
The Matte Sponge and Spill Sponge tools bulge or dent the polyhedrons a pre-selected amount. If the desired results are not achieved or the results are too extreme for the image, a manual method can be applied. The user should choose the Fine Tuning sliders, select a color region of interest and then move the appropriate slider to get the desired results.
For example, to remove spill, select a region of the composite image with spill on it. Move the spill or large poly slider to the right a little bit, the large poly will bulge and the spill should disappear. Move it a little more, if necessary. Moving this slider to the right removes spill (moves the colors from outside the large poly to between the medium and large polyhedrons) and moving it to the left, dents the large poly and moves that color region to outside the large poly.
If the user samples a foreground object shadow and then moves the Matte or medium poly slider to the right, the shadow will become more transparent. This is useful for matching composited shadows to shadows on the plate photography. It can also be used to make clouds or smoke more transparent.
If some foreground detail disappears during the composite, the user can select where the detail should be and then move the detail or small poly slider to the left. This dents the small poly in that color region and releases the detail pixels from the small poly into the visible region between the small and medium polyhedrons.
The Spill Sponge and Matte Sponge tools are 'shortcut tools' that automatically move the sliders a pre-selected amount as a timesaving step for the user. Other 'shortcut tools' include the Make FG Trans. tool and the Restore Detail tool.
These 'shortcut tools' are one-step operations where the user clicks on a color region of interest and Primatte performs a pre-calculated operation. Most operations using Primatte would only require these tools, but the manual operation of the sliders is always an option.
The Spill(-) tool bulges the large poly a small amount incrementally in the color region that is clicked on and the Spill(+) tool dents it a small amount with each click. The Matte(-) and Matte(+) tools do the same to the medium poly and the Detail(-) and Detail(+) do it to the small poly.