Match moving
Encyclopedia
In cinematography
, match moving is a visual-effects, cinematic techniques
that allows the insertion of computer graphics into live-action footage with correct position, scale, orientation, and motion relative to the photographed objects in the shot. The term is used loosely to refer to several different ways of extracting motion
information from a motion picture, particularly camera movement. Match moving is related to rotoscoping and photogrammetry
. It is sometimes referred to as motion tracking.
Match moving is sometimes confused with motion capture
, which is a hardware technology for recording the motion of objects, often human actors, in a controlled environment using special cameras and sensors. It is also distinct from motion control photography
which uses a robot
ic arm to execute multiple identical camera moves. Match moving, by contrast, is typically a software-based technology, applied after the fact to normal footage recorded in uncontrolled environments with an ordinary camera.
Match moving is primarily used to track the movement of a camera through a shot so that an identical virtual camera move can be reproduced in a 3D animation program. When new animated elements are composited back into the original live-action shot, they will appear in perfectly-matched perspective and therefore appear seamless.
As it is mostly software-based, match moving has become increasingly affordable as the cost of computer power has declined; it is now an established visual-effects tool and is even used in live television broadcasts
as part of providing effects such as the virtual yellow-down-line
in American football.
features. A feature is a specific point in the image that a tracking algorithm can lock onto and follow through multiple frames (SynthEyes calls them blips). Often features are selected because they are bright/dark spots, edges or corners depending on the particular tracking algorithm. Popular programs use template matching
based on NCC score and RMS error
. What is important is that each feature represents a specific point on the surface of a real object. As a feature is tracked it becomes a series of two-dimensional coordinates that represent the position of the feature across a series of frames. This series is referred to as a track. Once tracks have been created they can be used immediately for 2D motion tracking, or then be used to calculate 3D information.
.
To explain further: when a point on the surface of a three dimensional object is photographed its position in the 2D frame can be calculated by a 3D projection
function. We can consider a camera to be an abstraction that holds all the parameters necessary to model a camera in a real or virtual world. Therefore a camera is a vector that includes as its elements the position of the camera, its orientation, focal length, and other possible paremeters that define how the camera focuses light onto the film plane
. Exactly how this vector is constructed is not important as long as there is a compatible projection function P.
The projection function P takes as its input a camera vector (denoted camera) and another vector the position of a 3D point in space (denoted xyz) and returns a 2D point that has been projected onto a plane in front of the camera (denoted XY). We can express this:
The projection function transforms the 3D point and strips away the component of depth. Without knowing the depth of the component an inverse projection function can only return a set of possible 3D points, that form a line emanating from the center of the camera and passing through the projected 2D point. We can express the inverse projection as:
or
Let's say we are in a situation where the features we are tracking are on the surface of a rigid object such as a building. Since we know that the real point xyz will remain in the same place in real space from one frame of the image to the next we can make the point a constant even though we do not know where it is. So:
where the subscripts i and j refer to arbitrary frames in the shot we are analyzing. Since this is always true then we know that:
Because the value of XYi has been determined for all frames that the feature is tracked through by the tracking program, we can solve the reverse projection function between any two frames as long as P'(camerai, XYi) ∩ P'(cameraj, XYj) is a small set. Set of possible camera vectors that solve the equation at i and j (denoted Cij).
So there is a set of camera vector pairs Cij for which the intersection of the inverse projections of two points XYi and XYj is a non-empty, hopefully small, set centering around a theoretical stationary point xyz .
In other words, imagine a black point floating in a white void and a camera. For any position in space that we place the camera, there is a set of corresponding parameters (orientation, focal length, etc) that will photograph that black point exactly the same way. Since C has an infinite number of members, one point is never enough to determine the actual camera position.
As we start adding tracking points, we can narrow the possible camera positions. For example if we have a set of points {xyzi,0,...,xyzi,n} and {xyzj,0,...,xyzj,n} where i and j still refer to frames and n is an index to one of many tracking points we are following. We can derive a set of camera vector pair sets {Ci,j,0,...,Ci,j,n}.
In this way multiple tracks allow us to narrow the possible camera parameters. The set of possible camera parameters that fit, F, is the intersection of all sets:
The fewer elements are in this set the closer we can come to extracting the actual parameters of the camera. In reality errors introduced to the tracking process require a more statistical approach to determining a good camera vector for each frame, optimization
algorithms and bundle block adjustment are often utilized. Unfortunately there are so many elements to a camera vector that when every parameter is free we still might not be able to narrow F down to a single possibility no matter how many features we track. The more we can restrict the various parameters, especially focal length, the easier it becomes to pinpoint the solution.
In all, the 3D solving process is the process of narrowing down the possible solutions to the motion of the camera until we reach one that suits the needs of the composite we are trying to create.
. Since point clouds often reveal some of the shape of the 3D scene they can be used as a reference for placing synthetic objects or by a reconstruction program to create a 3D version of the actual scene.
. In this particular case we are referring to using match moving software to reconstruct a scene from incidental footage.
A reconstruction program can create three-dimensional objects that mimic the real objects from the photographed scene. Using data from the point cloud and the user's estimation, the program can create a virtual object and then extract a texture from the footage that can be projected onto the virtual object as a surface texture.
, Adobe After Effects
and Discreet Combustion, have two-dimensional motion tracking capabilities. This feature translates images in two-dimensional space and can add effects such as motion blur in an attempt to eliminate relative motion between two features of two moving images. This technique is sufficient to create verisimilitude when the two images do not include major changes in camera perspective. For example a billboard deep in the background of a shot can often be replaced using two-dimensional tracking.
Three-dimensional match moving tools make it possible to extrapolate three-dimensional information from two-dimensional photography. Programs capable of 3D match moving include:
These programs allow users to derive camera movement and other relative motion from arbitrary footage. The tracking information can be transferred to computer graphics software such as Blender
, 3ds Max, Maya
or LightWave
and used to animate virtual cameras and CGI objects.
The advantage of automatic tracking is that the computer can create many points faster than a human can. A large number of points can be analyzed with statistics
to determine the most reliable data. The disadvantage of automatic tracking is that, depending on the algorithm, the computer can be easily confused as it tracks objects through the scene. Automatic tracking methods are particularly ineffective in shots involving fast camera motion such as that seen with hand-held camera work and in shots with repetitive subject matter like small tiles or any sort of regular pattern where one area isn't very distinct from another. This tracking method also suffers when a shot contains a large amount of motion blur, making the small details it needs harder to distinguish.
The advantage of interactive tracking is that a human user can follow features through an entire scene and will not be confused by features that are not rigid. A human user can also determine where features are in a shot that suffers from motion blur; it is extremely difficult for an automatic tracker to correctly find features with high amounts of motion blur. The disadvantage of interactive tracking is that the user will inevitably introduce small errors as they follow objects through the scene, which can lead to drift.
Professional-level motion tracking is usually achieved using a combination of interactive and automatic techniques. An artist can remove points that are clearly anomalous and use tracking mattes to block confusing information out of the automatic tracking process. Tracking mattes are also employed to cover areas of the shot which contain moving elements such as an actor or a spinning ceiling fan.
compositing. However, the purpose of a tracking matte is to prevent tracking algorithms from using unreliable, irrelevant or non-rigid tracking points. For example, in a scene where an actor walks in front of a background, the tracking artist will want to use only the background to track the camera through the scene, knowing that motion of the actor will throw off the calculations. In this case, the artist will construct a tracking matte to follow the actor through the scene, blocking that information from the tracking process.
Most match moving applications seem based on similar algorithms for tracking and calibration. Often, the initial results obtained are similar. However, it seems that each program has different refining capabilities. Therefore, when choosing software, look closely at the refining process.
To achieve this, a number of components from hardware to software need to be combined. Products such as Take4D bring these together. The software collects all of the 6 degrees of freedom movement of the camera as well as metadata such as zoom, focus, iris and shutter elements from many different types of hardware devices, ranging from motion capture systems such as active LED marker based system from PhaseSpace, passive systems such as Motion Analysis or Vicon, to rotary encodes fitted to camera cranes and dollies such as Technocranes and Fisher Dollies, or inertia & gyroscopic sensors mounted directly to the camera. Take4D has also developed a laser based tracking system that can be attached to anything, including Steadycams, to track cameras outside in the rain at distances of up to 30 meters.
Motion control cameras can also be used as a source or destination for 3D camera data. Camera moves can be pre-visualised in advance and then converted into motion control data that drives a camera crane along precisely the same path as the 3D camera. Encoders on the crane can also be used in real time on-set to reverse this process to generate live 3D cameras in real time. The data can be sent to any number of different 3D applications such as Take4D's own renderer, MotionBuilder, Maya, and more, allowing 3D artists to modify their CGI elements live on set as well. The main advantage being that set design issues that would be timely and costly issues later down the line can be sorted out during the shooting process, ensuring that the actors "fit" within each environment for each shot whilst they do their performances.
Real time motion capture systems can also be mixed within camera data stream allowing virtual characters to be inserted into live shots on-set. This dramatically improves the interaction between real and non-real MoCap driven characters as both plate and CG performances can be choreographed together.
Software
Cinematography
Cinematography is the making of lighting and camera choices when recording photographic images for cinema. It is closely related to the art of still photography...
, match moving is a visual-effects, cinematic techniques
Cinematic techniques
- Basic Definitions of Terms :Aerial Shot:A shot taken from a crane, plane, or helicopter. Not necessarily a moving shot.Backlighting:The main source of light is behind the subject, silhouetting it, and directed toward the camera....
that allows the insertion of computer graphics into live-action footage with correct position, scale, orientation, and motion relative to the photographed objects in the shot. The term is used loosely to refer to several different ways of extracting motion
Motion (physics)
In physics, motion is a change in position of an object with respect to time. Change in action is the result of an unbalanced force. Motion is typically described in terms of velocity, acceleration, displacement and time . An object's velocity cannot change unless it is acted upon by a force, as...
information from a motion picture, particularly camera movement. Match moving is related to rotoscoping and photogrammetry
Photogrammetry
Photogrammetry is the practice of determining the geometric properties of objects from photographic images. Photogrammetry is as old as modern photography and can be dated to the mid-nineteenth century....
. It is sometimes referred to as motion tracking.
Match moving is sometimes confused with motion capture
Motion capture
Motion capture, motion tracking, or mocap are terms used to describe the process of recording movement and translating that movement on to a digital model. It is used in military, entertainment, sports, and medical applications, and for validation of computer vision and robotics...
, which is a hardware technology for recording the motion of objects, often human actors, in a controlled environment using special cameras and sensors. It is also distinct from motion control photography
Motion control photography
Motion control photography is a technique used in still and motion photography that enables precise control of, and optionally also allows repetition of, camera movements. It can be used to facilitate special effects photography. The process can involve filming several elements using the same...
which uses a robot
Robot
A robot is a mechanical or virtual intelligent agent that can perform tasks automatically or with guidance, typically by remote control. In practice a robot is usually an electro-mechanical machine that is guided by computer and electronic programming. Robots can be autonomous, semi-autonomous or...
ic arm to execute multiple identical camera moves. Match moving, by contrast, is typically a software-based technology, applied after the fact to normal footage recorded in uncontrolled environments with an ordinary camera.
Match moving is primarily used to track the movement of a camera through a shot so that an identical virtual camera move can be reproduced in a 3D animation program. When new animated elements are composited back into the original live-action shot, they will appear in perfectly-matched perspective and therefore appear seamless.
As it is mostly software-based, match moving has become increasingly affordable as the cost of computer power has declined; it is now an established visual-effects tool and is even used in live television broadcasts
PVI Virtual Media Services
PVI Virtual Media Services is one of the companies behind the virtual yellow-down-line shown on television broadcasts of American football games in the USA and Canada...
as part of providing effects such as the virtual yellow-down-line
1st & Ten (graphics system)
1st & Ten is the name for a computer system that generates and displays one version of the yellow first down line that a TV viewer sees during a live broadcast of a college or professional American football or Canadian football game A competing system that performs the same task using different...
in American football.
Tracking
The first step is identifying and trackingVideo tracking
Video tracking is the process of locating a moving object over time using a camera. It has a variety of uses, some of which are: human-computer interaction, security and surveillance, video communication and compression, augmented reality, traffic control, medical imaging and video editing...
features. A feature is a specific point in the image that a tracking algorithm can lock onto and follow through multiple frames (SynthEyes calls them blips). Often features are selected because they are bright/dark spots, edges or corners depending on the particular tracking algorithm. Popular programs use template matching
Template matching
Template matching is a technique in digital image processing for finding small parts of an image which match a template image. It can be used in manufacturing as a part of quality control, a way to navigate a mobile robot, or as a way to detect edges in images....
based on NCC score and RMS error
Root mean square deviation
The root-mean-square deviation is the measure of the average distance between the atoms of superimposed proteins...
. What is important is that each feature represents a specific point on the surface of a real object. As a feature is tracked it becomes a series of two-dimensional coordinates that represent the position of the feature across a series of frames. This series is referred to as a track. Once tracks have been created they can be used immediately for 2D motion tracking, or then be used to calculate 3D information.
Calibration
The second step involves solving for 3D motion. This process attempts to derive the motion of the camera by solving the inverse-projection of the 2D paths for the position of the camera. This process is referred to as calibrationCamera resectioning
Camera resectioning is the process of finding the true parameters of the camera that produced a given photograph or video. Usually, the camera parameters are represented in a 3 × 4 matrix called the camera matrix....
.
To explain further: when a point on the surface of a three dimensional object is photographed its position in the 2D frame can be calculated by a 3D projection
3D projection
3D projection is any method of mapping three-dimensional points to a two-dimensional plane. As most current methods for displaying graphical data are based on planar two-dimensional media, the use of this type of projection is widespread, especially in computer graphics, engineering and drafting.-...
function. We can consider a camera to be an abstraction that holds all the parameters necessary to model a camera in a real or virtual world. Therefore a camera is a vector that includes as its elements the position of the camera, its orientation, focal length, and other possible paremeters that define how the camera focuses light onto the film plane
Film plane
A film plane is the area inside any image taking device with a lens and a digital sensor or film; such as a camera. The film plane varies in distance from the lens focal point in each manufacturer...
. Exactly how this vector is constructed is not important as long as there is a compatible projection function P.
The projection function P takes as its input a camera vector (denoted camera) and another vector the position of a 3D point in space (denoted xyz) and returns a 2D point that has been projected onto a plane in front of the camera (denoted XY). We can express this:
- XY = P(camera, xyz)
- xyz ∈ P'(camera, XY)
or
- {xyz :P(camera, xyz) = XY}
Let's say we are in a situation where the features we are tracking are on the surface of a rigid object such as a building. Since we know that the real point xyz will remain in the same place in real space from one frame of the image to the next we can make the point a constant even though we do not know where it is. So:
- xyzi = xyzj
where the subscripts i and j refer to arbitrary frames in the shot we are analyzing. Since this is always true then we know that:
- P'(camerai, XYi) ∩ P'(cameraj, XYj) ≠ {}
Because the value of XYi has been determined for all frames that the feature is tracked through by the tracking program, we can solve the reverse projection function between any two frames as long as P'(camerai, XYi) ∩ P'(cameraj, XYj) is a small set. Set of possible camera vectors that solve the equation at i and j (denoted Cij).
- Cij = {(camerai,cameraj):P'(camerai, XYi) ∩ P'(cameraj, XYj) ≠ {})
So there is a set of camera vector pairs Cij for which the intersection of the inverse projections of two points XYi and XYj is a non-empty, hopefully small, set centering around a theoretical stationary point xyz .
In other words, imagine a black point floating in a white void and a camera. For any position in space that we place the camera, there is a set of corresponding parameters (orientation, focal length, etc) that will photograph that black point exactly the same way. Since C has an infinite number of members, one point is never enough to determine the actual camera position.
As we start adding tracking points, we can narrow the possible camera positions. For example if we have a set of points {xyzi,0,...,xyzi,n} and {xyzj,0,...,xyzj,n} where i and j still refer to frames and n is an index to one of many tracking points we are following. We can derive a set of camera vector pair sets {Ci,j,0,...,Ci,j,n}.
In this way multiple tracks allow us to narrow the possible camera parameters. The set of possible camera parameters that fit, F, is the intersection of all sets:
- F = Ci,j,0 ∩ ... ∩ Ci,j,n
The fewer elements are in this set the closer we can come to extracting the actual parameters of the camera. In reality errors introduced to the tracking process require a more statistical approach to determining a good camera vector for each frame, optimization
Optimization (mathematics)
In mathematics, computational science, or management science, mathematical optimization refers to the selection of a best element from some set of available alternatives....
algorithms and bundle block adjustment are often utilized. Unfortunately there are so many elements to a camera vector that when every parameter is free we still might not be able to narrow F down to a single possibility no matter how many features we track. The more we can restrict the various parameters, especially focal length, the easier it becomes to pinpoint the solution.
In all, the 3D solving process is the process of narrowing down the possible solutions to the motion of the camera until we reach one that suits the needs of the composite we are trying to create.
Point-cloud projection
Once the camera position has been determined for every frame it is then possible to estimate the position of each feature in real space by inverse projection. The resulting set of points is often referred to as a point cloud because of its raw appearance like a nebulaNebula
A nebula is an interstellar cloud of dust, hydrogen gas, helium gas and other ionized gases...
. Since point clouds often reveal some of the shape of the 3D scene they can be used as a reference for placing synthetic objects or by a reconstruction program to create a 3D version of the actual scene.
Ground-plane determination
The camera and point cloud need to be oriented in some kind of space. Therefore, once calibration is complete, it is necessary to define a ground plane. Normally, this is a unit plane that determines the scale, orientation and origin of the projected space. Some programs attempt to do this automatically, though more often the user defines this plane. Since shifting ground planes does a simple transformation of all of the points, the actual position of the plane is really a matter of convenience.Reconstruction
Reconstruction is the interactive process of recreating a photographed object using tracking data. This technique is related to photogrammetryPhotogrammetry
Photogrammetry is the practice of determining the geometric properties of objects from photographic images. Photogrammetry is as old as modern photography and can be dated to the mid-nineteenth century....
. In this particular case we are referring to using match moving software to reconstruct a scene from incidental footage.
A reconstruction program can create three-dimensional objects that mimic the real objects from the photographed scene. Using data from the point cloud and the user's estimation, the program can create a virtual object and then extract a texture from the footage that can be projected onto the virtual object as a surface texture.
2D vs. 3D
Match moving has two forms. Compositing programs, such as ShakeShake (software)
Shake is a discontinued image compositing package used in the post-production industry. Shake was widely used in visual effects and digital compositing for film, HD and commercials. Shake exposes its node graph architecture graphically. It enables complex image processing sequences to be designed...
, Adobe After Effects
Adobe After Effects
Adobe After Effects is a digital motion graphics and compositing software published by Adobe Systems, used in the post-production process of filmmaking and television production. Its main uses are the origination of 2D and 2.5D animation, visual effects compositing and finishing...
and Discreet Combustion, have two-dimensional motion tracking capabilities. This feature translates images in two-dimensional space and can add effects such as motion blur in an attempt to eliminate relative motion between two features of two moving images. This technique is sufficient to create verisimilitude when the two images do not include major changes in camera perspective. For example a billboard deep in the background of a shot can often be replaced using two-dimensional tracking.
Three-dimensional match moving tools make it possible to extrapolate three-dimensional information from two-dimensional photography. Programs capable of 3D match moving include:
- BlenderBlender (software)Blender is a free and open-source 3D computer graphics software product used for creating animated films, visual effects, interactive 3D applications or video games. The current release version is 2.60, and was released on October 19, 2011...
(open source; uses libmv) - Voodoo (freeware)
- Scenespector VooCAT
- Icarus (University of Manchester research project, now discontinued but still popular)
- Maya MatchMover 2010
- The Pixel Farm PFTrack, PFMatchit, PFHoe (based on PFTrack algorithms)
- REALVIZ MatchMover (Autodesk bought and re-released as part of Maya 2010 bundle)
- Science.D.Visions 3DEqualizer (which won an Academy Award for Technical AchievementAcademy Award for Technical AchievementThe Technical Achievement Award is a kind of Scientific and Technical Award given by the Academy of Motion Picture Arts and Sciences to those whose particular technical accomplishments have contributed to the progress of the motion picture industry and who are given a certificate, which describes...
) - Andersson Technologies LLC's SynthEyes
- Boujou (which won an Emmy awardEmmy AwardAn Emmy Award, often referred to simply as the Emmy, is a television production award, similar in nature to the Peabody Awards but more focused on entertainment, and is considered the television equivalent to the Academy Awards and the Grammy Awards .A majority of Emmys are presented in various...
in 2002) - NukeXNuke (software)Nuke is a node-based compositor produced by The Foundry, and used for film and television post-production. Nuke is cross-platform, and is available for Microsoft Windows, Mac OS X, and Linux...
from The Foundry - CameraTracker (a plug-in for Adobe After EffectsAdobe After EffectsAdobe After Effects is a digital motion graphics and compositing software published by Adobe Systems, used in the post-production process of filmmaking and television production. Its main uses are the origination of 2D and 2.5D animation, visual effects compositing and finishing...
) from The Foundry. - VideoTrace from Punchcard (innovative software for generating 3D models from video and images)
- IXIR 2D Track Editor It’s capable of 2D tracks and Mask files of softwares like 3D Equalizer, PFTrack, Boujou, SynthEyes, Matchmover, Movimento, Nuke, Shake, Fusion, After Effects, Combustion, Mocha, Silhouette
These programs allow users to derive camera movement and other relative motion from arbitrary footage. The tracking information can be transferred to computer graphics software such as Blender
Blender (software)
Blender is a free and open-source 3D computer graphics software product used for creating animated films, visual effects, interactive 3D applications or video games. The current release version is 2.60, and was released on October 19, 2011...
, 3ds Max, Maya
Maya (software)
Autodesk Maya , commonly shortened to Maya, is 3D computer graphics software that runs on Microsoft Windows, Mac OS and Linux, originally developed by Alias Systems Corporation and currently owned and developed by Autodesk, Inc. It is used to create interactive 3D applications, including video...
or LightWave
LightWave
LightWave 3D is a high end computer graphics program developed by NewTek. The latest release of LightWave runs on Windows and Mac OS X.- Overview:...
and used to animate virtual cameras and CGI objects.
Automatic vs. interactive tracking
There are two methods by which motion information can be extracted from an image. Interactive tracking (sometimes referred to as "supervised tracking") relies on the user to follow features through a scene. Automatic tracking relies on computer algorithms to identify and track features through a shot. The tracked points movements are then used to calculate a solution. This solution is composed of all the camera's information such as the motion, focal length, and lens distortion.The advantage of automatic tracking is that the computer can create many points faster than a human can. A large number of points can be analyzed with statistics
Statistics
Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....
to determine the most reliable data. The disadvantage of automatic tracking is that, depending on the algorithm, the computer can be easily confused as it tracks objects through the scene. Automatic tracking methods are particularly ineffective in shots involving fast camera motion such as that seen with hand-held camera work and in shots with repetitive subject matter like small tiles or any sort of regular pattern where one area isn't very distinct from another. This tracking method also suffers when a shot contains a large amount of motion blur, making the small details it needs harder to distinguish.
The advantage of interactive tracking is that a human user can follow features through an entire scene and will not be confused by features that are not rigid. A human user can also determine where features are in a shot that suffers from motion blur; it is extremely difficult for an automatic tracker to correctly find features with high amounts of motion blur. The disadvantage of interactive tracking is that the user will inevitably introduce small errors as they follow objects through the scene, which can lead to drift.
Professional-level motion tracking is usually achieved using a combination of interactive and automatic techniques. An artist can remove points that are clearly anomalous and use tracking mattes to block confusing information out of the automatic tracking process. Tracking mattes are also employed to cover areas of the shot which contain moving elements such as an actor or a spinning ceiling fan.
Tracking mattes
A tracking matte is similar in concept to a garbage matte used in traveling matteMatte (filmmaking)
Mattes are used in photography and special effects filmmaking to combine two or more image elements into a single, final image. Usually, mattes are used to combine a foreground image with a background image . In this case, the matte is the background painting...
compositing. However, the purpose of a tracking matte is to prevent tracking algorithms from using unreliable, irrelevant or non-rigid tracking points. For example, in a scene where an actor walks in front of a background, the tracking artist will want to use only the background to track the camera through the scene, knowing that motion of the actor will throw off the calculations. In this case, the artist will construct a tracking matte to follow the actor through the scene, blocking that information from the tracking process.
Refining
Since there are often multiple possible solutions to the calibration process and a significant amount of error can accumulate, the final step to match moving often involves refining the solution by hand. This could mean altering the camera motion itself or giving hints to the calibration mechanism. This interactive calibration is referred to as refining.Most match moving applications seem based on similar algorithms for tracking and calibration. Often, the initial results obtained are similar. However, it seems that each program has different refining capabilities. Therefore, when choosing software, look closely at the refining process.
Real time
On-Set real time camera tracking is becoming more widely used in feature film production to allow elements that will be inserted in post-production be visualised live on-set. This has the benefit of helping the director and actors improve performances by actually seeing set extensions or CGI characters whilst (or shortly after) they do a take. No longer do they need to perform to green / blue screens and have no feedback of the end result. Eye-line references, actor positioning, and CGI interaction can now be done live on-set giving everyone confidence that the shot is correct and going to work in the final composite.To achieve this, a number of components from hardware to software need to be combined. Products such as Take4D bring these together. The software collects all of the 6 degrees of freedom movement of the camera as well as metadata such as zoom, focus, iris and shutter elements from many different types of hardware devices, ranging from motion capture systems such as active LED marker based system from PhaseSpace, passive systems such as Motion Analysis or Vicon, to rotary encodes fitted to camera cranes and dollies such as Technocranes and Fisher Dollies, or inertia & gyroscopic sensors mounted directly to the camera. Take4D has also developed a laser based tracking system that can be attached to anything, including Steadycams, to track cameras outside in the rain at distances of up to 30 meters.
Motion control cameras can also be used as a source or destination for 3D camera data. Camera moves can be pre-visualised in advance and then converted into motion control data that drives a camera crane along precisely the same path as the 3D camera. Encoders on the crane can also be used in real time on-set to reverse this process to generate live 3D cameras in real time. The data can be sent to any number of different 3D applications such as Take4D's own renderer, MotionBuilder, Maya, and more, allowing 3D artists to modify their CGI elements live on set as well. The main advantage being that set design issues that would be timely and costly issues later down the line can be sorted out during the shooting process, ensuring that the actors "fit" within each environment for each shot whilst they do their performances.
Real time motion capture systems can also be mixed within camera data stream allowing virtual characters to be inserted into live shots on-set. This dramatically improves the interaction between real and non-real MoCap driven characters as both plate and CG performances can be choreographed together.
See also
- Structure from motionStructure from motionIn computer vision structure from motion refers to the process of finding the three-dimensional structure of an object by analyzing local motion signals over time....
- Virtual studioVirtual studioThe term virtual studio can refer to any number of technological tools which seek to simulate a physical television and/or movie studio. One such use of the term follows....
- 1st & Ten (graphics system)1st & Ten (graphics system)1st & Ten is the name for a computer system that generates and displays one version of the yellow first down line that a TV viewer sees during a live broadcast of a college or professional American football or Canadian football game A competing system that performs the same task using different...
- PVI Virtual Media ServicesPVI Virtual Media ServicesPVI Virtual Media Services is one of the companies behind the virtual yellow-down-line shown on television broadcasts of American football games in the USA and Canada...
External links
Videos on Youtube: Industry & Match Moving Training, A hole, A Car Race, A simple animation, Moving trainSoftware
- Take4D Real time on-set Virtual Studio Camera Tracking solutions, including Motion Control Cameras, Motion Capture Systems, Calibrated Zoom and Focus Metadata, and more.
- 3D-Equalizer
- Blender As of GSoC 2011 this 3d animation program has integrated 2d/3d tracking tools
- Voodoo a free and fully functional match moving program for LinuxLinuxLinux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
and Microsoft WindowsMicrosoft WindowsMicrosoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
. - VooCAT fast commercial match moving program based on Voodoo for Mac OS XMac OS XMac OS X is a series of Unix-based operating systems and graphical user interfaces developed, marketed, and sold by Apple Inc. Since 2002, has been included with all new Macintosh computer systems...
, LinuxLinuxLinux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
, and Microsoft WindowsMicrosoft WindowsMicrosoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
. - Icarus (Interactive Calibration And Reconstruction from Image Sequences).
- SynthEyes Camera tracker Provides widely used tools for camera tracking, motion capture, object building, object tracking, camera+object tracking, multiple-shot tracking, tripod (2.5-D) tracking, zooms, lens distortion, light solving, RAM playback, rotoscoped object separation and more.
- PFTrack Image analysis and match moving tools for the film effects industry
- PFMatchit Node-based camera tracking and match moving software
- PFHoe PFHoe is an extremely cost effective DV tracking application for Mac and Windows, aimed at DV hobbyists and budding CG artists.
- boujou Formerly a product of 2d3, boujou is now a product of sister company Vicon.
- REALVIZ MatchMover
- Mokey, Monet, Motor, Mocha
- PhaseSpace
- VideoTrace Innovative image-based modelling software for generating 3D models from video. Includes its own tracking and calibration engine, and supports data from other similar software such as Voodoo, boujou and SynthEyes.
- IXIR 2D Track Editor It’s capable of 2D tracks and Mask files of softwares like 3D Equalizer, PFTrack, Boujou, SynthEyes, Matchmover, Movimento, Nuke, Shake, Fusion, After Effects, Combustion, Mocha, Silhouette