In robotics

Robotics

Robotics is the branch of technology that deals with the design, construction, operation, structural disposition, manufacture and application of robots...

 and computer vision

Computer vision

Computer vision is a field that includes methods for acquiring, processing, analysing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions...

, visual odometry is the process of determining the position and orientation of a robot by analyzing the associated camera images. It has been used in a wide variety of robotic applications, such as on the Mars Exploration Rover

Mars Exploration Rover

NASA's Mars Exploration Rover Mission is an ongoing robotic space mission involving two rovers, Spirit and Opportunity, exploring the planet Mars...

s.

Overview

In navigation

Navigation

Navigation is the process of monitoring and controlling the movement of a craft or vehicle from one place to another. It is also the term of art used for the specialized knowledge used by navigators to perform navigation tasks...

, odometry

Odometry

Odometry is the use of data from moving sensors to estimate change in position over time. Odometry is used by some robots, whether they be legged or wheeled, to estimate their position relative to a starting location. This method is sensitive to errors due to the integration of velocity...

 is the use of data from the movement of actuators to estimate change in position over time through devices such as rotary encoder

Rotary encoder

A rotary encoder, also called a shaft encoder, is an electro-mechanical device that converts the angular position or motion of a shaft or axle to an analog or digital code. The output of incremental encoders provides information about the motion of the shaft which is typically further processed...

s to measure wheel rotations. While useful for many wheeled or tracked vehicles, traditional odometry techniques cannot be applied to robots with non-standard locomotion methods, such as legged robots. In addition, odometry universally suffers from precision problems, since wheels tend to slip and slide on the floor creating a non-uniform distance traveled as compared to the wheel rotations. The error is compounded when the vehicle operates on non-smooth surfaces. Odometry reading become increasingly unreliable over time as these errors accumulate and compound over time.

Visual odometry is the process of determining equivalent odometry information using sequential camera images to estimate the distance traveled. Visual odometry allows for enhanced navigational accuracy in robots or vehicles using any type of locomotion on any surface.

Algorithm

Most existing approaches to visual odometry are based on the following stages.

1. Acquire input images: using either single cameras., stereo camera
   Stereo camera
   A stereo camera is a type of camera with two or more lenses with a separate image sensor or film frame for each lens. This allows the camera to simulate human binocular vision, and therefore gives it the ability to capture three-dimensional images, a process known as stereo photography. Stereo...
   
   s, or omnidirectional camera
   Omnidirectional camera
   In photography, an omnidirectional camera is a camera with a 360-degree field of view in the horizontal plane, or with a visual field that covers the entire sphere...
   
   s.
2. Image correction: apply image processing
   Image processing
   In electrical engineering and computer science, image processing is any form of signal processing for which the input is an image, such as a photograph or video frame; the output of image processing may be either an image or, a set of characteristics or parameters related to the image...
   
    techniques for lens distortion removal, etc
3. Feature detection: define interest operators, and match features across frames and construct optical flow
   Optical flow
   Optical flow or optic flow is the pattern of apparent motion of objects, surfaces, and edges in a visual scene caused by the relative motion between an observer and the scene. The concept of optical flow was first studied in the 1940s and ultimately published by American psychologist James J....
   
    field.
   1. Use correlation to establish correspondence of two images, and no long term feature tracking.
   2. Feature extraction and correlation (Lucas–Kanade method).
   3. Construct optical flow field.
4. Check flow field vectors for potential tracking errors and remove outliers.
5. Estimation of the camera motion from the optical flow.
   1. Choice 1: Kalman filter
      Kalman filter
      In statistics, the Kalman filter is a mathematical method named after Rudolf E. Kálmán. Its purpose is to use measurements observed over time, containing noise and other inaccuracies, and produce values that tend to be closer to the true values of the measurements and their associated calculated...
      
       for state estimate distribution maintenance.
   2. Choice 2: find the geometric and 3D properties of the features that minimize a cost function based on the re-projection error between two adjacent images. This can be done by mathematical minimization or random sampling.
6. Periodic repopulation of trackpoints to maintain coverage across the image.

An alternative to feature-based methods is the "direct" or appearance-based visual odometry technique which minimizes an error directly in sensor space and subsequently avoids feature matching and extraction.

Another method, coined 'visiodometry' estimates the planar roto-translations between images using Phase correlation

Phase correlation

In image processing, phase correlation is a method of image registration, and uses a fast frequency-domain approach to estimate the relative translative offset between two similar images.- Example :...

instead of extracting features.