Hessian Affine region detector
Encyclopedia
The Hessian affine region detector is a feature detector
used in the fields of computer vision
and image analysis
. Like other feature detectors, the Hessian affine detector is typically used as a preprocessing step to algorithms that rely on identifiable, characteristic interest points
.
The Hessian affine detector is part of the subclass of feature detectors known as affine-invariant detectors: Harris affine region detector
, Hessian affine regions, maximally stable extremal regions
, Kadir–Brady saliency detector, edge-based regions (EBR) and intensity-extrema-based (IBR) regions.
. In fact, both algorithms were derived by Krystian Mikolajczyk and Cordelia Schmid in 2002,
,
based on earlier work in,
see also for a more general overview. See the Harris affine region detector
entry for a more complete description of the detection algorithm.
on the second-moment matrix. The Hessian affine also uses a multiple scale iterative algorithm to spatially localize and select scale & affine invariant points. However, at each individual scale, the Hessian affine detector chooses interest points based on the Hessian matrix
at that point:
where
is second partial derivative in the 
direction and 
is the mixed partial second derivative in the 
and 
directions. It's important to note that the derivatives are computed in the current iteration scale and thus are derivatives of an image smoothed by a Gaussian kernel: 
. As discussed in the Harris affine region detector
article, the derivatives must be scaled appropriately by a factor related to the Gaussian kernel:
.
At each scale, interest points are those points that simultaneously are local extrema of both the determinant and trace of the Hessian matrix. The trace of Hessian matrix is identical to the Laplacian of Gaussians (LoG):
As discussed in Mikolajczyk et al.(2005), by choosing points that maximize the determinant of the Hessian, this measure penalizes longer structures that have small second derivatives (signal changes) in a single direction. This type of measure is very similar to the measures used in the blob detection
schemes proposed by Lindeberg (1998), where either the Laplacian or the determinant of the Hessian were used in blob detection methods with automatic scale selection.
Like the Harris affine algorithm, these interest points based on the Hessian matrix are also spatially localized using an iterative search based on the Laplacian of Gaussians. Predictably, these interest points are called Hessian–Laplace interest points. Furthermore, using these initially detected points, the Hessian affine detector uses an iterative shape adaptation algorithm to compute the local affine transformation for each interest point. The implementation of this algorithm is almost identical to that of the Harris affine detector; however, the above mentioned Hessian measure replaces all instances of the Harris corner measure.
detectors. Mikolajczyk et al. analyzed both structured images and textured images in their evaluation. Linux binaries of the detectors and their test images are freely available at their webpage. A brief summary of the results of Mikolajczyk et al. (2005) follow; see A comparison of affine region detectors for a more quantitative analysis.
Overall, the Hessian affine detector performs second best to MSER. Like the Harris affine detector, Hessian affine interest regions tend to be more numerous and smaller than other detectors. For a single image, the Hessian affine detector typically identifies more reliable regions than the Harris-Affine detector. The performance changes depending on the type of scene being analyzed. The Hessian affine detector responds well to textured scenes in which there are a lot of corner-like parts. However, for some structured scenes, like buildings, the Hessian affine detector performs very well. This is complementary to MSER that tends to do better with well structured (segmentable) scenes.
Feature detection
In computer vision and image processing the concept of feature detection refers to methods that aim at computing abstractions of image information and making local decisions at every image point whether there is an image feature of a given type at that point or not...
used in the fields of computer vision
Computer vision
Computer vision is a field that includes methods for acquiring, processing, analysing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions...
and image analysis
Image analysis
Image analysis is the extraction of meaningful information from images; mainly from digital images by means of digital image processing techniques...
. Like other feature detectors, the Hessian affine detector is typically used as a preprocessing step to algorithms that rely on identifiable, characteristic interest points
Interest point detection
Interest point detection is a recent terminology in computer vision that refers to the detection of interest points for subsequent processing...
.
The Hessian affine detector is part of the subclass of feature detectors known as affine-invariant detectors: Harris affine region detector
Harris affine region detector
In the fields of computer vision and image analysis, the Harris affine region detector belongs to the category of feature detection. Feature detection is a preprocessing step of several algorithms that rely on identifying characteristic points or interest points so to make correspondences between...
, Hessian affine regions, maximally stable extremal regions
Maximally stable extremal regions
In computer vision, maximally stable extremal regions are used as a method of blob detection in images. This technique was proposed by Matas et al. to find correspondences between image elements from two images with different viewpoints...
, Kadir–Brady saliency detector, edge-based regions (EBR) and intensity-extrema-based (IBR) regions.
Algorithm description
The Hessian affine detector algorithm is almost identical to the Harris affine region detectorHarris affine region detector
In the fields of computer vision and image analysis, the Harris affine region detector belongs to the category of feature detection. Feature detection is a preprocessing step of several algorithms that rely on identifying characteristic points or interest points so to make correspondences between...
. In fact, both algorithms were derived by Krystian Mikolajczyk and Cordelia Schmid in 2002,
,
based on earlier work in,
see also for a more general overview. See the Harris affine region detector
Harris affine region detector
In the fields of computer vision and image analysis, the Harris affine region detector belongs to the category of feature detection. Feature detection is a preprocessing step of several algorithms that rely on identifying characteristic points or interest points so to make correspondences between...
entry for a more complete description of the detection algorithm.
How does the Hessian affine differ?
The Harris affine detector relies on interest points detected at multiple scales using the Harris corner measureCorner detection
Corner detection is an approach used within computer vision systems to extract certain kinds of features and infer the contents of an image. Corner detection is frequently used in motion detection, image registration, video tracking, image mosaicing, panorama stitching, 3D modelling and object...
on the second-moment matrix. The Hessian affine also uses a multiple scale iterative algorithm to spatially localize and select scale & affine invariant points. However, at each individual scale, the Hessian affine detector chooses interest points based on the Hessian matrix
Hessian matrix
In mathematics, the Hessian matrix is the square matrix of second-order partial derivatives of a function; that is, it describes the local curvature of a function of many variables. The Hessian matrix was developed in the 19th century by the German mathematician Ludwig Otto Hesse and later named...
at that point:
where
is second partial derivative in the direction and is the mixed partial second derivative in the and directions. It's important to note that the derivatives are computed in the current iteration scale and thus are derivatives of an image smoothed by a Gaussian kernel: . As discussed in the Harris affine region detectorHarris affine region detector
In the fields of computer vision and image analysis, the Harris affine region detector belongs to the category of feature detection. Feature detection is a preprocessing step of several algorithms that rely on identifying characteristic points or interest points so to make correspondences between...
article, the derivatives must be scaled appropriately by a factor related to the Gaussian kernel:
.At each scale, interest points are those points that simultaneously are local extrema of both the determinant and trace of the Hessian matrix. The trace of Hessian matrix is identical to the Laplacian of Gaussians (LoG):
As discussed in Mikolajczyk et al.(2005), by choosing points that maximize the determinant of the Hessian, this measure penalizes longer structures that have small second derivatives (signal changes) in a single direction. This type of measure is very similar to the measures used in the blob detection
Blob detection
In the area of computer vision, blob detection refers to visual modules that are aimed at detecting points and/or regions in the image that differ in properties like brightness or color compared to the surrounding...
schemes proposed by Lindeberg (1998), where either the Laplacian or the determinant of the Hessian were used in blob detection methods with automatic scale selection.
Like the Harris affine algorithm, these interest points based on the Hessian matrix are also spatially localized using an iterative search based on the Laplacian of Gaussians. Predictably, these interest points are called Hessian–Laplace interest points. Furthermore, using these initially detected points, the Hessian affine detector uses an iterative shape adaptation algorithm to compute the local affine transformation for each interest point. The implementation of this algorithm is almost identical to that of the Harris affine detector; however, the above mentioned Hessian measure replaces all instances of the Harris corner measure.
Robustness to affine and other transformations
Mikolajczyk et al. (2005) have done a thorough analysis of several state of the art affine region detectors: Harris affine, Hessian affine, MSER, IBR & EBR and salientKadir brady saliency detector
The Kadir–Brady saliency detector extracts features of objects in images that are distinct and representative. It was invented by Timor Kadir and Michael Brady in 2001 and an affine invariant version was introduced by Kadir and Brady in 2004....
detectors. Mikolajczyk et al. analyzed both structured images and textured images in their evaluation. Linux binaries of the detectors and their test images are freely available at their webpage. A brief summary of the results of Mikolajczyk et al. (2005) follow; see A comparison of affine region detectors for a more quantitative analysis.
Overall, the Hessian affine detector performs second best to MSER. Like the Harris affine detector, Hessian affine interest regions tend to be more numerous and smaller than other detectors. For a single image, the Hessian affine detector typically identifies more reliable regions than the Harris-Affine detector. The performance changes depending on the type of scene being analyzed. The Hessian affine detector responds well to textured scenes in which there are a lot of corner-like parts. However, for some structured scenes, like buildings, the Hessian affine detector performs very well. This is complementary to MSER that tends to do better with well structured (segmentable) scenes.
Software packages
- Affine Covariant Features: K. Mikolajczyk maintains a web page that contains Linux binaries of the Hessian-Affine detector in addition to other detectors and descriptors. Matlab code is also available that can be used to illustrate and compute the repeatability of various detectors. Code and images are also available to duplicate the results found in the Mikolajczyk et al. (2005) paper.
- lip-vireo: - binary code for Linux, Windows and SunOS from VIREO research group
External links
- http://vasc.ri.cmu.edu/~hebert/04workshop/presentations/schmid_sicily04.ppt - Presentation slides from Mikolajczyk et al. on their 2005 paper.
- http://lear.inrialpes.fr/software - Cordelia Schmid's Computer Vision Lab
- http://www.robots.ox.ac.uk/~vgg/research/affine/ - Code, test Images, bibliography of Affine Covariant Features maintained by Krystian Mikolajczyk and the Visual Geometry Group from the Robotics group at the University of Oxford.
- http://iris.usc.edu/Vision-Notes/bibliography/twod275.html - Bibliography of feature (and blob) detectors maintained by USC Institute for Robotics and Intelligent Systems