Affine shape adaptation
Encyclopedia
Affine shape adaptation is a methodology for iteratively adapting the shape of the smoothing kernels in an affine group
of smoothing kernels to the local image structure in neighbourhood region of a specific image point. Equivalently, affine shape adaptation can be accomplished by iteratively warping a local image patch with affine transformations while applying a rotationally symmetric filter to the warped image patches. Provided that this iterative process converges, the resulting fixed point will be affine invariant. In the area of computer vision
, this idea has been used for defining affine invariant interest point operators as well as affine invariant texture analysis methods.
or the multi-scale Harris corner detector
with automatic scale selection are invariant to translations, rotations and uniform rescalings in the spatial domain. The images that constitute the input to a computer vision system are, however, also subject to perspective distortions. To obtain interest points that are more robust to perspective transformations, a natural approach is to devise a feature detector that is invariant to affine transformations.
Interestingly, affine invariance can be accomplished from measurements of the same multi-scale windowed second moment matrix
as is used in the multi-scale Harris operator provided that we extend the regular scale-space concept obtained by convolution with rotationally symmetric Gaussian kernels to an affine Gaussian scale-space obtained by shape-adapted Gaussian kernels (Lindeberg 1994 section 15.3; Lindeberg and Garding 1997). For a two-dimensional image 
, let 
and let 
be a positive definite 2×2 matrix. Then, a non-uniform Gaussian kernel can be defined as
and given any input image
the affine Gaussian scale-space is the three-parameter scale-space defined as
Next, introduce an affine transformation
where 
is a 2×2-matrix, and define a transformed image 
as
.
Then, the affine scale-space representations
and 
of 
and 
, respectively, are related according to
provided that the affine shape matrices
and 
are related according to
.
Disregarding mathematical details, which unfortunately become somewhat technical if one aims at a precise description of what is going on, the important message is that the affine Gaussian scale-space is closed under affine transformations.
If we, given the notation
as well as local shape matrix 
and an integration shape matrix 
, introduce an affine-adapted multi-scale second-moment matrix according to
it can be shown that under any affine transformation
the affine-adapted multi-scale second-moment matrix transforms according to
.
Again, disregarding somewhat messy technical details, the important message here is that given a correspondence between the image points
and 
, the affine transformation 
can be estimated from measurements of the multi-scale second-moment matrices 
and 
in the two domains.
An important consequence of this study is that if we can find an affine transformation
such that 
is a constant times the unit matrix, then we obtain a fixed-point that is invariant to affine transformations (Lindeberg 1994 section 15.4; Lindeberg and Garding 1997). For the purpose of practical implementation, this property can often be reached by in either of two main ways. The first approach is based on transformations of the smoothing filters and consists of:
The second approach is based on warpings in the image domain and implies:
This overall process is referred to as affine shape adaptation (Lindeberg and Garding 1997; Baumberg 2000; Mikolajczyk and Schmid 2004; Tuytelaars and van Gool 2004; Lindeberg 2008). In the ideal continuous case, the two approaches are mathematically equivalent. In practical implementations, however, the first filter-based approach is usually more accurate in the presence of noise while the second warping-based approach is usually faster.
In practice, the affine shape adaptation process described here is often combined with interest point detection automatic scale selection as described in the articles on blob detection
and corner detection
, to obtain interest points that are invariant to the full affine group, including scale changes. Besides the commonly used multi-scale Harris operator, this affine shape adaptation can also be applied to other types of interest point operators such as the Laplacian/Difference of Gaussian blob operator and the determinant of the Hessian (Lindeberg 2008). Affine shape adaptation can also be used for affine invariant texture recognition and affine invariant texture segmentation.
Affine group
In mathematics, the affine group or general affine group of any affine space over a field K is the group of all invertible affine transformations from the space into itself.It is a Lie group if K is the real or complex field or quaternions....
of smoothing kernels to the local image structure in neighbourhood region of a specific image point. Equivalently, affine shape adaptation can be accomplished by iteratively warping a local image patch with affine transformations while applying a rotationally symmetric filter to the warped image patches. Provided that this iterative process converges, the resulting fixed point will be affine invariant. In the area of computer vision
Computer vision
Computer vision is a field that includes methods for acquiring, processing, analysing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions...
, this idea has been used for defining affine invariant interest point operators as well as affine invariant texture analysis methods.
Affine-adapted interest point operators
The interest points obtained from the scale-adapted Laplacian blob detectorBlob detection
In the area of computer vision, blob detection refers to visual modules that are aimed at detecting points and/or regions in the image that differ in properties like brightness or color compared to the surrounding...
or the multi-scale Harris corner detector
Corner detection
Corner detection is an approach used within computer vision systems to extract certain kinds of features and infer the contents of an image. Corner detection is frequently used in motion detection, image registration, video tracking, image mosaicing, panorama stitching, 3D modelling and object...
with automatic scale selection are invariant to translations, rotations and uniform rescalings in the spatial domain. The images that constitute the input to a computer vision system are, however, also subject to perspective distortions. To obtain interest points that are more robust to perspective transformations, a natural approach is to devise a feature detector that is invariant to affine transformations.
Interestingly, affine invariance can be accomplished from measurements of the same multi-scale windowed second moment matrix
as is used in the multi-scale Harris operator provided that we extend the regular scale-space concept obtained by convolution with rotationally symmetric Gaussian kernels to an affine Gaussian scale-space obtained by shape-adapted Gaussian kernels (Lindeberg 1994 section 15.3; Lindeberg and Garding 1997). For a two-dimensional image , let and let be a positive definite 2×2 matrix. Then, a non-uniform Gaussian kernel can be defined asand given any input image
the affine Gaussian scale-space is the three-parameter scale-space defined asNext, introduce an affine transformation
where is a 2×2-matrix, and define a transformed image as.Then, the affine scale-space representations
and of and , respectively, are related according toprovided that the affine shape matrices
and are related according to.Disregarding mathematical details, which unfortunately become somewhat technical if one aims at a precise description of what is going on, the important message is that the affine Gaussian scale-space is closed under affine transformations.
If we, given the notation
as well as local shape matrix and an integration shape matrix , introduce an affine-adapted multi-scale second-moment matrix according toit can be shown that under any affine transformation
the affine-adapted multi-scale second-moment matrix transforms according to.Again, disregarding somewhat messy technical details, the important message here is that given a correspondence between the image points
and , the affine transformation can be estimated from measurements of the multi-scale second-moment matrices and in the two domains.An important consequence of this study is that if we can find an affine transformation
such that is a constant times the unit matrix, then we obtain a fixed-point that is invariant to affine transformations (Lindeberg 1994 section 15.4; Lindeberg and Garding 1997). For the purpose of practical implementation, this property can often be reached by in either of two main ways. The first approach is based on transformations of the smoothing filters and consists of:- estimating the second-moment matrix
in the image domain, - determining a new adapted smoothing kernel with covariance matrix proportional to
, - smoothing the original image by the shape-adapted smoothing kernel, and
- repeating this operation until the difference between two successive second-moment matrices is sufficiently small.
The second approach is based on warpings in the image domain and implies:
- estimating
in the image domain, - estimating a local affine transformation proportional to
where 
denotes the square root matrix of 
, - warping the input image by the affine transformation
and - repeating this operation until
is sufficiently close to a constant times the unit matrix.
This overall process is referred to as affine shape adaptation (Lindeberg and Garding 1997; Baumberg 2000; Mikolajczyk and Schmid 2004; Tuytelaars and van Gool 2004; Lindeberg 2008). In the ideal continuous case, the two approaches are mathematically equivalent. In practical implementations, however, the first filter-based approach is usually more accurate in the presence of noise while the second warping-based approach is usually faster.
In practice, the affine shape adaptation process described here is often combined with interest point detection automatic scale selection as described in the articles on blob detection
Blob detection
In the area of computer vision, blob detection refers to visual modules that are aimed at detecting points and/or regions in the image that differ in properties like brightness or color compared to the surrounding...
and corner detection
Corner detection
Corner detection is an approach used within computer vision systems to extract certain kinds of features and infer the contents of an image. Corner detection is frequently used in motion detection, image registration, video tracking, image mosaicing, panorama stitching, 3D modelling and object...
, to obtain interest points that are invariant to the full affine group, including scale changes. Besides the commonly used multi-scale Harris operator, this affine shape adaptation can also be applied to other types of interest point operators such as the Laplacian/Difference of Gaussian blob operator and the determinant of the Hessian (Lindeberg 2008). Affine shape adaptation can also be used for affine invariant texture recognition and affine invariant texture segmentation.
See also
- corner detectionCorner detectionCorner detection is an approach used within computer vision systems to extract certain kinds of features and infer the contents of an image. Corner detection is frequently used in motion detection, image registration, video tracking, image mosaicing, panorama stitching, 3D modelling and object...
- blob detectionBlob detectionIn the area of computer vision, blob detection refers to visual modules that are aimed at detecting points and/or regions in the image that differ in properties like brightness or color compared to the surrounding...
- Harris affine region detector
- Hessian affine region detectorHessian Affine region detectorThe Hessian affine region detector is a feature detector used in the fields of computer vision and image analysis. Like other feature detectors, the Hessian affine detector is typically used as a preprocessing step to algorithms that rely on identifiable, characteristic interest points.The Hessian...
- scale-space
- Gaussian function