Reproducing kernel Hilbert space
Encyclopedia
In functional analysis
(a branch of mathematics
), a reproducing kernel Hilbert space is a Hilbert space
of functions
in which pointwise evaluation is a continuous linear functional
. Equivalently, they are spaces that can be defined by reproducing kernels. The subject was originally and simultaneously developed by Nachman Aronszajn (1907–1980) and Stefan Bergman
(1895–1977) in 1950.
In this article we assume that Hilbert spaces are complex
. The main reason for this is that many of the examples of reproducing kernel Hilbert spaces are spaces of analytic functions, although some real
Hilbert spaces also have reproducing kernels.
An important subset of the reproducing kernel Hilbert spaces are the reproducing kernel Hilbert spaces associated to a continuous kernel. These spaces have wide applications, including complex analysis
, quantum mechanics
, statistics
and harmonic analysis
.
of complex-valued functions on X. We say that H is a reproducing kernel Hilbert space if every linear map of the form
from H to the complex numbers is continuous for any x in X. By the Riesz representation theorem
, this implies that for every x in X there exists a unique element Kx of H with the property that:
The function Kx is called the point-evaluation functional at the point x.
Since H is a space of functions, the element Kx is itself a function and can therefore be evaluated at every point. We define the function
by
This function is called the reproducing kernel for the Hilbert space H and it is determined entirely by H because the Riesz representation theorem guarantees, for every x in X, that the element Kx satisfying (*) is unique.
.
A more sophisticated example is the Hardy space
H2(D)
, the space of square-integrable holomorphic functions on the unit disc. So here X=D, the unit disc. It can be shown that the reproducing kernel for H2(D) is
This kernel is an example of a Bergman kernel
, named for Stefan Bergman
.
In particular,
Note that
is an orthonormal sequence such that the closure of its span is equal to 
, then
The theorem first appeared in Aronszajn's Theory of Reproducing Kernels, although he attributes it to E. H. Moore
.
Theorem.
Suppose K is a symmetric, positive definite kernel on a set E. Then there is a unique Hilbert space of functions on E for which K is a reproducing kernel.
Proof.
Define, for all x in E,
.
Let H0 be the linear span of
.
Define an inner product on H0 by
The symmetry of this inner product follows from the symmetry of K and the non-degeneracy follows from the fact that K is positive definite.
Let H be the completion of H0 with respect to this inner product. Then H consists of functions of the form
where
. The fact that the above sum converges for every x follows from the Cauchy-Schwarz inequality.
Now we can check the RKHS property, (*):
To prove uniqueness, let G be another Hilbert space of functions for which K is a reproducing kernel. For any x and y in E,
(*) implies that
By linearity,
on the span of 
. Then G = H by the uniqueness of the completion.
s, for the Lebesgue measure
on D, that are holomorphic function
s. The theory is non-trivial in such cases as there are such functions, which are not identically zero. Then H is a reproducing kernel space, with kernel function the Bergman kernel; this example, with n = 1, was introduced by Bergman in 1922.
Functional analysis
Functional analysis is a branch of mathematical analysis, the core of which is formed by the study of vector spaces endowed with some kind of limit-related structure and the linear operators acting upon these spaces and respecting these structures in a suitable sense...
(a branch of mathematics
Mathematics
Mathematics is the study of quantity, space, structure, and change. Mathematicians seek out patterns and formulate new conjectures. Mathematicians resolve the truth or falsity of conjectures by mathematical proofs, which are arguments sufficient to convince other mathematicians of their validity...
), a reproducing kernel Hilbert space is a Hilbert space
Hilbert space
The mathematical concept of a Hilbert space, named after David Hilbert, generalizes the notion of Euclidean space. It extends the methods of vector algebra and calculus from the two-dimensional Euclidean plane and three-dimensional space to spaces with any finite or infinite number of dimensions...
of functions
Function space
In mathematics, a function space is a set of functions of a given kind from a set X to a set Y. It is called a space because in many applications it is a topological space, a vector space, or both.-Examples:...
in which pointwise evaluation is a continuous linear functional
Bounded operator
In functional analysis, a branch of mathematics, a bounded linear operator is a linear transformation L between normed vector spaces X and Y for which the ratio of the norm of L to that of v is bounded by the same number, over all non-zero vectors v in X...
. Equivalently, they are spaces that can be defined by reproducing kernels. The subject was originally and simultaneously developed by Nachman Aronszajn (1907–1980) and Stefan Bergman
Stefan Bergman
Stefan Bergman was a Polish-born American mathematician whose primary work was in complex analysis. He is best known for the kernel function he discovered while at Berlin University in 1922. This function is known today as the Bergman kernel...
(1895–1977) in 1950.
In this article we assume that Hilbert spaces are complex
Complex number
A complex number is a number consisting of a real part and an imaginary part. Complex numbers extend the idea of the one-dimensional number line to the two-dimensional complex plane by using the number line for the real part and adding a vertical axis to plot the imaginary part...
. The main reason for this is that many of the examples of reproducing kernel Hilbert spaces are spaces of analytic functions, although some real
Real number
In mathematics, a real number is a value that represents a quantity along a continuum, such as -5 , 4/3 , 8.6 , √2 and π...
Hilbert spaces also have reproducing kernels.
An important subset of the reproducing kernel Hilbert spaces are the reproducing kernel Hilbert spaces associated to a continuous kernel. These spaces have wide applications, including complex analysis
Complex analysis
Complex analysis, traditionally known as the theory of functions of a complex variable, is the branch of mathematical analysis that investigates functions of complex numbers. It is useful in many branches of mathematics, including number theory and applied mathematics; as well as in physics,...
, quantum mechanics
Quantum mechanics
Quantum mechanics, also known as quantum physics or quantum theory, is a branch of physics providing a mathematical description of much of the dual particle-like and wave-like behavior and interactions of energy and matter. It departs from classical mechanics primarily at the atomic and subatomic...
, statistics
Statistics
Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....
and harmonic analysis
Harmonic analysis
Harmonic analysis is the branch of mathematics that studies the representation of functions or signals as the superposition of basic waves. It investigates and generalizes the notions of Fourier series and Fourier transforms...
.
Definition
Let X be an arbitrary set and H a Hilbert spaceHilbert space
The mathematical concept of a Hilbert space, named after David Hilbert, generalizes the notion of Euclidean space. It extends the methods of vector algebra and calculus from the two-dimensional Euclidean plane and three-dimensional space to spaces with any finite or infinite number of dimensions...
of complex-valued functions on X. We say that H is a reproducing kernel Hilbert space if every linear map of the form
from H to the complex numbers is continuous for any x in X. By the Riesz representation theorem
Riesz representation theorem
There are several well-known theorems in functional analysis known as the Riesz representation theorem. They are named in honour of Frigyes Riesz.- The Hilbert space representation theorem :...
, this implies that for every x in X there exists a unique element Kx of H with the property that:
The function Kx is called the point-evaluation functional at the point x.
Since H is a space of functions, the element Kx is itself a function and can therefore be evaluated at every point. We define the function
byThis function is called the reproducing kernel for the Hilbert space H and it is determined entirely by H because the Riesz representation theorem guarantees, for every x in X, that the element Kx satisfying (*) is unique.
Examples
For example, when X is finite and H consists of all complex-valued functions on X, then an element of H can be represented as an array of complex numbers. If the usual inner product is used, then Kx is the function whose value is 1 at x and 0 everywhere else, and K(x,y) can be thought of as an identity matrix since K(x,y)=1 when x=y and K(x,y)=0 otherwise. In this case, H is isomorphic to.A more sophisticated example is the Hardy space
Hardy space
In complex analysis, the Hardy spaces Hp are certain spaces of holomorphic functions on the unit disk or upper half plane. They were introduced by Frigyes Riesz , who named them after G. H. Hardy, because of the paper...
H2(D)
H square
In mathematics and control theory, H2, or H-square is a Hardy space with square norm. It is a subspace of L2 space, and is thus a Hilbert space...
, the space of square-integrable holomorphic functions on the unit disc. So here X=D, the unit disc. It can be shown that the reproducing kernel for H2(D) is
This kernel is an example of a Bergman kernel
Bergman kernel
In the mathematical study of several complex variables, the Bergman kernel, named after Stefan Bergman, is a reproducing kernel for the Hilbert space of all square integrable holomorphic functions on a domain D in Cn....
, named for Stefan Bergman
Stefan Bergman
Stefan Bergman was a Polish-born American mathematician whose primary work was in complex analysis. He is best known for the kernel function he discovered while at Berlin University in 1922. This function is known today as the Bergman kernel...
.
The reproducing property
It is clear from the discussion above thatIn particular,
Note that
Orthonormal sequences
If is an orthonormal sequence such that the closure of its span is equal to , thenMoore-Aronszajn theorem
In the previous section, we defined a kernel function in terms of a reproducing kernel Hilbert space. It follows from the definition of an inner product that the kernel we defined is symmetric and positive definite. The Moore-Aronszajn theorem goes in the other direction; it says that every symmetric, positive definite kernel defines a unique reproducing kernel Hilbert space.The theorem first appeared in Aronszajn's Theory of Reproducing Kernels, although he attributes it to E. H. Moore
E. H. Moore
Eliakim Hastings Moore was an American mathematician.-Life:Moore, the son of a Methodist minister and grandson of US Congressman Eliakim H. Moore, discovered mathematics through a summer job at the Cincinnati Observatory while in high school. He learned mathematics at Yale University, where he was...
.
Theorem.
Suppose K is a symmetric, positive definite kernel on a set E. Then there is a unique Hilbert space of functions on E for which K is a reproducing kernel.
Proof.
Define, for all x in E,
.Let H0 be the linear span of
.Define an inner product on H0 by
The symmetry of this inner product follows from the symmetry of K and the non-degeneracy follows from the fact that K is positive definite.
Let H be the completion of H0 with respect to this inner product. Then H consists of functions of the form
where
. The fact that the above sum converges for every x follows from the Cauchy-Schwarz inequality.Now we can check the RKHS property, (*):
To prove uniqueness, let G be another Hilbert space of functions for which K is a reproducing kernel. For any x and y in E,
(*) implies that
By linearity,
on the span of . Then G = H by the uniqueness of the completion.Bergman kernel
The Bergman kernel is defined for open sets D in Cn. Take the Hilbert H space of square-integrable functionSquare-integrable function
In mathematics, a quadratically integrable function, also called a square-integrable function, is a real- or complex-valued measurable function for which the integral of the square of the absolute value is finite...
s, for the Lebesgue measure
Lebesgue measure
In measure theory, the Lebesgue measure, named after French mathematician Henri Lebesgue, is the standard way of assigning a measure to subsets of n-dimensional Euclidean space. For n = 1, 2, or 3, it coincides with the standard measure of length, area, or volume. In general, it is also called...
on D, that are holomorphic function
Holomorphic function
In mathematics, holomorphic functions are the central objects of study in complex analysis. A holomorphic function is a complex-valued function of one or more complex variables that is complex differentiable in a neighborhood of every point in its domain...
s. The theory is non-trivial in such cases as there are such functions, which are not identically zero. Then H is a reproducing kernel space, with kernel function the Bergman kernel; this example, with n = 1, was introduced by Bergman in 1922.