Positive definite kernel
Encyclopedia
In operator theory
, a branch of mathematics
, a positive definite kernel is a generalization of a positive-definite matrix.
be a sequence of (complex) Hilbert space
s and
be the bounded operators from Hi to Hj.
A map A on
where
is called a positive definite kernel if for all m > 0 and
, the following positivity condition holds:
Consider the set F0(X) of complex-valued functions f: X → C with finite support. With the natural operations, F0(X) is called the free vector space generated by X. Let δx be the element in F0(X) defined by δx(y) = δxy. The set {δx}x ∈ X is a vector space basis of F0(X).
Suppose now K: X × X → C is a positive definite kernel, then the Kolmogorov decomposition of K gives a Hilbert space
where F0(X) is "dense" (after possibly taking quotients of the degenerate subspace). Also, ⟨[δx], [δy]⟩ = K(x,y), which is a special case of the square root factorization claim above. This Hilbert space is called the reproducing kernel Hilbert space with kernel K on the set X.
Notice that in this context, we have (from the definition above)
being replaced by
Thus the Kolmogorov decomposition, which is unique up to isomorphism, starts with F0(X).
One can readily show that every Hilbert space is isomorphic to a reproducing kernel Hilbert space on a set whose cardinality is the Hilbert space dimension of H. Let {ex}x ∈ X be an orthonormal basis of H. Then the kernel K defined by K(x, y) = ⟨ex, ey⟩ = δxy reproduces a Hilbert space H. The bijection taking ex to δx extends to a unitary operator from H to H' .
of two Hilbert spaces via their kernels.
Consider two Hilbert spaces H(K, X) and H(L, Y). The disjoint union
of X and Y is the set
Define a kernel
on this disjoint union in a way that is similar to direct sum of positive matrices, and the resulting Hilbert space
is then the direct sum, in the sense of Hilbert spaces, of H(K, X) and H(L, Y).
For the tensor product, a suitable kernel
is defined on the Cartesian product X × Y in a way that extends the Schur product of positive matrices:
This positive kernel gives the tensor product of H(K, X) and H(L, Y),
in which the family { [δ(x,y)] } is a total set, i.e. its linear span is dense.
in the following sense:
Furthermore, if C and G is another pair, C an operator and G a Hilbert space, for which the above is true, then there exists a unitary operator U: G → HA such that B = UC.
The can be shown readily as follows. The matrix A induces a degenerate inner product <·, ·>A given by <x, y>A = <x, Ay>. Taking the quotient with respect to the degenerate subspace gives a Hilbert space HA, a typical element of which is an equivalence class we denote by [x].
Now let B: Cn → HA be the natural projection map, Bx = [x]. One can calculate directly that
.
So B*B = A. If C and G is another such pair, it is clear that the operator U: G → HA that takes [x]G in G to [x] in HA has the properties claimed above.
If {ei} is a given orthonormal basis of Cn, then {Bi = Bei} are the column vectors of B. The expression A = B*B can be rewritten as Ai, j = Bi*Bj. By construction, HA is the linear span of {Bi}.
The condition that HA = ∨B(n)Hn is referred to as the minimality condition. Similar to the scalar case, this requirement implies unitary freedom in the decomposition:
Operator theory
In mathematics, operator theory is the branch of functional analysis that focuses on bounded linear operators, but which includes closed operators and nonlinear operators.Operator theory also includes the study of algebras of operators....
, a branch of mathematics
Mathematics
Mathematics is the study of quantity, space, structure, and change. Mathematicians seek out patterns and formulate new conjectures. Mathematicians resolve the truth or falsity of conjectures by mathematical proofs, which are arguments sufficient to convince other mathematicians of their validity...
, a positive definite kernel is a generalization of a positive-definite matrix.
Definition
Letbe a sequence of (complex) Hilbert space
Hilbert space
The mathematical concept of a Hilbert space, named after David Hilbert, generalizes the notion of Euclidean space. It extends the methods of vector algebra and calculus from the two-dimensional Euclidean plane and three-dimensional space to spaces with any finite or infinite number of dimensions...
s and
be the bounded operators from Hi to Hj.
A map A on
whereis called a positive definite kernel if for all m > 0 and
, the following positivity condition holds:Examples
Positive definite kernels provide a framework that encompasses some basic Hilbert space constructions.Reproducing kernel Hilbert space
The definition and characterization of positive kernels extend verbatim to the case where the integers Z is replaced by an arbitrary set X. One can then give a fairly general procedure for constructing Hilbert spaces that is itself of some interest.Consider the set F0(X) of complex-valued functions f: X → C with finite support. With the natural operations, F0(X) is called the free vector space generated by X. Let δx be the element in F0(X) defined by δx(y) = δxy. The set {δx}x ∈ X is a vector space basis of F0(X).
Suppose now K: X × X → C is a positive definite kernel, then the Kolmogorov decomposition of K gives a Hilbert space
where F0(X) is "dense" (after possibly taking quotients of the degenerate subspace). Also, ⟨[δx], [δy]⟩ = K(x,y), which is a special case of the square root factorization claim above. This Hilbert space is called the reproducing kernel Hilbert space with kernel K on the set X.
Notice that in this context, we have (from the definition above)
being replaced by
Thus the Kolmogorov decomposition, which is unique up to isomorphism, starts with F0(X).
One can readily show that every Hilbert space is isomorphic to a reproducing kernel Hilbert space on a set whose cardinality is the Hilbert space dimension of H. Let {ex}x ∈ X be an orthonormal basis of H. Then the kernel K defined by K(x, y) = ⟨ex, ey⟩ = δxy reproduces a Hilbert space H. The bijection taking ex to δx extends to a unitary operator from H to H' .
Direct sum and tensor product
Let H(K, X) denote the Hilbert space corresponding to a positive kernel K on X × X. The structure of H(K, X) is encoded in K. One can thus describe, for example, the direct sum and the tensor productTensor product
In mathematics, the tensor product, denoted by ⊗, may be applied in different contexts to vectors, matrices, tensors, vector spaces, algebras, topological vector spaces, and modules, among many other structures or objects. In each case the significance of the symbol is the same: the most general...
of two Hilbert spaces via their kernels.
Consider two Hilbert spaces H(K, X) and H(L, Y). The disjoint union
Disjoint union
In mathematics, the term disjoint union may refer to one of two different concepts:* In set theory, a disjoint union is a modified union operation that indexes the elements according to which set they originated in; disjoint sets have no element in common.* In probability theory , a disjoint union...
of X and Y is the set
Define a kernel
on this disjoint union in a way that is similar to direct sum of positive matrices, and the resulting Hilbert space
is then the direct sum, in the sense of Hilbert spaces, of H(K, X) and H(L, Y).
For the tensor product, a suitable kernel
is defined on the Cartesian product X × Y in a way that extends the Schur product of positive matrices:
This positive kernel gives the tensor product of H(K, X) and H(L, Y),
in which the family { [δ(x,y)] } is a total set, i.e. its linear span is dense.
Motivation
Consider a positive matrix A ∈ Cn × n, whose entries are complex numbers. Every such matrix A has a "square root factorization"Square root of a matrix
In mathematics, the square root of a matrix extends the notion of square root from numbers to matrices. A matrix B is said to be a square root of A if the matrix product B · B is equal to A.-Properties:...
in the following sense:
- A = B*B where B: Cn → HA for some (finite dimensional) Hilbert space HA.
Furthermore, if C and G is another pair, C an operator and G a Hilbert space, for which the above is true, then there exists a unitary operator U: G → HA such that B = UC.
The can be shown readily as follows. The matrix A induces a degenerate inner product <·, ·>A given by <x, y>A = <x, Ay>. Taking the quotient with respect to the degenerate subspace gives a Hilbert space HA, a typical element of which is an equivalence class we denote by [x].
Now let B: Cn → HA be the natural projection map, Bx = [x]. One can calculate directly that
.So B*B = A. If C and G is another such pair, it is clear that the operator U: G → HA that takes [x]G in G to [x] in HA has the properties claimed above.
If {ei} is a given orthonormal basis of Cn, then {Bi = Bei} are the column vectors of B. The expression A = B*B can be rewritten as Ai, j = Bi*Bj. By construction, HA is the linear span of {Bi}.
Kolmogorov decomposition
This preceding discussion shows that every positive matrix A with complex entries can expressed as a Gramian matrix. A similar description can be obtained for general positive definite kernels, with an analogous argument. This is called the Kolmogorov decomposition:- Let A be a positive definite kernel. Then there exists a Hilbert space HA and a map B defined on Z where B(n) lies in
The condition that HA = ∨B(n)Hn is referred to as the minimality condition. Similar to the scalar case, this requirement implies unitary freedom in the decomposition:
- If there is a Hilbert space G and a map C on Z that gives a Kolmogorov decomposition of A, then there is a unitary operator