Inner product space - AbsoluteAstronomy.com

Mathematics

Mathematics is the study of quantity, space, structure, and change. Mathematicians seek out patterns and formulate new conjectures. Mathematicians resolve the truth or falsity of conjectures by mathematical proofs, which are arguments sufficient to convince other mathematicians of their validity...

, an inner product space is a vector space

Vector space

A vector space is a mathematical structure formed by a collection of vectors: objects that may be added together and multiplied by numbers, called scalars in this context. Scalars are often taken to be real numbers, but one may also consider vector spaces with scalar multiplication by complex...

with an additional structure

Mathematical structure

In mathematics, a structure on a set, or more generally a type, consists of additional mathematical objects that in some manner attach to the set, making it easier to visualize or work with, or endowing the collection with meaning or significance....

called an inner product. This additional structure associates each pair of vectors in the space with a scalar

Scalar (mathematics)

In linear algebra, real numbers are called scalars and relate to vectors in a vector space through the operation of scalar multiplication, in which a vector can be multiplied by a number to produce another vector....

quantity known as the inner product of the vectors. Inner products allow the rigorous introduction of intuitive geometrical notions such as the length

Length

In geometric measurements, length most commonly refers to the longest dimension of an object.In certain contexts, the term "length" is reserved for a certain dimension of an object along which the length is measured. For example it is possible to cut a length of a wire which is shorter than wire...

of a vector or the angle

Angle

In geometry, an angle is the figure formed by two rays sharing a common endpoint, called the vertex of the angle.Angles are usually presumed to be in a Euclidean plane with the circle taken for standard with regard to direction. In fact, an angle is frequently viewed as a measure of an circular arc...

between two vectors. They also provide the means of defining orthogonality

Orthogonality

Orthogonality occurs when two things can vary independently, they are uncorrelated, or they are perpendicular.-Mathematics:In mathematics, two vectors are orthogonal if they are perpendicular, i.e., they form a right angle...

between vectors (zero inner product). Inner product spaces generalize Euclidean space

Euclidean space

In mathematics, Euclidean space is the Euclidean plane and three-dimensional space of Euclidean geometry, as well as the generalizations of these notions to higher dimensions...

s (in which the inner product is the dot product

Dot product

In mathematics, the dot product or scalar product is an algebraic operation that takes two equal-length sequences of numbers and returns a single number obtained by multiplying corresponding entries and then summing those products...

, also known as the scalar product) to vector spaces of any (possibly infinite) dimension

Dimension (vector space)

In mathematics, the dimension of a vector space V is the cardinality of a basis of V. It is sometimes called Hamel dimension or algebraic dimension to distinguish it from other types of dimension...

, and are studied in functional analysis

Functional analysis

Functional analysis is a branch of mathematical analysis, the core of which is formed by the study of vector spaces endowed with some kind of limit-related structure and the linear operators acting upon these spaces and respecting these structures in a suitable sense...

.

An inner product naturally induces an associated norm

Norm (mathematics)

In linear algebra, functional analysis and related areas of mathematics, a norm is a function that assigns a strictly positive length or size to all vectors in a vector space, other than the zero vector...

, thus an inner product space is also a normed vector space

Normed vector space

In mathematics, with 2- or 3-dimensional vectors with real-valued entries, the idea of the "length" of a vector is intuitive and can easily be extended to any real vector space Rn. The following properties of "vector length" are crucial....

. A complete space

Complete space

In mathematical analysis, a metric space M is called complete if every Cauchy sequence of points in M has a limit that is also in M or, alternatively, if every Cauchy sequence in M converges in M....

with an inner product is called a Hilbert space

Hilbert space

The mathematical concept of a Hilbert space, named after David Hilbert, generalizes the notion of Euclidean space. It extends the methods of vector algebra and calculus from the two-dimensional Euclidean plane and three-dimensional space to spaces with any finite or infinite number of dimensions...

. An incomplete space with an inner product is called a pre-Hilbert space, since its completion with respect to the norm, induced by the inner product, becomes a Hilbert space

Hilbert space

. Inner product spaces over the field of complex numbers are sometimes referred to as unitary spaces.

Definition

In this article, the field

Field (mathematics)

In abstract algebra, a field is a commutative ring whose nonzero elements form a group under multiplication. As such it is an algebraic structure with notions of addition, subtraction, multiplication, and division, satisfying certain axioms...

of scalar

Scalar (mathematics)

s denoted

is either
the field of real number

Real number

In mathematics, a real number is a value that represents a quantity along a continuum, such as -5 , 4/3 , 8.6 , √2 and π...

or the field of complex number

Complex number

A complex number is a number consisting of a real part and an imaginary part. Complex numbers extend the idea of the one-dimensional number line to the two-dimensional complex plane by using the number line for the real part and adding a vertical axis to plot the imaginary part...

.

Formally, an inner product space is a vector space

Vector space

V over the field

together with an inner product, i.e., with a map

that satisfies the following three axiom

Axiom

In traditional logic, an axiom or postulate is a proposition that is not proven or demonstrated but considered either to be self-evident or to define and delimit the realm of analysis. In other words, an axiom is a logical statement that is assumed to be true...

s for all vectors

and all scalars

Conjugate
Complex conjugate
In mathematics, complex conjugates are a pair of complex numbers, both having the same real part, but with imaginary parts of equal magnitude and opposite signs...

symmetry:

Note that in

, it is symmetric.

Linear
Linear
In mathematics, a linear map or function f is a function which satisfies the following two properties:* Additivity : f = f + f...

ity in the first argument:

Positive-definiteness
Definite bilinear form
In mathematics, a definite bilinear form is a bilinear form B over some vector space V such that the associated quadratic formQ=B \,...

:

with equality only for

Notice that conjugate symmetry implies that

is real for all

, since we have

Conjugate symmetry and linearity in the first variable gives

so an inner product is a sesquilinear form.
Conjugate symmetry is also called Hermitian symmetry, and a conjugate symmetric sesquilinear form is called a Hermitian form.
While the above axioms are more mathematically economical, a compact verbal definition of an inner product is a positive-definite Hermitian form.

In the case of

, conjugate-symmetric reduces to symmetric, and sesquilinear reduces to bilinear.
So, an inner product on a real vector space is a positive-definite symmetric bilinear form.

From the linearity property it is derived that

implies

while from the positive-definiteness axiom we obtain the converse,

implies

Combining these two, we have the property that

if and only if

The property of an inner product space

that

and

is known as additivity.

Remark: Some authors, especially in physics

Physics

Physics is a natural science that involves the study of matter and its motion through spacetime, along with related concepts such as energy and force. More broadly, it is the general analysis of nature, conducted in order to understand how the universe behaves.Physics is one of the oldest academic...

and matrix algebra

Matrix algebra

Matrix algebra may refer to:*Matrix theory, is the branch of mathematics that studies matrices*Matrix ring, thought of as an algebra over a field or a commutative ring...

, prefer to define the inner product and the sesquilinear form with linearity in the second argument rather than the first. Then the first argument becomes conjugate linear, rather than the second.
In those disciplines we would write the product

(the bra-ket notation

Bra-ket notation

Bra-ket notation is a standard notation for describing quantum states in the theory of quantum mechanics composed of angle brackets and vertical bars. It can also be used to denote abstract vectors and linear functionals in mathematics...

of quantum mechanics

Quantum mechanics

Quantum mechanics, also known as quantum physics or quantum theory, is a branch of physics providing a mathematical description of much of the dual particle-like and wave-like behavior and interactions of energy and matter. It departs from classical mechanics primarily at the atomic and subatomic...

), respectively

(dot product as a case of the convention of forming the matrix product AB as the dot products of rows of A with columns of B). Here the kets and columns are identified with the vectors of V and the bras and rows with the dual vectors or linear functional

Linear functional

In linear algebra, a linear functional or linear form is a linear map from a vector space to its field of scalars. In Rn, if vectors are represented as column vectors, then linear functionals are represented as row vectors, and their action on vectors is given by the dot product, or the...

s of the dual space

Dual space

In mathematics, any vector space, V, has a corresponding dual vector space consisting of all linear functionals on V. Dual vector spaces defined on finite-dimensional vector spaces can be used for defining tensors which are studied in tensor algebra...

V^*, with conjugacy associated with duality. This reverse order is now occasionally followed in the more abstract literature, e.g., Emch [1972], taking

to be conjugate linear in x rather than y. A few instead find a middle ground by recognizing both

and

as distinct notations differing only in which argument is conjugate linear.

There are various technical reasons why it is necessary to restrict the basefield to

and

in the definition. Briefly, the basefield has to contain an ordered subfield

Ordered field

In mathematics, an ordered field is a field together with a total ordering of its elements that is compatible with the field operations. Historically, the axiomatization of an ordered field was abstracted gradually from the real numbers, by mathematicians including David Hilbert, Otto Hölder and...

(in order for non-negativity to make sense) and therefore has to have characteristic

Characteristic (algebra)

In mathematics, the characteristic of a ring R, often denoted char, is defined to be the smallest number of times one must use the ring's multiplicative identity element in a sum to get the additive identity element ; the ring is said to have characteristic zero if this repeated sum never reaches...

equal to 0. This immediately excludes finite fields. The basefield has to have additional structure, such as a distinguished automorphism. More generally any quadratically closed subfield of

will suffice for this purpose, e.g., the algebraic number

Algebraic number

In mathematics, an algebraic number is a number that is a root of a non-zero polynomial in one variable with rational coefficients. Numbers such as π that are not algebraic are said to be transcendental; almost all real numbers are transcendental...

s, but when it is a proper subfield (i.e., neither

nor

) even finite-dimensional inner product spaces will fail to be metrically complete. In contrast all finite-dimensional inner product spaces over

, such as those used in quantum computation, are automatically metrically complete and hence Hilbert spaces.

In some cases we need to consider non-negative semi-definite sesquilinear forms. This means that

is only required to be non-negative. We show how to treat these below.

Examples

A simple example is the real numbers with the standard multiplication as the inner product

More generally any Euclidean space

ⁿ with the dot product

Dot product

is an inner product space

The general form of an inner product on ⁿ is given by:

with M any Hermitian

Hermitian

A number of mathematical entities are named Hermitian, after the mathematician Charles Hermite:*Hermitian adjoint*Hermitian connection, the unique connection on a Hermitian manifold that satisfies specific conditions...

positive-definite matrix

Positive-definite matrix

In linear algebra, a positive-definite matrix is a matrix that in many ways is analogous to a positive real number. The notion is closely related to a positive-definite symmetric bilinear form ....

, and y^* the conjugate transpose

Conjugate transpose

In mathematics, the conjugate transpose, Hermitian transpose, Hermitian conjugate, or adjoint matrix of an m-by-n matrix A with complex entries is the n-by-m matrix A* obtained from A by taking the transpose and then taking the complex conjugate of each entry...

of y. For the real case this corresponds to the dot product of the results of directionally differential scaling

Scaling (geometry)

In Euclidean geometry, uniform scaling is a linear transformation that enlarges or shrinks objects by a scale factor that is the same in all directions. The result of uniform scaling is similar to the original...

of the two vectors, with positive scale factor

Scale factor

A scale factor is a number which scales, or multiplies, some quantity. In the equation y=Cx, C is the scale factor for x. C is also the coefficient of x, and may be called the constant of proportionality of y to x...

s and orthogonal directions of scaling. Up to an orthogonal transformation it is a weighted-sum

Weight function

A weight function is a mathematical device used when performing a sum, integral, or average in order to give some elements more "weight" or influence on the result than other elements in the same set. They occur frequently in statistics and analysis, and are closely related to the concept of a...

version of the dot product, with positive weights.

The article on Hilbert space
Hilbert space
The mathematical concept of a Hilbert space, named after David Hilbert, generalizes the notion of Euclidean space. It extends the methods of vector algebra and calculus from the two-dimensional Euclidean plane and three-dimensional space to spaces with any finite or infinite number of dimensions...

has several examples of inner product spaces wherein the metric induced by the inner product yields a complete
Complete space
In mathematical analysis, a metric space M is called complete if every Cauchy sequence of points in M has a limit that is also in M or, alternatively, if every Cauchy sequence in M converges in M....

metric space. An example of an inner product which induces an incomplete metric occurs with the space C[a, b] of continuous complex valued functions on the interval [a, b]. The inner product is

This space is not complete; consider for example, for the interval [−1,1] the sequence of "step" functions { f_k }_k where

f_k(t) is 0 for t in the subinterval [−1,0]
f_k(t) is 1 for t in the subinterval [1/k, 1]
f_k is affine in [0, 1/k].

This sequence is a Cauchy sequence

Cauchy sequence

In mathematics, a Cauchy sequence , named after Augustin-Louis Cauchy, is a sequence whose elements become arbitrarily close to each other as the sequence progresses...

which does not converge to a continuous function.

For random variable
Random variable
In probability and statistics, a random variable or stochastic variable is, roughly speaking, a variable whose value results from a measurement on some type of random process. Formally, it is a function from a probability space, typically to the real numbers, which is measurable functionmeasurable...

s X and Y, the expected value
Expected value
In probability theory, the expected value of a random variable is the weighted average of all possible values that this random variable can take on...

of their product

is an inner product. In this case, <X, X>=0 if and only if Pr

Probability

Probability is ordinarily used to describe an attitude of mind towards some proposition of whose truth we arenot certain. The proposition of interest is usually of the form "Will a specific event occur?" The attitude of mind is of the form "How certain are we that the event will occur?" The...

(X=0)=1 (i.e., X=0 almost surely

Almost surely

In probability theory, one says that an event happens almost surely if it happens with probability one. The concept is analogous to the concept of "almost everywhere" in measure theory...

). This definition of expectation as inner product can be extended to random vectors as well.

For square real matrices, with transpose as conjugation is an inner product.

Norms on inner product spaces

A linear space with a norm such as:

where p ≠ 2 is a normed space but not an inner product space, because this norm does not satisfy the parallelogram equality required of a norm to have an inner product associated with it.

However, inner product spaces have a naturally defined norm

Norm (mathematics)

based upon the inner product of the space itself that does satisfy the parallelogram equality:

This is well defined by the nonnegativity axiom of the definition of inner product space. The norm is thought of as the length of the vector x.
Directly from the axioms, we can prove the following:

Cauchy–Schwarz inequality: for x, y elements of V

with equality if and only if x and y are linearly dependent. This is one of the most important inequalities in mathematics. It is also known in the Russian mathematical literature as the Cauchy–Bunyakowski–Schwarz inequality.

Because of its importance, its short proof should be noted.

It is trivial to prove the inequality true in the case y = 0. Thus we assume ⟨y, y⟩ is nonzero, giving us the following:

The complete proof can be obtained by multiplying out this result.

Orthogonality: The geometric interpretation of the inner product in terms of angle and length, motivates much of the geometric terminology we use in regard to these spaces. Indeed, an immediate consequence of the Cauchy-Schwarz inequality is that it justifies defining the angle
Angle
In geometry, an angle is the figure formed by two rays sharing a common endpoint, called the vertex of the angle.Angles are usually presumed to be in a Euclidean plane with the circle taken for standard with regard to direction. In fact, an angle is frequently viewed as a measure of an circular arc...

between two non-zero vectors x and y in the case F = by the identity

We assume the value of the angle is chosen to be in the interval [0, +π]. This is in analogy to the situation in two-dimensional Euclidean space

Euclidean space

In mathematics, Euclidean space is the Euclidean plane and three-dimensional space of Euclidean geometry, as well as the generalizations of these notions to higher dimensions...

In the case F =

, the angle in the interval [0, +π/2] is typically defined by

Correspondingly, we will say that non-zero vectors x and y of V are orthogonal if and only if their inner product is zero.

Homogeneity
Homogeneous function
In mathematics, a homogeneous function is a function with multiplicative scaling behaviour: if the argument is multiplied by a factor, then the result is multiplied by some power of this factor. More precisely, if is a function between two vector spaces over a field F, and k is an integer, then...

: for x an element of V and r a scalar

The homogeneity property is completely trivial to prove.

Triangle inequality
Triangle inequality
In mathematics, the triangle inequality states that for any triangle, the sum of the lengths of any two sides must be greater than or equal to the length of the remaining side ....

: for x, y elements of V

The last two properties show the function defined is indeed a norm.

Because of the triangle inequality and because of axiom 2, we see that ||·|| is a norm which turns V into a normed vector space

Normed vector space

and hence also into a metric space

Metric space

In mathematics, a metric space is a set where a notion of distance between elements of the set is defined.The metric space which most closely corresponds to our intuitive understanding of space is the 3-dimensional Euclidean space...

. The most important inner product spaces are the ones which are complete with respect to this metric; they are called Hilbert space

Hilbert space

s. Every inner product V space is a dense

Dense set

In topology and related areas of mathematics, a subset A of a topological space X is called dense if any point x in X belongs to A or is a limit point of A...

subspace of some Hilbert space. This Hilbert space is essentially uniquely determined by V and is constructed by completing V.

Pythagorean theorem
Pythagorean theorem
In mathematics, the Pythagorean theorem or Pythagoras' theorem is a relation in Euclidean geometry among the three sides of a right triangle...

: Whenever x, y are in V and ⟨x, y⟩ = 0, then

The proof of the identity requires only expressing the definition of norm in terms of the inner product and multiplying out, using the property of additivity of each component.

The name Pythagorean theorem arises from the geometric interpretation of this result as an analogue of the theorem in synthetic geometry

Synthetic geometry

Synthetic or axiomatic geometry is the branch of geometry which makes use of axioms, theorems and logical arguments to draw conclusions, as opposed to analytic and algebraic geometries which use analysis and algebra to perform geometric computations and solve problems.-Logical synthesis:The process...

. Note that the proof of the Pythagorean theorem in synthetic geometry is considerably more elaborate because of the paucity of underlying structure. In this sense, the synthetic Pythagorean theorem, if correctly demonstrated is deeper than the version given above.

An induction

Mathematical induction

Mathematical induction is a method of mathematical proof typically used to establish that a given statement is true of all natural numbers...

on the Pythagorean theorem yields:

If x₁, ..., x_n are orthogonal vectors, that is, for distinct indices j, k, then

In view of the Cauchy-Schwarz inequality, we also note that

is continuous

Continuous function

In mathematics, a continuous function is a function for which, intuitively, "small" changes in the input result in "small" changes in the output. Otherwise, a function is said to be "discontinuous". A continuous function with a continuous inverse function is called "bicontinuous".Continuity of...

from V × V to F. This allows us to extend Pythagoras' theorem to infinitely many summands:

Parseval's identity: Suppose V is a complete inner product space. If {x_k} are mutually orthogonal vectors in V then

provided the infinite series on the left is convergent
Convergence (series)
Convergence is a series of books edited by Ruth Nanda Anshen and published by the Columbia University Press dealing with ideas that changed, or that are changing the world....

. Completeness of the space is needed to ensure that the sequence of partial sums

which is easily shown to be a Cauchy sequence

Cauchy sequence

In mathematics, a Cauchy sequence , named after Augustin-Louis Cauchy, is a sequence whose elements become arbitrarily close to each other as the sequence progresses...

, is convergent.

Parallelogram law
Parallelogram law
In mathematics, the simplest form of the parallelogram law belongs to elementary geometry. It states that the sum of the squares of the lengths of the four sides of a parallelogram equals the sum of the squares of the lengths of the two diagonals...

: for x, y elements of V,

The Parallelogram law is, in fact, a necessary and sufficient condition for the existence of a scalar
product, corresponding to a given norm. If it holds, the scalar product is defined by the
polarization identity

Polarization identity

In mathematics, the polarization identity is any one of a family of formulas that express the inner product of two vectors in terms of the norm of a normed vector space. Let \|x\| \, denote the norm of vector x and \langle x, \ y \rangle \, the inner product of vectors x and y...

which is a form of the law of cosines

Law of cosines

In trigonometry, the law of cosines relates the lengths of the sides of a plane triangle to the cosine of one of its angles. Using notation as in Fig...

Orthonormal sequences

Let V be a finite dimensional inner product space of dimension n. Recall that every basis

Basis (linear algebra)

In linear algebra, a basis is a set of linearly independent vectors that, in a linear combination, can represent every vector in a given vector space or free module, or, more simply put, which define a "coordinate system"...

of V consists of exactly n linearly independent vectors. Using the Gram-Schmidt Process we may start with an arbitrary basis and transform it into an orthonormal basis. That is, into a basis in which all the elements are orthogonal and have unit norm. In symbols, a basis

is orthonormal if

and

for each i.

This definition of orthonormal basis generalizes to the case of infinite dimensional inner product spaces in the following way. Let V be a any inner product space. Then a collection

is a basis for V if the subspace of V generated by finite linear combinations of elements of E is dense in V (in the norm induced by the inner product). We say that E is an orthonormal basis for V if it is a basis and

and

for all

.

Using an infinite-dimensional analog of the Gram-Schmidt process one may show:

Theorem. Any separable inner product space V has an orthonormal basis.

Using the Hausdorff Maximal Principle

Hausdorff maximal principle

In mathematics, the Hausdorff maximal principle is an alternate and earlier formulation of Zorn's lemma proved by Felix Hausdorff in 1914...

and the fact that in a complete inner product space

Hilbert space

orthogonal projection onto linear subspaces is well-defined, one may also show that

Theorem. Any complete inner product space

Hilbert space

V has an orthonormal basis.

The two previous theorems raise the question of whether all inner product spaces have an orthonormal basis. The answer, it turns out is negative. This is a non-trivial result, and is proved below. The following proof is taken from Halmos's A Hilbert Space Problem Book (see the references).

Proof

G is a dense subspace of an inner product space H, then any orthonormal basis for G is automatically an orthonormal basis for H. Thus, it suffices to construct an inner product space space H with a dense subspace G whose dimension is strictly smaller than that of H.

Let K be a Hilbert space of dimension

(for instance,

). Let E be an orthonormal basis of K, so

. Extend E to a Hamel basis

for K, where

. Since it is known that the Hamel dimension of K is c, the cardinality of the continuum, it must be that

.

Let L be a Hilbert space of dimension c (for instance,

). Let B be an orthonormal basis for L, and let

be a bijection. Then there is a linear transformation

such that

for

, and

for

.

Let

and let

be the graph of T. Let

be the closure of G in H; we will show

. Since for any

we have

, it follows that

.

Next, if

, then

for some

, so

; since

as well, we also have

. It follows that

, so

, and G is dense in H.

Finally,

is a maximal orthonormal set in G; if

for all

then certainly

, so

is the zero vector in G. Hence the dimension of G is

, whereas it is clear that the dimension of H is c. This completes the proof.

Parseval's identity

In mathematical analysis, Parseval's identity is a fundamental result on the summability of the Fourier series of a function. Geometrically, it is thePythagorean theorem for inner-product spaces....

leads immediately to the following theorem:

Theorem. Let V be a separable inner product space and {e_k}_k an orthonormal basis of V.
Then the map

is an isometric linear map V → ℓ² with a dense image.

This theorem can be regarded as an abstract form of Fourier series

Fourier series

In mathematics, a Fourier series decomposes periodic functions or periodic signals into the sum of a set of simple oscillating functions, namely sines and cosines...

, in which an arbitrary orthonormal basis plays the role of the sequence of trigonometric polynomial

Trigonometric polynomial

In the mathematical subfields of numerical analysis and mathematical analysis, a trigonometric polynomial is a finite linear combination of functions sin and cos with n a natural number. The coefficients may be taken as real numbers, for real-valued functions...

s. Note that the underlying index set can be taken to be any countable set (and in fact any set whatsoever, provided ℓ² is defined appropriately, as is explained in the article Hilbert space

Hilbert space

).
In particular, we obtain the following result in the theory of Fourier series:

Theorem. Let V be the inner product space

. Then the sequence (indexed on set of all integers) of continuous functions

is an orthonormal basis of the space

with the L² inner product. The mapping

is an isometric linear map with dense image.

Orthogonality of the sequence {e_k}_k follows immediately from the fact that if k ≠ j, then

Normality of the sequence is by design, that is, the coefficients are so chosen so that the norm comes out to 1. Finally the fact that the sequence has a dense algebraic span, in the inner product norm, follows from the fact that the sequence has a dense algebraic span, this time in the space of continuous periodic functions on

with the uniform norm. This is the content of the Weierstrass theorem on the uniform density of trigonometric polynomials.

Operators on inner product spaces

Several types of linear

Linear

In mathematics, a linear map or function f is a function which satisfies the following two properties:* Additivity : f = f + f...

maps A from an inner product space V to an inner product space W are of relevance:

Continuous linear maps
Continuous linear operator
In functional analysis and related areas of mathematics, a continuous linear operator or continuous linear mapping is a continuous linear transformation between topological vector spaces....

, i.e., A is linear and continuous with respect to the metric defined above, or equivalently, A is linear and the set of non-negative reals {||Ax||}, where x ranges over the closed unit ball of V, is bounded.
Symmetric linear operators, i.e., A is linear and ⟨Ax, y⟩ = ⟨x, A y⟩ for all x, y in V.
Isometries, i.e., A is linear and ⟨Ax, Ay⟩ = ⟨x, y⟩ for all x, y in V, or equivalently, A is linear and ||Ax|| = ||x|| for all x in V. All isometries are injective. Isometries are morphism
Morphism
In mathematics, a morphism is an abstraction derived from structure-preserving mappings between two mathematical structures. The notion of morphism recurs in much of contemporary mathematics...

s between inner product spaces, and morphisms of real inner product spaces are orthogonal transformations (compare with orthogonal matrix
Orthogonal matrix
In linear algebra, an orthogonal matrix , is a square matrix with real entries whose columns and rows are orthogonal unit vectors ....

).
Isometrical isomorphisms, i.e., A is an isometry which is surjective (and hence bijective). Isometrical isomorphisms are also known as unitary operators (compare with unitary matrix).

From the point of view of inner product space theory, there is no need to distinguish between two spaces which are isometrically isomorphic. The spectral theorem

Spectral theorem

In mathematics, particularly linear algebra and functional analysis, the spectral theorem is any of a number of results about linear operators or about matrices. In broad terms the spectral theorem provides conditions under which an operator or a matrix can be diagonalized...

provides a canonical form for symmetric, unitary and more generally normal operator

Normal operator

In mathematics, especially functional analysis, a normal operator on a complex Hilbert space H is a continuous linear operatorN:H\to Hthat commutes with its hermitian adjoint N*: N\,N^*=N^*N....

s on finite dimensional inner product spaces. A generalization of the spectral theorem holds for continuous normal operators in Hilbert spaces.

Generalizations

Any of the axioms of an inner product may be weakened, yielding generalized notions. The generalizations that are closest to inner products occur where bilinearity and conjugate symmetry are retained, but positive-definiteness is weakened.

Degenerate inner products

If V is a vector space and

a semi-definite sesquilinear form,
then the function ‖x‖ =

makes sense and satisfies all the properties of norm except that ‖x‖ = 0 does not imply x = 0 (such a functional is then called a semi-norm). We can produce an inner product space by considering the
quotient W = V/{ x : ‖x‖ = 0}. The sesquilinear form

factors through W.

This construction is used in numerous contexts. The Gelfand–Naimark–Segal construction is a particularly important example of the use of this technique. Another example is the representation of semi-definite kernel

Mercer's theorem

In mathematics, specifically functional analysis, Mercer's theorem is a representation of a symmetric positive-definite function on a square as a sum of a convergent sequence of product functions. This theorem, presented in , is one of the most notable results of the work of James Mercer...

s on arbitrary sets.

Nondegenerate conjugate symmetric forms

Alternatively, one may require that the pairing be a nondegenerate form, meaning that for all non-zero x there exists some y such that

though y need not equal x; in other words, the induced map to the dual space

is injective. This generalization is important in differential geometry: a manifold whose tangent spaces have an inner product is a Riemannian manifold

Riemannian manifold

In Riemannian geometry and the differential geometry of surfaces, a Riemannian manifold or Riemannian space is a real differentiable manifold M in which each tangent space is equipped with an inner product g, a Riemannian metric, which varies smoothly from point to point...

, while if this is related to nondegenerate conjugate symmetric form the manifold is a pseudo-Riemannian manifold

Pseudo-Riemannian manifold

In differential geometry, a pseudo-Riemannian manifold is a generalization of a Riemannian manifold. It is one of many mathematical objects named after Bernhard Riemann. The key difference between a Riemannian manifold and a pseudo-Riemannian manifold is that on a pseudo-Riemannian manifold the...

. By Sylvester's law of inertia

Sylvester's law of inertia

Sylvester's law of inertia is a theorem in matrix algebra about certain properties of the coefficient matrix of a real quadratic form that remain invariant under a change of coordinates...

, just as every inner product is similar to the dot product with positive weights on a set of vectors, every nondegenerate conjugate symmetric form is similar to the dot product with nonzero weights on a set of vectors, and the number of positive and negative weights are called respectively the positive index and negative index.

Purely algebraic statements (ones that do not use positivity) usually only rely on the nondegeneracy (the injective homomorphism

) and thus hold more generally.

The Minkowski inner product

The Minkowski inner product is typically defined in a 4-dimensional real vector space. It satisfies all the axioms of an inner product, except that it is not positive-definite

Definite bilinear form

In mathematics, a definite bilinear form is a bilinear form B over some vector space V such that the associated quadratic formQ=B \,...

, i.e., the Minkowski norm ||v|| of a vector v, defined as ||v||² = η(v,v), need not be positive. The positive-definite condition has been replaced by the weaker condition of nondegeneracy (every positive-definite form is nondegenerate but not vice-versa). It is common to call a Minkowski inner product an indefinite inner product, although, technically speaking, it is not an inner product according to the standard definition above.

Related products

The term "inner product" is opposed to outer product

Outer product

In linear algebra, the outer product typically refers to the tensor product of two vectors. The result of applying the outer product to a pair of vectors is a matrix...

, which is a slightly more general opposite. Simply, in coordinates, the inner product is the product of a 1×n covector with an n×1 vector, yielding a 1×1 matrix (a scalar), while the outer product is the product of an m×1 vector with a 1×n covector, yielding an m×n matrix. Note that the outer product is defined for different dimensions, while the inner product requires the same dimension. If the dimensions are the same, then the inner product is the trace of the outer product (trace only being properly defined for square matrices).

On an inner product space, or more generally a vector space with a nondegenerate form (so an isomorphism

) vectors can be sent to covectors (in coordinates, via transpose), so one can take the inner product and outer product of two vectors, not simply of a vector and a covector.

In a quip: "inner is horizontal times vertical and shrinks down, outer is vertical times horizontal and expands out".

More abstractly, the outer product is the bilinear map

sending a vector and a covector to a rank 1 linear transformation (simple tensor of type (1,1)), while the inner product is the bilinear evaluation map

given by evaluating a covector on a vector; the order of the domain vector spaces here reflects the covector/vector distinction.

The inner product and outer product should not be confused with the interior product and exterior product, which are instead operations on vector field

Vector field

In vector calculus, a vector field is an assignmentof a vector to each point in a subset of Euclidean space. A vector field in the plane for instance can be visualized as an arrow, with a given magnitude and direction, attached to each point in the plane...

s and differential form

Differential form

In the mathematical fields of differential geometry and tensor calculus, differential forms are an approach to multivariable calculus that is independent of coordinates. Differential forms provide a better definition for integrands in calculus...

s, or more generally on the exterior algebra

Exterior algebra

In mathematics, the exterior product or wedge product of vectors is an algebraic construction used in Euclidean geometry to study areas, volumes, and their higher-dimensional analogs...

.

As a further complication, in geometric algebra

Geometric algebra

Geometric algebra , together with the associated Geometric calculus, provides a comprehensive alternative approach to the algebraic representation of classical, computational and relativistic geometry. GA now finds application in all of physics, in graphics and in robotics...

the inner product and the exterior (Grassmann) product are combined in the geometric product (the Clifford product in a Clifford algebra

Clifford algebra

In mathematics, Clifford algebras are a type of associative algebra. As K-algebras, they generalize the real numbers, complex numbers, quaternions and several other hypercomplex number systems. The theory of Clifford algebras is intimately connected with the theory of quadratic forms and orthogonal...

) – the inner product sends two vectors (1-vectors) to a scalar (a 0-vector), while the exterior product sends two vectors to a bivector (2-vector) – and in this context the exterior product is usually called the "outer(alternatively,wedge) product". The inner product is more correctly called a scalar product in this context, as the nondegenerate quadratic form in question need not be positive definite (need not be an inner product).

Definition

Examples

Norms on inner product spaces

Orthonormal sequences

Operators on inner product spaces

Generalizations

Degenerate inner products

Nondegenerate conjugate symmetric forms

The Minkowski inner product

Related products

See also