Convergence of measures
Encyclopedia
In mathematics
, more specifically measure theory, there are various notions of the convergence of measures. Three of the most common notions of convergence are described below.
Let
be a measurable space.
The total variation
distance between two (positive) measures
and 
is then given by
If
and 
are both probability measures
, then the total variation distance is also given by
The equivalence between these two definitions can be seen as a particular case of the Monge-Kantorovich duality.
From the two definitions above, it is clear that the total variation distance between probability measures is always between 0 and 2.
To illustrate the meaning of the total variation distance, consider the following thought experiment. Assume that we are given two probability
measures
and 
, as well as a random variable 
. We know that 
has law either
or 
, but we do not know which one of the two. Assume now that we are given one single
sample distributed according to the law of
and that we are then asked to guess which one of the two distributions describes that law.
The quantity
then provides a sharp upper bound on the probability that our guess is correct.
a measurable space, a sequence 
is said to converge strongly to a limit 
if
for every set in
.
For example, as a consequence of the Riemann–Lebesgue lemma, the sequence
of measures on the interval [-1,1]
given by
converges strongly to Lebesgue measure, but it does not converge in total variation.
and statistics
, weak convergence (also known as narrow convergence or weak-* convergence which is a more appropriate name from the point of view of functional analysis
but less frequently used) is one of many types of convergence relating to the convergence of measures. It depends on a topology on the underlying space and thus is not a purely measure theoretic notion.
There are several equivalent definition
s of weak convergence of a sequence of measures, some of which are (apparently) more general than others. The equivalence of these conditions is sometimes known as the portmanteau theorem.
Definition. Let S be a metric space
with its Borel σ-algebra Σ. We say that a sequence of probability measure
s on (S, Σ), Pn, n = 1, 2, ..., converges weakly to the probability measure P, and write
if any of the following equivalent conditions is true:
In the case S = R with its usual topology, if Fn, F denote the cumulative distribution function
s of the measures Pn, P respectively, then Pn converges weakly to P if and only if limn→∞ Fn(x) = F(x) for all points x ∈ R at which F is continuous.
For example, the sequence where Pn is the Dirac measure located at 1/n converges weakly to the Dirac measure located at 0 (if we view these as measures on R with the usual topology), but it does not converge strongly. This is intuitively clear: we only know that 1/n is "close" to 0 because of the topology of R.
This definition of weak convergence can be extended for S any metrizable topological space
. It also defines a weak topology on P(S), the set of all probability measures defined on (S, Σ). The weak topology is generated by the following basis of open sets:
where
If S is also separable, then P(S) is metrizable and separable, for example by the Lévy–Prokhorov metric, if S is also compact or Polish
, so is P(S).
If S is separable, it naturally embeds into P(S) as the (closed) set of dirac measures, and its convex hull
is dense.
There are many "arrow notations" for this kind of convergence: the most frequently used are
, 
and 
.
be a probability space
and X be a metric space. If is a sequence of random variable
s then Xn is said to converge weakly (or in distribution or in law) to X as if the sequence of pushforward measure
s (Xn)∗(P) converges weakly to X∗(P) in the sense of weak convergence of measures on X, as defined above.
Mathematics
Mathematics is the study of quantity, space, structure, and change. Mathematicians seek out patterns and formulate new conjectures. Mathematicians resolve the truth or falsity of conjectures by mathematical proofs, which are arguments sufficient to convince other mathematicians of their validity...
, more specifically measure theory, there are various notions of the convergence of measures. Three of the most common notions of convergence are described below.
Total variation convergence of measures
This is the strongest notion of convergence shown on this page and is defined a follows.Let
be a measurable space.The total variation
Total variation
In mathematics, the total variation identifies several slightly different concepts, related to the structure of the codomain of a function or a measure...
distance between two (positive) measures
and is then given byIf
and are both probability measuresProbability measure
In mathematics, a probability measure is a real-valued function defined on a set of events in a probability space that satisfies measure properties such as countable additivity...
, then the total variation distance is also given by
The equivalence between these two definitions can be seen as a particular case of the Monge-Kantorovich duality.
From the two definitions above, it is clear that the total variation distance between probability measures is always between 0 and 2.
To illustrate the meaning of the total variation distance, consider the following thought experiment. Assume that we are given two probability
measures
and , as well as a random variable . We know that has law either
or , but we do not know which one of the two. Assume now that we are given one singlesample distributed according to the law of
and that we are then asked to guess which one of the two distributions describes that law.The quantity
then provides a sharp upper bound on the probability that our guess is correct.
Strong convergence of measures
For a measurable space, a sequence is said to converge strongly to a limit if
for every set in
.For example, as a consequence of the Riemann–Lebesgue lemma, the sequence
of measures on the interval [-1,1]given by
converges strongly to Lebesgue measure, but it does not converge in total variation.Weak convergence of measures
In mathematicsMathematics
Mathematics is the study of quantity, space, structure, and change. Mathematicians seek out patterns and formulate new conjectures. Mathematicians resolve the truth or falsity of conjectures by mathematical proofs, which are arguments sufficient to convince other mathematicians of their validity...
and statistics
Statistics
Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....
, weak convergence (also known as narrow convergence or weak-* convergence which is a more appropriate name from the point of view of functional analysis
Functional analysis
Functional analysis is a branch of mathematical analysis, the core of which is formed by the study of vector spaces endowed with some kind of limit-related structure and the linear operators acting upon these spaces and respecting these structures in a suitable sense...
but less frequently used) is one of many types of convergence relating to the convergence of measures. It depends on a topology on the underlying space and thus is not a purely measure theoretic notion.
There are several equivalent definition
Definition
A definition is a passage that explains the meaning of a term , or a type of thing. The term to be defined is the definiendum. A term may have many different senses or meanings...
s of weak convergence of a sequence of measures, some of which are (apparently) more general than others. The equivalence of these conditions is sometimes known as the portmanteau theorem.
Definition. Let S be a metric space
Metric space
In mathematics, a metric space is a set where a notion of distance between elements of the set is defined.The metric space which most closely corresponds to our intuitive understanding of space is the 3-dimensional Euclidean space...
with its Borel σ-algebra Σ. We say that a sequence of probability measure
Probability measure
In mathematics, a probability measure is a real-valued function defined on a set of events in a probability space that satisfies measure properties such as countable additivity...
s on (S, Σ), Pn, n = 1, 2, ..., converges weakly to the probability measure P, and write
if any of the following equivalent conditions is true:
- Enƒ → Eƒ for all boundedBounded functionIn mathematics, a function f defined on some set X with real or complex values is called bounded, if the set of its values is bounded. In other words, there exists a real number M...
, continuous functionContinuous functionIn mathematics, a continuous function is a function for which, intuitively, "small" changes in the input result in "small" changes in the output. Otherwise, a function is said to be "discontinuous". A continuous function with a continuous inverse function is called "bicontinuous".Continuity of...
s ƒ; - Enƒ → Eƒ for all bounded and Lipschitz functions ƒ;
- limsup Enƒ ≤ Eƒ for every upper semi-continuous function ƒ bounded from above;
- liminf Enƒ ≥ Eƒ for every lower semi-continuous function ƒ bounded from below;
- limsup Pn(C) ≤ P(C) for all closed setClosed setIn geometry, topology, and related branches of mathematics, a closed set is a set whose complement is an open set. In a topological space, a closed set can be defined as a set which contains all its limit points...
s C of space S; - liminf Pn(U) ≥ P(U) for all open setOpen setThe concept of an open set is fundamental to many areas of mathematics, especially point-set topology and metric topology. Intuitively speaking, a set U is open if any point x in U can be "moved" a small amount in any direction and still be in the set U...
s U of space S; - lim Pn(A) = P(A) for all continuity setContinuity setIn measure theory, a continuity set of a measure μ is any Borel set B such that |\mu| = 0\,. The class of all continuity sets for given measure μ forms a ring....
s A of measure P.
In the case S = R with its usual topology, if Fn, F denote the cumulative distribution function
Cumulative distribution function
In probability theory and statistics, the cumulative distribution function , or just distribution function, describes the probability that a real-valued random variable X with a given probability distribution will be found at a value less than or equal to x. Intuitively, it is the "area so far"...
s of the measures Pn, P respectively, then Pn converges weakly to P if and only if limn→∞ Fn(x) = F(x) for all points x ∈ R at which F is continuous.
For example, the sequence where Pn is the Dirac measure located at 1/n converges weakly to the Dirac measure located at 0 (if we view these as measures on R with the usual topology), but it does not converge strongly. This is intuitively clear: we only know that 1/n is "close" to 0 because of the topology of R.
This definition of weak convergence can be extended for S any metrizable topological space
Topological space
Topological spaces are mathematical structures that allow the formal definition of concepts such as convergence, connectedness, and continuity. They appear in virtually every branch of modern mathematics and are a central unifying notion...
. It also defines a weak topology on P(S), the set of all probability measures defined on (S, Σ). The weak topology is generated by the following basis of open sets:
where
If S is also separable, then P(S) is metrizable and separable, for example by the Lévy–Prokhorov metric, if S is also compact or Polish
Polish space
In the mathematical discipline of general topology, a Polish space is a separable completely metrizable topological space; that is, a space homeomorphic to a complete metric space that has a countable dense subset. Polish spaces are so named because they were first extensively studied by Polish...
, so is P(S).
If S is separable, it naturally embeds into P(S) as the (closed) set of dirac measures, and its convex hull
Convex hull
In mathematics, the convex hull or convex envelope for a set of points X in a real vector space V is the minimal convex set containing X....
is dense.
There are many "arrow notations" for this kind of convergence: the most frequently used are
, and .Weak convergence of random variables
Let be a probability spaceProbability space
In probability theory, a probability space or a probability triple is a mathematical construct that models a real-world process consisting of states that occur randomly. A probability space is constructed with a specific kind of situation or experiment in mind...
and X be a metric space. If is a sequence of random variable
Random variable
In probability and statistics, a random variable or stochastic variable is, roughly speaking, a variable whose value results from a measurement on some type of random process. Formally, it is a function from a probability space, typically to the real numbers, which is measurable functionmeasurable...
s then Xn is said to converge weakly (or in distribution or in law) to X as if the sequence of pushforward measure
Pushforward measure
In measure theory, a pushforward measure is obtained by transferring a measure from one measurable space to another using a measurable function.-Definition:...
s (Xn)∗(P) converges weakly to X∗(P) in the sense of weak convergence of measures on X, as defined above.
See also
- Convergence of random variablesConvergence of random variablesIn probability theory, there exist several different notions of convergence of random variables. The convergence of sequences of random variables to some limit random variable is an important concept in probability theory, and its applications to statistics and stochastic processes...
- Prokhorov's theoremProkhorov's theoremIn measure theory Prokhorov’s theorem relates tightness of measures to weak compactness in the space of probability measures. It is credited to the Soviet mathematician Yuri Vasilevich Prokhorov, who considered probability measures on complete separable metric spaces...
- Lévy–Prokhorov metric
- Tightness of measuresTightness of measuresIn mathematics, tightness is a concept in measure theory. The intuitive idea is that a given collection of measures does not "escape to infinity."-Definitions:...