Law of total probability
Encyclopedia
In probability theory
, the law (or formula) of total probability is a fundamental rule relating marginal probabilities to conditional probabilities.
that if
is a finite or countably infinite
partition
of a sample space (in other words, a set of pairwise disjoint event
s whose union
is the entire sample space) and each event
is measurable, then for any event 
of the same probability space
:
or, alternatively,
,
where, for any
for which 
these terms are simply omitted from the summation, because 
is finite.
The summation can be interpreted as a weighted average, and consequently the marginal probability,
, is sometimes called "average probability"; "overall probability" is sometimes used in less formal writings.
The law of total probability can also be stated for conditional probabilities. Taking the
as above, and assuming 
is not mutually exclusive
with
or any of the 
:
is the event 
. It follows that the probability of an event A is equal to the expected value
of the conditional probabilities
of A given
. That is,
where Pr(A|X) is the conditional probability
of A given X, and where EX denotes the expectation with respect to the random variable X.
This result can be generalized to continuous random variables (via continuous conditional density), and the expression becomes
where
denotes the sigma-algebra
generated by the random variable X.
.
Probability theory
Probability theory is the branch of mathematics concerned with analysis of random phenomena. The central objects of probability theory are random variables, stochastic processes, and events: mathematical abstractions of non-deterministic events or measured quantities that may either be single...
, the law (or formula) of total probability is a fundamental rule relating marginal probabilities to conditional probabilities.
Statement
The law of total probability is the propositionProposition
In logic and philosophy, the term proposition refers to either the "content" or "meaning" of a meaningful declarative sentence or the pattern of symbols, marks, or sounds that make up a meaningful declarative sentence...
that if
is a finite or countably infiniteCountable set
In mathematics, a countable set is a set with the same cardinality as some subset of the set of natural numbers. A set that is not countable is called uncountable. The term was originated by Georg Cantor...
partition
Partition of a set
In mathematics, a partition of a set X is a division of X into non-overlapping and non-empty "parts" or "blocks" or "cells" that cover all of X...
of a sample space (in other words, a set of pairwise disjoint event
Event (probability theory)
In probability theory, an event is a set of outcomes to which a probability is assigned. Typically, when the sample space is finite, any subset of the sample space is an event...
s whose union
Union (set theory)
In set theory, the union of a collection of sets is the set of all distinct elements in the collection. The union of a collection of sets S_1, S_2, S_3, \dots , S_n\,\! gives a set S_1 \cup S_2 \cup S_3 \cup \dots \cup S_n.- Definition :...
is the entire sample space) and each event
is measurable, then for any event of the same probability spaceProbability space
In probability theory, a probability space or a probability triple is a mathematical construct that models a real-world process consisting of states that occur randomly. A probability space is constructed with a specific kind of situation or experiment in mind...
:
or, alternatively,
,where, for any
for which these terms are simply omitted from the summation, because is finite.The summation can be interpreted as a weighted average, and consequently the marginal probability,
, is sometimes called "average probability"; "overall probability" is sometimes used in less formal writings.The law of total probability can also be stated for conditional probabilities. Taking the
as above, and assuming is not mutually exclusiveMutually exclusive
In layman's terms, two events are mutually exclusive if they cannot occur at the same time. An example is tossing a coin once, which can result in either heads or tails, but not both....
with
or any of the :Applications
One common application of the law is where the events coincide with a discrete random variable X taking each value in its range, i.e. is the event . It follows that the probability of an event A is equal to the expected valueExpected value
In probability theory, the expected value of a random variable is the weighted average of all possible values that this random variable can take on...
of the conditional probabilities
Conditional probability
In probability theory, the "conditional probability of A given B" is the probability of A if B is known to occur. It is commonly notated P, and sometimes P_B. P can be visualised as the probability of event A when the sample space is restricted to event B...
of A given
. That is,where Pr(A|X) is the conditional probability
Conditional probability
In probability theory, the "conditional probability of A given B" is the probability of A if B is known to occur. It is commonly notated P, and sometimes P_B. P can be visualised as the probability of event A when the sample space is restricted to event B...
of A given X, and where EX denotes the expectation with respect to the random variable X.
This result can be generalized to continuous random variables (via continuous conditional density), and the expression becomes
where
denotes the sigma-algebraSigma-algebra
In mathematics, a σ-algebra is a technical concept for a collection of sets satisfying certain properties. The main use of σ-algebras is in the definition of measures; specifically, the collection of sets over which a measure is defined is a σ-algebra...
generated by the random variable X.
Other names
The term law of total probability is sometimes taken to mean the law of alternatives, which is a special case of the law of total probability applying to discrete random variables. One author even uses the terminology "continuous law of alternatives" in the continuous case. This result is given by Grimmett and Welsh as the partition theorem, a name that they also give to the related law of total expectationLaw of total expectation
The proposition in probability theory known as the law of total expectation, the law of iterated expectations, the tower rule, the smoothing theorem, among other names, states that if X is an integrable random variable The proposition in probability theory known as the law of total expectation, ...
.