Normally distributed and uncorrelated does not imply independent
Encyclopedia
In probability theory
, two random variable
s being uncorrelated
does not imply their independence
. In some contexts, uncorrelatedness implies at least pairwise independence
(as when the random variables involved have Bernoulli distributions).
It is sometimes mistakenly thought that one context in which uncorrelatedness implies independence is when the random variables involved are normally distributed. However, this is incorrect if the variables are merely marginally normally distributed but not jointly normally distributed.
Suppose two random variables X and Y are jointly normally distributed. That is the same as saying that the random vector (X, Y) has a multivariate normal distribution. It means that the joint probability distribution of X and Y is such that for any two constant (i.e., non-random) scalars a and b, the random variable aX + bY is normally distributed. In that case if X and Y are uncorrelated, i.e., their covariance
cov(X, Y) is zero, then they are independent. However, it is possible for two random variables X and Y to be so distributed jointly that each one alone is marginally normally distributed, and they are uncorrelated, but they are not independent; examples are given below.
0 and variance 1. Let W = 1 or −1, each with probability 1/2, and assume W is independent of X. Let Y = WX. Then
Note that the distribution of X + Y concentrates positive probability at 0: Pr(X + Y = 0) = 1/2.
To see that X and Y are uncorrelated, consider
Probability theory
Probability theory is the branch of mathematics concerned with analysis of random phenomena. The central objects of probability theory are random variables, stochastic processes, and events: mathematical abstractions of non-deterministic events or measured quantities that may either be single...
, two random variable
Random variable
In probability and statistics, a random variable or stochastic variable is, roughly speaking, a variable whose value results from a measurement on some type of random process. Formally, it is a function from a probability space, typically to the real numbers, which is measurable functionmeasurable...
s being uncorrelated
Pearson product-moment correlation coefficient
In statistics, the Pearson product-moment correlation coefficient is a measure of the correlation between two variables X and Y, giving a value between +1 and −1 inclusive...
does not imply their independence
Statistical independence
In probability theory, to say that two events are independent intuitively means that the occurrence of one event makes it neither more nor less probable that the other occurs...
. In some contexts, uncorrelatedness implies at least pairwise independence
Pairwise independence
In probability theory, a pairwise independent collection of random variables is a set of random variables any two of which are independent. Any collection of mutually independent random variables is pairwise independent, but some pairwise independent collections are not mutually independent...
(as when the random variables involved have Bernoulli distributions).
It is sometimes mistakenly thought that one context in which uncorrelatedness implies independence is when the random variables involved are normally distributed. However, this is incorrect if the variables are merely marginally normally distributed but not jointly normally distributed.
Suppose two random variables X and Y are jointly normally distributed. That is the same as saying that the random vector (X, Y) has a multivariate normal distribution. It means that the joint probability distribution of X and Y is such that for any two constant (i.e., non-random) scalars a and b, the random variable aX + bY is normally distributed. In that case if X and Y are uncorrelated, i.e., their covariance
Covariance
In probability theory and statistics, covariance is a measure of how much two variables change together. Variance is a special case of the covariance when the two variables are identical.- Definition :...
cov(X, Y) is zero, then they are independent. However, it is possible for two random variables X and Y to be so distributed jointly that each one alone is marginally normally distributed, and they are uncorrelated, but they are not independent; examples are given below.
A symmetric example
Suppose X has a normal distribution with expected valueExpected value
In probability theory, the expected value of a random variable is the weighted average of all possible values that this random variable can take on...
0 and variance 1. Let W = 1 or −1, each with probability 1/2, and assume W is independent of X. Let Y = WX. Then
- X and Y are uncorrelated;
- Both have the same normal distribution; and
- X and Y are not independent.
Note that the distribution of X + Y concentrates positive probability at 0: Pr(X + Y = 0) = 1/2.
To see that X and Y are uncorrelated, consider
-
To see that Y has the same normal distribution as X, consider
-
(since X and −X both have the same normal distribution).
To see that X and Y are not independent, observe that |Y| = |X| or that Pr(Y > 1 | X = 1/2) = 0.
An asymmetric example
Suppose X has a normal distribution with expected valueExpected valueIn probability theory, the expected value of a random variable is the weighted average of all possible values that this random variable can take on...
0 and variance 1. Let
-
where c is a positive number to be specified below. If c is very small, then the correlationCorrelationIn statistics, dependence refers to any statistical relationship between two random variables or two sets of data. Correlation refers to any of a broad class of statistical relationships involving dependence....
corr(X, Y) is near 1; if c is very large, then corr(X, Y) is near −1. Since the correlation is a continuous functionContinuous functionIn mathematics, a continuous function is a function for which, intuitively, "small" changes in the input result in "small" changes in the output. Otherwise, a function is said to be "discontinuous". A continuous function with a continuous inverse function is called "bicontinuous".Continuity of...
of c, the intermediate value theoremIntermediate value theoremIn mathematical analysis, the intermediate value theorem states that for each value between the least upper bound and greatest lower bound of the image of a continuous function there is at least one point in its domain that the function maps to that value....
implies there is some particular value of c that makes the correlation 0. That value is approximately 1.54. In that case, X and Y are uncorrelated, but they are clearly not independent, since X completely determines Y.
To see that Y is normally distributed—indeed, that its distribution is the same as that of X—let us find its cumulative distribution functionCumulative distribution functionIn probability theory and statistics, the cumulative distribution function , or just distribution function, describes the probability that a real-valued random variable X with a given probability distribution will be found at a value less than or equal to x. Intuitively, it is the "area so far"...
:
-
(This follows from the symmetry of the distribution of X and the symmetry of the condition that |X| < c.)
Observe that the sum X + Y is nowhere near being normally distributed, since it has a substantial probability (about 0.88) of it being equal to 0, whereas the normal distribution, being a continuous distribution, has no discrete part, i.e., does not concentrate more than zero probability at any single point. Consequently X and Y are not jointly normally distributed, even though they are separately normally distributed. -
-
-
-
