Cochran's theorem
Encyclopedia
In statistics
, Cochran's theorem, devised by William G. Cochran, is a theorem
used in to justify results relating to the probability distribution
s of statistics that are used in the analysis of variance
.
standard normally distributed random variable
s, and an identity of the form
can be written, where each Qi is a sum of squares of linear combinations of the Us. Further suppose that
where ri is the rank
of Qi. Cochran's theorem states that the Qi are independent, and each Qi has a chi-squared distribution with ri degrees of freedom
.
Here the rank of Qi should be interpreted as meaning the rank of the matrix B(i), with elements Bj,k(i), in the representation of Qi as a quadratic form
:
Less formally, it is the number of linear combinations included in the sum of squares defining Qi, provided that these linear combinations are linearly independent.
then
is standard normal for each i. It is possible to write
(here, summation is from 1 to n, that is over the observations). To see this identity, multiply throughout by
and note that
and expand to give
The third term is zero because it is equal to a constant times
and the second term has just n identical terms added together. Thus
and hence
Now the rank of Q2 is just 1 (it is the square of just one linear combination of the standard normal variables). The rank of Q1 can be shown to be n − 1, and thus the conditions for Cochran's theorem are met.
Cochran's theorem then states that Q1 and Q2 are independent, with chi-squared distributions with n − 1 and 1 degree of freedom respectively. This shows that the sample mean
and sample variance are independent. This can also be shown by Basu's theorem
, and in fact this property characterizes the normal distribution – for no other distribution are the sample mean and sample variance independent.
Both these random variables are proportional to the true but unknown variance σ2. Thus their ratio is does not depend on σ2 and, because they are statistically independent, the distribution of their ratio is given by
where F1,n − 1 is the F-distribution with 1 and n − 1 degrees of freedom (see also Student's t-distribution). The final step here is effectively the defintion of a random variable having the F-distribution.
estimator of the variance of a normal distribution
Cochran's theorem shows that
and the properties of the chi-squared distribution show that the expected value of
is σ2(n − 1)/n.
is a standard multivariate normal random vector (here 
denotes the n-by-n identity matrix
), and if
are all n-by-n symmetric matrices with 
. Then, on defining 
, any one of the following conditions implies the other two:
Statistics
Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....
, Cochran's theorem, devised by William G. Cochran, is a theorem
Theorem
In mathematics, a theorem is a statement that has been proven on the basis of previously established statements, such as other theorems, and previously accepted statements, such as axioms...
used in to justify results relating to the probability distribution
Probability distribution
In probability theory, a probability mass, probability density, or probability distribution is a function that describes the probability of a random variable taking certain values....
s of statistics that are used in the analysis of variance
Analysis of variance
In statistics, analysis of variance is a collection of statistical models, and their associated procedures, in which the observed variance in a particular variable is partitioned into components attributable to different sources of variation...
.
Statement
Suppose U1, ..., Un are independentStatistical independence
In probability theory, to say that two events are independent intuitively means that the occurrence of one event makes it neither more nor less probable that the other occurs...
standard normally distributed random variable
Random variable
In probability and statistics, a random variable or stochastic variable is, roughly speaking, a variable whose value results from a measurement on some type of random process. Formally, it is a function from a probability space, typically to the real numbers, which is measurable functionmeasurable...
s, and an identity of the form
can be written, where each Qi is a sum of squares of linear combinations of the Us. Further suppose that
where ri is the rank
Rank (linear algebra)
The column rank of a matrix A is the maximum number of linearly independent column vectors of A. The row rank of a matrix A is the maximum number of linearly independent row vectors of A...
of Qi. Cochran's theorem states that the Qi are independent, and each Qi has a chi-squared distribution with ri degrees of freedom
Degrees of freedom (statistics)
In statistics, the number of degrees of freedom is the number of values in the final calculation of a statistic that are free to vary.Estimates of statistical parameters can be based upon different amounts of information or data. The number of independent pieces of information that go into the...
.
Here the rank of Qi should be interpreted as meaning the rank of the matrix B(i), with elements Bj,k(i), in the representation of Qi as a quadratic form
Quadratic form
In mathematics, a quadratic form is a homogeneous polynomial of degree two in a number of variables. For example,4x^2 + 2xy - 3y^2\,\!is a quadratic form in the variables x and y....
:
Less formally, it is the number of linear combinations included in the sum of squares defining Qi, provided that these linear combinations are linearly independent.
Sample mean and sample variance
If X1, ..., Xn are independent normally distributed random variables with mean μ and standard deviation σthen
is standard normal for each i. It is possible to write
(here, summation is from 1 to n, that is over the observations). To see this identity, multiply throughout by
and note thatand expand to give
The third term is zero because it is equal to a constant times
and the second term has just n identical terms added together. Thus
and hence
Now the rank of Q2 is just 1 (it is the square of just one linear combination of the standard normal variables). The rank of Q1 can be shown to be n − 1, and thus the conditions for Cochran's theorem are met.
Cochran's theorem then states that Q1 and Q2 are independent, with chi-squared distributions with n − 1 and 1 degree of freedom respectively. This shows that the sample mean
Arithmetic mean
In mathematics and statistics, the arithmetic mean, often referred to as simply the mean or average when the context is clear, is a method to derive the central tendency of a sample space...
and sample variance are independent. This can also be shown by Basu's theorem
Basu's theorem
In statistics, Basu's theorem states that any boundedly complete sufficient statistic is independent of any ancillary statistic. This is a 1955 result of Debabrata Basu....
, and in fact this property characterizes the normal distribution – for no other distribution are the sample mean and sample variance independent.
Distributions
The result for the distributions is written symbolically asBoth these random variables are proportional to the true but unknown variance σ2. Thus their ratio is does not depend on σ2 and, because they are statistically independent, the distribution of their ratio is given by
where F1,n − 1 is the F-distribution with 1 and n − 1 degrees of freedom (see also Student's t-distribution). The final step here is effectively the defintion of a random variable having the F-distribution.
Estimation of variance
To estimate the variance σ2, one estimator that is sometimes used is the maximum likelihoodMaximum likelihood
In statistics, maximum-likelihood estimation is a method of estimating the parameters of a statistical model. When applied to a data set and given a statistical model, maximum-likelihood estimation provides estimates for the model's parameters....
estimator of the variance of a normal distribution
Cochran's theorem shows that
and the properties of the chi-squared distribution show that the expected value of
is σ2(n − 1)/n.Alternative formulation
The following version is often seen when considering linear regression. Suppose that is a standard multivariate normal random vector (here denotes the n-by-n identity matrixIdentity matrix
In linear algebra, the identity matrix or unit matrix of size n is the n×n square matrix with ones on the main diagonal and zeros elsewhere. It is denoted by In, or simply by I if the size is immaterial or can be trivially determined by the context...
), and if
are all n-by-n symmetric matrices with . Then, on defining , any one of the following conditions implies the other two:-
-
(thus the 
are positive semidefinitePositive semidefiniteIn mathematics, positive semidefinite may refer to:* positive-semidefinite matrix* positive-semidefinite function...
) -
is independent of 
for 
See also
- Cramér's theoremCramér's theoremIn mathematical statistics, Cramér's theorem is one of several theorems of Harald Cramér, a Swedish statistician and probabilist.- Normal random variables :...
, on decomposing normal distribution - Infinite divisibility (probability)Infinite divisibility (probability)The concepts of infinite divisibility and the decomposition of distributions arise in probability and statistics in relation to seeking families of probability distributions that might be a natural choice in certain applications, in the same way that the normal distribution is...