Bernstein–von Mises theorem
Encyclopedia
In Bayesian inference
, the Bernstein–von Mises theorem provides the basis for the important result that the posterior distribution for unknown quantities in any problem is effectively independent of the prior distribution (assuming it obeys Cromwell's rule
) once the amount of information supplied by a sample of data is large enough.
The theorem is named after Richard von Mises and S. N. Bernstein even though the first proper proof was given by Joseph Leo Doob
in 1949 for random variables with finite probability space
. Later Lucien Le Cam
, his PhD student Lorraine Schwarz, David A. Freedman
and Persi Diaconis
extended the proof under more general assumptions. A remarkable result was found by Freedman in 1965: the Bernstein-von Mises theorem does not hold almost surely
if the random variable has an infinite countable probability space
.
The statistician A. W. F. Edwards
has remarked, "It is sometimes said, in defence of the Bayesian concept, that the choice of prior distribution is unimportant in practice, because it hardly influences the posterior distribution at all when there are moderate amounts of data. The less said about this 'defence' the better."
Bayesian inference
In statistics, Bayesian inference is a method of statistical inference. It is often used in science and engineering to determine model parameters, make predictions about unknown variables, and to perform model selection...
, the Bernstein–von Mises theorem provides the basis for the important result that the posterior distribution for unknown quantities in any problem is effectively independent of the prior distribution (assuming it obeys Cromwell's rule
Cromwell's rule
Cromwell's rule, named by statistician Dennis Lindley, states that one should avoid using prior probabilities of 0 or 1, except when applied to statements that are logically true or false...
) once the amount of information supplied by a sample of data is large enough.
The theorem is named after Richard von Mises and S. N. Bernstein even though the first proper proof was given by Joseph Leo Doob
Joseph Leo Doob
Joseph Leo Doob was an American mathematician, specializing in analysis and probability theory.The theory of martingales was developed by Doob.-Early life and education:...
in 1949 for random variables with finite probability space
Probability space
In probability theory, a probability space or a probability triple is a mathematical construct that models a real-world process consisting of states that occur randomly. A probability space is constructed with a specific kind of situation or experiment in mind...
. Later Lucien Le Cam
Lucien le Cam
Lucien Marie Le Cam was a mathematician and statistician. He obtained a Ph.D. in 1952 at the University of California, Berkeley, was appointed Assistant Professor in 1953 and continued working there beyond his retirement in 1991 until his death.Le Cam was the major figure during the period 1950...
, his PhD student Lorraine Schwarz, David A. Freedman
David A. Freedman (statistician)
David A. Freedman was Professor of Statistics at the University of California, Berkeley. He was a distinguished mathematical statistician whose wide-ranging research included the analysis of martingale inequalities, Markov processes, de Finetti's theorem, consistency of Bayes estimators, sampling,...
and Persi Diaconis
Persi Diaconis
Persi Warren Diaconis is an American mathematician and former professional magician. He is the Mary V. Sunseri Professor of Statistics and Mathematics at Stanford University....
extended the proof under more general assumptions. A remarkable result was found by Freedman in 1965: the Bernstein-von Mises theorem does not hold almost surely
Almost surely
In probability theory, one says that an event happens almost surely if it happens with probability one. The concept is analogous to the concept of "almost everywhere" in measure theory...
if the random variable has an infinite countable probability space
Probability space
In probability theory, a probability space or a probability triple is a mathematical construct that models a real-world process consisting of states that occur randomly. A probability space is constructed with a specific kind of situation or experiment in mind...
.
The statistician A. W. F. Edwards
A. W. F. Edwards
Anthony William Fairbank Edwards is a British statistician, geneticist, and evolutionary biologist, sometimes called Fisher's Edwards. He is a Life Fellow of Gonville and Caius College and retired Professor of Biometry at the University of Cambridge, and holds both the ScD and LittD degrees. A...
has remarked, "It is sometimes said, in defence of the Bayesian concept, that the choice of prior distribution is unimportant in practice, because it hardly influences the posterior distribution at all when there are moderate amounts of data. The less said about this 'defence' the better."