Bayesian experimental design
Encyclopedia
Bayesian experimental design provides a general probability-theoretical framework from which other theories on experimental design
can be derived. It is based on Bayesian inference
to interpret the observations/data acquired during the experiment. This allows accounting for both any prior knowledge on the parameters to be determined as well as uncertainties in observations.
The theory of Bayesian experimental design is to a certain extent based on the theory for making optimal decisions under uncertainty
. The aim when designing an experiment is to maximize the expected utility of the experiment outcome. The utility is most commonly defined in terms of a measure of the accuracy of the information provided by the experiment (e.g. the Shannon information or the negative variance
), but may also involve factors such as the financial cost of performing the experiment. What will be the optimal experiment design depends on the particular utility criterion chosen.
(PDF) is homogeneous and observational errors are normally distributed, the theory simplifies to the classical optimal experimental design theory
.
Given a vector
of parameters to determine, a prior PDF
over those parameters and a PDF 
for making observation 
, given parameter values 
and an experiment design 
, the posterior PDF can be calculated using Bayes' theorem
where
is the marginal probability density in observation space
The expected utility of an experiment with design
can then be defined
where
is some real-valued functional of the posterior PDF
after making observation 
using an experiment design 
.
noted that the expected utility will then be coordinate-independent and can be written in two forms
of which the latter can be evaluated without the need for evaluating individual posterior PDFs
for all possible observations 
. Worth noting is that the first term on the second equation line will not depend on the design 
, as long as the observational uncertainty doesn't. On the other hand, the integral of 
in the first form is constant for all 
, so if the goal is to choose the design with the highest utility, the term need not be computed at all. Several authors have considered numerical techniques for evaluating and optimizing this criterion, e.g. and .
Design of experiments
In general usage, design of experiments or experimental design is the design of any information-gathering exercises where variation is present, whether under the full control of the experimenter or not. However, in statistics, these terms are usually used for controlled experiments...
can be derived. It is based on Bayesian inference
Bayesian inference
In statistics, Bayesian inference is a method of statistical inference. It is often used in science and engineering to determine model parameters, make predictions about unknown variables, and to perform model selection...
to interpret the observations/data acquired during the experiment. This allows accounting for both any prior knowledge on the parameters to be determined as well as uncertainties in observations.
The theory of Bayesian experimental design is to a certain extent based on the theory for making optimal decisions under uncertainty
Optimal decision
An optimal decision is a decision such that no other available decision options will lead to a better outcome. It is an important concept in decision theory. In order to compare the different decision outcomes, one commonly assigns a relative utility to each of them...
. The aim when designing an experiment is to maximize the expected utility of the experiment outcome. The utility is most commonly defined in terms of a measure of the accuracy of the information provided by the experiment (e.g. the Shannon information or the negative variance
Variance
In probability theory and statistics, the variance is a measure of how far a set of numbers is spread out. It is one of several descriptors of a probability distribution, describing how far the numbers lie from the mean . In particular, the variance is one of the moments of a distribution...
), but may also involve factors such as the financial cost of performing the experiment. What will be the optimal experiment design depends on the particular utility criterion chosen.
Linear theory
If the model is linear, the prior probability density functionProbability density function
In probability theory, a probability density function , or density of a continuous random variable is a function that describes the relative likelihood for this random variable to occur at a given point. The probability for the random variable to fall within a particular region is given by the...
(PDF) is homogeneous and observational errors are normally distributed, the theory simplifies to the classical optimal experimental design theory
Optimal design
Optimal designs are a class of experimental designs that are optimal with respect to some statistical criterion.In the design of experiments for estimating statistical models, optimal designs allow parameters to be estimated without bias and with minimum-variance...
.
Approximate normality
In numerous publications on Bayesian experimental design, it is (often implicitly) assumed that all posterior PDFs will be approximately normal. This allows for the expected utility to be calculated using linear theory, averaging over the space of model parameters, an approach reviewed in . Caution must however be taken when applying this method, since approximate normality of all possible posteriors is difficult to verify, even in cases of normal observational errors and uniform prior PDF.Mathematical formulation
|
, given parameter values 
and design 
with design 
Given a vector
of parameters to determine, a prior PDFPrior probability
In Bayesian statistical inference, a prior probability distribution, often called simply the prior, of an uncertain quantity p is the probability distribution that would express one's uncertainty about p before the "data"...
over those parameters and a PDF for making observation , given parameter values and an experiment design , the posterior PDF can be calculated using Bayes' theoremBayes' theorem
In probability theory and applications, Bayes' theorem relates the conditional probabilities P and P. It is commonly used in science and engineering. The theorem is named for Thomas Bayes ....
where
is the marginal probability density in observation spaceThe expected utility of an experiment with design
can then be definedwhere
is some real-valued functional of the posterior PDFPosterior probability
In Bayesian statistics, the posterior probability of a random event or an uncertain proposition is the conditional probability that is assigned after the relevant evidence is taken into account...
after making observation using an experiment design .Gain in Shannon information as utility
If the utility is defined as the prior-posterior gain in Shannon informationDifferential entropy
Differential entropy is a concept in information theory that extends the idea of entropy, a measure of average surprisal of a random variable, to continuous probability distributions.-Definition:...
noted that the expected utility will then be coordinate-independent and can be written in two forms
of which the latter can be evaluated without the need for evaluating individual posterior PDFs
for all possible observations . Worth noting is that the first term on the second equation line will not depend on the design , as long as the observational uncertainty doesn't. On the other hand, the integral of in the first form is constant for all , so if the goal is to choose the design with the highest utility, the term need not be computed at all. Several authors have considered numerical techniques for evaluating and optimizing this criterion, e.g. and .See also
- Optimal DesignsOptimal designOptimal designs are a class of experimental designs that are optimal with respect to some statistical criterion.In the design of experiments for estimating statistical models, optimal designs allow parameters to be estimated without bias and with minimum-variance...
- Active LearningActive learning (machine learning)Active learning is a form of supervised machine learning in which the learning algorithm is able to interactively query the user to obtain the desired outputs at new data points. In statistics literature it is sometimes also called optimal experimental design.There are situations in which...