Propagation of uncertainty
Encyclopedia
In statistics
, propagation of error (or propagation of uncertainty) is the effect of variables
' uncertainties
(or errors
) on the uncertainty of a function
based on them. When the variables are the values of experimental measurements they have uncertainties due to measurement limitations
(e.g., instrument precision
) which propagate to the combination of variables in the function.
The uncertainty is usually defined by the absolute error. Uncertainties can also be defined by the relative error (Δx)/x, which is usually written as a percentage.
Most commonly the error on a quantity, Δx, is given as the standard deviation
, σ. Standard deviation is the positive square root of variance
, σ2. The value of a quantity and its error are often expressed as . If the statistical probability distribution
of the variable is known or can be assumed, it is possible to derive confidence limits to describe the region within which the true value of the variable may be found. For example, the 68% confidence limits for a one dimensional variable belonging to a normal distribution are ± one standard deviation from the value, that is, there is a 68% probability that the true value lies in the region . Note that the percentage 68% is approximate as the exact percentage that corresponds to one standard deviation is slightly larger than this.
If the variables are correlated, then covariance
must be taken into account.
be a set of m functions which are linear combinations of 
variables 
with combination coefficients 
.
or 
and let the variance-covariance matrix on x be denoted by
.
Then, the variance-covariance matrix
, of f is given by
.
This is the most general expression for the propagation of error from one set of variables onto another. When the errors on x are un-correlated the general expression simplifies to
Note that even though the errors on x may be un-correlated, their errors on f are always correlated.
The general expressions for a single function, f, are a little simpler.
Each covariance term,
can be expressed in terms of the correlation coefficient
by 
, so that an alternative expression for the variance of f is
In the case that the variables x are uncorrelated this simplifies further to
expansion, though in some cases, exact formulas can be derived that do not depend on the expansion.
where
denotes the partial derivative
of fk with respect to the i-th variable. Or in matrix notation,
where J is the Jacobian matrix. Since f0k is a constant it does not contribute to the error on f. Therefore, the propagation of error follows the linear case, above, but replacing the linear coefficients, Aik and Ajk by the partial derivatives,
and 
. In matrix notation,
.http://vision.ucsd.edu/sites/default/files/ochoa06.pdf
That is, the Jacobian of the function is used to transform the rows and columns of the covariance of the argument.
hence:
In the particular case that
, 
. Then
or
on account of using a truncated series expansion. The extent of this bias depends on the nature of the function. For example, the bias on the error calculated for log x increases as x increases since the expansion to 1+x is a good approximation only when x is small.
In data-fitting applications it is often possible to assume that measurements errors are uncorrelated. Nevertheless, parameters derived from these measurements, such as least-squares parameters, will be correlated. For example, in linear regression
, the errors on slope and intercept will be correlated and the term with the correlation coefficient, ρ, can make a significant contribution to the error on a calculated value.
In the special case of the inverse
where 
, the distribution is a Cauchy distribution
and there is no definable variance. For such ratio distribution
s, there can be defined probabilities for intervals which can be defined either by Monte Carlo simulation, or, in some cases, by using the Geary-Hinkley transformation.
, with standard deviations 
, correlation coefficient 
and precisely-known real-valued constants 
.
For uncorrelated variables the covariance terms are zero.
Expressions for more complicated functions can be derived by combining simpler functions. For example, repeated multiplication, assuming no correlation gives,
Define
where
is the absolute uncertainty on our measurement of 
.
The partial derivative of
with respect to 
is
Therefore, our propagated uncertainty is
where
is the absolute propagated uncertainty.
in which one measures current, I, and voltage
, V, on a resistor
in order to determine the resistance
, R, using Ohm's law
,
Given the measured variables with uncertainties, I±ΔI and V±ΔV, the uncertainty in the computed quantity, ΔR is
Statistics
Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....
, propagation of error (or propagation of uncertainty) is the effect of variables
Variable (mathematics)
In mathematics, a variable is a value that may change within the scope of a given problem or set of operations. In contrast, a constant is a value that remains unchanged, though often unknown or undetermined. The concepts of constants and variables are fundamental to many areas of mathematics and...
' uncertainties
Uncertainty
Uncertainty is a term used in subtly different ways in a number of fields, including physics, philosophy, statistics, economics, finance, insurance, psychology, sociology, engineering, and information science...
(or errors
Errors and residuals in statistics
In statistics and optimization, statistical errors and residuals are two closely related and easily confused measures of the deviation of a sample from its "theoretical value"...
) on the uncertainty of a function
Function (mathematics)
In mathematics, a function associates one quantity, the argument of the function, also known as the input, with another quantity, the value of the function, also known as the output. A function assigns exactly one output to each input. The argument and the value may be real numbers, but they can...
based on them. When the variables are the values of experimental measurements they have uncertainties due to measurement limitations
Observational error
Observational error is the difference between a measured value of quantity and its true value. In statistics, an error is not a "mistake". Variability is an inherent part of things being measured and of the measurement process.-Science and experiments:...
(e.g., instrument precision
Accuracy and precision
In the fields of science, engineering, industry and statistics, the accuracy of a measurement system is the degree of closeness of measurements of a quantity to that quantity's actual value. The precision of a measurement system, also called reproducibility or repeatability, is the degree to which...
) which propagate to the combination of variables in the function.
The uncertainty is usually defined by the absolute error. Uncertainties can also be defined by the relative error (Δx)/x, which is usually written as a percentage.
Most commonly the error on a quantity, Δx, is given as the standard deviation
Standard deviation
Standard deviation is a widely used measure of variability or diversity used in statistics and probability theory. It shows how much variation or "dispersion" there is from the average...
, σ. Standard deviation is the positive square root of variance
Variance
In probability theory and statistics, the variance is a measure of how far a set of numbers is spread out. It is one of several descriptors of a probability distribution, describing how far the numbers lie from the mean . In particular, the variance is one of the moments of a distribution...
, σ2. The value of a quantity and its error are often expressed as . If the statistical probability distribution
Probability distribution
In probability theory, a probability mass, probability density, or probability distribution is a function that describes the probability of a random variable taking certain values....
of the variable is known or can be assumed, it is possible to derive confidence limits to describe the region within which the true value of the variable may be found. For example, the 68% confidence limits for a one dimensional variable belonging to a normal distribution are ± one standard deviation from the value, that is, there is a 68% probability that the true value lies in the region . Note that the percentage 68% is approximate as the exact percentage that corresponds to one standard deviation is slightly larger than this.
If the variables are correlated, then covariance
Covariance
In probability theory and statistics, covariance is a measure of how much two variables change together. Variance is a special case of the covariance when the two variables are identical.- Definition :...
must be taken into account.
Linear combinations
Let be a set of m functions which are linear combinations of variables with combination coefficients . or and let the variance-covariance matrix on x be denoted by
.Then, the variance-covariance matrix
, of f is given by.This is the most general expression for the propagation of error from one set of variables onto another. When the errors on x are un-correlated the general expression simplifies to
Note that even though the errors on x may be un-correlated, their errors on f are always correlated.
The general expressions for a single function, f, are a little simpler.
Each covariance term,
can be expressed in terms of the correlation coefficientPearson product-moment correlation coefficient
In statistics, the Pearson product-moment correlation coefficient is a measure of the correlation between two variables X and Y, giving a value between +1 and −1 inclusive...
by , so that an alternative expression for the variance of f isIn the case that the variables x are uncorrelated this simplifies further to
Non-linear combinations
When f is a set of non-linear combination of the variables x, it must usually be linearized by approximation to a first-order Taylor seriesTaylor series
In mathematics, a Taylor series is a representation of a function as an infinite sum of terms that are calculated from the values of the function's derivatives at a single point....
expansion, though in some cases, exact formulas can be derived that do not depend on the expansion.
where
denotes the partial derivativePartial derivative
In mathematics, a partial derivative of a function of several variables is its derivative with respect to one of those variables, with the others held constant...
of fk with respect to the i-th variable. Or in matrix notation,
where J is the Jacobian matrix. Since f0k is a constant it does not contribute to the error on f. Therefore, the propagation of error follows the linear case, above, but replacing the linear coefficients, Aik and Ajk by the partial derivatives,
and . In matrix notation,.http://vision.ucsd.edu/sites/default/files/ochoa06.pdfThat is, the Jacobian of the function is used to transform the rows and columns of the covariance of the argument.
Example
Any non-linear function, f(a,b), of two variables, a and b, can be expanded ashence:
In the particular case that
, . Thenor
Caveats and warnings
Error estimates for non-linear functions are biasedBias of an estimator
In statistics, bias of an estimator is the difference between this estimator's expected value and the true value of the parameter being estimated. An estimator or decision rule with zero bias is called unbiased. Otherwise the estimator is said to be biased.In ordinary English, the term bias is...
on account of using a truncated series expansion. The extent of this bias depends on the nature of the function. For example, the bias on the error calculated for log x increases as x increases since the expansion to 1+x is a good approximation only when x is small.
In data-fitting applications it is often possible to assume that measurements errors are uncorrelated. Nevertheless, parameters derived from these measurements, such as least-squares parameters, will be correlated. For example, in linear regression
Linear regression
In statistics, linear regression is an approach to modeling the relationship between a scalar variable y and one or more explanatory variables denoted X. The case of one explanatory variable is called simple regression...
, the errors on slope and intercept will be correlated and the term with the correlation coefficient, ρ, can make a significant contribution to the error on a calculated value.
In the special case of the inverse
where , the distribution is a Cauchy distributionCauchy distribution
The Cauchy–Lorentz distribution, named after Augustin Cauchy and Hendrik Lorentz, is a continuous probability distribution. As a probability distribution, it is known as the Cauchy distribution, while among physicists, it is known as the Lorentz distribution, Lorentz function, or Breit–Wigner...
and there is no definable variance. For such ratio distribution
Ratio distribution
A ratio distribution is a probability distribution constructed as the distribution of the ratio of random variables having two other known distributions....
s, there can be defined probabilities for intervals which can be defined either by Monte Carlo simulation, or, in some cases, by using the Geary-Hinkley transformation.
Example formulas
This table shows the variances of simple functions of the real variables, with standard deviations , correlation coefficient and precisely-known real-valued constants .| Function | | Variance |
|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
For uncorrelated variables the covariance terms are zero.
Expressions for more complicated functions can be derived by combining simpler functions. For example, repeated multiplication, assuming no correlation gives,
Partial derivatives
Given
| Absolute Error | | Variance |
|---|---|
 |
Example calculation: Inverse tangent function
We can calculate the uncertainty propagation for the inverse tangent function as an example of using partial derivatives to propagate error.Define
where
is the absolute uncertainty on our measurement of .The partial derivative of
with respect to isTherefore, our propagated uncertainty is
where
is the absolute propagated uncertainty.Example application: Resistance measurement
A practical application is an experimentExperiment
An experiment is a methodical procedure carried out with the goal of verifying, falsifying, or establishing the validity of a hypothesis. Experiments vary greatly in their goal and scale, but always rely on repeatable procedure and logical analysis of the results...
in which one measures current, I, and voltage
Voltage
Voltage, otherwise known as electrical potential difference or electric tension is the difference in electric potential between two points — or the difference in electric potential energy per unit charge between two points...
, V, on a resistor
Resistor
A linear resistor is a linear, passive two-terminal electrical component that implements electrical resistance as a circuit element.The current through a resistor is in direct proportion to the voltage across the resistor's terminals. Thus, the ratio of the voltage applied across a resistor's...
in order to determine the resistance
Electrical resistance
The electrical resistance of an electrical element is the opposition to the passage of an electric current through that element; the inverse quantity is electrical conductance, the ease at which an electric current passes. Electrical resistance shares some conceptual parallels with the mechanical...
, R, using Ohm's law
Ohm's law
Ohm's law states that the current through a conductor between two points is directly proportional to the potential difference across the two points...
,
Given the measured variables with uncertainties, I±ΔI and V±ΔV, the uncertainty in the computed quantity, ΔR is
See also
- Experimental uncertainty analysisExperimental uncertainty analysisThe purpose of this introductory article is to discuss the experimental uncertainty analysis of a derived quantity, based on the uncertainties in the experimentally measured quantities that are used in some form of mathematical relationship to calculate that derived quantity...
- Errors and residuals in statisticsErrors and residuals in statisticsIn statistics and optimization, statistical errors and residuals are two closely related and easily confused measures of the deviation of a sample from its "theoretical value"...
- Accuracy and precisionAccuracy and precisionIn the fields of science, engineering, industry and statistics, the accuracy of a measurement system is the degree of closeness of measurements of a quantity to that quantity's actual value. The precision of a measurement system, also called reproducibility or repeatability, is the degree to which...
- Delta methodDelta methodIn statistics, the delta method is a method for deriving an approximate probability distribution for a function of an asymptotically normal statistical estimator from knowledge of the limiting variance of that estimator...
- Significance arithmeticSignificance arithmeticSignificance arithmetic is a set of rules for approximating the propagation of uncertainty in scientific or statistical calculations. These rules can be used to find the appropriate number of significant figures to use to represent the result of a calculation...
- Automatic differentiationAutomatic differentiationIn mathematics and computer algebra, automatic differentiation , sometimes alternatively called algorithmic differentiation, is a set of techniques to numerically evaluate the derivative of a function specified by a computer program...
- List of uncertainty propagation software
- Interval finite elementInterval finite elementThe interval finite element method is a finite element method that uses interval parameters. Interval FEM can be applied in situations where it is not possible to get reliable probabilistic characteristics of the structure. This is important in concrete structures, wood structures, geomechanics,...
External links
- A detailed discussion of measurements and the propagation of uncertainty explaining the benefits of using error propagation formulas and monte carlo simulations instead of simple significance arithmeticSignificance arithmeticSignificance arithmetic is a set of rules for approximating the propagation of uncertainty in scientific or statistical calculations. These rules can be used to find the appropriate number of significant figures to use to represent the result of a calculation...
. - Uncertainties and Error Propagation, Vern Lindberg's Guide to Uncertainties and Error Propagation.
- GUM, Guide to the Expression of Uncertainty in Measurement
- EPFL An Introduction to Error Propagation, Derivation, Meaning and Examples of Cy = Fx Cx Fx'
- uncertainties package, a program/library for transparently performing calculations with uncertainties (and error correlations).