Stationary process
Encyclopedia
In the mathematical sciences
, a stationary process (or strict(ly) stationary process or strong(ly) stationary process) is a stochastic process
whose joint probability distribution does not change when shifted in time or space. Consequently, parameters such as the mean
and variance
, if they exist, also do not change over time or position.
Stationarity is used as a tool in time series analysis, where the raw data are often transformed to become stationary; for example, economic
data are often seasonal and/or dependent on a non-stationary price level. An important type of non-stationary process that does not include a trend-like behavior is the cyclostationary process.
Note that a "stationary process" is not the same thing as a "process with a stationary distribution
". Indeed there are further possibilities for confusion with the use of "stationary" in the context of stochastic processes; for example a "time-homogeneous" Markov chain
(which condition is sometimes called by the name "stationary Markov chain") is sometimes said to have "stationary transition probabilities".
be a stochastic process
and let
represent the cumulative distribution function
of the joint distribution
of
at times 
. Then, 
is said to be stationary if, for all 
, for all 
, and for all 
,
Since
does not affect 
, 
is not a function of time.
is stationary. The sound of a cymbal
clashing, if hit only once, is not stationary because the acoustic power of the clash (and hence its variance) diminishes with time. However, it would be possible to invent a stochastic process describing when the cymbal is hit, such that the overall response would form a stationary process.
An example of a discrete-time stationary process where the sample space is also discrete (so that the random variable may take one of N possible values) is a Bernoulli scheme
. Other examples of a discrete-time stationary process with continuous sample space include some autoregressive and moving average
processes which are both subsets of the autoregressive moving average model
. Models with a non-trivial autoregressive component may be either stationary or non-stationary, depending on the parameter values, and important non-stationary special cases are where unit root
s exist in the model.
Let Y be any scalar random variable
, and define a time-series { Xt }, by
.
Then { Xt } is a stationary time series, for which realisations consist of a series of constant values, with a different constant value for each realisation. A law of large numbers
does not apply on this case, as the limiting value of an average from a single realisation takes the random value determined by Y, rather than taking the expected value
of Y.
As a further example of a stationary process for which any single realisation has an apparently noise-free structure, let Y have a uniform distribution
on (0,2π] and define the time series { Xt } by
Then { Xt } is strictly stationary.
is known as weak–sense stationarity, wide–sense stationarity (WSS) or covariance stationarity. WSS random processes only require that 1st and 2nd moments
do not vary with respect to time. Any strictly stationary process which has a mean
and a covariance
is also WSS.
So, a continuous
-time random process x(t) which is WSS has the following restrictions on its mean function
and autocorrelation
function
The first property implies that the mean function mx(t) must be constant. The second property implies that the correlation function depends only on the difference between
and 
and only needs to be indexed by one variable rather than two variables. Thus, instead of writing,
the notation is often abbreviated and written as:
This also implies that the autocovariance
depends only on
, since
When processing WSS random signals with linear
, time-invariant (LTI
) filter
s, it is helpful to think of the correlation function as a linear operator. Since it is a circulant
operator (depends only on the difference between the two arguments), its eigenfunctions are the Fourier
complex exponentials. Additionally, since the eigenfunction
s of LTI operators are also complex exponential
s, LTI processing of WSS random signals is highly tractable—all computations can be performed in the frequency domain
. Thus, the WSS assumption is widely employed in signal processing algorithm
s.
A process is second order stationary if the first and second order density functions satisfy
for all
, 
, and 
. Such a process will be wide sense stationary if the mean and correlation functions are finite. A process can be wide sense stationary without being second order stationary.
Mathematics
Mathematics is the study of quantity, space, structure, and change. Mathematicians seek out patterns and formulate new conjectures. Mathematicians resolve the truth or falsity of conjectures by mathematical proofs, which are arguments sufficient to convince other mathematicians of their validity...
, a stationary process (or strict(ly) stationary process or strong(ly) stationary process) is a stochastic process
Stochastic process
In probability theory, a stochastic process , or sometimes random process, is the counterpart to a deterministic process...
whose joint probability distribution does not change when shifted in time or space. Consequently, parameters such as the mean
Mean
In statistics, mean has two related meanings:* the arithmetic mean .* the expected value of a random variable, which is also called the population mean....
and variance
Variance
In probability theory and statistics, the variance is a measure of how far a set of numbers is spread out. It is one of several descriptors of a probability distribution, describing how far the numbers lie from the mean . In particular, the variance is one of the moments of a distribution...
, if they exist, also do not change over time or position.
Stationarity is used as a tool in time series analysis, where the raw data are often transformed to become stationary; for example, economic
Economics
Economics is the social science that analyzes the production, distribution, and consumption of goods and services. The term economics comes from the Ancient Greek from + , hence "rules of the house"...
data are often seasonal and/or dependent on a non-stationary price level. An important type of non-stationary process that does not include a trend-like behavior is the cyclostationary process.
Note that a "stationary process" is not the same thing as a "process with a stationary distribution
Stationary distribution
Stationary distribution may refer to:* The limiting distribution in a Markov chain* The marginal distribution of a stationary process or stationary time series* The set of joint probability distributions of a stationary process or stationary time series...
". Indeed there are further possibilities for confusion with the use of "stationary" in the context of stochastic processes; for example a "time-homogeneous" Markov chain
Markov chain
A Markov chain, named after Andrey Markov, is a mathematical system that undergoes transitions from one state to another, between a finite or countable number of possible states. It is a random process characterized as memoryless: the next state depends only on the current state and not on the...
(which condition is sometimes called by the name "stationary Markov chain") is sometimes said to have "stationary transition probabilities".
Definition
Formally, let be a stochastic processStochastic process
In probability theory, a stochastic process , or sometimes random process, is the counterpart to a deterministic process...
and let
represent the cumulative distribution functionCumulative distribution function
In probability theory and statistics, the cumulative distribution function , or just distribution function, describes the probability that a real-valued random variable X with a given probability distribution will be found at a value less than or equal to x. Intuitively, it is the "area so far"...
of the joint distribution
Joint distribution
In the study of probability, given two random variables X and Y that are defined on the same probability space, the joint distribution for X and Y defines the probability of events defined in terms of both X and Y...
of
at times . Then, is said to be stationary if, for all , for all , and for all ,Since
does not affect , is not a function of time.Examples
As an example, white noiseWhite noise
White noise is a random signal with a flat power spectral density. In other words, the signal contains equal power within a fixed bandwidth at any center frequency...
is stationary. The sound of a cymbal
Cymbal
Cymbals are a common percussion instrument. Cymbals consist of thin, normally round plates of various alloys; see cymbal making for a discussion of their manufacture. The greater majority of cymbals are of indefinite pitch, although small disc-shaped cymbals based on ancient designs sound a...
clashing, if hit only once, is not stationary because the acoustic power of the clash (and hence its variance) diminishes with time. However, it would be possible to invent a stochastic process describing when the cymbal is hit, such that the overall response would form a stationary process.
An example of a discrete-time stationary process where the sample space is also discrete (so that the random variable may take one of N possible values) is a Bernoulli scheme
Bernoulli scheme
In mathematics, the Bernoulli scheme or Bernoulli shift is a generalization of the Bernoulli process to more than two possible outcomes. Bernoulli schemes are important in the study of dynamical systems, as most such systems exhibit a repellor that is the product of the Cantor set and a smooth...
. Other examples of a discrete-time stationary process with continuous sample space include some autoregressive and moving average
Moving average model
In time series analysis, the moving-average model is a common approach for modeling univariate time series models. The notation MA refers to the moving average model of order q:...
processes which are both subsets of the autoregressive moving average model
Autoregressive moving average model
In statistics and signal processing, autoregressive–moving-average models, sometimes called Box–Jenkins models after the iterative Box–Jenkins methodology usually used to estimate them, are typically applied to autocorrelated time series data.Given a time series of data Xt, the ARMA model is a...
. Models with a non-trivial autoregressive component may be either stationary or non-stationary, depending on the parameter values, and important non-stationary special cases are where unit root
Unit root
In time series models in econometrics , a unit root is a feature of processes that evolve through time that can cause problems in statistical inference if it is not adequately dealt with....
s exist in the model.
Let Y be any scalar random variable
Random variable
In probability and statistics, a random variable or stochastic variable is, roughly speaking, a variable whose value results from a measurement on some type of random process. Formally, it is a function from a probability space, typically to the real numbers, which is measurable functionmeasurable...
, and define a time-series { Xt }, by
.Then { Xt } is a stationary time series, for which realisations consist of a series of constant values, with a different constant value for each realisation. A law of large numbers
Law of large numbers
In probability theory, the law of large numbers is a theorem that describes the result of performing the same experiment a large number of times...
does not apply on this case, as the limiting value of an average from a single realisation takes the random value determined by Y, rather than taking the expected value
Expected value
In probability theory, the expected value of a random variable is the weighted average of all possible values that this random variable can take on...
of Y.
As a further example of a stationary process for which any single realisation has an apparently noise-free structure, let Y have a uniform distribution
Uniform distribution (continuous)
In probability theory and statistics, the continuous uniform distribution or rectangular distribution is a family of probability distributions such that for each member of the family, all intervals of the same length on the distribution's support are equally probable. The support is defined by...
on (0,2π] and define the time series { Xt } by
Then { Xt } is strictly stationary.
Weak or wide–sense stationarity
A weaker form of stationarity commonly employed in signal processingSignal processing
Signal processing is an area of systems engineering, electrical engineering and applied mathematics that deals with operations on or analysis of signals, in either discrete or continuous time...
is known as weak–sense stationarity, wide–sense stationarity (WSS) or covariance stationarity. WSS random processes only require that 1st and 2nd moments
Moment (mathematics)
In mathematics, a moment is, loosely speaking, a quantitative measure of the shape of a set of points. The "second moment", for example, is widely used and measures the "width" of a set of points in one dimension or in higher dimensions measures the shape of a cloud of points as it could be fit by...
do not vary with respect to time. Any strictly stationary process which has a mean
Mean
In statistics, mean has two related meanings:* the arithmetic mean .* the expected value of a random variable, which is also called the population mean....
and a covariance
Covariance
In probability theory and statistics, covariance is a measure of how much two variables change together. Variance is a special case of the covariance when the two variables are identical.- Definition :...
is also WSS.
So, a continuous
Continuous function
In mathematics, a continuous function is a function for which, intuitively, "small" changes in the input result in "small" changes in the output. Otherwise, a function is said to be "discontinuous". A continuous function with a continuous inverse function is called "bicontinuous".Continuity of...
-time random process x(t) which is WSS has the following restrictions on its mean function
and autocorrelation
Autocorrelation
Autocorrelation is the cross-correlation of a signal with itself. Informally, it is the similarity between observations as a function of the time separation between them...
function
The first property implies that the mean function mx(t) must be constant. The second property implies that the correlation function depends only on the difference between
and and only needs to be indexed by one variable rather than two variables. Thus, instead of writing,the notation is often abbreviated and written as:
This also implies that the autocovariance
Autocovariance
In statistics, given a real stochastic process X, the autocovariance is the covariance of the variable with itself, i.e. the variance of the variable against a time-shifted version of itself...
depends only on
, sinceWhen processing WSS random signals with linear
Linear
In mathematics, a linear map or function f is a function which satisfies the following two properties:* Additivity : f = f + f...
, time-invariant (LTI
LTI system theory
Linear time-invariant system theory, commonly known as LTI system theory, comes from applied mathematics and has direct applications in NMR spectroscopy, seismology, circuits, signal processing, control theory, and other technical areas. It investigates the response of a linear and time-invariant...
) filter
Filter (signal processing)
In signal processing, a filter is a device or process that removes from a signal some unwanted component or feature. Filtering is a class of signal processing, the defining feature of filters being the complete or partial suppression of some aspect of the signal...
s, it is helpful to think of the correlation function as a linear operator. Since it is a circulant
Circulant matrix
In linear algebra, a circulant matrix is a special kind of Toeplitz matrix where each row vector is rotated one element to the right relative to the preceding row vector. In numerical analysis, circulant matrices are important because they are diagonalized by a discrete Fourier transform, and hence...
operator (depends only on the difference between the two arguments), its eigenfunctions are the Fourier
Fourier series
In mathematics, a Fourier series decomposes periodic functions or periodic signals into the sum of a set of simple oscillating functions, namely sines and cosines...
complex exponentials. Additionally, since the eigenfunction
Eigenfunction
In mathematics, an eigenfunction of a linear operator, A, defined on some function space is any non-zero function f in that space that returns from the operator exactly as is, except for a multiplicative scaling factor. More precisely, one has...
s of LTI operators are also complex exponential
Exponential function
In mathematics, the exponential function is the function ex, where e is the number such that the function ex is its own derivative. The exponential function is used to model a relationship in which a constant change in the independent variable gives the same proportional change In mathematics,...
s, LTI processing of WSS random signals is highly tractable—all computations can be performed in the frequency domain
Frequency domain
In electronics, control systems engineering, and statistics, frequency domain is a term used to describe the domain for analysis of mathematical functions or signals with respect to frequency, rather than time....
. Thus, the WSS assumption is widely employed in signal processing algorithm
Algorithm
In mathematics and computer science, an algorithm is an effective method expressed as a finite list of well-defined instructions for calculating a function. Algorithms are used for calculation, data processing, and automated reasoning...
s.
Second-order stationarity
The case of second-order stationarity arises when the requirements of strict stationarity are only applied to pairs of random variables from the time-series. The definition of second order stationarity can be generalized to Nth order (for finite N) and strict stationary means stationary of all orders.A process is second order stationary if the first and second order density functions satisfy
for all
, , and . Such a process will be wide sense stationary if the mean and correlation functions are finite. A process can be wide sense stationary without being second order stationary.Other terminology
The terminology used for types of stationarity other than strict stationarity can be rather mixed. Some examples follow.- Priestley uses stationary up to order m if conditions similar to those given here for wide sense stationarity apply relating to moments up to order m. Thus wide sense stationarity would be equivalent to "stationary to order 2", which is different from the definition of second-order stationarity given here.
- Honarkhah also uses the assumption of stationarity in the context of multiple-point Geostatistics, where higher n-point statistics are assumed to be stationary in the spatial domain.