Exponential smoothing
Encyclopedia
Exponential smoothing is a technique that can be applied to time series
data, either to produce smoothed data for presentation, or to make forecasts. The time series data themselves are a sequence of observations. The observed phenomenon may be an essentially random process
, or it may be an orderly, but noisy
, process. Whereas in the simple moving average the past observations are weighted equally, exponential smoothing assigns exponentially decreasing weights over time.
Exponential smoothing is commonly applied to financial market and economic data, but it can be used with any discrete set of repeated measurements. The raw data sequence is often represented by {xt}, and the output of the exponential smoothing algorithm is commonly written as {st}, which may be regarded as a best estimate of what the next value of x will be. When the sequence of observations begins at time t = 0, the simplest form of exponential smoothing is given by the formulas
where α is the smoothing factor, and 0 < α < 1.
where the choice of an integer k > 1 is arbitrary. A small value of k will have less of a smoothing effect and be more responsive to recent changes in the data, while a larger k will have a greater smoothing effect, and produce a more pronounced lag in the smoothed sequence. One disadvantage of this technique is that it cannot be used on the first k −1 terms of the time series.
such that 
and then using these weights to calculate the smoothed statistics {st}:
In practice the weighting factors are often chosen to give more weight to the most recent terms in the time series and less weight to older data. Notice that this technique has the same disadvantage as the simple moving average technique (i.e., it cannot be used until at least k observations have been made), and that it entails a more complicated calculation at each step of the smoothing procedure. In addition to this disadvantage, if the data from each stage of the averaging is not available for analysis, it may be difficult if not impossible to reconstruct a changing signal accurately (because older samples may be given less weight). If the number of stages missed is known however, the weighting of values in the average can be adjusted to give equal weight to all missed samples to avoid this issue.
in 1957, although the formulation below, which is the one commonly used, is attributed to Brown and is known as "Brown's simple exponential smoothing".
The simplest form of exponential smoothing is given by the formulae:
where α is the smoothing factor, and 0 < α < 1. In other words, the smoothed statistic st is a simple weighted average of the previous observation xt-1 and the previous smoothed statistic st−1. The term smoothing factor applied to α here is something of a misnomer, as larger values of α actually reduce the level of smoothing.
In the limiting case with α = 1 the output series is just the same as the original series.
Simple exponential smoothing is easily applied, and it produces a smoothed statistic as soon as two observations are available.
Values of α close to one have less of a smoothing effect and give greater weight to recent changes in the data, while values of α closer to zero have a greater smoothing effect and are less responsive to recent changes. There is no formally correct procedure for choosing α. Sometimes the statistician's judgment is used to choose an appropriate factor. Alternatively, a statistical technique may be used to optimize the value of α. For example, the method of least squares
might be used to determine the value of α for which the sum of the quantities (sn-1 − xn-1)2 is minimized.
Unlike some other smoothing methods, this technique does not require any minimum number of observations to be made before it begins to produce results. In practice, however, a "good average" will not be achieved until several samples have been averaged together; for example, a constant signal will take approximately 3/α stages to reach 95% of the actual value. To accurately reconstruct the original signal without information loss all stages of the exponential moving average must also be available, because older samples decay in weight exponentially. This is in contrast to a simple moving average, in which some samples can be skipped without as much loss of information due to the constant weighting of samples within the average. If a known number of samples will be missed, one can adjust a weighted average for this as well, by giving equal weight to the new sample and all those to be skipped.
This simple form of exponential smoothing is also known as an exponentially weighted moving average (EWMA). Technically it can also be classified as an Autoregressive integrated moving average
(ARIMA) (0,1,1) model with no constant term.
In other words, as time passes the smoothed statistic st becomes the weighted average of a greater and greater number of the past observations xt−n, and the weights assigned to previous observations are in general proportional to the terms of the geometric progression {1, (1 − α), (1 − α)2, (1 − α)3, …}. A geometric progression
is the discrete version of an exponential function
, so this is where the name for this smoothing method originated.
in the data. In such situations, several methods were devised under the name "double exponential smoothing".
One method, sometimes referred to as "Holt-Winters double exponential smoothing" works as follows:
Again, the raw data sequence of observations is represented by {xt}, beginning at time t = 0. We use {st} to represent the smoothed value for time t, and {bt} is our best estimate of the trend at time t. The output of the algorithm is now written as Ft+m, an estimate of the value of x at time t+m, m>0 based on the raw data up to time t. Double exponential smoothing is given by the formulas
where α is the data smoothing factor, 0 < α < 1, β is the trend smoothing factor, 0 < β < 1, and b0 is taken as (xn-1 - x0)/(n - 1) for some n > 1. Note that F0 is undefined (there is no estimation for time 0), and according to the definition F1=s0+b0, which is well defined, thus further values can be evaluated.
A second method, referred to as either Brown's linear exponential smoothing (LES) or Brown's double exponential smoothing works as follows.
where at, the estimated level at time t and bt, the estimated trend at time t are:
The sequence of observations is again represented by {xt}, beginning at time t = 0. {st} represents the smoothed value of the constant part for time t. {bt} represents the sequence of best estimates of the linear trend that are superimposed on the seasonal changes. {ct} is the sequence of seasonal correction factors for time t. L is the period of time of one cycle of seasonal change. The output of the algorithm is again written as Ft+m, an estimate of the value of x at time t+m, m>0 based on the raw data up to time t. Triple exponential smoothing is given by the formulas
where α is the data smoothing factor, 0 < α < 1, β is the trend smoothing factor, 0 < β < 1, and γ is the seasonal change smoothing factor, 0 < γ < 1.
Time series
In statistics, signal processing, econometrics and mathematical finance, a time series is a sequence of data points, measured typically at successive times spaced at uniform time intervals. Examples of time series are the daily closing value of the Dow Jones index or the annual flow volume of the...
data, either to produce smoothed data for presentation, or to make forecasts. The time series data themselves are a sequence of observations. The observed phenomenon may be an essentially random process
Stochastic process
In probability theory, a stochastic process , or sometimes random process, is the counterpart to a deterministic process...
, or it may be an orderly, but noisy
Statistical noise
Statistical noise is the colloquialism for recognized amounts of unexplained variation in a sample. See errors and residuals in statistics....
, process. Whereas in the simple moving average the past observations are weighted equally, exponential smoothing assigns exponentially decreasing weights over time.
Exponential smoothing is commonly applied to financial market and economic data, but it can be used with any discrete set of repeated measurements. The raw data sequence is often represented by {xt}, and the output of the exponential smoothing algorithm is commonly written as {st}, which may be regarded as a best estimate of what the next value of x will be. When the sequence of observations begins at time t = 0, the simplest form of exponential smoothing is given by the formulas
where α is the smoothing factor, and 0 < α < 1.
The simple moving average
Intuitively, the simplest way to smooth a time series is to calculate a simple, or unweighted, moving average. The smoothed statistic st is then just the mean of the last k observations:where the choice of an integer k > 1 is arbitrary. A small value of k will have less of a smoothing effect and be more responsive to recent changes in the data, while a larger k will have a greater smoothing effect, and produce a more pronounced lag in the smoothed sequence. One disadvantage of this technique is that it cannot be used on the first k −1 terms of the time series.
The weighted moving average
A slightly more intricate method for smoothing a raw time series {xt} is to calculate a weighted moving average by first choosing a set of weighting factors such that and then using these weights to calculate the smoothed statistics {st}:
In practice the weighting factors are often chosen to give more weight to the most recent terms in the time series and less weight to older data. Notice that this technique has the same disadvantage as the simple moving average technique (i.e., it cannot be used until at least k observations have been made), and that it entails a more complicated calculation at each step of the smoothing procedure. In addition to this disadvantage, if the data from each stage of the averaging is not available for analysis, it may be difficult if not impossible to reconstruct a changing signal accurately (because older samples may be given less weight). If the number of stages missed is known however, the weighting of values in the average can be adjusted to give equal weight to all missed samples to avoid this issue.
The exponential moving average
Exponential smoothing was first suggested by Charles C. HoltCharles C. Holt
Charles C. Holt is Professor Emeritus at the Department of Management at the McCombs School of Business at the University of Texas at Austin.Holt holds BS and MS degrees from the Massachusetts Institute of Technology . He later earned a MA and a PhD from the University of Chicago.- GSIA :In his...
in 1957, although the formulation below, which is the one commonly used, is attributed to Brown and is known as "Brown's simple exponential smoothing".
The simplest form of exponential smoothing is given by the formulae:
where α is the smoothing factor, and 0 < α < 1. In other words, the smoothed statistic st is a simple weighted average of the previous observation xt-1 and the previous smoothed statistic st−1. The term smoothing factor applied to α here is something of a misnomer, as larger values of α actually reduce the level of smoothing.
In the limiting case with α = 1 the output series is just the same as the original series.
Simple exponential smoothing is easily applied, and it produces a smoothed statistic as soon as two observations are available.
Values of α close to one have less of a smoothing effect and give greater weight to recent changes in the data, while values of α closer to zero have a greater smoothing effect and are less responsive to recent changes. There is no formally correct procedure for choosing α. Sometimes the statistician's judgment is used to choose an appropriate factor. Alternatively, a statistical technique may be used to optimize the value of α. For example, the method of least squares
Least squares
The method of least squares is a standard approach to the approximate solution of overdetermined systems, i.e., sets of equations in which there are more equations than unknowns. "Least squares" means that the overall solution minimizes the sum of the squares of the errors made in solving every...
might be used to determine the value of α for which the sum of the quantities (sn-1 − xn-1)2 is minimized.
Unlike some other smoothing methods, this technique does not require any minimum number of observations to be made before it begins to produce results. In practice, however, a "good average" will not be achieved until several samples have been averaged together; for example, a constant signal will take approximately 3/α stages to reach 95% of the actual value. To accurately reconstruct the original signal without information loss all stages of the exponential moving average must also be available, because older samples decay in weight exponentially. This is in contrast to a simple moving average, in which some samples can be skipped without as much loss of information due to the constant weighting of samples within the average. If a known number of samples will be missed, one can adjust a weighted average for this as well, by giving equal weight to the new sample and all those to be skipped.
This simple form of exponential smoothing is also known as an exponentially weighted moving average (EWMA). Technically it can also be classified as an Autoregressive integrated moving average
Autoregressive integrated moving average
In statistics and econometrics, and in particular in time series analysis, an autoregressive integrated moving average model is a generalization of an autoregressive moving average model. These models are fitted to time series data either to better understand the data or to predict future points...
(ARIMA) (0,1,1) model with no constant term.
Why is it "exponential"?
By direct substitution of the defining equation for simple exponential smoothing back into itself we find thatIn other words, as time passes the smoothed statistic st becomes the weighted average of a greater and greater number of the past observations xt−n, and the weights assigned to previous observations are in general proportional to the terms of the geometric progression {1, (1 − α), (1 − α)2, (1 − α)3, …}. A geometric progression
Geometric progression
In mathematics, a geometric progression, also known as a geometric sequence, is a sequence of numbers where each term after the first is found by multiplying the previous one by a fixed non-zero number called the common ratio. For example, the sequence 2, 6, 18, 54, ... is a geometric progression...
is the discrete version of an exponential function
Exponential function
In mathematics, the exponential function is the function ex, where e is the number such that the function ex is its own derivative. The exponential function is used to model a relationship in which a constant change in the independent variable gives the same proportional change In mathematics,...
, so this is where the name for this smoothing method originated.
Comparison with moving average
Exponential smoothing and moving average are similar in that they both assume a stationary, not trending, time series, therefore lagging behind the trend if one exists. They also both have roughly the same distribution of forecast error when α = 2/(k+1). They differ in that exponential smoothing takes into account all past data, whereas moving average only takes into account k past data points. Technically speaking, they also differ in that moving average requires that the past k data points be kept, whereas exponential smoothing only needs the most recent forecast value to be kept.Double exponential smoothing
Simple exponential smoothing does not do well when there is a trendTrend estimation
Trend estimation is a statistical technique to aid interpretation of data. When a series of measurements of a process are treated as a time series, trend estimation can be used to make and justify statements about tendencies in the data...
in the data. In such situations, several methods were devised under the name "double exponential smoothing".
One method, sometimes referred to as "Holt-Winters double exponential smoothing" works as follows:
Again, the raw data sequence of observations is represented by {xt}, beginning at time t = 0. We use {st} to represent the smoothed value for time t, and {bt} is our best estimate of the trend at time t. The output of the algorithm is now written as Ft+m, an estimate of the value of x at time t+m, m>0 based on the raw data up to time t. Double exponential smoothing is given by the formulas
where α is the data smoothing factor, 0 < α < 1, β is the trend smoothing factor, 0 < β < 1, and b0 is taken as (xn-1 - x0)/(n - 1) for some n > 1. Note that F0 is undefined (there is no estimation for time 0), and according to the definition F1=s0+b0, which is well defined, thus further values can be evaluated.
A second method, referred to as either Brown's linear exponential smoothing (LES) or Brown's double exponential smoothing works as follows.
where at, the estimated level at time t and bt, the estimated trend at time t are:
Triple exponential smoothing
Triple exponential smoothing takes into account seasonal changes as well as trends. It was first suggested by Holt's student, Peter Winters, in 1960.The sequence of observations is again represented by {xt}, beginning at time t = 0. {st} represents the smoothed value of the constant part for time t. {bt} represents the sequence of best estimates of the linear trend that are superimposed on the seasonal changes. {ct} is the sequence of seasonal correction factors for time t. L is the period of time of one cycle of seasonal change. The output of the algorithm is again written as Ft+m, an estimate of the value of x at time t+m, m>0 based on the raw data up to time t. Triple exponential smoothing is given by the formulas
where α is the data smoothing factor, 0 < α < 1, β is the trend smoothing factor, 0 < β < 1, and γ is the seasonal change smoothing factor, 0 < γ < 1.
See also
- Moving average
- Autoregressive moving average modelAutoregressive moving average modelIn statistics and signal processing, autoregressive–moving-average models, sometimes called Box–Jenkins models after the iterative Box–Jenkins methodology usually used to estimate them, are typically applied to autocorrelated time series data.Given a time series of data Xt, the ARMA model is a...
(ARMA) - Autoregressive integrated moving averageAutoregressive integrated moving averageIn statistics and econometrics, and in particular in time series analysis, an autoregressive integrated moving average model is a generalization of an autoregressive moving average model. These models are fitted to time series data either to better understand the data or to predict future points...
(ARIMA) - Errors and residuals in statisticsErrors and residuals in statisticsIn statistics and optimization, statistical errors and residuals are two closely related and easily confused measures of the deviation of a sample from its "theoretical value"...
External links
- Notes for a statistics class (Decision 411) at Duke University
- Data Smoothing by Jon McLoone, The Wolfram Demonstrations Project.
- The Holt-Winters Approach to Exponential Smoothing: 50 Years Old and Going Strong by Paul Goodwin (2010) Foresight: The International Journal of Applied ForecastingForesight: The International Journal of Applied ForecastingForesight: The International Journal of Applied Forecasting is a forecasting journal published quarterly by the International Institute of Forecasters...