Generalized additive model
Encyclopedia
In statistics
, the generalized additive model (GAM) is a statistical model
developed by Trevor Hastie and Rob Tibshirani for blending properties of generalized linear model
s with additive model
s.
The model specifies a distribution (such as a normal distribution, or a binomial distribution) and a link function g relating the expected value of the distribution to the m predictor variables, and attempts to fit functions fi(xi) to satisfy:
The functions fi(xi) may be fit using parametric or non-parametric means
, thus providing the potential for better fits to data than other methods. The method hence is very general – a typical GAM might use a scatterplot smoothing function such as a locally weighted mean for f1(x1), and then use a factor model for f2(x2). By allowing nonparametric fits, well designed GAMs allow good fits to the training data with relaxed assumptions on the actual relationship, perhaps at the expense of interpretability of results.
Overfitting
can be a problem with GAMs. The number of smoothing parameters can be specified, and this number should be reasonably small, certainly well under the degrees of freedom
offered by the data. Cross-validation can be used to detect and/or reduce overfitting problems with GAMs (or other statistical methods). Other models such as GLMs
may be preferable to GAMs unless GAMs improve predictive ability substantially for the application in question.
Statistics
Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....
, the generalized additive model (GAM) is a statistical model
Statistical model
A statistical model is a formalization of relationships between variables in the form of mathematical equations. A statistical model describes how one or more random variables are related to one or more random variables. The model is statistical as the variables are not deterministically but...
developed by Trevor Hastie and Rob Tibshirani for blending properties of generalized linear model
Generalized linear model
In statistics, the generalized linear model is a flexible generalization of ordinary linear regression. The GLM generalizes linear regression by allowing the linear model to be related to the response variable via a link function and by allowing the magnitude of the variance of each measurement to...
s with additive model
Additive model
In statistics, an additive model is a nonparametric regression method. It was suggested by Jerome H. Friedman and Werner Stuetzle and is an essential part of the ACE algorithm. The AM uses a one dimensional smoother to build a restricted class of nonparametric regression models. Because of this,...
s.
The model specifies a distribution (such as a normal distribution, or a binomial distribution) and a link function g relating the expected value of the distribution to the m predictor variables, and attempts to fit functions fi(xi) to satisfy:
The functions fi(xi) may be fit using parametric or non-parametric means
Nonparametric regression
Nonparametric regression is a form of regression analysis in which the predictor does not take a predetermined form but is constructed according to information derived from the data...
, thus providing the potential for better fits to data than other methods. The method hence is very general – a typical GAM might use a scatterplot smoothing function such as a locally weighted mean for f1(x1), and then use a factor model for f2(x2). By allowing nonparametric fits, well designed GAMs allow good fits to the training data with relaxed assumptions on the actual relationship, perhaps at the expense of interpretability of results.
Overfitting
Overfitting
In statistics, overfitting occurs when a statistical model describes random error or noise instead of the underlying relationship. Overfitting generally occurs when a model is excessively complex, such as having too many parameters relative to the number of observations...
can be a problem with GAMs. The number of smoothing parameters can be specified, and this number should be reasonably small, certainly well under the degrees of freedom
Degrees of freedom (statistics)
In statistics, the number of degrees of freedom is the number of values in the final calculation of a statistic that are free to vary.Estimates of statistical parameters can be based upon different amounts of information or data. The number of independent pieces of information that go into the...
offered by the data. Cross-validation can be used to detect and/or reduce overfitting problems with GAMs (or other statistical methods). Other models such as GLMs
Generalized linear model
In statistics, the generalized linear model is a flexible generalization of ordinary linear regression. The GLM generalizes linear regression by allowing the linear model to be related to the response variable via a link function and by allowing the magnitude of the variance of each measurement to...
may be preferable to GAMs unless GAMs improve predictive ability substantially for the application in question.
See also
- Additive modelAdditive modelIn statistics, an additive model is a nonparametric regression method. It was suggested by Jerome H. Friedman and Werner Stuetzle and is an essential part of the ACE algorithm. The AM uses a one dimensional smoother to build a restricted class of nonparametric regression models. Because of this,...
- Generalized additive model for location, scale, and shape (GAMLSS)
- Backfitting algorithmBackfitting algorithmIn statistics, the backfitting algorithm is a simple iterative procedure used to fit a generalized additive model. It was introduced in 1985 by Leo Breiman and Jerome Friedman along with generalized additive models...