The rich get richer (statistics)
Encyclopedia
In probability
and statistics
, the phrase "the rich get richer" is used to describe the self-reinforcing behavior of certain probability distribution
s and stochastic process
es, such as the Dirichlet process
and Chinese restaurant process. Note that this behavior is seen in the context of many, perhaps most, distributions, when considering a sequence of independent identically distributed observations drawn from a probability distribution with unknown parameter
and examining the conditional distribution
of one observation given all previous ones. For example, if an observation is drawn from a Gaussian distribution with unknown mean
(with uncertainty expressed by a prior distribution over the mean), then the posterior distribution over the mean will be shifted towards the observation, and the next observation will have a higher probability of being similar to the previous observation than it was under the prior distribution. In this case, regions that are "rich" in the sense of containing a large fraction of the observations are quite likely to get richer, whereas poor regions are less likely to get richer.
However, "the rich get richer" is often used particularly of the Chinese restaurant process (CRP), a stochastic process
closely related to the Dirichlet process
. In this model, the probability of an observation taking on a specific value is directly proportional to the number of times that value has already been seen. (In the colorful terminology of the CRP, the probability of a customer sitting at a particular infinitely large table in an infinitely large Chinese restaurant is directly proportional to the number of customers already seated at the table.) Note that in order for this model to work tractably, there are also a fixed number of pseudo-observations assumed, and with probability proportional to the number of pseudo-observations, an observation is given a new value that has never been seen before.
Probability
Probability is ordinarily used to describe an attitude of mind towards some proposition of whose truth we arenot certain. The proposition of interest is usually of the form "Will a specific event occur?" The attitude of mind is of the form "How certain are we that the event will occur?" The...
and statistics
Statistics
Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....
, the phrase "the rich get richer" is used to describe the self-reinforcing behavior of certain probability distribution
Probability distribution
In probability theory, a probability mass, probability density, or probability distribution is a function that describes the probability of a random variable taking certain values....
s and stochastic process
Stochastic process
In probability theory, a stochastic process , or sometimes random process, is the counterpart to a deterministic process...
es, such as the Dirichlet process
Dirichlet process
In probability theory, a Dirichlet process is a stochastic process that can be thought of as a probability distribution whose domain is itself a random distribution...
and Chinese restaurant process. Note that this behavior is seen in the context of many, perhaps most, distributions, when considering a sequence of independent identically distributed observations drawn from a probability distribution with unknown parameter
Parameter
Parameter from Ancient Greek παρά also “para” meaning “beside, subsidiary” and μέτρον also “metron” meaning “measure”, can be interpreted in mathematics, logic, linguistics, environmental science and other disciplines....
and examining the conditional distribution
Conditional distribution
Given two jointly distributed random variables X and Y, the conditional probability distribution of Y given X is the probability distribution of Y when X is known to be a particular value...
of one observation given all previous ones. For example, if an observation is drawn from a Gaussian distribution with unknown mean
Mean
In statistics, mean has two related meanings:* the arithmetic mean .* the expected value of a random variable, which is also called the population mean....
(with uncertainty expressed by a prior distribution over the mean), then the posterior distribution over the mean will be shifted towards the observation, and the next observation will have a higher probability of being similar to the previous observation than it was under the prior distribution. In this case, regions that are "rich" in the sense of containing a large fraction of the observations are quite likely to get richer, whereas poor regions are less likely to get richer.
However, "the rich get richer" is often used particularly of the Chinese restaurant process (CRP), a stochastic process
Stochastic process
In probability theory, a stochastic process , or sometimes random process, is the counterpart to a deterministic process...
closely related to the Dirichlet process
Dirichlet process
In probability theory, a Dirichlet process is a stochastic process that can be thought of as a probability distribution whose domain is itself a random distribution...
. In this model, the probability of an observation taking on a specific value is directly proportional to the number of times that value has already been seen. (In the colorful terminology of the CRP, the probability of a customer sitting at a particular infinitely large table in an infinitely large Chinese restaurant is directly proportional to the number of customers already seated at the table.) Note that in order for this model to work tractably, there are also a fixed number of pseudo-observations assumed, and with probability proportional to the number of pseudo-observations, an observation is given a new value that has never been seen before.
See also
- Dirichlet processDirichlet processIn probability theory, a Dirichlet process is a stochastic process that can be thought of as a probability distribution whose domain is itself a random distribution...
- Chinese restaurant process
- Pitman–Yor process, a generalization of the Dirichlet processDirichlet processIn probability theory, a Dirichlet process is a stochastic process that can be thought of as a probability distribution whose domain is itself a random distribution...
- Polya urn modelPolya urn modelIn statistics, a Polya urn model , named after George Pólya, is a type of statistical model used as an idealized mental exercise to understand the nature of certain statistical distributions.In an urn model, objects of real interest are represented as colored balls in an urn or...
- Double jeopardy (marketing)Double jeopardy (marketing)Double jeopardy is an empirical law in marketing where, with few exceptions, the lower market share brands in a market have both far fewer buyers in a time period and also lower brand loyalty....