Helmholtz machine
Encyclopedia
The Helmholtz machine is a name used by Geoff Hinton to describe a class of neural networks
which learn
the hidden structure of a set of data by being trained to create a generative model
which can produce the original set of data. The hope is that by learning economical representations
of the data, the underlying structure of the generative model should reasonably approximate the hidden structure of the data set. This is an unsupervised learning
algorithm.
The networks are characterized by bottom-up and top-down (reentrant) connections between layers of stochastic elements. The "Wake-Sleep" algorithm is an unsupervised learning rule which has been suggested as a means to train the Helmholtz machine.
Helmholtz machines may also be used in applications requiring a supervised learning algorithm (e.g. character recognition, or position-invariant recognition of an object within a field).
Neural Networks
Neural Networks is the official journal of the three oldest societies dedicated to research in neural networks: International Neural Network Society, European Neural Network Society and Japanese Neural Network Society, published by Elsevier...
which learn
LEARN
LEARN may refer to:* Law Enforcement Agency Resource Network, a website run by the Anti-Defamation League* LEARN diet, a brand name diet product...
the hidden structure of a set of data by being trained to create a generative model
Generative model
In probability and statistics, a generative model is a model for randomly generating observable data, typically given some hidden parameters. It specifies a joint probability distribution over observation and label sequences...
which can produce the original set of data. The hope is that by learning economical representations
Representations
Representations is an interdisciplinary journal in the humanities published quarterly by the University of California Press. The journals was established in 1983 and is the founding publication of the New Historicism movement of the 1980s. It covers topics including literary, historical, and...
of the data, the underlying structure of the generative model should reasonably approximate the hidden structure of the data set. This is an unsupervised learning
Unsupervised learning
In machine learning, unsupervised learning refers to the problem of trying to find hidden structure in unlabeled data. Since the examples given to the learner are unlabeled, there is no error or reward signal to evaluate a potential solution...
algorithm.
The networks are characterized by bottom-up and top-down (reentrant) connections between layers of stochastic elements. The "Wake-Sleep" algorithm is an unsupervised learning rule which has been suggested as a means to train the Helmholtz machine.
Helmholtz machines may also be used in applications requiring a supervised learning algorithm (e.g. character recognition, or position-invariant recognition of an object within a field).