Competitive learning
Encyclopedia
Competitive learning is a form of unsupervised learning
in artificial neural networks, in which nodes compete for the right to respond to a subset of the input data. A variant of Hebbian learning, competitive learning works by increasing the specialization of each node in the network. It is well suited to finding clusters within data.
Models and algorithms based on the principle of competitive learning include vector quantization
and self-organising maps (Kohonen maps).
1. (Set-up.) Let a set of sensors all feed into three different nodes, so that every node is connected to every sensor. Let the weights that each node gives to its sensors be set randomly between 0.0 and 1.0. Let the output of each node be the sum of all its sensors, each sensor's signal strength being multiplied by its weight.
2. When the net is shown an input, the node with the highest output is deemed the winner. The input is classified as being within the cluster corresponding to that node.
3. The winner updates each of its weights, moving weight from the connections that gave it weaker signals to the connections that gave it stronger signals.
Thus, as more and more data is received, each node "listens" more and more carefully to the sensors that relate to one cluster, and "listens" less and less to sensors that relate to other clusters.
Unsupervised learning
In machine learning, unsupervised learning refers to the problem of trying to find hidden structure in unlabeled data. Since the examples given to the learner are unlabeled, there is no error or reward signal to evaluate a potential solution...
in artificial neural networks, in which nodes compete for the right to respond to a subset of the input data. A variant of Hebbian learning, competitive learning works by increasing the specialization of each node in the network. It is well suited to finding clusters within data.
Models and algorithms based on the principle of competitive learning include vector quantization
Vector quantization
Vector quantization is a classical quantization technique from signal processing which allows the modeling of probability density functions by the distribution of prototype vectors. It was originally used for data compression. It works by dividing a large set of points into groups having...
and self-organising maps (Kohonen maps).
Example algorithm
Here is a simple competitive learning algorithm to find three clusters within some input data.1. (Set-up.) Let a set of sensors all feed into three different nodes, so that every node is connected to every sensor. Let the weights that each node gives to its sensors be set randomly between 0.0 and 1.0. Let the output of each node be the sum of all its sensors, each sensor's signal strength being multiplied by its weight.
2. When the net is shown an input, the node with the highest output is deemed the winner. The input is classified as being within the cluster corresponding to that node.
3. The winner updates each of its weights, moving weight from the connections that gave it weaker signals to the connections that gave it stronger signals.
Thus, as more and more data is received, each node "listens" more and more carefully to the sensors that relate to one cluster, and "listens" less and less to sensors that relate to other clusters.