Hebbian theory
Encyclopedia
Hebbian theory describes a basic mechanism for synaptic plasticity
wherein an increase in synaptic
efficacy arises from the presynaptic cell's repeated and persistent stimulation of the postsynaptic cell. Introduced by Donald Hebb
in 1949, it is also called Hebb's rule, Hebb's postulate, and cell assembly theory, and states:
The theory is often summarized as "Cells that fire together, wire together." It attempts to explain "associative learning", in which simultaneous activation of cells leads to pronounced increases in synaptic strength between those cells. Such learning is known as Hebbian learning.
. Hebb's theories on the form and function of cell assemblies can be understood from the following:
Gordon Allport
posits additional ideas regarding cell assembly theory and its role in forming engrams, along the lines of the concept of auto-association, described as follows:
Hebbian theory has been the primary basis for the conventional view that when analyzed from a holistic level, engrams are neuronal nets or neural networks
.
Work in the laboratory of Eric Kandel has provided evidence for the involvement of Hebbian learning mechanisms at synapses in the marine gastropod
Aplysia californica.
Experiments on Hebbian synapse modification mechanisms at the central nervous system
synapse
s of vertebrates are much more difficult to control than are experiments with the relatively simple peripheral nervous system
synapses studied in marine invertebrates. Much of the work on long-lasting synaptic changes between vertebrate neurons (such as long-term potentiation
) involves the use of non-physiological experimental stimulation of brain cells. However, some of the physiologically relevant synapse modification mechanisms that have been studied in vertebrate brains do seem to be examples of Hebbian processes. One such study reviews results from experiments that indicate that long-lasting changes in synaptic strengths can be induced by physiologically relevant synaptic activity working through both Hebbian and non-Hebbian mechanisms
This original principle is perhaps the simplest form of weight selection. While this means it can be relatively easily coded into a computer program
and used to update the weights for a network, it also prohibits the number of applications of Hebbian learning. Today, the term Hebbian learning generally refers to some form of mathematical abstraction of the original principle proposed by Hebb. In this sense, Hebbian learning involves weights between learning nodes being adjusted so that each weight better represents the relationship between the nodes. As such, many learning
methods can be considered to be somewhat Hebbian in nature.
The following is a formulaic description of Hebbian learning: (note that many other descriptions are possible)
where
is the weight of the connection from neuron
to neuron 
and 
the input for neuron 
. Note that this is pattern learning (weights updated after every training example). In a Hopfield network, connections 
are set to zero if 
(no reflexive connections allowed). With binary neurons (activations either 0 or 1), connections would be set to 1 if the connected neurons have the same activation for a pattern.
Another formulaic description is:
,
where
is the weight of the connection from neuron 
to neuron 
, 
is the number of training patterns, and 
the 
th input for neuron 
. This is learning by epoch (weights updated after all the training examples are presented). Again, in a Hopfield network, connections 
are set to zero if 
(no reflexive connections).
A variation of Hebbian learning that takes into account phenomena such as blocking and many other neural learning phenomena is the mathematical model of Harry Klopf. Klopf's model reproduces a great many biological phenomena, and is also simple to implement.
or the change in the
th synaptic weight 
is equal to a learning rate 
times the 
th input 
times the postsynaptic response 
. Often cited is the case of a linear neuron,
and the previous section's simplification takes both the learning rate and the input weights to be 1. This version of the rule is clearly unstable, as in any network with a dominant signal the synaptic weights will increase or decrease exponentially. However, it can be shown that for any neuron model, Hebb's rule is unstable. Therefore, network models of neurons usually employ other learning theories such as BCM theory
, Oja's rule
, or the Generalized Hebbian Algorithm
.
Synaptic plasticity
In neuroscience, synaptic plasticity is the ability of the connection, or synapse, between two neurons to change in strength in response to either use or disuse of transmission over synaptic pathways. Plastic change also results from the alteration of the number of receptors located on a synapse...
wherein an increase in synaptic
Synapse
In the nervous system, a synapse is a structure that permits a neuron to pass an electrical or chemical signal to another cell...
efficacy arises from the presynaptic cell's repeated and persistent stimulation of the postsynaptic cell. Introduced by Donald Hebb
Donald Olding Hebb
Donald Olding Hebb FRS was a Canadian psychologist who was influential in the area of neuropsychology, where he sought to understand how the function of neurons contributed to psychological processes such as learning...
in 1949, it is also called Hebb's rule, Hebb's postulate, and cell assembly theory, and states:
- Let us assume that the persistence or repetition of a reverberatory activity (or "trace") tends to induce lasting cellular changes that add to its stability.… When an axonAxonAn axon is a long, slender projection of a nerve cell, or neuron, that conducts electrical impulses away from the neuron's cell body or soma....
of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that As efficiency, as one of the cells firing B, is increased.
The theory is often summarized as "Cells that fire together, wire together." It attempts to explain "associative learning", in which simultaneous activation of cells leads to pronounced increases in synaptic strength between those cells. Such learning is known as Hebbian learning.
Hebbian engrams and cell assembly theory
Hebbian theory concerns how neurons might connect themselves to become engramsEngram (neuropsychology)
Engrams are a hypothetical means by which memory traces are stored as biophysical or biochemical changes in the brain in response to external stimuli....
. Hebb's theories on the form and function of cell assemblies can be understood from the following:
- "The general idea is an old one, that any two cells or systems of cells that are repeatedly active at the same time will tend to become 'associated', so that activity in one facilitates activity in the other."
- "When one cell repeatedly assists in firing another, the axon of the first cell develops synaptic knobs (or enlarges them if they already exist) in contact with the soma of the second cell."
Gordon Allport
Gordon Allport
Gordon Willard Allport was an American psychologist. Allport was one of the first psychologists to focus on the study of the personality, and is often referred to as one of the founding figures of personality psychology...
posits additional ideas regarding cell assembly theory and its role in forming engrams, along the lines of the concept of auto-association, described as follows:
- "If the inputs to a system cause the same pattern of activity to occur repeatedly, the set of active elements constituting that pattern will become increasingly strongly interassociated. That is, each element will tend to turn on every other element and (with negative weights) to turn off the elements that do not form part of the pattern. To put it another way, the pattern as a whole will become 'auto-associated'. We may call a learned (auto-associated) pattern an engram."
Hebbian theory has been the primary basis for the conventional view that when analyzed from a holistic level, engrams are neuronal nets or neural networks
Neural Networks
Neural Networks is the official journal of the three oldest societies dedicated to research in neural networks: International Neural Network Society, European Neural Network Society and Japanese Neural Network Society, published by Elsevier...
.
Work in the laboratory of Eric Kandel has provided evidence for the involvement of Hebbian learning mechanisms at synapses in the marine gastropod
Gastropoda
The Gastropoda or gastropods, more commonly known as snails and slugs, are a large taxonomic class within the phylum Mollusca. The class Gastropoda includes snails and slugs of all kinds and all sizes from microscopic to quite large...
Aplysia californica.
Experiments on Hebbian synapse modification mechanisms at the central nervous system
Central nervous system
The central nervous system is the part of the nervous system that integrates the information that it receives from, and coordinates the activity of, all parts of the bodies of bilaterian animals—that is, all multicellular animals except sponges and radially symmetric animals such as jellyfish...
synapse
Synapse
In the nervous system, a synapse is a structure that permits a neuron to pass an electrical or chemical signal to another cell...
s of vertebrates are much more difficult to control than are experiments with the relatively simple peripheral nervous system
Peripheral nervous system
The peripheral nervous system consists of the nerves and ganglia outside of the brain and spinal cord. The main function of the PNS is to connect the central nervous system to the limbs and organs. Unlike the CNS, the PNS is not protected by the bone of spine and skull, or by the blood–brain...
synapses studied in marine invertebrates. Much of the work on long-lasting synaptic changes between vertebrate neurons (such as long-term potentiation
Long-term potentiation
In neuroscience, long-term potentiation is a long-lasting enhancement in signal transmission between two neurons that results from stimulating them synchronously. It is one of several phenomena underlying synaptic plasticity, the ability of chemical synapses to change their strength...
) involves the use of non-physiological experimental stimulation of brain cells. However, some of the physiologically relevant synapse modification mechanisms that have been studied in vertebrate brains do seem to be examples of Hebbian processes. One such study reviews results from experiments that indicate that long-lasting changes in synaptic strengths can be induced by physiologically relevant synaptic activity working through both Hebbian and non-Hebbian mechanisms
Principles
From the point of view of artificial neurons and artificial neural networks, Hebb's principle can be described as a method of determining how to alter the weights between model neurons. The weight between two neurons increases if the two neurons activate simultaneously—and reduces if they activate separately. Nodes that tend to be either both positive or both negative at the same time have strong positive weights, while those that tend to be opposite have strong negative weights.This original principle is perhaps the simplest form of weight selection. While this means it can be relatively easily coded into a computer program
Computer program
A computer program is a sequence of instructions written to perform a specified task with a computer. A computer requires programs to function, typically executing the program's instructions in a central processor. The program has an executable form that the computer can use directly to execute...
and used to update the weights for a network, it also prohibits the number of applications of Hebbian learning. Today, the term Hebbian learning generally refers to some form of mathematical abstraction of the original principle proposed by Hebb. In this sense, Hebbian learning involves weights between learning nodes being adjusted so that each weight better represents the relationship between the nodes. As such, many learning
Learning
Learning is acquiring new or modifying existing knowledge, behaviors, skills, values, or preferences and may involve synthesizing different types of information. The ability to learn is possessed by humans, animals and some machines. Progress over time tends to follow learning curves.Human learning...
methods can be considered to be somewhat Hebbian in nature.
The following is a formulaic description of Hebbian learning: (note that many other descriptions are possible)
where
is the weight of the connection from neuronArtificial neuron
An artificial neuron is a mathematical function conceived as a crude model, or abstraction of biological neurons. Artificial neurons are the constitutive units in an artificial neural network...
to neuron and the input for neuron . Note that this is pattern learning (weights updated after every training example). In a Hopfield network, connections are set to zero if (no reflexive connections allowed). With binary neurons (activations either 0 or 1), connections would be set to 1 if the connected neurons have the same activation for a pattern.Another formulaic description is:
, where
is the weight of the connection from neuron to neuron , is the number of training patterns, and the th input for neuron . This is learning by epoch (weights updated after all the training examples are presented). Again, in a Hopfield network, connections are set to zero if (no reflexive connections).A variation of Hebbian learning that takes into account phenomena such as blocking and many other neural learning phenomena is the mathematical model of Harry Klopf. Klopf's model reproduces a great many biological phenomena, and is also simple to implement.
Generalization and stability
Hebb's Rule is often generalized asor the change in the
th synaptic weight is equal to a learning rate times the th input times the postsynaptic response . Often cited is the case of a linear neuron,and the previous section's simplification takes both the learning rate and the input weights to be 1. This version of the rule is clearly unstable, as in any network with a dominant signal the synaptic weights will increase or decrease exponentially. However, it can be shown that for any neuron model, Hebb's rule is unstable. Therefore, network models of neurons usually employ other learning theories such as BCM theory
BCM theory
BCM theory, BCM synaptic modification, or the BCM rule, named for Elie Bienenstock, Leon Cooper, and Paul Munro, is a physical theory of learning in the visual cortex developed in 1981...
, Oja's rule
Oja's rule
Oja's learning rule, or simply Oja's rule, named after a Finnish computer scientist Erkki Oja, is a model of how neurons in the brain or in artificial neural networks change connection strength, or learn, over time...
, or the Generalized Hebbian Algorithm
Generalized Hebbian Algorithm
The Generalized Hebbian Algorithm , also known in the literature as Sanger's rule, is a linear feedforward neural network model for unsupervised learning with applications primarily in principal components analysis...
.
See also
- Dale's principleDale's principleIn neuroscience, Dale's Principle is a rule attributed to the English neuroscientist Henry Hallett Dale. The principle basically states that a neuron performs the same chemical action at all of its synaptic connections to other cells, regardless of the identity of the target cell...
- Coincidence Detection in NeurobiologyCoincidence detection in neurobiologyCoincidence detection in the context of neurobiology is a process by which a neuron or a neural circuit can encode information by detecting the occurrence of timely simultaneous yet spatially separate input signals...
- LeabraLeabraLeabra stands for "Local, Error-driven and Associative, Biologically Realistic Algorithm". It is a model of learning which is a balance between Hebbian and error-driven learning with other network-derived characteristics. This model is used to mathematically predict outcomes based on inputs and...
- Long-term potentiationLong-term potentiationIn neuroscience, long-term potentiation is a long-lasting enhancement in signal transmission between two neurons that results from stimulating them synchronously. It is one of several phenomena underlying synaptic plasticity, the ability of chemical synapses to change their strength...
- MemoryMemoryIn psychology, memory is an organism's ability to store, retain, and recall information and experiences. Traditional studies of memory began in the fields of philosophy, including techniques of artificially enhancing memory....
- MetaplasticityMetaplasticityMetaplasticity is a term originally coined by W.C. Abraham and M.F. Bear to refer to the plasticity of synaptic plasticity. Until that time synaptic plasticity had referred to the plastic nature of individual synapses. However this new form referred to the plasticity of the plasticity itself, thus...
- Spike-timing-dependent plasticity
- Tetanic stimulationTetanic stimulationIn neurobiology, a tetanic stimulation consists of a high-frequency sequence of individual stimulations of a neuron. It is associated with long-term potentiation....
External links
- Overview
- Hebbian Learning tutorial (Part 1: Novelty Filtering, Part 2: PCA)