Spiking neural network
Encyclopedia
Spiking neural networks (SNNs) fall into the third generation of neural network
models, increasing the level of realism in a neural simulation. In addition to neuronal
and synaptic
state, SNNs also incorporate the concept of time into their operating model
. The idea is that neurons
in the SNN do not fire at each propagation cycle (as it happens with typical multi-layer perceptron networks
), but rather fire only when a membrane potential
- an intrinsic quality of the neuron related to its membrane electrical charge - reaches a specific value. When a neuron fires, it generates a signal which travels to other neurons which, in turn, increase or decrease their potentials in accordance with this signal.
In the context of spiking neural networks, the current activation level (modeled as some differential equation
) is normally considered to be the neuron's state, with incoming spikes pushing this value higher, and then either firing or decaying over time. Various coding methods exist for interpreting the outgoing spike train as a real-value number, either relying on the frequency of spikes, or the timing between spikes, to encode information.
and Andrew Huxley
in 1952. This model describes how action potential
s are initiated and propagated. Spikes, however are not generally transmitted directly between neuron
s, communication requires the exchange of chemical substances in the synaptic
gap, called neurotransmitter
s. The complexity and variability of biological models have resulted in various neuron models, such as the integrate-and-fire (1907), FitzHugh-Nagumo (1961-1962) and Hindmarsh-Rose
model (1984).
From the information theory
point of view, the problem is to propose a model that explains how information is encoded and decoded by a series of trains of pulses, i.e., action potentials. Thus, one of the fundamental questions of neuroscience is to determine if neurons communicated by a rate code or by a pulse code.
Early results with spiking neural models suggested that by using temporal coding
, networks of spiking neurons may gain more computational power than traditional neural networks. It was also suggested that, under certain conditions, any multilayered perceptron can be simulated closely by a network consisting of spiking neurons.
can in principle be used for information processing applications the same way as traditional artificial neural networks. However due to their more realistic properties, they can also be used to study the operation of biological neural circuits
. Starting with a hypothesis about the topology of a biological neuronal circuit and its function, the electrophysiological recordings
of this circuit can be compared to the output of the corresponding spiking artificial neural network simulated on computer
, determining the plausibility of the starting hypothesis
.
In practice, there is a major difference between the theoretical power of spiking neural networks and what has been demonstrated. They have proved useful in neuroscience, but not (yet) in engineering. To date, there have been no large scale spiking neural networks that solve computational tasks of the order and complexity of rate coded (second generation) neural networks. It is relatively easy to construct a spiking neural network model and observe its dynamics. It is much harder to develop a model with stable behavior that computes a specific function.
to simulate spiking neural networks. This software can be classified according to the use of the simulation:
, built at Stanford University, is a board that can simulate spiking neural networks directly in hardware.
Artificial neural network
An artificial neural network , usually called neural network , is a mathematical model or computational model that is inspired by the structure and/or functional aspects of biological neural networks. A neural network consists of an interconnected group of artificial neurons, and it processes...
models, increasing the level of realism in a neural simulation. In addition to neuronal
Artificial neuron
An artificial neuron is a mathematical function conceived as a crude model, or abstraction of biological neurons. Artificial neurons are the constitutive units in an artificial neural network...
and synaptic
Electrical synapse
An electrical synapse is a mechanical and electrically conductive link between two abutting neurons that is formed at a narrow gap between the pre- and postsynaptic neurons known as a gap junction. At gap junctions, such cells approach within about 3.5 nm of each other, a much shorter...
state, SNNs also incorporate the concept of time into their operating model
Operating Model
Operating model is a term that is used in many contexts. An operating model is the abstract representation of how an organization operates across process, organization, technology domains in order to deliver value defined by the organization in scope....
. The idea is that neurons
Artificial neuron
An artificial neuron is a mathematical function conceived as a crude model, or abstraction of biological neurons. Artificial neurons are the constitutive units in an artificial neural network...
in the SNN do not fire at each propagation cycle (as it happens with typical multi-layer perceptron networks
Perceptron
The perceptron is a type of artificial neural network invented in 1957 at the Cornell Aeronautical Laboratory by Frank Rosenblatt. It can be seen as the simplest kind of feedforward neural network: a linear classifier.- Definition :...
), but rather fire only when a membrane potential
Membrane potential
Membrane potential is the difference in electrical potential between the interior and exterior of a biological cell. All animal cells are surrounded by a plasma membrane composed of a lipid bilayer with a variety of types of proteins embedded in it...
- an intrinsic quality of the neuron related to its membrane electrical charge - reaches a specific value. When a neuron fires, it generates a signal which travels to other neurons which, in turn, increase or decrease their potentials in accordance with this signal.
In the context of spiking neural networks, the current activation level (modeled as some differential equation
Differential equation
A differential equation is a mathematical equation for an unknown function of one or several variables that relates the values of the function itself and its derivatives of various orders...
) is normally considered to be the neuron's state, with incoming spikes pushing this value higher, and then either firing or decaying over time. Various coding methods exist for interpreting the outgoing spike train as a real-value number, either relying on the frequency of spikes, or the timing between spikes, to encode information.
Beginnings
The first scientific model of a spiking neuron was proposed by Alan Lloyd HodgkinAlan Lloyd Hodgkin
Sir Alan Lloyd Hodgkin, OM, KBE, PRS was a British physiologist and biophysicist, who shared the 1963 Nobel Prize in Physiology or Medicine with Andrew Huxley and John Eccles....
and Andrew Huxley
Andrew Huxley
Sir Andrew Fielding Huxley, OM, FRS is an English physiologist and biophysicist, who won the 1963 Nobel Prize in Physiology or Medicine for his experimental and mathematical work with Sir Alan Lloyd Hodgkin on the basis of nerve action potentials, the electrical impulses that enable the activity...
in 1952. This model describes how action potential
Action potential
In physiology, an action potential is a short-lasting event in which the electrical membrane potential of a cell rapidly rises and falls, following a consistent trajectory. Action potentials occur in several types of animal cells, called excitable cells, which include neurons, muscle cells, and...
s are initiated and propagated. Spikes, however are not generally transmitted directly between neuron
Neuron
A neuron is an electrically excitable cell that processes and transmits information by electrical and chemical signaling. Chemical signaling occurs via synapses, specialized connections with other cells. Neurons connect to each other to form networks. Neurons are the core components of the nervous...
s, communication requires the exchange of chemical substances in the synaptic
Synapse
In the nervous system, a synapse is a structure that permits a neuron to pass an electrical or chemical signal to another cell...
gap, called neurotransmitter
Neurotransmitter
Neurotransmitters are endogenous chemicals that transmit signals from a neuron to a target cell across a synapse. Neurotransmitters are packaged into synaptic vesicles clustered beneath the membrane on the presynaptic side of a synapse, and are released into the synaptic cleft, where they bind to...
s. The complexity and variability of biological models have resulted in various neuron models, such as the integrate-and-fire (1907), FitzHugh-Nagumo (1961-1962) and Hindmarsh-Rose
Hindmarsh-Rose model
The Hindmarsh–Rose model of neuronal activity is aimed to study the spiking-bursting behavior of the membrane potential observed in experiments made with a single neuron. The relevant variable is the membrane potential, x, which is written in dimensionless units...
model (1984).
From the information theory
Information theory
Information theory is a branch of applied mathematics and electrical engineering involving the quantification of information. Information theory was developed by Claude E. Shannon to find fundamental limits on signal processing operations such as compressing data and on reliably storing and...
point of view, the problem is to propose a model that explains how information is encoded and decoded by a series of trains of pulses, i.e., action potentials. Thus, one of the fundamental questions of neuroscience is to determine if neurons communicated by a rate code or by a pulse code.
Early results with spiking neural models suggested that by using temporal coding
Temporal coding
The temporal coding is a type of neural coding which relies on precise timing of action potentials or inter-spike intervals.Combined with traditional rate coding models, temporal coding can provide additional information with the same rate....
, networks of spiking neurons may gain more computational power than traditional neural networks. It was also suggested that, under certain conditions, any multilayered perceptron can be simulated closely by a network consisting of spiking neurons.
Applications
This kind of neural networkArtificial neural network
An artificial neural network , usually called neural network , is a mathematical model or computational model that is inspired by the structure and/or functional aspects of biological neural networks. A neural network consists of an interconnected group of artificial neurons, and it processes...
can in principle be used for information processing applications the same way as traditional artificial neural networks. However due to their more realistic properties, they can also be used to study the operation of biological neural circuits
Biological neural network
In neuroscience, a biological neural network describes a population of physically interconnected neurons or a group of disparate neurons whose inputs or signalling targets define a recognizable circuit. Communication between neurons often involves an electrochemical process...
. Starting with a hypothesis about the topology of a biological neuronal circuit and its function, the electrophysiological recordings
Electrophysiology
Electrophysiology is the study of the electrical properties of biological cells and tissues. It involves measurements of voltage change or electric current on a wide variety of scales from single ion channel proteins to whole organs like the heart...
of this circuit can be compared to the output of the corresponding spiking artificial neural network simulated on computer
Computer
A computer is a programmable machine designed to sequentially and automatically carry out a sequence of arithmetic or logical operations. The particular sequence of operations can be changed readily, allowing the computer to solve more than one kind of problem...
, determining the plausibility of the starting hypothesis
Hypothesis
A hypothesis is a proposed explanation for a phenomenon. The term derives from the Greek, ὑποτιθέναι – hypotithenai meaning "to put under" or "to suppose". For a hypothesis to be put forward as a scientific hypothesis, the scientific method requires that one can test it...
.
In practice, there is a major difference between the theoretical power of spiking neural networks and what has been demonstrated. They have proved useful in neuroscience, but not (yet) in engineering. To date, there have been no large scale spiking neural networks that solve computational tasks of the order and complexity of rate coded (second generation) neural networks. It is relatively easy to construct a spiking neural network model and observe its dynamics. It is much harder to develop a model with stable behavior that computes a specific function.
Software
There is diverse range of application softwareApplication software
Application software, also known as an application or an "app", is computer software designed to help the user to perform specific tasks. Examples include enterprise software, accounting software, office suites, graphics software and media players. Many application programs deal principally with...
to simulate spiking neural networks. This software can be classified according to the use of the simulation:
- Software used primarily to simulate spiking neural networks which are present in the biology to study their operation and characteristics. In this group we can find simulators such as GENESISGENESIS (software)GENESIS is a simulation environment for constructing realistic models of neurobiological systems at many levels of scale including subcellular processes, individual neurons, networks of neurons, and neuronal systems.GENESIS was developed in the Caltech laboratory of Dr. James M...
(the GEneral NEural SImulation System) developed in James Bower's laboratory at Caltech; NEURONNeuron (software)NEURON is a simulation environment for modeling individual neurons and networks of neurons.It was primarily developed by Michael Hines, John W. Moore, and Ted Carnevale at Yale and Duke....
, mainly developed by Michael Hines, John W. Moore and Ted Carnevale in Yale UniversityYale UniversityYale University is a private, Ivy League university located in New Haven, Connecticut, United States. Founded in 1701 in the Colony of Connecticut, the university is the third-oldest institution of higher education in the United States...
and Duke UniversityDuke UniversityDuke University is a private research university located in Durham, North Carolina, United States. Founded by Methodists and Quakers in the present day town of Trinity in 1838, the school moved to Durham in 1892. In 1924, tobacco industrialist James B...
; and BrianBrian (software)Brian is an open source Python package for developing simulations of networks of spiking neurons. It is being developed by Romain Brette and Dan Goodman at the École Normale Supérieure in Paris.-Details:...
, developed by Romain Brette and Dan Goodman at the École Normale SupérieureÉcole Normale SupérieureThe École normale supérieure is one of the most prestigious French grandes écoles...
. This type of application softwareApplication softwareApplication software, also known as an application or an "app", is computer software designed to help the user to perform specific tasks. Examples include enterprise software, accounting software, office suites, graphics software and media players. Many application programs deal principally with...
usually supports the simulation of complex neural models with a high level of detail and accuracy. However large networks usually require very time-consuming simulationsComputer simulationA computer simulation, a computer model, or a computational model is a computer program, or network of computers, that attempts to simulate an abstract model of a particular system...
. - Software which addresses information processing tasks to solve problems. Computer programs such as SpikeNET are in this group, which has been developed by Delorme and Thorpe in collaboration between Centre de Recherche Cerveau et Cognition and SpikeNet Technology. This type of application software usually runs very fast simulations but does not allow the simulation of very complex or biologically-realistic neural modelsBiological neuron modelA biological neuron model is a mathematical description of the properties of nerve cells, or neurons, that is designed to accurately describe and predict biological processes...
. - Software which provides capabilities to support the simulation of relatively complex neural models efficiently so that it can also be convenient for information processing tasks. This software can exploit biological neuron characteristics to perform computation functions and at the same time allows the study of the functionality of these neural characteristics. In this software group we can find EDLUTEDLUTEDLUT is a computer application for simulating networks of spiking neurons.It was developed in the University of Granada and source code was released under GNU GPL version 3....
which has been developed in the University of GranadaUniversity of GranadaThe University of Granada is a public university located in Granada, Spain that enrolls approximately 80,000 students. The university also has campuses in Ceuta and Melilla. Every year, over 2,000 European students enroll in the UGR through the Erasmus Programme, making it the most popular...
. This application software must be efficient enough to run fast simulations, sometimes even in real time, and at the same time it must support neural models which are detailed and biologically plausible.
Hardware
NeurogridNeurogrid
Neurogrid is a piece of computer hardware that is designed specifically for simulation of biological brains. It uses analog computation to emulate ion channel activity, and digital communication to softwire structured connectivity patterns. Neurogrid simulates one million neurons and six billion...
, built at Stanford University, is a board that can simulate spiking neural networks directly in hardware.
External links
- Full text of the book Spiking Neuron Models. Single Neurons, Populations, Plasticity by Wulfram Gerstner and Werner M. Kistler (ISBN 0-521-89079-9)