Sparse coding
Encyclopedia
The sparse code is a kind of neural code in which each item is encoded by the strong activation of a relatively small set of neurons. For each item to be encoded, this is a different subset of all available neurons.
As a consequence, sparseness may be focused on temporal sparseness ("a relatively small number of time periods are active") or on the sparseness in an activated population of neurons. In this latter case, this may be defined in one time period as the number of activated neurons relative to the total number of neurons in the population. This seems to be a hallmark of neural computations since compared to traditional computers, information is massively distributed across neurons. A major result in neural coding from Olshausen et al. is that sparse coding of natural images produces wavelet
-like oriented filters that resemble the receptive fields of simple cells in the visual cortex.
As a consequence, sparseness may be focused on temporal sparseness ("a relatively small number of time periods are active") or on the sparseness in an activated population of neurons. In this latter case, this may be defined in one time period as the number of activated neurons relative to the total number of neurons in the population. This seems to be a hallmark of neural computations since compared to traditional computers, information is massively distributed across neurons. A major result in neural coding from Olshausen et al. is that sparse coding of natural images produces wavelet
Wavelet
A wavelet is a wave-like oscillation with an amplitude that starts out at zero, increases, and then decreases back to zero. It can typically be visualized as a "brief oscillation" like one might see recorded by a seismograph or heart monitor. Generally, wavelets are purposefully crafted to have...
-like oriented filters that resemble the receptive fields of simple cells in the visual cortex.
See also
- Rate codingRate codingThe rate coding model of neuronal firing communication states that as the intensity of a stimulus increases the frequency or rate of action potentials, or "spike firing", increases. Rate coding is sometimes called frequency coding....
- Independent-spike codingIndependent-spike codingThe independent-spike coding model of neuronal firing claims that each individual action potential, or "spike", is independent of each other spike within the spike train.Contrast this with correlation coding.-References:...
- Correlation codingCorrelation codingThe correlation coding model of neuronal firing claims that correlations between action potentials, or "spikes", within a spike train may carry additional information above and beyond the simple timing of the spikes...
- Population codingPopulation codingPopulation coding is a means by which information is coded in a group of neurons. In population coding, each neuron has a distribution of responses over some set of inputs, and the responses of many neurons may be combined to determine some value about the inputs...
- Grandmother cellGrandmother cellThe grandmother cell is a hypothetical neuron that represents a complex but specific concept or object. It activates when a person's brain "sees, hears, or otherwise sensibly discriminates" a specific entity, such as his or her grandmother. The term was coined around 1969 by Jerry Lettvin...