ADALINE is a single layer neural network

Neural network

The term neural network was traditionally used to refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks, which are composed of artificial neurons or nodes...

. It was developed by Professor Bernard Widrow

Bernard Widrow

Bernard Widrow is a U.S. professor of electrical engineering at Stanford University. He is the co-inventor of the Widrow–Hoff least mean squares filter adaptive algorithm with his then doctoral student Ted Hoff...

 and his graduate student Ted Hoff

Marcian Hoff

Marcian Edward "Ted" Hoff, Jr. , is one of the inventors of the microprocessor. Hoff joined Intel in 1967 as employee number 12, and is credited with coming up with the idea of using a "universal processor" rather than a variety of custom-designed circuits. His insight started the microprocessor...

 at Stanford University

Stanford University

The Leland Stanford Junior University, commonly referred to as Stanford University or Stanford, is a private research university on an campus located near Palo Alto, California. It is situated in the northwestern Santa Clara Valley on the San Francisco Peninsula, approximately northwest of San...

 in 1960. It is based on the McCulloch–Pitts neuron. It consists of a weight, a bias and a summation function.

The difference between Adaline and the standard (McCulloch-Pitts) perceptron

Perceptron

The perceptron is a type of artificial neural network invented in 1957 at the Cornell Aeronautical Laboratory by Frank Rosenblatt. It can be seen as the simplest kind of feedforward neural network: a linear classifier.- Definition :...

 is that in the learning phase the weights are adjusted according to the weighted sum of the inputs (the net). In the standard perceptron, the net is passed to the activation (transfer

Transfer function

A transfer function is a mathematical representation, in terms of spatial or temporal frequency, of the relation between the input and output of a linear time-invariant system. With optical imaging devices, for example, it is the Fourier transform of the point spread function i.e...

) function and the function's output is used for adjusting the weights.

There also exists an extension known as Madaline

Madaline

Madaline is a two layer neural network with a set of ADALINEs in parallel as its input layer and a single PE in its output layer. For problems with multiple input variables and one output, each input is applied to one Adaline. For similar problems with multiple outputs, madalines in parallel...

.

Definition

Adaline is a single layer neural network with multiple nodes where each node accepts multiple inputs and generates one output. Given the following variables:

• x is the input vector
• w is the weight vector
• n is the number of inputs
• some constant
• y is the output

then we find that the output is . If we further assume that

• 
• 

then the output reduces to the dot product of x and w

Learning Algorithm

Let us assume:

• is the learning rate (some constant)
• d is the desired output
• o is the actual output

then the weights are updated as follows . The ADALINE converges to the least squares error which is .