ALOPEX
Encyclopedia
ALOPEX is a correlation based machine learning algorithm first proposed by Tzanakou
and Harth in 1974.
, the goal is to train a system to minimize a cost function or (referring to ALOPEX) a response function. Many training algorithms, such as backpropagation
, have an inherent susceptibility to getting "stuck" in local minima or maxima of the response function. ALOPEX uses a cross-correlation of differences and a stochastic process to overcome this in an attempt to reach the absolute minimum (or maximum) of the response function.
Where:
based on a product of: the previous change in the variable 
, the resulting change in the cost function 
, and the learning rate parameter 
. Further, to find the absolute minimum (or maximum), the stochastic process 
(Gaussian or other) is added to stochastically "push" the algorithm out of any local minima.
Evangelia Micheli-Tzanakou
Evangelia Micheli-Tzanakou is a professor of biomedical engineering and the Director of Computational Intelligence Laboratories at Rutgers University. Dr...
and Harth in 1974.
Principle
In machine learningMachine learning
Machine learning, a branch of artificial intelligence, is a scientific discipline concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data, such as from sensor data or databases...
, the goal is to train a system to minimize a cost function or (referring to ALOPEX) a response function. Many training algorithms, such as backpropagation
Backpropagation
Backpropagation is a common method of teaching artificial neural networks how to perform a given task. Arthur E. Bryson and Yu-Chi Ho described it as a multi-stage dynamic system optimization method in 1969 . It wasn't until 1974 and later, when applied in the context of neural networks and...
, have an inherent susceptibility to getting "stuck" in local minima or maxima of the response function. ALOPEX uses a cross-correlation of differences and a stochastic process to overcome this in an attempt to reach the absolute minimum (or maximum) of the response function.
Method
ALOPEX, in its simplest form is defined by an updating equation:Where:
is the iteration or time-step.
is the difference between the current and previous value of system variable 
at iteration 
.
is the difference between the current and previous value of the response function 
at iteration 
.
is the learning rate parameter 
minimizes 
and 
maximizes 
Discussion
Essentially, ALOPEX changes each system variable based on a product of: the previous change in the variable , the resulting change in the cost function , and the learning rate parameter . Further, to find the absolute minimum (or maximum), the stochastic process (Gaussian or other) is added to stochastically "push" the algorithm out of any local minima.