ALOPEX

From Wikipedia, the free encyclopedia

ALOPEX (an acronym from "ALgorithms Of Pattern EXtraction") is a correlation based machine learning algorithm first proposed by Tzanakou and Harth in 1974.

1 Principle
2 Method
3 Discussion
4 References

[edit] Principle

In machine learning, the goal is to train a system to minimize (or possibly maximize) a cost function. Many training algorithms, such as backpropagation, have an inherent susceptibility to getting "stuck" in local minima of the cost function. ALOPEX uses cross-correlation and stochastic processes to overcome this in an attempt to reach the absolute minimum of the cost function.

[edit] Method

ALOPEX, in its simplest form is defined by an updating equation:

$\Delta\ W_{ij}(n) = \gamma\ \Delta\ W_{ij}(n-1) \Delta\ R(n) + r_i(n)$

Where:

$\Delta\ W_{ij}(n)$ is the weight value at time n.
$\Delta\ R(n)$ is the cost function or (formally) the response function.
$\gamma\$ is the learning rate parameter
$r_i(n) \sim\ N(0,\sigma\ ^2)$

[edit] Discussion

Essentially, ALOPEX changes weights $W i j (n)$ based on a product of: the previous change in the weights $W i j (n - 1)$ , the value of the cost function, and the learning rate parameter. Further, to find the absolute minimum (or maximum), the Gaussian Noise is added to stochastically "push" the algorithm out of any local minima.

[edit] References

Harth, E., & Tzanakou, E. (1974) Alopex: A stochastic method for determining visual receptive fields. Vision Research, 14:1475-1482. [Abstract from ScienceDirect]