ALOPEX

From Wikipedia, the free encyclopedia

ALOPEX (an acronym from "ALgorithms Of Pattern EXtraction") is a correlation based machine learning algorithm first proposed by Tzanakou and Harth in 1974.

1 Principle
2 Method
3 Discussion
4 References

[edit] Principle

In machine learning, the goal is to train a system to minimize a cost function or (referring to ALOPEX) a response function. Many training algorithms, such as backpropagation, have an inherent susceptibility to getting "stuck" in local minima or maxima of the response function. ALOPEX uses a cross-correlation of differences and a stochastic process to overcome this in an attempt to reach the absolute minimum (or maximum) of the response function.

[edit] Method

ALOPEX, in its simplest form is defined by an updating equation:

$\Delta\ W_{ij}(n) = \gamma\ \Delta\ W_{ij}(n-1) \Delta\ R(n) + r_i(n)$

Where:

$n \geq 0$ is the iteration or time-step.
$\Delta\ W_{ij}(n)$ is the difference between the current and previous value of system variable $\ W_{ij}$ at iteration $n \$ .
$\Delta\ R(n)$ is the difference between the current and previous value of the response function $\ R,$ at iteration $n \$ .
$\gamma\$ is the learning rate parameter $(\gamma\ < 0$ minimizes $R, \$ and $\gamma\ > 0$ maximizes $R \ )$
$r_i(n) \sim\ N(0,\sigma\ ^2)$

[edit] Discussion

Essentially, ALOPEX changes each system variable $W i j (n)$ based on a product of: the previous change in the variable $Δ$ $W i j (n - 1)$ , the resulting change in the cost function $Δ$ $R (n)$ , and the learning rate parameter $γ$ . Further, to find the absolute minimum (or maximum), the stochastic process $r i j (n)$ (Gaussian or other) is added to stochastically "push" the algorithm out of any local minima.

[edit] References

Harth, E., & Tzanakou, E. (1974) Alopex: A stochastic method for determining visual receptive fields. Vision Research, 14:1475-1482. Abstract from ScienceDirect