Kernel principal component analysis

From Wikipedia, the free encyclopedia

Kernel principal component analysis (kernel PCA) is an extension of principal component analysis (PCA) using techniques of kernel methods. Using a kernel, the originally linear operations of PCA are done in a reproducing kernel Hilbert space with a non-linear mapping.

[edit] Example

The two images show a number of data points before and after Kernel PCA. The color of the points is not part of the algorithm, it's only there to show how the data groups together before and after the transformation. Note in particular that the first principal component is enough to distinguish the three different groups, which is impossible using only linear PCA.

The kernel used in this example was:

$k(\boldsymbol{x},\boldsymbol{y}) = (\boldsymbol{x}^\mathrm{T}\boldsymbol{y} + 1)^2$

If instead a gaussian kernel is used: $k(\boldsymbol{x},\boldsymbol{y}) = e^\frac{-||\boldsymbol{x} - \boldsymbol{y}||)^2}{2\sigma^2},$ the result is shown in the next figure.