Kernel regression

From Wikipedia, the free encyclopedia

The kernel regression is a non-parametrical technique in statistics to estimate the conditional expectation of a random variable. The objective is to find a non-linear relation between a pair of random variables X and Y.

In any nonparametric regression, the conditional expectation of a variable Y relative to a variable X may be written:

\operatorname{E}(Y | X) = m(X)

where m is a non-parametric function.

Contents

[edit] Nadaraya-Watson kernel regression

Nadaraya (1964) and Watson (1964) proposed to estimate m as a locally weighted average, using a kernel as a weighting function. The Nadaraya-Watson estimator is:

\widehat{m}_h(x)=\frac{n^{-1}\sum_{i=1}^nK_h(x-X_i)Y_i }{n^{-1}\sum_{i=1}^nK_h(x-X_i)}

where K is a kernel with a bandwidth h.

[edit] Derivation

\operatorname{E}(Y | X) = \int y f(y|x) dy = \int y \frac{f(x,y)}{f(x)} dy

Using the kernel density estimation for the joint distribution f(x,y) and f(x) with a kernel K,

\hat{f}(x,y) = n^{-1} h^{-2} \sum_{i=1}^{n} K\left(\frac{x-x_i}{h}\right) K\left(\frac{y-y_i}{h}\right) ,
\hat{f}(x) = n^{-1} h^{-1} \sum_{i=1}^{n} K\left(\frac{x-x_i}{h}\right)

we obtain the Nadaraya-Watson estimator.

[edit] Priestley-Chao kernel estimator

\widehat{m}_{PC}(x) = h^{-1} \sum_{i=1}^n (x_i - x_{i-1}) K\left(\frac{x-x_i}{h}\right) y_i

[edit] Gasser-Müller kernel estimator

\widehat{m}_{GM}(x) = h^{-1} \sum_{i=1}^n \left[\int_{s_{i-1}}^{s_i} K\left(\frac{x-u}{h}\right) du\right] y_i

where si = (xi-1 + xi)/2

[edit] Kernel regression for image processing

Applications of Kernel regression for image processing purposes include but is not limited to denoising, deblurring, interpolation, super-resolution, and many other applications [1].

[edit] References

  1. ^ H. Takeda, S. Farsiu, and P. Milanfar. Kernel Regression for Image Processing and Reconstruction. IEEE Trans. on Image Processing, vol. 16, no. 2, pp. 349-366, Feb. 2007.

Nadaraya, E. A. (1964). "On Estimating Regression". Theory of Probability and its Applications 9 (1): 141–142. doi:10.1137/1109020. 

Simonoff, Jeffrey S. (1996). Smoothing Methods in Statistics. Springer. ISBN 0-387-94716-7. 

[edit] Statistical implementation

 kernreg2 y x, bwidth(.5) kercode(3) npoint(500) gen(kernelprediction gridofpoints)

[edit] External links

 This statistics-related article is a stub. You can help Wikipedia by expanding it.