Lehmann–Scheffé theorem

In statistics, the Lehmann–Scheffé theorem is prominent in mathematical statistics, tying together the ideas of completeness, sufficiency, uniqueness, and best unbiased estimation.[1] The theorem states that any estimator which is unbiased for a given unknown quantity and which is based on only a complete, sufficient statistic (and on no other data-derived values) is the unique best unbiased estimator of that quantity. The Lehmann–Scheffé theorem is named after Erich Leo Lehmann and Henry Scheffé, given their two early papers.[2][3]

If T is a complete sufficient statistic for θ and E(g(T)) = τ(θ) then g(T) is the minimum-variance unbiased estimator (MVUE) of τ(θ).

Statement

Let \vec{X}= X_1, X_2 \dots X_n be a random sample from a distribution that has p.d.f (or p.m.f in the discrete case) f(x:\theta) where \theta \in \Omega is a parameter in the parameter space. Suppose Y = u(\vec{X}) is a sufficient statistic for θ, and let \{ f_{Y}(y:\theta): \theta \in \Omega\} be a complete family. If \phi:\mathbb{E}[\phi(Y)] = \theta then \phi(Y) is the unique MVUE of θ.

Proof

By the Rao–Blackwell theorem, if Z is an unbiased estimator of θ then \phi(Y):= \mathbb{E}[Z|Y] defines an unbiased estimator of θ with the property that its variance is smaller than that of Z.

Now we show that this function is unique. Suppose W is another candidate MVUE estimator of θ. Then again \psi(Y):= \mathbb{E}[W|Y] defines an unbiased estimator of θ with the property that its variance is smaller than that of W. Then

\mathbb{E}[\phi(Y) - \psi(Y)] = 0, \theta \in \Omega.

Since \{ f_{Y}(y:\theta): \theta \in \Omega\} is a complete family

\mathbb{E}[\phi(Y) - \psi(Y)] = 0 \implies \phi(y) - \psi(y) = 0, \theta \in \Omega

and therefore the function \phi is the unique function of Y that has a smaller variance than any unbiased estimator. We conclude that \phi(Y) is the MVUE.

See also

References

  1. ↑ Casella, George (2001). Statistical Inference. Duxbury Press. p. 369. ISBN 0-534-24312-6.
  2. ↑ Lehmann, E. L.; Scheffé, H. (1950). "Completeness, similar regions, and unbiased estimation. I.". Sankhyā 10 (4): 305–340. JSTOR 25048038. MR 39201.
  3. ↑ Lehmann, E.L.; Scheffé, H. (1955). "Completeness, similar regions, and unbiased estimation. II.". Sankhyā 15 (3): 219–236. JSTOR 25048243. MR 72410.