Hoeffding's lemma
From Wikipedia, the free encyclopedia
In probability theory, Hoeffding's lemma is an inequality that bounds the moment-generating function of any bounded random variable. It is named after the Finnish–American mathematical statistician Wassily Hoeffding.
The proof of Hoeffding's lemma uses Taylor's theorem and Jensen's inequality. Hoeffding's lemma is itself used in the proof of McDiarmid's inequality.
Statement of the lemma
Let X be any real-valued random variable with expected value E[X] = 0 and such that a ≤ X ≤ b almost surely. Then, for all λ ∈ R,
Proof of the lemma
Since is a convex function, we have
So,
Let , and
Then, since
Taking derivative of ,
By Tayor's expansion,
Hence,
See also
This article is issued from Wikipedia. The text is available under the Creative Commons Attribution/Share Alike; additional terms may apply for the media files.