Neyman–Pearson lemma

From Wikipedia, the free encyclopedia

In statistics, the Neyman–Pearson lemma, named after Jerzy Neyman and Egon Pearson, states that when performing a hypothesis test between two point hypotheses H0: θ = θ0 and H1: θ = θ1, then the likelihood-ratio test which rejects H0 in favour of H1 when

\Lambda (x)={\frac {L(\theta _{0}\mid x)}{L(\theta _{1}\mid x)}}\leq \eta

where

P(\Lambda (X)\leq \eta \mid H_{0})=\alpha

is the most powerful test of size α for a threshold η. If the test is most powerful for all \theta _{1}\in \Theta _{1}, it is said to be uniformly most powerful (UMP) for alternatives in the set \Theta _{1}\,.

In practice, the likelihood ratio is often used directly to construct tests — see Likelihood-ratio test. However it can also be used to suggest particular test-statistics that might be of interest or to suggest simplified tests — for this one considers algebraic manipulation of the ratio to see if there are key statistics in it related to the size of the ratio (i.e. whether a large statistic corresponds to a small ratio or to a large one).

Proof

Define the rejection region of the null hypothesis for the NP test as

R_{{NP}}=\left\{x:{\frac {L(\theta _{{0}}|x)}{L(\theta _{{1}}|x)}}\leq \eta \right\}.

Any other test will have a different rejection region that we define as R_{A}. Furthermore, define the probability of the data falling in region R, given parameter \theta as

P(R,\theta )=\int _{R}L(\theta |x)\,dx,

For both tests to have size \alpha , it must be true that

\alpha =P(R_{{NP}},\theta _{0})=P(R_{A},\theta _{0})\,.

It will be useful to break these down into integrals over distinct regions:

P(R_{{NP}},\theta )=P(R_{{NP}}\cap R_{A},\theta )+P(R_{{NP}}\cap R_{A}^{c},\theta ),

and

P(R_{A},\theta )=P(R_{{NP}}\cap R_{A},\theta )+P(R_{{NP}}^{c}\cap R_{A},\theta ).

Setting \theta =\theta _{0} and equating the above two expression yields that

P(R_{{NP}}\cap R_{A}^{c},\theta _{0})=P(R_{{NP}}^{c}\cap R_{A},\theta _{0}).

Comparing the powers of the two tests, P(R_{{NP}},\theta _{1}) and P(R_{A},\theta _{1}), one can see that

P(R_{{NP}},\theta _{1})\geq P(R_{A},\theta _{1})\iff P(R_{{NP}}\cap R_{A}^{c},\theta _{1})\geq P(R_{{NP}}^{c}\cap R_{A},\theta _{1}).

Now by the definition of R_{{NP}},

P(R_{{NP}}\cap R_{A}^{c},\theta _{1})=\int _{{R_{{NP}}\cap R_{A}^{c}}}L(\theta _{{1}}|x)\,dx\geq {\frac {1}{\eta }}\int _{{R_{{NP}}\cap R_{A}^{c}}}L(\theta _{0}|x)\,dx={\frac {1}{\eta }}P(R_{{NP}}\cap R_{A}^{c},\theta _{0})
={\frac {1}{\eta }}P(R_{{NP}}^{c}\cap R_{A},\theta _{0})={\frac {1}{\eta }}\int _{{R_{{NP}}^{c}\cap R_{A}}}L(\theta _{{0}}|x)\,dx\geq \int _{{R_{{NP}}^{c}\cap R_{A}}}L(\theta _{{1}}|x)dx=P(R_{{NP}}^{c}\cap R_{A},\theta _{1}).

Hence the inequality holds.

Example

Let X_{1},\dots ,X_{n} be a random sample from the {\mathcal {N}}(\mu ,\sigma ^{2}) distribution where the mean \mu is known, and suppose that we wish to test for H_{0}:\sigma ^{2}=\sigma _{0}^{2} against H_{1}:\sigma ^{2}=\sigma _{1}^{2}. The likelihood for this set of normally distributed data is

L\left(\sigma ^{2};{\mathbf {x}}\right)\propto \left(\sigma ^{2}\right)^{{-n/2}}\exp \left\{-{\frac {\sum _{{i=1}}^{n}\left(x_{i}-\mu \right)^{2}}{2\sigma ^{2}}}\right\}.

We can compute the likelihood ratio to find the key statistic in this test and its effect on the test's outcome:

\Lambda ({\mathbf {x}})={\frac {L\left(\sigma _{0}^{2};{\mathbf {x}}\right)}{L\left(\sigma _{1}^{2};{\mathbf {x}}\right)}}=\left({\frac {\sigma _{0}^{2}}{\sigma _{1}^{2}}}\right)^{{-n/2}}\exp \left\{-{\frac {1}{2}}(\sigma _{0}^{{-2}}-\sigma _{1}^{{-2}})\sum _{{i=1}}^{n}\left(x_{i}-\mu \right)^{2}\right\}.

This ratio only depends on the data through \sum _{{i=1}}^{n}\left(x_{i}-\mu \right)^{2}. Therefore, by the Neyman–Pearson lemma, the most powerful test of this type of hypothesis for this data will depend only on \sum _{{i=1}}^{n}\left(x_{i}-\mu \right)^{2}. Also, by inspection, we can see that if \sigma _{1}^{2}>\sigma _{0}^{2}, then \Lambda ({\mathbf {x}}) is a decreasing function of \sum _{{i=1}}^{n}\left(x_{i}-\mu \right)^{2}. So we should reject H_{0} if \sum _{{i=1}}^{n}\left(x_{i}-\mu \right)^{2} is sufficiently large. The rejection threshold depends on the size of the test. In this example, the test statistic can be shown to be a scaled Chi-square distributed random variable and an exact critical value can be obtained.

See also

References

External links

  • Cosma Shalizi, a professor of statistics at Carnegie Mellon University, gives an intuitive derivation of the Neyman–Pearson Lemma using ideas from economics
This article is issued from Wikipedia. The text is available under the Creative Commons Attribution/Share Alike; additional terms may apply for the media files.