Consistent estimator

From Wikipedia, the free encyclopedia

In statistics, a consistent sequence of estimators is one which converges in probability to the true value of the parameter. In many cases, this is referred to simply as a consistent estimator.

A sequence is said to be strongly consistent if it converges almost surely to the correct value.

1 Formal definition
2 Properties
- 2.1 Proof
3 Further reading

[edit] Formal definition

Consider a set of samples $X_1,X_2,\ldots$ from a given probability distribution $f$ with an unknown parameter $\theta\in\Theta$ , where $Θ$ is the parameter space that is a subset of $\mathbb{R}^m$ . Let $U_n=U_n(X_1,\ldots,X_n)$ be an estimator of $θ$ based on the first $n$ samples.

We say that the sequence of estimators ${U n}$ is consistent (or that $U$ is a consistent estimator of $θ$ ), if $U i$ converges in probability to $θ$ for every $\theta\in\Theta$ . That is, for every $\varepsilon>0$ ,

$\lim_{n\rightarrow\infty}P(|U_n-\theta|\geq\varepsilon)=0$

for all $\theta\in\Theta$ .

[edit] Properties

Suppose $U$ is an estimator of $θ$ such that the sequence ${U n}$ is consistent. If $\alpha_n\to\alpha\in\mathbb{R}$ and $\beta_n\to\beta\in\mathbb{R}^m$ are two convergent sequences of constants with $0<|\alpha|<\infty$ and $|\beta|<\infty$ , then the sequence ${V n}$ , defined by $V_n \triangleq \alpha_n U_n+\beta_n$ is a consistent estimate of $αθ + β$ .

[edit] Proof

First, observe that

$\begin{align} |V_n-(\alpha\theta+\beta)| &= |\alpha_n U_n+\beta_n-\alpha\theta-\beta|\\ &\le |\alpha_nU_n-\alpha\theta|+|\beta_n-\beta|\\ &= |\alpha_nU_n-\alpha_n\theta+\alpha_n\theta-\alpha\theta|+|\beta_n-\beta|\\ &\le |\alpha_nU_n-\alpha_n\theta|+|\alpha_n\theta-\alpha\theta|+|\beta_n-\beta|\\ &= |\alpha_n||U_n-\theta|+|\alpha_n-\alpha||\theta|+|\beta_n-\beta|. \end{align}$

This implies

$\begin{align} & P(|V_n-(\alpha\theta+\beta)|\ge\varepsilon) \\ &\le P(|\alpha_n||U_n-\theta|+|\alpha_n-\alpha| |\theta|+|\beta_n-\beta|\ge\varepsilon) \\ &= P\left(|U_n-\theta|\ge\frac{\varepsilon-|\beta_n-\beta|-|\alpha_n-\alpha||\theta|}{|\alpha_n|}\right). \end{align}$

As $n\to\infty$ , we have $|\beta_n-\beta|\to 0$ , $|\alpha_n-\alpha||\theta|\to 0$ , and $|\alpha_n|\to|\alpha|\ne 0$ . So the last expression goes to $0$ as $n\to\infty$ . Therefore,