Edgeworth series

From Wikipedia, the free encyclopedia

The Gram–Charlier A series (named in honor of Jørgen Pedersen Gram and Carl Charlier), and the Edgeworth series (named in honor of Francis Ysidro Edgeworth) are series that approximate a probability distribution in terms of its cumulants. The series are the same; but, the arrangement of terms (and thus the accuracy of truncating the series) differ.

Gram–Charlier A series

The key idea of these expansions is to write the characteristic function of the distribution whose probability density function is ${\mathit {F}}$ to be approximated in terms of the characteristic function of a distribution with known and suitable properties, and to recover ${\mathit {F}}$ through the inverse Fourier transform.

Let ${\mathit {f}}$ be the characteristic function of the distribution whose density function is ${\mathit {F}}$ , and $\kappa _{r}$ its cumulants. We expand in terms of a known distribution with probability density function $\Psi$ , characteristic function $\psi$ , and cumulants $\gamma _{r}$ . The density $\Psi$ is generally chosen to be that of the normal distribution, but other choices are possible as well. By the definition of the cumulants, we have the following formal identity:

$f(t)=\exp \left[\sum _{{r=1}}^{\infty }(\kappa _{r}-\gamma _{r}){\frac {(it)^{r}}{r!}}\right]\psi (t)\,.$

By the properties of the Fourier transform, (it)^rψ(t) is the Fourier transform of (−1)^r D^r $\Psi$ (x), where D is the differential operator with respect to x. Thus, we find for F the formal expansion

$F(x)=\exp \left[\sum _{{r=1}}^{\infty }(\kappa _{r}-\gamma _{r}){\frac {(-D)^{r}}{r!}}\right]\Psi (x)\,.$

If $\Psi$ is chosen as the normal density with mean and variance as given by F, that is, mean μ = κ₁ and variance σ² = κ₂, then the expansion becomes

$F(x)=\exp \left[\sum _{{r=3}}^{\infty }\kappa _{r}{\frac {(-D)^{r}}{r!}}\right]{\frac {1}{{\sqrt {2\pi }}\sigma }}\exp \left[-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}\right]\,.$

By expanding the exponential and collecting terms according to the order of the derivatives, we arrive at the Gram–Charlier A series. If we include only the first two correction terms to the normal distribution, we obtain

$F(x)\approx {\frac {1}{{\sqrt {2\pi }}\sigma }}\exp \left[-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}\right]\left[1+{\frac {\kappa _{3}}{3!\sigma ^{3}}}H_{3}\left({\frac {x-\mu }{\sigma }}\right)+{\frac {\kappa _{4}}{4!\sigma ^{4}}}H_{4}\left({\frac {x-\mu }{\sigma }}\right)\right]\,,$

with H₃(x) = x³ − 3x and H₄(x) = x⁴ − 6x² + 3 (these are Hermite polynomials).

Note that this expression is not guaranteed to be positive, and is therefore not a valid probability distribution. The Gram–Charlier A series diverges in many cases of interest—it converges only if F(x) falls off faster than exp(−x²/4) at infinity (Cramér 1957). When it does not converge, the series is also not a true asymptotic expansion, because it is not possible to estimate the error of the expansion. For this reason, the Edgeworth series (see next section) is generally preferred over the Gram–Charlier A series.

Edgeworth series

Edgeworth developed a similar expansion as an improvement to the central limit theorem. The advantage of the Edgeworth series is that the error is controlled, so that it is a true asymptotic expansion.

Let {X_i} be a sequence of independent and identically distributed random variables with mean μ and variance σ², and let Y_n be their standardized sums:

$Y_{n}={\frac {1}{{\sqrt {n}}}}\sum _{{i=1}}^{n}{\frac {X_{i}-\mu }{\sigma }}.$

Let F_n denote the cumulative distribution functions of the variables Y_n. Then by the central limit theorem,

$\lim _{{n\to \infty }}F_{n}(x)=\Phi (x)\equiv \int _{{-\infty }}^{x}{\tfrac {1}{{\sqrt {2\pi }}}}e^{{-{\frac {1}{2}}q^{2}}}dq$

for every x, as long as the mean and variance are finite.

Now assume that the random variables X_i have mean μ, variance σ², and higher cumulants κ_r=σ^rλ_r. If we expand in terms of the standard normal distribution, that is, if we set

$\Psi (x)={\frac {1}{{\sqrt {2\pi }}}}\exp(-{\tfrac {1}{2}}x^{2})$

then the cumulant differences in the formal expression of the characteristic function f_n(t) of F_n are

$\kappa _{1}^{{F(n)}}-\gamma _{1}=0\,,$

$\kappa _{2}^{{F(n)}}-\gamma _{2}=0\,,$

$\kappa _{r}^{{F(n)}}-\gamma _{r}={\frac {\kappa _{r}}{\sigma ^{r}n^{{r/2-1}}}}={\frac {\lambda _{r}}{n^{{r/2-1}}}};\quad r\geq 3\,.$

The Edgeworth series is developed similarly to the Gram–Charlier A series, only that now terms are collected according to powers of n. Thus, we have

$f_{n}(t)=\left[1+\sum _{{j=1}}^{\infty }{\frac {P_{j}(it)}{n^{{j/2}}}}\right]\exp(-t^{2}/2)\,,$

where P_j(x) is a polynomial of degree 3j. Again, after inverse Fourier transform, the density function F_n follows as

$F_{n}(x)=\Phi (x)+\sum _{{j=1}}^{\infty }{\frac {P_{j}(-D)}{n^{{j/2}}}}\Phi (x)\,.$

The first five terms of the expansion are^[1]

${\begin{aligned}F_{n}(x)=&\Phi (x)\\&-{\frac {1}{n^{{1/2}}}}{\bigg (}{\tfrac {1}{6}}\lambda _{3}\,\Phi ^{{(3)}}(x){\bigg )}\\&+{\frac {1}{n}}{\bigg (}{\tfrac {1}{24}}\lambda _{4}\,\Phi ^{{(4)}}(x)+{\tfrac {1}{72}}\lambda _{3}^{2}\,\Phi ^{{(6)}}(x){\bigg )}\\&-{\frac {1}{n^{{3/2}}}}{\bigg (}{\tfrac {1}{120}}\lambda _{5}\,\Phi ^{{(5)}}(x)+{\tfrac {1}{144}}\lambda _{3}\lambda _{4}\,\Phi ^{{(7)}}(x)+{\tfrac {1}{1296}}\lambda _{3}^{3}\,\Phi ^{{(9)}}(x){\bigg )}\\&+{\frac {1}{n^{2}}}{\bigg (}{\tfrac {1}{720}}\lambda _{6}\,\Phi ^{{(6)}}(x)+{\big (}{\tfrac {1}{1152}}\lambda _{4}^{2}+{\tfrac {1}{720}}\lambda _{3}\lambda _{5}{\big )}\Phi ^{{(8)}}(x)\\&\qquad \quad +{\tfrac {1}{1728}}\lambda _{3}^{2}\lambda _{4}\,\Phi ^{{(10)}}(x)+{\tfrac {1}{31104}}\lambda _{3}^{4}\,\Phi ^{{(12)}}(x){\bigg )}\\&+O(n^{{-5/2}})\,.\end{aligned}}$

Here, $<var> Φ </var> (j) (x)$ is the j-th derivative of $<var> Φ </var>(\cdot)$ at point x. Remembering that the derivatives of the density of the normal distribution are related to the normal density by ϕ⁽ⁿ⁾(x) is (-1)ⁿH_n(x)ϕ(x), (where H_n is the Hermite polynomial of order n), this explains the alternative representations in terms of the density function. Blinnikov and Moessner (1998) have given a simple algorithm to calculate higher-order terms of the expansion.

Illustration: density of the sample mean of 3 Χ²

Density of the sample mean of three chi2 variables. The chart compares the true density, the normal approximation, and two edgeworth expansions

Take $X_{i}\sim \chi ^{2}(k=2)\qquad i=1,2,3$ and the sample mean ${\bar X}={\frac {1}{3}}\sum _{{i=1}}^{{3}}X_{i}$ .

We can use several distributions for ${\bar X}$ :

The exact distribution, which follows a gamma distribution: ${\bar X}\sim {\mathrm {Gamma}}\left(\alpha =n\cdot k/2,\theta =2/n\right)$ = ${\mathrm {Gamma}}\left(\alpha =3,\theta =2/3\right)$
The asymptotic normal distribution: ${\bar X}{\xrightarrow {n\to \infty }}N(k,2\cdot k/n)=N(2,4/3)$
Two Edgeworth expansion, of degree 2 and 3

Disadvantages of the Edgeworth expansion

Edgeworth expansions can suffer from a few issues:

They are not guaranteed to be a proper probability distribution as:
- The integral of the density needs not integrate to 1
- Probabilities can be negative
They can be inaccurate, especially in the tails, due to mainly two reasons:
- They are obtained under a Taylor series around the mean
- They guarantee (asymptotically) an absolute error, not a relative one. This is an issue when one wants to approximate very small quantities, for which the absolute error might be small, but the relative error important.

References

↑ Weisstein, Eric W., "Edgeworth Series", MathWorld.

Edgeworth series

Gram–Charlier A series

Edgeworth series

Illustration: density of the sample mean of 3 Χ²

Disadvantages of the Edgeworth expansion

See also

References

Further reading