Probability-generating function

From Wikipedia, the free encyclopedia

In probability theory, the probability-generating function of a discrete random variable is a power series representation (the generating function) of the probability mass function of the random variable. Probability-generating functions are often employed for their succinct description of the sequence of probabilities Pr(X = i) in the probability mass function for a random variable X, and to make available the well-developed theory of power series with non-negative coefficients.

Definition

Univariate case

If X is a discrete random variable taking values in the non-negative integers {0,1, ...}, then the probability-generating function of X is defined as ^[1]

$G(z)=\operatorname {E}(z^{X})=\sum _{{x=0}}^{{\infty }}p(x)z^{x},$

where p is the probability mass function of X. Note that the subscripted notations G_X and p_X are often used to emphasize that these pertain to a particular random variable X, and to its distribution. The power series converges absolutely at least for all complex numbers z with |z| ≤ 1; in many examples the radius of convergence is larger.

Multivariate case

If X = (X₁,...,X_d ) is a discrete random variable taking values in the d-dimensional non-negative integer lattice {0,1, ...}^d, then the probability-generating function of X is defined as

$G(z)=G(z_{1},\ldots ,z_{d})=\operatorname {E}{\bigl (}z_{1}^{{X_{1}}}\cdots z_{d}^{{X_{d}}}{\bigr )}=\sum _{{x_{1},\ldots ,x_{d}=0}}^{{\infty }}p(x_{1},\ldots ,x_{d})z_{1}^{{x_{1}}}\cdots z_{d}^{{x_{d}}},$

where p is the probability mass function of X. The power series converges absolutely at least for all complex vectors z = (z₁,...,z_d ) ∈ ℂ^d with max{|z₁|,...,|z_d |} ≤ 1.

Properties

Power series

Probability-generating functions obey all the rules of power series with non-negative coefficients. In particular, G(1⁻) = 1, where G(1⁻) = lim_z→1G(z) from below, since the probabilities must sum to one. So the radius of convergence of any probability-generating function must be at least 1, by Abel's theorem for power series with non-negative coefficients.

Probabilities and expectations

The following properties allow the derivation of various basic quantities related to X:

1. The probability mass function of X is recovered by taking derivatives of G

$p(k)=\operatorname {Pr}(X=k)={\frac {G^{{(k)}}(0)}{k!}}.$

2. It follows from Property 1 that if random variables X and Y have probability generating functions that are equal, G_X = G_Y, then p_X = p_Y. That is, if X and Y have identical probability-generating functions, then they have identical distributions.

3. The normalization of the probability density function can be expressed in terms of the generating function by

$\operatorname {E}(1)=G(1^{-})=\sum _{{i=0}}^{\infty }f(i)=1.$

The expectation of X is given by

$\operatorname {E}\left(X\right)=G'(1^{-}).$

More generally, the k^th factorial moment, ${\textrm {E}}(X(X-1)\cdots (X-k+1))$ of X is given by

${\textrm {E}}\left({\frac {X!}{(X-k)!}}\right)=G^{{(k)}}(1^{-}),\quad k\geq 0.$

So the variance of X is given by

$\operatorname {Var}(X)=G''(1^{-})+G'(1^{-})-\left[G'(1^{-})\right]^{2}.$

4. $G_{X}(e^{{t}})=M_{X}(t)$ where X is a random variable, $G_{X}(t)$ is the probability generating function (of X) and $M_{X}(t)$ is the moment-generating function (of X) .

Functions of independent random variables

Probability-generating functions are particularly useful for dealing with functions of independent random variables. For example:

If X₁, X₂, ..., X_n is a sequence of independent (and not necessarily identically distributed) random variables, and

$S_{n}=\sum _{{i=1}}^{n}a_{i}X_{i},$

where the a_i are constants, then the probability-generating function is given by

$G_{{S_{n}}}(z)=\operatorname {E}(z^{{S_{n}}})=\operatorname {E}(z^{{\sum _{{i=1}}^{n}a_{i}X_{i},}})=G_{{X_{1}}}(z^{{a_{1}}})G_{{X_{2}}}(z^{{a_{2}}})\cdots G_{{X_{n}}}(z^{{a_{n}}}).$

For example, if

$S_{n}=\sum _{{i=1}}^{n}X_{i},$

then the probability-generating function, G_Sn(z), is given by

$G_{{S_{n}}}(z)=G_{{X_{1}}}(z)G_{{X_{2}}}(z)\cdots G_{{X_{n}}}(z).$

It also follows that the probability-generating function of the difference of two independent random variables S = X₁ − X₂ is

$G_{S}(z)=G_{{X_{1}}}(z)G_{{X_{2}}}(1/z).$

Suppose that N is also an independent, discrete random variable taking values on the non-negative integers, with probability-generating function G_N. If the X₁, X₂, ..., X_N are independent and identically distributed with common probability-generating function G_X, then

$G_{{S_{N}}}(z)=G_{N}(G_{X}(z)).$

This can be seen, using the law of total expectation, as follows:

$G_{{S_{N}}}(z)=\operatorname {E}(z^{{S_{N}}})=\operatorname {E}(z^{{\sum _{{i=1}}^{N}X_{i}}})=\operatorname {E}{\big (}\operatorname {E}(z^{{\sum _{{i=1}}^{N}X_{i}}}|N){\big )}=\operatorname {E}{\big (}(G_{X}(z))^{N}{\big )}=G_{N}(G_{X}(z)).$

This last fact is useful in the study of Galton–Watson processes.

Suppose again that N is also an independent, discrete random variable taking values on the non-negative integers, with probability-generating function G_N and probability density $f_{i}=\Pr\{N=i\}$ . If the X₁, X₂, ..., X_N are independent, but not identically distributed random variables, where $G_{{X_{i}}}$ denotes the probability generating function of $X_{i}$ , then

$G_{{S_{N}}}(z)=\sum _{{i\geq 1}}f_{i}\prod _{{k=1}}^{i}G_{{X_{i}}}(z).$

For identically distributed X_i this simplifies to the identity stated before. The general case is sometimes useful to obtain a decomposition of S_N by means of generating functions.

Examples

The probability-generating function of a constant random variable, i.e. one with Pr(X = c) = 1, is

$G(z)=\left(z^{c}\right).\,$

The probability-generating function of a binomial random variable, the number of successes in n trials, with probability p of success in each trial, is

$G(z)=\left[(1-p)+pz\right]^{n}.\,$

Note that this is the n-fold product of the probability-generating function of a Bernoulli random variable with parameter p.

The probability-generating function of a negative binomial random variable on {0,1,2 ...}, the number of failures until the rth success with probability of success in each trial p, is

$G(z)=\left({\frac {p}{1-(1-p)z}}\right)^{r}.$

(Convergence for $|z|<{\frac {1}{1-p}}$ ).

Note that this is the r-fold product of the probability generating function of a geometric random variable with parameter 1−p on {0,1,2 ...}.

The probability-generating function of a Poisson random variable with rate parameter λ is

$G(z)={\textrm {e}}^{{\lambda (z-1)}}.\;\,$

Related concepts

The probability-generating function is an example of a generating function of a sequence: see also formal power series. It is occasionally called the z-transform of the probability mass function.

Other generating functions of random variables include the moment-generating function, the characteristic function and the cumulant generating function.

Notes

↑ http://www.am.qub.ac.uk/users/g.gribakin/sor/Chap3.pdf

References

Johnson, N.L.; Kotz, S.; Kemp, A.W. (1993) Univariate Discrete distributions (2nd edition). Wiley. ISBN 0-471-54897-9 (Section 1.B9)

v t e Theory of probability distributions

probability mass function (pmf) probability density function (pdf) cumulative distribution function (cdf) quantile function

raw moment central moment mean variance standard deviation skewness kurtosis L-moment

moment-generating function (mgf) characteristic function probability-generating function (pgf) cumulant combinant

This article is issued from Wikipedia. The text is available under the Creative Commons Attribution/Share Alike; additional terms may apply for the media files.