Wishart distribution

From Wikipedia, the free encyclopedia

In statistics, the Wishart distribution, named in honor of John Wishart, is any of a family of probability distributions for nonnegative-definite matrix-valued random variables ("random matrices"). These distributions are of great importance in the estimation of covariance matrices in multivariate statistics.

1 Definition of the Wishart distribution
2 Occurrence of the Wishart distribution
3 Probability density function
4 Characteristic Function
5 Theorem
- 5.1 Corollary 1
- 5.2 Corollary 2
6 Estimator of the multivariate normal distribution
7 See also

[edit] Definition of the Wishart distribution

The definition is as follows. Suppose

$X_1\sim N_p(0,V),$

i.e. X₁ is a p×1 column-vector-valued random variable (a "random vector") that follows a p-variate normal distribution, whose expected value is the p×1 column vector whose entries are all zero, and whose variance is the p×p nonnegative definite matrix V. We have

E (X) = 0

and

var(X) = E ((X - μ)(X - μ)') = E (X X') = V

where the transpose of any matrix A is denoted A′.

Further suppose X₁, ..., X_n are independent and identically distributed (i.i.d.). Then the Wishart distribution is the probability distribution of the p×p random matrix

$S=\sum_{i=1}^n X_i X_i'.$

One indicates that S has that probability distribution by writing

$S\sim W_p(V,n).$

The positive integer n is the number of degrees of freedom.

If p = 1 and V = 1 then this distribution is a chi-square distribution with $n$ degrees of freedom.

[edit] Occurrence of the Wishart distribution

The Wishart distribution arises frequently in likelihood-ratio tests in multivariate statistical analysis.

[edit] Probability density function

The Wishart distribution can be characterized by its probability density function, as follows.

Let ${\mathbf W}$ be a $p\times p$ symmetric matrix of random variables that is positive definite. Let ${\mathbf V}$ be a (fixed) positive definite matrix of size $p\times p$ .

Then, if $n\geq p$ , ${\mathbf W}$ has a Wishart distribution with $n$ degrees of freedom if it has a probability density function $f_{\mathbf W}$ given by

$f_{\mathbf W}(w)= \frac{ \left|w\right|^{(n-p-1)/2} \exp\left[ - {\rm trace}({\mathbf V}^{-1}w/2 )\right] }{ 2^{np/2}\left|{\mathbf V}\right|^{n/2}\Gamma_p(n/2) }$

where $\Gamma_p(\cdot)$ is the multivariate gamma function defined as

$\Gamma_p(n/2)= \pi^{p(p-1)/4}\Pi_{j=1}^p \Gamma\left[ (n+1-j)/2\right].$

In fact the above definition can be extended to any real $n > p - 1$ .

[edit] Characteristic Function

The characteristic function of the Wishart distribution is

$\left|{\mathbf I} - 2i\,{\mathbf\Theta}{\mathbf\Sigma}\right|^{-n/2}.$

In other words,

${\mathcal E}\left\{\mathrm{exp}\left[i\cdot\mathrm{trace}({\mathbf A}{\mathbf\Theta})\right]\right\} = \left|{\mathbf I} - 2i{\mathbf\Theta}{\mathbf\Sigma}\right|^{-n/2}$

where ${\mathcal E}(\cdot)$ denotes expectation.

[edit] Theorem

If ${\mathbf W}$ has a Wishart distribution with $m$ degrees of freedom and variance matrix ${\mathbf V}$ ---write ${\mathbf W}\sim{\mathbf W}_p({\mathbf V},m)$ ---and ${\mathbf C}$ is a $q\times p$ matrix of rank $q$ , then

${\mathbf C}{\mathbf W}{\mathbf C'} \sim {\mathbf W}_q\left({\mathbf C}{\mathbf V}{\mathbf C'},m\right)$

[edit] Corollary 1

If ${\mathbf z}$ is a nonzero $p\times 1$ constant vector, then ${\mathbf z'}{\mathbf W}{\mathbf z}\sim\sigma_z^2\chi_m^2$ .

In this case, $\chi_m^2$ is the chi-square distribution and $\sigma_z^2={\mathbf z'}{\mathbf V}{\mathbf z}$ (note that $\sigma_z^2$ is a constant; it is positive because ${\mathbf V}$ is positive definite).

[edit] Corollary 2

Consider the case where ${\mathbf z'}=(0,\ldots,0,1,0,\ldots,0)$ (that is, the j-th element is one and all others zero). Then corollary 1 above shows that

$w_{jj}\sim\sigma_{jj}\chi^2_m$

gives the marginal distribution of each of the elements on the matrix's diagonal.

Noted statistician George Seber points out that the Wishart distribution is not called the "multivariate chi-square distribution" because the marginal distribution of the off-diagonal elements is not chi-square. Seber prefers to reserve the term multivariate for the case when all univariate marginals belong to the same family.

[edit] Estimator of the multivariate normal distribution

The Wishart distribution is the probability distribution of the maximum-likelihood estimator (MLE) of the covariance matrix of a multivariate normal distribution. The derivation of the MLE is perhaps surprisingly subtle and elegant. It involves the spectral theorem and the reason why it can be better to view a scalar as the trace of a 1×1 matrix than as a mere scalar. See estimation of covariance matrices.

[edit] See also

	Probability distributions [ view • talk • edit ]
	Univariate	Multivariate
Discrete:	Bernoulli • binomial • Boltzmann • compound Poisson • degenerate • Gauss-Kuzmin • geometric • hypergeometric • logarithmic • negative binomial • parabolic fractal • Poisson • Rademacher • Skellam • uniform • Yule-Simon • zeta • Zipf • Zipf-Mandelbrot	Ewens • multinomial
Continuous:	Beta • Beta prime • Cauchy • chi-square • Dirac delta function • Erlang • exponential • exponential power • F • fading • Fisher's z • Fisher-Tippett • Gamma • generalized extreme value • generalized hyperbolic • generalized inverse Gaussian • Half-Logistic • Hotelling's T-square • hyperbolic secant • hyper-exponential • hypoexponential • inverse chi-square • inverse Gaussian • inverse gamma • Kumaraswamy • Landau • Laplace • Lévy • Lévy skew alpha-stable • logistic • log-normal • Maxwell-Boltzmann • Maxwell speed • normal (Gaussian) • Pareto • Pearson • polar • raised cosine • Rayleigh • relativistic Breit-Wigner • Rice • Student's t • triangular • type-1 Gumbel • type-2 Gumbel • uniform • Voigt • von Mises • Weibull • Wigner semicircle • Wilks' lambda	Dirichlet • Kent • matrix normal • multivariate normal • von Mises-Fisher • Wigner quasi • Wishart
Miscellaneous:	Cantor • conditional • exponential family • infinitely divisible • location-scale family • marginal • maximum entropy • phase-type • posterior • prior • quasi • sampling • singular