Normal-Wishart distribution

Normal-Wishart
Notation (\boldsymbol\mu,\boldsymbol\Lambda) \sim \mathrm{NW}(\boldsymbol\mu_0,\lambda,\mathbf{W},\nu)
Parameters \boldsymbol\mu_0\in\mathbb{R}^D\, location (vector of real)
\lambda > 0\, (real)
\mathbf{W} \in\mathbb{R}^{D\times D} scale matrix (pos. def.)
\nu > D-1\, (real)
Support \boldsymbol\mu\in\mathbb{R}^D ; \boldsymbol\Lambda \in\mathbb{R}^{D\times D} covariance matrix (pos. def.)
PDF f(\boldsymbol\mu,\boldsymbol\Lambda|\boldsymbol\mu_0,\lambda,\mathbf{W},\nu) = \mathcal{N}(\boldsymbol\mu|\boldsymbol\mu_0,(\lambda\boldsymbol\Lambda)^{-1})\ \mathcal{W}(\boldsymbol\Lambda|\mathbf{W},\nu)

In probability theory and statistics, the normal-Wishart distribution (or Gaussian-Wishart distribution) is a multivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a multivariate normal distribution with unknown mean and precision matrix (the inverse of the covariance matrix).[1]

Definition

Suppose

\boldsymbol\mu|\boldsymbol\mu_0,\lambda,\boldsymbol\Lambda \sim \mathcal{N}(\boldsymbol\mu|\boldsymbol\mu_0,(\lambda\boldsymbol\Lambda)^{-1})

has a multivariate normal distribution with mean \boldsymbol\mu_0 and covariance matrix (\lambda\boldsymbol\Lambda)^{-1}, where

\boldsymbol\Lambda|\mathbf{W},\nu \sim \mathcal{W}(\boldsymbol\Lambda|\mathbf{W},\nu)

has a Wishart distribution. Then (\boldsymbol\mu,\boldsymbol\Lambda) has a normal-Wishart distribution, denoted as

(\boldsymbol\mu,\boldsymbol\Lambda) \sim \mathrm{NW}(\boldsymbol\mu_0,\lambda,\mathbf{W},\nu) .

Characterization

Probability density function

f(\boldsymbol\mu,\boldsymbol\Lambda|\boldsymbol\mu_0,\lambda,\mathbf{W},\nu) = \mathcal{N}(\boldsymbol\mu|\boldsymbol\mu_0,(\lambda\boldsymbol\Lambda)^{-1})\ \mathcal{W}(\boldsymbol\Lambda|\mathbf{W},\nu)

Properties

Marginal distributions

By construction, the marginal distribution over \boldsymbol\Lambda is a Wishart distribution, and the conditional distribution over \boldsymbol\mu given \boldsymbol\Lambda is a multivariate normal distribution. The marginal distribution over \boldsymbol\mu is a multivariate t-distribution.

Generating normal-Wishart random variates

Generation of random variates is straightforward:

  1. Sample \boldsymbol\Lambda from a Wishart distribution with parameters \mathbf{W} and \nu
  2. Sample \boldsymbol\mu from a multivariate normal distribution with mean \boldsymbol\mu_0 and variance (\lambda\boldsymbol\Lambda)^{-1}

Related distributions

Notes

  1. ↑ Bishop, Christopher M. (2006). Pattern Recognition and Machine Learning. Springer Science+Business Media. Page 690.

References