Rectified Gaussian distribution

In probability theory, the rectified Gaussian distribution is a modification of the Gaussian distribution when its negative elements are reset to 0 (analogous to an electronic rectifier). It is essentially a mixture of a discrete distribution (constant 0) and a continuous distribution (a truncated Gaussian distribution with interval (0,\infty)).

Density function

The probability density function of a rectified Gaussian distribution, for which random variables X having this distribution are displayed as X \sim \mathcal{N}^{\textrm{R}}(\mu,\sigma^2) , is given by

f(x;\mu,\sigma^2) =\Phi(-\frac{\mu}{\sigma})\delta(x)+ \frac{1}{\sqrt{2\pi\sigma^2}}\; e^{ -\frac{(x-\mu)^2}{2\sigma^2}}\textrm{U}(x).
A comparison of Gaussian distribution, rectified Gaussian distribution, and truncated Gaussian distribution.

Here, \Phi(x) is the cumulative distribution function (cdf) of the standard normal distribution:

\Phi(x) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^x e^{-t^2/2} \, dt \quad x\in\mathbb{R},

\delta(x) is the Dirac delta function

\delta(x) = \begin{cases} +\infty, & x = 0 \\ 0, & x \ne 0 \end{cases}

and, \textrm{U}(x) is the unit step function:

\textrm{U}(x)=\begin{cases} 0, & x \leq 0, \\ 1, & x > 0. \end{cases}

Alternative form

Often, a simpler alternative form is to consider a case, where,

s\sim\mathcal{N}(\mu,\sigma^2),x=\textrm{max}(0,s),

then,

x\sim\mathcal{N}^{\textrm{R}}(\mu,\sigma^2)

Application

A rectified Gaussian distribution is semi-conjugate to the Gaussian likelihood, and it has been recently applied to factor analysis, or particularly, (non-negative) rectified factor analysis. Harva [1] proposed a variational learning algorithm for the rectified factor model, where the factors follow a mixture of rectified Gaussian; and later Meng [2] proposed an infinite rectified factor model coupled with its Gibbs sampling solution, where the factors follow a Dirichlet process mixture of rectified Gaussian distribution, and applied it in computational biology for reconstruction of gene regulatory network.

References

  1. ↑ Harva, M.; Kaban, A. (2007). "Variational learning for rectified factor analysis☆". Signal Processing 87 (3): 509. doi:10.1016/j.sigpro.2006.06.006.
  2. ↑ Meng, Jia; Zhang, Jianqiu (Michelle), Chen, Yidong, Huang, Yufei (1 January 2011). "Bayesian non-negative factor analysis for reconstructing transcription factor mediated regulatory networks". Proteome Science 9 (Suppl 1): S9. doi:10.1186/1477-5956-9-S1-S9.