Observed information

From Wikipedia, the free encyclopedia

In statistics, the Observed Information is the negative of the second derivative of the log-likelihood.

[edit] Definition

Suppose we observe random variables $X_1,\ldots,X_n$ , independent and identically distributed with density f(X; θ), where θ is a (possibly unknown) vector. Then the log-likelihood of the parameters $θ$ given the data $X_1,\ldots,X_n$ is

$l(\theta | X_1,\ldots,X_n) = \sum_{i=1}^n \log f(X_i| \theta)$ .

We define the Observed Information Matrix at $θ *$ as

$\mathcal{J}(\theta^*) = - \left. \nabla \nabla^{\top} l(\theta) \right|_{\theta=\theta^*}$

$= - \left. \left( \begin{array}{cccc} \tfrac{\partial^2}{\partial \theta_1^2} & \tfrac{\partial^2}{\partial \theta_1 \partial \theta_2} & \cdots & \tfrac{\partial^2}{\partial \theta_1 \partial \theta_n} \\ \tfrac{\partial^2}{\partial \theta_2 \partial \theta_1} & \tfrac{\partial^2}{\partial \theta_2^2} & \cdots & \tfrac{\partial^2}{\partial \theta_2 \partial \theta_n} \\ \vdots & \vdots & \ddots & \vdots \\ \tfrac{\partial^2}{\partial \theta_n \partial \theta_1} & \tfrac{\partial^2}{\partial \theta_n \partial \theta_2} & \cdots & \tfrac{\partial^2}{\partial \theta_n^2} \\ \end{array} \right) l(\theta) \right|_{\theta = \theta^*}$