Quantum relative entropy

From Wikipedia, the free encyclopedia

In quantum information theory, quantum relative entropy is a measure of distinguishability between two quantum states. It is the quantum mechanical analog of relative entropy.

Contents

[edit] Motivation

For simplicity, it will be assumed that all objects in the article are finite dimensional.

We first discuss the classical case. Suppose the probabilities of a finite sequence of events is given by the probability distribution P = {p1...pn}, but somehow we mistakenly assumed it to be Q = {q1...qn}. For instance, we can mistake an unfair coin for a fair one. According to this erroneous assumption, our uncertainty about the j-th event, or equivalently, the amount of information provided after observing the j-th event, is

\; - \log q_j.

The (assumed) average uncertainty of all possible events is then

\; - \sum_j p_j \log q_j.

On the other hand, the Shannon entropy of the probability distribution p, defined by

\; - \sum_j p_j \log p_j,

is the real amount of uncertainty before observation. Therefore the difference between these two quantities

\; - \sum_j p_j \log q_j - (- \sum_j p_j \log p_j) = \sum_j p_j \log p_j - \sum_j p_j \log q_j

is a measure of the distinguishability of the two probability distributions p and q. This is precisely the classical relative entropy, or Kullback–Leibler divergence:

D_{\mathrm{KL}}(P\|Q) = \sum_j p_j \log \frac{p_j}{q_j} \!.

Note

  1. In the definitions above, the convention that 0·log 0 = 0 is assumed, since limx → 0 x log x = 0. Intuitively, one would expect that an event of zero probability to contribute nothing towards entropy.
  2. The relative entropy is not a metric. For example, it is not symmetric. The uncertainty discrepancy in mistaking a fair coin to be unfair is not the same as the opposite situation.

[edit] Definition

As with many other objects in quantum information theory, quantum relative entropy is defined by extending the classical definition from probability distributions to density matrices. Let ρ be a density matrix. The von Neumann entropy of ρ, which is the quantum mechanical analaog of the Shannon entropy, is given by

S(\rho) = - \operatorname{Tr} \rho \log \rho.

For two density matrices ρ and σ, the quantum relative entropy of ρ with respect to σ is defined by

S(\rho \| \sigma) = - \operatorname{Tr} \rho \log \sigma - S(\rho) = \operatorname{Tr} \rho \log \rho - \operatorname{Tr} \rho \log \sigma = \operatorname{Tr}\rho (\log \rho - \log \sigma).

We see that, when the states are classical, i.e. ρσ = σρ, the definition coincides with the classical case.

[edit] Non-finite relative entropy

In general, the support of a matrix M, denoted by supp(M), is the orthogonal complement of its kernel. When consider the quantum relative entropy, we assume the convention that - s· log 0 = ∞ for any s > 0. This leads to the definition that

S(\rho \| \sigma) = \infty

when

supp(\rho) \cap supp(\sigma)^{\perp} \neq {0}.

This makes physical sense. Informally, the quantum relative entropy is a measure of our ability to distinguish two quantum states. But orthogonal quantum states can always be distinguished, via projective measurement. In the present context, this is reflected by non-finite quantum relative entropy.

In the interpretation given in the previous section, if we erroneously assume the state ρ has support in supp(ρ)⊥, this is an error impossible to recover from.

[edit] Klein's inequality

[edit] Corresponding classical statement

For the classical Kullback–Leibler divergence, it can be shown that

D_{\mathrm{KL}}(P\|Q) = \sum_j p_j \log \frac{p_j}{q_j} \geq 0,

and equality holds if and only if P = Q. Colloquially, this means that the uncertainty calculated using erroneous assumptions is always greater than the real amount of uncertainty.

To show the inequality, we rewrite

D_{\mathrm{KL}}(P\|Q) = \sum_j p_j \log \frac{p_j}{q_j} = \sum_j (- \log \frac{q_j}{p_j})(p_j).

Notice that log is a concave function. Therefore -log is convex. Applying Jensen's inequality to -log gives

D_{\mathrm{KL}}(P\|Q) = \sum_j (- \log \frac{q_j}{p_j})(p_j) \geq - \log ( \sum_j \frac{q_j}{p_j} p_j ) = 0.

Jensen's inequality also states that equality holds if and only if, for all i, qi = (∑qj) pi, i.e. p = q.

[edit] The result

Klein's inequality states that the quantum relative entropy

S(\rho \| \sigma) = \operatorname{Tr}\rho (\log \rho - \log \sigma).

is non-negative in general. It is zero if and only ρ = σ.

Proof

Let ρ and σ have spectral decompositions

\rho = \sum_i p_i v_i v_i ^* \; , \; \sigma = \sum_i q_i w_i w_i ^*.

So

\log \rho = \sum_i (\log p_i) v_i v_i ^* \; , \; \log \sigma = \sum_i (\log q_i)w_i w_i ^*.

Direct calculation gives

S(\rho \| \sigma)
= \sum_k p_k \log p_k - \sum_{i,j} (p_i \log q_j) | v_i ^* w_j |^2
= \sum_i p_i ( \log p_i - \sum_j \log q_j | v_i ^* w_j |^2)
\;= \sum_i p_i (\log p_i - \sum_j (\log q_j )P_{ij}), where Pi j = |vi*wj|2.

Since the matrix (Pi j)i j is a doubly stochastic matrix and -log is a concave function, the above expression is

\geq \sum_i p_i (\log p_i - \log (\sum_j q_j P_{ij})
\; = \sum_i p_i (\log p_i - \log (\sum_j q_j P_{ij}).

Define ri = ∑jqj Pi j. Then {ri} is a probability distribution. From the non-negativity of classical relative entropy, we have

S(\rho \| \sigma) \geq \sum_i p_i \log \frac{p_i}{r_i} \geq 0.

The second part of the claim follows from the fact that, since -log is strictly convex, equality is achieved in

\sum_i p_i (\log p_i - \sum_j (\log q_j )P_{ij}) \geq \sum_i p_i (\log p_i - \log (\sum_j q_j P_{ij})

if and only if (Pi j) is a permutation matrix, which implies ρ = σ, after a suitable labeling of the eigenvectors {vi} and {wi}.

[edit] An entanglement measure

Let a composite quantum system have state space

H = \otimes _k H_k

and ρ be a density matrix acting on H.

The relative entropy of entanglement of ρ is defined by

\; D_{\mathrm{REE}} (\rho) = \min_{\sigma} S(\rho \| \sigma)

where the minimum is taken over the family of separable states. A physical interpretation of the quantity is the optimal distinguishability of the state ρ from separable states.

Clearly, when ρ is not entangled

\; D_{\mathrm{REE}} (\rho) = 0

by Klein's inequality.