Law of total expectation

The proposition in probability theory known as the law of total expectation,^[1] the law of iterated expectations, the tower rule, the smoothing theorem, and Adam's Law among other names, states that if $X$ is an integrable random variable (i.e. $\operatorname {E} |X|<\infty )$ and $Y$ is any random variable, not necessarily integrable, on the same probability space, then

\operatorname {E} (X)=\operatorname {E} (\operatorname {E} (X\mid Y)),

i.e., the expected value of the conditional expected value of $X$ given $Y$ is the same as the expected value of $X$ .

One special case states that if ${\left\{A_{i}\right\}}_{i}$ is a finite or countable partition of the sample space, then

\operatorname {E} (X)=\sum _{i}{\operatorname {E} (X\mid A_{i})\operatorname {P} (A_{i})}.

Example

Suppose that two factories supply light bulbs to the market. Factory $X$ 's bulbs work for an average of 5000 hours, whereas factory $Y$ 's bulbs work for an average of 4000 hours. It is known that factory $X$ supplies 60% of the total bulbs available. What is the expected length of time that a purchased bulb will work for?

Applying the law of total expectation, we have:

$\operatorname{E} (L) = \operatorname{E}(L \mid X) \operatorname{P}(X)+\operatorname{E}(L \mid Y) \operatorname{P}(Y) = 5000(.6)+4000(.4)=4600$

where

$\operatorname{E} (L)$ is the expected life of the bulb;
$\operatorname {P} (X)={6 \over 10}$ is the probability that the purchased bulb was manufactured by factory $X$ ;
$\operatorname {P} (Y)={4 \over 10}$ is the probability that the purchased bulb was manufactured by factory $Y$ ;
$\operatorname{E}(L \mid X)=5000$ is the expected lifetime of a bulb manufactured by $X$ ;
$\operatorname{E}(L \mid Y)=4000$ is the expected lifetime of a bulb manufactured by $Y$ .

Thus each purchased light bulb has an expected lifetime of 4600 hours.

Proof in the finite and countable cases

{\begin{aligned}\operatorname {E} \left(\operatorname {E} (X\mid Y)\right)&=\operatorname {E} {\Bigg [}\sum _{x}x\cdot \operatorname {P} (X=x\mid Y){\Bigg ]}\\[6pt]&=\sum _{y}{\Bigg [}\sum _{x}x\cdot \operatorname {P} (X=x\mid Y=y){\Bigg ]}\cdot \operatorname {P} (Y=y)\\[6pt]&=\sum _{y}\sum _{x}x\cdot \operatorname {P} (X=x,Y=y).\end{aligned}}

If both summations are finite, then we can switch them around, and the previous expression will become

{\begin{aligned}\sum _{x}\sum _{y}x\cdot \operatorname {P} (X=x,Y=y)&=\sum _{x}x\sum _{y}\operatorname {P} (X=x,Y=y)\\[6pt]&=\sum _{x}x\cdot \operatorname {P} (X=x)\\[6pt]&=\operatorname {E} (X),\end{aligned}}

so we are done.

If at least one of the two summations is infinite, we still can switch their order without changing the sum if the series converges absolutely. We prove absolute convergence by observing that

{\begin{aligned}\sum _{y}\sum _{x}{\Bigl |}x\cdot \operatorname {P} (X=x,Y=y){\Bigr |}&=\sum _{y}\sum _{x}|x|\cdot \operatorname {P} (X=x,Y=y)\\[6pt]&=\sum _{x}\sum _{y}|x|\cdot \operatorname {P} (X=x,Y=y)\\[6pt]&=\sum _{x}|x|\cdot \operatorname {P} (X=x)\\[6pt]&=E|X|<\infty ,\end{aligned}}

as required. (The last calculation once again uses the fact that the members of an absolutely converging series can be transposed arbitrarily without altering the sum).

Proof in the general case

Let $(\Omega ,{\mathcal {F}},\operatorname {P} )$ be a probability space on which two sub σ-algebras $\mathcal{G}_1 \subseteq \mathcal{G}_2 \subseteq \mathcal{F}$ are defined. For a random variable $X$ on such a space, where $\operatorname {E} |X|<\infty$ , the smoothing law states that

\operatorname {E} [\operatorname {E} [X\mid {\mathcal {G}}_{2}]\mid {\mathcal {G}}_{1}]=\operatorname {E} [X\mid {\mathcal {G}}_{1}]\quad {\text{(a.s.)}}.

Proof. Since a conditional expectation is a Radon–Nikodym derivative, verifying the following two properties establishes the smoothing law:

$\operatorname{E}[ \operatorname{E}[X \mid \mathcal{G}_2] \mid \mathcal{G}_1] \mbox{ is } \mathcal{G}_1$ -measurable
$\int _{G_{1}}\operatorname {E} [\operatorname {E} [X\mid {\mathcal {G}}_{2}]\mid {\mathcal {G}}_{1}]d\operatorname {P} =\int _{G_{1}}Xd\operatorname {P}$ , for all $G_{1}\in {\mathcal {G}}_{1}$ , and both integrals exist.

The first of these properties holds by the definition of the conditional expectation. To prove the second one, note that

\int _{G_{1}}|X|d\operatorname {P} \leq \int _{\Omega }|X|d\operatorname {P} =\operatorname {E} |X|<\infty ,

so the integral $\int _{G_{1}}Xd\operatorname {P}$ on the right-hand side exists for every $G_{1}\in {\mathcal {G}}_{1}$ . The second property thus holds since $G_1 \in \mathcal{G}_1 \subseteq \mathcal{G}_2$ implies

\int _{G_{1}}\operatorname {E} [\operatorname {E} [X\mid {\mathcal {G}}_{2}]\mid {\mathcal {G}}_{1}]d\operatorname {P} =\int _{G_{1}}\operatorname {E} [X\mid {\mathcal {G}}_{2}]d\operatorname {P} =\int _{G_{1}}Xd\operatorname {P} .

Corollary. In the special case when $\mathcal{G}_1 = \{\empty,\Omega \}$ and $\mathcal{G}_2 = \sigma(Y)$ , the smoothing law reduces to

\operatorname{E}[ \operatorname{E}[X \mid Y]] = \operatorname{E}[X].

Proof of partition formula

{\begin{aligned}\sum \limits _{i}\operatorname {E} (X\mid A_{i})\operatorname {P} (A_{i})&=\sum \limits _{i}\int \limits _{\Omega }X(\omega )\operatorname {P} (d\omega \mid A_{i})\cdot \operatorname {P} (A_{i})\\&=\sum \limits _{i}\int \limits _{\Omega }X(\omega )\operatorname {P} (d\omega \cap A_{i})\\&=\sum \limits _{i}\int \limits _{\Omega }X(\omega )I_{A_{i}}(\omega )\operatorname {P} (d\omega )\\&=\sum \limits _{i}\operatorname {E} (XI_{A_{i}}),\end{aligned}}

where $I_{A_{i}}$ is the indicator function of the set $A_{i}$ .

If the partition ${\{A_{i}\}}_{i=0}^{n}$ is finite, then, by linearity, the previous expression becomes

\operatorname {E} \left(\sum \limits _{i=0}^{n}XI_{A_{i}}\right)=\operatorname {E} (X),

and we are done.

If, however, the partition ${\{A_{i}\}}_{i=0}^{\infty }$ is infinite, then we use the dominated convergence theorem to show that

\operatorname {E} \left(\sum \limits _{i=0}^{n}XI_{A_{i}}\right)\to \operatorname {E} (X).

Indeed, for every $n\geq 0$ ,

\left|\sum _{i=0}^{n}XI_{A_{i}}\right|\leq |X|I_{\mathop {\cup } \limits _{i=0}^{n}A_{i}}\leq |X|.

Since every element of the set $\Omega$ falls into a specific partition $A_{i}$ , it is straightforward to verify that the sequence ${\left\{\sum _{i=0}^{n}XI_{A_{i}}\right\}}_{n=0}^{\infty }$ pointwise-converges to $X$ . By initial assumption, $\operatorname {E} |X|<\infty$ . Applying the dominated convergence theorem yields the desired.

References

↑ Weiss, Neil A. (2005). A Course in Probability. Boston: Addison–Wesley. pp. 380–383. ISBN 0-321-18954-X.

Billingsley, Patrick (1995). Probability and measure. New York: John Wiley & Sons. ISBN 0-471-00710-2. (Theorem 34.4)
Christopher Sims, "Notes on Random Variables, Expectations, Probability Densities, and Martingales", especially equations (16) through (18)

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.