Markov kernel

In probability theory, a Markov kernel (or stochastic kernel) is a map that plays the role, in the general theory of Markov processes, that the transition matrix does in the theory of Markov processes with a finite state space.^[1]^[2]

Formal definition

Let $(X,\mathcal A)$ , $(Y,\mathcal B)$ be measurable spaces. A Markov kernel with source $(X,\mathcal A)$ and target $(Y,\mathcal B)$ is a map $\kappa \colon X \times \mathcal B \to [0,1]$ with the following properties:

The map $x \mapsto \kappa(x,B)$ is $\mathcal A$ - measureable for every $B \in \mathcal B$ .
The map $B \mapsto \kappa(x,B)$ is a probability measure on $(Y, \mathcal B)$ for every $x \in X$ .

(i.e. It associates to each point $x \in X$ a probability measure $\kappa(x,.)$ on $(Y,\mathcal B)$ such that, for every measurable set $B\in\mathcal B$ , the map $x\mapsto \kappa(x,B)$ is measurable with respect to the $\sigma$ -algebra $\mathcal A$ .)

Examples

Simple random walk: Take $X=Y=\Z$ and $\mathcal A = \mathcal B = \mathcal P(\Z)$ , then the Markov kernel $\kappa$ with

\kappa(x,B)=\frac{1}{2}\delta_{x-1}(B)+\frac{1}{2}\delta_{x+1}(B), \quad \forall x \in \Z \forall B \in \mathcal P(\Z)

describes the transition rule for the random walk on $\Z$ .

Galton-Watson process: Take $S=Y=\N$ , $\mathcal A = \mathcal B = \mathcal P(\N)$ , then

\kappa(x,B)=\begin{cases} \delta_0(B) & \quad x=0,\\ P[\xi_1 + \dots + \xi_x \in B] & \quad \text{else,}\\ \end{cases}

with i.i.d. random variables $\xi_i$ .

General Markov processes with finite state space: Take $X=Y$ , $\mathcal A = \mathcal B = \mathcal P(X) = \mathcal P(Y)$ and $|X|=|Y|=n$ , then the transition rule can be represented as a stochastic matrix $(K_{ij})_{1 \leq i,j \leq n}$ with $\Sigma_{j \in Y}K_{ij}=1$ for every $i \in X$ . In the convention of Markov kernels we write

\kappa(i,B)=\Sigma_{j \in B}K_{ij} \quad \forall i \in X \forall B \in \mathcal B

Properties

Semidirect product

Let $(X, \mathcal A, P)$ be a probability space and $\kappa$ a Markov kernel from $(X, \mathcal A)$ to some $(Y, \mathcal B)$ . Then there exists a unique measure $Q$ on $(X \times Y, \mathcal A \otimes \mathcal B)$ , s.t.

Q(A \times B) = \int_A \kappa(x,B)dP(x), \quad \forall A \in \mathcal A \forall B \in \mathcal B

Regular conditional distribution

Let $(S,Y)$ be a Borel space, $X$ a $(S,Y)$ - valued random variable on the measure space $(\Omega, \mathcal F,P)$ and $\mathcal G \subseteq \mathcal F$ a sub- $\sigma$ -algebra. Then there exists a Markov kernel $\kappa$ from $(\Omega, \mathcal G)$ to $(S,Y)$ , s.t. $\kappa(.,B)$ is a version of the conditional expectation $E[\mathbf 1_{\{X \in B\}}| \mathcal G]$ for every $B \in Y$ , i.e.

P[X \in B|\mathcal G]=E[\mathbf 1_{\{X \in B\}}|\mathcal G]=\kappa(\omega,B), \quad P-a.s. \forall B \in \mathcal G

It is called regular conditional distribution of $X$ given $\mathcal G$ and is not uniquely defined.

References

↑ Epstein, P.; Howlett, P.; Schulze, M. S. (2003). "Distribution dynamics: Stratification, polarization, and convergence among OECD economies, 1870–1992". Explorations in Economic History 40: 78. doi:10.1016/S0014-4983(02)00023-2.
↑ Reiss, R. D. (1993). "A Course on Point Processes". Springer Series in Statistics. doi:10.1007/978-1-4613-9308-5. ISBN 978-1-4613-9310-8.

Bauer, Heinz (1996), Probability Theory, de Gruyter, ISBN 3-11-013935-9

§36. Kernels and semigroups of kernels