Empirical process

For the process control topic, see Empirical process (process control model).

In probability theory, an empirical process is a stochastic process that describes the proportion of objects in a system in a given state. For a process in a discrete state space a population continuous time Markov chain^[1]^[2] or Markov population model^[3] is a process which counts the number of objects in a given state (without rescaling). In mean field theory, limit theorems (as the number of objects becomes large) are considered and generalise the central limit theorem for empirical measures. Applications of the theory of empirical processes arise in non-parametric statistics.^[4]

Definition

For X₁, X₂, ... X_n independent and identically-distributed random variables in R with common cumulative distribution function F(x), the empirical distribution function is defined by

F_n(x)=\frac{1}{n}\sum_{i=1}^n I_{(-\infty,x]}(X_i),

where I_C is the indicator function of the set C.

For every (fixed) x, F_n(x) is a sequence of random variables which converge to F(x) almost surely by the strong law of large numbers. That is, F_n converges to F pointwise. Glivenko and Cantelli strengthened this result by proving uniform convergence of F_n to F by the Glivenko–Cantelli theorem.^[5]

A centered and scaled version of the empirical measure is the signed measure

G_n(A)=\sqrt{n}(P_n(A)-P(A))

It induces a map on measurable functions f given by

f\mapsto G_n f=\sqrt{n}(P_n-P)f=\sqrt{n}\left(\frac{1}{n}\sum_{i=1}^n f(X_i)-\mathbb{E}f\right)

By the central limit theorem, $G_n(A)$ converges in distribution to a normal random variable N(0, P(A)(1 − P(A))) for fixed measurable set A. Similarly, for a fixed function f, $G_nf$ converges in distribution to a normal random variable $N(0,\mathbb{E}(f-\mathbb{E}f)^2)$ , provided that $\mathbb{E}f$ and $\mathbb{E}f^2$ exist.

Definition

\bigl(G_n(c)\bigr)_{c\in\mathcal{C}}

is called an empirical process indexed by

\mathcal{C}

, a collection of measurable subsets of S.

\bigl(G_nf\bigr)_{f\in\mathcal{F}}

is called an empirical process indexed by

\mathcal{F}

, a collection of measurable functions from S to

\mathbb{R}

A significant result in the area of empirical processes is Donsker's theorem. It has led to a study of Donsker classes: sets of functions with the useful property that empirical processes indexed by these classes converge weakly to a certain Gaussian process. While it can be shown that Donsker classes are Glivenko–Cantelli classes, the converse is not true in general.

Example

As an example, consider empirical distribution functions. For real-valued iid random variables X₁, X₂, ..., X_n they are given by

F_n(x)=P_n((-\infty,x])=P_nI_{(-\infty,x]}.

In this case, empirical processes are indexed by a class $\mathcal{C}=\{(-\infty,x]:x\in\mathbb{R}\}.$ It has been shown that $\mathcal{C}$ is a Donsker class, in particular,

\sqrt{n}(F_n(x)-F(x))

converges weakly in

\ell^\infty(\mathbb{R})

to a Brownian bridge B(F(x)) .

References

↑ Bortolussi, L.; Hillston, J.; Latella, D.; Massink, M. (2013). "Continuous approximation of collective systems behaviour: A tutorial". Performance Evaluation. doi:10.1016/j.peva.2013.01.001.
↑ Stefanek, A.; Hayden, R. A.; Mac Gonagle, M.; Bradley, J. T. (2012). "Mean-Field Analysis of Markov Models with Reward Feedback". Analytical and Stochastic Modeling Techniques and Applications. Lecture Notes in Computer Science 7314. p. 193. doi:10.1007/978-3-642-30782-9_14. ISBN 978-3-642-30781-2.
↑ Dayar, T. R.; Hermanns, H.; Spieler, D.; Wolf, V. (2011). "Bounding the equilibrium distribution of Markov population models". Numerical Linear Algebra with Applications 18 (6): 931. doi:10.1002/nla.795.
↑ Mojirsheibani, M. (2007). "Nonparametric curve estimation with missing data: A general empirical process approach". Journal of Statistical Planning and Inference 137 (9): 2733–2758. doi:10.1016/j.jspi.2006.02.016.
↑ Wolfowitz, J. (1954). "Generalization of the Theorem of Glivenko-Cantelli". The Annals of Mathematical Statistics 25: 131. doi:10.1214/aoms/1177728852.

External links

Empirical Processes: Theory and Applications, by David Pollard, a textbook available online.
Introduction to Empirical Processes and Semiparametric Inference, by Michael Kosorok, another textbook available online.

Stochastic processes

Discrete time	Bernoulli process Branching process Chinese restaurant process Galton–Watson process Independent and identically distributed random variables Markov chain Moran process Random walk Loop-erased Self-avoiding

Continuous time	Bessel process Birth–death process Brownian motion Bridge Excursion Fractional Geometric Meander Cauchy process Contact process Continuous-time random walk Cox process Diffusion process Empirical process Feller process Fleming–Viot process Gamma process Hunt process Interacting particle systems Itō diffusion Itō process Jump diffusion Jump process Lévy process Local time Markov additive process McKean–Vlasov process Ornstein–Uhlenbeck process Poisson process Compound Non-homogeneous Point process Schramm–Loewner evolution Semimartingale Sigma-martingale Stable process Superprocess Telegraph process Variance gamma process Wiener process Wiener sausage

Both	Branching process Gaussian process Hidden Markov model (HMM) Markov process Martingale Differences Local Sub- Super- Random dynamical system Regenerative process Renewal process White noise

Fields and other	Dirichlet process Gaussian random field Gibbs measure Hopfield model Ising model Potts model Boolean network Markov random field Percolation Pitman–Yor process Point process Cox Poisson Random field Random graph

Time series models	Autoregressive conditional heteroskedasticity (ARCH) model Autoregressive integrated moving average (ARIMA) model Autoregressive (AR) model Autoregressive–moving-average (ARMA) model Generalized autoregressive conditional heteroskedasticity (GARCH) model Moving-average (MA) model

Financial models	Black–Derman–Toy Black–Karasinski Black–Scholes Chen Constant elasticity of variance (CEV) Cox–Ingersoll–Ross (CIR) Garman–Kohlhagen Heath–Jarrow–Morton (HJM) Heston Ho–Lee Hull–White LIBOR market SABR volatility Vašíček

Actuarial models	Bühlmann Cramér–Lundberg Risk process Sparre–Anderson

Queueing models	Bulk Fluid Generalized queueing network M/G/1 M/M/1 M/M/c

Properties	Càdlàg paths Continuous Continuous paths Ergodic Exchangeable Feller-continuous Gauss–Markov Markov Mixing Piecewise deterministic Predictable Progressively measurable Self-similar Stationary Time-reversible

Limit theorems	Central limit theorem Donsker's theorem Doob's martingale convergence theorems Ergodic theorem Fisher–Tippett–Gnedenko theorem Large deviation principle Law of large numbers (weak/strong) Law of the iterated logarithm Maximal ergodic theorem Sanov's theorem

Inequalities	Burkholder–Davis–Gundy Doob's martingale Kunita–Watanabe

Tools	Cameron–Martin formula Convergence of random variables Doléans-Dade exponential Doob decomposition theorem Doob–Meyer decomposition theorem Doob's optional stopping theorem Dynkin's formula Feynman–Kac formula Filtration Girsanov theorem Infinitesimal generator Itō integral Itō's lemma Kolmogorov continuity theorem Kolmogorov extension theorem Lévy–Prokhorov metric Malliavin calculus Martingale representation theorem Optional stopping theorem Prohorov theorem Quadratic variation Reflection principle Skorokhod integral Skorokhod's representation theorem Skorokhod space Snell envelope Stochastic differential equation Tanaka Stopping time Stratonovich integral Uniform integrability Usual hypotheses Wiener space Classical Abstract

Disciplines	Actuarial mathematics Econometrics Ergodic theory Extreme value theory (EVT) Large deviations theory Mathematical finance Mathematical statistics Probability theory Queueing theory Renewal theory Ruin theory Statistics Stochastic analysis Time series analysis Machine learning

List of topics Category

Empirical process

Definition

Example

See also

References

Further reading

External links