Talagrand's concentration inequality

In probability theory, Talagrand's concentration inequality is an isoperimetric-type inequality for product probability spaces.[1][2] It was first proved by the French mathematician Michel Talagrand.[3] The inequality is one of the manifestations of the concentration of measure phenomenon.[2]

Statement

The inequality states that if \Omega = \Omega_1 \times \Omega_2 \times \cdots \times \Omega_n is a product space endowed with a product probability measure and A is a subset in this space, then for any t \ge 0

\Pr[A] \cdot \Pr\left[\overline{A_t}\right] \le e^{-t^2/4} \, ,

where \overline{A_t} is the complement of At where this is defined by

A_t = \{ x \in \Omega ~:~ \rho(A,x) \le t \}

and where \rho is Talagrand's convex distance defined as

\rho(A,x) = \max_{\alpha, \|\alpha\|_2 \le 1} \ \min_{y \in A} \ \sum_{i~:~x_i \neq y_i} \alpha_i

where \alpha \in \mathbf{R}^n, x,y \in \Omega are n-dimensional vectors with entries \alpha_i, x_i, y_i respectively and \|\cdot\|_2 is the \ell^2-norm. That is,

\|\alpha\|_2=\left(\sum_i\alpha_i^2\right)^{1/2}

Explanation

Probability deals with uncertainty. The inference of a future event or process requires the identification of key variables related to it. Certain assumptions about the nature of those variables (random variables) and their functional relationship can be drawn. Yet there are cases where the functional relationships of such variables are not directly available. But in spite of such difficulty Talagrand's concentration inequality helps us to describe some properties (in probabilistic terms) related to the variables an their functional representation. The authors [4] clearly describe what the Talagrand's concentration inequality implies under certain assumptions-"..if a function of many independent random variables does not depend too much on any of the variables then it is concentrated in the sense that with high probability, it is close to its expected value..."- and by such implication, we can for example (under many other possible applications), quantify the empirical risk of the model we are using to forecast some future event.

References

  1. Alon, Noga; Spencer, Joel H. (2000). The Probabilistic Method (2nd ed.). John Wiley & Sons, Inc. ISBN 0-471-37046-0.
  2. 2.0 2.1 Ledoux, Michel (2001). The Concentration of Measure Phenomenon. American Mathematical Society. ISBN 0-8218-2864-9.
  3. Talagrand, Michel (1995). Concentration of measure and isoperimetric inequalities in product spaces. Publications Mathématiques de L'IHÉS (Springer-Verlag). doi:10.1007/BF02699376. ISSN 0073-8301.
  4. Boucheron, Stephanie;Lugosi,Gábor;jMassart,Pascal (2013).Concentration Inequalities: A Nonasymptotic Theory of Independence.OUP Oxford.ASIN: B00BPS6QQI.