Sub-gaussian random variables

A number of distributions, notably Gaussian and Bernoulli, are known to satisfy certain concentration of measure inequalities. We will analyze this phenomenon from a more general perspective by considering the class of sub-Gaussian distributions  [link] .

A random variable $X$ is called sub-Gaussian if there exists a constant $c>0$ such that

$$\mathbb{E}\left(exp,\left(,X,t,\right)\right)\le exp(c^2{t}^2/2)$$

holds for all $t\in \mathbb{R}$ . We use the notation $X\sim \mathrm{Sub}\left(c^2\right)$ to denote that $X$ satisfies [link] .

The function $\mathbb{E}\left(exp,\left(,X,t,\right)\right)$ is the moment-generating function of $X$ , while the upper bound in [link] is the moment-generating function of a Gaussian random variable. Thus, a sub-Gaussian distribution is one whose moment-generating function is bounded by that of a Gaussian. There are a tremendous number of sub-Gaussian distributions, but there are two particularly important examples:

A common way to characterize sub-Gaussian random variables is through analyzing their moments. We consider only the mean and variance in the following elementary lemma, proven in  [link] .

(buldygin-kozachenko [link] )

If $X\sim \mathrm{Sub}\left(c^2\right)$ then,

and

[link] shows that if $X\sim \mathrm{Sub}\left(c^2\right)$ then $\mathbb{E}\left(X^2\right)\le c^2$ . In some settings it will be useful to consider a more restrictive class of random variables for which this inequality becomes an equality.

A random variable $X$ is called strictly sub-Gaussian if $X\sim \mathrm{Sub}\left({\sigma }^2\right)$ where ${\sigma }^2=\mathbb{E}\left(X^2\right)$ , i.e., the inequality

$$\mathbb{E}\left(exp,\left(,X,t,\right)\right)\le exp({\sigma }^2{t}^2/2)$$

holds for all $t\in \mathbb{R}$ . To denote that $X$ is strictly sub-Gaussian with variance ${\sigma }^2$ , we will use the notation $X\sim \mathrm{SSub}\left({\sigma }^2\right)$ .

We now provide an equivalent characterization for sub-Gaussian and strictly sub-Gaussian random variables, proven in  [link] , that illustrates their concentration of measure behavior.

(buldygin-kozachenko [link] )

A random variable $X\sim \mathrm{Sub}\left(c^2\right)$ if and only if there exists a $t_0\ge 0$ and a constant $a\ge 0$ such that

for all $t\ge t_0$ . Moreover, if $X\sim \mathrm{SSub}\left({\sigma }^2\right)$ , then [link] holds for all $t>0$ with $a=\sigma$ .

Finally, sub-Gaussian distributions also satisfy one of the fundamental properties of a Gaussian distribution: the sum of two sub-Gaussian random variables is itself a sub-Gaussian random variable. This result is established in more generality in the following lemma.

Suppose that $X=[X_1,X_2,...,X_N]$ , where each $X_i$ is independent and identically distributed (i.i.d.) with $X_i\sim \mathrm{Sub}\left(c^2\right)$ . Then for any $\alpha \in {\mathbb{R}}^N$ , $⟨X,\alpha ⟩\sim \mathrm{Sub}\left(c^2,{\parallel \alpha \parallel }_2^2\right)$ . Similarly, if each $X_i\sim \mathrm{SSub}\left({\sigma }^2\right)$ , then for any $\alpha \in {\mathbb{R}}^N$ , $⟨X,\alpha ⟩\sim \mathrm{SSub}\left({\sigma }^2,{\parallel \alpha \parallel }_2^2\right)$ .

Since the $X_i$ are i.i.d., the joint distribution factors and simplifies as:

If the $X_i$ are strictly sub-Gaussian, then the result follows by setting $c^2={\sigma }^2$ and observing that $\mathbb{E}\left({⟨X,\alpha ⟩}^2\right)={\sigma }^2{\parallel \alpha \parallel }_2^2$ .