Variance

In the treatment of the mathematical expection of a real random variable X , we note that the mean value locates the center of the probability mass distribution induced by X on the real line. In this unit, we examine how expectation may be used for further characterizationof the distribution for X . In particular, we deal with the concept of variance and its square root the standard deviation . In subsequent units, we show how it may be used to characterize the distribution for a pair $\{X,Y\}$ considered jointly with the concepts covariance , and linear regression

Variance

Location of the center of mass for a distribution is important, but provides limited information. Two markedly different random variables may have the same mean value.It would be helpful to have a measure of the spread of the probability mass about the mean. Among the possibilities, the variance and its square root, the standard deviation,have been found particularly useful.

Definition . The variance of a random variable X is the mean square of its variation about the mean value:

The standard deviation for X is the positive square root σ _X of the variance.

Remarks

• If $X\left(\omega \right)$ is the observed value of X , its variation from the mean is $X\left(\omega \right)-{\mu }_X$ . The variance is the probability weighted average of the square of these variations.
• The square of the error treats positive and negative variations alike, and it weights large variations more heavily than smaller ones.
• As in the case of mean value, the variance is a property of the distribution, rather than of the random variable.
• We show below that the standard deviation is a “natural” measure of the variation from the mean.
• In the treatment of mathematical expectation, we show that
  $$E\left[{(X-c)}^2\right]\text{is}\text{a}\text{minimum}\text{iff}c=E\left[X\right],\text{in}\text{which}\text{case}E\left[{(X-E\left[X\right])}^2\right]=E\left[X^2\right]-E^2\left[X\right]$$
  This shows that the mean value is the constant which best approximates the random variable, in the mean square sense.

Basic patterns for variance

Since variance is the expectation of a function of the random variable X , we utilize properties of expectation in computations. In addition, we find it expedient to identifyseveral patterns for variance which are frequently useful in performing calculations. For one thing, while the variance is defined as $E\left[{(X-{\mu }_X)}^2\right]$ , this is usually not the most convenient form for computation. The result quoted above gives an alternate expression.

• (V1) Calculating formula . $\mathrm{Var}\left[X\right]=E\left[X^2\right]-E^2\left[X\right]$ .
• (V2) Shift property . $\mathrm{Var}[X+b]=\mathrm{Var}\left[X\right]$ . Adding a constant b to X shifts the distribution (hence its center of mass) by that amount. The variation of the shifted distribution about the shifted centerof mass is the same as the variation of the original, unshifted distribution about the original center of mass.
• (V3) Change of scale . $\mathrm{Var}\left[aX\right]=a^2\mathrm{Var}\left[X\right]$ . Multiplication of X by constant a changes the scale by a factor $\left|a\right|$ . The squares of the variations are multiplied by a ² . So also is the mean of the squares of the variations.
• (V4) Linear combinations
  1. ${\displaystyle \mathrm{Var}[aX\pm bY]=a^2\mathrm{Var}\left[X\right]+b^2\mathrm{Var}\left[Y\right]\pm 2ab\left(E,\left[,X,Y,\right],-,E,\left[,X,\right],E,\left[,Y,\right]\right)}$
  2. More generally,
     $$\mathrm{Var}\left[\sum _{k=1}^n,a_k,X_k\right]=\sum _{k=1}^n{a}_k^2\mathrm{Var}\left[X_k\right]+2\sum _{i<j}{a}_i{a}_j\left(E,\left[X_i{X}_j\right],-,E,\left[X_i\right],E,\left[X_j\right]\right)$$
  The term $c_{ij}=E\left[X_i{X}_j\right]-E\left[X_i\right]E\left[X_j\right]$ is the covariance of the pair $\{X_i,X_j\}$ , whose role we study in the unit on that topic. If the $c_{ij}$ are all zero, we say the class is uncorrelated .