The Long-Run Average Is Not a Promise

You are offered a game. With probability one-half you win 10 dollars; otherwise you lose 8 dollars. Would you play?

The average outcome over many plays is positive. But that does not mean you’ll win one dollar on your next attempt.

Expected value is probability-weighted

For discrete outcomes $x_i$ with probabilities $p_i$,

$$E[X]=\sum_i x_i p_i.$$

For this game,

$$E[X]=10(.5)+(-8)(.5)=1.$$

The expected gain is one dollar per play. In a single play the gain is either 10 dollars or a loss of 8 dollars; the expected value is not even a possible outcome.

Variance describes the spread

Expectation alone tells us little about risk. A game that always pays one dollar and a game that pays 1,001 dollars or loses 999 dollars with equal probabilities both have expectation one dollar. Their variability is rather different.

The population variance of a discrete random variable is

$$\operatorname{Var}(X)=E[(X-E[X])^2] =\sum_i p_i(x_i-\mu)^2.$$

For the first game with mean $1$, deviations are $9$ and $-9$. Variance is $81$ and standard deviation is $9$.

A useful computational shortcut is $\operatorname{Var}(X)=E[X^2]-E[X]^2$.

Units and linear transformations

For constants $a$ and $b$,

$$E[a+bX]=a+bE[X],\quad \operatorname{Var}(a+bX)=b^2\operatorname{Var}(X).$$

Adding ten dollars changes the mean by ten but leaves the spread unchanged. Multiplying dollar amounts by two doubles the standard deviation and quadruples the variance.

If $X$ and $Y$ are independent, the variance of their sum is the sum of their variances. Without independence, a covariance term is required. Don’t silently assume independence because it makes a formula convenient.

A binomial connection

For $X\sim\mathrm{Binomial}(n,p)$, expectation is $np$ and variance is $np(1-p)$, derived from a sum of independent Bernoulli indicators.

Check yourself

1. If $X$ equals 0 with probability .75 and 8 with probability .25, find $E[X]$.
2. Find the variance and standard deviation of that variable.
3. If all values are multiplied by three, what happens to the variance?

Worked responses

1. $E[X]=2$.

2. Variance is $.75(0-2)^2+.25(8-2)^2=12$, so the standard deviation is $\sqrt{12}\approx3.464$.

3. It is multiplied by $3^2=9$.

Derive the variance identities

Expanding the centered square gives $E[(X-\mu)^2]=E[X^2]-2\mu E[X]+\mu^2=E[X^2]-\mu^2$, assuming the second moment is finite. This identity is algebraically convenient, but direct subtraction of two nearly equal large values can lose floating-point accuracy. Computing centered deviations is often numerically safer.

Linearity of expectation does not require independence: $E[X+Y]=E[X]+E[Y]$ whenever the expectations are defined appropriately. Variance does require attention to dependence:

$$\operatorname{Var}(X+Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)+2\operatorname{Cov}(X,Y).$$

For a difference the covariance term has a minus sign. If $Y=X$, the sum is $2X$ and its variance is $4\operatorname{Var}(X)$, not twice the variance. Independence is sufficient for zero covariance when moments exist, but zero covariance need not imply independence. For example, a symmetric variable $X$ and $Y=X^2$ can have zero covariance despite a deterministic relationship.

A Bernoulli indicator $I$ equals one for success and zero otherwise. Since $I^2=I$, $E[I]=p$ and $\operatorname{Var}(I)=p-p^2$. A binomial count is the sum of $n$ independent indicators with the same success probability, giving mean $np$ and variance $np(1-p)$. For the sample proportion $X/n$, the variance is $p(1-p)/n$. This derivation explains the square-root sample-size denominator in proportion standard errors.

Further problems and solutions

1. If $E[X]=3$ and $\operatorname{Var}(X)=4$, find the mean and variance of $10-2X$.
2. If $X,Y$ each have variance 4 and covariance 3, find the variance of their difference.
3. Why can expected profit alone be insufficient for comparing two investments or games?

Worked further solutions

1. Mean $4$; variance $(-2)^2(4)=16$; standard deviation $4$.
2. $4+4-2(3)=2$.
3. Equal means can accompany different tail risks, loss probabilities, and variances. The full distribution and the decision maker’s constraints matter; expectation is one summary.