Density Is Not Probability

A probability density can be taller than one. This surprises people who were just told that probabilities lie between zero and one.

There is no contradiction: a density is not a probability. It is a function whose area over a range gives a probability.

Probability lives in an area

For a continuous random variable with density $f$,

$$P(a\le X\le b)=\int_a^b f(x)\,dx.$$

The total area is one. The probability of one exact point is zero under an absolutely continuous distribution, even at the point where the density is tallest.

For a uniform variable between 0 and 2, $f(x)=1/2$ inside that interval. The probability of falling between .5 and 1.5 is the width 1 multiplied by density .5, giving .5.

A perfectly flat model

For $X\sim U(a,b)$, each equal-width interval inside the range has equal probability. The density is $1/(b-a)$.

Expected value is $(a+b)/2$ and variance is $(b-a)^2/12$. Uniform randomness is useful as a model and as a simulation ingredient, but many real physical measurements are anything but uniform.

Waiting for an event

In a simple Poisson process with rate $\lambda$ events per unit time, the time $T$ until the next event has an exponential distribution:

$$f(t)=\lambda e^{-\lambda t},\ t\ge0;\quad P(T>t)=e^{-\lambda t}.$$

If the event rate is two per hour, the chance of waiting more than half an hour is $e^{-1}\approx.3679$.

The exponential model is memoryless: conditional on waiting already, the remaining waiting-time distribution has the same form. This is a strong modeling assumption, not a universal law of waiting rooms.

Normality is one model among many

A normal distribution also has a density, but unlike a uniform or exponential density, it has a symmetric bell shape across the real line.

We should decide which model to use from the science and observed patterns, not because one curve is easier to integrate or has appeared on more classroom slides.

Check yourself

1. What is $P(.2<X<.7)$ when $X\sim U(0,1)$?
2. Why is $P(X=1)=0$ for a continuous model not saying that the value can never be observed?
3. For an exponential waiting time with rate 3/hour, calculate $P(T>1\text{ hour})$.

Worked responses

1. $.7-.2=.5$.

2. A continuous model assigns probability to intervals rather than positive mass to individual exact points.

3. $e^{-3}\approx.0498$.

A density is not a point probability

A probability density is nonnegative and integrates to one over its domain. Its units are inverse units of the variable, so multiplying by a small interval width produces a dimensionless approximate probability. A density can exceed one: the uniform density on $[0,0.2]$ is 5, yet its total area is exactly one. For a continuous variable, $P(X=x)=0$ at each single point. Zero point probabilities do not imply zero probability for an interval containing uncountably many points.

For $X\sim U(a,b)$, $E[X]=(a+b)/2$ and $\operatorname{Var}(X)=(b-a)^2/12$. These follow by integrating $xf(x)$ and $(x-E[X])^2f(x)$. For a uniform variable on $[2,8]$, the mean is 5 and variance 3. The chance of lying in $[3,6]$ is its length divided by total length, $3/6=1/2$.

For exponential waiting time with rate $r>0$, the survival function is $P(T>t)=e^{-rt}$ for $t\geq0$. Consequently

$$P(T>s+t\mid T>s)=\frac{e^{-r(s+t)}}{e^{-rs}}=e^{-rt}.$$

This is the memoryless property. It is a model implication, not a universal description of waiting. Aging equipment and scheduled service intervals usually require different assumptions. The mean is $1/r$ and variance $1/r^2$; doubling the rate halves expected waiting time.

The cumulative distribution function $F(x)=P(X\leq x)$ gives interval probabilities by $F(b)-F(a)$ for $a<b$ in the continuous case. For a differentiable CDF, its derivative is the density. CDFs are bounded by zero and one; densities need not be.

Further problems

1. For a uniform variable on $[10,14]$, find mean, variance, and $P(X>13)$.
2. For an exponential rate of 2 per hour, find the chance of waiting more than another hour after already waiting two hours.
3. Explain why a probability density of 4 at a point does not violate probability axioms.

Worked further solutions

1. Mean 12, variance $16/12=4/3$, and probability $1/4$.
2. $e^{-2}\approx0.13534$, by memorylessness under the exponential model.
3. The density is probability per unit of the variable, not a point probability. Area over an interval, not height, must lie in the probability range.