A Poll's Margin of Error Is Not Magic

A poll finds four supporters among twenty respondents. Our observed proportion is $4/20=.20$.

Can we conclude that twenty percent of the population supports the proposal? Only if the sample design permits that inference. And even then, twenty observations carry substantial random uncertainty.

The deceptively simple interval

A textbook may suggest the approximate interval

$$\hat p\pm1.96\sqrt{\frac{\hat p(1-\hat p)}{n}}.$$

This Wald interval is memorable and notoriously unreliable in small samples or when the true proportion lies near zero or one. In particular, if none of twenty trials succeeds, it returns $[0,0]$: a ludicrous assertion of certainty.

The Wilson score alternative

A commonly preferable approximate interval is the Wilson score interval. With $z=1.96$, sample size $n$, and observed proportion $\hat p$, define

$$c=\frac{\hat p+z^2/(2n)}{1+z^2/n}, \qquad h=\frac{z}{1+z^2/n} \sqrt{\frac{\hat p(1-\hat p)}{n}+\frac{z^2}{4n^2}}.$$

The interval is $[c-h,c+h]$. For four successes in twenty independent binomial trials, a 95% Wilson interval is approximately [0.081, 0.416].

Notice how wide that is. Twenty observations leave open a considerable range of possible population proportions.

At zero observed successes, Wilson still produces a positive upper bound instead of falsely declaring the event impossible.

The assumptions are not an optional footnote

A binomial proportion requires independent Bernoulli observations with a common success probability under the model. In a clustered or weighted survey, its variance will often require a design-based calculation.

No interval, Wilson or otherwise, can fix systematic response bias. If the twenty respondents were gathered outside a political rally, the method’s mathematical sophistication does not confer representativeness.

Reproduce it

R’s prop.test performs a score-based procedure with its own implementation details, including a continuity-correction option. Software procedures need to be identified precisely rather than indiscriminately called “the 95% interval.”

prop.test(4, 20, correct = FALSE)

Compare that answer with the Wilson calculation above.

Check your understanding

1. Find the sample proportion for 18 successes in 50 trials.
2. Why should a zero-success sample not necessarily have a zero-width interval?
3. Does choosing a different interval estimator eliminate selection bias?

Worked responses

1. 0.36.

2. An unobserved event can still have a positive underlying probability.

3. No. The sampling mechanism remains a distinct issue.

Further reading: R prop.test documentation.

Wilson’s interval comes from inverting a test

For an independent binomial sample with observed proportion $\hat p=X/n$, a score test of a candidate $p$ uses $(\hat p-p)/\sqrt{p(1-p)/n}$. Retain candidate parameters satisfying

$$(\hat p-p)^2\leq z^2p(1-p)/n.$$

Multiplying out gives $(n+z^2)p^2-(2n\hat p+z^2)p+n\hat p^2\leq0$. Solving this quadratic produces the Wilson endpoints already displayed. The denominator contains the candidate $p$ under the test, rather than replacing it immediately by $\hat p$. This inversion explains why Wilson avoids the zero-width failure of the Wald interval when all outcomes agree.

For zero successes in ten trials and $z\approx1.96$, Wilson’s lower endpoint is zero and its upper endpoint is $z^2/(10+z^2)\approx0.2775$. The sample proportion is zero, yet a wide range of positive risks remains compatible with this small sample under the procedure. If every trial succeeds, symmetry gives a lower endpoint near $0.7225$ and upper endpoint one.

No nonrandomized binomial interval has exact nominal coverage at every parameter because its sample space is discrete. Wilson is generally better behaved than the elementary Wald interval, not universally exact. The Clopper–Pearson construction instead inverts binomial tail probabilities and is conservative in the usual coverage sense. Choose the method for its properties, and label it rather than simply calling every range a “95% interval.”

Additional practice

1. Calculate the Wilson center for $X=4$, $n=20$, $z=1.96$.
2. Explain the Wald failure for $X=0$.
3. Does using Wilson repair biased recruitment?

Worked additional solutions

1. $(0.2+3.8416/40)/(1+3.8416/20)\approx0.24834$, shifted toward one-half compared with the raw proportion.
2. Its estimated standard error $\sqrt{\hat p(1-\hat p)/n}$ is zero, producing a misleading zero-width interval despite a finite sample.
3. No. Interval calibration presumes an appropriate sampling model. It does not supply representativeness or independence absent from the design.