Quantiles, Quartiles, and Outliers

The mean and standard deviation are useful, but they can be dramatically affected by unusual values. Sometimes it is more useful to summarize where the middle of the observations sits.

To do that, we sort the data.

Percentiles and quartiles

The $p$th percentile roughly describes the point at or below which $p$ percent of the data lie. The median is the 50th percentile.

The first quartile, $Q_1$, is the 25th percentile; the third quartile, $Q_3$, is the 75th percentile. The interquartile range is

$$\mathrm{IQR}=Q_3-Q_1.$$

It describes the spread of the central portion of the data. We commonly report a median and IQR together for distributions that are skewed or contain extreme observations.

There is a subtlety: for a finite sample, software packages use several different rules for determining percentiles. They can disagree, particularly for small datasets. This is not necessarily a software bug. It is a choice of definition.

One explicitly stated convention

For the laboratories on this site, we use a common linear interpolation rule (sometimes called type 7). For $n$ sorted values $x_{(1)} \le \cdots \le x_{(n)}$, estimate a quantile at probability $p$ by interpolating at zero-based position

$$h=(n-1)p.$$

If $h$ falls between two integers, interpolate between the corresponding adjacent observations.

For example, take

$$2,\;4,\;6,\;8,\;10.$$

For $p=.25$, we have $h=4(.25)=1$, so $Q_1=4$. Similarly, $Q_3=8$, the median is 6, and $\mathrm{IQR}=4$.

This definition is easy to calculate consistently. Other perfectly legitimate textbook conventions may give different quartiles.

Is it an outlier?

A common exploratory rule calls observations outside the inner fences potential outliers:

$$Q_1-1.5\,\mathrm{IQR},\qquad Q_3+1.5\,\mathrm{IQR}.$$

For our five observations, the fences are $4-6=-2$ and $8+6=14$. No value lies beyond them.

Importantly, this is a screening convention, not a scientific definition of an error. A high observation might be completely legitimate. Deleting it automatically can introduce bias and remove the very phenomenon we hope to understand.

A boxplot often shows the median, quartiles, whiskers, and observations outside the chosen fence. Different plotting packages can also use different rules for whiskers.

What not to infer

The interquartile range does not describe every observation. A small IQR does not rule out extreme tails. And quartiles alone cannot tell us whether a distribution has one peak or two.

No single statistic describes everything about a dataset. That is why we use several summaries together.

Check your understanding

1. For the sorted data $2,4,6,8,10$, what are the median, IQR, and upper 1.5-IQR fence using this lesson’s rule?
2. Does an observation beyond the upper fence have to be a measurement error?
3. Why should we state which method produced the sample quartiles when reporting small datasets?

Suggested responses

1. Median 6, IQR 4, upper fence 14. 2. No: it is an observation worth investigating, not automatically a mistake. 3. Multiple accepted algorithms can yield different sample percentiles.

Interpolation and boxplot conventions

For sorted $[1,2,4,8]$, the type-7 first-quartile position is $(4-1)(0.25)=0.75$ in zero-based indexing. Interpolate 75% of the way from 1 to 2, giving $Q_1=1.75$. The third-quartile position is 2.25, yielding $Q_3=5$. Thus IQR is $3.25$, with fences $-3.125$ and $9.875$. The maximum 8 is not flagged.

A boxplot’s whiskers commonly end at the most extreme observed values within the fences, rather than at the numerical fences themselves. In this example whiskers end at 1 and 8. R’s boxplot hinges can differ from type-7 quartiles, so an independently calculated quantile table and a package boxplot can legitimately disagree unless conventions are aligned.

Quantiles are resistant to changing one extreme value because they depend mainly on order positions, but they are not immune to contamination. The median’s resistance is often summarized by an approximately 50% breakdown point in suitable large-sample settings: enough altered observations can move the center arbitrarily far. The phrase “robust” names a specific behavior under perturbation, not a guarantee that the statistic answers the right scientific question.

Additional practice

1. Calculate type-7 quartiles for $[0,4,8,12]$.
2. If the largest observation 12 becomes 120, which quartile changes under this interpolation rule?
3. Why should an outlier analysis preserve an audit of any exclusion?

Worked additional solutions

1. Positions $0.75$ and $2.25$ yield $Q_1=3$, $Q_3=9$, IQR 6.
2. $Q_1$ stays 3; $Q_3=8+0.25(120-8)=36$. Small-sample interpolated quartiles can still depend on an extreme endpoint. Resistance is not complete invariance.
3. Exclusion changes the evidence and possibly the estimand. Record the rule, reason, timing, and sensitivity of conclusions with and without the observation.