Three Means Enter, One F Statistic Leaves

Two means? We can compare them. Three means? We could conduct three pairwise tests, but that invites the multiplicity problem and avoids a more fundamental question.

Do the groups show more separation than we’d expect from the variation within them?

That is the central question in a one-way analysis of variance, or ANOVA.

Separate two kinds of variation

Suppose fictional outcomes are

• Group A: $2,3,4$
• Group B: $4,5,6$
• Group C: $6,7,8$

The group means are 3, 5, and 7. The grand mean is 5.

The between-group sum of squares weights how far each group mean lies from the grand mean:

$$SS_B=\sum_{g=1}^{G}n_g(\bar x_g-\bar x)^2.$$

Here each group has three observations, so $SS_B=3(4+0+4)=24$.

The within-group sum of squares adds the squared deviations of observations from their own group means:

$$SS_W=\sum_g\sum_{i\in g}(x_i-\bar x_g)^2.$$

Each group contributes $1^2+0^2+1^2=2$, giving $SS_W=6$.

Construct the F ratio

With three groups and nine total observations, the between-group degrees of freedom are $G-1=2$ and within-group degrees of freedom are $n-G=6$.

The mean squares are

$$MS_B=24/2=12,\qquad MS_W=6/6=1.$$

Therefore $F=12/1=12$.

Under the classical normal independent-observation model with equal group variances and a null of equal group means, this statistic has an F reference distribution with 2 and 6 degrees of freedom.

What an omnibus rejection means

If the global equality hypothesis is rejected, we have evidence that not all population group means are equal under the model.

It does not automatically tell us exactly which pair differs. Planned contrasts or appropriately adjusted pairwise comparisons answer more specific questions.

Classical ANOVA assumes independent observations, an appropriate within-group error distribution, and equal variances. When spreads differ substantially, Welch’s ANOVA or another design-appropriate method may be preferable.

R demonstration

score <- c(2,3,4,4,5,6,6,7,8)
group <- factor(rep(c("A","B","C"), each=3))
summary(aov(score ~ group))

Then inspect group means and variance. Never use an F statistic as a substitute for understanding which participants entered each group.

Check your understanding

1. Why isn’t $F=12$ itself a probability?
2. What does rejecting equal means across three groups establish?
3. If observations are repeated on the same people, is independent one-way ANOVA automatically appropriate?

Worked responses

1. It is a ratio of mean squares; the p-value is calculated from its reference distribution.

2. Evidence that at least one population mean differs, not identification of every differing pair.

3. No. Repeated measurements introduce dependence that requires a suitable design-aware method.

Prove the decomposition and complete the inference

Write each deviation as $x_{ig}-\bar x=(x_{ig}-\bar x_g)+(\bar x_g-\bar x)$. Squaring and summing makes the cross term vanish because $\sum_i(x_{ig}-\bar x_g)=0$ within each group. Therefore $SS_T=SS_W+SS_B$. In the example, total squared deviation is $30$, partitioned into within-group $6$ and between-group $24$.

The completed analysis table is:

Source	Sum of squares	Degrees of freedom	Mean square	F
Between groups	24	2	12	12
Within groups	6	6	1
Total	30	8

The upper-tail probability $P(F_{2,6}\geq12)=0.008$. Under the stated equal-variance independent normal-error model, a prespecified 5% omnibus test rejects equality of all three population means. The illustrative sample is tiny, so assumption assessment cannot be delegated to a reassuring software label. The observed group means and spreads must accompany the result.

The descriptive effect fraction $\eta^2=SS_B/SS_T=24/30=0.8$ describes between-group separation in this sample. It is not evidence that group membership caused 80% of individual variation, and it can overestimate a population explained fraction in small samples. Adjusted measures and uncertainty require their own definitions.

A planned contrast asks a more specific question. For independent groups with common variance estimated by $MS_W$, a contrast $L=\sum_gc_g\bar x_g$, with $\sum_gc_g=0$, has estimated standard error $\sqrt{MS_W\sum_gc_g^2/n_g}$. Comparing C with A here gives $L=4$ and $SE=\sqrt{2/3}\approx0.81650$. If many contrasts are selected or searched, the family of comparisons must be accounted for. An omnibus rejection does not identify them automatically.

Additional practice

1. Show the total degrees of freedom equals the sum of the two component degrees of freedom.
2. If every outcome is increased by 10, what happens to $F$? If every outcome is multiplied by 2?
3. Why does a global rejection not establish that every pair differs?

Worked additional solutions

1. $(G-1)+(n-G)=n-1$.
2. Adding a constant leaves all centered deviations unchanged. Multiplying by two multiplies both sums of squares and both mean squares by four; their ratio remains the same.
3. The null is a conjunction of all group-mean equalities. Its rejection establishes that at least one equality fails under the model, not which ones or how many.