Opening
We have eight fictional participants and exactly seventy ways to choose four for treatment A. The remarkable thing is that an exact test is suddenly small enough to inspect entirely.
Teaching narrative
Under a sharp null of no individual treatment effect, outcomes remain fixed while treatment labels vary across the assignments permitted by the experiment's randomization mechanism.
Every allocation's statistic can be calculated. The proportion of equally likely allocations as extreme as the observed result gives an exact design-based p-value, avoiding a normal approximation in this tiny example.
The bootstrap asks a different question. It repeatedly samples with replacement from the empirical data, approximating sampling variability under that empirical distribution. It does not create genuinely new observations from the wider population.
These methods are powerful because the computation makes assumptions visible. Permuting observational labels does not grant an observational study randomized causal validity, and resampling a biased sample does not remove the bias.
Before the experiment
What does the bootstrap fail to learn about people who never entered the original sample?
Live demonstration
Enumerate the seventy possible allocations and their differences, then resample the waiting-time dataset with replacement 1,000 times.
Exit question
Contrast the source of randomness in the exact randomization test and in bootstrap resampling.