Association, Confounding, and Causal Claims

Suppose people who spend more time exercising tend to live longer. Should everyone exercise? That’s a plausible public-health recommendation, but the association itself is not enough to establish the size of exercise’s causal effect.

People who exercise may differ from people who do not in age, existing health, socioeconomic conditions, diet, access to care, and countless other ways. Those differences complicate the leap from association to cause.

Statistics can identify relationships. Causal inference asks what would happen if we actually intervened.

The counterfactual question

For a treatment $T$ with two levels, imagine that every person has two potential outcomes: $Y(1)$ if treated and $Y(0)$ if untreated.

An individual causal effect is

$$Y(1)-Y(0).$$

The challenge is immediate: we generally observe only one of these outcomes for the same person under the same circumstances.

This is sometimes called the fundamental problem of causal inference. The missing counterfactual is not a software bug we can patch by collecting another variable.

Instead, we use research designs and assumptions to make comparisons informative.

Why random assignment matters

In a randomized experiment, treatment assignment is governed by a known random mechanism rather than participant choice or investigator preference.

That doesn’t guarantee equal measured characteristics in a particular small sample. What it does is remove systematic selection into treatment in expectation and provide a principled basis for estimating uncertainty from the assignment mechanism.

Randomization is not the same thing as random sampling:

• Random sampling helps generalize appropriately from a sample to a population.
• Random assignment helps identify treatment effects within the study by reducing systematic baseline differences between treatment groups.

An experiment can have randomized assignment among volunteers while those volunteers are not representative of the broader population.

Confounding: the third-variable problem

A confounder is a variable that complicates causal comparison because it is associated with treatment choice and relevant to the outcome, in a way that can create or distort the observed association.

Suppose one hospital uses a particular treatment mainly for difficult cases, while another treatment is used mostly for mild cases. Pooled treatment success rates are not comparing similar patient mixes.

In our Simpson’s paradox example, stone size changes the way the observed treatment rates compare when the data are stratified.

But stratification itself doesn’t prove which method causes better outcomes. Other relevant variables could still differ. The historical data were not a randomized head-to-head trial.

Statistical adjustment helps, but cannot conjure missing evidence

Methods such as regression adjustment, stratification, matching, and weighting can improve observational comparisons under appropriate assumptions.

They cannot automatically remove the influence of unmeasured confounders. Adjustment for variables measured after treatment can even introduce new biases. A sophisticated model is not a substitute for understanding the causal question and how the data arose.

When planning research, ask: What exactly is the intervention, who receives it, how are groups formed, what outcomes are measured, and which conclusions will the design support?

Communicating results responsibly

A good causal claim should say more than “the p-value was significant.” It should explain the population, design, comparison, outcome, effect estimate, uncertainty, and most important limitations.

A cautious result is not a weak result. Sometimes the most useful answer is that the available evidence does not identify the causal effect we hoped to measure.

Exercise

1. Explain the distinction between random sampling and random assignment.
2. Why might controlling for observed stone size fail to establish causation in the historical treatment comparison?
3. What additional design information would you want before interpreting a study’s regression slope causally?

Worked responses

1. Sampling is about selection from a target population; assignment is about how interventions are distributed among participants.

2. Treatment groups could differ in other observed or unobserved characteristics and time periods, even within size strata.

3. How treatment or exposure was chosen; possible confounding, temporal order, measurement validity, potential interference, sample selection, and the population to which conclusions would apply.

Read further

The ASA GAISE recommendations call for statistical thinking that integrates data context, ethical practice, and well-scoped communication. For broader introductory inference and applications, see OpenStax Introductory Statistics 2e.

Define the effect before selecting a regression

For person $i$, let $Y_i(1)$ and $Y_i(0)$ be outcomes under treatment and control. Their individual causal effect is $Y_i(1)-Y_i(0)$, but ordinarily only one potential outcome is observed. The sample average treatment effect averages those unobserved differences over recruited people; a population effect averages over a specified target population. They need not coincide.

Random assignment makes treatment independent of potential outcomes under the design, allowing unbiased comparisons in expectation for suitable estimators and assignment schemes. It does not guarantee exact balance in the realized experiment. A causal interpretation also needs a sufficiently defined treatment and assumptions about interference: one person’s assignment must not silently alter another’s outcome in a model that assumes independent treatment exposure.

For observational adjustment, a common identifying assumption is conditional exchangeability: within measured covariates X, treatment assignment is unrelated to potential outcomes. Positivity requires both treatment possibilities to occur with positive probability in the covariate groups relevant to the target. Consistency connects the observed outcome to the potential outcome for the treatment actually received. These assumptions are stronger than a regression’s ability to produce coefficients and cannot all be checked from the observed association table.

Further problems

1. Explain why comparing volunteers who choose treatment with those who decline leaves a counterfactual problem.
2. What would positivity failure look like if every person in one age group received treatment?
3. Distinguish a variable measured before treatment from an outcome caused by treatment when considering adjustment.

Worked further solutions

1. The groups may differ in what their outcomes would have been even under the same treatment. Only one outcome per person is observed.
2. There are no untreated comparisons in that group; the data cannot directly identify its untreated outcome distribution without extrapolation or additional assumptions.
3. A pretreatment confounder may help make comparisons exchangeable. A post-treatment mediator lies on a causal pathway; adjusting for it can remove part of the total effect or create other bias.