Association and Correlation

Suppose you measure how many hours students studied and their examination scores. If higher study time tends to accompany higher scores, the variables are associated.

Now for the critical part: that does not automatically mean the extra study time caused higher scores.

Students differ in prior preparation, difficulty of coursework, sleep, access to resources, and many other ways. Statistics can help us identify patterns. Inferring causes requires more thought.

A scatterplot preserves the pairs

A dataset containing $n$ pairs of observations $(x_i,y_i)$ can be drawn as a scatterplot. Each point represents a case.

A plot can suggest a positive association, negative association, curved relationship, clusters, or no obvious pattern. Before calculating a single coefficient, inspect the graph.

If the relationship is curved, a measure of linear association may summarize it poorly.

Pearson’s correlation

The sample Pearson correlation coefficient is

$$r=\frac{\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})} {\sqrt{\sum_{i=1}^{n}(x_i-\bar{x})^2\,\sum_{i=1}^{n}(y_i-\bar{y})^2}}.$$

For nonconstant paired data, $r$ lies between $-1$ and $1$.

• Near $+1$ indicates a strong positive linear association.
• Near $-1$ indicates a strong negative linear association.
• Near $0$ indicates weak linear association; there might still be a pronounced nonlinear relationship.

Correlation is unchanged if we add a constant to either variable or multiply both by positive constants. It is not immune to outliers.

If either variable has zero variability, Pearson correlation is undefined. There is simply no direction of linear covariation to standardize.

The case of one surprising observation

Imagine seven observations following a clear upward trend. Now add an eighth observation very far above or below the trend.

That one observation can change the correlation dramatically. A statistic summarizing the entire sample may look convincing even when one data point exerts disproportionate influence.

Try the laboratory below. Move the last observation vertically, keeping the first seven fixed. Watch the graph and the correlation value together.

In a real analysis, this would be a reason to investigate data collection, fit diagnostics, and the scientific context—not to discard a point because the result is inconvenient.

A final warning

A correlation between ice cream sales and swimming-related emergencies might be associated with season or temperature rather than ice cream causing emergencies. An observational association alone cannot identify the correct causal explanation.

We will return to causal interpretation, confounding, and regression in the later modeling unit. For now, the most important habit is simple: look at the paired data before telling a story about the coefficient.

Check your understanding

1. If one variable is constant, what happens to Pearson’s $r$?
2. Does $r=0$ necessarily imply the variables are independent?
3. If moving one point greatly changes $r$, what should the analyst do next?

Suggested responses

1. It is undefined. 2. No: relationships can be nonlinear, and zero correlation alone does not imply independence. 3. Examine the point, data quality, plots, model assumptions, and substantive context; do not remove it reflexively.

Centering, scaling, and the limits of a single number

Define $S_{xx}=\sum(x_i-\bar x)^2$, $S_{yy}=\sum(y_i-\bar y)^2$, and $S_{xy}=\sum(x_i-\bar x)(y_i-\bar y)$. Pearson’s correlation is $r=S_{xy}/\sqrt{S_{xx}S_{yy}}$, provided both sums of squares are positive. Cauchy–Schwarz gives $|r|\leq1$. If either variable is constant, the denominator is zero and correlation is undefined, not zero.

For pairs $(-1,1),(0,0),(1,1)$, the response is exactly $y=x^2$. Yet $S_{xy}=0$, so $r=0$. Zero linear correlation does not imply absence of a relationship. A scatterplot should precede or accompany the coefficient.

Adding a constant to a variable leaves correlation unchanged. Multiplying one variable by a positive constant leaves it unchanged; multiplying by a negative constant reverses its sign. Thus changing centimetres to metres does not change $r$, while reversing a scale’s orientation can. The regression slope changes magnitude under unit changes because it has response-per-predictor units; correlation is dimensionless.

Additional practice

1. Compute $r$ for $(1,2),(2,4),(3,6)$.
2. What is $r$ if every response in those pairs is negated?
3. Why can a single influential point increase or decrease $r$ dramatically in a small sample?

Worked additional solutions

1. The points lie exactly on a line with positive slope, giving $r=1$; direct centered sums give the same result.
2. $r=-1$.
3. The coefficient uses all centered cross-products and squared deviations. A distant point can dominate those sums. Investigate its provenance and scientific role rather than deleting it merely to obtain a preferred coefficient.