A Statistical Investigation from Question to Reproducible Report

You have reached the end of the textbook portion of this course. We can now ask a more interesting question than whether you remember the formula for a standard error.

Can you conduct a statistical investigation from beginning to end without deceiving yourself or your reader?

That is the capstone.

The research brief

Start with a question that can be answered using genuinely observed, documented data. The provided starting dataset is R’s historical airquality records from New York in 1973.

One suitable question is:

Among the observed days with complete measurements, how does recorded ozone concentration relate to temperature, wind, and solar radiation?

That is a descriptive and model-based question. It is deliberately not “What is the causal effect of increasing temperature by one degree on ozone?” You do not have an experiment manipulating the weather.

Alternatively, use another verifiably sourced dataset whose use is permitted, with a data dictionary and enough observations to support a suitable analysis.

1. Frame the investigation

Identify the population you would like to understand and the actual observational units represented by the records. Define the response variable, predictors, time range, measurement units, and how observations were selected.

Write down which claim the design could support. If the evidence is observational, explain which sources of confounding, selection, or temporal structure prevent you from assuming causality.

2. Audit the source

For the provided R dataset:

data("airquality", package = "datasets")
dim(airquality)
colSums(is.na(airquality))
summary(airquality)

You should find 153 rows. There are 37 missing ozone values and seven missing solar-radiation values. A model containing ozone, solar radiation, wind, and temperature uses only 111 complete observations with the default missingness handling.

Explain what was lost, what remains, and why missingness may matter for generalization.

3. Explore before modeling

Make at least two appropriate visualizations. One should show the response variable’s distribution; another should investigate its association with a predictor or the structure of missingness.

Summarize center and spread with justified choices. If the data are skewed, explain why the median or quantiles might complement a mean. Label every plotted unit and source.

Do not allow the default graphing software to decide your scientific interpretation.

4. Specify and calculate

For an air-quality analysis, a plausible model is:

$$E(\mathrm{Ozone}\mid \mathrm{Temp},\mathrm{Wind},\mathrm{Solar.R}) =\beta_0+\beta_1\mathrm{Temp} +\beta_2\mathrm{Wind}+\beta_3\mathrm{Solar.R}.$$

You can reproduce it in R:

required <- c("Ozone", "Temp", "Wind", "Solar.R")
complete <- na.omit(airquality[required])
fit <- lm(Ozone ~ Temp + Wind + Solar.R, data = complete)
summary(fit)
plot(fitted(fit), resid(fit))
abline(h=0, lty=2)

Identify the number of analytic observations, estimated coefficients with their units, and at least one appropriate uncertainty summary. Inspect the residual plot and discuss violations or ambiguities.

To compare the temperature-only model with the multivariable model, fit both on the same complete-case records. Otherwise a changed coefficient can be due to both a new specification and a changed sample.

5. Test an alternative explanation

What happens if you add month as a factor? What happens if you use only temperature and wind? Would a curved temperature term improve the descriptive fit? You need not try every conceivable model. Choose one scientifically motivated sensitivity analysis and explain what question it answers.

Disclose every alternative you tried. Do not simply select the model with the most appealing p-value and pretend that it was the original plan.

6. Write an actual report

Submit a concise report with:

1. Research question and design. What are you estimating, for whom, and from which observations?
2. Source and provenance. Who recorded the data? When? How can someone recover the source?
3. Data audit. Observation count, missingness, exclusions, definitions, and transformations.
4. Visual exploration. Two figures with captions that stand on their own.
5. Methods. Equations or algorithms, assumptions, and why they fit the question.
6. Results. Estimates, units, uncertainty, and the principal findings.
7. Sensitivity and limitations. One alternative analysis, possible bias, and the distinction between association and causation.
8. Reproducible code. An R script that runs on a clean installation with the specified dataset.

The reproducibility requirement is not optional. If the script cannot recreate the results from the documented source, revise the submission.

Scoring rubric (100 points)

Criterion	Points
Well-scoped empirical question and design	10
Verified provenance, dictionary, and reproducibility	15
Missingness and data-quality audit	15
Appropriate, clearly labeled visualizations	10
Correct statistical method and assumptions	20
Correct interpretation of estimates and uncertainty	15
Sensitivity analysis, ethics, and limitations	10
Clear, concise, transparent communication	5
Total	100

A report should not receive full credit on the methods or interpretation criteria if it makes unsupported causal claims, misreports denominators, or conceals exclusions.

Final self-critique

Before submitting, answer these questions in a short paragraph:

• Which conclusion would a statistically inexperienced reader be most likely to draw incorrectly?
• What important information is missing from your dataset?
• Would you be comfortable sharing the source and code with someone determined to prove you wrong?
• If you had resources for one better follow-up study, how would you design it?

What completion means

Completing this project is stronger evidence of competence than clicking through a lecture. Yet it is not a licensed university credential. The surrounding project is independent educational material, and an instructor would still need to assess the submitted work.

The outcome we want is not merely a student who can calculate. It is someone who can look at a quantitative claim, ask where the data came from, and know exactly what the mathematics does—and does not—establish.

Starting sources

• Official R documentation: airquality
• ASA GAISE College recommendations
• Course data field notebook
• R Statistics Workbench

A reference structure for the final report

Use a title that names the actual question, followed by an abstract stating the design, data, main estimate, uncertainty, and principal limitation. In the methods section define the unit of observation, inclusion rules, missingness handling, transformations, estimand, model assumptions, and any planned comparisons. A reader should be able to reconstruct which rows produced every denominator.

The results should begin with the analysis sample and its descriptive distribution, then present the primary estimate with units and uncertainty. Separate a primary model from exploratory extensions. Explain at least one plausible alternative explanation or modeling choice and report how it changes the conclusion. A discussion should distinguish the population directly described by the observations from any broader population or causal claim being considered.

Include an executable script, a data dictionary, source and rights information, and a short computational checklist. Confirm that code can run in a clean session, tables agree with the prose, and a changed seed does not overturn a Monte Carlo conclusion merely because too few repetitions were used. Where exact computation is possible, compare it with simulation.

A self-contained design critique

A hypothetical city posts an online questionnaire and receives 2,000 replies, 70% supporting a park. A report declares that citywide support is $70\%\pm2\%$. The respondent proportion is descriptive evidence. Its generalization to residents requires a selection and response model. A textbook independent-sample margin does not account for self-selection merely because the reply count is large. A defensible capstone would identify the unsupported inferential step, state what the data do establish, and propose a sampling design or sensitivity analysis rather than attach a more complicated formula to the same unsupported claim.

Final oral-defense prompts

1. What exactly is your estimand, and which population does it concern?
2. Which assumption would most damage your conclusion if false?
3. Which numerical result can another reader independently verify?
4. What additional data would most reduce the uncertainty you actually face?

What a satisfactory defense demonstrates

There is no single numerical answer. A satisfactory response ties the estimand to the design, identifies assumptions beyond software syntax, reproduces at least one calculation from documented inputs, and distinguishes sampling precision from bias or causal uncertainty. More observations are useful only if they address the relevant information gap.