Learning from Data
All models are wrong, but some are useful.
— George Box, statistician
In probability, we know how the dice are loaded and ask what will happen. In statistics, we see what happened and ask how the dice must be loaded. Probability reasons from causes to effects; statistics reasons back from effects to causes. Every estimate of a population, every opinion poll, every clinical trial and every scientific measurement is a statistical inference: a conclusion about the world drawn from limited, noisy data.
Statistics is a young subject compared with most of mathematics. Its modern form was created in the early twentieth century by Karl Pearson, Ronald Fisher, Jerzy Neyman and others, often driven by practical problems: brewing beer (the \(t\)-test was invented at the Guinness brewery), growing crops, testing medicines. Today, with the explosion of data, statistics and its close relative machine learning are among the most important applications of mathematics.
Why statistics matters
Decisions about medicines, public health, economic policy, climate and justice all rest on statistical evidence. Misreading it has real consequences: a patient frightened by a false positive (Section 12.2), a policy based on a chance fluctuation (Section 13.1), a computer system whose small errors pile up (Section 13.6). Understanding what data can and cannot tell you, and how uncertain any conclusion is, is part of being an informed citizen, and a professional necessity for scientists, doctors, engineers and economists.
What you will meet at university
A first course in statistics covers estimation (finding good estimates of unknown quantities and saying how precise they are), hypothesis testing (deciding whether data are consistent with a hypothesis), confidence intervals, regression (relating one variable to others), and the design of experiments and surveys. The central ideas are sampling distributions (an estimate is a random variable), bias and variance, and likelihood. Later courses add Bayesian inference, multivariate analysis, time series and the methods of machine learning, which are statistics at enormous scale.
The questions that run through this chapter
Is it real or is it noise? How much variation should we expect by chance, and when is a change worth taking seriously?
How should we estimate? Given data, what is the best estimate of an unknown quantity, and how do we compare different methods?
How does randomness enter the data? What do the ways data are collected, protected or processed do to our conclusions?
The chapter starts with reading newspaper statistics critically (Section 13.1), then turns to estimation: counting voles by capture–recapture, estimating a total from serial numbers, and comparing the mean with the median. It ends with three questions about how randomness enters data: randomised surveys that protect privacy, rounding errors that accumulate or cancel, and the construction of variables with prescribed correlations.
Ideas in this chapter
| § | Topic | Idea |
|---|---|---|
| 13.1 | Reading statistics | Rates, natural variation, trends |
| 13.2 | Capture–recapture | Maximum likelihood |
| 13.3 | Serial numbers | Bias, variance and sufficiency |
| 13.4 | Estimating a centre | Efficiency and robustness |
| 13.5 | Randomised response | Mixtures and total variance |
| 13.6 | Rounding errors | Systematic versus random error |
| 13.7 | Correlation | The Cholesky construction |
Section 13.7 uses covariance and correlation, which were removed from the STEP specification in 2019; it explains what it needs.
How to read this chapter
Remember that every estimate is a random variable: if the data had come out differently, so would the estimate. For each method in this chapter, ask what its distribution would be over many repetitions: is it right on average, and how much does it vary? That single question is the heart of statistical thinking.