Reasoning About Chance
Probability theory is nothing but common sense reduced to calculation.
— Pierre-Simon Laplace, Théorie analytique des probabilités (1812)
Laplace’s famous remark is half true. Probability is common sense made precise; but, as this chapter shows again and again, common sense left to itself goes wrong. People think a positive test for a rare disease means you probably have it; that the second passenger to board a plane is as badly off as the last; that a sequence of coin tosses cannot beat another sequence of the same length; that the average of many measurements always settles down. All of these are false, and in each case a short calculation reveals the truth and, better still, the reason for it.
Probability began in the seventeenth century with questions about gambling, in the letters between Blaise Pascal and Pierre de Fermat. It became a branch of rigorous mathematics only in 1933, when Andrey Kolmogorov set out its axioms. In between, and since, it has grown into one of the most widely used parts of mathematics.
Why probability matters
Every field that deals with uncertainty uses probability: medicine (diagnostic tests and clinical trials), law (the weight of evidence), finance (risk and insurance), engineering (reliability and floods), computer science (randomised algorithms, cryptography, machine learning), physics (quantum mechanics and statistical mechanics) and biology (genetics and epidemics). Modern artificial intelligence is, in large part, applied probability: a language model predicts the probability of the next word. And the ability to reason correctly about risk, chance and evidence is one of the most valuable skills any educated person can have.
What you will meet at university
First-year probability covers the axioms, conditional probability and Bayes’ theorem, independence, discrete and continuous random variables, expectation and variance, generating functions, and the two great limit theorems: the law of large numbers and the central limit theorem. Later courses study random processes (Markov chains, Poisson processes, random walks, branching processes), measure-theoretic probability (which puts limits and continuous distributions on a firm footing) and martingales, and connect probability with statistics, analysis, combinatorics and physics.
The questions that run through this chapter
What exactly is the question? What is the random experiment, what is being conditioned on, and what does “at random” mean?
What is the clever way in? When can symmetry, indicators, conditioning or a well-chosen state replace a long calculation?
What happens in the long run? How do random processes behave over time, and what do sums, averages and maxima of many random quantities look like?
The chapter has five parts. Thinking clearly (Section 12.1 to Section 12.4) deals with the traps: conditioning, base rates, the meaning of “random” and the meaning of independence. Symmetry and clever arguments (Section 12.5 to Section 12.9) collects problems with beautiful short solutions. Expectation, games and stopping (Section 12.10 to Section 12.14) turns to averages and decisions: when to stop, whom to shoot at, how to randomise. Random processes (Section 12.15 to Section 12.19) follows randomness through time and space. And Limits and continuous probability (Section 12.20 to Section 12.25) ends with geometric probability and the great limit theorems.
Ideas in this chapter
| § | Topic | Idea |
|---|---|---|
| 12.1 | Conditioning | “A specific one” versus “at least one” |
| 12.2 | Bayes’ theorem | Base rates and natural frequencies |
| 12.3 | Bertrand’s paradox | “Random” needs a definition |
| 12.4 | Independence | Pairwise is not mutual |
| 12.5 | The airline seat | Symmetry between two seats |
| 12.6 | One more coin | Complementary symmetric events |
| 12.7 | Tournaments | Counting matches and pairs |
| 12.8 | Penney’s game | First-step analysis; non-transitivity |
| 12.9 | Fair coins | Extracting fairness from bias |
| 12.10 | Random loops | A memoryless sum of indicators |
| 12.11 | Cups and \(e\) | Factorials and the exponential series |
| 12.12 | Optimal stopping | The \(\frac1e\) rule |
| 12.13 | The truel | Strategic weakness |
| 12.14 | Game theory | Randomised strategies; minimax |
| 12.15 | Branching processes | Extinction as a fixed point |
| 12.16 | Markov chains | Stationary distributions; reversal |
| 12.17 | Banach’s matchboxes | The negative binomial |
| 12.18 | The exponential | Memorylessness and independence |
| 12.19 | Spatial randomness | Nearest neighbours; Rayleigh |
| 12.20 | Buffon’s needle | Geometric probability and \(\pi\) |
| 12.21 | Random triangles | Probability as area |
| 12.22 | The Cauchy distribution | A distribution with no mean |
| 12.23 | Stirling’s formula | Factorials from the normal curve |
| 12.24 | Central limit theorem | Moment generating functions |
| 12.25 | Extreme values | The Gumbel distribution |
One section, Section 12.24, uses moment generating functions, which were removed from the STEP specification in 2019; it explains what it needs.
How to read this chapter
When a probability question seems confusing, write down the sample space for a small case and count. When an answer comes out suspiciously simple, look for the symmetry that explains it. And when your intuition and the calculation disagree, trust the calculation, then find out why your intuition was wrong: that is where the learning happens.