09Probability Axioms & the Sample Space

Chapter 8 handed us an arrow that finds the bottom of any landscape, and with it Part 0 closed. Every tool the field quietly assumes is now built. This chapter points the whole toolbox at the thing it was sharpened for, which is putting an honest number on an event that has not happened yet. You already own one such number. Favourable over total is the rule you were handed at about nine years old, and you have never had cause to doubt it. We are going to break it in the first two minutes. Then we rebuild probability from three plain rules about weight, and out of those three rules the entire familiar list falls: the complement rule, the empty event, monotonicity, and the addition law with its subtracted overlap. Somewhere in the middle the school formula walks back in, wearing a different name. It turns out to be a theorem that quietly assumes every outcome is equally likely, which is a claim about the physical world and not a fact of mathematics. By the end you will be able to take a probability question you have never seen, build its sample space, decide whether counting is even licensed, and finish with 23 people in a room and a result that ought to feel impossible.

Before the first die is rolled, here is where this page sits in the course. Part 0 is behind us and Part 1 begins right here, at the root of everything that follows. Tap a downstream node in the panel below and watch the road run back to this page.

Three of those roads are worth naming now. Chapter 11 turns an event's weight into expectation, which is just that weight used to average a payoff. Chapter 27 prices an option as a discounted expected payoff, so the whole of derivatives pricing is this chapter's machinery run at altitude. And Chapter 16 admits the uncomfortable part: in real markets nobody hands you the weights. You estimate them from a finite, noisy sample.

01The formula that breaks

Let's start with the one probability fact everybody in the room already owns. Count the outcomes you want, divide by the total number of outcomes, and that is the probability. A fair coin has one head out of two sides, so heads is 1/2, and a fair die has three even faces out of six, so even is 3/6. It works, it is quick, and almost nobody is ever told what it assumes.

So here is a coin. It has been bent in a vice until it lands heads seven times in ten, and we have thrown it ten thousand times to be sure. It still has exactly one head, and it still has exactly two sides. Commit to an answer before you touch the panel: what does favourable over total say the probability of heads is, and what is it really?

The formula answers 0.500, because one head out of two sides is all it can see. The coin answers 0.700. Nothing about the counting was done wrong. The formula was asked a question it has no equipment to handle: it counts outcomes, and this coin's two outcomes are not interchangeable.

You might reasonably reply that bent coins are a stunt. Fair enough. Here is the same failure with perfectly fair equipment and no trickery at all. Roll two ordinary dice and add the faces. The possible sums are 2 through 12, so there are eleven outcomes on the list. How often does a seven come up?

Favourable over total says one sum out of eleven, which is 0.0909. Roll the dice and a seven arrives about 0.1667 of the time, which is nearly double. Both dice were fair, and every sum on the list was genuinely possible. The list was still the wrong thing to count, and that is the crack we spend this chapter sealing.

Notice what actually went wrong, because it is the same fault twice. In both cases we listed the outcomes correctly and then treated them as though they were interchangeable. The bent coin's head and tail are not. A sum of seven and a sum of two are not, because seven can be built six different ways and two can be built only one. We need a definition of probability that never needed them to be interchangeable in the first place.

02Every event is a subset

Rebuilding starts with something almost insultingly simple. Take an experiment, meaning any procedure with an uncertain result, and write down every distinguishable result it can produce. For one roll of a die that list is {1, 2, 3, 4, 5, 6}. That set has a name: it is the sample space, written Ω, and its elements are called outcomes or atoms.

Now the move that most courses spend half a sentence on, and that is worth a great deal more than half a sentence. Take a statement about the experiment, something like "the roll is even". Ask which outcomes make that statement true, and the honest answer is 2, 4 and 6. So the statement has quietly handed you a set, {2, 4, 6}, and that set is what we call the event.

The word event fights you here, and it is worth naming why. In English an event is something that happens in time, so your mind pictures the die tumbling and coming to rest. Put that picture down. In this subject an event is a static subset of Ω, chosen before anything is rolled. Drive the panel below and watch sentences turn into sets.

Two extremes in that panel are worth pausing on. "The roll is a seven" makes no outcome true, so it hands you the empty set ∅, the impossible event. "The roll is between 1 and 6" makes every outcome true, so it hands you Ω itself, the certain event. Impossible and certain are not special cases bolted on afterwards — they are simply the smallest and largest subsets you can name.

And now the payoff for Chapter 1. Once events are sets, the ordinary words of logic stop being vague. "A or B" is the union A ∪ B, because an outcome makes the compound sentence true exactly when it lies in one set or the other. "A and B" is the intersection A ∩ B. "Not A" is the complement Aᶜ. The set algebra you learned in Chapter 1 turns out to be the logic of events, unchanged and pointed somewhere new.

Take A as "even", which is {2, 4, 6}, and B as "at least 4", which is {4, 5, 6}. Then A ∩ B = {4, 6} and A ∪ B = {2, 4, 5, 6}. One more word to bank while we are here. Two events with no outcome in common are called disjoint, or mutually exclusive, which just means A ∩ B = ∅. That word is about to do an enormous amount of work.

03One unit of weight

We have the sets. Now we need the number, and the shape of the object that supplies it is where beginners genuinely stall. Every function in Part 0 took a number and returned a number. This one takes a set. So P(A) is not P multiplied by A, and it is not a typo — it is a function being fed a subset of Ω.

Here is the picture, and we keep it for the rest of the chapter. Imagine one unit of weight, a single kilogram of sand, and you get to spread it over the six faces of the die however you like. Spread it evenly and the die is fair. Pile it all on the six and the die is loaded. Whatever you do, the total on the table stays one kilogram.

The amount sitting on a single outcome is that outcome's probability mass, and the little table of "how much on each atom" is the probability mass function, or pmf. Now circle any event, meaning any subset, and ask the obvious question. How much weight is inside the circle? That total is P(A). P is a weighing machine for regions.

Drag the sand around in that panel and the definition becomes something you can feel. The probability of an event is the sum of the masses of the outcomes inside it, and nothing more mysterious than that. On a fair die each face carries 1/6, so the event {2, 4, 6} weighs 1/6 + 1/6 + 1/6 = 0.500.

One more hard stare at P before we constrain it, this time at its type. Our Ω has six atoms, so the number of subsets you could hand to P is 2⁶ = 64. Sixty-four legal inputs, from the empty set up to Ω itself, and every one of them comes back as a single number between 0 and 1.

That panel is the whole domain of P laid out at once. Click any of the 64 doors and the answer lands on the rail. Two of them are pinned by meaning rather than by arithmetic: the empty set weighs 0 because it contains no sand, and Ω weighs 1 because it contains all of it. Hold onto those two, because they are about to stop being observations and become axioms.

04★ Three rules and nothing else

In 1933 Andrey Kolmogorov wrote down what a probability has to be, and the striking thing is how little he needed. Three rules. Everything else in this chapter, and a fair amount of the rest of the course, is squeezed out of them. Read each one as a statement about sand rather than as a commandment.

Axiom 1, non-negativity. For every event A, we require P(A) ≥ 0. You cannot place a negative amount of sand on a face, because there is no such physical thing — and where there is no such thing, there is no such probability.

Axiom 2, normalization. P(Ω) = 1, meaning the whole pile is exactly one unit. This is the axiom that fixes the scale, and it is the reason probabilities are comparable at all.

Axiom 3, additivity. If A ∩ B = ∅, then P(A ∪ B) = P(A) + P(B). Two piles that share no grain of sand combine by plain addition, because nothing gets counted twice. This is the workhorse, and every theorem below is really this axiom applied to a cleverly chosen cut.

That is the entire definition of probability. Notice what is not in it. There is no mention of coins, of fairness, of equal likelihood, or of dividing anything by anything. A bent coin satisfies all three axioms perfectly well. Try to build a P that does not.

Push the weights to the extremes in that panel and watch which lamp goes red. Drag a face below zero and Axiom 1 fails on the spot. Pile up more than one unit and Axiom 2 fails. What you cannot do is find a spread of sand that feels wrong and yet satisfies all three. The axioms are not a filter placed on top of the weight picture. They are the weight picture, written as rules.

Axiom 3 deserves its own experiment, because its condition is the one people forget under pressure. It only licenses addition when the two events are disjoint. Watch what the sum does when you let the circles touch.

Keep the events apart and P(A) + P(B) tracks P(A ∪ B) exactly. Slide them together and the sum runs high, and the excess is precisely the weight in the shared region. That overshoot is not a flaw in the axiom. It is the axiom telling you, in numbers, where its permission stops, and we cash that in as a theorem when we prove the addition law.

05Everything else is a theorem

Here is where three lines start paying rent. Every law you were ever handed about probability can be derived from those axioms, and each derivation follows the same recipe. Rewrite the event you care about as a union of disjoint pieces, apply Axiom 3, and use Axiom 2 wherever Ω shows up. No new assumptions enter anywhere.

Start with the complement rule, where an event A and its complement Aᶜ share nothing and together cover everything. So A ∪ Aᶜ = Ω and A ∩ Aᶜ = ∅. Axiom 3 then gives P(A) + P(Aᶜ) = P(Ω), and Axiom 2 says that right-hand side is 1. Rearrange and you have P(Aᶜ) = 1 − P(A), which will do more work in this course than almost any other line.

Step that rack of derivations and you get three more for free. Take A = Ω in the complement rule and out drops P(∅) = 0, so the impossible event carries no weight. For monotonicity, cut a larger event B into the disjoint pair A and B ∖ A. Then P(B) = P(A) + P(B ∖ A). Axiom 1 makes that last term non-negative, so A ⊆ B forces P(A) ≤ P(B). Put B = Ω in that and you get the ceiling, P(A) ≤ 1.

Sit with that for a second, because it is the substance of the chapter: nobody decreed that probabilities top out at one. It is forced, by two axioms and one cut. If a model of yours ever reports a probability of 1.4, you have not found an exotic case; you have broken an axiom somewhere upstream.

That leaves the addition law, which is the one people half-remember from Chapter 1's counting. You cannot just write P(A ∪ B) = P(A) + P(B), because Axiom 3 refuses to apply to overlapping events. So we have to earn it, and the way you earn it is with scissors.

The cut in that stepper is the whole proof. Slice A ∪ B into three pieces that touch nowhere: the part of A outside B, the shared lens A ∩ B, and the part of B outside A. Now Axiom 3 is legal on all three, so their weights add. Reassemble P(A) + P(B) and you find the lens has been counted twice, so subtract it once. The result is P(A ∪ B) = P(A) + P(B) − P(A ∩ B).

Check it on the die. With A = {2, 4, 6} and B = {4, 5, 6} on fair weights, P(A) + P(B) = 0.5 + 0.5 = 1.0, which is already absurd for an event that misses the faces 1 and 3. The lens {4, 6} weighs 2/6, and taking it off leaves 4/6 ≈ 0.667. That is exactly the weight of {2, 4, 5, 6}, counted directly.

This looks like Chapter 1's inclusion–exclusion, and it should, because it is the same picture. The difference is the status of the claim. In Chapter 1 we counted and saw the overlap double-counted. Here we proved it, from an axiom, for any weights at all, fair or bent. Same shape, sturdier foundation.

06★★ Where favourable-over-total comes from

We have built quite a lot without ever dividing favourable by total. So where does the formula you have used your whole life actually live? It is hiding inside Axiom 3, with one extra assumption strapped to its back, and that assumption is doing all the work.

Take a fair die and add up the weight of the even faces the slow way. The event {2, 4, 6} is the disjoint union of the three single-outcome events {2}, {4} and {6}, so Axiom 3 applies three times over. That gives P(even) = 1/6 + 1/6 + 1/6 = 3/6. Look hard at where those two numbers came from before you go on.

The 3 is nothing but the number of even faces, which is |A|. The 6 appears only because every face happened to carry the identical weight 1/6, which is 1/|Ω|. Write it in general and the collapse is immediate: if all |Ω| outcomes carry the same mass, then P(A) = |A| × (1/|Ω|) = |A|/|Ω|. Favourable over total is additivity, plus one substitution.

That substitution has a name, and it is worth saying out loud, because nobody ever states it. It is equal likelihood, the claim that every outcome in Ω carries exactly the same weight. Not a theorem but a physical claim about the equipment. And equipment can be bent.

So that panel asks you to commit before it reveals anything. Load the die so the six carries 1/3 of the sand and the other five faces share the rest evenly, at 2/15 apiece. Does favourable over total still work? Most people say yes, and the recomputation says otherwise. The true weight of the even faces becomes 2/15 + 2/15 + 1/3 = 0.600, while the formula sits there insisting on 0.500.

Now check the three lamps on that loaded die. Every weight is still non-negative, so Axiom 1 holds. They still total exactly one, so Axiom 2 holds. Disjoint events still add, so Axiom 3 holds. The axioms survive a loaded die without a scratch, and the school formula does not. That is the whole reason we bothered with three rules instead of one fraction.

Which leaves you with a question to ask before you ever write |A|/|Ω| again. Not "how many outcomes are there", but "does this experiment actually hand every outcome the same weight?" Some do, honestly and by construction. Others only look like they do.

Work through those cases and a pattern shows itself. Equal likelihood is earned by physical symmetry, meaning the outcomes are related by something that genuinely cannot tell them apart, like the faces of a machined cube or the cards in a shuffled deck. It is destroyed by bending, by weighting, and above all by summarising, which is what the eleven dice-sums did to us in the opening.

07★ Counting, done honestly

Suppose the assumption is earned. Then the whole problem changes character, and that is the good news for the rest of Part 1. Computing P(A) becomes a pure counting exercise. And counting you already know, from Chapter 1: the multiplication principle, factorials, combinations.

The failure mode is no longer the arithmetic. It is building the sample space wrong in the first place. Let's go back to the two dice and do it properly, because the classic error lives right there.

Step that grid and the honest Ω appears. Each die independently shows one of six faces, so the multiplication principle gives 6 × 6 = 36 outcomes, laid out as a grid your eye can count. Six of those cells sum to seven, so P(sum = 7) = 6/36 = 0.1667. That is the number the dice actually produce.

Now watch the stepper fold the grid. Treat the dice as indistinguishable, so that (2,5) and (5,2) become one outcome, and Ω shrinks from 36 to 21. Every count in sight is still correct. But the 21 atoms no longer weigh the same, because a mixed pair like {2,5} arrived from two grid cells while a double like {3,3} arrived from one. Apply the formula anyway and a seven comes out as 3/21 = 0.1429, which is wrong.

A fair objection arrives here, and it is a good one. We fold outcomes together all the time in counting problems, and it usually works fine. So when exactly is folding legal? The rule is sharper than most courses admit.

Folding is safe when every group you collapse has the same size. Take the panel's example: 20 stocks, 5 of which will beat the market, and you pick 3 at random. Counting ordered picks gives 20 × 19 × 18 = 6840 in the denominator and 5 × 4 × 3 = 60 on top. Counting unordered hands gives C(20,3) = 1140 and C(5,3) = 10. Both routes return 0.0088, or about a 0.88% chance.

They agree because every unordered triple folds from exactly 3! = 6 orderings, top and bottom alike, so the sixes cancel. The dice sums had no such luck. A seven folds from six ordered cells and a two folds from one, so the groups were ragged and the equal weights died in the fold. That single test, are the groups all the same size, is what separates a clean simplification from a broken sample space.

08Twenty-three people

Time to spend all of it at once. You are in a room and you want the probability that at least two people share a birthday. Assume 365 equally likely days and no twins, which are the honest simplifications we will name and keep. How many people do you need before that probability passes one half?

Commit to a number now, out loud if you can, because the guess is the point. Most people reason that there are 365 days, so you need somewhere near half of them, and they land around 180. The real answer is 23. Before we explain why, here is how you compute it.

Attacking "at least one shared birthday" head-on is a nightmare of overlapping cases, because two people might match, or three, or two separate pairs might match. So flip it. The complement of "at least one shared" is "all distinct", and that one is a single clean count. The first person may have any birthday, so 365/365, and the second must dodge one taken day, so 364/365. The third dodges two, so 363/365, and so on down.

That is real code, executed, and the printed table is its output. At 22 people the chance of a shared birthday is 0.4757. Add one more person and it jumps to 0.5073, crossing the line. Keep going and it climbs fast: 0.7063 at 30 people, 0.9704 at 50, and 0.9901 at 57.

Every ingredient in that computation is something we built on this page. "At least one" became 1 − P(none) by the complement rule from the axioms. "All distinct" was a multiplication-principle count over an equally likely sample space. No new machinery anywhere. That is exactly the claim this chapter has been building toward.

Computing the answer is not the same as understanding it. The shock deserves an explanation, not a shrug. Your intuition missed by so much because you were quietly asking the wrong question: "how many people must there be before someone matches me?" That question really does need about 253 people to reach a half.

The room does not care about you. A collision needs a pair, and 23 people contain C(23,2) = 253 distinct pairs, each one an independent chance to collide. Your gut counted people, which grows linearly. The collisions live among pairs, which grow like n²/2. That mismatch between a line and a parabola is the entire paradox, and once you see it the result stops being a trick and becomes obvious.

This matters well beyond party games. Scan a hundred trading strategies for one that beats the market and you are not running one test. You are running the pair-count's cousin, a hundred chances for a fluke to look like a signal. Chapter 19 gives that failure its proper name and its correction.

So here is the chapter as one procedure you can carry out the door. Write down Ω, the set of distinguishable outcomes. Write the event as the subset that makes your sentence true. Ask whether the outcomes really carry equal weight, and only then reach for counting. If the question says "at least one", flip to the complement first. If two events overlap, subtract the lens once.

And hold onto the shape underneath it, because it never changes after this. Probability is one unit of weight spread over a sample space. P is a function that weighs whatever region you circle. Three rules govern that weight: none negative, the whole pile is one, separate piles add. Every law in this chapter came out of those three, including the school formula, which turned out to be a theorem carrying an assumption it never declared.

Which sets up the next question rather neatly. Everything we have weighed so far was measured against the whole of Ω. But information arrives. Someone tells you the roll came out even, and now half the sample space is dead. What happens to the weights that are left?

Drag that panel and you can watch the answer take shape. The surviving region gets renormalised so its weight is one again, because the axioms demand it, and every event inside is re-measured against the smaller world. That operation is conditional probability, and Chapter 10 is where it stops being a picture and becomes the sharpest tool in the course. Run it backwards and it is Bayes' theorem.