12The Waiting-Time Family

Last chapter ended on a map with one control still at a stop. We had chopped the window into finer and finer slices, and then we deliberately stopped chopping. We had to stop, because every slice still had to be a countable trial you could point at. This chapter turns that dial all the way down to zero. Once the slots dissolve, the geometric question "which trial?" becomes the question "what time?", and the answer is the exponential distribution. Almost nothing new gets imported. You already derived the hard part in Chapter 11, when the geometric tail q^m turned into e^(−λt) and the two halves of that chapter shook hands. That handshake has a name: the duality. Here's the plan. We build the survival function first, because it is the thing you can actually photograph, and we let the density fall out of it as a slope. Then we find that the whole law has one knob. One knob means one length, and that forces the mean and the spread to be the same number. Then comes the strange part. This thing does not age. A component that has run for a thousand hours has a thousand hours left, on average, exactly like a fresh one. Your gut will fight that, and it should, because your gut is right about real bulbs. One instrument sorts out which world is which, and it is the hazard rate. Then the keystone, which is the deepest thing here: no memory doesn't merely describe this curve. It builds it, and leaves room for nothing else. At the end we read four famous laws off one timeline of dots, without generating a single new random number.

01When the slots dissolve

Turn the dial we left alone. In Chapter 11 the window was chopped into slices, each slice a little Bernoulli trial, and the wait was a count of slices. The geometric lived on that grid. Now shrink the slices toward zero width. The grid disappears, and the bars have nowhere left to stand, so watch what happens to the picture.

The bars fuse into a curve, and that looks like a cosmetic swap. It isn't one. Something breaks when the grid dissolves, and it breaks quietly. On the grid, "the wait is 3" meant slot three, and P(X = 3) was a real positive number you could compute. In flowing time, ask for the probability that the wait is exactly three hours and the answer is zero. Not small. Zero. Chapter 8 already told you why. A point carries no probability. Only an interval does, and a density is not a probability. So a reader carrying the discrete habit walks up, asks for P(T = 3), gets nothing back, and concludes the model is broken. The model is fine. The question was broken. Watch both readouts as you narrow the slot from an hour to a second.

The probability of any one named slot slides to zero, while the probability of "somewhere in the first hour" doesn't move at all. The point survives nothing. The interval survives everything. So when the slots dissolve, only questions about intervals are left standing, and that tells us which Chapter 11 fact to carry across. We proved P(X > k) = q^k there, and that statement was already about an interval: it says the first k slots all missed. It is the one geometric fact built to survive a dissolving grid, so it is the one we take with us. In flowing time it becomes S(t) = P(T > t), the chance you are still waiting at time t. That is the survival function, and for waiting problems it is the main character. That claim probably sounds like bookkeeping, since S(t) = 1 − F(t) is just the CDF read from the other end. So let me show you why the survival function is the thing you can actually measure.

Here is the whole reason: you can photograph the survival curve, but you have to compute the density. A hundred and twenty bulbs, all lit at t = 0, and a camera pointed at the rack. S(t) is the fraction still glowing, and you count it straight off the photograph — sixty bulbs alight out of a hundred and twenty means S = 0.5. Now ask the same rack for the density. You'd have to take two frames, subtract the counts, and divide by the gap between them. That is noisier, it is derived, and it sits one step away from anything you actually saw. So the survival curve is the measurement and the density is the computation. Almost every textbook opens on the density, which is exactly backwards, and it costs the reader real money later. We are going the other way, and that choice pays off three more times before this chapter ends: at the mean, at the racing clocks, and at the gamma.

02Build the stream, and the law falls out

Now we build the thing that produces the events, by hand, rather than accept it as a word. Take a rate of λ = 2 arrivals an hour. Here's the first thing nobody checks: λ is measured in events per hour, so it can be 5 or 500, while a probability can never exceed 1. So the slice probability p cannot be λ. Chop the hour into n slices instead, and hand each slice its own independent coin. There is exactly one honest way to spread λ events across n slices: each slice takes p = λ/n, so that n slices times p per slice gives the rate back. That fraction is forced, not chosen. Try n = 2 and you get p = 1, which means two arrivals guaranteed, one per half-hour, and no randomness left at all. Chop the hour into minutes and p = 1/30. Chop it into seconds and p = 1/1800. Drag the slicing and watch the product.

np never leaves 2. You have built this limit before: n → ∞, p → 0, with np = λ pinned. That is Chapter 11's Poisson setup, word for word. Same limit, new question. Poisson asked "how many?" and we are asking "how long?" The stream itself has a name, the Poisson process, and it is just a constant-rate run of events along a timeline. Now notice the gift sitting inside the construction, because it is worth more later than it costs now. Different slices are different coins, tossed independently, so what happens in one stretch of the timeline tells you nothing about a stretch that does not overlap it. That property is called independent increments, and we never had to declare it. We got it by building it.

Now ask the stream our surviving question: what is the chance you are still waiting at time t? Write it yourself before any algebra. In t hours there are nt slices. The wait exceeds t exactly when every one of those nt slices missed, and the slices are independent, so their probabilities multiply. Cut the hour into 3600 one-second slices and a three-hour wait is 10,800 misses in a row. You have just written q^k with k = nt. That is the geometric's own line, and the only new thing is what the exponent counts. Step it through.

There it is: S(t) = (1 − λ/n)^(nt) = [(1 − λ/n)^n]^t → [e^(−λ)]^t = e^(−λt). Look at how little that cost. The bracket in the middle is the deadlock from Chapter 11 — the same tug-of-war, the same e, standing on the same rung a second time. So the e in e^(−λt) is not there because "exponentials are natural". It is there because of one specific limit you already watched freeze digit by digit. The law of waiting in a constant-rate stream is S(t) = e^(−λt), and at two arrivals an hour that says 13.5% of waits run past the first hour (e^(−2) = 0.135). The random variable T that obeys this law is called the exponential distribution, written Exp(λ). Don't take my word for the limit. Let's build the sliced stream in code and measure the fraction still waiting.

The measured fraction lands on e^(−λt) to four decimals, and the machine only ever flipped biased coins. It never once evaluated an exponential. That is a receipt, not an argument. The argument was the algebra; the code just confirms nobody cheated.

03One knob is the whole distribution

We have the survival curve, so the density costs exactly one derivative. You already own f = F′ from Chapter 8 and S = 1 − F from ten minutes ago. Differentiate, and f(t) = −S′(t) = λe^(−λt). At two an hour, the density one hour in is 2 × 0.135 = 0.271. Now I want to stop on the λ out front, because it is the exact inch where readers start memorizing. Every book hands you λe^(−λt) whole, so you parse it as two glued facts: there is an e^(−λt) shape, and there is also a λ scale factor. That is not what it is. The λ is the chain rule pulling the exponent's coefficient out front, and it appears whether you want it or not. So drag along the survival curve and watch its steepness trace out the density.

The density is nothing but the steepness of the thing you photographed on the bulb rack. Now the callback that closes the loop. In Chapter 10, the number out front of the bell was 1/(σ√(2π)), and its job was to force the total area to 1. The λ here does that same job, and you can check it on screen: ∫₀^∞ λe^(−λt) dt = 1 exactly. Drop the λ and the area comes out as 1/λ instead, which at two an hour is 0.5, not 1. Here is the difference. Chapter 10's constant had to be bolted on. This one falls out of a derivative. A term that could have been inevitable usually gets handed to you as arbitrary. Not here.

Next, the mean — and I want to run the units before the integral. λ is a rate, so its units are 1/time, which makes 1/λ a time. At two arrivals an hour that is half an hour, thirty minutes. Now look in the box, where λ is the only parameter there is. If you want this distribution to hand you a time — the mean, the SD, the median, the 90th percentile, anything — there is nothing else with units of time to build it from. Every one of those numbers is forced to be a plain number times 1/λ. That is the one-scale law, and it lets you predict before you compute: the mean and the SD must be proportional. Same constant, though? Run the integrals and see.

You can now write the mean as 1/λ. But a rate carries a clock with it, and that is where the first trouble bites. A rate of 2 per hour is the same stream as 1/30 per minute: the arrivals never changed, only the clock did. The mean flips the units too, because λ counts events per unit time while 1/λ is time per event. That one stream has a mean wait of 0.5 hours, which is the same thing as 30 minutes. Switch the clock and both numbers must move together, or your answer turns to nonsense. Set a rate below, change the clock, and watch what stays fixed.

E[T] = ∫₀^∞ t·λe^(−λt) dt = 1/λ, and the variance integral gives 1/λ², so SD = 1/λ. At two an hour the mean is 0.5 hours and the SD is 0.5 hours as well. Both constants came out as 1. Be clear about what did which job here. The proportionality was forced by the units, and you knew it before any integral ran. The equality is the exponential's own fingerprint, the same kind of signature as the Poisson's mean = variance = λ. And the one-scale law is something your eye can check directly. Drag λ and the entire picture just stretches sideways. Nothing else about it ever changes, because there is no second knob. The normal had μ and σ. This has one number and no shape parameter at all. Hold that hole in your hand. When the gamma turns up at the end of the chapter, its r is exactly the knob that is missing here.

Now some numbers that will fight you, and they fight you because of where you have just been. Chapter 10 was a symmetric bell, where the mean sits on the median and everything is tidy, so you imported that equality without noticing you imported anything. This curve is skewed. Solve S(t) = ½ and you get e^(−λt) = ½, so t = ln2/λ ≈ 0.693/λ, which at two an hour is about 21 minutes. That median sits to the left of the mean at 1/λ, half an hour. Here's the question that catches almost everyone. A bulb's mean life is 1000 hours, so what fraction of bulbs last longer than 1000 hours? Commit to a number before you look.

Not half. S(1000) = e^(−1) = 0.368, so only 37% of bulbs outlast the average, and nearly two-thirds are already dark by 1000 hours. The long thin tail hauls the mean to the right while most of the mass sits early, which is the same balance-point argument that made the geometric's average six while its most likely wait was one. Run it the other way and it becomes a working tool. A physicist measures a half-life, drags that marker, and reads off λ = ln2/t½: a half-life of 1000 hours gives λ = 0.693/1000 = 0.000693 per hour. That inversion is how a decay constant gets into a textbook in the first place. One more, because we should fix it out loud: e^(−1/2) is 0.607, so half a mean life leaves 61% still alive, not the "about 70%" you'll hear people eyeball. Loyalty is to the subject.

Those numbers hide an everyday lesson worth feeling in the body. The median sits to the left of the mean, and only 37% of items outlast the average. Put it in human terms. Told the average wait is 10 minutes, your gut hears "usually about ten minutes". It is not. Most waits are short, a handful are enormous, and those rare monsters quietly own the average — about 63 people in a hundred are already served inside those ten minutes. So send a hundred people to wait, and watch where they actually land.

04The thing that doesn't age

Here's the property this whole family turns on, and it is the one your gut will refuse. Your bulb has a mean life of 1000 hours, and it has already burned for 1000 hours. How much life does it have left, on average? Every gut in the room says much less, because it is old, it is tired, it is closer to death. The answer is 1000 hours. Exactly a fresh bulb, with no discount for the thousand hours it already served. Don't take that from me. Grab the handle and try to make the remaining-life curve move.

It never changes shape at any setting you choose — not flatter, not shifted, not shrunk. The algebra behind that is three lines, and Chapter 5 already wrote them for us. Conditioning shrinks the world to what you know, and there is one sub-step worth saying out loud that books skip. If you have outlasted t+s, then you certainly outlasted s, so the event {T > t+s} sits inside {T > s}. Their intersection is therefore just {T > t+s} itself. So P(T > t+s | T > s) = S(t+s)/S(s) = e^(−λ(t+s))/e^(−λs) = e^(−λt) = S(t). Put numbers in it: at two an hour, a wait already 3 hours old passes 4 hours with probability e^(−8)/e^(−6) = e^(−2) = 0.135, the same 0.135 a fresh wait has of passing one hour. The exponents subtract, and the past cancels out of the algebra exactly as literally as it cancels out of the physics. That is memorylessness: the thing does not age.

Now the honest part, because your intuition is not stupid and I am not going to overrule it. Your intuition is correct about real bulbs. Filaments wear out, bearings wear out, people wear out. So when a book says "the exponential is memoryless" as though that were a fact about bulbs, it is telling you something false about your fridge. Memorylessness is a fact about a constant-rate stream, not about hardware. What genuinely does not age is a radioactive nucleus. A uranium-238 atom that is five billion years old is statistically identical to one made this morning. Arrivals at a call centre do not age either, because nothing in the stream knows a call just happened. So keep both intuitions, sorted into the right boxes. The question is which world you are in, and there is an instrument that tells you.

Go back to the rack and ask the question a survivor would ask. At hour t, how many bulbs die in the next minute? That is f(t)·dt·1000. But a bulb that is still glowing doesn't care about that number, because that number is watered down by every bulb that already burned out. The living bulb asks a sharper question: of us survivors, what fraction goes dark this minute? That is f(t)dt / S(t). The division isn't exotic — it is Chapter 5's conditioning, with the survivors as the new sample space. That quantity has a name, the hazard rate, and for the exponential it works out as λe^(−λt)/e^(−λt). The e's cancel. It's λ, flat, forever. For the 1000-hour bulb that is 1/1000 = 0.001 per hour, at hour one and at hour ten thousand alike. The bulb at hour ten thousand faces precisely the risk it faced in its first second. That is what "no memory" looks like when you can touch it. Now flip the toggle, because flat is only one curve in a world of possible curves. Rising hazard means the thing ages, and the bathtub curve — high, then flat, then climbing — is the real shape for most manufactured things. So here is your diagnostic, and it is the most transferable thing in this chapter: check the hazard. If it isn't flat, the exponential is a lie, and you now know that before you fit anything.

05No memory leaves no room

Fine — the exponential is memoryless. Surely it is just one of many, so go and build me a memoryless law that isn't exponential. I'll even give you the test as a physical gesture, so you don't need any algebra to run it. Take a survival curve and cut it at time s. Throw away everything to the left, then stretch what is left vertically by 1/S(s) — at two an hour, cutting at one hour means stretching by 1/0.135, about 7.4 times. That gesture is conditioning. It is exactly "given it survived to s", drawn instead of written. If the law is memoryless, the rescaled tail must land dead on top of the original curve, for every s you pick. Try it on a straight ramp, try it on a bathtub, then try it on e^(−λt). Then read what your own hand just did.

I just told you the formula is now disposable, that two words rebuild it. That is a strong claim, so do not take it on faith. Take the two words — constant rate and no memory — and build the whole distribution back yourself, one honest rung at a time. Pick the right next piece at each step and its reason locks in beside it. Pick a wrong one and you will see exactly why it cannot stand. By the end you will have rebuilt e^(−λt), its density, and its mean, with no formula in front of you.

"The cut tail lands on the original" is S(t+s)/S(s) = S(t), which rearranges to S(t+s) = S(t)S(s). That is the whole of memorylessness, written as an equation about a curve. Read what it demands: the curve has to turn adding time into multiplying probability. How many functions can do that — dozens, a family, an infinite zoo? Exactly one of them can, and the proof runs to six lines with no calculus anywhere in it. Let a = S(1), some number strictly between 0 and 1; at two an hour it is 0.135. Then S(2) = S(1+1) = S(1)S(1) = a², which is 0.135² = 0.018, and S(3) = a³, and S(n) = aⁿ for every whole n. Now go the other way. S(½)S(½) = S(1) = a, so S(½) = a^(1/2). The same trick gives S(m/n) = a^(m/n) for every rational. The fractions m/n crowd in arbitrarily close to every point on the line, and S is an unbroken curve, so the in-between values get pinned too, with no freedom left anywhere. Therefore S(t) = a^t = e^(t·ln a) = e^(−λt), with λ = −ln a > 0 — and −ln(0.135) is 2, the rate we started from.

Sit with that for a second, because it is the deepest sentence in the chapter and it is usually thrown away in half a line. There was never a second law. There was never room for one. Memorylessness and the exponential are not two facts about one distribution. They are the same statement in two costumes. And the λ isn't even a real ingredient — it is your own choice of a = S(1) wearing a logarithm, since λ = −ln a. So you can throw the formula away now. Give me four words, constant rate and no memory, and you can rebuild e^(−λt) from scratch, plus the density, plus the mean, plus every number in this chapter. That is a thing you cannot lose.

06One stream, two readings

Now let's collect the debt Chapter 11 left on the table. You have been carrying T, the wait for the first event, from this chapter, and N(t), the count of events inside a window, from the last one. They feel like two different random variables from two different formulas. They aren't. There is only one random thing in this room, and it is dots on a timeline. T and N(t) are two questions asked of the same dots, so watch what happens when you drag one.

Both readouts twitch, because both instruments point at the same stream. Read the gaps between consecutive dots and the histogram fills in the exponential. Slide a window along and count the dots inside, and that histogram fills in the Poisson. Now shade [0, t] and say both sentences out loud. "The first dot lands after t." "No dots in [0, t]." Point at each one on the timeline. They are the same shaded region — not equivalent, identical. So P(T > t) = P(N(t) = 0), and that's the duality, sitting there as a matter of looking rather than deriving. Then check it against a formula you already own. Put k = 0 into Chapter 11's Poisson PMF with mean λt: e^(−λt)(λt)⁰/0! = e^(−λt). At two an hour over one hour that reads 0.135 × 1 / 1 = 0.135, the exact number the survival curve gave in section two by slicing the timeline and taking a limit. Two roads, one number. The duality isn't a slogan. It is a checkable fact, and you just checked it.

One more property before we build upward, and it is the cheapest rung in the chapter. Every book says "the gamma is the sum of r independent exponentials", and both of the important words in that sentence get smuggled past you for free. Why should the second gap be independent of the first? And why the same λ, when a dot just landed a moment ago? We already own both halves of that sentence. Identical: the hazard is flat forever, so the stream cannot know a dot just happened. Independent: different slices are different coins, which is the independent increments we built in section two. Put them together and the inter-arrival gaps are iid Exp(λ) — at two an hour, every gap has the same mean of 0.5 hours, whatever came before it. Click anywhere on the timeline and check it yourself.

That restart claim carries a quiet assumption: the rate λ is the same at every instant. Real arrivals often break it. A call centre is swamped at 9am and idle at 3am, so the gaps run short in the rush and long at night. Nothing here is aging. No part wears out, and the hazard check from before would happily pass. What is moving is the rate itself, and that is a second, different way for the model to be wrong. Pool every gap, fit one exponential, and see whether it holds.

Stand mid-gap, or stand right on top of a dot. Either way the clock resets to zero, and the forward view is statistically identical every single time. That's the restart property: the stream begins again at every instant, including the instant an event just happened. Nothing new was assumed there, because both halves were already paid for.

07Waiting for the rth, and the family closes

Let's ask a bigger question of the same dots. Stop waiting for the first event and wait for the rth one instead; call that time T_r. This is where readers quit, because the answer contains a t^(r−1) that looks conjured out of nothing, sitting next to an (r−1)! that looks like a fudge bolted on to make an integral behave. Neither is true. We are going to assemble the formula out of parts you already hold, exactly the way Chapter 9 read C(n,k)pᵏq^(n−k) factor by factor. Ask when the rth dot lands in the sliver [t, t+dt]. Two things must happen, on disjoint stretches of timeline, so independent increments lets us multiply them.

First, exactly r−1 dots landed in [0, t], and that's Chapter 11's Poisson PMF with mean λt, read straight off the shelf: e^(−λt)(λt)^(r−1)/(r−1)!. Second, one dot lands in the sliver, with probability λ dt — the rate itself. Multiply the two and you have it: f(t) = λ^r t^(r−1) e^(−λt) / (r−1)!. That's the gamma distribution, Gamma(r, λ). Every factor now has a name and nothing is arbitrary. The t^(r−1) is the r−1 events that had to already happen. The (r−1)! is Poisson's own denominator, not a fudge. The leading λ is "one event, right now", the same rate factor as before. Set r = 1 and the whole thing collapses back to λe^(−λt). And one name defused in a line: for whole r, the gamma function Γ(r) is just (r−1)!, so Γ(3) = 2! = 2. The capital Greek letter is what lets r go fractional later, and it is nothing else.

The gamma is not a new animal. It is the exponential with r turned up, and its mean is r/λ, which is just r gaps of mean 1/λ laid end to end: three gaps at two an hour give 3 × 0.5 = 1.5 hours. Chapter 15 will turn "averages add" into a theorem, so I'm flagging that I use it early. And remember the hole from section three, where the exponential had one knob and no shape. Turn r and watch the hole get filled.

At r = 1 the density piles up against zero, because the most likely wait for the very first dot is no wait at all. Turn r up and the pile lifts off the wall and becomes a bump, because waiting for three dots and having all three arrive instantly is genuinely unlikely. That's the shape parameter doing its job. There is one more trap here, and it hides inside a single word.

The word is "three". Three of what? Three events you must live through, or three servers racing to help you? The same number attaches to two opposite structures, and the formula alone will not tell you which one you are in. Only the picture will. Worse, your gut carries a hard prior that more means faster, which is right for one of these and exactly backwards for the other. Look at both before you write anything down, and commit to a guess.

Three gaps end to end is series: you are waiting for the third dot and you must sit through all three. That's the gamma, mean 3/λ, three times slower. Three clocks started together, stopping the moment the first one rings, is parallel: that's a minimum, mean 1/(3λ), three times faster. At two an hour, the series wait averages 1.5 hours and the parallel wait averages 10 minutes. Same three, same λ, and a factor of nine between them. And here's the survival function paying for itself one last time. "The min exceeds t" means all three clocks exceed t, and they're independent, so the survivals multiply: S_min(t) = [e^(−λt)]³ = e^(−3λt), which is Exp(3λ), read straight off section two. The minimum of exponentials is exponential again — still memoryless, just faster. Try that with densities and you'll be integrating for an hour. With survival it is one multiplication, and that is precisely why we opened this chapter on S and not on f.

So let's close the family, and let's do it with code rather than a claim. Simulate one constant-rate stream and nothing else: draw iid Exp(λ) gaps, then cumulative-sum them into dot times. That array is the entire random world of this chapter. Now interrogate it four ways, without generating a single new random number.

Histogram the gaps and the exponential PDF lands on them. Count dots per unit window and the Poisson PMF lands on that, with mean and variance both λ, exactly the Chapter 11 fingerprint. Sum every three consecutive gaps and Gamma(3, λ) lands, mean 3/λ, which is 1.5 hours at two an hour. Take the min of each triple and Exp(3λ) lands, mean 1/(3λ), ten minutes. One array, four laws, and the code introduced no new probability at all. Every one of those was proven above, so this is a receipt, not an argument. Then break it on purpose. Replace the constant hazard with a rising one and every fit fails at once, because the thing now ages and no member of this family will ever describe it.

Step back and look at what you are carrying out of here. Not four distributions — one stream, read four ways. Point yourself at any constant-rate process, whether it's decay or arrivals or failures or packets, and you can read it from either end knowing the two readings are the same fact. Given a half-life you can recover λ. Given λ you can state the mean, the SD, the median and the fraction still alive from memory, because there's only one length in the box: at two an hour, a mean of 0.5 hours, an SD of 0.5 hours, a median near 21 minutes. You can answer "it has run a thousand hours, now what?" without flinching. Better, you can say exactly when that answer is a lie. Check the hazard, and if it climbs, the thing ages and this family is the wrong family. Here's the map.

And the road out is already paved. Twice in this chapter we got a density by differentiating a survival curve, rather than by substituting into a formula. Do that once and it's a trick. Do it twice and it's a method. Chapter 13 generalizes it into the full change-of-variables machine: to get the law of Y = g(X), you route through the CDF and then differentiate, and you never substitute a density into g. The chain-rule factor that pops out of that differentiation has a name, the Jacobian, and you'll recognise it on sight. It's the same factor that has been standing in front of the exponential this whole chapter.