11Random Variables, Expectation & Variance

Chapter 10 handed us one move, and it was a good one. A fact arrives, your world shrinks to where that fact held, and you rescale the survivors until they add to one again. Every answer we produced with it was a probability. Not one of them was an amount. That is the gap this chapter closes, because a probability on its own cannot be traded, hedged, budgeted for or lost. This page attaches a number to every outcome, and then builds the one tool that collapses all those numbers into a single honest summary. The tool is called expectation, written E[·], and here is the thing worth knowing before we start. It is not four separate tools. The mean is E[X], and the variance is that very same machine pointed at squared distance from the mean. The moment generating function of Chapter 12, the covariance of Chapter 18, and the risk-neutral price e^(−rT)·E^Q[payoff] of Chapter 27 are that identical machine, aimed somewhere else. Learn to point it once and the rest of the course stops being a list of formulas to hold in your head.

The first die can wait. Almost everything downstream in this course is waiting on a number rather than a probability, and one page supplies it. The panel below traces the roads that lead back here.

Three of those roads are worth naming now. Chapter 16 measures how much an estimate wobbles, and wobble is variance wearing a different coat. Chapter 26 explains why a volatile asset grows slower than its own average return, and that whole argument is one inequality we prove at the end of this page. Chapter 27 prices an option as a discounted expected payoff, which is E[·] pointed at a contract.

01The rule that pins a number to every outcome

Here is a bet, and it is deliberately small enough to hold in your head. We roll one fair die, and if it lands on 1, 2 or 3 you pay me £2. If it lands on 4 or 5 I pay you £1. If it lands on a 6 I pay you £6.

Nothing about the die has changed since Chapter 9. The sample space is still Ω = {1,2,3,4,5,6} and each face still carries weight 1/6. What is new is the table taped beside it, turning each face into a pound figure. That table is the entire idea of this section.

Call the table X. It takes an outcome and returns a number, which makes it a function from Ω into the real numbers, and we call it a random variable. Now for the question that trips almost everyone, and I want you to answer it before reading on. Is X a number?

It is not. X is the whole rule, the entire two-column table, and it never changes while we play. X(6) is a number, and that number is +6. The name is genuinely misleading, because the word variable makes you picture the algebra unknown you have been solving for since school. There is nothing here to solve for.

So where has the randomness gone? It sits upstream of the table entirely, in the question of which outcome ω actually occurs. Once the die stops rolling, the table is a plain deterministic lookup with no chance left in it at all. That split is worth holding onto, because it is what lets us do everything else in this chapter.

One piece of notation falls straight out of the table, and it will do real work later. Write {X = x} for the set of outcomes the rule sends to the value x. It is not an equation waiting to be solved. It is a collection of faces, and collections of outcomes are exactly the events Chapter 9 taught us to weigh.

Gather them and the bet's structure appears. Three faces feed the value −2, so {X = −2} = {1,2,3} and its probability is 0.5000. Two faces feed +1, giving 1/3. One face feeds +6, giving 1/6. Every event is a plain subset of Ω, and we already know how to weigh those.

02Forget the outcomes, keep the weights

Look at what we just produced: three values, three weights, and a die that now feels like scaffolding we could quietly take down. That instinct is correct, and it is the first real economy of the subject.

The list of values with their probabilities is called the probability mass function, written p(x) = P(X = x). For our bet it reads p(−2) = 1/2, p(1) = 1/3 and p(6) = 1/6. Those three add to exactly 1, and they have to, because every face of the die was sorted into precisely one of the three groups.

Here the pmf is honest in a way that is about to stop being true, because each bar is a probability. You can read 0.5000 straight off the first bar, and that really is the chance of paying me two pounds.

Now change the question to something continuous, like the closing price of a stock. It can land anywhere in a range, and a range holds uncountably many candidates. Ask for the probability that it closes at exactly £100.000000…, on the nose forever, and the honest answer is zero. There are too many candidates for any single one to hold a positive share of a fixed unit of weight.

That looks like it destroys the whole apparatus, and it would if probability had to live on points. It does not, because probability lives in area instead. Build a histogram, then shrink the bins and watch what survives and what dies.

Take the little triangular shape in that panel, whose curve is f(x) = 2x on the interval from 0 to 1. With four bins the rightmost bar covers [0.75, 1] and holds probability 0.4375, so its height is 0.4375 / 0.25 = 1.75. Now shrink to twenty bins. The rightmost bar holds only 0.0975, and its height has risen to 1.95.

That is the whole lesson, so let's say it plainly. Bar height is probability divided by width. Both the probability and the width are racing to zero, and their ratio settles on a finite curve. The probability of any single bar dies, and the height lives.

The curve it settles on is the probability density function, written f(x). It is probability per unit length, in the same sense that a material has mass per unit volume, and that is why it is called a density. Notice that the far right of our triangle reaches f(1) = 2, which is a density of two. If f were a probability that number would be flat nonsense, and this is the exact inch where continuous probability tears for most readers.

So burn in the replacement reading. f(x) is not the probability of x. The small quantity f(x)dx is a probability, because it is a height times a width, the area of a very thin slab. Probability is the area under the density, always, and never the height of it.

There is one more object to meet, and it is the one that never breaks. Define F(x) = P(X ≤ x) and call it the cumulative distribution function. It asks a question that makes sense for any random variable at all, discrete or continuous or a mixture of the two.

Drag the line across and watch F fill up behind it. On our discrete bet it climbs in steps, reaching 0.5000 at x = −2 and 0.8333 at x = 1 before topping out at 1. Each step's height is exactly the pmf bar underneath it. On the triangle, F(x) = x² rises smoothly, so F(0.75) = 0.5625, and its slope at any point is the density there.

One sentence covers it. The CDF accumulates the density, and the density is the rate at which the CDF climbs. Chapter 3's derivative and integral just walked into probability theory wearing new names.

03★ The balance point

We now have a distribution, which is a whole shape. Often you want to hand somebody a single number instead. Should you take this bet? That is a question about one figure, not about three bars.

The obvious move is to average the values, and the obvious move is wrong. Averaging −2, 1 and 6 gives 1.667, and that answer quietly assumes the three outcomes are equally likely. They are not, because paying me two pounds happens half the time while being counted as one case in three.

So weight each value by how often it actually happens. Multiply, then add:

E[X] = Σ x · p(x)

For the bet that is (−2)(1/2) + (1)(1/3) + (6)(1/6), which comes to −1 + 0.3333 + 1 = 0.3333. The bet is worth about thirty-three pence a roll to you. We call this number the expectation, or the mean, and we write it μ.

Now the trap in the name, and it catches nearly everybody once. Roll a plain fair die and compute E[X]. Every face carries 1/6, so the answer is (1+2+3+4+5+6)/6 = 3.5. Commit to what that means before you touch the panel.

A die can never roll 3.5, not rarely and not approximately but never. So "expected value" is a poor name for the thing it labels, because E[X] is not a value you should expect to see. Drag the bars around on that rail and the better picture appears. E[X] is where the weighted mass balances, exactly like a centre of mass on a seesaw.

There is a second reading, and it is the one that makes the number useful at a desk. Play the same game over and over and average what you actually win. Those running averages settle down on E[X], which is why a thirty-three pence edge per roll is worth having. Chapter 13 turns that observation into the law of large numbers and says precisely how fast the settling happens.

The weights carry more load than they look like they do. It is easy to nod at "weight by probability" without feeling how much it changes.

A raffle ticket pays £1000 with probability 0.001 and nothing otherwise. Average the two values and you get £500. Weight them properly and the ticket is worth 0.001 × 1000 = £1. Unweighted averaging is not slightly off here. It is wrong by a factor of five hundred, and every mispriced tail risk in finance is a version of that mistake.

04The same average, with a density

Our formula sums over a list of values. A continuous variable has no list, so the sum has nowhere to start. This is the moment most textbooks write an integral sign and move on, and it is worth slowing down instead.

Go back to the thin bars, and chop the range into slices of width dx, treating each slice as if it were a single discrete outcome. Its value is the x at that slice. Its weight is the probability that X lands inside the slice, which is the area of the slab, f(x)dx.

That gives us a genuine discrete weighted average over the slices, Σ x · f(x)dx, using nothing we have not already built. Now let the slices get thin.

Drive the bin slider on the triangle and watch the running total. Five bins give 0.6600, ten bins give 0.6650, and forty bins give 0.6666, closing in on the exact answer 2/3. That is Chapter 3's Riemann sum doing exactly the job it was built for, and in the limit we write it as an integral:

E[X] = ∫ x · f(x) dx

So the dx in that expression is not punctuation, and it is not there to tell you which letter to integrate. It is half the weight. Pair it with f(x) and you have the continuous stand-in for p(x), doing the identical job of saying how much this value counts.

Hold the two formulas side by side and notice how little actually changed. Sum becomes integral, and p(x) becomes f(x)dx. The idea underneath is untouched, and it is still one sentence: multiply each value by its weight and add up.

05★★ One machine, any function

Here is where the chapter turns, and everything downstream is a consequence of the next few paragraphs. In practice you almost never want the average of X itself. You want the average of some function of it.

An option pays max(S − K, 0), which is a bent function of the price. A risk report wants E[X²], because squares are how spread gets measured. A growth calculation wants E[e^X], because compounding is exponential. In every case the thing you care about is g(X) rather than X.

So take the smallest possible example and try it honestly. Let X be uniform on {−1, 0, 1, 2}, so each value carries weight 1/4. What is E[X²]?

Your instinct says to build the new variable first. Call it Y = X², work out what values Y takes and with what probabilities, and only then average. Squaring the four values gives 1, 0, 1, 4, so Y lands on 0 with probability 1/4, on 1 with probability 1/2, and on 4 with probability 1/4. Average that and you get 0(0.25) + 1(0.5) + 4(0.25) = 1.5.

That is correct, and it is also more work than it needed to be. Watch the shortcut.

$ python lotus.py
▍

Skip Y entirely, and instead square each original value, weight it by X's own probability, and add: ((−1)² + 0² + 1² + 2²)/4 = 6/4 = 1.5. Same number, and the distribution of X² never had to be constructed at all. Written out in general, that shortcut is

E[g(X)] = Σ g(x) · p(x)  and  E[g(X)] = ∫ g(x) · f(x) dx

It carries a slightly rude name, the Law of the Unconscious Statistician, or LOTUS. The joke is that people use it without noticing they have skipped a step. We are not going to skip it, because the step is one honest line and it is the reason the whole thing is legal.

Here it is. On the event {X = x}, the transformed value g(X) is g(x). There is nothing left to be uncertain about. So whatever probability that event carried, g(x) now carries the same amount, and the weights transfer across unchanged. No new distribution is needed because no new randomness was introduced.

What we actually have is bigger than a computational trick. Set g(x) = x and LOTUS hands back E[X]. Set g(x) = x² and it hands back E[X²]. Expectation is one machine, written E[·], and the function you point it at is a free choice.

And one warning arrives free with the machine, so take it now. In our example E[X] = (−1+0+1+2)/4 = 0.5, so E[X]² = 0.25. But E[X²] = 1.5. Those are not close and they are not meant to be. The mean of the square is not the square of the mean. That gap will turn out to be the most useful quantity in the chapter.

There is a fair objection to LOTUS hiding in that very example, and it deserves an answer rather than a shrug. Squaring is not one-to-one, since both −1 and +1 land on the same output. When LOTUS lists them as two separate terms, is it double-counting something?

$ python collision_audit.py
▍

It is not, and the audit shows why. The long route merges those two outcomes into the single value Y = 1 and adds their weights, giving 0.25 + 0.25 = 0.5. LOTUS keeps them apart and contributes 1²(0.25) + 1²(0.25), which is the same 0.5. Addition happened either way, and LOTUS simply does it inside the sum instead of beforehand.

06★ Linearity does not care about dependence

Point the machine at g(x) = ax + b and something very convenient falls out. Pull the constant a outside the sum and separate the b term, which is legal for any finite sum, and you land on E[aX + b] = a·E[X] + b. Scaling and shifting pass straight through expectation untouched.

The version that matters is the one for sums of two variables:

E[X + Y] = E[X] + E[Y]

Read that carefully, because there is no condition attached to it. No independence requirement, no mention of correlation, nothing about how X and Y were generated. It is true when the two are strangers and it is true when one is a strict function of the other.

Your gut will refuse this, and the refusal is understandable. Chapter 10 spent a whole page showing how much dependence changes, so surely it must change an average too. Here is the reason it cannot. Go outcome by outcome across the sample space and write the weighted sum of X + Y. Split it into the X part and the Y part. Nothing else happened there, so all you did was reorder a finite sum, and reordering never asks whether the terms know each other.

# explicit JOINT tables — no independence
seed = 42
def E(P, f):        # expectation the HARD way
  return sum(f(x,y)*P[x][y]   # weight
             for x,y in cells)  # all 36
assert E(x+y) == E(x)+E(y)  # ALWAYS
try:
  assert E(x*y) == E(x)*E(y) # ?
except AssertionError:
  print('products do not split')

That panel runs three couplings that could hardly be more different. The first is two independent dice, and the second is the same die counted twice, so Y = X and the two move in perfect lockstep. The third is Y = 7 − X, which is as perfectly opposed as two variables can get, since their sum is pinned at 7 forever. All three print E[X + Y] = 7.0000.

Their variances are 5.8333, 11.6667 and 0. So dependence is enormously important, and Chapter 18 is largely about measuring it. It simply has no grip on the mean, because the mean only ever adds. Spread multiplies terms together, and multiplication is exactly where the entanglement bites.

On its own the rule looks almost too plain to be a tool. It turns problems that appear to need a full joint distribution into a short sum of easy pieces.

Here is the classic. n people check their hats at a restaurant, and a careless attendant hands them back in a completely random order. How many people, on average, get their own hat back? Commit to an answer, and notice whether your answer depends on n.

# block 1 — this loop actually ran ↓
from itertools import permutations
for n in range(1,9):
  fp = 0
  for p in permutations(range(n)):
    fp += sum(p[i]==i for i in range(n))
  print(n, fp/factorial(n))  # → 1.0

The answer is 1, and not roughly one or one only once n gets large. It is exactly one, for two people and for two million. Here is the whole argument. Define I_k as 1 if person k gets their own hat and 0 otherwise, so the total number of matches is I_1 + I_2 + … + I_n. Each person is equally likely to receive any hat, so E[I_k] = 1/n. Add up n of those and the total is n × (1/n) = 1.

Look at what we never did. Those indicators are badly tangled, since person 1 receiving hat 3 changes the odds for everyone else. Working out their joint behaviour is genuinely hard. We did not need it, because we only ever added them, and addition is blind to dependence. That move has a name worth remembering: write the thing you want as a sum of indicators, then take expectations one at a time.

07Aim the machine at spread

Two investments both return 5% on average. The first returns between 4% and 6% every year without fail. The second returns 40% or loses 30%, roughly at random. The mean cannot tell them apart, and no sane person would treat them as the same instrument.

So we need a second number, one that says how far X typically wanders from μ. The natural first attempt is the plain average deviation, E[X − μ]. Try it on our bet before reading on, and guess what it gives.

The three weighted deviations come out as −1.1667, +0.2222 and +0.9444, and they add to precisely 0. That is not bad luck with these particular numbers. Linearity says E[X − μ] = E[X] − μ, and μ is defined as E[X], so the answer is zero for every distribution that has a mean.

That failure is telling us something true. The positives and negatives cancel exactly, and that perfect cancellation is precisely what makes μ the balance point in the first place. Our measuring tool destroyed the very signal it was built to read, so the sign has to die before we average.

Two ways to kill a sign. Take the absolute value, or take the square. The absolute value is honest and people do use it, but it has a kink at zero and is not differentiable there. Squaring is smooth, which matters enormously once Chapter 8's calculus starts optimising these quantities. It also punishes one big miss far more than several small ones, which is usually what a risk manager wants.

So point the machine at squared distance from the mean, and give the result its name:

Var(X) = E[(X − μ)²]

On our bet the three weighted squared deviations are 2.7222, 0.1481 and 5.3519, summing to 8.2222. Now a tamer bet with the identical mean of 0.3333, paying £1 a third of the time and nothing otherwise. Its variance is 0.2222. Same centre, wildly different behaviour, and at last a number that can see the difference.

One nuisance came along with the square, and it is not a small one. Our payoffs were in pounds, so the variance is in pounds squared. Nobody has ever been paid in pounds squared. Take the square root and you land back in the original units:

σ = √Var(X)

That is the standard deviation. For our bet it comes to √8.2222 = 2.8674 pounds, against 0.4714 pounds for the tame version. This is exactly why finance quotes volatility as a percentage rather than a percentage squared, and Chapter 16 will lean on that habit constantly.

08The identity you derive, not memorise

The definition works, but computing it is clumsy. You have to find μ first, then go back through every outcome subtracting it before you can square anything. Two passes. There is a cleaner route, and it is worth deriving rather than accepting.

Step through it and every line is something you already own. Expand the square algebraically: (X − μ)² = X² − 2μX + μ². Apply E[·] to all three terms, and pull the constants out using linearity, which is now available to us. That gives E[X²] − 2μ·E[X] + μ². Finally, E[X] just is μ, so the middle term becomes 2μ² and eats one copy of the last one:

Var(X) = E[X²] − E[X]²

In words, the mean of the square minus the square of the mean. Check it on the LOTUS example, where we already have both pieces: 1.5 − 0.25 = 1.25. Squaring the four deviations directly and averaging gives (2.25 + 0.25 + 0.25 + 2.25)/4 = 1.25 as well.

People flip the order of those two terms constantly, so here is the fix that never fails. Variance is an average of squares, so it can never be negative. The bigger term has to come first, which means E[X²] ≥ E[X]² always holds. If you ever write it the other way you will be claiming a negative variance, and the sign will tell on you immediately.

That inequality is more than a sign check. Earlier we found that E[X²] and E[X]² disagree, and now we know which one is larger, and by exactly how much. The gap between them is the variance. No coincidence, and the last section is about what it generalises to.

One honest footnote before we leave the identity. It is beautiful algebra and a slightly dangerous formula on a computer, because subtracting two large nearly-equal numbers destroys precision. Chapter 39 meets Welford's algorithm, which is how production risk systems actually compute a variance.

09★ Which way it leans

We know that E[g(X)] and g(E[X]) are generally different things, because averaging then transforming is not the same as transforming then averaging. What we have not answered is which of the two comes out bigger. The answer is more satisfying than it has any right to be.

Start with the case we already proved. For g(x) = x² we know E[X²] ≥ E[X]², because the difference is a variance. So for squaring, the average of the outputs sits above the output of the average. Squaring is the standard example of a convex function, and Chapter 3 drew convexity as a curve that bows upward with its chords lying above it.

Drag the two points and the picture explains itself. Put X at 1 or 3 with equal weight. The average input is 2, so g(E[X]) = 4, which is a point on the curve. The average output is (1 + 9)/2 = 5, which is the midpoint of the chord. The chord sits above the curve, so 5 sits above 4, and the gap of 1 is precisely Var(X).

That is Jensen's inequality, and it holds for every convex function rather than just for squaring:

g convex ⇒ E[g(X)] ≥ g(E[X])

Flip the curvature and the inequality flips with it. For a concave function such as ln, the chord lies below the curve, so E[g(X)] ≤ g(E[X]). With the same two points, ln(E[X]) = ln 2 ≈ 0.6931 while E[ln X] = (0 + 1.0986)/2 ≈ 0.5493. Equality arrives only when the function is a straight line, or when X is a constant and there is no spread to bend.

So the whole rule is curvature, and nothing else. Convex bows up and the average of the outputs overshoots, while concave bows down and it undershoots instead. A straight line has no curvature at all, which is exactly why linearity was exact.

Now the payoff, and it is the seed of an entire chapter. Suppose an asset's log return is +0.2 or −0.2 with equal chance, so the average log return is exactly 0. The actual growth factor is e^X, and e^x is convex, so Jensen guarantees a gap. Working it out, E[e^X] = (1.2214 + 0.8187)/2 ≈ 1.0201 against e^(E[X]) = 1.0000.

Read that gap slowly, because it is the shape of a real market fact. The arithmetic average return is above the compounded growth an investor actually experiences, and the gap widens with volatility. Here the excess is about 0.0201, and σ²/2 = 0.02 sits right beside it. Chapter 26 shows that for a lognormal asset the relationship is exact, and the correction μ − σ²/2 is the famous volatility drag.

So let's collect the chapter, because it really is one object. A random variable pins a number to every outcome. The distribution carries those numbers with their weights, as bars if the values are countable and as a density if they are not. Expectation is the probability-weighted average of whatever function of the outcome you choose.

Everything else on this page was that same machine, aimed somewhere new. Point E[·] at x and you get the mean. Point it at (x − μ)² and you get the variance. Point it at e^(tx) and Chapter 12 will call it the moment generating function. Point it at a discounted option payoff and Chapter 27 calls it a price.

Carry three things forward. Linearity lets you split a sum without ever asking how the pieces are related, and that is the sharpest tool on this page. E[X²] is never E[X]², and the gap between them has a name. And whenever a curved function stands between you and an average, check which way it bends before you trust the answer.

Next we stop inventing distributions one bet at a time. There is a small family of shapes that keeps reappearing, and they are related to each other by simple limits rather than by coincidence. Learn the root of that tree and the branches regenerate on demand.