16Sampling, Standard Error & Estimators

Chapter 15 closed on a sentence that has been quietly true since Chapter 9. An expected value is never a fact about the world. It is the output of a model somebody chose, with three slots filled in by hand. Kelly ate the number p and never asked where it came from, and we watched what happened when the answer was "a hundred noisy flips". So here is the question this chapter turns on, and the whole of Part 2 grows out of it: who fills those slots in when nobody knows the answer? From this page the arrow runs backwards. You hold the data, and the model is the unknown. That reversal does something strange and wonderful to a number you compute from data. It makes that number random — not the digits on your screen, which are finished, but the procedure that produced them. A number computed from data is itself a random variable, so it has a distribution of its own. The width of that distribution, and not the width of your data, is exactly how much you are allowed to believe yourself. By the end you will be able to draw a cloud you can never observe, using one sample and a square root. You will know why n−1 is a derivation and not a convention. And you will know why a century of market history still cannot tell you what a stock is expected to return.

Look at what this page stands on. Chapter 9 gave us the sample space and favourable-over-total. Chapter 11 gave us E[X], linearity, variance and Jensen's inequality. Chapter 12 gave us the Normal and the 68-95-99.7 rule. Chapter 13 did the heaviest lifting: the √n law, the standardized average, and the central limit theorem, plus two words it used and deliberately did not define. Those two words were standard error and sampling distribution, and this chapter is where that debt gets paid in full.

That is the seam. Everything before it ran model → data: here is p, now tell me what the coin will do. Everything after it runs data → model, and the first casualty of that reversal is a habit almost nobody notices they have.

01The arrow turns around

Here is a price series with a hundred daily returns in it, and I want you to tell me what this stock is expected to return on an average day.

You will average them, because that is obviously the right thing to do, and you will get 0.043% a day. Now let me tell you something you could not have known. That series came out of a machine, and I set the machine's true expected daily return to 0.020%. Your answer was off by more than double.

Press the machine again, with the same settings and the same hidden truth, over a completely fresh hundred days. This time the average comes back −0.011%, which does not even have the right sign. Press it once more and you get 0.061%. The truth never moved. Your answer moved every single time.

That is the phenomenon, and now it earns its two names. The fixed number out in the world is the estimand, written θ, and in this case it is μ = 0.020%. It is real, it is fixed, and you will never see it. The number you computed is the estimate, written θ̂ and read "theta hat", and here it is x̄ = 0.043%. It is known, it is sitting on your screen, and it is different every time.

These are two different objects, and almost everyone arrives believing they are one object seen at two levels of focus. That belief is completely reasonable and it is the thing this chapter has to take away from you. In every problem you have ever solved, "the mean" was handed to you in the question, so computing an average felt like arithmetic rather than estimation. Nothing so far has ever put a number in front of you that is real, fixed, and permanently invisible.

Now the second inch, and this one is aimed at finance readers in particular. The word population sounds like it means a big group of people, so it sounds like something you could in principle finish collecting. Suppose you own every daily return the S&P 500 has printed since 1928. That is about twenty-five thousand numbers, the complete historical record, nothing missing. Surely now you know μ exactly?

You do not, and the reason is the whole of Part 2 in one sentence. The population is the process, not a bigger dataset. What generated those twenty-five thousand numbers was a data-generating process — a machine of firms and traders and shocks — and the record is one long draw from it. Chapter 15 already told you that the model is something somebody chose, never something the world hands over. The one thing you can never do is take a second draw of the twentieth century.

So the arrow has turned around, and it has left us holding exactly two objects. There is a θ we want and cannot see. There is a θ̂ we have and cannot trust. Everything else in this chapter is the relationship between them.

02Where the randomness actually lives

Let us make that relationship precise, and the first step is a piece of notation that is switched silently on every page of every statistics textbook. Once your data is on the screen it is not moving. Those hundred returns are finished. So what, exactly, is random?

The randomness is in the design, not in the numbers. Before you look, the sample is X1, X2, …, Xn — n random variables, each an independent draw from the same population. After you look it is x1, x2, …, xn, which are n perfectly ordinary numbers. Capital letters are the plan. Lowercase letters are what the plan happened to produce this once.

Press the button and dots fall out. Press it again and different dots fall out, from a machine that never changed. Every probability statement in statistics is a statement about that press. It is never a statement about the dots that already landed.

This matters more than notation usually does. When you read E[X̄], you are not being asked for the expectation of a quantity that is already determined, which would be either nonsense or a tautology. You are being asked what the averaging procedure does across the presses you did not make. Hold that and the rest of the chapter parses. Miss it and every sentence from here reads as word salad.

Now the assumption hiding inside "each an independent draw from the same population". Those three words get compressed into the letters iid, and they are usually muttered before the real work like a spell nobody unpacks. They are the load-bearing assumption of everything that follows, and in market data they are false.

Turn on the dial that makes each draw lean toward the one before it, and watch the sample stop looking like a fair picture of the population. You get clusters, you get runs, and you get a mean that wanders further than it should. That leaning has a name — autocorrelation — and Chapter 21 gives it a full working. Meet it once here as a picture, so that its name later lands on something you have already seen.

And take the honest cost with you now rather than later. At a stickiness of 0.5, the spread of the sample mean measured across 200,000 runs is 1.72× what the independent-draws formula predicts. So the error bar you would have quoted is 72% too narrow, and you would have quoted it with total confidence. Everything in this chapter assumes the button was pressed cleanly. Markets do not press it cleanly, and Chapters 21 and 32 exist because of that. The formula is not wrong, by the way, and that is the uncomfortable part. It answers a question about a world where each draw ignores the last one.

03An average is a noun

A statistic is any number you compute from the sample. The mean is one of them, and so are the variance, the maximum, the median, and a Sharpe ratio. Feed random variables into a function and what comes out is a random variable, and that is not a new theorem at all.

It is Chapter 11's definition, read on a bigger sample space. A random variable was only ever a rule that takes an outcome and returns a number. Here the outcome is the whole sample and the rule is "add them up and divide by n". The definition already covers this case, and Chapter 9 built a sample space out of one roll rather than out of a hundred rolls at once. Nothing in that construction ever objected to the larger version. Saying so out loud is the entire move, and almost no course says it.

Which means X̄ has to stop being a verb. Right now it is something you do: add, divide, done. It has to become a noun — an object with a distribution, a mean, a variance and a shape. As long as it stays a verb, the phrase "the variance of the sample mean" is unparseable, because verbs do not have variances.

Here is the smallest example that forces the change, and you already own every piece of it. Roll two dice. But instead of taking the sum, take the mean of the two faces.

There are 36 equally likely samples, so lay them out in a 6×6 grid and colour each cell by its mean. Sweep the cells into a histogram and a familiar triangle appears. It is the same triangle you met in Chapter 9 as the distribution of the sum, and it turns out to have been a sampling distribution the whole time, wearing different axis labels.

Read the numbers straight off it. There are 11 possible values, from 1.0 to 6.0 in steps of 0.5. The mean of those values is 3.5, exactly μ. And the standard deviation is 1.2076, which is where this gets good.

Check that against Chapter 13's √n law. One die has σ = √(35/12) = 1.7078. Divide by √2 and you get 1.7078 / 1.4142 = 1.2076. It matches to seven decimal places. We did not assume the √n law here and we did not approximate it. We counted 36 cases with Chapter 9's favourable-over-total, and the previous chapter's promise came true inside the answer.

Notice what this kills. Most readers assume the distribution of X̄ needs the central limit theorem, or a simulation, or some heavy machinery arriving later — and so they file it away as an approximation they will have to take on trust. It is not an approximation at all: for a small enough case it is exactly computable by counting, and you have just done it.

04The sampling distribution

That histogram has a name, and it is the centre of the entire subject. The sampling distribution is the law of a statistic across all the samples you could have drawn.

Sit with the strangeness of that for a second, because it is genuinely strange. You will only ever draw one sample. The distribution is spread over repetitions that never happen. With the dice it felt safe, because all 36 samples were visible at once on the grid. With real data, 35 of them are invisible forever.

And the miracle this whole chapter is built on is that you can describe that distribution anyway, from the single sample you actually hold. We will get there in a moment. First we have to clear up a collision that causes more wrong error bars in finance than any other single confusion.

Three different objects now wear the word "distribution". There is the population's distribution, which is a fact about the world. There is your sample's own histogram, which is the only one you will ever hold in your hands. And there is the statistic's distribution, the cloud we just met. Textbooks draw all three with the same axis label, at different scales, on different pages. Readers fuse the sample with the population, which is survivable, and then fuse the sample with the statistic, which is not. That second fusion is the error behind almost every misquoted error bar in finance.

So here they are on one screen, with the axes locked to the same ruler and a slider on n. Panel one is the population, and it is wide, fixed, and completely unmoved by anything you do to the slider. Panel two is one sample. It jitters whenever you resample, and here is the part to watch: it stays about as wide as panel one no matter how large you make n. Panel three is the cloud of x̄ values. It jitters too, it sits far narrower, and it visibly collapses as n rises.

Your eye settles the whole confusion in about three seconds. Panel two's width is a fact about the world and it refuses to shrink. Panel three's width is a fact about your knowledge and it shrinks like 1/√n. Chapter 13 separated data from average with two histograms; this is that picture with the middle panel put back where it belongs.

Now take panels one and three away, and leave only panel two. That is the real situation. Always. You have one sample and no way to see the cloud your estimate came from.

Run it in two beats, and commit out loud on both. Beat one: your hundred returns average 0.043%, and I am about to draw a fresh hundred from the identical world. Write down what the new average will be. Most people write 0.043 or something near it, and some protest that it cannot be predicted — and both of those answers are the admission we need. The new average comes back 0.061%. Draw again and it is −0.008%. Let 500 fresh samples rain down as dots and pile into a heap, and the number you had been treating as the answer is one dot in that heap. It never was anything else.

Beat two is the stroke. Wipe the 500 dots off the screen and put back your one sample, which is the real situation. Now, how many more samples do you need before you can say how wide that heap was? Every honest answer is a number. A few. A hundred. As many as possible.

The answer is zero. The heap's width is σ/√n, and the one sample in front of you already estimates σ. Draw the predicted curve from that single sample — one number and one square root — and then drop the 500 real dots back on top of it. They land inside a curve that was drawn without them.

That is the moment statistics becomes possible, and it is worth being explicit about what just happened. The cloud you can never observe was drawn from the one sample you did. Every confidence interval, every p-value, every t-statistic and every error bar on a Sharpe ratio for the rest of this course is that curve, drawn that way.

So the cloud is the object, and it has exactly three properties worth asking about. Where is it centred? How wide is it? What shape is it? Those three questions are called bias, standard error and the central limit theorem, and the next three sections take them in order.

05The centre — bias, and what it does not promise

The distance from the cloud's centre to the truth is the bias: bias(θ̂) = E[θ̂] − θ. For the sample mean that distance is exactly zero, and the proof is one line.

Take the expectation of the average. E[X̄] = (1/n)·ΣE[Xi] = (1/n)·(nμ) = μ. That is linearity of expectation from Chapter 11 and nothing else. Notice what the argument never used: independence appears nowhere in it. Each of the n draws contributes μ/n to the answer, and the n copies add straight back up to μ. This is Chapter 13's equal-shares lemma standing up again in new clothes.

So the sample mean is an unbiased estimator of μ. Now let me take most of that word away from you, because it buys far less than it sounds like it buys.

"Unbiased" is a compliment in English before it is a definition in statistics, so it gets heard as accurate, correct, trustworthy. It means none of those things. It means the cloud is centred on the truth, averaged over infinitely many samples you will never take. It makes no promise whatsoever about the one estimate you are holding.

Look at the dot near the edge of the cloud in that panel. That dot is somebody's Tuesday. Their estimator was unbiased the entire time it was lying to them, and every honest quant has shipped one. An unbiased estimate can be wildly wrong; unbiasedness is a property of the procedure, not of your answer.

There is a second trap here, and it detonates years later if we leave it. Readers assume unbiasedness passes through functions. If s² is unbiased for σ², then surely s is unbiased for σ?

It is not, and the tool that settles it is one you used two chapters ago on volatility drag. The square root is concave, so Jensen's inequality says E[√(s²)] ≤ √(E[s²]) = σ. The sample standard deviation is biased low, even though the sample variance is not.

Put numbers on it so it stops being abstract. For Normal data, E[s] sits 20.21% below σ at n = 2, 6.00% below at n = 5, and 0.25% below at n = 100. It is the curvature of the square root doing all of that work. It is the same curvature that made ln E[G] ≥ E[ln G] in Chapter 15, charging the same kind of fee for the same reason.

06The width — the standard error

The second question about the cloud is how wide it is. The standard deviation of a sampling distribution has its own name — the standard error — and Chapter 13 already handed us the formula for the sample mean.

Var(X̄) = σ²/n, so SE(X̄) = σ/√n. That is the number that answers "how much should I believe my own estimate", and it is a completely different quantity from the σ that describes the data. Chapter 13 named the words and promised a working; this is the working.

Now the most expensive confusion in applied statistics, which is a naming failure rather than a conceptual one. σ and SE are both standard deviations. Both get written as a ± after a number. Both are printed in the same units. Nothing in the words tells you which random variable each one belongs to, and that is the entire distinction.

Two rulers on one screen, locked to the same scale, with one slider. Ruler A is labelled "how spread out the returns are". Ruler B is labelled "how spread out my answer is". Before you drag n from 10 to 10,000, predict which one moves. A good fraction of readers expect both to shrink, and that prediction is exactly the misconception worth extracting.

Ruler A does not move a millimetre, and it never will, because it is a fact about the market. Ruler B collapses towards a point. So σ describes the world and does not shrink with data; SE describes your knowledge and shrinks like 1/√n. Only one of the two is bought with data.

Two live failures follow from mixing them up, and they run in opposite directions. Forwards: a reader with 100 daily returns of standard deviation 1.2% reports their mean return as 0.04% ± 1.2%, which is wrong by a factor of ten. Backwards: a reader believes that collecting more data makes the returns themselves less volatile, as though the market calmed down because they opened a longer spreadsheet.

Now land it in the real numbers, because the numbers are the whole argument. A hundred days of returns with σ = 1.2% a day gives SE = 1.2/10 = 0.12%. The sample mean was 0.043%. The error bar is nearly three times the size of the thing being measured. That single line is why quantitative research is hard, and it is not rhetoric. It is division.

One more honesty note before we move, because readers hear a guarantee where none was offered. A range of ±1 SE is not a bound. Under a Normal cloud it covers about 68% of the time, which means roughly one time in three your estimate lands outside it. It is a typical wobble, not a fence. So when a research note prints an estimate with a ± beside it, the first question is always which of the two numbers that is.

07The σ you do not have

Now look hard at the formula we just wrote down. SE = σ/√n. You have n, obviously; you counted your own data. You do not have σ.

That σ is a property of the population, and the population is the thing you are trying to learn about in the first place. So your error bar contains an unknown. Before you can quote your own uncertainty, you have to estimate the spread as well as the mean.

I want to slow down on this step even though nothing hard happens in it, because skipping it is what makes the next three sections feel arbitrary. In most courses, Bessel's n−1 arrives as an unmotivated rule in a box, next to a formula nobody had any reason to want. If you never felt the need for s, then n−1 is a convention — and a convention can only be memorised and then forgotten, never re-derived.

There is a second thing hiding here that most books never say at all. Your error bar has an error bar. You are about to substitute an estimate for σ and carry on as though nothing happened, and something did happen. That something is precisely why the t-distribution exists, and we will meet it later in this chapter rather than having it appear from nowhere in the next one.

So take the obvious candidate and use the sample's own spread: average the squared distances from x̄, dividing by n. It is the natural guess, and we now have a tool for judging it. Ask whether it is biased.

It is, and it is biased low. Not scattered low — shifted low, every time. And the reason has nothing to do with probability at all.

x̄ is the point that minimises the total squared distance to your data. That is what an average is, and one line of Chapter 3 proves it. Differentiate f(a) = Σ(xi−a)² to get −2Σ(xi−a), set that to zero, and a = x̄ falls straight out. The second derivative is 2n > 0, so it is a genuine minimum and not just a stationary point.

So squared distances measured from x̄ are smaller than squared distances from any other point. Including from the true μ. Always, in every sample.

Do not take that on trust — hunt for a counterexample. Two points on a line, a marker for the true μ, and two bars drawn live: the sum of squared distances to x̄ in one colour, the sum to μ in another. Drag the points anywhere you like, through as many configurations as you have patience for, and try to find a case where the x̄ bar is longer. You cannot. The two bars touch only in the instant x̄ lands exactly on μ.

That is bias seen as an inequality, with your own hands, before any algebra. And an inequality is a much stronger fact than a tendency. Readers who accept there is a bias usually assume it is bad luck that could go either way and would average out. It cannot go the other way.

Here is the sentence that makes it permanent. x̄ is the point that best fits your data, so measuring the scatter around it is grading your own homework. And flag the forward link now, because it is the same fact wearing a different hat: "the point minimising squared distance" is exactly what OLS computes in Chapter 20, one dimension at a time.

08Exactly one observation's worth

How much too small, exactly? Predict before you read on, because the surprise lives in the size rather than in the fact. Is the shortfall a fixed fraction of the variance? Is it proportional to n? Does it fade away as the data piles up?

Most people guess proportional. The answer is a constant.

The derivation is three lines and each one earns its place. Start by writing the deviation from the sample mean in terms of the deviation from the true mean: Xi − X̄ = (Xi − μ) − (X̄ − μ). That is add-and-subtract μ, and it is the only trick in the whole thing.

Square it and sum over i. The cross term is −2(X̄−μ)·Σ(Xi−X̄), and Σ(Xi−X̄) = 0 by the definition of X̄, so that term dies for free. What is left is clean: Σ(Xi−X̄)² = Σ(Xi−μ)² − n(X̄−μ)².

Now take expectations of both sides, and stop on the second term, because this is where every honest attempt at this derivation stalls. The first term is easy: E[Σ(Xi−μ)²] = nσ², straight from the definition of variance. The second term is n·E[(X̄−μ)²], and readers stare at that as if it were a new quantity to compute.

It is not new. Look at it once more: E[(X̄−μ)²] is a variance — it is Var(X̄) written in its long form, because μ is the mean of X̄. And Chapter 13 already told us that Var(X̄) = σ²/n. So the second term is n·(σ²/n) = σ².

Put the two together: E[Σ(Xi−X̄)²] = nσ² − σ² = (n−1)σ². The shortfall is exactly one σ². Not a fraction of the data, not something that fades with n — exactly one observation's worth of variance, stolen by the act of locating the mean. Divide by n−1 instead of n and it is restored, exactly. That is Bessel's correction, and you just derived it. Notice that nothing in those three lines used the shape of the population, so the result holds for dice, for returns, and for anything with a finite variance.

The constant gap and its shrinking importance are two separate facts, so let us keep them separate. The missing σ² is the same size at every n. What changes is its share of the total. At n = 2 the naive rule returns half the true variance, so the correction is +100%. At n = 10 it returns 90% and the correction is +11.1%. At n = 1000 it returns 99.9% and the correction is +0.1%. Same missing piece throughout. It just matters less when there is more of everything else.

And because we already have a case small enough to count, let us not believe any of this on authority. Go back to the two dice and read those same 36 samples for spread instead of for centre. The naive rule takes the values 0, 0.25, 1, 2.25, 4 and 6.25 across those 36 cells, and its average is 1.4583. The true variance of one die is 35/12 = 2.9167. The naive rule returns exactly half of it, which is what (n−1)/n predicts at n = 2. Divide the same sums by n−1 = 1 instead and the average becomes 2.9167 — the true variance, to the last digit, by counting.

One honesty note that this counted panel makes unavoidable, and it ties straight back to what bias does not promise. The naive rule lands below the truth in 30 of the 36 samples. The unbiased rule still lands below the truth in 24 of the 36, which is two thirds of the time. Unbiased means the average is right. It has never meant that the typical sample is right.

09Degrees of freedom, defined

Now the phrase everyone repeats can finally be defined, and it deserves the attention because you will meet it again as n−k in regression, as the parameter of the t and chi-square curves, and as a number every software package prints without explanation.

A degree of freedom is a freedom of the residuals. Once x̄ is computed from the data, the n residuals xi − x̄ are no longer n free numbers. They must sum to zero, so tell me any n−1 of them and the last one is forced.

So a sample of n carries n−1 independent pieces of information about spread, and dividing the squared total by n−1 is dividing by how many free numbers actually went into it. Most readers can happily say "we used up a degree of freedom estimating the mean" and, if pressed for what a degree of freedom is, have nothing. That is a bad state to be in for something this reusable.

So pull on the constraint until you can feel it. Draw n dots with residual bars hanging off the mean line, and drag any bar you like. As you drag, another bar moves by itself to keep the total at zero. Drag n−1 of them into any configuration you please, and the last bar is no longer draggable. It is being held in place by the others.

Then take it to n = 1, which settles the whole thing. One data point, one residual, and that residual is forced to be exactly 0. Bessel's formula returns 0/0 — undefined — and that is the correct answer. One observation genuinely tells you nothing about variability, and the honest formula refuses to answer. Divide by n instead and it reports "variance = 0", which is a confident lie about a question that has no answer.

There is a geometric version of this, and it is worth having because Chapter 20 will need exactly this picture. Stack the residuals into a vector in n dimensions. The sum-to-zero constraint says that vector is perpendicular to the all-ones direction (1,1,…,1), using Chapter 6's dot product. So it is confined to an (n−1)-dimensional subspace, and its squared length is built from n−1 independent directions rather than n. That is what the number counts. And it is why a regression that fits k coefficients will divide by n−k.

10The shape — and the two places it lies

Centre and width are settled, so the last property of the cloud is its shape. Chapter 13 proved that an average of iid draws is driven towards the Normal. Read on this side of the course, that says something much stronger than it did there.

It says the sampling distribution of X̄ has a known shape. And a known shape is what converts a distance into a probability. "My estimate is 1.8 standard errors from that value" becomes a number you can act on, which is the entire bridge into hypothesis testing. In Chapter 13 the CLT described a mathematical limit; here it is a licence to attach probabilities to your own error, and that licence rests on its hypotheses being true of your data.

Three things need measuring rather than arguing about, so this one is a real run.

Panel one standardizes with the true σ. The histogram lies on the standard Normal at n = 5 and at n = 100 alike, and the fraction of runs beyond ±1.96 comes out 0.0504 and 0.0502 against the 0.05 it should be. Panel two standardizes with s from the same samples. At n = 100 you can barely tell the difference: the tail fraction is 0.0535. At n = 5 it is 0.1216, which is more than double what it should be.

That is the debt from the plug-in step coming due, and it now has a name. Once you divide by s instead of σ, the standardized quantity is no longer Normal, because the denominator is random too — and sometimes it is too small, which throws the ratio out into the tail. That fatter curve is the t-distribution, and Chapter 17 does the working. The extra weight in its tails is the exact price of not knowing σ, and it is a price that falls away as the sample grows. Put concretely: at n = 5 you need 2.776 standard errors for 95% coverage, not 1.960. The folklore rule "use n > 30" is standing in for this whole story, and at n = 30 the honest number is 2.045.

Panel three is the honest counterexample, and it matters more than it looks. Having learned that sampling distributions are Normal, readers apply that to every statistic — medians, maxima, variances, correlations, Sharpe ratios. The CLT is a theorem about averages.

So take the sampling distribution of the sample maximum, drawn from a perfectly Normal population, and measure its skewness as n grows. A Normal has skewness 0.0000. The maximum gives +0.30 at n = 5, then +0.55 at 30, +0.66 at 100, +0.79 at 1000, and +0.87 at 10,000. It is not converging to Normal. It is walking away from Normal, toward a different limit entirely, and more data makes it worse. That is exactly why Chapter 32 needs a separate theory for tails.

And keep the fine print you already met in Chapter 13. The CLT converges in the middle first and in the tails last, which is a problem precisely because a risk number lives in the tail. Everything in this section also assumed independent draws, which is the hypothesis this entire chapter rests on.

11What you actually pay

One last question about estimators, and it is the one that stops "unbiased" from hardening into a superstition. Two errors matter, not one: how far the cloud's centre sits from the truth, and how wide the cloud is.

Put them together and you get the mean squared error: E[(θ̂−θ)²] = Var(θ̂) + bias². The derivation is the same add-and-subtract move as before, this time around E[θ̂], with the cross term dying for the same reason. Bias and variance are measured in the same squared units here, which is exactly what lets you add them and get one score. And MSE is the only thing you actually pay.

So here is a rigged contest. I offer two estimators of the mean daily return: the sample mean, and the sample mean multiplied by 0.9. Which is better? Everyone picks the sample mean, because the other one is obviously biased.

Score them both on our running sample — 100 days, σ = 1.2%, true μ = 0.020%. The sample mean has zero bias and a variance of σ²/n, giving MSE = 1.44×10⁻⁶, so its typical error is 0.120%. The shrunken one carries a little bias and a lot less variance, giving MSE = 1.167×10⁻⁶ and a typical error of 0.108%. The biased estimator carries 81% of the unbiased one's MSE. It wins.

Turn the shrinkage dial and you can watch the trade directly. As the multiplier c falls from 1, the squared bias climbs from zero while the variance falls like c², so the total dips below the unbiased point before turning back up. That dip is the same picture Chapter 22 will call the bias-variance tradeoff, and it is why regularization, shrinkage and Bayesian priors will read as sensible engineering rather than as cheating.

The honest limit of the demonstration, stated plainly: the best multiplier here is c* = μ²/(μ² + σ²/n) = 0.027, which would cut the typical error to 0.020%. But that formula uses the true μ, which nobody has. It is a ceiling, not a recipe, and closing the gap between the two is what Chapters 19 and 22 are for.

Two more words finish the vocabulary, and they are routinely used as synonyms when they are not. Consistency means the whole cloud collapses onto θ as n grows, which is Chapter 13's law of large numbers seen from this side. Efficiency means that among unbiased estimators, yours has the narrower cloud. Unbiasedness is about the centre at a fixed n. An estimator can have either property without the other.

Now cash the whole chapter out where it decides real money. Estimating a mean return and estimating a volatility are not equally hard. They are not even close, and the reason is one line of Chapter 5.

Before the numbers, commit to an answer. A strategy shows a Sharpe ratio of 1.0, meaning its annual return is one annual standard deviation. How many years of track record would you want before you believed it was not luck? Guesses land anywhere from a few months to a lifetime.

Now derive it, and start from the one fact that makes it work. Over T years, log returns add, so the total has mean μT and variance σ²T. The estimated annual mean is that total divided by T, so SE(μ̂) = σ√T / T = σ/√T. Look at what is missing from that expression. There is no n in it. Sampling frequency does not appear at all.

The run confirms it and the table is worth staring at. One year of data sampled daily, hourly and by the minute gives standard errors on the annual mean of 0.2029, 0.1984 and 0.2023 — all of them σ/√T = 0.20, unmoved. Over the same three rows the standard error of the estimated volatility collapses from 0.0089 to 0.0033 to 0.0009. Sampling faster does nothing for the mean and almost everything for the vol.

That asymmetry is the punchline, and it is genuinely surprising the first time. The precision of a mean is set by the calendar. The precision of a volatility is set by the count. So μ takes decades and σ takes weeks — one year of daily data pins a 20% volatility to about ±0.9 percentage points, while pinning the mean to ±2% a year takes a century.

That century is not a figure of speech. With σ ≈ 20% a year for equities, a hundred years of data gives SE(μ̂) = 20/10 = 2% a year, and that is the best anyone will ever do with that dataset. Read against an expected return of roughly 8%, a 95% interval spans about four points either side. People hold "we have a hundred years of data" as though it settles things. It settles the risk. It barely touches the return.

And now the sentence a quant lives by falls out in one more step. The t-statistic is the estimate over its own standard error, so t = μ̂/SE = (μ/σ)·√T = SR·√T. A Sharpe of 1.0 needs four years to reach two standard errors. A Sharpe of 0.5 needs sixteen. A Sharpe of 0.3 needs forty-four. That is how long an edge must survive before anyone should believe in it, and it is why so many track records prove nothing at all.

Two conventions you have been using now have reasons instead of customs. Variances add over time, so annualizing a daily volatility means multiplying the variance by 252 and taking the root — that is Chapter 13's √252, and it is not a fudge factor. And finance quotes σ rather than variance because σ is in percent, which a human can feel, while variance is in percent-squared, which nobody can.

One honesty note on the volatility side, because I would rather flag a debt than over-claim. The relative error of an estimated σ is about 1/√(2n) for Normal data, which is where the ±0.9 points came from. Deriving that properly needs fourth moments, and those arrive with kurtosis in Chapter 29. Take the number now and the derivation then.

Look at what you can do now that you could not do at the top of the page. Shown any number computed from data, you reach automatically for three properties of the cloud behind it: where it is centred, how wide it is, and what shape it has. You know all three can be had from the single sample in front of you, which is the trick the whole of statistics is built on. You will never again confuse σ with SE, because you watched one refuse to move while the other collapsed.

You can derive n−1 rather than reciting it, and you can answer the question almost no practitioner can. A degree of freedom is a freedom of the residuals, which must sum to zero, so n numbers carry n−1 independent pieces of information about spread. And when n = 1 the formula honestly refuses to answer instead of confidently lying.

Everything from here is one more sentence about the sampling distribution. Chapter 17's p-value is a tail probability of it, and its confidence interval is its width read backwards. Chapter 19's multiple-testing problem is what happens when you sample enough clouds to find a lucky dot. Chapter 20's regression standard errors are this same object for a slope, with n−k in place of n−1. The bootstrap is this object computed by brute force when no formula exists, and Chapter 30 finally gives the Sharpe ratio the error bar it has been missing.

And the crack is already visible, which is the honest part. Every result on this page assumed independent draws and a finite variance. Markets break both, and they break them exactly when it matters most. That is the door Chapter 21's autocorrelation and Chapter 32's fat tails walk through.

One sentence to carry out of here, smaller and more useful than any formula on the page. Before you believe a number you computed, ask how wide the cloud behind it was.