14Two Variables at Once

Chapter 13 handed you a machine with one moving part, and it never failed you. Point it at Y = g(X), chase the event, collect the Jacobian, and a new density falls out the other end. Then it hit a wall. We wanted χ²(k) = Z₁² + Z₂² + … and we could not prove it, and I told you plainly why. The machine re-rules one axis, so adding needs two variables standing in the same room, and we had no room. So here is the gap, said out loud. Everything in this book so far has been one variable at a time. Real problems have several variables moving together, and your instinct already has an answer ready for what that means. Two variables, two distributions, so just do Chapter 8 twice. That instinct is wrong. It is not slightly wrong. It is missing an entire object, and a fair die is going to prove that to you in the first two minutes. Here is where we come out. Two variables at once is not two distributions. It is one pile of probability mass spread over a plane instead of a line. And every question you can ask about the pair is one of exactly two physical moves on that pile. You can project the pile onto an axis, which is the law of total probability, drawn. Or you can slice the pile and rescale the cut face to area 1, which is Chapter 5's shrink the world and re-measure, finally drawn. And independence turns out to be exactly the case where every slice has the same shape.

01Two numbers off one roll

We are going to build nothing new here. Chapter 8 told you that a random variable is a function from outcomes to numbers, written X : Ω → ℝ, and Chapter 13 made us read that definition again because it had quietly rotted. So take the machine you already own and run it twice on the same outcome. One experiment happens, one outcome lands, and two readers each report a number about that one outcome. That is not new theory. It is a second reading of a thing that has already happened.

There is Ω, the same rack of real outcomes it has always been, and there is the same P measuring it. What is new is that two arrows leave each outcome now instead of one, and they land on a pair of numbers, (X, Y). Nothing was re-randomised. The experiment was not re-run. So here is the one question nobody has asked you in thirteen chapters. Do the two outputs remember each other? You have never needed to ask it, because you have never had two readings of one outcome on the table at the same time. Let me show you why the question is not academic, using the smallest example I can build.

Two worlds, both made out of a fair die. In World A you roll two dice: X is the first die, Y is the second. In World B you roll one die: X is the die, and Y = 7 − X. Now look at the two 1D pictures: in both worlds X is uniform on 1..6. In both worlds Y is uniform on 1..6 as well, because 7 − X runs over exactly those six values, each with probability 1/6. The two pictures are not merely similar. They are identical, digit for digit, and here they are printed side by side. So commit to an answer before you scroll: what is P(X + Y = 7) in each world?

Your gut multiplied. It said 1/6 in both worlds, because that is the only move a pair of separate 1D tables permits. World A really is 1/6, so half the gut was right. World B is 1.000. Dead certainty. The pair in World B always sums to seven, because we built Y out of X to make it so. Sit with what just happened, because nothing was hidden from you and nothing mystical occurred. The two worlds have identical shadows and they behave in opposite ways. So the two 1D tables simply do not contain the answer, and no amount of staring at them will produce it. Something real lives between the two variables, and it is invisible from either axis alone. That gap is why this chapter exists.

02One pile of mass on a plane

So let's write down the thing that lives between the two variables. I want to build it before I name it, because the name is going to sound like a new axiom and it is not one. Go back to the two dice worlds and just count. Thirty-six equally likely outcomes sit on a 6×6 grid, with die one down the side and die two across the top. Fill in each cell with the probability that the pair lands in that cell. No theory, just counting.

You have just built two grids with the same edges and completely different insides. But you only counted what was handed to you, so pin those four edge totals down and try to move the mass around yourself. How many different grids can hide behind one pair of shadows?

World A puts 1/36 in every cell, flat as a table. World B puts 1/6 down the anti-diagonal and 0 in the other thirty cells, because those thirty pairs simply cannot happen. Both grids still hold all the mass: thirty-six cells at 1/36 total 1, and six cells at 1/6 total 1 as well. Look at the edges of the two grids and you get the same totals, exactly as promised. Look at the interiors and you are looking at completely different worlds. What the shadows lost is now something your eye can just see. Now the name. That grid of numbers is the joint PMF, written p(x,y). And read the comma out loud, because it is doing more work than it looks: p(3,4) means P(X=3 AND Y=4). That comma is Chapter 3's ∩ and nothing else.

Which means the joint added no probability theory at all. Watch how cheap it was. {X=3} is an event, because Chapter 8 built it that way, and {Y=4} is an event for the same reason. Their intersection is an event too, by Chapter 3, and Chapter 3 has known how to measure events since Part I. So p(x,y) is a probability you have been able to compute since the second week of this book. The grid is not a new object. It is only where we choose to write the numbers down. And the cells are disjoint, and together they cover everything, so the mass on the grid sums to exactly 1, which is Chapter 3's total mass, unmoved.

Now let the cells shrink, exactly the way Chapter 8 shrank them to cross from a PMF to a PDF. That was the sum-to-integral fork, and we are about to run the same fork one dimension over. But two things are about to change meaning underneath you, and nobody says them out loud, so they cause real damage. Chapter 8 drilled in that probability is the area under the curve. Promoted to two dimensions, that sentence is now wrong. And the word density is about to stop meaning per unit length and start meaning per unit area. That switch usually happens silently. So let me say it out loud instead, and let the units do the proof.

In one dimension the mass in a thin sliver was f(x)·dx, which is height × width, which is an area. Now take a grid cell here and shrink it to dx by dy. The mass in that cell is f(x,y)·dx·dy, which is height × tile, which is a volume. That is the whole promotion, and it is arithmetic rather than convention. So f(x,y) is the joint PDF, and it is priced per unit area. The probability of a region is the volume of mass sitting over that region. The total is ∫∫f dx dy = 1. The grid just became a landscape. Now let's make sure Chapter 8's oldest sin didn't sneak back in wearing a new costume, because the picture is new and your gut is going to try it again.

Drag the region around and the enclosed mass ticks up honestly, accumulating as volume. Now shrink that region toward a single point and watch two readouts at once. The mass runs to 0. The colour under your finger never changes. So P(X=x, Y=y) = 0, exactly as in Chapter 8, and the height was never a probability. The height is a rate: it tells you how thickly the mass is piled at that spot, and it only becomes a probability once you multiply it by some room to sit in.

03Project it — the marginal

Now we have the pile, so let's make the first of the two moves you can make on it. Go back to World B's grid and ask for P(X=3), where you simply do not care what Y did. Everyone's instinct is to add up the row, all six of its cells, and everyone's instinct is right. But nobody ever tells you why that is legal. The licence is a theorem you already own and already paid for.

Here is the argument, and it is Chapter 6 word for word. Y must take some value. The six cells {Y=y} in that row are disjoint, and together they cover the row entirely. So they are a partition, and the law of total probability says P(X=3) = Σ_y P(X=3, Y=y). That is not a definition to memorise. It is Chapter 6 with the partition being "what did Y do?", and that is the whole justification. Watch the row's six cells detach and fly into a single cell out at the side of the table. Now the name, and the name is embarrassing once you see where it comes from. Row totals get written in the margin of the table, so we call p_X(x) = Σ_y p(x,y) the marginal PMF. The word is a place on the page. It never meant unimportant, which is roughly the opposite of what this operation does.

And those marginals are the shadows. They are the two identical 1D pictures the two dice worlds handed us. So the two 1D pictures that failed us back in the dice worlds finally have a proper name, and that name is the marginal.

Run the same move through Chapter 8's fork. The row becomes a column of the landscape, and the sum becomes an integral: f_X(x) = ∫ f(x,y) dy. People call this projection, and the word plants a picture in your head that is wrong. You imagine a silhouette, as if you walked around the side of the surface and traced its outline against the wall. It is not an outline at all. It is an accumulation: you pour the entire column of mass straight down and let it stack up into a bar. So predict before you look. Which projects higher, the tall thin spike or the long low ridge?

The silhouette model says the spike, because the spike is taller. The accumulation says the ridge, because there is more of the ridge to pour. Being wrong about that once is worth more than any sentence I could write, and the silhouette should now be dead. But a second question is sitting here, and every textbook walks straight past it. Why is ∫f(x,y)dy a density and not a probability? You have spent five chapters watching ∫f dx be a probability. Integration now feels like the operation that produces probabilities, so watching it hand back a density instead looks like an inconsistency, and most readers quietly file that under "don't ask".

So let's ask, because the answer is arithmetic on the units: f(x,y) is mass per (unit x)(unit y). Integrate over all of y and you spend the "per unit y". What is left over is mass per unit x, and that is a 1D density. That is not a convention someone chose, just the units doing their own bookkeeping. Watch the label on the vertical axis change as the integral runs, then check the derived marginal the way you would check any density: ∫f_X dx = 1, still. Nothing leaked.

04Slice it — the conditional

That was the first move. Here is the second, and it is the one you will use for the rest of your life. Instead of pouring the columns down, hold x still. Fix x = x₀ and read f(x₀, y) as a function of y alone. What you get is a cut face through the landscape, and it looks like the answer to "how does Y behave when X is x₀". Its shape is exactly right. Its size is wrong, and that gap is the whole trap. The shape looks so right that fixing the size is going to feel like fussy bookkeeping.

Drag the vertical line and watch the area readout on the cut face. It is never 1, at any x₀, ever. So the cut face is not a density, whatever it looks like. Now here is the part that defuses everything coming next. That area is exactly the height of the marginal bar at x₀, the bar we built a moment ago by pouring columns down. Of course it is. We defined the marginal at x₀ as the total mass in that column, and this slice is that column, seen face-on instead of end-on. So the slice's area is f_X(x₀). That is not a new fact, just the same object looked at from a different angle. Remember it, because a denominator is about to appear and it is going to look like a rabbit out of a hat.

Except we cannot get to it yet, because a contradiction is sitting in the road and I am not going to drive around it. Chapter 5 defined P(A|B) = P(A∩B)/P(B). Chapter 8 proved that P(X=x₀) = 0 for a continuous variable. Put those two sentences together and "condition on X = x₀" reads 0/0. That flatly contradicts two things this book taught you explicitly. Nearly every course writes f(y|x) = f(x,y)/f_X(x) anyway and hopes you don't notice. If you felt that, you were right, and here is the honest fix, with nothing swept under the rug.

Don't condition on a point. Condition on a thin strip, x₀ ≤ X < x₀+dx, which is a real event with a real, positive probability of about f_X(x₀)·dx. Now Chapter 5's definition applies with nothing bent: the top of the fraction is the little tile's mass, f(x₀,y)·dy·dx. The bottom is the strip's mass, f_X(x₀)·dx. The dx sits in both, and it cancels on screen. Then let dx → 0 and watch the answer. It never moves. The ratio survives the limit even though the top and the bottom both run to zero, and that is why conditioning at a probability-zero point is legal. We did not dodge the division by zero. We showed that the ratio was never the problem in the first place.

So we have earned it: f(y|x₀) = f(x₀, y) / f_X(x₀). Now look at what that division physically does, because as algebra it is forgettable and as a picture it is permanent. The slice has area f_X(x₀). Divide the whole slice by its own area and the new area is exactly 1, by the plain definition of dividing a thing by itself. That is Chapter 13's renormalise discipline, verbatim. A density must be rescaled so the total mass stays 1.

That was a button on a smooth landscape. Now make the same move on numbers you can add up by hand. Here is a small joint table, read straight from a story. Grab one row, divide it by its own total, and you have Y's law for the case where X is fixed.

The cut face inflates in place. The shape is untouched, the height changes, and the area readout snaps to 1.000. So the conditional PDF takes its shape from the joint and its size from the renormaliser, and that is the entire definition. And now the sentence this chapter has been walking toward since the very first roll. This is Chapter 5's shrink the world and re-measure, finally drawn. The world shrank to a single vertical line, and the re-measuring is the rescale you just watched. Conditioning was never a formula. It was always this picture.

05When the shadows are enough

Time to put a real landscape on the screen and turn a knob on it. Take two standard bells from Chapter 10, stand them up in two dimensions, and you get the 2D Gaussian. It comes with a dial called ρ. At ρ = 0 its contours are perfect circles. Turn ρ up and the blob tilts into an ellipse. Now let me be straight with you about that symbol, because I am not going to hand you a token you cannot cash. ρ is a knob, and it tilts the blob. Its real name and its real meaning arrive in Chapter 16, and until then I am not going to pretend you own it.

And I am not going to tell you that tilted means dependent either, because that would make dependence a visual vibe. Let's earn it with the slicer instead. Set ρ = 0.8 and drag the slice. The cut face's peak slides as x moves, which means learning X moved Y's whole picture. That is Chapter 5's definition of dependence, in motion, in your hand. Now set ρ = 0 and drag the slice again. The peak sits dead still. So dependence is not a number yet. It is a thing you just watched move.

Which brings us to the question the whole chapter has been for. When are the two shadows enough? There is an obvious answer available, and I want you to commit to it first. On the screen is the tilted ρ = 0.8 blob. Beside it are its own two marginals, f_X and f_Y, computed live from that exact blob, in front of you, by pouring its columns down. Now multiply the shadows. Build the surface f_X(x)·f_Y(y). What comes back?

Your gut said the same tilted blob, and it said so with total confidence. Obviously the marginals came out of that blob, so the blob must be what is in them. Press the button. A perfectly circular blob appears, the tilt is gone, and you have just watched your own reconstruction lose something real. Two different worlds, identical shadows: this is the two dice worlds again, in a smarter costume. Now turn on the mismatch map, which is just joint − product, and it glows exactly where the product lied. Positive along the tilt. Negative across it. That glow is the dependence that projection destroyed, and it was never in the shadows to begin with.

Now take the ρ dial and kill the mismatch with your own hand. Turn ρ toward 0 and the mismatch drains to black, hitting exactly zero at ρ = 0. That is the one setting where the product IS the joint. So here is the keystone, and the last two minutes were spent earning it: it is a real theorem, not a tautology. X and Y are independent exactly when learning X never changes Y's picture, meaning f(y|x) = f_Y(y) for every value of x. And that is algebraically identical to f(x,y) = f_X(x)·f_Y(y). Both sentences describe the same surface.

Let's turn that crank, because it takes one line and it uses nothing you don't already have. Say every slice has the same shape g(y). Then the surface is f(x,y) = f_X(x)·g(y), because a fixed shape scaled by a number that depends only on x is exactly what that sentence draws. Now integrate over x and let the marginal machine run: f_Y(y) = g(y)·∫f_X(x)dx = g(y). A marginal integrated over its own variable is 1, so that integral simply vanishes and the common slice shape must be the marginal. We did not assume that anywhere. It was forced on us. Multiply back up by f_X(x) and you have f(x,y) = f_X(x)f_Y(y).

Two things fall out of that for free. First, the equation is symmetric in x and y, and a product does not care which letter you solve for. So if X tells you nothing about Y, then Y tells you nothing about X, and independence is mutual with no second argument required. Second, it is a claim about every pair of values at once, not about any single cell. Checking one cell, seeing a product, and declaring independence is the classic way to be wrong. Put it on the dice grids. In World A the cell at (3,4) holds 1/36, and its row total 1/6 times its column total 1/6 is 1/36 as well, so that cell factors, and so does every other cell. In World B that same cell holds 1/6, while the same product is still 1/36, six times smaller. Section one's belief was that the marginals are the story, and to that belief f = f_X·f_Y reads as a triviality: of course the joint is the product of its marginals, where else would they have come from? But this is exactly what factorization is not. You have now watched the product fail on screen. Factorization is the exact condition under which the two flat shadows contain the whole solid, and most joints do not meet it.

One more thing before we leave factorization, because this structure is not a probability trick and I don't want you filing it away in the probability drawer.

Physicists make this exact move, and they have a name for it. When one of them solves the heat equation, the first step is to posit that u(x,t) = X(x)·T(t), and the name of that step is separation of variables. It only works when the solution genuinely has that shape: a fixed profile in x, scaled by a number that depends on t. Put that next to what we just drew: a fixed shape in y, scaled by a number that depends on x. Those are the same sentence, written by two different fields about two different things. And in both of them it is a bet rather than a trick. The thing either factors or it doesn't. When it doesn't, the product quietly hands you the wrong world.

The mismatch map lit up on its own. But independence is a promise about every cell, and the glow never told you which cell broke it. So test the cells yourself: clear them all, or catch the single one that fails.

06The trap in the shape of the region

Now let me show you the way this goes wrong in practice, because you are about to walk into it and almost everybody does. Throw a dart uniformly at a round dartboard of radius one. The density is f(x,y) = 1/π on the disc, which is about 0.32 of mass per unit area at every single point of it, so the density is a constant. It is the most symmetric object we have drawn all chapter. A constant obviously factors, since you can write it as (a constant) × (1), so X and Y are independent. That reasoning is flatly wrong, and the test that convicts it is the one you built ten minutes ago.

Run the slicer. At x = 0 the slice is a wide flat plateau, spanning y from −1 to 1. At x = 0.99 the slice is a narrow sliver pinned near zero, spanning about −0.14 to 0.14. That half-width is the circle doing the work: the square root of 1 − 0.99² is 0.141. The shape changed. So by the keystone they are dependent, and you can feel exactly why with no algebra at all. Learning X = 0.99 tells you a great deal about where Y can possibly be. Now let the algebra confess what it was hiding. What you actually wrote down was f(x,y) = (1/π)·1[x²+y²≤1], and that indicator is part of the density. It cannot be split into a function of x times a function of y.

The region where the mass is allowed to live has a name, and it is the support. Here is the rule to carry out. Factorization has to hold at every point of the plane, indicator and all. The range of y is allowed to depend on x, and that range is part of the density. So if the support is not a rectangle, the pair is not independent, no matter how clean the formula looks. The support was doing the coupling the whole time. It was invisible only because nobody had said the word.

07Independence lives in the coordinates

Keep the dart exactly where it is. We are going to describe the same dart a second way, and something strange is going to happen. But first I owe you a rung, and without that rung the ending is a rabbit. Chapter 13 taught you that changing coordinates makes a density pick up a Jacobian, and in one dimension that factor was |dx/dy|, a length ratio. In two dimensions the conservation is the same and one thing honestly changes: the factor becomes an area ratio. A gut error sits right underneath, and it is Chapter 13's original sin in a new costume. It says that changing coordinates just relabels the same points, so the density should carry over unchanged. Let's just draw the grid and look.

Overlay a polar grid on the plane, and the cells out near r = 3 are visibly fatter than the cells at r = 0.3. Put a number on it: at ten times the radius, a tile has ten times the area. A tile spans dr in the radial direction and an arc of length r·dθ across, so its area is r·dr·dθ, not dr·dθ, and dragging a tile outward makes its area climb linearly in r. Now finish it with the conservation argument you already believe. The mass in a tile cannot change when you rename its corners, because renaming touched no outcomes. So if the tile's area changed, the mass per unit area must change to compensate. That is Chapter 13's rescale, verbatim, one dimension up, and the exchange rate is the factor r.

Now back to the dart, and this time commit to an answer first. You have watched the disc's slices change shape, so (X,Y) are dependent, and you felt it. Same dart, same disc. But now describe that dart by how far out and which way, as R and Θ. Does knowing the angle tell you anything at all about the radius?

Staring at a symmetric disc, your gut says no, and this time your gut is right. Here is the arithmetic, in one line, using the rung we just built. f(r,θ) = (1/π)·r = (2r) · (1/(2π)). That is a function of r, times a function of θ. It factors. Check it at a point: at r = 0.5 the left side is 0.5/π ≈ 0.159, and 2 × 0.5 times 1/(2π) is 0.159 too. And the r that made it factor is not a fudge someone slipped in. It is the Jacobian, the area ratio you just watched with your own eyes. So the same dart is tangled in (X,Y) and free in (R,Θ).

That should feel like a cheat, and I want to say plainly why it isn't. Chapter 5 hammered that independence is a physical claim, not a default, and the dartboard slicer had you feel this exact dependence in your hands. So how can it depend on the coordinate system? The resolution is that "the two things" changed: X and Y are not R and Θ. Independence was never a property of the dart at all. It is a property of the pair of questions you decided to ask about the dart. The tangle in (X,Y) was never in the dart. It was in the square grid we insisted on measuring the dart with.

Let's finish with a receipt instead of a formula. One array of darts, sampled uniformly on the disc once, and that array is the entire random world of this section. Every number from here on is the same darts wearing different labels, and we never call the generator again.

So the margins are strictly less than the joint. That leaves an honest question this chapter never closed: what is exactly enough? Keep one shadow, add the conditional, and watch the whole pair come back with nothing lost.

Histogram the darts in (x,y), build the two marginals from that data, multiply them, and subtract: the mismatch map glows. Now relabel the same darts as (r,θ), do the identical thing, and the mismatch comes out as flat noise around zero. Nothing was re-randomised between those two panels. The rule did all the work, which is exactly the claim, checked by you, on data.

So what you carry out of here is not a set of formulas for joint distributions. It is a pile of mass and two moves. Project the pile and you get the marginal, which is the law of total probability wearing an integral. Slice the pile and rescale the cut face to area one and you get the conditional, which is Chapter 5 drawn at last. You can now look at any joint heatmap and read dependence off it by eye: drag a slice, and ask whether the shape changes. You can re-derive f(y|x) instead of recalling it. And you can say exactly why conditioning on a probability-zero point is legal, because you watched the dx cancel.

But one thing sits above all of it: the marginals are strictly less than the joint. That is not a slogan you have to take on trust, because the mismatch map is a picture you have watched glow. Here is where you are, and here is where that fact is about to get paid.

It pays off twice, almost immediately. Chapter 15 is going to tell you that E[X+Y] = E[X] + E[Y] even when X and Y are wildly dependent, which sounds like magic. It isn't, and you can already see why. The sum only ever touches the marginals. It never touches the dependence structure at all. Then Chapter 16 will warn you that zero correlation does not imply independence, and you are already halfway there. Covariance is going to be one number summarising a whole mismatch map, and one number can be zero while the map is not. Next, though, we do something simpler and just as sharp. We take a distribution, joint or not, and squeeze it down to two honest numbers: its center and its spread.