08Multivariable Calculus & Optimization

Chapter 7 closed on a shape. A positive-definite matrix pushes every direction of a circle outward, and the surface behind such a matrix curves upward everywhere, like a bowl. That told us a bottom exists. It never told us which way to walk to reach it, and reaching it is the entire job of this chapter. Fitting a model and choosing a portfolio turn out to be the same physical act. You walk a landscape of many variables toward its lowest point. The whole of it hangs on one arrow. The gradient is a vector of ordinary slopes, and it always points the steepest way uphill, so you reach the bottom by moving against it until the arrow shrinks to nothing. A constraint changes almost nothing about that story. It just says stop the moment your arrow lines up with the constraint's arrow. By the end you will be able to take an optimisation you have never seen, name its surface, find where its gradient dies, check whether that flat spot is really the bottom, and say what a budget line does to the answer.

This chapter sits at a junction. Almost everything in Parts 2 through 5 is an optimisation in disguise, wearing a different name each time. Click a downstream node in the panel below and watch the road run back to this page.

Four of those roads are worth naming now, because they are the reason this chapter earns its length. OLS regression in Chapter 20 is what you get when you set the gradient of the squared error to zero. Gradient boosting in Chapter 23 is descent performed in the space of functions rather than numbers. Markowitz in Chapter 30 is a constrained quadratic, solved with the Lagrange multiplier we build here. And the double integral at the end becomes expectation itself in Chapter 11.

01The landscape you are standing on

For five chapters a function has meant one thing to us. You feed in a number, you get a number back, and the picture is a curve drawn over a line of inputs. That picture has to break now, because a portfolio has a weight for every asset and a model has a parameter for every feature. One input is not enough.

So add a second input. The inputs no longer sit on a line. They sit on a flat plane, one axis for each input, and the output becomes a height above every point of that plane. The curve has become a surface. That is the whole change, and it is one extra direction rather than a new kind of object.

You already own the reading device for this, and it has been in your pocket for years. A weather map paints temperature over a country, where the flat map is the input and the colour is the output. A hiker's map does the same job with altitude, and it uses a trick worth stealing right now. It draws contour lines, each one joining every place at the same height.

Tilt the hill in that panel and watch the rings on the flat map beneath it move in step. The two pictures are the same function, said twice. Here is the one confusion worth killing before it starts. The flat map is not the surface. The flat map is the input plane, every point of it is a pair of numbers you are allowed to feed in, and the height is what comes back out.

Let's pin a specific hill down so every number in this chapter is checkable. Take x as hundreds of metres east of a trailhead and y as hundreds of metres north, with the height in metres given by

f(x, y) = 30 + 3x + 4y − x² − y² + xy

At the trailhead itself both inputs are zero, so the height is 30 metres. Walk east and the 3x term lifts you. Walk north and the 4y term lifts you faster. The two squared terms bend the whole thing over into a dome, and the xy term is the one that makes east and north interact. Keep an eye on that last term, because it is the source of the only genuine subtlety in the first half of this chapter.

Now drag the walking line across the contour map in that panel and read the height profile that draws itself beside it. Where the rings are far apart the profile is nearly flat, and where they crowd together the profile climbs hard. That is not a coincidence and it is worth saying out loud. The rings are drawn at equal height steps, so squeezing the same climb into less ground is what steepness means.

Which means the map was already encoding slope before we did any calculus at all. A cartographer with no derivatives can look at a contour map and tell you where the hard climbing is. Our job in the next three sections is to turn that visual fact into a number, and then into an arrow.

02One axis at a time

Stand at the trailhead and ask the simplest possible slope question. Face east, ignore north entirely, and ask how steeply the ground rises under your feet. You have not left Chapter 3 to ask that. You have restricted yourself to a single line of inputs, so there is only one variable left moving, and the slope of a one-variable function is a thing you already know how to compute.

Watch what the panel does with that. It cuts the surface with a vertical plane running east to west, and where the plane meets the surface you get a curve. An ordinary curve, the kind Chapter 3 spent its whole length on. Slide the plane north and south and the curve changes shape, because you are slicing a different part of the hill. But at any fixed position it is just a curve, and its slope at your feet is just a derivative.

That number has a name and a slightly theatrical symbol. It is the partial derivative of f with respect to x, written ∂f/∂x, and the curly ∂ means nothing more than "there were other variables in the room, and I held them still." Flip the panel's toggle to slice the other way and you get ∂f/∂y, the slope facing north. Same operation, different plane.

Now for the step that people quietly refuse to believe. To compute ∂f/∂x you treat y as a constant, exactly as you would treat the 3 in 3x. It feels arbitrary. Who gave us permission to freeze a variable that is obviously still there?

The panel settles it by doing the experiment rather than arguing. Set y to the literal number 2 and substitute it everywhere in our hill. The 4y becomes 8, the −y² becomes −4, and the xy becomes 2x. Collect the terms and the whole surface has collapsed into

f(x, 2) = 34 + 5x − x²

There is nothing multivariable left. It is a one-variable polynomial, and Chapter 3's power rule differentiates it in one line to give 5 − 2x. Now do it the other way, keeping y as a symbol and treating it as a constant. The 3x gives 3, the 4y and −y² are constants so they give 0, the −x² gives −2x, and the xy gives y. So ∂f/∂x = 3 − 2x + y. Set y = 2 in that and you get 5 − 2x.

The same answer, by two roads. Freezing y was never a special rule you had to be granted. It is what the arithmetic already does when you walk along a line where y genuinely does not change. The other partial comes out the same way, and for our hill the pair is

∂f/∂x = 3 − 2x + y  ·  ∂f/∂y = 4 − 2y + x

At the trailhead, where x and y are both zero, those read 3 and 4. So the ground rises three metres per hundred metres east, and four metres per hundred metres north. Two honest numbers, both measured with tools you already had.

One notation note so nothing surprises you later. Because ∂f/∂x is itself a function of both variables, you can differentiate it again, and differentiating it with respect to y gives ∂²f/∂x∂y. For our hill that is 1, the coefficient on the xy term. It measures how much the east slope changes as you move north, and it is about to matter.

03★ Two slopes, added

Nobody walks due east. A real step goes diagonally, some east and some north at the same time, so we need to know what a diagonal step costs in height. The obvious guess is that you pay for each direction separately and add the bills:

Δheight ≈ (east slope × east step) + (north slope × north step)

Your gut should fight that, and the objection is a good one. The two directions are not independent on this hill. We put an xy term in on purpose, which means the north slope 4 − 2y + x genuinely depends on where you are east-west. Step east first and the northward climb you were quoted is no longer the climb you get. So why is adding two separately-measured effects allowed at all?

Commit to a guess before you hit reveal, then let the panel run the numbers. Take a step of 0.1 east and 0.1 north from the trailhead. The prediction is 3(0.1) + 4(0.1) = 0.70 metres. The true height change, computed straight from the formula, is 0.69. So the recipe is wrong by 0.01, and the objection was real.

Now halve the step and watch the miss, because this is where the argument is won. At 0.05 each way the prediction is 0.35 and the truth is 0.3475. The miss went from 0.01 to 0.0025. You halved the step and the error fell to a quarter. That is the fingerprint of a term proportional to h², and Chapter 4 taught us exactly what to do with those.

Here is where the missing 0.01 comes from, named precisely. The north slope at the trailhead is 4. After stepping 0.1 east, the north slope has become 4 − 0 + 0.1 = 4.1, because ∂²f/∂x∂y = 1. So the slope you were quoted drifted during the walk, by 1 × 0.1, and over a northward step of 0.1 that drift costs 1 × 0.1 × 0.1 = 0.01. The interaction is real, and it is exactly one h² in size.

So the addition is not an assumption and it is not a lie. It is a local linearization, correct to first order, with an honest second-order remainder that dies faster than the step itself. Chapter 4 already licensed this move for one variable. All we did is apply it once per axis.

Worth stating plainly so nobody over-claims it. This approximation is only good for small steps. Take a step of size 1 across this hill and the h² term is the same size as the linear one, and the prediction is worthless. Everything in the rest of this chapter takes small steps, and where an algorithm takes big ones it breaks in exactly this way.

Now look at the shape of what we wrote, because it is begging to be tidied. The expression f_x·Δx + f_y·Δy is a sum of products, taken slot by slot, between two pairs of numbers. Chapter 6 gave that operation a name and a symbol. It is the dot product of (f_x, f_y) with (Δx, Δy).

Drag the step arrow in that panel and watch the two contribution bars respond, then watch the dot-product readout agree with their sum. Nothing new is being claimed. We are writing the same arithmetic in fewer symbols. Give the first of those two vectors a name and the tidying is done:

∇f = (∂f/∂x, ∂f/∂y) — the gradient, the partials packed into one vector

At the trailhead our gradient is (3, 4). The whole linearization compresses to one line:

Δheight ≈ ∇f · step

Let me be honest about what has been earned so far. Bundling numbers into a vector and drawing an upside-down triangle on it is not yet an insight. It is bookkeeping. The steepest-direction question is where the bookkeeping pays, and it pays in something you can see.

04★★ The steepest way up

Here is the question the whole chapter turns on. You are standing at the trailhead and you may take one step of fixed length in any direction you like. Which direction gains you the most height?

Set it up with what you already hold. Write the direction as a unit vector û, meaning an arrow of length exactly one that only carries a heading. Then the height gained is ∇f · û by the line we just wrote. And Chapter 6 gave us a second way to read every dot product, in terms of the angle between the two arrows:

∇f · û = |∇f| |û| cos θ = |∇f| cos θ

Read the right-hand side slowly, because the answer is sitting in plain sight. The length |∇f| is fixed by where you are standing, and you cannot change it by choosing a direction. The length |û| is 1 by construction. So cos θ is the only thing in that expression you control.

Predict before you touch the dial. Of every possible heading, which one makes |∇f| cos θ as large as it can be? Commit to an answer, then spin it.

Cosine is largest when the angle is zero. There is no cleverness in that, it is just what the cosine curve looks like. And θ = 0 means your step points in exactly the same direction as ∇f. So the steepest uphill direction is straight along the gradient, and the climb rate you get there is cos 0 = 1 times |∇f|, which is |∇f| itself.

That is the keystone of this chapter, and notice that it was never a rule to memorise. It is one substitution and one glance at a cosine.

∇f points the steepest way uphill, and |∇f| is that steepest slope.

Check it on our hill. The numbers here are unusually kind. At the trailhead ∇f = (3, 4), so |∇f| = √(9 + 16) = 5. The unit vector along it is (0.6, 0.8). Feed that in and the dot product is 3(0.6) + 4(0.8) = 1.8 + 3.2 = 5, matching the length exactly.

Now spin the dial elsewhere and watch the readout fall off. Point due east and you get (3,4)·(1,0) = 3, which is the east partial, as it must be. Point at (−0.8, 0.6), which is square across the gradient, and you get −2.4 + 2.4 = 0, so that heading gains no height at all. Point at (−0.6, −0.8) and you get −5, the fastest possible way down.

That last reading is the one this chapter runs on. The steepest descent direction is −∇f, for exactly the same reason and with no extra work. Cosine is smallest at 180°.

The general name for ∇f · û is the directional derivative, the rate of change along a chosen heading. The two partials are simply the special cases where û is due east or due north. So the partials were never the fundamental objects. They were two samples of a whole circle of slopes, and the gradient is the compact object that generates all of them.

05Where the arrow dies

Now read the keystone backwards, and an optimisation condition falls straight out of it. Suppose you are standing somewhere and ∇f is not the zero vector. Then it names a direction, and stepping that way gains height, so you cannot possibly be at the top. The only place with no direction left to climb is the place where the arrow has no length at all.

Drag the marker across the contour map in that panel and watch the arrow shrink as you climb. At the trailhead it reads |∇f| = 5.00. At (2, 2) the gradient is (1, 2) and the length has fallen to 2.24. At (3, 3) it is (0, 1) with length 1.00. Keep going and at (3.33, 3.67) the arrow vanishes and the height reads 42.33 metres. That is the summit.

Solve for it directly and it agrees. Set both partials to zero, so 3 − 2x + y = 0 and 4 − 2y + x = 0, and the pair gives x = 10/3 and y = 11/3. A point where the gradient is the zero vector is called a stationary point, and the condition is written the way you will see it everywhere from here on:

∇f = 0 — the optimum condition

Two traps worth naming while we are here. First, ∇f = 0 does not mean f = 0. Our summit has a gradient of zero and a height of 42.33 metres, and those are different quantities entirely. Second, ∇f = 0 is a vector equation, so it is really n separate equations, one per input. For a model with a million parameters that is a million equations at once.

And now the exception, because ∇f = 0 is a candidate rather than a verdict. Look at the surface g(x, y) = x² − y² in that panel. Its gradient is (2x, −2y), which is dead zero at the origin, so the origin passes the test. Walk east from there and the ground rises like a parabola. Walk north and it falls like an upside-down one.

That is a saddle point, and the everyday version of it is a mountain pass. From the pass you go up toward either peak and down toward either valley, and you are at the bottom of one journey and the top of another at the same time. In one variable this barely happens. In many variables it is the normal state of affairs, and in a large neural network the overwhelming majority of stationary points are saddles rather than minima.

So the gradient alone cannot finish the job. It tells you where the ground is flat. It cannot tell you which way the ground is curving, and curvature is a second derivative.

Chapter 4 already handled this in one variable. If f′(a) = 0 and f″(a) > 0 the curve is cup-shaped and you are at a minimum, and if f″(a) < 0 it is a cap and you are at a maximum. We need the same test with more directions available, so collect every second partial into a square grid:

H = [ f_xx   f_xy ;   f_yx   f_yy ] — the Hessian, the matrix of second partials

This is where the definition becomes a picture, and it is Chapter 7 in a new costume. A minimum needs the surface to curve upward in every direction, not just along the two axes. Chapter 7 told us exactly which matrices push every direction outward. Those are the positive-definite ones, the ones whose eigenvalues are all positive.

all eigenvalues of H > 0 ⟹ bowl, a minimum  ·  all < 0 ⟹ dome, a maximum  ·  mixed signs ⟹ a saddle

Step the three cards in that panel and check each one against the surface it draws. The mixed case is the most instructive, so take f = 2xy, whose Hessian is [0  2 ;  2  0] with eigenvalues +2 and −2 along (1,1) and (1,−1). Walk the (1,1) diagonal and the height goes as +2t², climbing. Walk the (1,−1) diagonal and it goes as −2t², falling. The eigenvectors told you where to look and the signs told you what you would find.

Our own hill closes the loop nicely. Its Hessian is [−2  1 ;  1  −2], with eigenvalues −1 and −3 along (1,1) and (1,−1). Both negative, so the summit really is a maximum, which the walking confirmed. And those same eigen-directions are the long and short axes of every contour ellipse on the map, which is why the rings came out tilted.

One more property earns its own beat, because it explains a habit that runs through the rest of this course. A surface that curves upward everywhere, at every point rather than just at one, is called convex. Chapter 4 defined that for a curve and the meaning is unchanged. It is a bowl.

Drop marbles from random starting points in that panel and compare the two landscapes. On the convex bowl every marble arrives at the same place, because there is only one flat spot for it to find. On the bumpy surface beside it the marbles scatter into different dents, and each one stops somewhere that is lowest only in its own neighbourhood.

That is the gift, and it is worth stating as a sentence you can carry:

On a convex surface, a local minimum is the global minimum.

Which is why quants engineer their objectives to be convex whenever they possibly can. Mean-variance portfolio optimisation is convex, and so is ridge regression, and both were designed that way rather than discovered that way. When the surface is convex, finding one stationary point ends the search. When it is not, you have found a local answer and you should say so out loud.

06Walking down the hill

We have a condition for the answer and a test for whether it is the right kind. What we do not have is a way to actually solve ∇f = 0 when the problem is real.

On our hill it was two linear equations and a minute of algebra. Now count what a neural network asks for. Millions of parameters, so millions of simultaneous equations, all of them nonlinear because of the activation functions stacked between the layers. There is no closed form. Nobody is solving that system, ever.

So stop trying to solve it and use what the keystone already gives you for free. You may not know where the bottom is, but at any point you are standing on, the gradient tells you which way is down. Take a small step that way. Then look again and repeat.

w ← w − η ∇f(w) — gradient descent, with η the learning rate

The minus sign is the keystone doing its work, since −∇f is the steepest way down. The η is yours to choose. It says how far to trust a slope measured at a single point before you go and measure again.

this loop actually runs ↓
w = [2.0, 2.0]
eta = 0.10
for i in range(8):
  L = w[0]**2 + 4*w[1]**2
  print(i, w[0], w[1], L)
  g = [2*w[0], 8*w[1]]  # ∇L
  w[0] -= eta*g[0]
  w[1] -= eta*g[1]

That panel is a genuine CodeRun. The loop you can read is the loop that executes, and the trace printed underneath is its real output rather than a drawing of one. It minimises L(w₁, w₂) = w₁² + 4w₂², an elongated bowl, starting from (2, 2) where the loss is 20.

Run it at η = 0.1 and follow the numbers. The gradient at the start is (4, 16), so the first step lands on (1.6, 0.4) and the loss prints 3.2. The next two iterations print 1.664 and 1.0496. The loop never solved anything. It just kept asking which way was down and going there.

Now push the learning-rate slider up and break it deliberately, because the failure teaches more than the success. At η = 0.3 the w₂ coordinate overshoots the valley floor and lands further out on the far wall, so the next step overshoots harder. The printed loss grows without bound. Too small a rate fails the other way, quietly. The loop converges, but it crawls, and you pay for every wasted iteration in compute.

A lock worth carrying, because this pattern shows up far outside finance. A ball rolling in a bowl with friction is doing the same thing physically, since gravity supplies −∇f and the ball has no idea where the bottom is either. A PID controller in a robot is the same shape again, measuring an error and feeding a correction proportional to it. Measure the local error, move against it, repeat.

One thing is still missing before this is usable on a real model. Our loss was written directly in terms of the two weights, so the gradient was easy to write down. A real model is built in layers, where the loss depends on the output, which depends on the last layer, which depends on the one before it. How does a slope travel back through a stack like that?

Step the derivation in that panel and notice that no new machinery arrives. Suppose you walk a path across the hill, so your position (x(t), y(t)) depends on time. Take the linearization we already proved, Δf ≈ ∇f·Δposition, divide both sides by Δt, and let Δt shrink. What comes out is the multivariable chain rule:

df/dt = ∇f · velocity = f_x·(dx/dt) + f_y·(dy/dt)

Read that in words and it is completely reasonable. The rate the height changes is how fast you are moving east times the east slope, plus how fast you are moving north times the north slope. Every route into f contributes, and the contributions add. That addition is the single thing beginners drop, and it is the whole difference from the one-variable chain rule.

The second beat of that panel checks it rather than asserting it. Walk the curve x = t, y = t², and look at time t = 1, which puts you at the point (1, 1). There the gradient is (2, 3) and the velocity is (1, 2), so the chain rule says 2(1) + 3(2) = 8. Now take the other road. Substitute the path into f first, which gives 30 + 3t + 3t² + t³ − t⁴, differentiate that with Chapter 3's power rule to get 3 + 6t + 3t² − 4t³, and evaluate at t = 1. That reads 3 + 6 + 3 − 4 = 8. Identical.

Now flip the panel's last beat and run the same rule backwards through two layers. A loss depends on a prediction. The prediction depends on a weight. So the loss's sensitivity to that weight is the product of the local slopes along the chain. When a weight feeds several places, you sum over the paths, exactly as the two terms above are summed. Sweep that computation from the loss end back toward the inputs and you have backpropagation.

Which is worth saying flatly, because the word carries far more mystique than the mechanism deserves. Backprop is the chain rule, run backwards, with the shared work cached instead of recomputed. That is the entire idea. Chapter 23 will use the same move to explain gradient boosting, where each new tree is fitted to the residual because the residual is the negative gradient of squared error.

07★ When a fence is in the way

Everything so far assumed you may step wherever the arrow points. Real problems almost never allow that. Portfolio weights must sum to one because you invest all your capital and no more, a risk budget caps your exposure, and a position limit is a hard wall. The optimum you actually want is the best point on the allowed set, which is usually nowhere near the free one.

Before we can handle a fence, we need one geometric fact about the gradient, and it is one line of work. Take a step along a contour line. By the definition of a contour, the height does not change, so the directional derivative in that direction is zero.

∇f · (tangent to the contour) = 0 ⟹ the gradient is perpendicular to the contour

Turn on the arrow field in that panel and look at how the arrows meet the rings. Every one crosses at a right angle. Everywhere on the map, no exceptions. You have seen this in the physical world without naming it. Rain landing on a hillside runs straight downhill, and its track crosses each contour line square on, because water takes the steepest way down.

Check the arithmetic at the trailhead once so it is not just a picture. The gradient is (3, 4), and the contour through that point runs along (−0.8, 0.6). Their dot product is −2.4 + 2.4 = 0. Perpendicular, exactly.

This rung is the one most treatments skip, and skipping it is precisely why the next result feels like a magic trick when you meet it elsewhere. We are not going to skip it.

Now the constrained problem, with numbers small enough to check by hand. You have 10 units of capital to split between two desks, so x + y = 10. Their combined output is f = xy, because each desk amplifies the other. The contours of xy are hyperbolas, one for each output level, and the constraint is a straight line across them.

Drag the marker along that line and watch the output readout. As long as you are crossing contours, the output is still changing, so one of the two directions along the line makes it better and you are not done. Keep sliding. The moment you stop improving is the moment the line stops crossing contours and merely grazes one.

That word is the whole method. At the constrained optimum, the constraint is tangent to a contour of the objective. And two curves that are tangent at a point share the same perpendicular direction there. We just proved that each curve's perpendicular is its own gradient. So at the optimum the two gradients must lie along the same line:

∇f = λ ∇g — the Lagrange condition, with λ the multiplier

The λ is there for a plain reason rather than a mysterious one. Two gradients point the same way. They have no reason at all to be the same length, so a scaling number is needed to turn a parallelism into an equation.

Solve our case and see how little work it is. Here ∇f = (y, x) and ∇g = (1, 1), so the condition gives y = λ and x = λ, which forces x = y. Feed that into the constraint and you get x = y = 5, with output 25 and λ = 5. The tangent hyperbola is xy = 25, touching the budget line at exactly one point.

And λ is not scaffolding to be thrown away after solving. It has a meaning, and in finance it is the most useful number in the whole calculation. Drag the budget in that panel and watch both the answer and λ move together.

Loosen the budget from 10 to 10.1. The new optimum is (5.05, 5.05) and the best output rises from 25 to 25.5025. So one tenth of extra capital bought you 0.5025 of extra output, against a prediction of λ × 0.1 = 0.5. The 0.0025 mismatch is second order in the loosening, which is the same h² we met in the missing-0.01 audit.

So λ is a price. It is how much the best achievable value improves per unit of constraint relaxed, which is why it is called the shadow price of the constraint. A risk manager reads it as the value of one more unit of risk budget. A desk reads it as what an extra unit of capital is worth before anyone quotes a rate.

One last case and then we are done with constraints. Most real limits are inequalities rather than equalities, since a position limit says at most this much, not exactly this much. The single question is when such a constraint counts.

Picture a wall you might or might not be leaning on. Toggle the two cards in that panel and both cases are visible at once. Minimise (x−3)² + (y−1)², whose free minimum sits at (3, 1). Under the limit x + y ≤ 6 that free minimum is already legal, so the wall does nothing and you may delete it from the problem. Its multiplier is 0.

Now tighten the limit to x + y ≤ 2. The free minimum is outside the allowed region, so you are pressed flat against the wall and the best legal point is (2, 0). There the constraint is tight, it behaves exactly like the equality we already solved, and its multiplier is nonzero. Loosen it to 2.1 and the minimum falls from 2 to 1.805, which is the same shadow-price reading as before.

That two-case split is the whole content of the KKT conditions, named for Karush, Kuhn and Tucker. The bookkeeping line that encodes it is complementary slackness, and it says only this: either the multiplier is zero, or the constraint is tight. Never both nonzero at once. We meet the full machinery in Chapter 30, where a long-only portfolio has one of these per asset.

08The accumulation twin

The derivative side is finished. One loose end left. Chapter 3 taught us that the derivative has a twin: the integral accumulates area under a curve rather than measuring slope. Two inputs must have a twin too.

And it does. Better still, we already own the move that builds it. Under a curve you accumulate area. Under a surface you accumulate volume, the amount of space between the flat input plane and the sheet floating above it.

Here is the machine, and it is partial differentiation run backwards. Freeze x at some value. What sits above that frozen line is an ordinary one-variable curve, exactly as it was when we sliced for a partial derivative, and Chapter 3 already tells you the area under it. That area is one thin slice of the volume. Now sweep the frozen line across and add up every slice.

Sweep the slice in that panel and watch the running volume fill in beneath it. Take f(x, y) = x + y over the unit square, meaning both inputs run from 0 to 1. Freeze x and integrate over y, giving a slice area of x + ½. The readout runs from 0.5 at the left edge to 1.5 at the right.

Then integrate those slice areas across x, which gives ½ + ½ = 1. The volume under that sheet is exactly one cubic unit. On paper, the two integral signs are just the two passes:

∬ f dA = ∫ ( ∫ f dy ) dx — the double integral, computed by iteration

Flip the panel's order switch and slice the other way, freezing y first. Every slice is a different shape and the running total climbs differently, but the final volume is 1 again. Which makes sense, since the volume under a fixed sheet cannot depend on which way you chose to chop it.

An honest flag on where this leads, and on what it is not yet. In Chapter 11 the surface becomes a joint probability density, and the volume over a region becomes the probability of landing in that region. This exact machine is how E[X] gets defined for continuous variables. That sentence needs probability, which we have not built. So here it is pure geometry, and the geometry is already complete.

So here is the chapter as one image you can carry out the door. A function of many inputs is a landscape. Its partial derivatives are ordinary slopes taken one axis at a time, and packing them into the gradient turns the local height change into a single dot product. That dot product forces the keystone, since |∇f|cosθ is largest when you walk along the arrow. Read it forward and you get descent. Read it as an equation and you get ∇f = 0. Read it sideways and you get perpendicularity to the contours, which is Lagrange. The Hessian's eigenvalue signs tell you whether the flat spot you found is a bowl, and if the surface is convex then that one flat spot is the whole answer.

You now hold a template that fits an optimisation you have never seen. Name the surface, find where the gradient dies or walk downhill until it does, check the curvature so the flat spot is really the bottom, and attach a multiplier for every budget that binds. Chapter 20 runs it on squared error and out come the normal equations. Chapter 23 runs it on residuals and out comes boosting. Chapter 30 runs it on portfolio variance with weights summing to one and out comes the efficient frontier.

Part 0 ends here. We have spent eight chapters building the algebra, the series, the calculus and the linear algebra that the field quietly assumes, and the toolbox is now genuinely complete. Chapter 9 points all of it at the target it was built for. We start putting numbers on uncertain events, from three axioms up, and the first thing probability will ask of you is a function defined on sets.