◈ behind the scenes · a fun ML prediction research project

How I built it — and every way it went wrong first

On 19 July 2026, Argentina and Spain walk out at MetLife Stadium for the World Cup final. My model gives Spain about a 54-in-100 chance of lifting the trophy, Argentina about 46. This page is the workshop behind that number: ten acts, four genuine wrong turns, seven expert agents built to mark my homework, one dead model exposed by Malta, and a betting market I learned to stop throwing away. Every number here traces to a real dataset, a real bug, or a real test — nothing is invented, including the mistakes.

00The warm-up: three tools, no magic

Start with the only three ideas this whole project runs on. No machine learning yet. No football knowledge required. If these three click, nothing later can lose you.

Tool one: a single match is nearly a coin-flip. The better team loses football matches all the time — one deflection, one offside toe, one keeper's afternoon. So an honest forecaster never says "Spain will win." The honest output is a probability — and here is what that number really is. Picture a parallel-world machine simulation: the computer plays the same match ten thousand times, and I simply count who wins how many — "Spain 54%" just means Spain won 5,400 of 10,000 simulated finals. That counting picture is the single most important idea on either of these pages. Hold onto it.

Tool two: goals are counts of rare events. A team takes a dozen-ish shots; each shot scores with some small chance; the goals that result are a small count built from many small-chance tries. There is a hundred-year-old formula purpose-built for exactly that shape — the Poisson — and it needs just one number: the team's scoring rate, written λ (lambda). Give the formula λ and it tells you how often that team scores 0, 1, 2, 3 goals. Give it both teams' λs and it prices every possible scoreline of the final — and from the scorelines, the win, the draw, the loss.

Tool three: training is just "least surprised." When I say a model is trained, I mean something plain: the fit chooses the λs that make the matches that actually happened look least surprising. If Spain keeps winning 2–0 and 3–0, a low Spain λ makes those results look like miracles — so the fitting process raises it. That's all training is. No magic, no black box: a knob turned until reality stops looking surprising.

Armed with exactly these three tools, I built my first machine. It looked great. It was lying to me twice.

aThe first build: it looked fine

First decision, before any code: what do I even claim? A confident "I called it" was never on the table — fork one, taken in the first minute. The claim would be a calibrated probability, audited hard enough to deserve printing. That choice quietly dictated everything that follows, because a probability can be tested — and testing is where all my trouble, and all my credibility, came from.

Second decision: data. I went keyless — sources that need no login, no API key, no paywall. Keyless does not mean fake. ESPN's public scoreboard API returns every real 2026 World Cup match — 102 completed games with real goals, shots, possession, and (I'd only later realise how much this mattered) real betting odds. I checked the paid alternatives honestly: API-Football wants a key, football-data.org locks its statistics behind a paywall, Sportmonks' free tier covers Danish and Scottish leagues — fine leagues, wrong continent of relevance. Verified, declined, logged.

Then I built two models side by side. One was the Poisson goals model from the warm-up — two knobs per team, attack and defence. The other was XGBoost, a powerful modern learner that stacks thousands of tiny if-this-then-that rules, with hundreds of effective knobs, fed rolling form, shot conversion, possession differentials. Flexible, fashionable, and on my data, about to embarrass itself.

Here is what happened. Trained on the tournament's first 94 matches, XGBoost called the final Argentina 70%. Then a bug-fix restored eight missing matches — games that had gone to extra time, which my ingestion had silently dropped — and on 102 matches the same model said Spain 63%. Eight added games. The winner flipped. That swing has a name: overfitting. The model was never holding an opinion about strength. It was memorizing the last thing it saw.

The deeper reason is arithmetic, not mystique. A hundred matches is a tiny purse of signal. Spend it across hundreds of knobs and every knob buys mostly noise; spend it on two knobs per team and the only thing those knobs can learn is strength. That's why, at this data size, the humble model with fewer parameters beats the clever one — fork four. The Poisson, for its part, barely blinked when the eight matches arrived: Argentina 37, Draw 25, Spain 37 before, near enough the same after. Stable. Symmetric. Satisfying.

I benched XGBoost and trusted the steady Poisson: 37–25–37. Stable, symmetric, satisfying. Before publishing, I did one paranoid thing — I created seven agents in their respective domains and asked them to mark my homework.

bThe red-team: nobody marks their own homework

It looked fine. That is precisely the moment to build agents whose only job is to break it, because the one reviewer who is structurally incapable of finding your blind spots is you. So I created seven agents, each an expert in its respective domain — one hunting leakage, one judging the football-domain modelling, one auditing the statistics, one reading the data engineering, one testing market realism, one asking what was missing entirely, one guarding teaching honesty — and asked them to mark my homework, with a lead reviewer to synthesize the verdict. None of them wrote the model. None of them owed me anything.

The verdict came back: right direction — rebuild the spine. Five findings carried the force. One: the model was answering the wrong question — a final cannot end in a draw, yet a quarter of its probability sat on "draw" with nowhere to go. Two: the cruellest — the deep history was never actually wired in; the model had trained on 102 matches, roughly seven per finalist, and the beautiful symmetric 37/37 might be silence, not parity. Three: real DraftKings odds sat cached for every match, used for nothing — while a borrowed Opta figure wore the label "the market." Four: no model had ever been scored on matches it hadn't seen. Five: the headline blended the trustworthy Poisson fifty-fifty with the very XGBoost I had just documented as broken — never average in a model you've called unreliable. Fork nine died in that sentence. And ten smaller findings besides.

One piece of good news, worth exactly as much as the bad: the leak I feared most — the final itself contaminating training — was checked and verified not firing. An audit that only ever finds problems is a mood, not a measurement.

The cruellest accusation was the second: your stable 37/37 isn't parity — it's silence. I refused to believe a number that stable could mean nothing. So I tested it.

cThe 37/37 mirage and the Malta test

I started the rebuild — the full history, thousands of matches, the works (Act d has the details). And here is the scare that taught me the most valuable lesson of the whole project: the number barely moved. Still ~37/37. Two teams, a hundred-plus real matches each now, and the model shrugged: dead even. A number that refuses to move when you change its world is not being stable. It is being absent.

So I ran a five-second test that beats a week of mathematics. Don't ask the model about Argentina and Spain — ask it about a mismatch so absurd that any working model must get it emphatically right. I asked it for Spain versus Malta.

It answered: Spain 1.35, Malta 1.30. Read that again. The model expected the world's number-one side to edge Malta by five hundredths of a goal. It thought every team on Earth was average. The 37/37 was never parity between two giants — it was a dead model outputting the global mean for everyone, and looking profound while doing it.

The autopsy took one line of code. Regularization is a leash: it stops a model bolting after every noisy match, and every sensible fit wears one. But pull the leash too tight and the model cannot reach the data at all. My library's default tension — alpha = 1.0, a perfectly reasonable default for other problems — was, at my sample size, strangling every team's strength number — its coefficient — down to nearly zero. Over-regularization: the equal-and-opposite failure to Act a's overfitting. XGBoost memorized because its leash was too loose; the Poisson flatlined because its leash was too tight.

Loosen the dial and watch the model wake up. At alpha = 0.001, Spain–Malta separates to 3.9 – 0.3 — a scoreline a football fan would actually recognise — and the final, asked again, moves to Argentina 34.3, Draw 27.5, Spain 38.1. Alive, discriminating, and leaning Spain — the same direction as the market. The deep history had mattered all along. The bug had been hiding it.

Alive at last — and leaning Spain, like the market. But 0.001 was a guess. What earns a knob its value? The same thing that earns any number: matches the model never saw. First, though, I needed something to be measured against.

dThe rebuild: 8,218 matches and a bug wall

A dead model means the data never got a vote. So: rebuild the electorate. There is a public, keyless dataset — a community-maintained CSV — holding 49,520 real international results from 1872 to this morning. I filtered it to the current era, 2018 through kickoff, and weighted what remained twice over: recency (an exponential half-life of three years, so last summer outvotes 2019 — and the clock is anchored to the final's date, never to "now", so the build reproduces identically forever) and importance (a World Cup knockout counts fully; a friendly counts 0.4 of a vote). What survived: 8,218 weighted matches. Argentina now speaks with 110 real results, Spain with 111 — not seven.

Rebuilds surface bugs, and I kept mine on the wall deliberately, because every one of them produced plausible output while quietly poisoning the numbers. The extra-time bug: ESPN tags matches that finish after 120 minutes with a different status string, so my filter silently dropped eight real knockout games — including two of Argentina's — until a routine data-proof caught the dataset at 94 matches instead of 102. The label bug: a shootout win was being recorded as a draw, corrupting the form features. The dead feature: my "neutral venue" flag was constant in training and hardcoded at prediction — a correction that corrected nothing, silently. The off-by-one: the prediction row's form window stopped one match early, omitting each finalist's semifinal — the single most relevant game either had played.

The meta-lesson outranks any single fix: bugs in data pipelines do not crash — they hand you plausible numbers. Count your rows. Test your flags. Distrust your defaults. And when a paranoid five-minute check feels beneath you, remember it caught what a week of modelling had missed.

The engine was rebuilt and honest. Now — measured against what? It turned out I'd been holding the answer since day one and throwing it away.

eThe yardstick: what the market actually is

Sitting in my own cache, ignored since day one: DraftKings' closing prices for all 102 matches. Betting odds are probabilities wearing a disguise, and learning to undress them honestly is one genuinely transferable skill of this project. Take the final itself. The closing prices were Spain +125, Draw +200, Argentina +255. A $100 bet on Spain at +125 returns $225 — so that price only breaks even if Spain wins about 44.4% of the time. Convert all three prices the same way and add them up: 105.9%. The maths didn't break. The extra 5.9% is the bookmaker's fee, baked directly into the prices — the house edge, hiding in plain sight. De-vigging simply squeezes that fee back out so the three outcomes describe one honest set of probabilities again: Spain 41.9, Draw 31.5, Argentina 26.6. Across the whole tournament the fee averaged 4.5%. The house always gets paid.

One category error to kill while I'm here, because I committed it myself. Early on, I "benchmarked against the market" using a published Opta figure. But Opta is not the market — Opta is a rival model, a fellow forecaster with its own opinions. The market is something rarer: a price, formed by people staking real money, that eats information for a living. Three different objects — a price, a peer model, a rating ladder — and only the price is a benchmark. I keep all three on screen from here on, each in its own labelled lane, never blended.

A yardstick needs a scale, and "how often was it right" is the wrong one for a coin-flip sport. The right scale is log-loss — a fine for misplaced confidence. Every match, the forecaster pays a fine based on the probability it gave the outcome that actually happened: say 70% and be wrong, the fine is heavy; say 55% and be wrong, it barely stings; be certain and wrong, the fine is catastrophic. Over 102 matches, the honest forecaster is the one with the smallest total fine.

The market's average fine over the tournament: 0.794. For scale, guessing ⅓–⅓–⅓ every match costs 1.099. That 0.794 is the line — the number any model claiming skill must approach. And held against it, my then-current numbers showed exactly the tilt the red-team had predicted: overrating Argentina, underrating the draw.

Yardstick in hand. Time for the exam — one the model cannot cheat.

fThe backtest: a nose behind Vegas, inside the noise

The exam has two rules. Rule one: the model may study only what happened before the tournament — it trains on the 8,116 internationals up to 11 June, then must predict all 102 World Cup matches blind. Rule two: the wall is a date, not a flag. Leakage — the classic silent killer of published results — is letting the model sit the exam with the answer key in its pocket; a mutable "completed" flag can leak, a hard timestamp cannot. Nothing on or after kickoff enters any fit, the final itself included. I verified it: zero rows past the wall.

This exam also settles Act c's open debt. That alpha dial I loosened by eye? Now it's tuned by measurement: try each tension, score each on the 102 unseen matches, keep the one with the smallest fine. The data chose 0.0001 — looser still than my guess. A knob earns its value on matches the model never saw, or it hasn't earned anything. One asterisk, worn openly: those same 102 matches both sat the exam and chose the knob — which tints the 0.840 slightly optimistic. The leash is a single monotone dial, so the tint is small; the footnote still belongs on the record.

The verdict board, in full. Random guessing: 1.099. Always predicting the tournament's overall averages — the same win/draw/loss frequencies for every match, blind to who is playing: 1.050. My model: 0.840. The market: 0.794. Two readings, both true. First — the model has real skill: it beats both naive baselines by a distance, on games it never saw, despite training only on pre-tournament data while the market watched every match happen. Second — the gap to the market is +0.046. How sure can a gap measured on only 102 matches be? I re-drew and re-scored those matches a thousand times — a resampling trick called the bootstrap — and it put the gap's 95% confidence interval at [−0.003, +0.094]. That interval straddles zero — though most of its mass sits above it. Read it the honest way round: my model is probably a nose behind the market, by a gap that 102 matches cannot distinguish from zero.

So here is the phrasing law I adopted, said proudly rather than sheepishly: my model does not beat Vegas — nobody's should claim to. It sits just behind, closer than the data can tell apart. (Hit-rate tells the same story with a blunter instrument: 67% for me, 69% for the market. If you ever needed proof that accuracy can't referee a coin-flip sport, two points of hit-rate separating forecasters whose fines differ by noise is it.)

One exam passed. One embarrassment left: my beautiful validated number answered a question nobody asked. A final cannot end in a draw.

gThe right question: who lifts the trophy

Refit through kickoff with the tuned dial, the probabilities read: Argentina 32.1, Draw 29.0, Spain 38.9 — expected goals 1.04 against 1.17. A validated forecast of the ninety minutes. And a forecast with 29% of its worlds ending in a result the World Cup final does not permit. Somebody lifts the trophy in every single tomorrow. My target was wrong, exactly as the red-team's first finding said.

Two refinements fix it. The first is a humility about finals: they are cagey, low-scoring, nobody-wants-the-mistake affairs, so I apply the Dixon–Coles correction — a standard, gentle reshaping that shifts a little probability toward low-scoring outcomes (rho = −0.05, mild). Watch what it does: my draw moves from 29.0 to 30.3 percent. The market's draw price, which I never fit to, sits at 31.5. When a correction you chose for football reasons lands your number beside a price you never touched — unforced agreement — that is validation, not coincidence.

The second refinement resolves the draw worlds. In every tomorrow that ends 90 minutes level, the final continues: thirty minutes of extra time — which I model as the same two teams at one-third rates, the Poisson machinery unchanged — and if still level, a shootout, which I score as a fair coin. Not because shootouts are fair coins, but because every tempting tilt — Argentina's celebrated shootout keeper included — is numerically speculative, and this project does not bake anecdotes into arithmetic. The tilt lives in a footnote, on purpose — and the footnote has numbers: hand Argentina a 55/45 shootout edge and the trophy moves 53.8 → 53.0. Under a point, because only 17% of worlds ever reach the coin. Restraint is part of the method, and here it was also cheap.

Now let the draw mass flow downhill. 30.3% of worlds go to extra time; extra time resolves some and passes the rest to the coin; the coin splits what remains. Summed across all worlds: Spain 53.8%, Argentina 46.2% — and, re-running the entire fit a hundred times on gently reshuffled data (the bootstrap again), a 90% band of roughly 42–66 for Spain. The number fans actually ask for, carrying its own honesty about how much one hundred-ish matches per team can pin down.

One model, one earned number. But one model agreeing with itself proves little. I went looking for strangers.

hConvergence: three strangers agree

The last fork asked: more data? Any new source had to be keyless and genuinely independent — another opinion, not my own data echoed back. The World Football Elo ratings qualify: a rating ladder maintained for decades — after every match the winner takes rating points from the loser, more for an upset — built by people who have never heard of my model. On the eve of the final it ranked Spain #1 at 2232 and Argentina #2 at 2200 — the top two teams on Earth, thirty-two points apart. Feed that gap through Elo's own win-expectancy formula and it says Spain ~54.6, Argentina ~45.4.

Line up the three instruments. My goals model, built from scratch out of public data and audited within an inch of its life: Spain 53.8. The Elo ladder, decades old, method entirely different: ~54.6. The betting market, pricing the ninety minutes with real money: Spain 41.9 against 26.6 on its own win-in-90 scale — the same lean, priced in its own currency. Three strangers, three methods, one direction. Convergence of independent instruments is stronger validation than any single model's confidence — and it is as close to "right" as an honest forecaster gets to feel the night before the world finds out.

One last experiment closed the loop. I tried chaining the three instruments — a weighted blend, every one of 903 weight combinations, two mixing rules, cross-validated so the weights couldn't cheat. The search's verdict: put 100% on the market, every fold, nothing on me. That is not a defeat — it is the cleanest lesson in the project. A blend only beats its parts when the parts make different mistakes; the models are built from the same results the market already digested, so there was no independent error left to harvest. What building them bought instead was trust — you only know the market is worth 100% after you've matched it from scratch — and reach: the market prices ninety minutes, and the extra-time-and-penalties machinery is what turns its price into a trophy number (Spain 58.2 / Argentina 41.8, beside the independent 53.8 / 46.2 — the gap between them is honest information, not embarrassment). And yes — I even staged XGBoost's retrial. Re-fed with 19,617 matches of honest features, the Act-a exhibit came back sane: no flip-flop, a reasonable near-even call on the final. It also finished last (0.881), and the four-way weight search handed it zero. The signal here is smooth — strength plus a little form — and trees buy variance, not insight, where there are no sharp corners to find. Right data made it sane. Right structure is what the Poisson had all along. (One detail for the record: the Elo in that search was not the official ladder from earlier — I built my own, replaying 26 years of matches with the winner taking rating points from the loser, more for upsets and big margins, then a small regression turning each rating gap into win/draw/loss probabilities. It reached the same verdict as the official one: Spain 2115, Argentina 2110 — a four-point whisper.)

Now walk the whole road at once.

iThe map of every fork

Strip away the drama and this is what modelling actually is: a sequence of forks, each with a road taken, a road refused, and a reason you should be able to say in one line. Here is the whole journey as one map — every green branch taken, every red stub pruned, and the receipt for each. If you take one artifact away from these two pages, take this: not the answer, but the shape of how an answer gets earned.

And the honest closing panel, because a map that hides its edges is a brochure. This model sees nations, not line-ups: it cannot know who is rested, who is carrying a knock, who starts on the bench — projected elevens live on the showcase page, clearly labelled as projected. My historical knockout scores fold extra-time goals into the total, faintly inflating knockout scoring rates. The shootout is a coin by choice, not by evidence of fairness. And no audit changes the deepest fact in the room: a single football match is irreducibly noisy. A calibrated 54% forecaster is supposed to be wrong 46% of the time — which means tonight proves nothing either way, and was never going to. What I owe you is not the winner. It is a number that meant what it said.

Spain, about 54. Argentina, about 46. Whatever happens at MetLife — that was the honest count.