Football · published error bar

The football house projection backtest

This page is fantasy football. The house model keeps vendor volume and replaces vendor efficiency with each player’s own prior-season rates, shrunk toward the positional mean. That claim is testable. The script is tools/backtest.mjs. Run it. NBA, MLB, and NHL do not run this house model — they rank on per-game (or per-appearance) rates × eGP, without the football missed-game haircut, and they have no number here.

Local guest state No trackers Published methodology Works offline

Mean absolute error

14.09 from 18.04 pts

Improvement

21.8% less MAE, held out

Rank correlation

0.972 Spearman, from 0.952

Cohort

2,264 skill players, 7 season pairs

What was measured

Leave-one-season-out cross-validation over seven consecutive season pairs, 2018→2019 through 2024→2025. The shrink strength is fitted on six pairs and scored on the seventh, then rotated, so every number above is one the model had not seen. Truth is each player’s actual next-season line, scored through the same scorer as the prediction, so a scoring convention cannot show up as model error. Positional means come from the earlier season only.

Both arms are handed identical volume — carries, receptions, pass attempts — so the comparison isolates one question: does shrinking last year’s efficiency beat using it raw? Which volume they are handed is the whole argument, and both answers are below.

The number above is a ceiling. Here is the floor.

The 21.8% hands both arms the player’s actual future volume. Nobody has that in August. Hand both arms last season’s volume instead and the same cross-validation returns 0.9% — mean absolute error 37.00 against 36.69, and negative on two of the seven folds. Add back the players who had a real season and then did not come back at all, 865 of them, and cross-validation picks no shrinkage whatsoever.

Efficiency regression buys little when the volume is also stale, because volume error swamps it: 37 points of error against 18. That is the honest shape of this model.

Production sits between those two numbers, and we cannot tell you where. The vendor supplies a projected volume for the season being drafted — a real forecast, not a copy of last year — and the house model replaces only the efficiency. So the ceiling arm overstates the gain and the floor arm understates it. The blend weight is capped at 0.35 for exactly this reason: the cap is the hedge against the ceiling, not a tuning knob.

Other limits

Seven season pairs is better than one. It is not many. The survivor cohort requires six games on both sides, which excludes the player who was hurt in week two — that is what the washout arm above exists to price, and it is the arm where the model looks worst. The production blend partner is the vendor’s full projection, which is stronger than the raw-efficiency baseline benchmarked here, so none of this is a claim about beating a vendor.

This is a comparison to a baseline — extrapolating last season’s efficiency forward, the assumption most projections ship with — not a claim that any other product is wrong.

NBA, MLB, and NHL are not on this page. Those packs ship with the house model off. Season value is rate × eGP. Football’s missed-game haircut is not applied. ESPN hockey and baseball rest-of-season totals are scaled to a rate before they blend with Marcel. Do not cite 21.8% or 0.9% as a basketball, baseball, or hockey accuracy figure.

Reproduce it

From the repo: node tools/backtest.mjs. It runs offline against tools/season-lines.json and prints all three arms, every fold, and the HOUSE_MODEL block that public/engine.js ships. node tools/backtest.mjs --check runs on every push and fails the build if these numbers drift from tools/backtest-pin.json. If the page and the script ever disagree, believe the script — and the build should already have caught it.