October 8, 2026 · Research result
Gate 2 failed. The estimated advantage is too uncertain to establish that comps predict NFL production better than recruiting rank.
We aligned 257 quarterbacks at their first 19 career FBS-team games with a passing attempt. Of those, 96 have mature NFL production outcomes and recruiting ranks; 37 qualify for the temporal holdout.
| Outcome | Held out | Comps error | Recruit error | 95% improvement interval | Result |
|---|---|---|---|---|---|
| First 3 NFL seasons — primary | 37 | 3,222.30 | 3,287.46 | -587.84 to 685.43 | Failed |
| First 4 NFL seasons | 21 | 5,203.74 | 4,938.37 | -1,145.23 to 625.47 | Failed |
| Drafted indicator | 58 | 0.34 | 0.32 | -0.09 to 0.03 | Failed |
| Draft round | 58 | 2.49 | 2.45 | -0.40 to 0.32 | Failed |
Errors are mean absolute errors. Improvement = recruiting error minus comps error; the lower confidence bound must exceed zero to pass.
The primary outcome is adjusted passing yards: passing yards + 20 × passing touchdowns − 45 × interceptions, summed across the first three entry-calendar NFL regular seasons. Complete NFL data ends in 2025. First-four-season and draft outcomes are secondary checks.
For each quarterback, training players must have an earlier college cutoff and NFL outcomes available before that cutoff. Both models use five neighbors. The unchanged rule requires ten eligible training players, ten held-out players, and a positive lower bound from 2,000 paired bootstrap samples. Feature weights and thresholds were not tuned after these results.
Of the 257 quarterbacks, 97 have supported NFL or draft identities; 160 remain unknown and are not labeled zero or undrafted. Verified undrafted labels cover NFL entrants, not every undrafted college quarterback. This selection limit and gaps in national college coverage constrain the result. EPA, success rate, opponent rank and spread were unavailable for the common-feature model.
The separate start-aligned check still has nine windows and zero eligible held-out predictions: blocked. The homepage continues to show those published first-19-start windows. This research cohort is a separate study, not a replacement for the homepage snapshot.
CollegeFootballData supplies college statistics. nflverse open releases supply draft, roster and NFL season outcomes. No ESPN or sports-reference pages were scraped.