NBA Draft ProjectionsV4

Method

How V4 works

The fourth go at the same question I started with in 2021: from what a prospect did before the draft, how good will he be in the NBA? V4 is mostly about making the answer honest.

Where this came from

V1 was a notebook before the 2021 draft. I scraped Tankathon and Basketball Reference, averaged a few regression models and ranked the class (write-up here). V2 in 2024 turned that into a Streamlit dashboard with two ratings and ceiling and floor comps. V3 rebuilt the data from open sources and finally checked the model against the order teams actually drafted in.

V3 had two problems I didn't see at the time. Its settings were picked on the same backtest it reported, so its numbers were flattering. And the way it counted busts quietly broke the draft-order baseline on Immediate. V4 fixes both, adds two per-possession targets, and only keeps upgrades that held up on classes the model hadn't seen.

The data

Every pick from 2000 to 2026, 1,602 players, comes from the NBA's draft history through nba_api, with combine measurements and birthdates for players Basketball Reference doesn't have.

College numbers are built from ESPN box scores published by the SportsDataverse hoopR project. V4 adds up a player's last three college seasons instead of just the last one, weighted 5/3/2 toward the most recent, which gives per-40 numbers, shooting splits and rate stats from a bigger sample. From 2013-14 on I also use ESPN's play-by-play for shot types: how many of his shots came at the rim, from mid range and from dunks, how well he finished at the rim and from mid range, and how many of his makes were assisted. The assister on a made shot matches the "Assisted by" text 99.9% of the time, so I trust it. Before 2013-14 the play-by-play mostly only has made shots, so those classes don't get shot types. Matching names between ESPN and the NBA worked for 99% of college picks from 2005.

NBA outcomes are Basketball Reference's season VORP and BPM from a public CSV mirror of their tables, through 2025-26. For plus-minus I pulled every five-man lineup's possessions and points for and against from stats.nba.com, one team and season at a time, 570 requests spaced out and cached, from 2007-08 (the first season it has) to 2025-26.

303 picks came from overseas, 36 from high school and 16 from the G League or Overtime Elite. They stay in, with a background flag and no college numbers, so the model judges them on age, size and path.

The targets

The two headline ratings are the same as V2 and V3:

The bust fix. V3 counted a season a player didn't play as 0, which is replacement level, but kept a negative VORP for a player who did play and was bad. So a second-rounder who never played ranked above half the rotation players picked 6 to 30. On Immediate, the median never-played pick finished at 0.58 inside his class (1 is first), ahead of the median player with 500+ minutes at 0.44. Now a season below replacement counts as replacement, the same as not playing, since a team can always get replacement level for free. That one change took the draft order's Immediate correlation from 0.06 to 0.42 and its Peak correlation from 0.45 to 0.57. V3's big Immediate win over the teams was mostly this bug. The new version also tracks later careers better: it correlates 0.64 with a player's BPM in seasons 3 to 6, against 0.54 for the old one.

VORP rewards minutes, and high picks get minutes, so part of the draft order's score is just opportunity. Two targets that only count how good a player was per possession:

Nothing on any target counts below replacement. The VORP ratings stay the headline on the board, since that's the question I care about. Peak BPM and Peak RAPM show up in each player's drawer. Immediate RAPM is only on the backtest page, because two seasons of plus-minus for young players is mostly the prior.

Careers are still going for recent classes, so the peak targets only count classes up to 2020 and Immediate up to 2024. Newer players never go into training, and the board shows their numbers so far.

The model

Two models, blended. One is LightGBM with small trees. The other is a ridge regression on the same inputs, with missing values filled by the median plus a flag for each. Both learn where each player finished within his own class, from 0 to 1, since the draft is a ranking problem. Recent public draft models keep finding that a regularised linear model does as well as boosting at this sample size, and CraftedNBA blends the two, so I did the same. The ridge mattered most on Immediate. The tuning can also force "younger is better" on age in LightGBM, and it chose that in most classes.

To turn a score back into VORP, BPM or plus-minus, I bin out-of-fold scores into 20 slices and use what past players in each slice actually did.

There are two versions. Stats only never sees the draft slot. It's the headline, because it's the one that can tell you something the teams don't already know. Stats + draft slot adds the log of the pick number, with the constraint that a later pick can only lower the projection. It reuses the settings tuned for the stats-only version and only picks its blend weight again. That's the version to use if you want the best guess, since every strong public model with a headline number mixes in consensus somehow.

The backtest, done properly this time

For every class from 2010 on, the model only gets the drafts before it. Inside those, the settings are tuned the same way: each of the last six earlier classes is predicted from the classes before it, Optuna tries 40 settings for LightGBM, the ridge strength and the blend weight are picked on the same earlier classes, and only then is the test class ranked. None of the test classes ever touch the tuning. I report Spearman within each class, how many of the real top 10 it had in its top 10, NDCG@10 (with each player's gain being how far he got above replacement), and how often the 80% range held.

Averaged over the test classes, stats only against the draft order:

So the teams still win on the long-run questions on stats alone, and it's roughly even on the first two years. The pre-draft numbers do add something the teams miss: the model with the slot clearly beats the slot on its own on Immediate and both plus-minus targets. On Peak and Peak BPM it's level, inside the noise. All of this is on the backtest page, next to V3's published numbers.

One step does look at the test classes: deciding which groups of features to keep. Settings are tuned blind, but I compared each feature idea on these same classes with fixed settings. I decided the rule before running it: an idea stays only if it helps on average and in most classes. With 11 to 15 classes per target, anything under about 0.03 in Spearman is noise anyway.

What I tried that didn't earn a place

The code for all of it is still in the repo, switched off, so the comparison can be rerun with python ablation.py.

Ranges

Each projection comes with an 80% range, calibrated on earlier classes only. I take the 10th to 90th percentile curves of what past players did at each score from half the earlier classes, then use the other half, which includes the most recent class, to widen or narrow them until 80% land inside. It's conformalised quantile regression with one twist. So many careers sit exactly at replacement level that a plain 10th to 90th range covered close to 90%, so the calibration step picks the percentile width instead of adding a margin. On the test classes, 81% of Peak, 84% of Immediate, 79% of Peak BPM and 83% of Peak RAPM outcomes landed inside. The ranges for the top prospects are wide, often from replacement level to a 10. That's what 80% honestly looks like for a top-ten pick.

Why a projection is what it is

Each player's drivers are SHAP values for the whole blend: tree SHAP for LightGBM, the exact linear version for the ridge (each coefficient times how far the player is from average), mixed with the blend weight. A missing-value flag counts toward the feature it flags. They show how far each feature moved him within his class, in percentile points. Age is still the biggest driver, then true shooting, team strength and steals.

Comps

Comps work the same way as in V2 and V3. Pre-draft features are standardised and compared by cosine similarity, using only the features both players have, against earlier classes whose careers are long enough to grade. Out of the ten closest, the best career is the ceiling and the worst is the floor.

What it can't see

Sources I leaned on

Rebuilding it

pip install -r requirements-model.txt
python build_dataset.py && python rapm.py && python train.py && python predict.py

Raw downloads and API responses are cached, so a rerun doesn't hit anyone's servers again. The nested backtest takes about six minutes on a 32-thread machine. The code is on GitHub.