clock menu more-arrow no yes

Filed under:

Statistical Draft Modelling: Where the Models Have Missed

New, comments

Where have vlj110's draft models missed most over the years?

Kirby Lee-USA TODAY Sports

I know a lot of people here use my projection models to help inform their evaluations of prospects. I am generally happy with the performance of both the EWP and HUMBLE models, but there is no question that they sometimes make mistakes. The goal of this post is to pull up some of the more extreme historical cases where the models flubbed. Hopefully, some patterns will emerge that help us gauge the reliability of the model for different 2015 prospects.

I took all of the players drafted between 1990 and 2012, and subtracted their observed NBA wins peak from their projected average between EWP and HUMBLE models. At this level, the results are a bit awkward, since guy like Dirk, Paul, and Durant, all of whom the models loved, top the list of "underrated". To make sure I am focusing on guys who the models actually incorrectly assessed, I limit the "overrated" set to guys with an AVG expectation greater than 5 and the "underrated" set to guys with an AVG expectation of less than 5.

I will present these lists with some limited commentary, but I hope for some discussion of patterns in the misses in the comments.

Point Guards:

Overrated:

Name Pick Season Pos MP EWP HUM AVG OBS DIF
Jay Williams 2 2002 1.0 3573 6.8 6.9 6.8 0.9 -5.9
Ricky Rubio 5 2009 1.2 3684 12.3 8.6 10.4 7.5 -2.9
Kyrie Irving 1 2011 1.2 303 11 9.9 10.4 8 -2.4
T.J. Ford 8 2003 1.0 2178 5.1 6.8 5.9 3.9 -2
Dion Waiters 4 2012 1.5 1462 3.6 7.4 5.5 3.8 -1.7

In general, the models do a good job of not becoming overly excited by point-guard prospects. Jay Williams tops the list, but he failed for a reason that was completely impossible to predict. Of the non-injury cases, both the top two are great players who simply haven't (yet) lived up to lofty model expectations.

Underrated:

Name Pick Season Pos MP EWP HUM AVG OBS DIF
Steve Nash 15 1996 1.1 3402 2.2 3.2 2.7 12.3 9.6
Sam Cassell 24 1993 1.4 2344 2.4 3.1 2.8 9.5 6.7
Deron Williams 3 2005 1.1 3198 3.3 6 4.7 10 5.3
Terrell Brandon 11 1991 1.0 2175 3.2 4.3 3.8 9 5.2
Rafer Alston 39 1998 1.0 1030 1.7 2.3 2 6.6 4.6

Interestingly, scouts seem to do a good job of picking out great prospects who my models miss on. Both Steve Nash and Deron Williams were drafted well above where the numbers recommended and became stars. Some other cases including Westbrook, Wall, and Rose passed the 5 AVG cutoff but were still definitely underrated by the numbers as well.

Shooting Guards:

Overrated:

Name Pick Season Pos MP EWP HUM AVG OBS DIF
Nick Calathes 45 2009 1.5 2641 7.2 3.9 5.5 2.5 -3
Todd Day 8 1992 2.3 2840 5.7 6.2 6 3.9 -2.1
Tyreke Evans 4 2009 2.0 1073 5.9 6.8 6.3 4.7 -1.6
Bonzi Wells 11 1998 2.2 3540 7.4 5.7 6.6 5.2 -1.4
Jim Jackson 4 1992 2.4 3165 4.5 6.5 5.5 4.1 -1.4

None of these are actually that bad. Calathes (who is probably more a PG) was clearly overrated by the EWP model, but the rest are pretty accurately pegged. This is great news for D'Angelo Russell, Emmanuel Mudiay, and Delon Wright are are projected as "shooting guards" (Pos 1.5-2.5) and score 10.6, 7, and 5.3 respectively.

Underrated:

Name Pick Season Pos MP EWP HUM AVG OBS DIF
Brent Barry 15 1995 2.0 3123 2.3 4.8 3.5 9.2 5.7
Brandon Roy 6 2006 2.3 2838 2.6 5.6 4.1 9.7 5.6
Michael Redd 43 2000 2.0 3324 3 2.5 2.8 8 5.2
Cuttino Mobley 41 1998 1.7 2762 1.6 1.1 1.4 6.6 5.2
Lance Stephenson 40 2010 2.2 902 1.7 1.5 1.6 6.8 5.2

The model seems to have bigger issues in under rather than overrating shooting-guard prospects. The model completely missed on several excellent shooting-guards. Barry and Redd can both be explained by dramatically improved three-point shooting after college. Roy was flagged for poor steals and blocks which apparently wasn't an issue. I still don't know what to think about hyper-recruit and bland college player Lance Stephenson. Devin Booker and R.J. Hunter should be happy about this.

Small Forwards:

Overrated:

Name Pick Season Pos MP EWP HUM AVG OBS DIF
DerMarr Johnson 6 2000 2.8 879 5.3 6.7 6 2.4 -3.6
Joe Smith 1 1995 3.4 2098 10.1 9.1 9.6 6.5 -3.1
Billy Owens 3 1991 3.2 2403 5.8 7.9 6.8 5.9 -0.9
Tracy Murray 18 1992 3.1 2949 5.5 5 5.2 4.4 -0.8
Toni Kukoc 29 1990 3.0 115 8.4 NA 8.4 7.7 -0.7

Basically DerMarr Johnson, is the only real error on this list. You might be thinking "Joe Smith isn't a 3" and you'd be right. Positions are assigned by a model projecting NBA position from NCAA stats. Joe Smith is apparently an error on that model as well. The model actually pegged the last three quite accurately.

Underrated:

Name Pick Season Pos MP EWP HUM AVG OBS DIF
Gerald Wallace 25 2001 3.3 824 4.9 3.8 4.3 10.3 6
Bryon Russell 45 1993 3.3 2334 1.8 1.9 1.8 7.8 6
Joe Johnson 10 2001 2.6 1605 2.6 4.4 3.5 8.2 4.7
Jimmy Butler 30 2011 2.8 3033 2.9 2.8 2.8 7.3 4.5
Richard Jefferson 13 2001 2.9 2211 4.3 4.9 4.6 8.6 4

The good news is that even if you struggle by the models' standards, you could still end up an NBA star from the SF spot. The patterning of under/overrated is is great news for Winslow (8.9), Johnson (7.2), Oubre (5.9), Hezonja (5.2), and Rondae Hollis-Jefferson (5.2) (as well as Wiggins from last season and Otto from 2013). Small forwards seem much more likely to exceed rather than fall short of expectations.

Power Forwards:

Overrated:

Name Pick Season Pos MP EWP HUM AVG OBS DIF
Tyrus Thomas 4 2006 4.0 829 12 9.1 10.6 4 -6.6
Michael Beasley 2 2008 3.7 1194 9.1 8.8 8.9 2.9 -6
Doug Smith 6 1991 3.7 1993 4.9 5.4 5.2 1 -4.2
Brandan Wright 8 2007 3.8 1014 6.4 6.9 6.7 3.5 -3.2
Stromile Swift 2 2000 4.1 1332 6 6.9 6.5 4 -2.5

Relative to the wing positions, it looks like 4s are more likely to bust. Tyrus Thomas and Michael Beasley were huge misses. Unfortunately, these were all cases where both the numbers and the scouts were mislead. That makes it difficult to sort out the busts in the future.

Underrated:

Name Pick Season Pos MP EWP HUM AVG OBS DIF
Antonio Davis 45 1990 4.1 991 1.4 1.2 1.3 7.4 6.1
LaMarcus Aldridge 2 2006 4.4 1602 3.8 4.7 4.2 10.1 5.9
Paul Millsap 47 2006 4.0 3239 4.4 4.6 4.5 9.6 5.1
Carlos Boozer 34 2002 4.0 2621 4.3 3.9 4.1 9 4.9
David Lee '05 30 2005 3.9 3171 3.4 3.1 3.2 8 4.8

The model also struggles in the other direction with power forwards. The models actually did better than the scouts on Millsap, Boozer, and Lee... but the professional decision-makers were much smarter in their praise of LaMarcus Aldridge.

Centers:

Overrated:

Name Pick Season Pos MP EWP HUM AVG OBS DIF
Greg Oden 1 2007 5.0 925 13.1 11.2 12.1 4.3 -7.8
Shawn Bradley 2 1993 5.0 984 12.7 8.9 10.8 7.5 -3.3
Bismack Biyombo 7 2011 4.6 239 6.2 8.7 7.4 5 -2.4
Andrea Bargnani 1 2006 4.5 2365 5 5.8 5.4 3.1 -2.3
Enes Kanter 3 2011 4.8 697 7 6.4 6.7 4.6 -2.1

I won't worry about missing on Oden. Bradley is an unsurprising wonky case, and given that he was actually pretty solid I won't worry about that one either. The bigger issue seems to be international big-man prospects. This could be a flag on Porzingis who I am already worried about.

Underrated:

Name Pick Season Pos MP EWP HUM AVG OBS DIF
Brook Lopez 10 2008 5.0 1487 2.8 3.5 3.1 8.8 5.7
P.J. Brown 29 1992 4.6 2539 2.8 3.2 3 8.3 5.3
DeAndre Jordan 35 2008 4.9 781 3.1 2.2 2.7 7.7 5
Theo Ratliff 18 1995 4.5 2926 2.5 3.6 3 7.6 4.6
Samuel Dalembert 26 2001 5.0 1265 2.5 3.3 2.9 7.3 4.4

Doesn't look to be a common theme with these misses, and with the exception of Brook Lopez, it appears the models and scouts agreed on these players.