I know a lot of people here use my projection models to help inform their evaluations of prospects. I am generally happy with the performance of both the EWP and HUMBLE models, but there is no question that they sometimes make mistakes. The goal of this post is to pull up some of the more extreme historical cases where the models flubbed. Hopefully, some patterns will emerge that help us gauge the reliability of the model for different 2015 prospects.
I took all of the players drafted between 1990 and 2012, and subtracted their observed NBA wins peak from their projected average between EWP and HUMBLE models. At this level, the results are a bit awkward, since guy like Dirk, Paul, and Durant, all of whom the models loved, top the list of "underrated". To make sure I am focusing on guys who the models actually incorrectly assessed, I limit the "overrated" set to guys with an AVG expectation greater than 5 and the "underrated" set to guys with an AVG expectation of less than 5.
I will present these lists with some limited commentary, but I hope for some discussion of patterns in the misses in the comments.
In general, the models do a good job of not becoming overly excited by point-guard prospects. Jay Williams tops the list, but he failed for a reason that was completely impossible to predict. Of the non-injury cases, both the top two are great players who simply haven't (yet) lived up to lofty model expectations.
Interestingly, scouts seem to do a good job of picking out great prospects who my models miss on. Both Steve Nash and Deron Williams were drafted well above where the numbers recommended and became stars. Some other cases including Westbrook, Wall, and Rose passed the 5 AVG cutoff but were still definitely underrated by the numbers as well.
None of these are actually that bad. Calathes (who is probably more a PG) was clearly overrated by the EWP model, but the rest are pretty accurately pegged. This is great news for D'Angelo Russell, Emmanuel Mudiay, and Delon Wright are are projected as "shooting guards" (Pos 1.5-2.5) and score 10.6, 7, and 5.3 respectively.
The model seems to have bigger issues in under rather than overrating shooting-guard prospects. The model completely missed on several excellent shooting-guards. Barry and Redd can both be explained by dramatically improved three-point shooting after college. Roy was flagged for poor steals and blocks which apparently wasn't an issue. I still don't know what to think about hyper-recruit and bland college player Lance Stephenson. Devin Booker and R.J. Hunter should be happy about this.
Basically DerMarr Johnson, is the only real error on this list. You might be thinking "Joe Smith isn't a 3" and you'd be right. Positions are assigned by a model projecting NBA position from NCAA stats. Joe Smith is apparently an error on that model as well. The model actually pegged the last three quite accurately.
The good news is that even if you struggle by the models' standards, you could still end up an NBA star from the SF spot. The patterning of under/overrated is is great news for Winslow (8.9), Johnson (7.2), Oubre (5.9), Hezonja (5.2), and Rondae Hollis-Jefferson (5.2) (as well as Wiggins from last season and Otto from 2013). Small forwards seem much more likely to exceed rather than fall short of expectations.
Relative to the wing positions, it looks like 4s are more likely to bust. Tyrus Thomas and Michael Beasley were huge misses. Unfortunately, these were all cases where both the numbers and the scouts were mislead. That makes it difficult to sort out the busts in the future.
|David Lee '05||30||2005||3.9||3171||3.4||3.1||3.2||8||4.8|
The model also struggles in the other direction with power forwards. The models actually did better than the scouts on Millsap, Boozer, and Lee... but the professional decision-makers were much smarter in their praise of LaMarcus Aldridge.
I won't worry about missing on Oden. Bradley is an unsurprising wonky case, and given that he was actually pretty solid I won't worry about that one either. The bigger issue seems to be international big-man prospects. This could be a flag on Porzingis who I am already worried about.
Doesn't look to be a common theme with these misses, and with the exception of Brook Lopez, it appears the models and scouts agreed on these players.