The (Ir)Relevance of Early Hype for NBA Draft Prospects

Working from prior expectations is an important mental mooring that keeps us from being swept away by variance. Priors are good, but priors built from vastly inferior sources of information should be quickly abandoned. The case of recruiting status relative to actual college production is a good example of this. High school scouts do a remarkable job of identifying the best teenage basketball players given the massive number of prospects and lack of comparability across competitive venues, but they are far from perfect. For every Anthony Davis or Kevin Durant who lives up to or exceeds the hype, there are several players like Donnell Harvey or Kelvin Torbert who fall far short. NCAA competition offers a vastly superior system for assessing players' ultimate value in the NBA. Performance against other excellent athletes in a standardized setting obviates any information gained watching a player pick apart chunky 6'4" centers or throw down dunks in summer camp. Unfortunately, people far too often fail to change their opinion of prospects after watching them struggle in college and use ‘potential' as a cognitive caulking to fill the void between hype and observed performance.

Shabazz Muhammad and sticky-thinking

I think the best example of this particular brand of sticky-thinking is the story of Shabazz Muhammad. Bazz was a consensus top-2 recruit coming out of high-school and most of us following closely had been hearing his name for at least a year prior to his arrival at UCLA. He was ultimately drafted at the end of the lottery in spite of projection models and many scouts' ‘eye tests' labeling him closer to a 2^nd round talent. The reason I like Shabazz as an example is that the failure to update information spanned beyond simple on-court production. We found out that the whole time Shabazz was destroying high-school competition and garnering ecstatic scouting reports he was actually a year older than everyone thought he was. It may seem silly to put that much emphasis on such a small age difference, but anyone who has spent time trying to project NBA numbers knows this is a big deal. There is a real difference between 18 and 19 and I have little doubt that subjective impressions of Bazz' high-school performance would have been different if scouts knew his actual age. In this case, even his high-school acclaim itself was called into question based on new information, but at least some NBA decision-makers failed to update and Bazz was drafted well above where a similar player without the hype would have landed.

High-school hype matters... until it doesn't

Obviously I do not expect one incomplete anecdote to convince you that allowing high-school hype to influence evaluation is a bad idea. In fact, you should probably be thinking... "Look at how many stars were top prospects coming out of high-school and how few marginal recruits go on to successful pro careers. Obviously high-school hype is important!" I am going to do some simple regression analysis using RSCI (Recruiting Services Consensus Index), NBA production measured in "Win Shares", and basic college statistics to demonstrate how this protest is true but misleading.

First I want to show a plot of the relationship between recruit ranking and NBA Win Shares (in best season before turning 26).

As you can see, recruiters do a good job of identifying future NBA talent. The three most productive NBA players were all top-10 recruits and a much higher percentage of unheralded recruits end up failing to ever make an NBA roster. In order to measure just how good of a predictor high-school hype is of pro success, I built a simple linear regression model with RSCI predicting NBA Win Shares. I transformed the RSCI variable to account for the fact that the difference between say prospects #1 and #2 is much larger than that between #50 and #51, then plugged it into the model. The RSCI variable is a highly significant predictor and explains 17.5% of the variance in NBA performance. That is far from a perfect evaluation tool, but strong enough to pay close attention to. However, this seemingly strong relationship becomes much less relevant once we run prospects through the filter of NCAA competition.

I took the first model and added the following variables that speak to college performance: points, shot attempts, assists, turnovers, rebounds, steals, blocks, and personal fouls all per 40 minutes and pace-adjusted, as well as age and team strength-of-schedule (SOS) and margin-of-victory (MOV). This collection of variables represents easily available information about college production. The resulting model does a much better job of identifying how a college player will do in the pros than just looking at his recruit ranking, but more importantly, after including the production variables the information added by RSCI drops considerably. Here are the standardized beta coefficients which give us the relative importance of each variable for accurately predicting pro success:

I ordered the variables from most to least important. As you can see, the transformed RSCI variable falls to the bottom of the list. In fact, it is not even a statistically significant predictor after controlling for all of the other variables (n = 1123, p-value = 0.08). I also wouldn't be surprised if what predictive power remains is simply a function of a self-fulfilling prophecy wherein top recruits are force-fed opportunities to succeed even when they don't necessarily deserve them, while unheralded recruits fall under the radar and never get a chance to prove themselves. What all of this tells us is that top-tier recruits often make better pros than less impressive high-school players, but top-tier recruits who don't back that hype up in college have little or no more potential than anyone else. To help put this point into perspective, as far back as I have RSCI data (1998) not a single player has gone from a highly acclaimed recruit who struggled in college (defined as less than 5 EWP in my prediction model) to a dominant professional career. The best example I can find is Dan Gadzuric who posted 6.1 ‘wins' in 04/05 despite college production that pegged him as a complete bust. Harrison Barnes who posted a 4.2 EWP as a freshman and a 2.4 as a sophomore has a shot at beating Gadzuric's mark.

Guys who do not look like great prospects based on college production sometimes succeed in the NBA, but looking at pre-college hype does not help identify which players that will be. I should note that this analysis is a bit limited in that it misses key players who went directly to the NBA out of high-school. However, the only way that really matters is if you think LeBron James or Dwight Howard would have struggled against college competition. I personally can live with the assumption that they would have done just fine.

Producing in obscurity:

Not only does the infatuation with high-school hype lead to over-drafting players whose limitations are revealed in college, but it also hurts decision-makers in the other direction. Teams are often slow to catch on to future stars like Dwayne Wade, Kevin Martin, Stephen Curry, Danny Granger, Jameer Nelson, Damian Lillard, and Gilbert Arenas because they slipped into the NCAA relatively unnoticed. Often this just means forcing them to stay in college a couple years longer than less-productive but more hyped prospects, but sometimes the stink of being missed by college scouts stays with a player until he is several seasons into their NBA career. This is especially true for those who lack elite athleticism.

This issue is typically mediated by the problem of evaluators not giving enough respect to players in mid-major conferences. This can be seen when comparing the coefficients of my ‘EWP' (link) and ‘Humble' (link) models. The EWP model includes all of the basic production variables as well as SOS and MOV, both of which are strong positive predictors of NBA success. The Humble model is largely the same, but also includes a variable to capture scout rankings. In this model, SOS and MOV are negative predictors of NBA success. What this says is that mid-major players typically outperform their draft slot. It is important to account for the differences in competition between conferences, but the current consensus has taken this too far.

There is a huge difference between true NBA stars and anyone on the outside looking in, but at the margins there are likely many players who deserve a shot over current bench-warmers. Often those watching games from home or Europe ended up at a lesser program coming out of high-school and never captured much attention because they were putting up numbers in a weak conference or on losing teams. Ultimately these players miss NBA-level training during key developmental years and become 22/23-year-old late 2^nd rounders or undrafted free-agents who never get an honest opportunity to make an NBA team.

Conclusion:

High-school hype might help you make preseason big-boards, but you must be ready to abandon those boards shortly after the conference schedule begins. Maybe Andrew Wiggins, Jabari Parker, and Julius Randle are still the top prospects in the 2014 draft, but I hope you would have reached that assessment having never heard of them until November. I should conclude by noting that NBA professionals have largely moved towards this same conclusion. There are definitely examples of residual-hype impacting draft decisions (ex: Rivers and Mayo), but this bias is not nearly as strong as you see among fans or ESPN personalities. Instead, physical profile seems to dominate RSCI in the eyes of teams looking for ‘potential' and this draws attention to mediocre recruits with excellent athleticism like Zach LaVine, while leading scouts to ignore top recruits who don't `look the part' like Kyle Anderson. This is the issue I will address in my next post.

You can see my 2014 draft projection models here or find me on Twitter @VJL_bball