How Do We Assess "Potential" Among NBA Draft Prospects?

My goal with this article is to investigate the concept of ‘potential' (AKA ‘ceiling' or ‘upside'). These are terms that scouts, media draft experts, and sports fans love to throw around as we approach the NBA draft. The simplest feature of potential is age. An 18-year-old freshman doing the same thing as a 23-year-old senior clearly has more ‘potential'. I do not think there is much serious debate over that idea, so I am more interested in discussing potential within a given age group. While people often leave potential undefined when they use it to describe NBA prospects, there seems to be strong inter-rater agreement regarding which players have it and which do not. I want to highlight some of the key assumptions that seem to be built into the potential concept and challenge them, or at least open them up for debate.

Potential as physical tools:

The key feature of potential (as used in prospect evaluation) is the idea of physical profile above all else. How tall is he? How high does he jump? How good would he be if his mortal shell was inhabited by a spirit with perfect basketball skills and understanding? I admit that this definition is not entirely fair. If it were that simple, D.J. Stephens would have been the #1 pick in 2013 with the intention of turning him into the next great shooting-guard. However, an emphasis on physical traits clearly plays the central role in evaluating potential.

There is some solid logic behind this idea. It is possible to study rates of improvement across different skill-sets, but there will always be ambiguity regarding how much better a player can become at shooting, dribbling, defending, finishing at the rim... Whereas we all know that "you can't teach height." I quibble with this perspective to a certain extent. For example, the current poster-boy for drafting on athleticism, Paul George, grew 2" in his first season with the Pacers and that has undoubtedly played a role in his rapid improvement. We are seeing something similar in 2014 with Giannis Antetokounmpo. Not only that, but there are many players who can credit aggressive body-reshaping for a significant portion of their success. Look at Kevin Love from draft-day to now for a great example of this.

This is important to note, because it leads us to ask things like "Should we expect Nik Stauskas who made an impressive physical change between his freshman and sophomore year to have the potential to continue improving physically?"

I bring up these examples to challenge the default assumption that the physical is set in stone and all improvement must come elsewhere, but I do think it is safe to say physical profile is less malleable than most components of a successful basketball player.

So granting that, let's look at skill development and some of the unstated assumptions that go into how that works with respect to ‘potential'.

Shortcomings as potential:

I often hear comments like: "He just needs to improve his shot..." or "once he learns how to handle better..." These statements speak to the idea that shortcomings in certain areas, while not technically positives, signal room for growth (i.e. potential). Nothing wrong with that, but it carries with it the assumption that guys who already possess those skills do not have the same potential to get that much better at them. People have an intuitive appreciation for diminishing returns to skill learning and I think that is where this assumption comes from. However, I can think of at least two models for skill development that include diminishing returns, one of which does not lead to the conclusion that ‘room for improvement' says anything about potential.

Model 1 imagines all players progressing along the same diminishing curve with some players positioned further along than others. Model 2 imagines different curves for every individual and their current production is simply a function of how much work they have invested and where that places them based on their fundamental capacity. When someone treats a shortcoming in a player's game as "room for growth" they are essentially working from model 1. Because player "B" is lower on the curve than player "A", another year of investment in training will have greater returns for "B" than "A". However, if we assume model 2 this is not the case. Additional investment will have the same returns for each player. "B" will become ‘serviceable' in that area, while "A" will become ‘great'. If we assume model 2, there is no difference in ‘potential' between players "A" and "B", "A" is simply better.

So which model is correct? In order to address that question, I took a bunch of players who left college before turning 20, then looked at their progression from their 20 years old NBA season to their 22 years old NBA season. The plots below compare progression across a range of statistics for the top and bottom half of performers at each statistic in college.

Unsurprisingly, the players who were better at a given skill in college were better at all ages in the NBA, but what we are interested is in the change across the three years. The plot for ASTs/TOs in the top right corner is exactly what we would expect if Model 2 was the best way to describe individual differences in skill-level. Both groups progress in making decisions with the ball at an even rate with the high-skilled players reaching new heights and the low-skilled players holding the gap steady. The progression in points (per 40) actually shows the high-skilled college scorers improving at a faster rate than the low-skilled which is completely inconsistent with Model 1. Alternatively, scoring efficiency fits Model 1 very nicely. The more efficient college scorers are seeing minimal returns to increased development while the low-skilled group closes the gap between age 20 and 21. The high-skilled group also appears to be hitting a wall in improving free-throw percentage and foul rates, allowing those who struggled in college to catch up. Interestingly, we see no development in rebounds, steals, or blocks (high-skilled blockers actually decrease after their first season something we also see in college which I attribute to opponent game-planning) for either group. This is consistent with analyses I have done in the past. These traits are something a player either has or does not have. Do not expect a prospect who cannot block, steal, or board to figure out how once he enters the NBA (not that this never happens of course). Instead, these traits should be viewed as a part of the baseline a player has to work from, much as height and leaping ability are popularly understood.

We see here that neither Model 1 nor Model 2 works as a universal perspective on skill development. Not only is this the case across different skills, but the models are likely variably applicable across players. For example, most prospects have had considerable basketball training throughout their lives resulting in relatively equal investment and arguing for Model 2, however in cases like Joel Embiid where a player had a late start it may be much more appropriate to assume Model 1.

Potential as an underlying cognitive trait:

Players who fill up box-scores and show a high level of skill across a variety of traits are often denied access to the ‘high potential' category if they lack impressive athleticism. Just look at how some current NBA stars were viewed entering the league. Draft Express had this to say about Kevin Love in 2008: "... There are serious doubts about how his proficiency will translate to the pro level... there really aren't many players at his height with his lack of athleticism in the pros, and it's tough to guess how high in the draft a team will be willing to take a chance on him." Here are some comments on Marc Gasol "The current leader in efficiency rating in the ACB League, Marc Gasol has built a pretty mistake-free style of game that helps him to emerge as a statistical standout.... How much will Gasol's lack of athleticism get exposed in the NBA? I guess that's the question every single decision maker will be asking himself" before he fell to the second round in 2007. Paul Millsap dropped to the late second round in 2006 in spite of impressive collegiate production. The reason for Millsap's drop is likely captured in NBADraft.net's lowly 7/10, 5/10. And 7/10 ratings for athleticism, size, and potential. Stephen Curry and James Harden both earned a "limited upside?" flag from Draft Express; Curry due to his "frail frame" and "average athleticism, lateral quickness, and wingspan", Harden due to his "average size and athleticism." These are all players I would put on the far right of the ‘raw' to ‘skilled' continuum, yet they moved along faster growth trajectories in the pros than any of their peers. Those growth trajectories are exactly what the concept of potential is supposed to be measuring.

[BTW... I do not mean to rag on DX here, they do a much better job than other draft media. See here for a great and relevant DX article. I only use their comments because they are easily accessible and represent commonly held opinions at the time]

The key to the success of all of these players is that they started with a really nice skill-set, but then added to those tools every season. This makes sense if we expand the idea of unique individual skill-curves to some core underlying trait that applies more broadly than any single skill. Call it learning-ability, work-ethic, BBIQ, coordination, or whatever else you want. The key is that a player has some trait (or collection of traits) that results in more rapid accretion of skills. If we believe this is the case, a player who already has an ‘old man game' as a freshman in college should be labeled as ‘high potential' rather than just the back-handed ‘NBA ready', because we can expect him to continue to develop at a faster rate than his peers.

This is a more complex phenomenon to try to test, but I did throw together a tentative analysis using two traits that I think are probing that underlying quality better than others. Players who collect lots of steals and have a good assist-to-turnover ratio in college tend to perform better in the NBA, not just at passing and stealing, but in general. In fact, I found that NCAA steals are actually one of the better predictors of NBA offensive RAPM when building my draft models. Steals and assist-to-turnover ratio seem to say something about a player's awareness and understanding of the game that makes them useful markers of potential. What I did here is look at progression in points from age 20 to 22 as we did above, but instead of splitting players into groups based on college points, I split them into high/low-ast-to-tov and high/low-steals:

As you can see, in both cases, the high-skill groups continue progressing at a steady rate, while the low-skill groups hit a wall at 21. Remember, this is happening even though we are looking at skill in completely different statistics. A lot more needs to be done to properly test this theory of ‘potential', but this shows some support for looking at individual differences in expected growth that may have nothing to do with athleticism.

This should start to make us worry about those ‘raw' prospects who succeed at the college level based purely on physical tools. Athleticism alone is not going to cut it at the NBA level, and if a player does not show a diverse skill-set or high BBIQ by the time he is in college, we really should worry about his potential to ever do so. The list of players who my draft projection model thought were great prospects based on college production but failed in the NBA is littered with these kinds of players. Tyrus Thomas, Stromile Swift, Eddie Griffin, Patrick O'Bryant, Hassan Whiteside, Anthony Randolph, Tyreke Evans... There are a lot more raw athletes to drastically under-perform based on college production than there are highly skilled mediocre athletes.

A place for athleticism:

If you were nodding along to the previous sections, you may be asking questions like "who really has more potential between say... Kyle Anderson and Andrew Wiggins, when the former is superior across nearly every basketball skill." I really do think this is a line of reasoning that should be given more weight, but I want to take some time to point out that there is evidence supporting the role of physical tools in assessing potential.

Based on my production-focused draft model, the clearest example I can find of athleticism triumphing over college production is Richard Jefferson. I would have pegged Jefferson as only the 3^rd best prospect on the Arizona Wildcats in 2001, way behind Gilbert Arenas (who inexplicably slipped to the 2^nd) and just behind Loren Woods as a fringe 1^st rounder. However, due to his freakish athleticism (along with other factors like shutting down Jason Richardson in the tournament) Jefferson was drafted in the lottery. I would have scoffed at the pick if a team made it today, but it clearly paid-off. Eric Bledsoe, who looked like a complete dud at Kentucky, may ultimately become an even better example.

There are other less extreme anecdotes as well. DeMar DeRozan looked like a 2nd-rounder based on production, but after years of being force-fed opportunities he has finally turned into a very good player. Russell Westbrook looked like a fringe lottery pick based on college production (EWP 5.4), but he was drafted 4^th for his physical tools and became a star. Derrick Rose looked like a definite lottery pick (EWP 8.1) based on production, but not a guy to target at #1... that ended up being wrong. My model actually has Andre Drummond as the second best prospect in 2012 (EWP 8.2), but only because it was a weak class. Drummond will quickly eclipse that prediction. Similarly, Paul George didn't look quite like the future star he has become (EWP 8.3). All of these cases are consistent with the ‘athleticism as upside' narrative, but I could probably string together a list of surprises to support any storyline. In order to really test the role of athleticism we need to look more comprehensively.

I include height in all of my draft projection models because although most of its value is captured in block and rebound rates, there is something about height that speaks to future success beyond those statistics. Other physical attributes are more difficult to usefully quantify. Several studies have successfully used combine measurements to explain NBA success, either as part of a complete model including production (shutupandjam), or as a lone indicator of potential (Hardwood Paroxysm, and my ATH model). The latter approach is not particularly telling, because it fails to evaluate whether athleticism carries information above and beyond actual on-court production, but the fact that some physical measurements improve predictions even along with box-score statistics tells us we should be paying attention.

To give a sense of the role athleticism plays in helping explain potential, I ran a regression predicting future NBA production of college players using age, strength-of-schedule, margin-of-victory, and standard box-score statistics. I then merged the predictions generated by that production-only model with different combine measurements to try to improve prediction accuracy. By far the most useful measurement is no-step vertical (which is only a little better than max vertical, but they are highly correlated measures). Vertical leap seems to be a very good catch-all measure of basketball athleticism, which is consistent with the popular approach to identifying the athletic level of different players. What is really interesting is that when I simply include the production-based prediction along with vertical leap as linear predictors, the vertical measure carries some information but is not quite statistically significant even with a sample of nearly 500 players. However, vertical leap becomes much more informative when I include an interaction effect between production and leaping. What this tells us is that while athleticism says something about potential, most of that information only applies to highly-skilled players.

The above plot shows the expected wins peak for hypothetical players scoring a 10 (red), 5 (green) and 2.5 (blue) in the production-only model after accounting for vertical leaping ability. For example, take a player who only looks like a solid prospect based on production like say... Andrew Wiggins with a 4.8 EWP. Even if we assume the draft-combine record no-step vertical of 40" for Wiggins, the athleticism adjustment only bumps his prediction up to a 6.4 expected wins peak. Now, imagine a player predicted to be a borderline star producing 10 wins at his peak based purely on college production. Combined with that same 40" vertical, his predicted peak rises to 15.7 wins which is superstar territory.

This is only a quick and dirty test of athleticism's role in assessing potential. Lumping all of the other variables into a single production effect before pairing it with athleticism makes the analysis easier to communicate, but I also think it exaggerates the actual effect-size of vertical leap in this case. Furthermore, appropriately quantifying athleticism is difficult. Obviously vertical leap is not all of the story, and the combine has many limitations as a measurement tool (especially as performance is increasingly skewed by special training). I think this does give a solid picture of the type of relationship we should expect between athleticism, production, and potential. Athleticism may be an important tool for separating the superstars from the stars, but it does very little to excuse mediocre collegiate production.

Conclusion:

The analyses I did in this article are relatively slapdash. I glossed over a lot of complexity and failed to pursue any questions with much depth. I do not intend anything you just read to offer a final answer, but I do hope to push people to reevaluate their concept of potential. I'm tired of the word being carelessly thrown around, all too often as a crutch for lazy thinking. Yes, it matters that one player is more athletic than another, but how much do you weight that against other traits when trying to predict growth curves? Yes, I understand "you can't teach height", but can you teach rebounding, ball-stealing, or shot-blocking?... There are so many unanswered questions about what factors shape ‘potential' and we will not answer them until more people are challenged to actually defend their assertions about who has it and who does not.

You can see my 2014 draft projection models here or find me on Twitter @VJL_bball