"What's is the difference between a religion and a superstition? A superstition is specific. You step on a line and your team will lose. A religion in intricate. It fills in all the gaps. You answer one question, another goes in its place. That is why it is so hard to change people's minds." - Bill James at 2012 Sloan Sports Analytics Conference
At its basic level, the Sloan Sports Analytics Conference is about statistics and the ways that those statistics are used to analyze sports. What is the advantage of going 2-of-1 at the end of a quarter? Was it more effective for the Dallas Mavericks to start DeShawn Stevenson or J.J. Barea in the NBA Finals? Does Kevin Durant have a higher shooting percentage from the right wing or the left wing?
At its core, however, the Conference is about something much more important: the ways people think and how, if those ways are challenged, better decisions can often be made as a result.
Bill James, referred to as the Godfather of Analytics, challenged the ways people thought about analyzing the game of baseball. His ideas were not popular or widely accept at first, but over time, the truth of what James was saying began to cause cracks in the foundations of conventional wisdom. Those cracks, like fissures in a dam, have grown to the point where his original ideas have themselves become conventional wisdom and are being challenged by even newer ways of thinking.
Below the fold is a little bit of my personal experience with analytics, along with a recap of this year's event. Whether you already use advanced stats on a regular basis, or you are skeptical, hopefully it will at least spark some conversation.
Challenging long-held ideas is rarely easy. As we go through life, all of us accumulate ideas and customs that we accept as indisputable facts. We develop systems of belief - in sports, in politics, in religion and in other areas of life - that are uncomfortable to challenge.
I am as good of an example of this as anyone, and it is not exactly a secret that my feelings towards so-called "advanced statistics" have not always been positive. I distrusted per-minute stats, dismissed concepts like true shooting percentage and detested aggregate stats like PER and Win Scores.
This attitude was shaped by a variety of factors:
- Laziness. As an English major who spent as little time studying math as possible, concepts like coefficients, standard deviation and regression were not easy to understand and it took considerably less effort to ignore them than to learn them
- Stubbornness. After years of analyzing traditional box scores and believing certain idioms to be true - that Kobe Bryant is clutch, for example - it can be difficult to admit the inaccuracies with those positions
- Naivety. You don't know what you don't know. Without ever having been exposed to concepts like 'efficiency', there was a natural ignorance and fear of the unknown
- Genuine questions. There is still much that advanced stats cannot account for, and as such, traditional theories acted as a 'God of the Gaps'. These gaps keep shrinking as analytics become more advanced, but new questions always seem to pop up in their place
Last year was my first trip to the Sloan Conference, and at that time, the concept of advanced stats in my mind went from being the fascination of a few power geeks to a concept with real world applications that was used by decision makers within professional teams, sports leagues and media organizations.
Front office personnel, such as Mike Zerran of the Boston Celtics, use analytics as a significant tool in player development, transactions and in-game strategy. Zerran also readily admits that traditional scouting and visual observations must be taken into account in order to make any analysis complete. You can attempt to look at statistical production from the college ranks when making draft decisions, but small sample sizes and uncertainty about how those stats will translate to the pro game make statistics insufficient by themselves. You can give a coach or a player a specific piece of information, but if that information is not broken down so that it is easily understood, the analytics are worthless.
At this year's Conference, I spent less time in the panels with the big name guests (although I still went to some), and spent more time watching research paper presentations about new advancements and ideas. Instead of sitting back as a fly on the wall and taking everything in, I had conversations with some key figures in the analytics community and picked their brains. Like last year, it was a rewarding experience that has not only enhanced my knowledge of the game, but has encouraged me to keep searching for new ways of analysis.
Day 1
I'm not used to waking up so early. The alarm clock in my hotel room rang out at 6:30, which would have been 5:30 back in Minnesota. I crawled out of bed, somehow was coherent enough to look halfway presentable, and stumbled down to the Hynes Convention Center.
The first face I saw when I walked into the main hallway was that of ESPN True Hoop's Henry Abbott. [Author's Note: True Hoop's coverage of the Conference this year has been fantastic.] After exchanging hellos, we discussed one of the research papers that was attempting to prove two things: that experience does not have a direct correlation with winning in the playoffs; and that there is an advantage to keeping a winning team in tact. Both are ideas that tend to fly in the face of conventional wisdom, especially from broadcasters, who often declare that youth cannot win in the postseason and that swapping a couple of pieces (like Kendrick Perkins for Jeff Green, for examples) will often improve a team right away.
The presentation was happening later on in the day and I made a note to attend that session. As a side note, all of these research papers are available on www.sloansportsconference.com.
After eating a couple of free breakfast pastries, we all assembled in the main ballroom for the opening panel. This panel was on "The Evolution of Sports Leagues", and featured NHL commissioner Gary Bettman, agent Scott Boras, Major League Baseball executive Rob Manfred, NBA Deputy Commissioner Adam Silver, New York Giants executive Steve Tisch and moderator Michael Wilbon.
For the most part, this panel consisted of sports executives defending the decisions made by sports executives. Gary Bettman essentially said that missing an entire season was exactly what the NHL needed and has made the league more viable in the long run. Adam Silver seemed optimistic about the new NBA collective bargaining agreement. Not a single piece of analytics was mentioned during the entire panel. Of course, looking at those names, that's not exactly a surprise.
Once the panel ended, I went to a couple of research paper presentations.
The first paper was the presentation on "Experience and Winning in the NBA".
There were a number of conclusions that this study's author, James Tarlow, was able to make based upon his research. The first is that there is no positive correlation between NBA experience and winning in the postseason. There is a positive correlation, however, between experience and winning the regular season. In other words, the data shows that veterans are better to help a team get to the postseason, but aren't as valuable once the team is there.
The paper concluded that coaches who are not effective will not get better with time. Recycled coaches who have not had a record of success will not generally improve.
Lastly, the paper's data showed that keeping winning teams in tact is important, and that doing the opposite can have a negative impact on winning. This would appear to confirm that idea of team chemistry, which is an idea largely ignored by advanced stats.
The next paper presentation was entitled, "Big 2's and Big 3's: Analyzing How a Team's Best Players Complement Each Other".
Don't let the name fool you. The paper was not all about the Miami's of the world. The study's author, Robert Ayer, essentially looked at player archetypes and which types worked best with one another. It did not look at player skill - for example, it put players like Kobe Bryant and Steve Francis in the same category - but was only interested in playing style. The study concluded that a high-rebounding, high-scoring center, a high usage point guard and a three-point shooting wing are the best trio to put together. Duplicates, such as two high-usage guards, tend not to perform as well. Players like Danilo Gallinari came out looking incredibly valuable.
There were some problems with the study. For one, it used per game stats, instead of per-minute stats, to make its conclusions. For two, forcing every player into one of 14 archetypes did not always seem effective or accurate. It was interesting, however, to look at questions of fit and how teams can make better pairings with their personnel.
A third paper presentation that I attended was called, "Predicting the Next Pitch".
The conference was not exclusively a basketball conference, so occasionally, I attended panels/presentations that involved other professional sports. In this paper, researchers Gartheeban Ganeshapillai and John Guttag attempted to model a way in which baseball players could learn how to anticipate pitches. The practical implications of a model like this are obvious for hitters, but to relate it to the game of basketball, think of the advantages of coaches could anticipate the types of shots that opposing players would take based upon certain conditions.
It is hard to have any type of reliable prediction model in a game that is so fluid and in which every decision is influenced by a host of other factors, but models like these could at least get players anticipating the types of events they might see from a given opponent.
After the presentations, it was time for lunch. A nice catered lunch was provided by the Conference, but the best part - which was also true of other periods of down time - was the opportunity to chat with some of the panelists and organizers. Over the course of both days, I was able to: discuss the development of Kevin Love with Daryl Morey; pick the brains of ESPN legends like John Hollinger and Larry Coon; chide Bill Simmons about his doubts regarding Ricky Rubio; get gambling advice from Jeffrey Ma; crow about my high school tennis record with Todd Martin; chat sports management with Drew Carey (more on him later); argue about topics like Ryan Anderson vs. Kevin Love with other bloggers; and offer humble praise to Jeff Van Gundy.
I don't mean to mention this to name drop. But it's important to note the willingness that panel participants had to discuss sports with conference goers. The interactive quality of Sloan is definitely one of its key selling points.
Once lunch was over, there were two more paper presentations to attend.
The first was on "Effort vs. Concentration: the Asymmetric Impact of Pressure on NBA Performance".
The study's authors, Matthew Goldman and Justin Rao, showed that self-focus - thinking about the details of how one should accomplish a goal, such as hitting a golf shot - can actually be counterproductive. Their study showed that these tasks are often performed better when there are distractions that prevent the brain from self-focusing.
In basketball, the free throw is an example of a task that involves self-focus. Data shows that visiting teams tend to shoot better from the foul line than home teams, based upon the idea that the distraction of the crowd actually helps visiting players. Crowds should really go deadly silent during opposing player free throw attempts in order to be most effective.
Of course, that's not quite as much fun, so it's doubtful the idea catches on.
The second was on "NBA Chemistry: Positive and Negative Synergies in Basketball".
Phillip Maymin, the study's author, began the presentation by asking the audience whether they would prefer Chris Paul or Deron Williams. He launched into an explanation of his method where he evaluated each player's offense and defense in three categories of scoring, rebounding and ball-handling. Maymin followed that based on these attributes, certain players will work mesh together better than others. Groundbreaking, I know.
Getting back to his original question, Maymin concluded that - based on their skills - Chris Paul would've been more effective with the Jazz and Deron Williams would've been more effective with the Hornets. The study had some obvious problems. The most glaring problem was that Sasha Vujacic's ball-handling skills were rated higher than LeBron James' ball-handling skills.
ESPN stats researcher Dean Oliver once said that any statistical formula needs to pass the "laugh test" in order to be taken seriously. The Vujacic > Love stat clearly didn't do that. The study's author also pronounced names wrong on a consistent basis, like Nicolas Bat-Um, Kevin Durr-Aunt and Durr-Aun Williams, which suggested that he was not much of a basketball fan.
The study was not very convincing or particularly helpful.
Instead of another research paper, it was time for the Basketball Analytics panel, which featured moderator Jackie MacMullen, Jeff Van Gundy, John Hollinger, Mike Zarren and Dean Oliver. Oliver took Mark Cuban's place, since Cuban suddenly became unavailable.
Some of the most interesting revelations from the panel?
A handful of NBA teams have installed cameras in their arenas that can track more sophisticated player data, such as shot location, shot height, player position, speed and many others. The Dallas Mavericks used to have this technology, but have since stopped using it, after the NBA mandated that teams have to share their data. The application of all this data is still being explored, since there is so much to be digested, but clearly, there are advancements and new ways of looking at the game that will continue to evolve thanks to improvements in technology.
When Jeff Van Gundy needed to make a point to his players, and he couldn't get info that would back up his point, he would make up the data.
Dean Oliver's insightful quote that, "there's counting things, and then there's analyzing the things you count."
One of the recurring themes that came up across the Conference was how data should be communicated from GMs and analysts to coaches, and from coaches to players. Van Gundy and Zerran talked about the latter, and how players often have revisionist history when it comes to past events. You have to pick your battles, pick statistics that are easily actionable and often use video to convince players of what you're trying to explain.
Jeff Van Gundy brought up the idea of 2-for-1's. Statistically, this is a smart strategy. Van Gundy noted, however, that in some cases, it can increase player selfishness and decrease player chemistry, since shooting early in the possession is often necessary to accomplish the 2-for-1. This is an example of where the statistics are not enough to fully complete the picture and that other observations, which are often intangible, are necessary.
As an aside, Van Gundy, while clearly not as influenced by analytics as the other panelists, made brilliant observations and is clearly a guy who is in favor of using all of the available and relevant information to make his team better. I could listen to him talk about sports all day long, even if he has no more of a mathematical mind than I do.
The next panel, Coaching Analytics, touched on many of the same themes.
The panelists were an interesting mix. Daryl Morey, who has never been a coach, was the moderator. Detroit Lions defensive end Lawrence Jackson and Grantland writer Bill Simmons, neither of whom have been coaches, were panelists. Van Gundy and Eric Mangini - both of whom were coaches at one time - rounded out the group.
Jackson actually did add some key insights from the player's perspective. Van Gundy offered more jokes. Mangini and Simmons argued back and forth about football strategy. Simmons also tried pushing some strange theory about how coaches should stick to a set number of minutes for their star players in order to increase their effectiveness. It was not well received by any of the other panelists.
This panel did not have many interesting revelations, but thanks to Van Gundy, it was still entertaining. He largely defended coaches, and suggested that GMs should be fired more often than they are, but he also made some great points about how coaches can unfairly get a "recycled" label due to unwinnable situations.
The last event of the day was a live B.S. Report between Simmons and analytics pioneer Bill James.
James is a very interesting thinker with a lot of fantastic ideas. Unfortunately, this B.S. Report didn't really cover many of these ideas, since Simmons' questions were largely things like, "Which advancement is your favorite?" and "Did you feel validated by the Mavericks winning the championship?"
James seems like a very humble person and he enjoys self-depricating humor as much as anyone. He noted more than once that he can't believe he got away with some of the material he used to publish, since now those ideas seem very antiquated. I definitely enjoyed hearing from such a revered figured in the analytics world.
The first day of the Conference lasted from around 7:30 AM to about 7 PM at night, and I was dead tired when I returned to my hotel room. I threw on the Sixers/Warriors game and fell asleep before it was over.
Day 2
I began Day Two with some more free breakfast pastries and had a fun chat with Larry Coon on some aspects of the new collective bargaining agreement.
When the panels began, I went to another research presentation on "The Power of Belief in Sports Performance Research".
I thought the presentation was going to be about religious belief and players like Tim Tebow who cite that belief as a reason for effective performance. It was actually a talk by sports scientist Peter Blanch on the placebo effect with regards to certain performance-enhancing substances, such as creatine. Clinical trials have shown that athletic performance can improve as much as a full percentage point if the athlete believes he has been given creatine, whether or not the drug has actually been administered. In fact, trials show that when the athlete is given creatine, but is not told about the administration of the drug, there is no performance increase.
Dr. Blanch performed these trials on individual sports, such as cycling and swimming, since it is easy to measure these sports with time intervals. Team sports, like basketball, are harder to measure in a straightforward way. Belief, which might otherwise be called confidence in this case, is an important piece of athletics and I was pleased to see that the results of this study confirmed this importance.
The next research presentation was on "Deconstructing the Rebound with Optical Tracking Data".
Using the data from the same camera system that was mentioned earlier in the Basketball Analytics panel, the study's team of authors looked at the ways in which rebounds are gathered in the NBA. This data allowed the team - led by Rajiv Maheswaran from the University of Southern California - to analyze the height at which rebounds are gathered, the distribution patterns of rebounds and the areas of the court where offensive or defensive players have an advantage. There is an impact between shot location and rebound rates, which in terms of practical applications, can assist players and coaches in floor spacing, transition basketball and a number of other techniques.
Like was said earlier, this type of data is fairly new, but is going to become much more wide spread. As more and more people gain access to this data, the world of analytics is going to become much more detailed than is currently possible.
The last research presentation of the day, "New Visual and Spatial Analytics for the NBA", also used this new data.
It tackled the question, "Who is the best shooter in the NBA?" Given that current metrics are insufficient, and that players like Tyson Chandler - who are not shooters at all - lead the league in FG%, author Kirk Goldsberry wanted to come up with a different method of analysis.
Using thousands of shot attempt data, he mapped out charts that showed where on the floor players were most effective. For instance, Leandro Barbosa is the best shooter in the NBA from the left corner three. Amar'e Stoudemire shoots the highest percentage from the right elbow. Goldsberry weighted shots in order to arrive at an ultimate statistic for shooting range, which concluded that Steve Nash and Ray Allen are the top two shooters in the NBA. Tyson Chandler finished near the bottom, according to his metric.
What was more interesting to me than the answer to Goldsberry's question was the potential of this data. By mapping out the areas where a player shoots the highest percentages, it is possible to see, for example, that Ray Allen is least effective from the right wing or that Kobe Bryant is incredibly effective in the paint area. Coaches and players could easily apply this kind of data into their offensive and defensive strategies.
When the research presentations finished, I went into a panel on Soccer Analytics.
I'm not a huge soccer fan, but Drew Carey was on the panel, and I wanted to hear him give his thoughts. Carey is actually the owner and chairman of the Seattle Sounders. His ideas on sports are common sense, and at the same time, visionary.
For example, Carey believes that fans should be able to vote on a team GM every four years. He says that many owners do not invest in winning ideas because those owners care about profit, not winning. He explained that, for his team, there is no idea that he will not listen to and no information that he does not think important, as long as it helps towards the objective of winning. He was one of the biggest hits of the Conference, both for his acceptance of analytics, and his willingness to use them in ways that make sense.
The rest of the panel was probably fantastic for soccer fans, but since I'm not as familiar with the lingo, I was not all that understanding. Sorry I cannot provide a better recap here, but I'm sure if you're really interested, you can look that up somewhere on the Interwebs.
The last panel of the Conference was on "Fanalytics".
Bill Simmons was the moderator and the panelists were Tim Brosnan of Major League Baseball, Nathan Hubbard of Ticketmaster, John Walsh of ESPN and Jonathan Kraft of the Kraft Group/New England Patriots.
The panel was mostly centered around ticketing and improving the fan experience. Hubbard described how Ticketmaster uses analytics for dynamic pricing models and how secondary markets, like StubHub, are capitalizing on ineffective pricing. Walsh occasionally added how ESPN is analyzing how their networks are performing and Jonathan Kraft discussed the issues with providing Wi-Fi in his stadium.
Simmons interjected his brand of humor into the panel, including repeated attempts to joke about Major League Baseball's YouTube policy that prevents clips from being shown.
Finally, to end the Conference, Simmons and Mark Cuban sat down for a live B.S. Report.
They discussed topics that included the new CBA, this year's Mavericks, winning the title and the future of analytics. Many Conference-goers were particularly disappointed when Cuban said that he thinks sports management majors are not valuable for many sports organizations and that the likelihood of finding a job in sports is very low.
This B.S. Report, and the other one with Bill James, are both available on Grantland.com and the Sloan Sports website.
Conclusion
Out of the many ideas discussed at this year's Conference, the two that resounded with me the most were: A) Analytics are great, but only if you can explain them in a way that makes them practical to apply to real world situations and B) Analytics is a forever-changing world, where old ideas are constantly replaced by fresh data, and teams are consistently searching for new ways to gain an advantage over competitors.
Advanced stats are not scary, as long as they can be put in the proper perspective as an important part of a much larger picture. Scouting will never go away and context is mandatory for any comprehensive analysis. But those who ignore advanced stats completely, whether they are fans or team personnel, will be stuck at a disadvantage when it comes to understanding the game.