With the NHL season right around the corner, this marks my first ever post on hockey. Clearly hockey is not my sport, but this shows how math and analytics can cast a net over all sports. One of the constant questions in sports is how much of a team’s or player’s success is due to skill and how much is due to luck. Face-offs is a perfect setting in hockey to ask (and answer) this question.
To answer this question I will follow a very elegant approach that originated in the sabermetrics community and was popularized in the book “The Success Equation“, where Mauboussin ordered the four major sports based on the level of luck that they involve. This approach is based on a very simple mathematical equation, which states that for two independent random variables X and Y, the variance of their sum is equal to the sum of their variances, i.e., var(X+Y) = var(X)+var(Y). In our case the independent variables X and Y correspond respectively to the skill and luck associated with the observed face-off win percentages of the players; the sum of these two variables includes everything observed.
I downloaded data on the face-offs won and lost for each player for the 2016-17 NHL season from hockey-reference.com. Half of the players took less than 3 face-offs, while only 30% of the players took more than 40 face-offs. In order, to avoid skewing the results from the high variability from players taking very few face-offs, I considered only the players with at least 40 face-offs, which leaves us with a sample of 255 skaters. The following table shows the top and bottom 5 players with respect to face-off won %.
I first calculated the variance of the observed face-off win% (FOW%) for the players in the data, which is equal to var(observed) = 38.27 (this is the var(X+Y) in the equation above). To calculate the variance expected in a completely random (with respect to face-offs) NHL, we model each face-off a player faced as a Bernoulli trial with probability of success equal to 0.5 (i.e., a coin flip). For each player, we flip the coin as many times as the face-offs they had in the season and calculate the simulated (purely random) face-off win%. For example, Matt Duchene took 1,098 face-offs, from which he won 687 of them. To obtain an estimate for Duchene’s FOW% in a purely random world we flip a coin 1,098 times and we keep track of how many times the coin lands on the “face-off win” for Duchene. Our coin flip series provides a 49.7% FOW% for Duchene in a world of pure luck. Repeating this process for all the players allows us to calculate the FOW% for each player in a completely random league. We can then calculate the corresponding variance over all the players in this random world, which gives us var(luck) = 12.65. Simply put, the contribution of luck in face-off success (or lack thereof) is only about 33% of the observed variance of the players’ face-off win %. In other words, 67% of face-off success can be attributed on the player’s skill!