In order to explore our hypothesis we used play-by-play data from the 2014-2016 NFL seasons. In particular, for every game in our dataset we calculated for each team the utilization of passing as the fraction of passing plays over the total number of its offensive snaps. We further calculated the average expected points added for the passing plays for each team and each game. We have adjusted the expected points for strength of defense. The following figure presents the results (binned for better visualization), where to reiterate passing efficiency is the expected points added per passing play. As we can see there is a declining trend for the passing efficiency as we increase its utilization.
The correlation coefficient is . These results, while they account for quality of passing defense, they do not account for the quality of the rushing game as well as the overall passing ability of the team that can impact the results. Therefore, we build a regression model where the independent variable is the average expected points added per passing play (adjusted for defense) within a game, while the dependent variables include:
The table above presents our results where we can see that the utilization is still negatively correlated with the expected points added per passing play. The interaction term also shows that this correlation depends on the rushing ability of the offense. In particular, the effect of passing utilization on its efficiency is , namely, if the offense runs the ball better the negative relationship between and is less strong. In particular, with (the maximum observed value in our dataset), the corresponding coefficient is -0.42 — compared with a coefficient of -1.33 for the minimum value of in our dataset, i.e., -0.73.
So how much should a team run? Obviously the question depends on many factors but it should be evident that calling passing plays all the time is going to have diminishing returns. While the passing efficiency might still be greater than that of rushing even when , this does not mean that it is the best the team can do. What we a team is interested is maximizing the efficiency on a per-play basis regardless of the type of play, i.e.,
The following figure presents the passing utilization that maximizes the above equation for different values of and . As one might have expected for teams with better passing rating a higher utilization is recommended for fixed rushing ability, while better running game reduces the optimal passing utilization. Note that a rushing EPA higher than 0.3 per play per game is rather unrealistic, and so is having . For the average rushing EPA (marked with the vertical line), the optimal fraction of passing plays is 0.3, 0.47 and 0.63, for a bad, average and great passing offense respectively.
I’d like to note here that the results of the analysis are not and should not be treated as causal, that is, running more does not necessarily cause passing to be more efficient. It might as well be the case that teams that are trailing in the score turn to more passing and this bias the results. In some of my past analysis I have explored the possibility of similar reverse causality and there are not strong indicators for it. Furthermore, we have treated rushing as being constant regardless of its utilization. While rushing skill curves are weaker as compared to passing the final results will quantitatively (not qualitatively) change. However, it should be evident that there is a clear interaction between passing efficiency and utilization that makes rushing still a piece of the puzzle in the NFL.
]]>
We have also included the following interaction terms:
We have used play-by-play data to build this model from the 2014-15 and 2015-16 NBA seasons and following is the reliability curve we obtained.
As we can see the obtained reliability curve is practically the y=x curve, which means that our in-game win probabilities are well-calibrated.
We can then use this win-probability model to calculate the win probability added (WPA) for a lineup during every stint it played and then calculate its win probability aded efficiency (i.e., win probability added per 100 possessions) as: . After obtaining these raw win probability efficiencies we can use the Bayesian average to adjust the ratings in such a way that accounts for the different number of observations (i.e., possessions) we have for each team: , where is the (league) average total win probability added and the average number of observations for all the lineups. We have essentially considered that our prior belief for lineups we have never seen is that of a league average lineup. The more observations we obtain for lineup l, they will eventually outweight our prior belief and when .
But what is the reason for doing all of this? Does this win probability added indeed provide us with a different view of the lineups? To examine that we trained a win probability model using the play-by-play data from the 2014-15 NBA season and calculated the WPA efficiency and the net efficiency rankings for every lineup in the 2015-16 season. We then calculated the ranking correlation between the two rankings obtained from the different metrics. The correlation coefficient is 0.71 and there are two observatins to be made:
The two metrics agree to a large extend with regards to which lineups add value to a team. This is important since the net efficiency rating is in general considered a good lineup evaluation metric.
You might be wondering “Then if this WPA efficiency for 5-men lineup is so great then why isn’t used more?”. The major drawback of this approach is the fact that it is dependent on the in-game win probability model used, and therefore, different models might provide different ratings. In contrast, the efficiency ratings are purely based on points scored and allowed, which is unambiguous. Of course, this is not something that hasn’t happened before in sports analytics. For example, (and full disclaimer I am not a big baseball fan and hence, I am not very well aware of the analytics literature) in baseball there are many different versions of wins above replacement! What is important is the high level idea; how one develops it further and implements it is secondary – at least at this stage – but certainly is critical for the success of the “metric”.
]]>The ratings we got after the end of the regular season are:
Using these ratings we get the following projections (they will be updated after every game):
As we can see CSKA opens as the bit favorite followed by Real Madrid and the reign champion Fener. Many fans (at least in Greece) are looking forward to a Greek final, but the chances of this happening are only about 1.5%.
After the first two games of the series two teams have made the break, Real and Zalgiris. On the other hand CSKA and Fener have made a strong case for being present at the final 4 once more, while the chance of a Greek final is still around 1.5%.
Final Four
We are only days away from the final four and here are our updated projections for the semifinals and the finals:
Big Final
CSKA once more defied the probabilities and … did not win the title, so we are headed for a very balanced final between Fener and Real Madrid. Will Obradovic win his 10th title, or Real win her 10th title? Numbers do not help since they do not give any favorite (in fact Fener is a slight favorite with 50.5% win probability if this is considered a favorite). We are in for a great final!
]]>These ratings are obtained through a regression model (a simplified version of it can be found here) and as we can see Monaco and Tenerife are the top-2 teams with Riesen Ludwigsburg following at the third place. Using these ratings we simulate the playoffs 10,000 times to make our predictions for the final ultimate winner. For the final 4 pairings, we draw them randomly in every simulation (since after all the same will happen when the 4 finalists are known!). Following are our initial projections:
It is not a suprise that Monaco and Tenerife are the favorites, while the cluster of Nymburk, Strasbourg, Neptunas and AEK that are matching up together at the top-16 and top-8 is the most balanced one in terms of probability of making it to the playoffs!
With the first leg of the top-16 round coming up following is the distribution of the expected point differential for each of these games:
Each colored area gives the win probability for the corresponding team (or else positive values for point differential correspond to home team win, while negative values correspond to visiting team win). As we see the two games that are expected to be the most balanced are PAOK-Pinar and Bayreuth-Basiktas.
With the first leg of the top-16 round over, following are the updates in our predictions. As you will see there are some pretty big swings in favor of Banvit, Nymburk and Riesen Ludwigsburg.
Taking the results of the top-16 leg 1 into account following are the projections for each of the upcoming rounds:
Tenerife is still at the top of the race, while Ludwigsburg has emerged as the second favorite, while Monaco’s poor outing (compared to the expectations) at leg 1 dropped their chances for the trophy to 10%.
Quarter finals
The first round of this year’s playoffs involved two major upsets (based on the results of the first leg and the team ratings). The first BCL champions, and the favorite for the back-to-back championship, Tenerife got disqualified after a sensetional performance by Murcia! Before leg 2, Murcia had a mere 1% chance of advancing! At the same time AEK with the (practically) buzzer beater from Punter covered in Nymburk the 10 point deficit from leg 1 in Athens and qualified in the top-8 after having just about 12% chance of doing so! Quarter finals are here and following are the point differential predictions for leg 1 and the updated probabilities for the rest of the tournament. AS Monaco is back at the top of its game and our ratings. However, don’t count out anyone just yet! An exciting round of quarter finals is ahead!
After the first leg AS Monaco remains the favorite for the title but there have been some favorites emerging for participating in the final four. After leg 1 of the quarter finals here are the updated odds.
Final 4
The final 4 is here and after the pairings’ draw here are the projections:
AS Monaco is the big favorite but AEK – playing in front of its fans – has good chances to advance to the final and challenge the French team.
Final
Our predictions for the semi-finals were correct and the final is going to be between AEK and AS Monaco. Monaco is the heavy favorite with a 74% win probability but never count out AEK in front of its fans (and I hope I am wrong — just to reveal my personal preferences if you have not figured them out yet :)).
As you all might know by now AEK defied the odds and whon its 3rd European title. Closing out these predictions here is the in-game win-probability from the big final:
Looking forward to a new season!
]]>As you can see we can easily get the offensive and defensive rating (efficiency) for the lineup, as well as its net rating (simply the difference between the offensive and defensive rating). We also know the minutes played by the lineup and its pace. Using these two we can get an approximation for the number of possessions that the lineup played. The pace value is the number of possessions per 48 minutes for the lineup and therefore the specific lineup shown above played a total of (335/48)*96.5 = 673.5 possessions. When we want to compare two lineups we can check the ratings provided on the NBA’s website and simply see which lineup has higher (lower) offensive (defensive) rating. Right?
Well, not so fast! There are lineups like the one above that have played more than 600 possessions, while there are lineups that have played less than 10 possessions (e.g., Irving, Larkin, Morris, Rozier and Theis have played a whopping 3 possessions!). How confident are we that the lineup ratings we have obtained, are indeed their true ratings, especially for lineups that have played few possessions? We could calculate a probability that lineup A is better than lineup B by making an assumption for (or learning through data) the distribution of the actual performance of a lineup . For example, Wayne Winston in his book Mathletics indicates that when it comes to a lineup’s +/- rating, the actual performance of the lineup over 48 minutes is normally distributed with a mean equal to the lineups +/- rating and a standard deviation of points. Therefore, a lineup that has played a few only minutes will be associated with a high variance and we will be able to further calculate the probability that this lineup is better than another lineup of the team (for which we can also model its performance through a similar normal distribution). However, even if this probabilistic analysis were to be the most accurate representation of reality, when you are presenting your analysis to the coaching staff you should have a simple (yet concrete) message. Probabilistic comparisons of lineups are great but too cumbersome to digest, especially if you are not trained in probabilities and statistics. So is there a way that we can adjust the lineup ratings to account for the fact that different lineups have played many more or less possessions and hence, their true efficiency might be different than the one reported on the NBA’s website (or the one you calculated on your own from the play-by-play data)? Luckily the answer is yes!
In order to achieve our goal we will make use of the notion of Bayesian average. The idea behind the Bayesian average is that when we have a small number of observations (possessions in our case) for an object of interest (lineup in our case), the simple average can provide us with a distorted view. Consider the case of the lineup mentioned above with 3 (offensive) possessions observed. In this situation, all three possessions can easily end up in a made 3 point shot, which will lead to an offensive efficiency of 300 (points/100 posessions). However, it is also very possible that all of the 3 possessions end up with a missed shot, a turnover etc., leading to an offensive efficiency of 0! Simply put, when we have few observations it is very likely to obtain extreme values just by chance. So here is where the Bayesian approach comes into play. In the case of probability estimates, obtaining new evidence allows us to use Bayes theorem and update a prior belief we had for an event:
What does this have to do with our lineup ratings? Well we can adjust the ratings based on some prior belief we have for them. In our case this prior belief can be the team weighted average efficiency of a lineup (or the league weighted average efficiency of a lineups). In particular, considering the team weighted average, the Bayesian adjusted efficiency of lineup i is:
Essentially for every new lineup we begin with a prior belief that this is a (team/league) average lineup. Then every time we obtain a new observation (i.e., a new possession) we can update our rating for the lineup. It should be evident that as we accumulate enough observations for a lineup (i.e., is large compared to ), the impact of our prior belief gets smaller and smaller. For example, while the Bayesian adjusted rating of the lineup in the above figure is 111.6 (practically equal to its “raw” rating of 111.9), for a lineup with fewer observations there can be significant differences. For instance, the Celtics lineup Baynes, Brown, Ojeleye, Rozier and Smart have played 33 possessions with a raw offensive rating of 60.5. However, the Bayesian adjusted rating of this lineup is 78.1, since we have considered a prior based on 24 possessions on average for each Boston lineup and a 102.6 offensive efficiency. The following figure presents the raw and Bayesian-adjusted efficiency ratings for all the Celtics’ lineups. The size of each point corresponds to the number of possessions observed for every lineup. As we can see for lineups with many observations the two ratings have a good correlation. In fact, there is a negative correlation (-0.25, p-value < 0.001) between the absolute difference of the two ratings for a lineup and the number of possessions observed for the lineups, i.e., the fewer the observations the larger the adjustment.
Furthermore, the Spearman ranking correlation between the two ratings is 0.83, which means that while there is a good relationship between the two ratings, there are differences in the rankings that they provide.
As it should be evident one can do the same with defensive and net efficiency ratings. I hope we will start seeing these Bayesian adjustments in mainstream statistics.
The initial projections are given in the following:
Patriots and Steelers (closely followed by the Chiefs) are the AFC favorites, while the Vikings are the NFC top favorite (despite the Eagles being the top-seed). The most possible matchup at the moment is Patriots-Vikings, which appeared in almost 10% of the simulations, while the probability of having a Pennsylvania Super Bowl (Steelers-Eagles) is around 7.5%.
1/7/2018
Wild card round is over, and Bills, Panthers, Chiefs and Rams are eliminated from contention. The latest projections are as follows:
1/16/2018
Divisional round is over and 4 teams remain in contention:
1/22/2018
Super bowl matchup is set and the Patriots are opening as the favorite:
]]>You can observe a few differences (e.g., AEK is taking many more shots from the restricted area, while PAOK is taking more shots from the paint), but overall there is little that you can say with regards to “how much” different the shooting tendencies of teams (players) are. One could possibly provide information about the exact locations of shots and obtain some type of shot density and compare. This is certainly possible but cumbersome. On the other hand, the fact that the shooting charts are mainly similar, it might mean that there are underline patterns that all of the teams (players) follow to a different extend. If we could identify these patterns (some type of shooting dictionary) we could then describe a team/player through these patterns.
One way to identify similar latent patterns in data that can be represented through a matrix S is matrix factorization. With matrix factorization one tries to express the original data matrix as a product of two (or more) factor matrices (e.g., WH). These factors include latent patterns of the original data. There are various techniques to perform this task, but for our case we will focus on Non-negative Matrix Factorization. In our case our data can be represented through a matrix S whose columns represent locations on the court and the columns represent teams (or players). However, what is court location? One could use the actual x,y coordinates by overlaying a grid over the court and obtaining the counts of shots in every grid cell. However, this would give fairly sparse and noisy results in our case where we have a fairly small number of games for each team/player. Another approach is to use as location the 12 court zones (Restricted area, paint, midrange slot, etc.) and then the element (i,j) of the matrix S represents the number of shots taken by team (player) i from court zone j.
Using S as our data (shot) matrix, Non-negative Matrix Factorization (NMF) aims at identifying matrices W and H such that:
where dist(S,WH) represents a distance metric between the original data matrix S and the product of the factor matrices WH. Furthermore, we constrain the factor matrices to be non-negative. This allows for easier interpretation of the results since the data in the original matrix are also non-negative. With regards to the distance metric, we have used the Frobenius norm of the matrices.
The next step is to decide on the number of patterns we want to find. This essentially corresponds to the number of rows for matrix H (which is equal to the number of columns of matrix W). Choosing the number of patterns is not trivial and is essentially very similar to the problem of choosing the number of clusters in a clustering problem. One approach to choose the number of pattern is by examining how good the approximation of S is with the factor matrices product. However, the approximation is monotonically increasing as we increase the number of patterns and hence there is a tradeoff between finding trivial patterns and approximation quality. This is the same as the problem of bias and variance; obtaining a large number of patterns essentially provides us with an overfitted model where practically every pattern represents a team/player. For our purposes we have used a number of patterns k = 7, since it provides a good tradeoff between approximation and interpretability (non-overfitting). Figure 7 presents the quality of approximation for the case of player matrix as a function of the number of patterns k (the results for the teams’ matrix are similar).
Every pattern (i.e., a row of matrix H) is essentially a 12-dimensional vector, each element of which correspond to one of the court zones. The value of the element further captures the strength of the corresponding court zone in the pattern. For example, one of the patterns identified from the players’ matrix is the following:
Simply put this pattern includes shots mainly from the restricted area and a few from the paint. Once these patterns are identified, the other factor matrix W can be used to obtain the coordinates of a player/team with regards to the basis of the patterns identified. This essentially allows us to express a player/team as a linear combination of these shooting latent patterns.
The following figure presents the 7 player patterns identified and the corresponding coefficients for some of the players (the coefficients for all the players can be found here). The coefficients are proportional to the number of shots a player takes. For example, Manny Harris appears to be getting the majority of his shots from pattern 1 (midrange shots), while Abromaitis is a corner 3 (pattern 2) and restricted area (pattern 3) shooter.
Each one of these patterns is also associated with an expected point per shot. In order to identify this we need:
Then the expected point per shots for each pattern r is:
The following table shows the expected points per pattern, where as we can see patterns 3 and 6 are the most efficient ones as one might have expected, since they include shots from the restricted area and the (left) corner.
We performed the same analysis for the 32 team and the patterns we identified are presented in the following together with some of the coefficients for select teams (all the coefficients can be found here).
Furthermore, the points per pattern for the team patterns are:
The above present a fairly simple application of matrix factorization in basketball. It provides a better understanding of the offensive/shooting tendencies of teams. Even more insights can be obtained if two matrices are analyzed, namely, one for made shots and one for missed shots. In this case, we can really identify potential inefficiencies of upcoming opponents. For instance, we can identify specific shooting patterns that a team is not successful at and force them to take those.
With regards to the factorization itself, the KL divergence usually works better as compared to the Frobenius norm that we have used in our analysis. Furthermore, one can overlay a grid (e.g., 1×1 meters) and use the grid cells as the locations. This will provide a very long matrix, but the NMF will essentially reduce the dimensionality. However, in this case there can be more noise as compared to when using the court zones and in this case it is better to apply NMF over an intensity surface instead of the raw counts.
ACK: I would like to thank Basketball Champions League for providing me access to the data.
]]>Using data for all the shots taken from the 32 teams in BCL we calculate the expected point per shot for each team and for each are of the court. The court is divided in 13 areas as shown in the figure below.
Given the field goal percentage of a team from each area of the court, we can easily compute the points per shot that the team is expected to add whenever taking a shot from that area. For example, if the field goal percentage of our team from the paint is 55%, the expected points from a shot taken from the pain are 0.55*2 = 1.1. The following figure shows the expected points per shot from each area on the court when considering all the shots taken from all the teams in BCL.
Similar to the NBA, the most efficient shots are the ones taken from the restricted area (under the basket), followed by all the three point shots. The most inneficient shots are the ones from midrange. Now, among the three point shots, the most efficient ones are the corner threes! An idea that has been thrown around for the higher efficiency of corner threes has been the shorter distance to the hoop. In particular, in the NBA, the corner threes are at a 22 feet (6.7m), while above the break the distance is 23.75 feet (7.24m). This is a fairly large difference. In contrast, the three point line in FIBA competitions varies much less, from 22.15 feet (6.24m) above the break to 21.65 feet (6.60m) at the corners. If distance was a big factor for the increased efficiency of the corner threes in the NBA, this difference in the efficiency should not be as much pronounced in FIBA competitions. This setting is exactly the setting of a natural experiment. The fact that in FIBA competitions the corner threes are still more efficient as compared to threes above the break, allow us to reject the hypothesis that the shorter distance at the corners is responsible for the majority of the efficiency difference. On the other hand, in both the NBA and FIBA contests, the fraction of corner threes that are assisted is much higher compared to the threes above the break. The actual fractions differ significantly between the NBA and FIBA competitions , but this is mainly an artifact of how assists are counted in the different competitions. The following table presents the fraction of shots from each three-point area in BCL.
Area | Fraction of assisted shots |
Top of the key | 20% |
Left wing | 27% |
Right wing | 27% |
Left corner | 39% |
Right corner | 35% |
Overeall, above the break threes are assisted at a rate 25.4%, while corner threes are assisted at a rate 37.3%. A two-proportion z-test further allows us to reject a null hypothesis that the two rates are equal (p-value < 0.001). Given that assisted shots tend to be of higher quality (e.g., they tend to be open more frequent that not), the fact that corner threes are more efficient than above the break threes is not surprising. Of course, the question is why corner threes are more assisted, but this is a topic deserving its own in depth study and analysis, which we are currently performing.
Using the notion of expected points per shot we can have a quantitative way to evaluate the efficiency of a team. An efficient team will make shot choices that lead to larger expected points per shot. The above figure presents the league average expecte points per shot, which means that a team with similar efficiency as the league average, should attempt to take more shots under the basket and corner threes. However, this is not always necessarily true for all teams. Teams might not have the right personel for creating corner three opportunities. Similarly it seems inadvisible for a team to take may midrange shots, but there are several players (many of whom are future hall-of-famers – Dirk Nowitzki, Chris Paul etc.) who have made a career based on their efficiency from the midrange. Therefore, in order to estimate the expected points per shot for each team, we cannot use the league average numbers for the expected points per shot, but rather quantify these variables for each team individually. In particular, with f(z,t) being the FG% from area z for team t, n(z,t) being the number of shots team t has taken from area z and p(z) being the number of points awarded from zone z, the expected points xPTS[t] per shot for team t are:
The following figure presents the z-score for the expected points per shot for all 32 teams. As we can see the top-3 teams in terms of offensive efficiency (as captured by expected points per shot) are all from group A (having the first 3 spots in the group) !! Pinar Karsiyaka, EWE Baskets Oldenburgh and AS Monaco!
However offense is only half of the game. Defense can impact the efficiency of a team by either reducing the FG% from the different court zones, or by forcing the offense to take shots from court zones with low FG%. The following figure exhibits the z-score of the defensive expected points per shot for each team, that is the expected points of the shots allowed from the defense. Therefore the lower the better, i.e., the team allows less points per shot than an average team.
As we can see AS Monaco and Tenerife have the most efficient defense until now in the competition allowing more than two standard deviations less points compared to an average team. On the contrary EWE Basketbs Oldenburg allows more than 2 standard deviations more points compared to an average team.
Note that the above are raw numbers, i.e., they do not adjust for who a team faces. For example, scoring 1 point per shot against EWE Baskets Oldenburg is not as good as scoring 1 point per shot against AS Monaco. In order to adjust the xPTS (both offensive and defensive) for each team we solve the following optimization problem:
where x is a vector with the xPTS ratings for every team (offensive and defensive respectively), h is the home edge with regards to xPTS and m is the league average xPTS. Every game i will essentially provide us with two data points for the above optimization objective. Solving the above optimization problem we obtain a home edge h= 0.014 points/shot, and a league average xPTS of m = 1.038. The following table provides the results.
Note that for the defensive ratings a negative value is better (i.e., the team allows less points per shot than average). One could also get ratings for specific court zones. However, given the small number of games to begin with and the even sparser data with regards to shots in specific areas for specific games, we might not be able to provide a robust solution to the above optimization.
You can explore the (currently only offensive) efficiency of the BCL teams in the interactive app here.
Acknowledgments: I would like to thank Basketball Champions League for providing me with access to the data.
]]>Let us assume that we are evaluating a new kicker in practice. We ask the kicker to take 20 50-yard field goals. He makes 16 of them. What can we say about his success rate at 50-yard field goals?
In order to get a good estimate of the probability distribution of the kicker’s success rate σ, we will make use of the Bayes theorem:
In the above equation π(σ) is the prior probability distribution for the success rate of the kicker, while π(σ|data) is the posterior distribution we estimate taking into consideration the data we observed (in our case the 16/20 FGs). f(data|σ) is the likelihood of observing the data given the success rate σ. Finally, f(data) is the total probability of observing the data:
What is the prior distribution that we can use? We can simply look into all NFL kickers and use the distribution of their collective success rate in 50-yard FGs. The average success rate is around 70%. There are some kickers that are exceptional and way above average in 50-yard FGs (e.g., Justin Tucker), while the majority of the kickers are around average. Therefore, one could use a Beta distribution for the prior π(σ), with an average of 0.7. Given that the average of a Beta distribution is given by α/(α+β), where α and β are the distribution parameters, we choose α=5 and β=2. This gives us the following prior distribution:
The next element we have to calculate is the likelihood function f(data|σ). Simply put we need to calculate the likelihood of observing 16 successful kicks and 4 misses, given the success rate σ. This is nothing more than the binomial distribution:
Finally, the total probability of observing the data is:
Combining all of these we obtain the following posterior probability function for the success rate σ of our kicker at 50-yard FGs:
As we can see there is smaller uncertainty associated with this posterior probability since we now have some data to support this probability. For example, but calculating the area under the posterior distribution between σ = 0.8 and σ = 1, we find that the kicker has a 42% chance of being an 80% or better kicker at 50-yard field goals. A generic NFL kicker (i.e., one that is drawn from the prior distribution) has only a 34% probability of being an 80% or better kicker at 50-yard field goals. With more data we can further update our beliefs. For example, if he we give the kicker another 30 attempts and he makes 25 of them, our updated posterior distribution for the success rate is:
Using this posterior distribution we now can say that our kicker has a 58% probability of being an 80% or better kicker at 50-yard FGs.
It should be evident that Bayes theorem is a very powerful tool that allows us to make probabilistic inferences, updated for every new data point we obtain. Brian Burke of ESPN has used a similar analysis to find that Garoppolo has an edge over rookie QB’s. In particular, there is a 64% chance that Garoppolo is better than a generic-first round QB.
The code associated with the above analysis can be found here.
]]>
To answer this question I will follow a very elegant approach that originated in the sabermetrics community and was popularized in the book “The Success Equation“, where Mauboussin ordered the four major sports based on the level of luck that they involve. This approach is based on a very simple mathematical equation, which states that for two independent random variables X and Y, the variance of their sum is equal to the sum of their variances, i.e., var(X+Y) = var(X)+var(Y). In our case the independent variables X and Y correspond respectively to the skill and luck associated with the observed face-off win percentages of the players; the sum of these two variables includes everything observed.
I downloaded data on the face-offs won and lost for each player for the 2016-17 NHL season from hockey-reference.com. Half of the players took less than 3 face-offs, while only 30% of the players took more than 40 face-offs. In order, to avoid skewing the results from the high variability from players taking very few face-offs, I considered only the players with at least 40 face-offs, which leaves us with a sample of 255 skaters. The following table shows the top and bottom 5 players with respect to face-off won %.
I first calculated the variance of the observed face-off win% (FOW%) for the players in the data, which is equal to var(observed) = 38.27 (this is the var(X+Y) in the equation above). To calculate the variance expected in a completely random (with respect to face-offs) NHL, we model each face-off a player faced as a Bernoulli trial with probability of success equal to 0.5 (i.e., a coin flip). For each player, we flip the coin as many times as the face-offs they had in the season and calculate the simulated (purely random) face-off win%. For example, Matt Duchene took 1,098 face-offs, from which he won 687 of them. To obtain an estimate for Duchene’s FOW% in a purely random world we flip a coin 1,098 times and we keep track of how many times the coin lands on the “face-off win” for Duchene. Our coin flip series provides a 49.7% FOW% for Duchene in a world of pure luck. Repeating this process for all the players allows us to calculate the FOW% for each player in a completely random league. We can then calculate the corresponding variance over all the players in this random world, which gives us var(luck) = 12.65. Simply put, the contribution of luck in face-off success (or lack thereof) is only about 33% of the observed variance of the players’ face-off win %. In other words, 67% of face-off success can be attributed on the player’s skill!
]]>