Evaluating lineups has traditionally been one of the tasks that “analytics” are commissioned with in basketball (and other sports with frequent substitutions). 5, 3 and 2 men lineups are typically analyzed, with the latter two offering a larger sample to work with. The metric that is typically utilized to evaluate the performance of a lineup is the offensive and defensive efficiency (rating), which is the points scored and allowed by the lineup respectively per 100 possessions. The unit of observation for calculating efficiency is a possession (rather than the number of minutes played). Therefore, if we want to calculate the efficiencies of a lineup, we can calculate the number of points scored/allowed while on the floor and also obtain an estimate for the number of (offensive and defensive) possessions. Both of these can be estimated from play-by-play data! However, you do not need to do this since the NBA is offering detailed statistics for th efficiencies of various lineups. For example, the following is the 5-men lineup for the Boston Celtics that has been on the court the most.
As you can see we can easily get the offensive and defensive rating (efficiency) for the lineup, as well as its net rating (simply the difference between the offensive and defensive rating). We also know the minutes played by the lineup and its pace. Using these two we can get an approximation for the number of possessions that the lineup played. The pace value is the number of possessions per 48 minutes for the lineup and therefore the specific lineup shown above played a total of (335/48)*96.5 = 673.5 possessions. When we want to compare two lineups we can check the ratings provided on the NBA’s website and simply see which lineup has higher (lower) offensive (defensive) rating. Right?
Well, not so fast! There are lineups like the one above that have played more than 600 possessions, while there are lineups that have played less than 10 possessions (e.g., Irving, Larkin, Morris, Rozier and Theis have played a whopping 3 possessions!). How confident are we that the lineup ratings we have obtained, are indeed their true ratings, especially for lineups that have played few possessions? We could calculate a probability that lineup A is better than lineup B by making an assumption for (or learning through data) the distribution of the actual performance of a lineup . For example, Wayne Winston in his book Mathletics indicates that when it comes to a lineup’s +/- rating, the actual performance of the lineup over 48 minutes is normally distributed with a mean equal to the lineups +/- rating and a standard deviation of points. Therefore, a lineup that has played a few only minutes will be associated with a high variance and we will be able to further calculate the probability that this lineup is better than another lineup of the team (for which we can also model its performance through a similar normal distribution). However, even if this probabilistic analysis were to be the most accurate representation of reality, when you are presenting your analysis to the coaching staff you should have a simple (yet concrete) message. Probabilistic comparisons of lineups are great but too cumbersome to digest, especially if you are not trained in probabilities and statistics. So is there a way that we can adjust the lineup ratings to account for the fact that different lineups have played many more or less possessions and hence, their true efficiency might be different than the one reported on the NBA’s website (or the one you calculated on your own from the play-by-play data)? Luckily the answer is yes!
In order to achieve our goal we will make use of the notion of Bayesian average. The idea behind the Bayesian average is that when we have a small number of observations (possessions in our case) for an object of interest (lineup in our case), the simple average can provide us with a distorted view. Consider the case of the lineup mentioned above with 3 (offensive) possessions observed. In this situation, all three possessions can easily end up in a made 3 point shot, which will lead to an offensive efficiency of 300 (points/100 posessions). However, it is also very possible that all of the 3 possessions end up with a missed shot, a turnover etc., leading to an offensive efficiency of 0! Simply put, when we have few observations it is very likely to obtain extreme values just by chance. So here is where the Bayesian approach comes into play. In the case of probability estimates, obtaining new evidence allows us to use Bayes theorem and update a prior belief we had for an event:
What does this have to do with our lineup ratings? Well we can adjust the ratings based on some prior belief we have for them. In our case this prior belief can be the team weighted average efficiency of a lineup (or the league weighted average efficiency of a lineups). In particular, considering the team weighted average, the Bayesian adjusted efficiency of lineup i is:
Essentially for every new lineup we begin with a prior belief that this is a (team/league) average lineup. Then every time we obtain a new observation (i.e., a new possession) we can update our rating for the lineup. It should be evident that as we accumulate enough observations for a lineup (i.e., is large compared to ), the impact of our prior belief gets smaller and smaller. For example, while the Bayesian adjusted rating of the lineup in the above figure is 111.6 (practically equal to its “raw” rating of 111.9), for a lineup with fewer observations there can be significant differences. For instance, the Celtics lineup Baynes, Brown, Ojeleye, Rozier and Smart have played 33 possessions with a raw offensive rating of 60.5. However, the Bayesian adjusted rating of this lineup is 78.1, since we have considered a prior based on 24 possessions on average for each Boston lineup and a 102.6 offensive efficiency. The following figure presents the raw and Bayesian-adjusted efficiency ratings for all the Celtics’ lineups. The size of each point corresponds to the number of possessions observed for every lineup. As we can see for lineups with many observations the two ratings have a good correlation. In fact, there is a negative correlation (-0.25, p-value < 0.001) between the absolute difference of the two ratings for a lineup and the number of possessions observed for the lineups, i.e., the fewer the observations the larger the adjustment.
Furthermore, the Spearman ranking correlation between the two ratings is 0.83, which means that while there is a good relationship between the two ratings, there are differences in the rankings that they provide.
As it should be evident one can do the same with defensive and net efficiency ratings. I hope we will start seeing these Bayesian adjustments in mainstream statistics.
- People that oppose the use of Bayesian approaches often cite that the choice of prior might not be objective. Ignoring the fact that this might indeed be a benefit of Bayesian approach (i.e., incorporating our subjective belief with objective evidence), the more evidence we accumulate the less impact our prior belief will have on the final result.
- In our discussion above we chose the team average lineup rating as our prior. However, one could have other choices. For instance, we could take the team average of lineups that only includes players from the lineup under consideration. We could further temporally weight the individual observations, giving more weight to recent games/possessions.
- The raw ratings are not adjusted for opponent (or home field). This can be done before applying Bayesian adjustment. In particular, with being the raw rating of lineup , we can adjust for opponent strength using the average (defensive) efficiency of the lineups faced by as follows: , where is the (league) average efficiency, and is the average efficiency of the lineups faced by lineup (for defensice efficiency adjustment one should use the reciprocal ratio).