The upcoming season the NFL will introduce a new, pilot, rule for the touchback. In particular, after every touchback the returning team will start the drive from its own 25 yard line instead of the 20 yard line. These 5 extra yards are part of an effort to reduce the incentives for kick returns that result in strong collisions between the players. There have been various opinions that have been expressed for this decision – both pro and against, but one that is of particular interest from an analytical point of view is the impact that it will have on the offense. The hypothesis put forward by the league committee is that these 5 yeards provide an advantage to the returning team and hence, they will not attempt a return. However, are these 5 extra yards really making any difference in the offense?
I used the play-by-play data from the past seven NFL seasons and I extracted the outcome of every single drive that started at the offense’s 20 (old touchback line), 25 (new touchback line), 30, 35 and 40 yard line. The outcome of every drive was either 7 points (i.e., touchdown – even though in all formality the touchdown is worth 6 points), or 3 points (i.e., field goal) or 0 points (i.e., punt). I then performed a simple statistical comparisson for the means of the points per drive scored for the drives that start at the 20 yard line and each one of the other options. The following table shows the difference in the corresponding means.
As we can see while there is an increase in the expected number of points by moving the touchback line to the 25 yard line, this increase is not statistically significant (in fact the p-value is greater than 0.1). A significant increase in the points per drive scored begins to appear for drives that start at the 35 yard line. So assuming that the percentage of kickoffs that result in a touchback does not change (e.g., a possible reason for this to change is trying more for “mortar” kicks, which I discuss later), I do not expect to see any significant difference in the points scored by the teams (purely due to the touchback rule).
Apart from the points scored many analysts are interested in the touchdowns scored, and in particular in the touchdown percentage (even though games are won based on which team scored more points rather than which team scored more touchdowns). The touchdown percentage for drives starting at yard line X is simply the fraction of drives that resulted in a touchdown. The following figure depicts the 95% confidence intervals for the touchdown percentage for the various starting points examined above.
We can see again here that the touchdown percentage is clearly higher at the 35 yard line and beyond. However, there is a small (but significant) increase when the drive starts from the 25 yard line. However, this might be an artifact of the very large dataset for the drives that start at the 20 yard line (9649 data points) that essentially leads to smaller confidence interval. Also the small sample for the 25 yard line (982 data points) might not be capturing accurately the touchdown percentage. Furthermore, if we focus on the last season alone the touchdown percentage for the 20 yard line was 17.9%, while for the drives that started at the 25 yard line it was 20.8%. However, the corresponding confidence intervals were [17%, 18.8%] and [17.2%, 24.4%], i.e., they overlap and hence, we can reject the hypothesis that these two success rates are the same! In general, even if there is a small impact in the touchdown percentage this is not necessarily significant and also it is not the crucial metric that wins games; the points per drive are and it seems that in that respect there will be very small impact (and certainly not statistically significant).
In the following I have also broken down the expected point benefits per team given their performance during the last 7 seasons. The y-axis corresponds to the difference between the expected points for a drive starting at the 20-yard line, minus the expected points for a drive starting at the 25-yard line. Hence, a negative value means that the team is going to benefit from the rule change. However, what we observe is that the p-value for the differential is for almost all the teams greater than the typical significance level of 0.05 (marked with the vertical dashed line) and hence, we cannot reject the hypothesis that there will be no impact on the offense of the teams. However, there are 3 teams (Chicago, Washingthon and Pittsburgh) that seem to be posed to enjoy a significant change in their scoring. Nevertheless, we need to be careful with the interpretation of these results. When examining significance at a pre-defined level there is always the chance that the rejection of the null was a false positive. In fact, if you repeat the same experiment several times you are almost guaranteed to see some experiments leading to rejection the of the null even if the null is true (the implications of this on the reproducability of published research are huge but I am not going to get into this topic here; after all we are talking about sports here). Therefore, if we repeat the experiment 32 times (as many as the NFL teams), we expect to get a significant result (at the 0.05 level) about 2 times! Therefore, the results for Chicago, Washington and Pittsburgh can be just flukes (false positives). Actually, the fact that Chicago and Washington appear to be significantly “penalized” if they start with 5 extra yards, it leads us to believe that these 2 data points are indeed false positive with high probability. Maybe only the Steelers will actually take advantage of the new touchback rule!! Or not…
So overall it does not seem that the new rule change will have any practical impact on the offense. It will definetely have a perception impact for the 5 free yards. Another dimension crucial for how things will play out in the field next year has to do with the fact that the kicking team will attempt more of these “mortar” kicks, trying to pin the ball within the 10 or 5 yard line. While this is certainly possible – and I expect this to happen – performing a good/successful mortar kick is not trivial and definitely requires a good skill set. Unfortunately, there are not good data (at least in my possesion) to assess the success rate of a mortar kick attempt. If anyone has such data I would be more than happy to learn more!
One thing I did though is to assume that kicking teams are indeed capable of pinning the ball between the 5 and 10 yard line of the offense and then calculate what is the expected difference in points scored by the returning team for these starting positions as compared to the 20-yard line. In this case the offense is expected to score 0.24 points less per drive (p-value < 0.00001)! Hence, it is not clear who will benefit from this rule change (seems to be the kicking team if anyone) but I really hope it will serve its purpose and reduce the kick returns that lead to strong collisions between players.
[Update week 14, 2016 season]
It has been 14 weeks with the new rule and I set to examine what has been the impact on the percentage of kickoffs that were returned as well as the starting field position that can affect the points scored (I am not looking at the difference in the points scored here simply because this might be an attribute of potent offenses and/or bad defenses – which we have somehow seen during the last few years). The following figure presents the fraction of the kickoffs that were returned. As we can see there is not any statistically significant difference between the fraction of kickoffs returned this year and the past year. OF course as we can also notice, there is a statistically significant increase in this fraction every two years, which might mean that we should monitor the impact of the rule next year as well. Furthremore, it is notable that the biggest impact on the touchbacks was the change at the kickoff rule in 2011, where the kick was taken from the own 35 yard line of the kicking team instead of the 30. The following figure also shows the average starting field position where as we can see again there is not any statistically significant change (the only change was again observed in 2011).
Overall the rule does not seem to have the impact that the league wanted to have but we should possibly wait to collect more evidence.