Should bonus points be included in the Six Nations Championship? Niven Winchester Joint Program on the Science and Policy of Global Change Massachusetts Institute of Technology 77 Massachusetts Avenue, E19-439f Cambridge MA 02139-4307 USA Tel: +1 617-253-6958 E-mail: niven@mit.edu We construct strength measures based on wins and potential bonuses and use a prediction model to determine bonuses that are significantly correlated with team strength. The results reveal that a bonus for losing by seven or fewer points is strongly statistical significant, bonuses for scoring three or more net tries are moderately significant, and bonuses for scoring four or more tries are insignificant. These findings indicate that including carefully-crafted bonuses in Six Nations league tables will more accurately rank teams according to their ability than rewarding only wins and draws. If bonuses are included to reward strong teams, the Six Nations Championship should adopt the bonus system used in French domestic competitions rather than that used in the Rugby World Cup and elsewhere. Keywords: Non-linear least squares, Six Nations Championship, Sports ranking systems.
I. Introduction Bonus points are currently awarded in most major rugby competitions. The Six Nations Championship the world s oldest international rugby competition is a notable exception. This may change in the near future, as Six Nations Rugby Ltd is currently considering the introduction of bonus points (Rees, 2013). Two bonus point systems are currently used in rugby. In most competitions, including the Rugby World Cup, four points are awarded for a win, two points for a draw, one point for losing by seven or fewer points and one point for scoring four or more tries. In competitions organized by the French Professional League (Ligue Nationale de Rugby, LNR), the try bonus is awarded for scoring three or more net tries (tries scored minus tries conceded) and competition points awarded for other outcomes are the same as above. The Six Nations Championship is contested annually by national sides from England, France, Ireland, Italy and Scotland. In each year, each team plays the other teams once. The team that records the most wins (with a draw counting as half a win) is declared the champion, with several tie-breaking procedures used to separate teams with an equal number of wins. A controversial issue is that, under a bonus points system, a team could beat all other teams (and achieve a grand slam ) and finish second in a competition. For example, this could occur if a team won all of its five games without earning any bonuses (for a total of 20 competition points) and another team won four games and picked up a one narrow loss bonus and five try bonuses (for a total of 22 competition points). To insure that a team that completes a grand slam is always the champion, one suggestion is to award a grand slam bonus worth three competition points. In this note, we evaluate whether or not including bonus points in the Six Nations Championship more accurately ranks teams from strongest to weakest than only rewarding wins and draws, and evaluate the suitability of alternative bonus systems. This note has three further 2
sections. The next section outlines our modeling framework and results. Section III discusses how bonus points would change league standings in previous years. The final section concludes. II. Modeling Framework and Results Building on the methodology developed by Winchester (2008), we determine bonus values that are best at rewarding strong teams using a prediction model. For a match between home team i and away team j in round r of year y, we relate the winning margin of the home team ( ) to home advantage and an average of net competition points earned per game by each team in previous games. Our analysis employs the following equation: { (1) ( ) } where translates differences in competition points to differences in predicted net scores;,, and are the number of points awarded for, respectively, winning, scoring more than a certain number of tries in a game, losing by a narrow margin, and achieving a grand slam;,, are, respectively, the net average number of wins, try bonuses and loss bonuses accumulated per game by the home team; is a binary variable equal to one if a team completed a grand slam; 1 and is an error term. For each team, the average number of wins, try bonuses and loss bonuses earned per game used in the prediction model is calculated using a time-varying weighted average of outcomes from previous and current seasons. In the first game of the season, averages are based on all games from the previous season. For each subsequent game played, the weight on outcomes from the previous season is reduced by one-third and that for outcomes from the 1 The net grand slam variable is divided by five in Equation 1, as the grand slam bonus is awarded based on results from five games (in most seasons). 3
current season is increased by one-third until the weight on outcomes from the current season is equal to one. For each team s fourth and (after 1999) fifth match of the season, strength measures used in the prediction model are based on results from matches played in the current season. Parameters to be estimated in Equation 1 include,,,, and. As multiplying competition points by any positive scalar will not change team rankings, we normalize competition points with respect to. That is, we set = 1 and express values for, and relative to the number of competition points awarded for a win. We estimate Equation 1 using non-linear least squares for all matches played in the Six Nations from 1997 to 2012. We do not consider games played prior to 1997 so that our analysis only includes data from games played since rugby became a professional sport (in 1996). Regression results are reported in Table 1. Specification (a) estimates Equation 1 without any bonus points. The p-value for is less than 1%, which indicates that wins from previous matches are a statistically significant determinant of match outcomes. The point estimate for suggests that a team that always wins will beat a team that always loses by 21 points, plus or minus home advantage of around five points. A bonus for losing by seven or fewer points is added to the prediction model in specification (b). The loss bonus is a significant determinant at a 1% significance level and also improves the goodness of fit in the model, as measured by the adjusted and the root mean square error. The point estimate for the loss bonus indicates that a narrow loss bonus should be equal to 84% of the points awarded for a win. Although this value is larger than most stakeholders would find acceptable, it is consistent with estimates for the Super Rugby competition (see Winchester, 2008) and the National Football League (see Winchester and Stefani, 2013). 4
The next two specifications include a try bonus and a bonus for completing a grand slam. The try bonus in specification (c) is awarded for scoring four or more tries, and three or more net tries are required to earn this bonus in specification (d). In both specifications, the grand slam bonus is not statistically significant. The four-try bonus is also not significantly different from zero at conventional significance levels. The net try bonus is significant at a 10% significance level, but not at a 5% significance level. These results match the findings from Winchester (2008), who finds that the four-try bonus is not significantly correlated with team strength but a bonus based on net tries is. Overall, our results provide strong evidence that awarding a bonus for losing by seven or fewer points improves the ability of Six Nations league tables to rank teams from strongest to weakest. Support for a try bonus is less concrete. The bonus for scoring four or more tries is not significantly correlated with team strength, but there is moderate evidence that awarding a bonus for scoring three or more net tries can improve the accuracy of league standings. There is no evidence to suggest that awarding a bonus for completing a grand slam (in addition to competition points awarded for winning each game) improves the accuracy of league standings. III. Alternative League Standings We examine the impact of bonus points on Six Nations standings for the 1993 (the first year the value of a try was increased from four to five points in the Six Nations Championship) to 2012 competitions. We consider standard and French bonus systems. Ranking changes under at least one of the bonus systems compared to the current system are listed in Table 2. Under the current system, in each season, teams are ranked by the number of wins, then the number of draws, and then by season net game points (points scored minus points conceded). Under the two 5
bonus systems, teams are ranked by the number of competition points earned, and then by season net points. If they had been employed, both bonus systems would have resulted in different champions than under the existing system in 2002 and 2007. In 2002, France completed a grand slam, but is ranked second under both bonus systems. Despite recording one less win than France, England would have claimed the Championship in that year if bonuses were included as, relative to France, they earned three more try bonuses (under both systems) and a narrow loss bonus, and had a superior points differential. However, if the number of wins was used to separate teams with the same number of competition points, as in Super Rugby since 2011, France would have been champions under both bonus point systems. On the other 10 occasions a team completed a grand slam between 1993 and 2012, that team would have also been the champion under both bonus systems. In 2007, France and Ireland recorded the same number of wins but France claimed the title by scoring more net points during the competition. Under both bonus systems, however, Ireland would have finished ahead of France, as they picked up one more narrow-loss bonus and at least as many try bonuses. Other ranking changes due to bonus points also involve a reordering of teams with the same number of wins. In the years considered, France is demoted four times under a bonus point system and is only promoted once. IV. Conclusions Using a prediction model for the Six Nations Championship, we related team strength to wins, draws, narrow losses, the number of times certain try thresholds were exceeded, and a grand slam bonus. We found that team strength was significantly correlated with a bonus for losing by 6
seven or fewer points, moderately correlated with a bonus for scoring three or more net tries, but not significantly correlated with a bonus for scoring four or more tries or completing a grand slam. These findings suggest that including carefully designed bonuses in the Six Nations Championship more accurately ranks teams according to their ability than only rewarding wins. There is strong evidence that awarding a bonus for losing by seven or fewer points improves the accuracy of league standing, but mixed support for a try bonus. If a try bonus is to be included, the findings here and elsewhere suggest that this bonus should follow the French system and be awarded for scoring three or more net tries. The research also showed that awarding a bonus for completing a grand slam (on top of awarding points for winning each game) does not improve the accuracy of league standings. To decrease (but not eliminate) the possibility of a grand slam winner finishing second in the competition, teams with the same number of competition points could be ranked according to the number of wins. References Rees, P. (2013) Six Nations countries may kick tradition into touch for bonus points, The Observer, January 5. Winchester, N. (2008) Shifting the goal posts : Optimizing the allocation of competition points for sporting contests, Journal of Quantitative Analysis in Sports, 4, Article 1. Winchester, N. and R. Stefani (2013) An innovative approach to National Football League standings using bonus points, Applied Economics, 45, 123-134. 7
(a) No Bonuses Table 1. Regression results (b) Loss bonus only (c) Absolute try bonus (d) Net try bonus Loss bonus - 7 or fewer points 7 or fewer points 7 or fewer points Try bonus - - 4 or more tries 3 or more net tries 4.498 *** (1.120) 4.618 *** (1.068) 4.642 *** (1.059) 4.627 *** (1.060) 21.552 *** (2.493) 25.676 *** (2.611) 21.961 *** (3.402) 20.904 *** (3.537) Loss bonus ( ) 0.839 *** (0.142) 0.915 *** (0.184) 0.909 *** (0.185) Try bonus ( ) 0.377 (0.244) 0.462 * (0.275) Slam bonus ( -0.084 (0.587) 0.033 (0.623) Adjusted R 2 0.252 0.319 0.324 0.327 Root mean square error 16.856 16.086 16.026 15.994 Note: The net score of the home team ( ) is the dependent variable. ***, **, * denotes statistical significance at, respectively, 1%, 5 and 10% significance levels. Robust standard errors are reported in parentheses. The sample size is 225 for all specifications. 8
Ranking under alternative bonus systems Table 2. Rankings changes under bonus point systems Bonus points Competition points Team Year None Standard French Played Win Draw Net pts Try Net try Loss Standard French Scotland 1993 2 3 3 4 2 0 10 0 0 0 8 8 England 1993 4 2 2 4 2 0 0 0 1 1 9 10 Ireland 1999 4 5 4 4 1 0-24 0 0 1 5 5 France 1999 5 4 5 4 1 0-25 1 0 1 6 5 Scotland 2000 5 6 5 5 1 0-50 0 0 0 4 4 Italy 2000 6 5 6 5 1 0-122 1 0 0 5 4 France 2002 1 2 2 5 5 0 81 1 1 0 21 21 England 2002 2 1 1 5 4 0 131 4 4 1 21 21 France 2007 1 2 2 5 4 0 69 2 2 0 18 18 Ireland 2007 2 1 1 5 4 0 65 2 3 1 19 20 Scotland 2008 5 6 6 5 1 0-54 0 0 1 5 5 Italy 2008 6 5 5 5 1 0-57 0 0 2 6 6 France 2009 3 4 4 5 3 0 23 1 1 0 13 13 Wales 2009 4 3 3 5 3 0 19 1 1 2 15 15 France 2011 2 2 3 5 3 0 26 1 1 1 14 14 Ireland 2011 3 3 2 5 3 0 12 0 1 2 14 15 Note: The standard system awards four points for a win, two points for a draw, one point for losing by seven or fewer points and one point for scoring four or more tries. The French system awards four points for a win, two points for a draw, one point for losing by seven or fewer points and one point for scoring three or more net tries. 9