Leveling the Playing Field

Transcription

1 Statistical Analysis of Competitiveness in Sporting Codes and the Effectiveness of Equalisation Policies Geoff Edwards Sasakawa Fellow Haas School of Business, University of California, Berkeley S545 Student Services Building #1900, Berkeley, CA Tel April 2003 This article develops new statistical measures of competitiveness in sporting competitions and uses them to shed light on the effectiveness of equalisation efforts in the Australian Football League (AFL). These new statistics confirm that the AFL s equalisation policy has had a dramatic effect on the game. Indeed, it appears that the AFL is more competitive today than at any other time in its 105 year history. Evidence of the AFL s positive experience with equalisation is instructive for sports administrators in other codes implementing or considering whether to introduce similar policies. The statistics introduced here can be adapted to measure and compare competitiveness over time in other sports. Unfortunately, recent turmoil in the NRL precludes a conclusion at this stage whether the NRL s salary cap policy has been effective in enhancing competitiveness in that code.

2

3 Statistical Analysis of Competitiveness in Sporting Codes and the Effectiveness of Equalisation Policies Recent salary cap scandals in both the Australian Football League (AFL) and the National Rugby League (NRL) have generated much discussion in the Australian media and amongst supporters. While the focus of discussion was rightly on those that had breached the rules, some commentators asked whether, given the possibility of extensive rorting by many (or even most) teams, equalisation policies that center on salary cap restraints are really effective. Others took the opportunity to challenge the merit of salary cap systems that restrict the wages of elite sporting talent and prevent successful and wealthy clubs from monopolizing the best players. While debates over equalisation policies like salary caps regularly arise in various sporting contexts around the globe, they are typically conducted in a vacuum of information as to whether equalisation policies are effective in their purported aims of leveling the playing field. Challenges to salary cap systems and other policies aimed at equalisation would be strengthened if there were evidence that in fact there is little or no public benefit from equalisation policies in terms of enhanced competitiveness. Conversely, the defence of equalisation policies would be enhanced if there were strong evidence that the policies have had the effect of leveling the playing

4 Edwards field, ensuring regular opportunities for supporters of all teams to enjoy on and off field success. The primary purpose of this article is to introduce new robust measures of the competitiveness of sporting codes. A secondary purpose is to demonstrate how such measures can assist assessments of the success or otherwise of policies like equalisation. To illustrate their value, the new measures are applied to the experiences of the AFL and the NRL. For the AFL, these measures provide compelling evidence that the AFL s explicit policy of equalisation, initiated in 1986 in conjunction with the beginning of the national competition the following year, has worked well to even the competition and provide all clubs with regular opportunities for success. Indeed, it appears that the AFL is today more competitive than it has been in its entire history. Similar analysis of the NRL is more equivocal. Discontinuities in team participation in a competition over time make it difficult to assess competitiveness and the NRL s experience with over-expansion of team franchises during the 1990s prevents a conclusion being reached at this stage on the effects of the NRL s equalisation attempts. Equalisation Equalisation policies exist in various forms in many sporting codes. In Australia, the AFL introduced its equalisation policy in 1986 in conjunction with the beginning of the expansion from a Victorian based league to a national competition the following year and the NRL has imposed salary caps since In the United States, teams in the National Football League have long been subject to revenue sharing rules, salary caps and draft systems, and in Major League Baseball teams are subject to revenue sharing and luxury 2

5 taxes on big spending clubs like the New York Yankees. By one estimate, the Yankees will transfer $50 million to their competitors in 2003 (Curry, 2002). The AFL sees its equalisation policy as fundamental to the long-term health of Australian football. Apart from providing for an equal sharing among the 16 clubs of revenue from broadcasting, corporate sponsorship and hospitality and other income, the policy imposes a National Draft system and Total Player Payments Caps on player salaries which enables an even spread of player talent to be distributed among all 16 clubs (Australian Football League, n.d.). Evening the spread of talent among the clubs should in theory provide all clubs with regular opportunities to reach the finals and contend for the premiership, further assisting the financial viability of all clubs (through memberships, club-level corporate sponsorships and gate receipts). The purpose and anticipated benefits of equalisation are stated by the AFL as follows: The AFL Commission views its policy as being overwhelmingly directed towards creating an environment in which all clubs can compete. It believes that clubs in a sporting competition are best served by a policy that assists all competitors to function effectively in the marketplace. The equalisation policy promotes, but does not guarantee, greater financial stability for individual clubs. It also promotes competitiveness and evenness on the field, allowing for uncertainty of outcomes in games and the opportunity for surprise results. This uncertainty maximises public interest, increasing the potential revenue generated by broadcasting rights, membership sales, gate receipts and corporate sponsorship. The maximisation of revenue from these sources allows admission prices to be kept as low as possible. (Australian Football League, n.d.) Has Equalisation Worked in the AFL? Regarding the evidence on the positives from equalization, the AFL states: all key performance indicators attendances, club memberships, broadcast audiences, media coverage, sales of licensed products and overall participation 3

6 Edwards rates have grown substantially since the competition developed nationally under the equalisation policy. (Australian Football League, n.d.) While these outcomes might be associated with equalisation of the competition, there may be no causal relationship these outcomes may simply reflect increased publicity and expansion into new markets since the commencement of the national competition. We would like to find and apply much more direct measures of competitiveness in a sporting code before being comfortable in concluding that the AFL s equalisation policy has worked. Few AFL administrators or supporters would doubt that the equalisation policy has had the desired effect in their code. The Brisbane Lions have come from some dismal performances in the early 1990s to win the last two premierships. Meanwhile, historically wealthy perennial finalists Carlton are struggling, despite attempts to rort the salary cap system. Anecdotal case studies aside, how can we best assess whether the competition is more intense now than before the equalisation policy came into effect? How can we robustly establish whether the most naturally disadvantaged teams have more regular opportunities to succeed today than before the equalisation policy took effect? Before we can answer these and similar questions we must address an important preliminary question: how can we best measure the competitiveness of a sporting code at various points in time, where a more competitive competition is defined in accord with the common goal of equalisation policies small variation in performances across teams and regular opportunities for all teams to achieve success? For the AFL, simple supportive statistics abound (all calculations in this paper are based on data from Lovett, 2002). For example, in the 13 years since 1990, 11 clubs have 4

7 played in the grand final, while only 8 played in the previous 20 grand finals. Again since 1990, eight clubs have won premierships, while only 5 managed to do so in the previous 20 years. In the ten years from 1993 to 2002, clubs churned from the bottom half of the ladder to the top half in successive years on 29 occasions. By contrast, in the ten years prior to the institution of the national competition ( ), churn occurred only 21 times, which was at least much better than the ten years from 1964 to 1973 perhaps the dimmest years for competition in the game when clubs moved out of the bottom half only 11 times. We could also look at the number of times a club churned from the bottom four all the way to the top four in successive years. This occurred 6 times over the last ten years, 4 times over the ten years to 1986 and just once in the ten years to Finally, we could ask how many times a club has remained in the top four in successive years. In the ten-year periods to 1973 and 1986 this occurred 25 and 23 times respectively. In the last ten years, clubs have managed to remain in the top four on only 12 occasions. While these statistics might seem to provide a compelling case that the AFL s equalisation policy has worked, they are arguably not sufficient. They are potentially distorted by the introduction to the competition of powerful new teams with large supporter bases like the West Coast Eagles and the Adelaide Crows and the expansion of the competition from 12 teams prior to 1987 to 16 teams after These statistics also cannot confirm whether or not some clubs have remained perennially poor performers after the equalisation policy commenced. More involved statistical analysis, described below, can provide a far richer picture of the competition. 5

8 Edwards New Measures of Competitiveness I approach the search for new, robust measures of competitiveness by considering competitiveness to be negatively correlated with variance in performance across the teams in a competition. The more widely dispersed are the performances of teams (measured in terms of their number of wins or their winning percentages) the less competitive I consider the competition to have been. Single Year Measures None of the measures discussed in the previous section are able to provide a sensible measure of the competitiveness of a sporting competition in a single year. I suggest that a good measure of how competitive a sporting code is in any single year is simply the standard deviation across the teams of the number of wins during the regular season. For the convenience of sporting fans and commentators who might prefer to deal with a more intuitive measure than standard deviations, an alternative, highly correlated measure of the variance in performance in any year would be the difference between the total number of regular season wins by, say, the top four teams together and the total number of wins by the bottom four. While looking at the top and bottom four teams suits the 16 team AFL code quite well, this alternative measure could obviously be adjusted to suit analysis of a sporting code with more or less teams. The virtue of these measures is that they give a sense of the disparity in the performance extremes of teams over a year. Where competition is tight and few teams are persistently poor or outstanding over the year, both the standard deviation in performances and the difference in the performances of the top and bottom four teams will be small. 6

9 To be able to use these measures to compare years when teams played more or less games, it is preferable to consider winning percentages, rather than raw numbers of wins. Again, a smaller standard deviation in these percentages, or the less the difference between the winning percentages of the top four and bottom four teams, the tighter is the competition during any year. For example, in 1986 the standard deviation of winning percentages across the 12 teams then in the AFL was 0.197, while in 2002, the standard deviation across the 16 teams in the competition that year was For the alternative statistic, in 1986 the top four teams in the AFL won 62 out of 88 games between them for a collective winning percentage of 0.75 while the bottom four teams won just 23 games for a winning percentage of Therefore, in 1986 the difference in winning percentages between the top and bottom four teams was In 2002 this difference was Multi-Year Measures While it is novel, and can be useful for some purposes, to have measures of competitiveness of a sporting code based on data from single years, there are good reasons to want to derive measures of competitiveness over longer periods. A truly competitive sporting code should offer all teams the ability to rise to the top at regular intervals and not have to languish near the bottom of the ladder in perpetuity. The single year measures described above do not inform us as to whether the bottom teams in any year are doing poorly perpetually or are having the chance to perform better in adjacent years. To analyze long-term competitiveness in a sporting code and to properly assess whether the AFL is more competitive since the equalisation policy took effect, we could instead consider five or ten consecutive years of performances by each team. 7

10 Edwards For example, over each five-year period from to there are 98 of these we can calculate the percentage of wins out of games played by each AFL team. To illustrate, for the period, Collingwood played 85 games and won 66 of them, for a five-year winning percentage of For each of these five-year periods we can then calculate the standard deviation in the five-year winning percentages across the teams. Alternatively, for each of the five-year periods, we can choose the top four performing teams and take an average of their winning percentages over that period, do the same for the bottom four teams, and then calculate the difference between the average fiveyear winning percentages of the top and bottom four performers. The smaller the standard deviation in five-year winning percentages across the teams, or the smaller the difference in five-year performances of the top and bottom four teams, the more competitive the game has been over those five years. If, over the five years, all teams spent some time in the top half of the ladder and some time in the bottom half, both measures would be quite small, whereas if there was little or no churn, they would likely be very large. Columns 1 and 3 of Table 1 report the results for the modern (post-war) era. [Insert Table 1 about here] On some occasions when developing these sorts of statistics care will be needed with the data as discontinuities in team participation can cause problems. In analyzing AFL data and comparing periods in the modern era before and after the equalisation policy was introduced, I encountered three discontinuities that tend to bias the statistics substantially for certain years. First, in 1987 the Brisbane Bears and West Coast Eagles were added to the competition making it truly national. Brisbane performed very poorly 8

11 until 1994, while West Coast very quickly settled near the top of the ladder, finishing fourth in 1988, third in 1990, first in 1991 and winning the premiership in 1992 and As a result of these extremes in performances, five-year standard deviations and top fourbottom four winning percentage difference statistics for the eras to are both biased downwards to the extent that they do not incorporate the performances of these new teams. This effect can be seen most vividly in column 3 of Table 1 where the top four-bottom four winning percentage difference statistics climbs rapidly after the era as the performances of Brisbane and West Coast are finally included. Second, after recording several years of poor performances Fitzroy exited the competition through a merger with Brisbane at the end of Uncorrected for this, the entries in columns 1 and 3 of Table 1 fall suddenly for the era as Fitzroy s poor record is dropped from the calculations. Third, Fremantle entered the competition in 1995 and has performed poorly in every year since, yet not until the era are Fremantle s results included in the statistics. Without some correction to the data, both these discontinuities in team participation in the AFL bias the results for the to eras downwards, making the competition seem more competitive than it really was. Fortunately in this case I have been able to make a reasonable and conservative correction for both the exit of Fitzroy and the entry of Fremantle by imagining that Fremantle took over Fitzroy s role as cellar dweller in 1997, and ascribing to Fremantle all of Fitzroy s history up to that year (including, conservatively, Fitzroy s bottom of the ladder finishes in 1995 and 1996). The second and fourth columns in Table 1 report the corrected numbers (the shaded areas show where these numbers differ from columns 1 and 3). 9

12 Edwards While two other teams, Adelaide and Port Adelaide, entered in 1991 and 1997 respectively, no further corrections are needed as both these teams performed around the average of all clubs each year for at least their first five years, thereby introducing no bias to either the standard deviation or top four-bottom four difference statistics. Even after making a conservative correction for Fitzroy and Fremantle, the evidence on the effectiveness of the equalisation policy is striking. Assuming that it took at least five years after its introduction for the equalisation policy to have its full impact (a reasonable assumption given the initial inequalities in the AFL in terms of talent distribution and player salaries in 1986) we can compare standard deviations in five-year performances or five-year winning percentage differences between the best four and worst four teams prior to 1987 and after, say, For each and every five-year period from to , the five-year standard deviations have been less than in any fiveyear period prior to and including Similarly, for each five-year period from to , the top four-bottom four difference has been less than in any five-year period from to This is despite the fact that after 1986 the AFL has grown in the number of participating teams from 12 to 14 in 1987, to 15 in 1991 and to 16 in All else equal, and assuming performances follow a somewhat normal distribution, a larger number of teams involved in a competition should be expected to lead to greater variance in performance between those teams, not less. Remarkably, taking into account data distortions caused by the introduction of Hawthorn, North Melbourne and Footscray in the mid-1920s, both measures suggest that the five years to 2001 were the most competitive in the entire history of the competition (a standard deviation of just and a top four-bottom four difference of just 0.248). 10

13 Using the standard deviation measure, the AFL s worst five-year period in terms of competitiveness was the era. This is a somewhat surprising result given that this era produced the only premiership for St Kilda, the worst performing team in the game s history. It was, however, a particularly dismal period for two teams Fitzroy and Footscray. It is possible to use the data derived from these sorts of competitiveness measures in statistical tests of the effect of exogenous changes such as the introduction of an equalisation policy. For the AFL, I performed an ordinary least squares regression of postwar five-year performance (winning percentage) standard deviations on a dummy variable coded 1 for each five-year period from to (assuming again that the AFL s equalization policy did not take full effect for at least five years after its introduction in 1987) and coded 0 for all periods up to (dropping data for the transitional to eras). As is to be expected in time series analysis with the dependent variable constructed in the way that it is, there is positive autocorrelation in the residuals (Durbin-Watson statistic = 0.495). Specifically, a regression of the residuals on lagged residuals reveals first order auto-regressive (AR(1)) autocorrelation. An ordinary least squares analysis with an AR(1) pattern in the residuals will report inefficient and biased standard errors. To correct for this, I performed a feasible generalized least squares regression using the Cochrane and Orcutt (1949) method. Table 2 reports the results which suggest that the equalisation policy has had the effect of reducing the standard deviation in five-year performances by a very substantial 6 percentage points. In other words, around one-third of the variation in five-year performances has been removed. The hypothesis that the policy has improved competitiveness is strongly supported. Not only do we find a marked increase in 11

14 Edwards competitiveness after the equalisation policy was introduced, but this is very unlikely to be the result of random fluctuations. The results are robust to variations in the year chosen for the dummy to switch from 0 to 1, and variations in the length of the pre period (for example: beginning with the era instead of ; or ending with the era instead of ). [Insert Table 2 about here] We can also look at standard deviations in ten-year performances or, alternatively, the differences in the average ten-year winning percentages of the top four and bottom four teams over each ten-year period from to Table 3 presents the uncorrected and corrected statistics for the modern era in similar format to Table 1. [Insert Table 3 about here] Using statistics based on ten years of performances probably provides even better measures of competitiveness in the AFL. Ten year periods are long enough as to leave no doubt that teams in an equalized competition would have the opportunity to go from the bottom of the ladder to premiership contenders (and maybe even back again). The results are again impressive. After again correcting for the exit of Fitzroy and the entry of Fremantle I find on both measures that the last ten years have been more competitive than any other ten consecutive years in the AFL s history. Statistical tests report very similar results to the test reported in Table 2 using five-year differences. The hypothesis that the AFL s equalization policy has improved competitiveness is again supported. 12

15 Addendum: The NRL The NRL, by contrast, has so far enjoyed less success in enhancing the competitiveness of their game. I constructed the same five and ten year measures to assess competitiveness in the NRL since its inception in Table 4 reports uncorrected five and ten year measures for the post-war period. I find that although the NRL introduced a salary cap system in 1988, the five years to 1997 were among the least competitive in the NRL s history. This was no doubt the result of over-expansion of the league at that time, giving rise to the demand for the short lived Super-League of elite teams that year. [Insert Table 4 about here.] There is a bright side for NRL administrators today. Following significant consolidation of the number of teams in 1999 and 2000, the NRL appears to have returned to the level of competitiveness that it enjoyed in Admittedly, there is downward bias in the five year numbers in Table 3 for the most recent five-year periods due to the elimination (by merger or otherwise) of three poorly performing teams in 1999 and Gold Coast left the league in 1999, and in 2000, Balmain merged with Wests and South Sydney was forced out of the competition. While South Sydney was reinstated to the competition in 2002 its poor performances over recent seasons are not included in the measures for the eras to 2000, 2001 and Nonetheless, with the exception of South Sydney, those teams remaining in the NRL today appear to have been as competitive as a group over the past five years as were the teams in the league in the five years to Using simple measures of competitiveness such as numbers of different premiers, numbers of drawn games, the number of points scored and conceded by the top and bottom teams 13

16 Edwards and the amount of bottom half to top half churn, some have argued that the NRL s salary cap policy has failed to improve competitiveness (Leigh and Wolfers, 2002). While my measures also suggest no improvement in competitiveness since 1988, my conclusion on the NRL s equalisation efforts is more circumspect. Over-expansion in the number of teams during the 1990s substantially reduced competitiveness in the NRL and likely masked any positive effects of the salary cap system. More time is needed with a stable number of teams before the effectiveness of the NRL s salary cap system can properly be assessed. Conclusion This article has developed new ways of measuring the competitiveness of sporting codes that are more robust than reliance on simple statistics such as numbers of different premiers or finalists. Conceptually, these measures ask whether the performances of teams in a competition in a single year or averaged over a number of years are bunched closely or widely dispersed. Particularly valuable is that the statistical methods introduced here are able to track teams over periods of time longer than a single year and long enough to allow a reasonable time for any team to rise from the bottom if the competition is truly competitive. While the application of the method in this article was to the AFL, with an addendum regarding the NRL, administrators and commentators in many other sporting codes can readily adapt the method to analyze the degree of competition in their sport. For example, the choice of the number of consecutive years to consider will depend on the sporting code being analyzed and how long a period it is reasonable to expect a team to be able to rise from the bottom to the top of the ladder given a level playing field. I consider that, given the size of player lists and the time needed to groom a draft choice into a star 14

17 player and to turn a bottom of the ladder team into a premiership contender, any period less than five years would be too short for the AFL. This might not be the case for other codes such as basketball, where player lists are much smaller. This article has also provided strong evidence that the AFL s equalisation policy has worked remarkably well. By these measures, the AFL is now more competitive than it has ever been. In my opinion, the jury is out on the effectiveness of the NRL s attempts at equalisation due to the disruption in that code during the 1990s, but in time, under stable conditions and with a commitment to enforcement, I expect the NRL to enjoy similar success. References Australian Football League (n.d.). A Policy of Equalisation. Retrieved April 27, 2003, from Cochrane, D. & Orcutt, G. (1949). Application of Least-Squares Regression to Relationships Containing Autocorrelated Error Terms. Journal of the American Statistical Association, 44, Curry, J. (2002, August 31). Don t Worry, Yanks Will Keep Spending. New York Times, p. D2. Leigh, A., & Wolfers, J. (2002, August 26). Numbers Crunch Salary Cap s Logic. Sydney Morning Herald, p.13. Lovett, M. (Ed.). (2002). AFL 2002: The Official Statistical History of the AFL. Melbourne: AFL Publishing. 15

18 Edwards Table 1: AFL Competitiveness Five -Year Statistics Years Standard Deviations Top Four-Bottom Four Differences Uncorrected Corrected Uncorrected Corrected (1) (2) (3) (4)

19 Table 2: Cochrane-Orcutt Regression of Post-War Five-Year Winning Percentage Standard Deviations ( to and to ) on an Equalisation Dummy (zero for to and one for to ) Coefficient Standard Error t-statistic Probability> t Equalisation Constant Number of observations: 43 F(1,41): R-squared: Rho: DW statistic (original): DW statistic (transformed):

20 Edwards Table 3: AFL Competitiveness Ten-Year Statistics Years Standard Deviations Top Four-Bottom Four Differences Uncorrected Corrected Uncorrected Corrected (1) (2) (3) (4)

21 Table 4: NRL Competitiveness Five and Ten Years to Five-Year Statistics Ten-Year Statistics Standard Deviations Top Four-Bottom Four Differences Standard Deviations Top Four-Bottom Four Differences (1) (2) (3) (4)