Last September, just four months before the. Smashing the racket IN DETAIL

Transcription

1 Smashing the racket Are some professional tennis players really taking backhanders to throw matches? Tim Paulden explains how spotting anomalous movements in the betting markets can help shine a light into the murky world of tennis match-fixing Last September, just four months before the phenomenon of tennis match-fixing was catapulted into the public gaze by the BBC (bbc.in/1v04tna) and Buzzfeed News (bzfd.it/1p9k9lw), we delivered a presentation at the 2015 RSS Conference entitled Smashing the racket: Detecting match-fixing in tennis via in-play betting irregularities. This branch of research represented our first foray into the fledgling field of forensic sports analytics the application of statistical modelling to help identify and eliminate corruption within the sports sector. To set the scene, spend a few moments pondering the following question: How might an unscrupulous tennis player go about making money through match-fixing? When we pose this challenge to various audiences, they most often suggest that the player should secretly agree to throw (that is, deliberately lose) a particular match, and have a complicit third party place hefty bets on his opponent winning. When the match is over and these crooked bets have paid off, the spoils can be then be divided up between the fixer and bettor. Despite its simplicity, this strategy is understood to be one of the most likely modes of tennis match-fixing Dr Tim Paulden is innovation and development manager at ATASS Sports particularly at the lower echelons of the sport, where the size of the betting market for a single first-round match may dwarf the prize money for an entire tournament. However, the perpetrators of such a fix might inadvertently leave behind an important fragment of evidence: the perturbations to the market odds caused by their bets. To see how and why such perturbations might arise, we must first understand the basic dynamics of a pre-match tennis betting market (where pre-match means that the bets are placed in advance of the match starting). In simple terms, whenever the market experiences a large influx of money in support of player A rather than player B, the odds on player A will tend to shorten (that is, pay off less generously) and those on player B will lengthen (pay off more generously) just as a share price will rise if the demand to buy exceeds the available supply. Under normal circumstances (see Betting odds, page 17), such an odds movement will be relatively short-lived in particular, a further influx of money on player A is unlikely, as the odds have become less attractive. However, what if these large bets actually came from our shady third-party bettor, who knows that player B intends to throw the match? In this Main image: Maxim Zarya/iStock/Thinkstock 16 SIGNIFICANCE June The Royal Statistical Society

2 Betting odds: How do they evolve during a normal match? What are betting odds, and what do they tell us about each player s chance of winning a match? Picture the final of the 2016 Australian Open, in which Andy Murray came up against Novak Djoković. Immediately before the match started, one of the major online betting exchanges was quoting the odds on Murray winning as 5.0, which simply means that a 10 bet on Murray would return 50 if he won the match, for a gain of 40. In other words, decimal odds of 5.0 are equivalent to 4 to 1 in old bookmaker s notation. Meanwhile, the initial odds on Djoković winning were 1.25 so a successful 10 bet on him would return 12.50, for a gain of The market s implied probability for each player is simply the reciprocal of the odds (that is, 1 divided by the odds). For instance, in the above example, the market s implied probability for Murray at the start of the match was 1/5 = 0.2, while that for Djoković was 1/1.25 = 0.8. Intuitively speaking, if the market believes a successful bet on Murray should pay off more than a successful bet on Djoković, then logically Murray must be much less likely to win the match. If the odds on a player shorten, this simply means that the odds have moved closer to 1.0 in other words, the market deems the player more likely to win. Conversely, if the odds on a player lengthen, the odds have moved further from 1.0, and the player is deemed less likely to win. When a match begins, the betting market is said to go in-play, and the players odds will fluctuate continuously as the scoreline evolves. (See Scoring in tennis, below, for a quick overview of tennis scoring terminology.) For example, in the Australian Open final mentioned above, Djoković dominated the play in the opening set and his odds gradually shortened, reaching 1.09 at the moment he won the first set an implied probability of Meanwhile, Murray s odds lengthened to 12.5 an implied probability of In the second set, Murray fought back valiantly, with his odds shortening at one point to 7.5, but ultimately Djoković won the set by 7 games to 5. With a commanding lead of two sets, Djoković s odds now stood at 1.02 an implied probability of He went on to win the third set and with it, the tournament. instance, the bettor is already virtually certain of the final result, and will be willing to continue betting on player A, even at odds that would normally be considered unfavourable thus distorting the market further still. Therefore, if we discover there are players who regularly lose matches in which there has been a substantial lengthening in their pre-match odds, then the circumstances surrounding these matches may warrant further scrutiny. In fact, Buzzfeed News conducted an analysis of exactly this kind in January 2016 as one component of the BBC Buzzfeed exposé (bit.ly/23o6bvc), building on earlier work in this domain by Rodenberg and Feustel. 1 After examining prematch betting activity from more than matches, and running significance tests, Buzzfeed concluded that there were 15 players who regularly lost matches in which there had been lopsided betting activity before the match began a judgement made based on the difference between the opening odds and closing odds. However, there are some fundamental weaknesses in considering solely the gap between the opening and closing pre-match odds as a metric of odds movement. Firstly, these two isolated numbers provide only a very simplistic picture of how the odds are changing. Secondly, it is often difficult to reliably pin down their values particularly in the case of the opening odds. Indeed, a detailed critique published by the well-respected DW on Sport blog in late January 2016 cast serious doubts on the inclusion of one of the 15 names in Buzzfeed s list, and highlighted several instances in which the odds data used by Buzzfeed appeared to be either erroneous or misleading. These concerns and the dangers of trial by algorithm were echoed by an article published in the Guardian newspaper the following week (bit.ly/1sjejl2). The shortcomings of focusing exclusively on the opening and closing odds naturally raise the more complex question of whether anomalous odds movements can be detected inplay that is, while the match is actually taking place. If our ultimate goal is to identify potential instances of match-fixing, If our ultimate goal is to identify potential instances of match-fixing, it is critically important to scrutinise in play betting it is critically important to scrutinise in-play betting, since this form of betting is now offered by all major exchanges, with volumes often exceeding pre-match volumes by a factor of 10 or more making it highly likely that the majority of suspicious third-party betting would occur in-play. Furthermore, in-play odds data does not suffer from the data integrity issues highlighted in the previous paragraph if we have access to a detailed stream of odds data captured throughout the match, Scoring in tennis Tennis has one of the most esoteric scoring systems of any mainstream sport, with a match being organised hierarchically into sets, games, and points. Loosely speaking, each bout of action (which can be won by one player or the other) is known as a point, and a player wins the current game by accumulating sufficiently more points than their opponent. Similarly, a player wins the current set by accumulating sufficiently more games than their opponent. The first player to win two sets (or three sets in the case of men s Grand Slam tournaments) is declared the winner of the match. A comprehensive description of the tennis scoring system can be found on Wikipedia at bit.ly/1vxmjew. June 2016 significancemagazine.com 17

3 then the presence of one inaccurate data point will have a negligible impact on the overall results, rather than entirely scuppering the analysis. However, moving over to an in-play scenario presents some serious complications. How are we going to spot when the market odds shift anomalously if they are already fluctuating continuously due to the changing match scoreline? Clearly, we are going to need some additional statistical machinery. Setting the scene In order to meaningfully classify odds movements as anomalous, we need to get a handle on how each player s win probability should have evolved given the sequence of points in the match, accounting for the fact that one player may be naturally stronger than the other. In our initial study, we chose to focus solely on professional men s tennis (ATP), spanning all levels down to the Challenger Tour. To make our work accessible to non-statisticians in the tennis world, we also made a conscious decision to keep our methods as parsimonious as possible, while still retaining real world applicability. Arguably, the simplest possible model of tennis that incorporates a notion of player superiority is to assume that the two players A and B, say have a fixed probability of winning each point (p and 1 p respectively, for some probability p). As any tennis fan will tell you, however, this abstraction is unrealistic because there is an intrinsic advantage to being the server in a game (that is, the player who hits the initial shot to begin each point), particularly at the professional level. So, it is natural to suppose instead that each player has a certain probability of winning a point when serving, which is fixed throughout the match say, p A for player A, for player B. To meaningfully classify odds movements as anomalous, we need to get a handle on how the win probabilities should have evolved during a match This simple assumption turns out to be surprisingly powerful. If we know the values of p A for a match, we can use this information to calculate the probability that each player will win a game when serving, by figuring out the probability of each possible sequence of points arising. From these game probabilities, we can then determine the probability of each player winning a set, by considering each possible sequence of games, and finally, we can use these set probabilities to find the probability of each player winning The O Malley tennis formulae The O Malley tennis formulae 2,3 allow us to calculate the probability that a particular player wins a game, a set, or the whole match, given only the two players probabilities of winning a point on serve (which are assumed to be fixed throughout the match). For instance, the O Malley formula for the probability of the server winning a game (starting at 0 0) is p Gp ( ) = p 15 4p 1 2p 1 ( p) where p is the server s point-win probability. As an illustration, if p = 0.63, then G(p) is approximately 0.795; if p = 0.66, then G(p) is approximately O Malley s set-win and match-win formulae are much more complicated to write down than the game-win formula above, but they follow the same underlying principle: namely, the only inputs required for the calculation are the two players point-win probabilities. Full details of the formulae are available in the references highlighted above. the match overall. In other words, the players respective point-win probabilities on serve uniquely determine their match-win probabilities. What is the best way to go about computing these match-win probabilities? Rather than build up the calculation completely from scratch, the tidiest approach is to apply the celebrated tennis formulae published in 2008 by O Malley (see box above). 2 By feeding the values of p A into the relevant O Malley formulae, we can determine not only the players win probabilities at the start of the match, but their win probabilities in every possible match scenario. To illustrate this, consider a best-of-three-sets match in which p A = 0.66 = 0.63 (that is, player A has a probability of 0.66 of winning each point he serves, while for player B this probability is 0.63). In this instance, the O Malley formulae tell us that the two players chances of winning the match initially stand at m A = and m B = respectively in other words, player A s modest serving edge over player B means he is nearly twice as likely to win the match overall. Suppose we now fast-forward to a point a little later in the same match, when player B is leading 1 0 in sets, and the score in the second set is 1 1. By applying the O Malley formulae to this mid-match scenario, we find that the match-win probabilities have now become m A = and m B = that is, player B is now the strong favourite to win. After experimenting further with the properties of the model, it became evident to us that the match-win probabilities we were calculating depended far more strongly on the gap between p A than on their specific values a discovery that we found was consistent with previous work by Klaassen and Magnus. 4 For instance, if we were to modify the example in the previous paragraph so that p A = 0.56 = 0.53 (that is, we dial down each player s on-serve advantage, 18 SIGNIFICANCE June 2016

4 while still maintaining the same gap), then the initial matchwin probabilities turn out to be m A = and m B = almost exactly the same as the values we obtained for p A = 0.66 = Motivated by this property, we made one last simplification to reduce the number of variables from two to one, which was to specify that p A must be spaced symmetrically about in other words, p A = = 0.645, for some dominance parameter that encodes how much better one player is than the other. Here, the value of represents an average on-serve pointwin probability for men s tennis, as suggested originally by Klaassen and Magnus. 5 Breaking new ground We now turn to the million-dollar question: how well can this simple model capture the evolution of a real betting market? Figure 1 provides a graphical representation of a typical match from July 2014, with the evolution of the market s win probability for player A at each point of the match shown in red. The nine grey trajectories show how the model win probability for player A varies during the course of the match for different values of the dominance parameter (ranging from 0.08 at the bottom to 0.08 at the top, in steps of 0.02). In this instance, we see the market trajectory in red corresponds to a dominance parameter somewhere between 0 and 0.02 (as indicated by the shaded region in Figure 1). In order to obtain a more precise answer for the best, we can minimise the discrepancy score, which we define as the mean absolute difference between the market probability and model probability over the course of the If a match has a small discrepancy score, we may conclude that the market evolved rationally if not, the match may be worthy of further investigation match (or, roughly speaking, the area of the region between the red and grey lines). If a match has a small discrepancy score, then we may conclude that the market probabilities evolved rationally if not, then the match may be worthy of further investigation. In this example, the minimum discrepancy score of was obtained for = 0.01 (corresponding to p A = = 0.635); the trajectory for this model is shown by the blue line in Figure 1. Using the framework above, we then investigated an archive of around 5000 ATP matches spanning the period from 2013 to mid-2015, all of which had successfully passed a set of data integrity checks. For each match, we determined the best value of simply by minimising the discrepancy score over a discrete grid (on which ranged from 0.12 to 0.12 in steps of ), and recorded the optimal along with its FIGURE 1 The evolution of a typical match. The red plot represents the market probability for player A at each point of the match. Each grey trajectory represents the corresponding model probability at each point of the match for different dominance parameters, ranging from = 0.08 (bottom line) to = 0.08 (top line) in steps of The blue trajectory represents the model probability at each point of the match for = 0.01, which is the value minimising the discrepancy score June 2016 significancemagazine.com 19

5 corresponding score. (We later developed more intelligent heuristics or statistical rules of thumb to predict the position of the likely optimum, which allowed us to generate the same results as the naïve grid search in a small fraction of the time.) The results obtained were extremely encouraging. As shown in the histogram in Figure 2, the discrepancy score was found to be small for almost all matches examined, with 80% of matches having a score less than 0.03, and 99% of matches having a score less than (To put these values into context, note that the close correspondence between the red and blue lines in Figure 1 represents a discrepancy score of , which is slightly to the right of the histogram s peak. In other words, this quality of fit was typical across the matches examined.) These results provide powerful empirical evidence that even an extremely parsimonious model can provide a reasonable approximation to the evolution of a betting market across the vast majority of tennis matches. Importantly, since our approach considers whether the market probabilities are consistent with any value of the dominance parameter, it is In one match, the eventual winner was deemed significantly more likely to win when trailing by a set than they had been at the start of the match robust to the phenomenon of a player having an off day, and simply underperforming relative to expectations. Naturally, not all matches falling within the right-hand tail of the histogram represent instances of match-fixing: the presence of a discrepancy could be due to an injury, or any number of extraneous factors. However, based on the score assigned by our model, these are the matches that merit closer scrutiny. Out of bounds When we examined the matches in the rightmost 1% of the distribution (that is, those with a discrepancy score exceeding 0.06), we found there were matches in which the odds evolved in a highly irregular fashion that we could not rationally explain. By way of illustration, we present graphs from two of these matches: one played in August 2014 (Figure 3a) and the other in February 2015 (Figure 3b). As is evident from these graphs, under our model, there is no value of that is remotely consistent with the evolution of the betting market. In fact, for both matches, the eventual winner would have needed to be a dead cert at the start of the match (as shown by the starting point of the blue model line) for the market probabilities observed later on to make sense. In the second of these matches, the market jolted so irregularly that the eventual winner was deemed significantly more likely to win when trailing by a set than they had been at the start of the match a clearly absurd situation. The final piece of the jigsaw slotted into place when we investigated the background to these particular matches in detail. We found that in both cases, the evolution of the betting market had been identified as being highly irregular by numerous tennis blogs including DW on Sport and other websites, such as Slate.com. In other words, our system had been able to successfully filter down a collection of several thousand matches to a small subset that included those specifically flagged as suspicious by tennis experts. FIGURE 2 Histogram of the discrepancy score over the set of matches examined A final volley Given some of the negative reactions elicited by Buzzfeed s pre-match odds analysis back in January, it bears repeating once again that anomalous patterns in odds movements 20 SIGNIFICANCE June 2016

6 can never provide cast-iron proof of match-fixing. However, to the trained statistician s eye, some of the results we have obtained are profoundly unsettling. What is the explanation for the irregular market movements in these matches, where the link between events unfolding on the court and the market odds has apparently been completely severed? In what respect do these specific matches differ from the overwhelming majority of matches that do appear to unfold rationally? With the scourge of tennis matchfixing now firmly in the public eye, and fresh corruption revelations surfacing almost every month, we are committed to obtaining credible answers to these questions, and will be sharing our findings with the Tennis Integrity Unit (TIU) the sport s watchdog in the hope that they may be able to shed further light on these puzzling irregularities. Watch this space. n (a) Note Since our initial work was undertaken last summer, this stream of research has blossomed into a three-year collaborative PhD project with Lancaster University on the topic of forensic sports analytics. (b) FIGURE 3 Two anomalous matches (a) from August 2014 and (b) from February 2015 in which the in-play odds evolved in a highly irregular fashion. As before, the red trajectory represents the market probability at each point of the match, while the blue trajectory represents the closest fit achieved by the model. For match (a), the optimal value of was , achieving a discrepancy score of 0.092; for match (b), the optimal was 0.08, with a discrepancy score of References 1. Rodenburg, R. and Feustel, E. D. (2014) Forensic sports analytics: Detecting and predicting match-fixing in tennis. Journal of Prediction Markets, 8(1), O Malley, A. J. (2008) Probability formulas and statistical analysis in tennis. Journal of Quantitative Analysis in Sports, 4(2). 3. Madurska, A. M. (2012) A set-by-set analysis method for predicting the outcome of professional singles tennis matches. MEng project, Department of Computing, Imperial College London. bit.ly/1w6wjvg 4. Klaassen, F. J. G. M. and Magnus, J. R. (2003) Forecasting the winner of a tennis match. European Journal of Operational Research, 148, bit.ly/1w6zlqo 5. Klaassen, F. J. G. M. and Magnus, J. R. (2001) Are points in tennis independent and identically distributed? Evidence from a dynamic binary panel data model. Journal of the American Statistical Association, 96(454), bit.ly/1rud16v June 2016 significancemagazine.com 21