Predictive Analysis of Football Matches using In-play Data

Transcription

1 Department of AI, Faculty of ICT, University of Malta Predictive Analysis of Football Matches using In-play Data Matthew J. Zammit September 11, 2018

2 It s the 2 nd of January 2017 at 15:48 (GMT), Goodison Park Ref. Kevin Friend signals for halftime of Match Game 20 between Everton FC & Southampton FC. Halftime result - Goalless draw

3 Match day - Halftime Analysis (1) Figure: Shots per team Figure: Crosses per team 3/41 Matthew J. Zammit MSc. Dissertation Viva Voce

4 Match day - Halftime Analysis (2) Figure: Dribbles per team Figure: Tackles per team 4/41 Matthew J. Zammit MSc. Dissertation Viva Voce

5 Match day - Halftime Analysis (3) Figure: Passes per team 5/41 Matthew J. Zammit MSc. Dissertation Viva Voce

6 How do you think is the match going to end? Match result market at halftime; Everton to win, odds at 2.86 (0.35). S hampton to win, odds at 4.00 (0.25). Ends in a draw, odds at 2.50 (0.40).

7 Motivation Rise in popularity of betting exchanges through the Internet Prediction Markets, have been found to be accurate Sports data is recently being captured at precise and granular levels than ever before 7/41 Matthew J. Zammit MSc. Dissertation Viva Voce

8 Why Machine Learning? Multitude of complex variables associated with a football match Difficult for humans to think in terms of probability and to react to market changes Emotions might hinder the performance of humans to make rational decisions 8/41 Matthew J. Zammit MSc. Dissertation Viva Voce

9 Aims and Objectives Predict the fulltime result (H/D/A) of matches drawn at halftime using in-play match data Investigate whether using Feature Selection (FS) by a Genetic Algorithm (GA) process would remove certain predictors and increase classification accuracy Test if the addition of pre-match data to the in-play game statistics would improve accuracy rate Compare the probabilistic classification of the classifier with that of the implied probability from the betting exchange market 9/41 Matthew J. Zammit MSc. Dissertation Viva Voce

10 Major Issues No publicly accessible datasets available No previously conducted studies using same data to compare with Most similar study was carried out using a Case Based Reasoning approach on the over/under 2.5 goals market 10/41 Matthew J. Zammit MSc. Dissertation Viva Voce

11 Dissertation Schematic 11/41 Matthew J. Zammit MSc. Dissertation Viva Voce

12 Halftime/Fulltime Result Base Rule Data was retrieved from Fooball-Data We investigated the transition probability of the halftime result to that of the fulltime result of each match Consists of 77,553 match instances spanning several major and minor leagues across Europe over multiple seasons We found that for a high percentage of matches the fulltime result remained the same as that of the halftime Base rule BR ftr (R htr ) = R htr (1) 12/41 Matthew J. Zammit MSc. Dissertation Viva Voce

13 In-play Dataset Football-Data data was not granular enough for the study Data was parsed from Whoscored website and had to be engineered using conditional rules Main benefits why this site was chosen: 1 Data recorded at a play-by-play rate 2 Actions are labeled with a type, the x and y coordinates of the ball 3 Opta as the source 4 Continually being updated with data of major European competitions. Most importantly, English Premier League, Italian Serie A, Spanish La Liga, French Ligue and the German Bundesliga from 2009/10 to present 13/41 Matthew J. Zammit MSc. Dissertation Viva Voce

14 Feature Engineering of In-play Predictors Dataset URL: CORNER_FAVOUR SHOT_TOTAL FOUL_RECEIVED SHOT_ON_GOAL CROSS_FAV_TOTAL ASSIST_SHOT CROSS_FAV_SUCCESS ASSIST_INTENT L OFFSIDE_COMMITTED ASSIST_INTENT L_GOAL POSSESSION_TOTAL PASS_TOTAL/SUCCESS POSSESSION_ATT PASS_LONG POSSESSION_DEF PASS_FORW/BACK INTERCEPTION PASS_TRGT_FINAL_TRD CARD_YELLOW/RED PASS_TRGT_MID_TRD TACKLE_TOT/SUCCESS PASS_TRGT_DEF_TRD DRIBBLE_TOT/SUCCESS 14/41 Matthew J. Zammit MSc. Dissertation Viva Voce

15 Feature and Target Vector The feature vector constitutes of the difference in the halftime statistics between the home and away team A positive value for a particular feature means that the home team had accumulated more of that statistic till the halftime than the away team The target vector consists of only one element for each feature vector. The value could be from the set {0,1,2}, where the elements represents home, draw and away win respectively 15/41 Matthew J. Zammit MSc. Dissertation Viva Voce

16 match target (FTR) shottotaldiff shotongoaldiff passtotaldiff passlongdiff passsuccessdiff passbackwarddiff passtargetfinalthirddiff... tackletotaldiff tacklesuccessdiff dribbletotaldiff Feature and Target Vector Example Arsenal v Aston Villa Arsenal v Cardiff Arsenal v Chelsea Arsenal v Crystal Palace Arsenal v Everton Arsenal v Fulham Arsenal v Hull Arsenal v Liverpool Arsenal v Man City Arsenal v Man Utd Arsenal v Newcastle Arsenal v Norwich Arsenal v Southampton Arsenal v Stoke Arsenal v Sunderland Arsenal v Swansea Arsenal v Tottenham Arsenal v West Brom Arsenal v West Ham /41 Matthew J. Zammit MSc. Dissertation Viva Voce

17 Manual Feature Selection Experiments Instances from the English 2015/16 season were used as a sample for initial experimentation Features were iteratively being added to the feature space depending on the accuracy and based on our football intuition Machine Learning Algorithms used; Neural Nets (NN), Naive Bayes (NB), Decision Tree (DT), Random Forest (RF) Dataset was normalized for algorithms which trained faster and perform better with scaled data Random Forest was found to be consistently accurate across all the tests 17/41 Matthew J. Zammit MSc. Dissertation Viva Voce

18 Automated Feature Selection using GA Using only the Random Forest algorithm with custom GA separately for each league Investigate classification performance and predictors chosen with default parameter settings and with model tuning Nested Cross Validation (CV) Grid search used for parameter tuning Fitness function promotes fewer predictors Growth function for mutation rate Stopping criteria: score of latest generation subtracted by the mean of the scores from the previous ten generations greater than a threshold (0.1) 18/41 Matthew J. Zammit MSc. Dissertation Viva Voce

19 Genetic Algorithm - Feature selection process Chromosomes Parenta Parentb Random cutoff Cross-Over Parenta Parentb Switch ends from cutoff point Mutation Childa Childb Mu Mu Mu Randomly select genes for mutations Final Childa Childb 19/41 Matthew J. Zammit MSc. Dissertation Viva Voce

20 Genetic Algorithm - Mutation Threshold Mutation Threshold mt = tanh ( 2i n ) (2) Where i is the current epoch and n is the maximum number of epochs. 20/41 Matthew J. Zammit MSc. Dissertation Viva Voce

21 Nested CV with Parameter Tuning p1 Outer folds X1 V1 X2 V2 X2... Vm Xm p1 p1 X1,1 V1,1 X2,1 V2,1 Xm,1 Vm,1 s1 s1... s1 p2 p2 p2 X1,1 V1,1 X2,1 V2,1 Xm,1 Vm,1 Inner folds s2 s2... s pn pn pn X1,1 V1,1 X2,1 V2,1 Xm,1 Vm,1 sn sn... sn 21/41 Matthew J. Zammit MSc. Dissertation Viva Voce

22 Evaluation of Classifiers Accuracy Accuracy = TP + TN TP + TN + FP + FN (3) TP - True Positive TN - True Negative FP - False Positive FN - False Negative 22/41 Matthew J. Zammit MSc. Dissertation Viva Voce

23 Pre-match data Prematch data added to contextualise the instances Prematch feature set includes simple attributes: Goals Scored Goals Conceded Points and computed ones Team Form based on the teams latest performances Attacking Strength Defensive Strength Inner partitioning loop customised to train/test on a seasonal basis because of temporal data Same as with in-play data, the feature vector consisted of the difference between the home and away team pre-match statistics 23/41 Matthew J. Zammit MSc. Dissertation Viva Voce

24 Pre-match Features - Att & Def Strength where, t denotes the team for which the strength is being calculated, n represents the match game and m describes the total number of teams. S and C represent the goals scored and goals conceded matrices, respectively. Attack Strength AttackStr(t, n, m) = Defence Strength DefenceStr(t, n, m) = 1 n 1 mn n S it i=1 m j=1 i=1 1 n 1 mn n (4) n S ij C it i=1 m j=1 i=1 (5) n C ij 24/41 Matthew J. Zammit MSc. Dissertation Viva Voce

25 Pre-match Features - Form Form Form(t, j) = 5 ( 1 2 )i (y j i,t ) i=0 (6) 5 ( 1 2 )i i=0 Importance is given to the result of the previous games depending on how recent they have been played by assigning them different weights. 25/41 Matthew J. Zammit MSc. Dissertation Viva Voce

26 Pre-match Form - Examples 26/41 Matthew J. Zammit MSc. Dissertation Viva Voce

27 Custom Inner Loop for Prematch data Split Data Set / / / / /16 t 1 v 1 t 2 v 2 t 3 v 3 t 4 v 4 Score S 1 S 2 S 3 S 4 S 27/41 Matthew J. Zammit MSc. Dissertation Viva Voce

28 Evalutation of Probability Outcomes Brier Score Function 1 n n i=1 r (p ij o ij ) 2 (7) j=1 Where, n is the total number of instances and r is the number of possible outcomes (three is our case). p i,j is the probability of the j th outcome for the i th instance from the model. For example, when i = 1 the probability vector is [0.7, 0.2, 0.1]. o i,j is the actual probability of the j th outcome for the i th instance after its occurrence. For example, for the same instance i = 1, the actual result was [1.0, 0.0, 0.0]. 28/41 Matthew J. Zammit MSc. Dissertation Viva Voce

29 Base Rule Matrices Results HTR - Halftime Result, FTR - Fulltime Result. A darker color represents a lower value. 29/41 Matthew J. Zammit MSc. Dissertation Viva Voce

30 BR Matrices Results by Goal Difference (1) 30/41 Matthew J. Zammit MSc. Dissertation Viva Voce

31 BR Matrices Results by Goal Difference (2) 31/41 Matthew J. Zammit MSc. Dissertation Viva Voce

32 Time Series Results One tailed paired t-test showed that the accuracy of the time series random forest was not significantly different from that of the base-rule with t-statistic of 1.33 and p-value of /41 Matthew J. Zammit MSc. Dissertation Viva Voce

33 Predictors Chosen from GA and Tuning 33/41 Matthew J. Zammit MSc. Dissertation Viva Voce

34 Decision Tree Example 34/41 Matthew J. Zammit MSc. Dissertation Viva Voce

35 Results Summary Leagues RF HT+PM RF GA(T) RF GA RF TS RF 10CV BR English Premier League (±.041) (±.034) (±.028) (±.052) Italian Serie A (±.051) (±.039) (±.035) (±.018) Spanish La Liga (±.038) (±.031) (±.035) (±.013) German Bundesliga (±.037) (±.047) (±.043) (±.053) French Ligue (±.036) (±.040) (±.046) (±.037) Mean (±0.020) (±.016) (±.011) (±0.040) (±0.041) All leagues (±.027) (±.015) BR - Base Rule RF 10CV - Random Forest 10-fold Cross Validation RF TS - Random Forest Time Series RF GA - Default Random Forest with Genetic Algorithm RF GA(T) - Random Forest with Genetic Algorithm & Tuned RF HT+PM - Random Forest & Genetic Algorithm with In-play & Pre-match Data 35/41 Matthew J. Zammit MSc. Dissertation Viva Voce

36 Comparision with Betting Exchange (Italy) Brier Score for the PM was and for the RF was Match Prediction Home Draw Away Actual AC Milan vs Cagliari Random Forest BetFair Actual Crotone vs Empoli Random Forest BetFair Actual Empoli vs Udinese Random Forest BetFair Actual Lazio vs Chievo Random Forest BetFair Actual Lazio vs Crotone Random Forest BetFair Actual Napoli vs Pescara Random Forest BetFair Match Prediction Home Draw Away Actual Napoli vs Pescara Random Forest BetFair Actual Palermo vs Inter Random Forest BetFair Actual Roma vs Cagliari Random Forest BetFair Actual Sampdoria vs Empoli Random Forest BetFair Actual Sassuolo vs Torino Random Forest BetFair Actual Udinese vs AC Milan Random Forest BetFair /41 Matthew J. Zammit MSc. Dissertation Viva Voce

37 Comparision with Betting Exchange (Eng) Brier Score for the PM was and for the RF, Match Prediction Home Draw Away Actual Arsenal vs Burnley Random Forest BetFair Actual Burnley vs Random Forest Southampton BetFair Actual Hull vs Bournemouth Random Forest BetFair Actual Liverpool vs Swansea Random Forest BetFair Actual Man City vs Burnley Random Forest BetFair Man City vs Tottenham Watford vs Middlesbrough West Ham vs Man Utd Actual Random Forest BetFair Actual Random Forest BetFair Actual Random Forest BetFair /41 Matthew J. Zammit MSc. Dissertation Viva Voce

38 Everton won the match by three goals to nil. Everton vs Southampton Actual Random Forest BetFair

39 Conclusion We have derived a base rule for predicting fulltime results at the halftime interval of football matches Parsed in-play data from an Opta source and developed a dataset consisting of the differences between team statistics till the halftime for both pre-match and match day data Shown that random forest using both types of available data produced the best results Similar accuracy as the betting market when considering probabilities for predictions, in some cases out-performing the market and thus giving an edge to the user 39/41 Matthew J. Zammit MSc. Dissertation Viva Voce

40 Future Work Addition of other predictors such as inclusion of key players in team, player individual form and their scoring and defensive abilities Rate or number of entrances into opposition penalty box and dangerous areas Split match data into several minute time-frames and investigate predictions along the time of play Investigate predictions on other markets such as over/under goals and next team to score Use predictions as part of a betting strategy 40/41 Matthew J. Zammit MSc. Dissertation Viva Voce

41 Thank you for your attention