Standing Between a Bayesian and a Frequentist: An Emperical Bayes Exploration of Movies, Baseball, and Long Beach Basketball.

Transcription

1 Standing Between a Bayesian and a Frequentist: An Emperical Bayes Exploration of Movies, Baseball, and Long Beach Basketball Arthur Berg Pennsylvania State University

2 Arthur Berg Standing Between a Bayesian and a Frequentist 2 / 28

3 Bayesian and Frequentist Representatives Rev. Thomas Bayes FRS ( ) English Mathematician Presbyterian Minister Sir Ronald Fisher FRS ( ) English Statistician Evolutionary Biologist, Geneticist P (H E) = P (E H)P (H) P (E) Let the data speak for itself. Arthur Berg Standing Between a Bayesian and a Frequentist 3 / 28

4 Bayes Estimator as a Convex Combination 1 st Goal: List the top 250 movies of all time. Movies are rated on a scale of 1 to 10. Some movies are rated by many people, and some by only a few. Movies with fewer than 3000 votes are not considered. All movies have an average rating of C = 6.9. µ i represents the mean rating by everyone who has seen movie i. The real goal is to construct the best estimate of µ i, then pick the top 250. The frequentist approach uses only X i, the average rating for movie i. = X i ˆµ (Fisher) i The Bayesian approach shrinks X i towards C with more shrinking applied when the number of votes for movie i is small. ˆµ (Bayes) i = α i Xi + (1 α i )C where α i (0, 1) Arthur Berg Standing Between a Bayesian and a Frequentist 4 / 28

5 Internet Movie Database Top 250 Rank WR R Title Votes The Shawshank Redemption (1994) 546, The Godfather (1972) 427, The Godfather: Part II (1974) 257, The Good, the Bad and the Ugly (1966) 170, Pulp Fiction (1994) 436, Inception (2010) 265, Schindler s List (1993) 289, Angry Men (1957) 126, One Flew Over the Cuckoo s Nest (1975) 225, The Dark Knight (2008) 487, Black Swan (2010) 20, Avatar (2009) 285, True Grit (2010) 6,444 Arthur Berg Standing Between a Bayesian and a Frequentist 5 / 28

6 IMDb Weighted Ranking a true Bayesian estimate WR i = v ir i + mc v i + m = v i v i + m α i R i = average rating of the movie i ( X i ) R i + Xi v i = total number of votes from regular voters m v i + m 1 α i m = minimum # of votes to make the list = 3000 C = grand mean across all movies in the database = 6.9 C Arthur Berg Standing Between a Bayesian and a Frequentist 6 / 28

7 A Bayesian Calculation X i = (X i,1,..., X i,vi ) represents the v i ratings of movie i. prior: µ i N (µ 0, σ 2 0 ) conditional: X i,j µ i iid N (µi, σ 2 ) (j = 1,..., v i ) ˆµ (Bayes) i = E[µ i X i ] v i = ( v i + σ 2 /σ0 2 ) X i + ( σ2 /σ0 2 v i + σ 2 /σ0 2 ) µ 0 v i = v i + m R m i + v i + m C µ 0 = C, m = σ 2 /σ0 2 Arthur Berg Standing Between a Bayesian and a Frequentist 7 / 28

8 1 Does 2 How shrinking really help? much to shrink by? Prediction Error = n i=1 (µ i ˆµ i ) 2

9 Standing Between a Bayesian and a Frequentist In 1956, Charles Stein proved the existence of an estimator better than the sample mean under certain assumptions. In 1961, Willard James and Charles Stein explicitly constructed such an estimator. Arthur Berg Standing Between a Bayesian and a Frequentist 9 / 28

10 The James-Stein Estimator (n 4) µ i N (µ 0, σ 2 0) X i µ i iid N (µi, σ 2 ) (i = 1,... n) σ 2 ˆµ (Bayes) i = E [µ i X i ] = ( σ0 2 + σ2 α )µ 0 + ( σ0 2 σ0 2 + )X i σ2 1 α (n ˆµ (JS) 3)σ2 i = ( (X i X) 2 α ) X + ( 1 In practice, if σ 2 is unknown, an estimate is used. (n 3)σ2 (X i X) 2 )X i 1 α Arthur Berg Standing Between a Bayesian and a Frequentist 10 / 28

11 Predicting Batting Averages 2 nd Goal: Predict final batting averages from pre-season performances. Pre-season batting averages for 18 major league players are provided. Season final batting averages for the same players are also recorded. Data is from the 1970 season and is published in JASA (1975) and Scientific American (1977) by Efron and Morris. The frequentist approach uses only X i, the pre-season batting average for player i. = X i ˆp (Fisher) i The Emperical Bayes approach shrinks X i towards X by some empirically determined amount. ˆp (Stein) i = ˆαX i + (1 ˆα) X where ˆα (0, 1) Arthur Berg Standing Between a Bayesian and a Frequentist 11 / 28

12 Name hits/ab pre-season (ˆµ (ML) ) season final (µ) 1 Clemente 18/ Robinson 17/ Howard 16/ Johnstone 15/ Berry 14/ Spencer 14/ Kessinger 13/ Alvarado 12/ Santo 11/ Swoboda 11/ Unser 10/ Williams 10/ Scott 10/ Petrocelli 10/ Rodriguez 10/ Campaneris 9/ Munson 8/ Alvis 7/ Arthur Berg Standing Between a Bayesian and a Frequentist 12 / 28

13 Batting Average Dataset 1977 Batting Averages Dataset (Efron) Batting Average pre season season final Arthur Berg Standing Between a Bayesian and a Frequentist 13 / 28

14 James-Stein Estimation of Batting Averages 1977 Batting Averages Dataset (Efron) Batting Average pre season season final Arthur Berg Standing Between a Bayesian and a Frequentist 14 / 28

15 Ranking Bias Emperical Bayes + Order Statistics 1977 Batting Averages Dataset (Efron) Genome-wide association studies SNPS: AA/Aa/aa or 0/1/2 ( 10 7 ) Batting Average pre season season final ranking bias estimator part frequentist, part Bayesian with robust properties Estimated effects of the top SNPs are biased up. (winner s curse) Applied to 2 GWAS studies with 2,000 cases and 3,000 controls Crohn s Disease Type 1 Diabetes Arthur Berg Standing Between a Bayesian and a Frequentist 15 / 28

16 49ers Statistics Arthur Berg Standing Between a Bayesian and a Frequentist 16 / 28

17 Opponents Over 3 Seasons 08-09, 09-10, opponent # alaska anchorage 1 arizona state 1 boise state 1 byu cougars 1 byu hawaii 1 cal poly 7 cal state fullerton 6 cal state northridge 6 clemson 2 cs monterey bay 1 duke 1 green bay 2 idaho 1 idaho state 1 iowa 1 kentucky 1 loyola marymount 2 montana 1 montana state 1 new mexico state 1 north carolina 1 notre dame 1 oregon 1 pacific 8 pepperdine 2 saint mary s 1 saint peter s 1 san diego state 1 san francisco state 1 syracuse 1 temple 1 texas 1 uc davis 6 uc irvine 6 uc riverside 6 uc santa barbara 7 ucla 1 univ. san francisco 1 utah state 2 washington 1 weber state 2 west virginia 1 wisconsin 1 Arthur Berg Standing Between a Bayesian and a Frequentist 17 / 28

18 Winning Percentages All Games All 3 Seasons (93) 56% Season (30) 50% Season (33) 52% Season (30) 67% Conference Games All 3 Seasons 67% Season 63% Season 50% Season 88% Arthur Berg Standing Between a Bayesian and a Frequentist 18 / 28

19 spread Spread = 49ers Score Opponent Score (10 11 Season) uc santa barbara cal state northridge uc davis cal poly uc riverside cal state fullerton pacific uc irvine Arthur Berg Standing Between a Bayesian and a Frequentist 19 / 28

20 spread Spread = 49ers Score Opponent Score (10 11 Season) uc santa barbara cal state northridge uc davis cal poly uc riverside cal state fullerton pacific uc irvine Arthur Berg Standing Between a Bayesian and a Frequentist 20 / 28

21 Over/Under (Total Score) uc irvine Over/Under = 49ers Score + Opponent Score (10 11 Season) cal state fullerton cal state northridge uc riverside Arthur Berg Standing Between a Bayesian and a Frequentist 21 / 28 uc davis pacific uc santa barbara cal poly

22 Over/Under (Total Score) uc irvine Over/Under = 49ers Score + Opponent Score (10 11 Season) cal state fullerton cal state northridge uc riverside Arthur Berg Standing Between a Bayesian and a Frequentist 22 / 28 uc davis pacific uc santa barbara cal poly

23 Conversion Formulas x = LB Score y = Opponent Score Over/Under = x + y Spread = x y Over/Under + Spread x = 2 Over/Under Spread y = 2 Arthur Berg Standing Between a Bayesian and a Frequentist 23 / 28

24 Predictions Over Under Rank Opponent LB Score O. Score Spread 2 Cal Poly Cal State Northridge Pacific UC Santa Barbara Cal State Fullerton UC Riverside UC Irvine UC Davis Arthur Berg Standing Between a Bayesian and a Frequentist 24 / 28

25 How good are the predictions? Using the season to predict the season: adjusted prediction error for spread unadjusted prediction error spread = = 58% adjusted prediction error for over/under unadjusted prediction error over/under = = 63% Using the season to predict the season: adjusted prediction error for spread unadjusted prediction error spread = = 78% adjusted prediction error for over/under unadjusted prediction error over/under = = 69% Arthur Berg Standing Between a Bayesian and a Frequentist 25 / 28

26 LB vs UCI Vegas Odds (as of 3am on game day) All bets are pay $110 to win $100. Long Beach is the favorite; UCI is the underdog. Casino Spread Over/Under LV Hilton Wynn MGM Mirage -10 NA Predicted These predictions recommend betting on UCI (still expecting LB to win) and betting on over for the over/under option. Arthur Berg Standing Between a Bayesian and a Frequentist 26 / 28

27 Disclaimers: 1 I do not necessarily encourage sports betting. 2 I am not liable for any bets made based my presentation. Arthur Berg Standing Between a Bayesian and a Frequentist 27 / 28

28 Thank You!! Beach.ArthurBerg.com Arthur Berg Standing Between a Bayesian and a Frequentist 28 / 28