Activity: the race to beat Ruth

Transcription

1 Activity: the race to beat Ruth Background In baseball, a pitcher from the fielding team throws a ball at a batter from the batting team, who attempts to hit the ball with a stick. If she hits the ball, she then tries to run around the playing area, while the players in the fielding team try first to catch the ball and then to throw the ball at a base she is running towards. If the batter is very good, she might manage to run all the way around the playing area without the fielding team managing to hit her next base. In this case, she obtains a home run. In 1961, America was enthralled by a race between two baseball players, Mickey Mantle and Roger Maris, both of the New York Yankees baseball team. This pair were battling to overcome the legendary Babe Ruth s 1 record of 60 home runs in a season, set in As Mlodinow writes, a good looking, good natured fellow, Mantle came across as the all-american boy everyone hoped would set records. Maris was gruff a nasty sort who didn t like kids 2. It was the gruff Maris who broke the record, with 61 home runs, but he never again hit more than 40 home runs in a season. Was Maris better than Ruth? Although we re focusing on a sport for this activity, the idea of ranking individuals or organisations is a serious affair. In some countries, schools are ranked by their pupils performance in exams, and hospitals are ranked by how many patients die during surgery, for example; this naturally influences their ability to attract students and patients. In this activity, we ll be looking at rankings and relative performance in a series of games assuming we know the true ability of the players. Since games hinge on chance events, we need to look at the distribution of performances over alternative realisations. One of the best ways to assess distributions is by simulation, and in this activity we ll be simulating games on a computer using the R environment. Objectives This is an introductory exercise that presumes no knowledge of R. It is intended to bring together various facets of the language and prepare you for the activities in the second and third meetings. We have therefore given you code that you can directly use to tackle the problem. There are several ways you can use this code. You can blind yourself to the code if you wish to try to work out how to do it for yourself, or you can use the code as the foundation for your own version, or you can just use it as-is. If the latter, you should make sure you attempt to understand each line (otherwise you re not learning anything, and you ll have problems with the subsequent activities). One thing you can do to help with this is to type the name of any variable you create; this causes R to print the contents of that variable to the screen. If after doing that, you re still stuck, ask the people near you or the facilitators. Specifically by completing the exercise you will learn to: 1. Start R or RStudio. 2. Change your working directory. 3. Install and load packages. 4. Load data from a file into R. 5. Transform data and create new variables. 6. Write data from R to a file. 7. Plot data. 8. Output plots to file. 9. Write a script. 10. Write a function. 11. Use loops and logical statements. 12. Perform a basic simulation. 1 Babe Ruth is regularly voted as one of America s favourite sportsmen, even now, more than 60 years after his death. Check out his Wikipedia page! 2 Leonard Mlodinow (2008) The Drunkard s Walk. London: Penguin.

2 Data We ll be using data from the Lahman Baseball Database, owned by Sean Lahman ( This is licensed under Creative Commons Attribution-ShareAlike 3.0 Unported License, so you re allowed to download and use it. This contains the annual records of ~ baseball players born in the period , including Ruth, Mantle and Maris. As you can imagine, it s a large database. You want to use two files: Master.csv, which contains details of each player, including names and a player ID (one line per player), and batting.csv, which contains one line per player per season, indexed by their player ID, and which contains statistics on their performance that year. In particular, the following columns will be useful: playerid, yearid, AB (which is the number of at bats the player had that season, namely the number of chances he had to get a home run) and HR (which is the number of home runs he had that season). Tasks: 1. Start up RStudio if you have it installed. If you don t, start R instead or install RStudio now. The instructions assume you are using RStudio. You should see a GUI with several parts to it. One part has some legal information. This is where you type commands. You might try some simple ones to see what happens, say 6+9 # then hit enter x=999 # an equivalent alternative is x<-999 x # # indicates a comment by the way x = 4 # spaces are optional y=x+1 z=c(1,2,3) omega=1:3 omega[2] sin(omega) The first line above just adds 6 and 9 and prints to the screen. It doesn t do store the result anywhere. The second line assigns the value of 999 to x. Since x did not exist yet, R creates it first and then assigns the value of 999 to it. After you type this, you may see the variable x appear in the workspace pane. However, the number 999 is not printed to screen. To print to screen, type the variable name (line 3; you can also type print(x) if you want, and there are other ways using the function cat() for example). Work out what the other lines do. If you d like to find more basic R commands, check out Short s R reference card. Note that these commands can be rerun one at a time by using the up arrow to select them. However, a better way to rerun commands, and to allow you to rerun them at a later date, or to let someone else rerun them, is to make a script. You ll see in RStudio a pane in the top left with some buttons Run and Source. This is the script pane. If you put the commands you want to run here, you can select and run them (individually, groups or all) by lingering the cursor on a line and hitting ctrl-r or ctrl-enter, or hitting the Run button; or selecting a group of lines and doing the same; or ctrl-a ctrl-r to select all lines and run them all. It would make sense, if you re using the commands provided here, to copy them into the R script part of the screen, and save it as something like baseball.r.

3 2. Now start looking at the data by loading it into R. For convenience, you can change the working directory to the place your data are stored in your computer. In order to perform certain analysis later on, you need to install and load packages that you will use later. setwd("c:/t/yalenus/rworkshop") #[1] b=read.csv('batting.csv') #[2] m=read.csv('master.csv') library(lattice) #[3] Comments on the numbered lines follow: [1]. Set the working directory that you will read data from and write data into. Remember to change it to suit your directory structure. Note that Windows uses an unorthodox system for labelling directories with backslashes (i.e. the \ symbol) in place of the forward slash ( / ) used in other operating systems. In R, however, you should use the forward slash regardless of your operating system. [2]. The function, read.csv(file,header), is used to read a data file saved in csv format, where file is the name of the file which the data are to be read from, and header is the a logical value indicating whether the file contains the names of the variables as its first line. In this case, the data are read and stored in a variable called b. header by default is set as TRUE so if you are reading in data without column titles, you should use something like read.csv( myfilename.csv,header=false). Note that single and double quote marks can both be used, but don t pair them or you will cause chaos. Note that if you d like to explore this dataset and followed the advice we gave you before (i.e. typing the name of the variable, here b), you re going to get a massive stream of numbers flowing onto your screen. If you d like to see just the top of the dataset, type head(b). You ll see that the data are arranged in a rectangular grid, a bit like an excel spreadsheet, with names of the columns along the top. This is what R calls a data frame. If you d like to extract just one column from the data, you use the $ symbol, like this: b$playerid. [3]. We are going to use something called a package later, in this case, a package called lattice which is used to make certain graph types. Packages contain bundles of functions that were written by someone else and, well, packaged together for others to use. There are thousands of packages available: the most essential ones are already installed on your computer, and others are available on the comprehensive R archive network (CRAN). Before you use this command to load a package, it needs to be installed on your computer, either already or by you. You only need to install once, and next time you want to use this package, you only need to load the package using the library() function. If lattice is not already installed on your computer, click the "packages" tab in the lower right pane, then click "install packages", key in the name of the package and click "Install" 3. 3 Note: installing packages is one of the most annoyingly tricky things in R. Some packages depend on others being installed, perhaps of a particular version, and some may require some non-r libraries to be running on your computer. If you get an error during the class, let us know and we ll try to fix it! If you get an error outside of class, try googling.

4 3. Next, explore the data a little. Visualisation is the simplest way to give you an idea of the distribution of the data. You might plot the number of home runs, the number of at-bats, the ratio of home runs to at-bats (i.e. the fraction of possible home runs that were obtained), the joint distribution of ratios to at-bats, and so on. You might also plot the number of home runs over time that Ruth, Mantle and Maris got, and perhaps their ratios. What were Ruth s, Mantle s and Maris best home run percentages? What is the average home run percentage for all players? Above are only some measures you may use to compare the three players; you can certainly come up with your own. # First, explore the data a little. You might plot the number of home runs, histogram(b$hr,freq=false,xlab='homeruns',col='darkorange', breaks=seq(0,75,5)) #[4] # --> Most players get less than HRs a season # --> Very few get more than # --> Very very few get more than [4]. histogram() is a function in the lattice package which, yes, creates an histogram and prints it to the screen 4. You ll notice that unlike the functions above, this function has rather a lot of input variables or arguments. To find out what the arguments of a function are, you can consult the help files. In RStudio you can also type the name of the function, an opening bracket, and then hit tab (the autocomplete key): this will then show you what arguments the function takes. Some arguments are optional, in which case not specifying them makes the function use default values. How does R know which argument is which? If you do not specify the name of the argument, R will assume the first unspecified one corresponds to the first argument in the default ordering. For example, you don t specify the value of the argument x that histogram is expecting, and similarly, you don t specify the argument to which the vector b$hr from you dataset corresponds, so R assumes you wanted x=b$hr. On the other hand, you can also specify the names of arguments, in which case the ordering doesn t matter. Examples are freq, xlab, col, and breaks above. freq=false specifies that we are plotting the proportions rather than frequencies (i.e. counts) of each group. breaks is a numeric vector that specifies the breakpoints between bars in the histogram. xlab is text string giving labels for the x-axis. col specifies colour of the bars in the histogram, in this case, a handsome Yale-NUS-like orange. Other arguments you could use include xlim, which gives the range of the x-axis, ylab and ylim. Plotting the histogram gives an idea how your data are distributed and may influence how you analyse them further. # the number of at-bats, histogram(b$ab,xlab='at bats,col='darkorange',breaks=seq(0,750,50)) # --> The distribution is pretty weird! A lot get less than 100 ABs, but if you get more # than 100, the distribution is pretty uniform up to about 600, then it tapers off # the ratio of home runs to at-bats (i.e. the fraction of possible home runs that were # obtained), pchr=100*b$hr/b$ab histogram(pchr,xlab='home run %age',col='darkorange',breaks=seq(0,100,1),xlim=c(-1,20)) # --> Most players have a very low chance of getting a HR. Most are < % # --> There is a small number with HR %ages between and ish # --> There's also a very long tail and some have, ostensibly, a 100% HR percentage 4 You can also make it print to a file. See later for syntax.

5 # --> We can investigate this: mean(pchr>5,na.rm=true) #[5] mean(pchr>6,na.rm=true) mean(pchr>7,na.rm=true) mean(pchr>8,na.rm=true) mean(pchr>9,na.rm=true) mean(pchr>10,na.rm=true) mean(pchr>20,na.rm=true) # About 1 in have a HR %age over 20, and 2 in have a HR %age over 10 [5]. mean(x, na.rm=true) gives the mean of values in vector x, and any missing values are excluded from the analysis. In this case, we re asking for the mean of the statement pchr>5. What does that mean?! pchr is a vector of values, with one entry per batter per season, with the percentage of home runs. pchr>5 is a logical statement which returns a vector of TRUEs and FALSEs. R interprets TRUE as being 1 and FALSE as 0 in arithmetic expressions, so when we work out the mean of pchr>5, we re first converting pchr to a vector of TRUEs and FALSEs, then that vector to a vector of 1s and 0s, then taking the mean of that vector, which is the proportion of times it is equal to 1. So, mean(pchr>5) is merely a quick way to ask for the proportion of times the home run percentage is more than 5%. Oh, the na.rm term is needed as some players had no at-bat opportunities some years, which leads to an NA in their entry which would muck up the calculation. # the joint distribution of ratios to at-bats, and so on. scatter.smooth(b$ab,pchr,col=rgb(1,0,0,0.1),pch=16,xlab='atbase',ylab='homerun %age') #[6] # --> Why do some people have a 100% record? # This suggests we ought to exclude those # with small (say <100) ABs # --> There's a slight trend for better players to have more a-bats. But the effect # is quite small so can be ignored without much impact [6]. scatter.smooth() plots and adds a smooth curve to a scatter plot. col=rgb(red, green, blue, alpha) is another way to specify colours other than using the names directly (e.g. col='darkorange'). Using this colour specification, an alpha transparency value between 0 and 1 can be specified (0 means fully transparent and 1 means opaque). If you have a lot of points, making them semi-transparent makes it easier to visualise the density of points. # output the number of home runs, at-bats, and the ratio of home runs to at-bats to a # csv file output=data.frame(hr=b$hr,ab=b$ab,pchr=pchr) #[7] write.csv(output,'baseball.csv',row.names=false) #[8] [7]. As mentioned before, a data frame is used to store data tables. This is how to make a data frame within R (rather than reading it in from a file). In this output data frame we created, we have three columns with names HR, AB, and pchr. [8]. write.csv() outputs a data frame to an outside comma-separated values (csv) file for future use. This format can be read by excel, libreoffice calc, and most other statistics packages. It is also a text file, so can be read into scripts in other programming languages such as C. The names of the data frame and the name of the output file need to be specified. row.names=false specifies that there be no row names in the csv file. Usually you don t want row names.

6 # You might also plot the number of home runs over time that Ruth, Mantle and Maris got, id_ruth=as.character(m$playerid[which(m$namelast=='ruth')]) #[9] id_maris=as.character(m$playerid[which(m$namelast=='maris')]) id_mantle=as.character(m$playerid[which(m$namelast=='mantle')]) i=which(b$playerid==id_ruth) AB_ruth=b$AB[i] #[10] HR_ruth=b$HR[i] yr_ruth=b$yearid[i] i=which(b$playerid==id_maris) AB_maris=b$AB[i] HR_maris=b$HR[i] yr_maris=b$yearid[i] i=which(b$playerid==id_mantle) AB_mantle=b$AB[i] HR_mantle=b$HR[i] yr_mantle=b$yearid[i] plot(yr_ruth,hr_ruth,xlab='',ylab='home runs',pch=16,col='steelblue',xlim=c(1910,1970),ylim=c(0,70)) #[11] lines(loess.smooth(yr_ruth,hr_ruth),col='steelblue',lwd=2) #[12] points(yr_mantle,hr_mantle,pch=16,col='darkorange') #[13] lines(loess.smooth(yr_mantle,hr_mantle),col='darkorange',lwd=2) points(yr_maris,hr_maris,pch=16,col='black') lines(loess.smooth(yr_maris,hr_maris),col='black',lwd=2) text(1925,65,'ruth',col='steelblue') #[14] text(1955,65,'mantle',col='darkorange') text(1965,65,'maris',col='black') # --> Who looks best: Ruth, Maris or Mantle? [9]. which(x) gives the indices where the argument is TRUE. For example, if x=c(1,3,5) then which(x==3) returns 2 and which(x>=3) returns the vector c(2,3). 5 This line gives us indices of entries where the namelast column is Ruth. It turns out that all of these do actually correspond to Babe Ruth (you should always check). [10]. x[i] gives the ith element of the vector x. i can be a vector. This line returns all the at-bat numbers correspond to Ruth in the data set. [11]. plot(x, y,...) plots y versus x on a new graph. If both x and y are vectors of numbers then this will by default create a scatter plot with a point per datum. There are many graphical parameter arguments that can be used in this function to customize the graph. Ask us if you d like to customise it. [12], [13], [14]. lines(), points(), text() are used to add extra lines, points and texts, respectively on an existing graph. Note that you need to use plot() to plot a graph first before you can add lines, points or text onto that graph. 5 A double equals sign is different from the single equals sign we saw before. The statement x=3 assigns the value of 3 to the variable x. The statement x==3 asks is x equal to 3?.

7 # and perhaps their ratios. plot(yr_ruth,100*hr_ruth/ab_ruth,xlab='',ylab='home runs',pch=16,col='steelblue',xlim=c(1910,1970),ylim=c(0,13)) #[15] lines(loess.smooth(yr_ruth,100*hr_ruth/ab_ruth),col='steelblue',lwd=2) points(yr_mantle,100*hr_mantle/ab_mantle,pch=16,col='darkorange') lines(loess.smooth(yr_mantle,100*hr_mantle/ab_mantle),col='darkorange',lw d=2) points(yr_maris,100*hr_maris/ab_maris,pch=16,col='black') lines(loess.smooth(yr_maris,100*hr_maris/ab_maris),col='black',lwd=2) text(1925,12.5,'ruth',col='steelblue') text(1955,12.5,'mantle',col='darkorange') text(1965,12.5,'maris',col='black') # --> Who looks better: Ruth, Maris or Mantle? [15]. If you add one line png('three players.png',height=480,width=480) before plot() and add one line dev.off() after the last text(), then instead of getting a graph appearing on your screen, a png file will be created in your working directory, that can, for example, be inserted into reports in Microsoft Word or LaTeX documents. # What were Ruth s, Mantle s and Maris best home run percentages? max(100*hr_ruth/ab_ruth) max(100*hr_mantle/ab_mantle) max(100*hr_maris/ab_maris) # --> % Ruth # --> % Mantle # --> % Maris # What is the average home run percentage for all players? mean(100*b$hr/b$ab,na.rm=true) # --> %

8 4. Next, limit your attention to the period 1927 (when Ruth set the 60 home runs record) and 1997 (roughly the start of the steroid era). Exclude Ruth (as we re going to investigate the breaking of his record). Assume that the number of home runs a player gets in a season is binomially distributed (see box 1). If all players over this time period (except Ruth) had a 5% chance of getting a home run at every at-base opportunity, simulate how many times Ruth s record would be broken over this 70 year period. If this were as high as 8%, how many times would his record be beaten? If each player s home run chance were randomly selected from all the observed home run percentages in the dataset, how many times would Ruth s record be beaten? # Next, limit your attention to the period... : i=which((b$ab>=100)&(b$yearid<=1997)&(b$yearid>=1927)&(b$playerid!=id_rut h)) # i.e. exclude small ABs, early/late years, and Ruth hrpc=100*b$hr[i]/b$ab[i] # the empirical home run percentage histogram(hrpc,xlab='home run %age',col='darkorange',breaks=seq(0,100,1),xlim=c(-1,14)) # --> How does this plot differ from last time? # If all players over this time period (except Ruth) had a 5% chance of getting a home run # at every at-base opportunity, simulate how many times Ruth s record would be broken over # this 70 year period. This is the number of chances each player had each year: ABs_historic=b$AB[i] # --> This function takes a scalar/vector of homerun probabilities, a vector of at-bats, # and simulates a lot of binomial distributions (one per player per season). It then # counts the number of times Ruth s record of 60 is beaten. The simulated number of # records is output by the function. # If verbose is TRUE, these numbers are printed to the screen homerunsimulator=function(homerunprob=0.1,abs=abs_historic,verbose=false) #[16] { n=rbinom(length(abs),abs,homerunprob) #[17] sim_new_records=sum(n>60.5) if(verbose) { if(length(homerunprob)==1)cat('using home run probability', homerunprob,'\n===============================\n') cat('simulated seasons beating Ruth:',sim_new_records,'\n') } #[18] return(sim_new_records) #[19] } [16]. This is a (home-made) function. It takes in one or more inputs or arguments, processes the inputs, and produces output. In this case, the input is probability of having a home run when there is an at-bat, the number of at-bats, and a logical variable verbose which controls whether the function reports what it s doing to screen. The output is the total number of times someone would beat Ruth s record of 60 runs. [17]. rbinom(n,n,p) simulates N random numbers from a binomial distribution with n trials (the number of at-bats in a season) and success probability p (the probability of getting a home run). [18]. If verbose equals to TRUE, R will output some text to the screen. [19]. Output the value to where the function is called. As soon as a function hits a return command, it will stop and output the argument of return.

9 hrsim=homerunsimulator(0.05,verbose=true) #[20] # --> What does it return? What does this mean? # How likely is it that someone would beat Ruth if everyone were very good # But that s just ONE simulation. What if you did it again? What is the average # number of times Ruth s record would be beaten over a large number (say 100) # of 70 year periods? [20]. this line runs the homerunsimulator function and stores it in the variable hrsim. NSIM=100 hrsims=rep(0,nsim) for(sim in 1:NSIM)hrsims[sim]=homerunsimulator(0.05,verbose=FALSE) histogram(hrsims,xlab='simulated home runs') # --> In 100 alternative universes, # how many would give someone beating Ruth if every baseball player were very good? # If this were as high as 8%, how many times would his record be beaten? hrsim=homerunsimulator(0.08,verbose=true) # --> The number of simulated seasons should be something around. # --> If everyone were excellent, what would have happened? NSIM=1000 # Takes a long time (~1min) hrsims=rep(0,nsim) for(sim in 1:NSIM)hrsims[sim]=homerunsimulator(0.08,verbose=FALSE) histogram(hrsims,xlab='simulated home runs') # --> Sampling distribution is around. The mean is around and the sd is # around. The distribution is approximately normal # If each player s home run chance were randomly selected from all the observed home run # percentages in the dataset, how many times would Ruth s record be beaten? homerunsimulator(hrpc/100,verbose=true) # --> Probably somewhere around. NSIM=1000 # Takes a long time (~1min) hrsims=rep(0,nsim) for(sim in 1:NSIM)hrsims[sim]=homerunsimulator(hrpc/100,verbose=FALSE) histogram(hrsims,xlab='simulated home runs') # --> The sampling distribution is weird (non-normal). How likely is it for someone to # beat his record, even if no one were actually as talented as Ruth?

10 Box 1: binomial distribution If you have n attempts to do something, and for each have a probability p of success, and succeed in X of them, then we say that X comes from a binomial distribution. We denote this X~Bin(n, p). If so, X can take any integer value between 0 and n, inclusive. The probability that the random variable X is equal to a specific value, say k, given n and p, is Pr(X = k μ) = n k pk (1 p) n k where the first term is called n choose k and indicates the number of combinations you can select k items from a group of n of them (e.g. there are 36 choose 6 ways to select six students from a class of 36). Some examples of the binomial distribution are presented below: In R you may find the following commands useful: rbinom(n,n,p) # simulates N random variates from a binomial distribution # with n trials and probability p dbinom(k,n,p) # calculates the probability that a random variable which # is binomial with n trials and probability p of success # would be equal to k What now? If you ve finished this activity and you ve got some learning time left before makan, try our related football activity! You ll find it on the web. On Friday, we ll be leaving sports, and America, and looking at three other areas: sex, dying, and education, all Singapore style!