Effects of Incentives: Evidence from Major League Baseball Guy Stevens April 27, 2013 1
Contents 1 Introduction 2 2 Data 3 3 Models and Results 4 3.1 Total Offense................................... 4 3.2 Home Run Hitters................................ 5 4 Discussion 6 1
1 Introduction Incentives are stimuli, financial or otherwise, that promote certain behaviors. They are often used in the workplace to encourage improved job performance by employees. While there is an intuitive relationship between financial rewards such as an end-of-year bonus and productivity increases, studying this connection in practice is difficult. Quantifying job performance is very difficult in many fields, making empirical research into human response to incentives almost impossible. Job performance is quantified in almost unparalleled detail in Major League Baseball, making it useful for an investigation into the effects of incentives. Every pitch, swing, hit, and catch are recorded with extensive accompanying information, and there are constantly new data sources providing even better information. For example, Sportvision s PITCHf/x system records velocity, spin, and location of every single Major League pitch. These new sources of data, as well as modern statistical measures, give an opportunity for a fresh look at performance in the presence of financial incentives. Baseball players may significantly alter their own future earnings by performing well before reaching free agency. During free agency, a right typically granted to players who have accrued six seasons of Major League service time, any of the 30 Major League clubs may bid for their services. MLB free agency is the source of some of the most exorbitant contracts in the history of professional sports, including two contracts over $250million for Alex Rodriguez and a $240million contract given to Albert Pujols by the Los Angeles Angels of Anaheim that will pay him one of the highest salaries in American sports beyond age 40. Players typically enjoy their prime in their late 20 s to early 30 s, but by their late 30 s are almost always in a state of significant decline in skill. In deciding how much to pay a free agent, teams must project how the player will perform in the future based on his past performance and assign a value to that performance. In many cases, teams accept overpaying for the last few years of a large contract in exchange for signing a player through his prime years. Identifying what causes changes in performance is important to assessing what level of performance can be expected in the future and how much risk is involved. One potential source of performance change that may or may not exist is due to the presence of financial incentives due to impending free agency. If such incentives affect performance, teams would favor prior performance to that of the player s contract year. In accordance with this issue, focusing a study on how players perform before signing their contract, rather than how they are paid or how they perform afterward, could be more enlightening than the alternative. Stolovitch, Clark, and Condly (2002) wrote an authoritative, consensual picture of incentive systems in general after conducting extensive research. They found evidence of several different successful incentive systems, most notably quota-based systems. These give employees rewards if they meet numerical quotas in certain fields, i.e. meeting a targeted number of sales or bringing in a certain number of clients. The positive responses to quota-based incentives stem from two main factors. Firstly, employees feel that they have control over whether or not they reach their targets. Additionally, they can set well-defined, numerical goals for themselves to help push themselves towards meaningful thresholds. 2
Applying this theory to baseball performance, I hypothesize that players may strive to reach certain milestones in the year before reaching free agency. This would likely involve the most glamorous statistic that can lead to increased pay: home runs (HR). It is widely accepted that power is heavily rewarded in free agency, so players may aim to elevate their status as power hitters in advance of their free agency. Evidence of this would support the performance to statistics theory, named as such because players emphasize performance in significant statistics. This change in approach may not show up as increased totals in certain statistics, as those are limited by the player s true ability level, but rather in some relevant, underlying metrics. Players actively trying to increase HR totals would see increases in their rate of fly balls per ball in play (FB%) while not necessarily seeing the desired HR results. HR are highly correlated with future earnings, so this would be a reasonable way for players to attempt to earn additional salary in free agency. Prior research has explored the issues of performance pay, long-term incentives, and human response, both in a baseball context and in the general workforce. Research based on actual performance rather than expected output finds that contract year performance is more highly correlated with future salary and future performance than previous seasons output (Hochberg, 2011). This would support reward-seeking players decisions to attempt to maximize performance in contract seasons. Evidence from outside the baseball world has found strong human response to financial incentives, both short- and long-term (Park, 2012). In Section 2, I will outline the data to be used in my analysis. The subsequent section will outline the models and analyze the regression outputs, followed by a discussion of the importance of these results, both inside and outside of baseball. 2 Data I included in my analysis every player-season where a player recorded at least 300 plate appearances, over the nine seasons from 2003 to 2011. This eliminates those who played partial seasons due to injury or poor performance. This is one potential source of survivor bias, as those with very poor performance are less likely to reach that cutoff point. I am not overly concerned with this bias, but it is worth noting and revisiting at a later point. Every hitter was evaluated by his actual run production using a system of linear weights incorporated into a single metric: woba. Linear weights assigns a run value to every at bat result based on that event s average contribution to run scoring over years of data. This system is one of the most widely used in the baseball analytics community today. Using a projection system based on regressing recent performance to league average with a builtin age component, I compared each hitter s actual woba to their projected woba. The projections were created by Tom Tango s Marcel The Monkey Forecasting System, then compiled and provided in database format by Jeff Zimmerman of Baseball Heat Maps. The projection involves weighting performance in the three previous seasons, with the most recent weighted the most heavily, and then regressing to league average up to a certain amount of playing time. Players who received less playing time, and thus provide less data, will be 3
regressed more heavily. The age component is a simple multiplicative age adjustment based on a peak age of 29. Almost every non-pitcher in the dataset was also evaluated by their propensity for hitting fly balls. I ruled out of this analysis players who average fewer than five HR per season, as they are players who do not derive significant value from hitting home runs and thus are less likely to change their approach this way. Additionally, a spike in the number of fly balls hit by a non-home run hitter may not indicate a statistics-based change in approach but rather simple regression to the mean, because hitters classified as non-home run hitters likely hit very few fly balls previously. 3 Models and Results 3.1 Total Offense The first model compared how overall offensive performance, measured by woba, relates to projected offensive performance in contract years. I used a linear regression model woba = β 0 + β 1 (projwoba) + β 2 (ContractY ear) (1) where ContractY ear was a dummy variable with a value of 1 if a player will be a free agent following the season and 0 otherwise. Equation (1) is the simplest version of the analysis. The regression model yielded the following results: which gives an equation of Estimate S.E. t-value Pr(> t ) Intercept.0576.0078 7.35 0 projwoba.8138.0225 36.09 0 ContractYear -.0009.0017 -.52.603 Table 1: Results of regression in equation 1 woba =.0576 +.8138(projwOBA).0009(ContractY ear) (2) The negative sign on the ContractYear coefficient suggests that players are negatively impacted by impending free agency, but the lack of significance given by a two-sided p-value of.603 means there is not enough evidence to suggest performance relative to expectations, measured in terms of woba, is affected by a player s contract status at all. I had hoped to see an effect, but I am not surprised by the lack of such an effect. One issue is that the projections are partially based on historical performance by age via the built-in age component. Contract status and age are related to a certain extent, so the use of this component may mitigate the observed effect of contract status. The relationship is based on the distribution of free agent ages, which is sparse at ages below 28 but very populous between 29 and 32. 4
3.2 Home Run Hitters The other major model evaluates if players with home run ability hit more fly balls in their contract year. The regression model F B% = β 0 + β 1 (pastf B%) + β 2 (ContractY ear) (3) with pastf B% representing their career average fly-ball percentage to-date and ContractY ear the same dummy variable as in (1). The results of this regression model gave the following results: Estimate S.E. t-value Pr(> t ) Intercept.4219.0124 33.90 0 pastfb% -.0643.0321-2.01.0451 ContractYear.0089.0045 1.97.0489 which yields a regression equation of Table 2: Results of regression in equation 3 F B% = 0.4219.0643(pastF B%) +.0089(ContractY ear) (4) Let s start by analyzing the statistical significance in the model. Both slope parameters have p-values under a standard significance threshold of α =.05. However, they are just barely under that cutoff, so the evidence is not overwhelming. Regardless, it is sufficient to conclude that there is a relationship between a player s contract year status and their F B% when controlling for past F B%. While I am not particularly interested in the value of the coefficient on pastf B% in the model, the result is so surprising that it must be addressed. The model says that when controlling for ContractY ear, higher pastf B% values correspond to lower F B% values. Thus, players who hit lots of fly balls in the past will hit fewer fly balls than those who did not hit as many previously. This result seems so counterintuitive as to be almost unbelievable. The coefficient should, in theory, be relatively close to one, as past fly-ball tendencies are certainly predictive of future fly-ball tendencies to some extent. While the degree to which that is true may not be as high as I would have expected, an inverse relationship leads me to believe there are significant issues with the sample or data. A thorough investigation of the data did not yield any evidence that the numbers were incorrect. This is an almost unexplainable result in terms of baseball theory, but it leads me to question the usefulness of the results. The significance of the coefficients may not provide the result I had hoped for all along. While the p-value under.05 for the pastf B% coefficient suggests statistical significance relative to a null hypothesis that there is no relationship, statistical significance would appear 5% of the time even if no relationship existed. I believe that the true relationship is more likely to be positive than negative despite these results, and that this result is more likely the cause of some bias in the sample. 5
4 Discussion These two models do not support the hypothesis that players are affected by their upcoming free agency. Overall offensive performance relative to expectations, as measured by linear weights-driven woba, is not significantly different for players in their contract year. Players with some home run ability tend to slightly increase their fly-ball tendencies in contract years, but the increase is slight and the model s parameters are otherwise in question. Therefore, I hesitate to state conclusively that this is truly significant. Regardless of context, this provides no evidence of responses to incentives similar to quota-based systems. While this does not mean workers as a whole are unaffected by these systems, it suggests that evidence of such phenomena overall may not be found in Major League Baseball hitters. Professional baseball players do not appear to alter performance with free agency looming, despite previous findings in other fields suggesting that employees respond to performance-based incentives. Continued efforts on this research may yield more fruitful results, but thus far I am unenthusiastic about the evidence I have found. No evidence is still worth discussing, as differences from other fields are still meaningful. However, this could also be instructive about my methodology, and there may be value in expanding this analysis into other areas, such as pitching and defense, that I considered previously. 6
References Heywood, John S. (2006). Performance Pay and Job Satisfaction. Journal of Industrial Relations (0022-1856), 48 (4), p. 523. Hochberg, Daniel. The Effect of Contract Year Performance on Free Agent Salary in Major League Baseball. 2011. Konstantatos, Evan, Performance of Major League Baseball Players during Their Free Agency Season (2009). Honors Projects in Finance. Paper 8. Larkin, Ian (2012). The psychological costs of pay-for-performance: Implications for the strategic compensation of employees. Strategic Management Journal (0143-2095), 33 (10), p. 1194. McNabb, Robert (2007). The impact of varying types of performance-related pay and employee participation on earnings. International Journal of Human Resource Management (0958-5192), 18 (6), p. 1004. Park, Sanghee (2012). How and What You Pay Matters: The Relative Effectiveness of Merit Pay, Bonuses and Long-Term Incentives on Future Job Performance. Compensation and Benefits Review (0886-3687), 44 (2), p. 80. Stolovitch, Harold D., Richard E. Clark, and Steven J. Condly. Incentives, Motivation and Workplace Performance: Research and Best Practices. Rep. The International Society for Performance Improvement, 2002. 7