A Visible (Hot) Hand? Expert Players Bet on the Hot Hand and Win

Transcription

1 A Visible (Hot) Hand? Expert Players Bet on the Hot Hand and Win Joshua B. Miller and Adam Sanjurjo October 31, 2018 Abstract Since its inception, the hot hand fallacy literature has tended to focus on whether the hot hand exists, rather than the fitness of hot hand beliefs. We provide the first evidence that people here experienced practitioners can profitably exploit their hot hand beliefs. In particular, using the data from the original hot hand field study we find that players bets predict future outcomes. We use simulations to demonstrate how underpowered tests and misinterpreted effect sizes led the original study to the opposite conclusions JEL Classification Numbers: C12; C14; C18; C19; C91; D03; G02. Keywords: Hot Hand Fallacy; Betting; Beliefs. Fundamentos del Análisis Económico, Universidad de Alicante. Financial support from the Spanish Ministries of Education and Science and Economics and Competitiveness (ECO P) and Generalitat Valenciana (Research Projects Gruposo3/086 and PROMETEO/2013/037) is gratefully acknowledged. Both authors contributed equally, with names listed in alphabetical order. This draft has benefitted from helpful comments and suggestions from Jeremy Arkes, Phil Birnbaum, Vincent Crawford, Carlos Cueva, Mitchel Lichtman, Daniel Stone, and Tom Tango. All mistakes and omissions remain our own. 1

2 1 Introduction The hot hand fallacy refers to the belief that expert practitioners have a tendency to exhibit transient increases in their baseline ability the hot hand when they do not. Owing largely to the results of the seminal paper (Gilovich, Vallone, and Tversky 1985), the apparent robustness of its conclusion that the hot hand is a cognitive illusion, 1 and the sustained resistance of expert practitioners to these conclusions, 2 a consensus has emerged that the hot hand is a myth, and the associated belief a massive and widespread cognitive illusion (Kahneman 2011; Thaler and Sunstein 2008; Tversky and Gilovich 1989a,b). Accordingly, the hot hand fallacy has been used as a candidate explanation for various puzzles and behavioral anomalies in the domains of financial markets, 3 sports wagering, 4 casino gambling, 5 and lotteries. 6 In line with the long-standing consensus view of belief in the hot hand as a fallacy, and the subsequent literature often focusing on hot hand performance rather than beliefs, to our knowledge there exists no direct evidence that holding hot hand beliefs can be profitable. Here, we provide the first such evidence. Our finding of profitable hot hand beliefs is of particular relevance in light of the recent finding of strong evidence of hot hand performance (Miller and Sanjurjo 2018). 7 We test for the ability of decision makers to predict sequential performance outcomes in an incentivized field betting task. The data that we study is from Gilovich et al. (1985, GVT ), in which collegiate basketball players repeatedly bet on the outcomes of teammates (and their own) subsequent basketball shots. The data has several features that make it of more general interest: (i) the performance outcomes being bet on come from a naturalistic domain rather than an artificial laboratory environment, 8 (ii) while the outcomes that bettors formulate beliefs over are simple, bettors have an opportunity to inform these beliefs by observing rich information about the more complex underlying data generating process (player shooting), as in many real world environments 1 Many researchers have been so sure that the original Gilovich results were wrong that they set out to find the hot hand. To date, no one has found it (Thaler and Sunstein 2008). 2 Amos Tversky remarked on the stubbornness of practitioners to heed the evidence: I ve been in a thousand arguments over this topic. I ve won them all, and I ve convinced no one (Moskowitz and Wertheim 2011, p.229). By contrast, Tversky and Kahneman (1971) successfully convinced expert researchers that common research practices reflected a systematic misunderstanding of the sampling variability (Tversky and Kahneman 1974), though in practice their advice went unheeded (Gelman and Loken 2014). 3 See, e.g. Barberis and Thaler (2003); De Bondt (1993); De Long, Shleifer, Summers, and Waldmann (1991); Kahneman and Riepe (1998); Loh and Warachka (2012); Malkiel (2011); Rabin and Vayanos (2010) 4 See, e.g. Arkes (2011); Avery and Chevalier (1999); Brown and Sauer (1993); Camerer (1989); Durham, Hertzel, and Martin (2005); Lee and Smith (2002); Paul and Weinbach (2005); Sinkey and Logan (2013) 5 See, e.g. Croson and Sundali (2005); Narayanan and Manchanda (2012); Smith, Levere, and Kurtzman (2009); Sundali and Croson (2006); Xu and Harvey (2014) 6 See, e.g. Galbo-Jørgensen, Suetens, and Tyran (2016); Guryan and Kearney (2008); Yuan, Sun, and Siu (2014) 7 The existence of hot hand performance suggests that peoples hot hand beliefs may be profitable, but does not necessitate that they are, e.g. people may detect hot hand performance in moments when it is not present. 8 An important feature of GVT s contribution was that it constituted the first field evidence to corroborate existing findings from laboratory and survey work on the misperception of randomness (Falk 1981; Tversky and Kahneman 1974; Wagenaar 1972). 2

3 such as betting and financial markets, and (iii) the bettors are experienced within this domain, so their performance can be expected to be more informative than that of novices. In their analysis of the betting data Gilovich et al. (1985) find no evidence that players can predict their teammates, or their own, shot outcomes. This finding has been referred to by Gilovich as...the most important bit of evidence against the hot hand. 9 However, we observe that the authors empirical approach is critically underpowered because it relies solely on a large number of separate small-sample individual level tests. Further, we observe that the seemingly low correlations between bets and respective shot outcomes that GVT find are in fact not low. In particular, we benchmark observed correlation sizes to what one would expect from various models of hot hand shooting (by simulation), and find that they are in line with what one would expect from a bettor who perfectly detects the shooter s state, i.e. his/her current probability of success. In light of these observations, we perform an improved analysis of the original GVT betting data in which we greatly increase statistical power by: (i) pooling data across bettors, and (ii) performing an analysis on the collection of individual test results. In contrast with the conclusions of GVT s study, we find that bettors are successful at predicting shot outcomes, and that the effect sizes are considerable. Beyond our results constituting the first evidence of profitable hot hand beliefs, they simultaneously reverse the strongest remaining evidence from GVT that expert hot hand beliefs are mistaken. 10 This reversal is notable given that: (i) GVT s conclusions have had a large influence on the thinking of researchers and laypeople alike, and (ii) their use of betting data in a field setting to study the empirical relevance of expert practitioners hot hand beliefs remains unique to the literature. 11 Because the human perception of sequential dependence applies to a broad range of important decision making domains, and the accumulating evidence of sometimes considerable hot hand performance (Arkes 2010; Bocskocsky, Ezekowitz, and Stein 2014; Green and Zwiebel 2017; Miller and Sanjurjo 2014, 2015, 2018; Yaari and Eisenmann 2011), we view the results presented here as an important step forward in the pursuit of better understanding the nature, and accuracy, of hot hand beliefs. Section 2 presents the GVT betting design, Section 3 the original analysis and its limitations, and Section 4 our analysis of the original betting data. 9 The more complete quote is as follows:...what I ve always considered the most important bit of evidence against the hot hand... the fact that players in our studies could not predict before taking a shot whether they were more or less likely to make it. The content is from a 2015 letter that Gilovich granted us permission to share publically. 10 GVT also administer an unincentivized survey in which 8 NBA players and a coach express an almost unanimous belief in the hot hand. However, these beliefs can no longer be considered fallacious (Miller and Sanjurjo 2018). See Appendix C for more on GVT s survey of beliefs. 11 Rao (2009b) performed a study in which college students (rather than players) bet on shot outcomes in video excerpts of NBA telecasts (rather than in a live, controlled, field shooting setting). The author does not report whether subjects are successful or not at the prediction task, as the analysis is instead focused on measuring to what extent subjects hold hot hand or gambler s fallacy beliefs, as a function of streak length. 3

4 2 The Betting Data In GVT s betting task players predicted the shot outcomes of GVT s controlled shooting study. In the shooting study each of 26 players from Cornell University s intercollegiate men and women s basketball teams shot 100 times from fixed distance, but switching locations after each shot. For each player, the experimenters chose the distance from which they anticipated that he/she would make roughly half of the shots. In the betting task both the shooter and an observer (another player that was not currently shooting) bet on the outcome of each upcoming shot by either placing a high or low stakes bet on hit. Under a high stakes bet they earn +$0.05 for a hit and $0.04 for a miss, while under a low stakes bet they earn +$0.02 for a hit and $0.01 for a miss. Therefore, a risk neutral bettor should bet high if she believes that the probability of a hit is greater than.5, and low otherwise. Accordingly, the authors reasonably interpret high bets as hit predictions and low bets as miss predictions. 3 The original analysis and its limitations GVT find that while experts predictions are positively correlated with the outcome of the previous shot (.40 for shooters and.42 for observers, on average), they are relatively uncorrelated with the outcome of the shot that is bet on (.02 for shooters and.04 for observers, on average). Further, they find no evidence of any exemplary individual predictors. Thus, in sum, the authors find that experts are unsuccessful in predicting the shooters performance (p. 309). While their analysis and interpretation of betting data provide a valuable first step, the authors misinterpret their correlation measures of predictability. In particular, because the hot hand is likely a relatively infrequent phenomenon, even if a bettor is highly skilled at detecting when a player is hot, the correlation of the bettor s predictions with the shot outcomes is expected to be low. To illustrate, imagine a hypothetical shooter who shoots 12 percentage points better in his hot state than in his normal state, and is in the hot state on 10 percent of his shots. Such a hot hand effect would be considerable for basketball standards, and is roughly equal to the conservative estimate of the (average) hot hand effect reported in Miller and Sanjurjo (2018). 12 Now, suppose that there is a predictor who can perfectly detect the shooter s hot hand whenever it occurs, so always bets high when the shooter is in the hot state and low when the shooter is not in the hot state. In this extreme case the expected correlation between predictions and respective shot 12 The difference between the median three point shooter and the top three point shooter in the NBA season was 12 percentage points, per ESPN, NBA Player 3-Point Shooting Statistics /stat/3-points [accessed September 24, 2016]. 4

5 outcomes would (only) be around.07. Of course, if the predictor were instead less than perfect at detecting the hot state, then the expected correlation would be even lower. 13 The reason why this correlation measure is expected to lead to a surprisingly low underestimate of prediction ability is closely related to Stone (2012) s work on measurement error and the hot hand. 14 In order to gauge the statistical power of the original tests we conduct simulations in Appendix A which reveal that the correlation between shot outcomes and the predictions of a perfect detector would have just a 10 percent chance of being significant at the 5 percent level. 15 By contrast, if predictions from many predictors were instead pooled, say n = 52 predictors (to match GVT s design), then the correlation between predictions and shot outcomes would be significant with near-certainty. 16 Observe that the.07 expected correlation of perfect detectors is not vastly different from the average correlation of.04 of GVT s observers. This suggests that the original betting data may contain evidence that players can successfully predict outcomes at rates better than chance evidence that has gone undetected because the previous analysis used only underpowered individuallevel tests. 4 An improved analysis of the betting data Before analyzing the raw data, a quick inspection of GVT s Table 6 (p. 310) reveals that 34 out of 52 bettors exhibit a positive correlation between predictions and shot outcomes, which is significantly more than expected by chance (p <.05, binomial test). Further, 8 bettors out of 52 (4 observers and 4 shooters) exhibit a statistically significant (p <.05) positive correlation between predictions and shot outcomes. 17 An analysis of the collection of these individual test results reveals that 8 significant findings would be extremely unlikely for bettors who are predicting at random (p <.01, binomial test). 13 The correlation follows by simple calculation. In particular, call a successful shot outcome hit, the hot state hot, and a bet on success bet = hot. Then, set the following values: (i) P(hit) =.5, which is GVT s design goal, (ii) P(hot) =.1, which is roughly equal to the expected relative frequency of streaks of three hits (.13), and (iii) P(hit hot) P(hit hot) =.12, which is equal to Miller and Sanjurjo (2018) s (conservative) estimate of the hot hand effect. Together this implies that P(hit hot) =.608 and P(hit hot) =.488. It follows that ρ(bet, hit) = ρ(hot, hit) = Cov(hot,hit) =.072. See Appendix A for a more complete treatment. 14 σ(hot)σ(hit) In particular, Stone (2012) showed that the serial correlation in hits is expected to be far lower than the serial correlation in the player s hit probability, as a hit is only a noisy measure of a player s underlying probability of success. In this case we have shown that the correlation between bet hit and hit is expected to be far lower than the correlation between bet hit and the player s hit probability, for the same reason. 15 In Appendix A, using a Hidden Markov Chain model of the hot hand, we extend this simple illustration in order to explore how expected correlations and statistical power depend on the size of the hot hand, its frequency, and predictor ability. The simulations use 10,000 repetitions. 16 For the pooled case only 3 of 10,000 simulated datasets returned insignificant results. 17 We are not the first to observe this pattern in GVT s table, as Wardrop (1999) mentions that... obtaining four... significant results [for shooters] is noteworthy (p. 7). 5

6 Correlation predictor +/- S.E. 95% CI Figure 1: The correlation and associated confidence intervals for each of the 44 sequences of predictions. The standard errors are approximated using Fischer s z-transform. While GVT assert that the correlations observed among these 8 players, which fall between.20 and.24, are quite low (p. 309), our example with the perfect detector in Section 3 indicates that this is not the case. 18 In fact, in Appendix A we report, for example, that if a hypothetical player were to shoot 30 percentage points better when hot, and have the hot hand on 15 percent of his/her shots, then the expected correlation between shot outcomes and the predictions of a perfect detector would (only) be.20. Relative to the individual tests performed in the original study, pooling data across all bettors would allow for more powered statistical tests. To this end, we collect the available raw data from the original betting task, which consists of all of the bets placed on 22 of the 26 shooters (44 sequences of predictions total). 19 In Figure 1, for each of the 44 sequences of predictions, we plot the correlation along with its associated confidence interval. Upon pooling the bets of all participants, we find that the average correlation between predictions and shot outcomes is ˆρ =.07, and highly significant (p <.001, permutation test with predictor/shooter stratification). 20 While the correlation is a unitless measure, we can get a sense 18 An alternative argument for why these correlations are not small, is that they are consistent with fan beliefs (Wardrop 1999). 19 We thank Tom Gilovich for providing us with the raw data. We were informed that the data for the remaining four shooters could not be located. The 44 sequences of predictions and outcomes were entered by two independent coders, and cross-validated. 20 In the case of shooters predicting their own shots, the average correlation is ˆρ =.070 (p <.01). In the case of the 6

7 of its magnitude by examining the increase in shooting percentage when a bettor bets on a hit, rather than a miss. 21 The actual increase across bettors is 7.7 percentage points (p <.001, S.E. = 1.6), 22 which is comparable in magnitude to an NBA shooter going from slightly above average to elite in three point shooting percentage. 23,24 In Appendix B we briefly discuss possible mechanisms for bettors ability to successfully predict shot outcomes. 5 Conclusion We provide the first evidence of profitable hot hand beliefs. The domain is a naturalistic field setting in which experts bet on simple outcomes generated from a complex process about which they observe rich information, as in real world betting and financial markets. In addition to our results being novel, they also reverse the canonical study s strongest remaining evidence that belief in the hot hand is fallacious. Hot hand fallacy studies have often sought to determine whether the widespread belief in the hot hand is justified or not. However, in light of recent work demonstrating that belief in the hot hand is justified (Arkes 2013; Bocskocsky et al. 2014; Green and Zwiebel 2017; Miller and Sanjurjo 2014, 2018; Stone 2012), it is natural for hot hand research to now shift more towards understanding the fitness of peoples hot hand beliefs. We view the present study as providing an important step forward in this direction. predictions of an observer, the average correlation is ˆρ =.066 (p <.01). There is a slight downward bias in these estimates of approximately due the fact that correl(bet t, hit t 1) For a single predictor, the correlation is approximately equal to the OLS regression coefficient ˆβ p from the linear probability model hit t = β 0 + β p bet t + ɛ t, because the variance in predictions is sufficiently close to the variance in shot outcomes for most bettors. 22 These results come from a linear probability model with fixed effects and robust standard errors. As mentioned in Footnote 20, correl(bet t, hit t 1).41. Thus, the errors in the model are correlated, and the coefficient is slightly downward biased, by approximately ESPN, NBA Player 3-Point Shooting Statistics /stat/3- points [accessed September 24, 2016]. 24 An alternative intuition for the magnitude of the average correlation can be derived from the simulations reported in Appendix A. In particular, one can see in Figure 2 that if a predictor were to identify the shooter s state 50 percent of the time, and if the shooter were to shoot 30 percentage points better when in the hot (rather than normal) state and be in the hot state on 15 percent of his shots, then the expected correlation would be.07. 7

8 References Aharoni, G. and O. H. Sarig (2011): Hot hands and equilibrium, Applied Economics, 44, Albert, J. (1993): Comment on A Statistical Analysis of Hitting Streaks in Baseball by S. C. Albright, Journal of the American Statistical Association, 88, Arkes, J. (2010): Revisiting the Hot Hand Theory with Free Throw Data in a Multivariate Framework, Journal of Quantitative Analysis in Sports, 6. (2011): Do Gamblers Correctly Price Momentum in NBA Betting Markets? Journal of Prediction Markets, 5, (2013): Misses in Hot Hand Research, Journal of Sports Economics, 14, Attali, Y. (2013): Perceived Hotness Affects Behavior of Basketball Players and Coaches, Psychological Science, forthcoming. Avery, C. and J. Chevalier (1999): Identifying Investor Sentiment from Price Paths: The Case of Football Betting, Journal of Business, 72, Barberis, N. and R. Thaler (2003): A survey of behavioral finance, Handbook of the Economics of Finance, 1, Bocskocsky, A., J. Ezekowitz, and C. Stein (2014): The Hot Hand: A New Approach to an Old Fallacy, 8th Annual Mit Sloan Sports Analytics Conference. Brown, W. A. and R. D. Sauer (1993): Does the Basketball Market Believe in the Hot Hand? Comment, American Economic Review, 83, Camerer, C. F. (1989): Does the Basketball Market Believe in the Hot Hand,? American Economic Review, 79, Cao, Z. (2011): Essays on Behavioral Economics, Ph.D. thesis, Oregon State University. Croson, R. and J. Sundali (2005): The Gamblers Fallacy and the Hot Hand: Empirical Data from Casinos, Journal of Risk and Uncertainty, 30, De Bondt, W. P. (1993): Betting on trends: Intuitive forecasts of financial risk and return, International Journal of Forecasting, 9, De Long, J. B., A. Shleifer, L. H. Summers, and R. J. Waldmann (1991): The Survival of Noise Traders In Financial-markets, Journal of Business, 64, Durham, G. R., M. G. Hertzel, and J. S. Martin (2005): The Market Impact of Trends and Sequences in Performance: New Evidence, Journal of Finance, 60, Falk, R. (1981): The perception of randomness, Proceedings of the fifth international conference for the psychology of mathematics education, 1,

9 Galbo-Jørgensen, C. B., S. Suetens, and J.-R. Tyran (2016): Predicting Lotto Numbers A natural experiment on the gamblers fallacy and the hot hand fallacy, Journal of the European Economic Association, 14, working Paper. Gelman, A. and E. Loken (2014): The Statistical Crisis in Science, American Scientist, 102, 460. Gilovich, T., R. Vallone, and A. Tversky (1985): The Hot Hand in Basketball: On the Misperception of Random Sequences, Cognitive Psychology, 17, Green, B. S. and J. Zwiebel (2017): The Hot Hand Fallacy: Cognitive Mistakes or Equilibrium Adjustments? Management Science, working Paper. Guryan, J. and M. S. Kearney (2008): Gambling at Lucky Stores: Empirical Evidence from State Lottery Sales, American Economic Review, 98, Kahneman, D. (2011): Thinking, Fast and Slow, Farrar, Straus and Giroux. Kahneman, D. and M. W. Riepe (1998): Aspects of Investor Psychology: Beliefs, preferences, and biases investment advisors should know about, Journal of Portfolio Management, 24, Lee, M. and G. Smith (2002): Regression to the mean and football wagers, Journal of Behavioral Decision Making, 15, Loh, R. K. and M. Warachka (2012): Streaks in Earnings Surprises and the Cross-Section of Stock Returns, Management Science, 58, Malkiel, B. G. (2011): A random walk down Wall Street: the time-tested strategy for sucessful investing, New York: W. W. Norton & Company. Miller, J. B. and A. Sanjurjo (2014): A Cold Shower for the Hot Hand Fallacy, Working Paper. Available at SSRN: (2015): Is the Belief in the Hot Hand a Fallacy in the NBA Three Point Shootout? Working Paper. Available at SSRN: (2018): Surprised by the Hot Hand Fallacy? A Truth in the Law of Small Numbers, Econometrica, forthcoming. Moskowitz, T. and L. J. Wertheim (2011): Scorecasting: The Hidden Influence Behind How Sports are Played and Games are Won, New York: Crown Archetype. Narayanan, S. and P. Manchanda (2012): An empirical analysis of individual level casino gambling behavior, Quantitative Marketing and Economics, 10, Neiman, T. and Y. Loewenstein (2011): Reinforcement learning in professional basketball players, Nature Communications, 2:569. Paul, R. J. and A. P. Weinbach (2005): Bettor Misperceptions in the NBA: The Overbetting of Large Favorites and the Hot Hand, Journal of Sports Economics, 6,

10 Rabin, M. and D. Vayanos (2010): The Gamblers and Hot-Hand Fallacies: Theory and Applications, Review of Economic Studies, 77, Rao, J. M. (2009a): Experts Perceptions of Autocorrelation: The Hot Hand Fallacy Among Professional Basketball Players, Working Paper. (2009b): When the Gambler s Fallacy becomes the Hot Hand Fallacy: An Experiment with Experts and Novices, Working Paper. Sinkey, M. and T. Logan (2013): Does the Hot Hand Drive the Market? Eastern Economic Journal, Advance online publication, doi: /eej Smith, G., M. Levere, and R. Kurtzman (2009): Poker Player Behavior After Big Wins and Big Losses, Management Science, 55, Stone, D. F. (2012): Measurement error and the hot hand, The American Statistician, 66, 61 66, working paper. Sundali, J. and R. Croson (2006): Biases in casino betting: The hot and the gamblers fallacy, Judgement and Decision Making, 1, Thaler, R. H. and C. R. Sunstein (2008): Nudge: Improving Decisions About Health, Wealth, and Happiness, Yale University Press. Tversky, A. and T. Gilovich (1989a): The cold facts about the hot hand in basketball, Chance, 2, (1989b): The Hot Hand : Statistical Reality or Cognitive Illusion? Chance, 2, Tversky, A. and D. Kahneman (1971): Belief in the Law of Small Numbers, Psychological Bulletin, 2, (1974): Judgment under Uncertainty: Heuristics and Biases, Science, 185, Wagenaar, W. A. (1972): Generation of Random Sequences by Human Individuals: A Critical Survey of the Literature, Psychological Bulletin, 77, Wardrop, R. L. (1999): Statistical Tests for the Hot-Hand in Basketball in a Controlled Setting, Unpublished manuscript, 1, Xu, J. and N. Harvey (2014): Carry on winning: The gambler s fallacy creates hot hand effects in online gambling, Cognition, 131, Yaari, G. and S. Eisenmann (2011): The Hot (Invisible?) Hand: Can Time Sequence Patterns of Success/Failure in Sports Be Modeled as Repeated Random Independent Trials? PLoS One, 6, Yuan, J., G.-Z. Sun, and R. Siu (2014): The Lure of Illusory Luck: How Much Are People Willing to Pay for Random Shocks, Journal of Economic Behavior & Organization, forthcoming. 10

11 A Appendix: Simulations of expected predictor performance, and statistical power We extend the example from Section 3. Let the player s shot outcomes Y i be determined by their probability of success P(Y t = 1), which in turn is determined by the shooter s latent state, where the probability of hitting a shot in the normal state (n) is p n = P(Y t = 1 S t = n), and in the hot state p h = P(Y t = 1 S t = h). The latent state process is governed by the following Markov chain: Q := ( 1 qnh q nh q hn 1 q hn where q nh is the probability going from the normal state to the hot state, and q hn is the probability of going from the hot state back to the normal state. 25 The initial state is determined by the Markov chain s stationary distribution π = (π n, π h ), which satisfies πq = π. In the following simulations we consider different hot hand effect sizes d := (p h p n ) {.1,.2,.3,.4,.5}, and fractions of time that the shooter is in the relatively rare hot state π h {.05,.1,.15,.2,.25}, under the constraints that q nh =.05 and, following GVT s design, that the overall probability of success p is.5, where p n and p h are defined so that p = p n + π h d = We assume that bettors (henceforth predictors ) make a guess about the shooter s state G t {n, h}, and that their bets on success X t accord with their guesses, i.e. X t := 1 [Gt=h]. Finally, we consider different values for the predictor s ability a to detect the shooter s state ( hot or not hot ), P(G t = S t ) = a {0,.25,.5,.75, 1}, so that: S t G t = G w.p. a w.p. 1 a where G {n, h} represents a uniform random guess, as the bettor is indifferent given GVT s design goal of selecting a distance for each player from which he/she would be expected to make 50 percent of the shots. In each panel of Figure 2 we report the expected correlation (y-axis) between predictions and shot outcomes, for a predictor whose ability to detect (x-axis) the shooter s state varies from zero to perfect, as the frequency that the shooter is in the hot state changes. In addition, across panels we vary the strength of the hot hand effect. The figure makes clear that the correlations of around.20 that correspond to GVT s 8 exemplary predictors cannot reasonably be considered quite low (p. 309). For example,.20 is at least as large as the expected correlations that would result from 25 See Albert (1993) for a similar approach to modeling the hot hand. 26 Note that the transition probability from the hot to normal state is determined by q hn = πn π h q nh. ) 11

12 Hot Hand of +10pp Hot Hand of +20pp Hot Hand of +30pp Expected Correlation Expected Correlation Expected Correlation Hot Hand of +40pp Hot Hand of +50pp Expected Correlation Expected Correlation % of shots in hot state Figure 2: The expected correlation as a function of predictor ability, for various sizes of the hot hand, its relative frequency. a predictor who perfectly detects the hot hand, i.e. a = 1, for nearly all models that contain substantial (and frequent) hot hand effects up to and including 30pp. Further, if predictors are less than perfect at detecting the hot hand, i.e. a < 1, then a.20 correlation can be consistent with even larger hot hand effects. In each panel of Figure 2 we report the power (y-axis) of the test corresponding to the correlation between predictions and shot outcomes, for an individual predictor whose ability to detect (x-axis) the shooter s state again varies from zero to perfect, as the frequency that the shooter is in the hot state changes. In addition, as before, we vary the strength of the hot hand effect across panels. The figure makes clear that the probability of observing a significant correlation (p <.05, Spearman test) exceeds.80 in only the most extreme models of hot hand shooting and predictor ability. Relative to the limited power of individual tests reported in Figure 3, one can increase power greatly by performing an analysis of the collection of individual test results. Accordingly, in Figure 4 we report the probability of observing a statistically significant number (p <.05, Binomial test) of statistically significant correlations (Spearman test) among 52 tests (following GVT s design), for the same set of models of the hot hand and predictor ability reported in Figures 2 and 4. The figure makes clear that the power of the this approach is substantially greater than the power of individual tests. For example, power reaches acceptable levels (.80) for models in which hot hand effects are at least +20pp, the frequency of hot hand shooting is not too low (> 10 percent of the 12

13 Hot Hand of +10pp Hot Hand of +20pp Hot Hand of +30pp Hot Hand of +40pp Hot Hand of +50pp % of shots in hot state Figure 3: The probability of observing a statistically significant (p <.05) correlation in a single sequence as a function of predictor ability, for various sizes of the hot hand, and its relative frequency. time), and predictor abilities are strictly greater than.5. A way of increasing power even further, relative to analyzing the collection of individual tests, is to pool sequences of predictions across predictors and test for a significant average correlation. We refrain from providing a figure with the corresponding simulation results, as power increases to near-certainty for most combinations of parameters. 13

14 Hot Hand of +10pp Hot Hand of +20pp Hot Hand of +30pp Hot Hand of +40pp Hot Hand of +50pp % of shots in hot state Figure 4: The probability of observing a statistically significant (p <.05) number of statistically significant correlations between predictions and shot outcomes within 52 sequences, as a function of predictor ability, for various sizes of the hot hand, and its relative frequency. 14

15 B Appendix: A discussion of mechanisms that can account for profitable betting We briefly discuss alternative accounts of how the bettors we study, who believe in the hot hand, are able to predict outcomes successfully. Given the rich information available for bettors to base their beliefs on in GVT s natural field betting task, any insights that we can glean from this environment could in turn be useful for understanding the fitness of hot hand beliefs more generally, e.g. in real world betting and investment environments. A simple candidate explanation for bettors ability to successfully predict shot outcomes is that a default belief in the hot hand happens to be adaptive. In particular, because there happens to be hot hand shooting in the data, mechanically betting on success after recent success will lead to bets that predict shot outcomes at rates that are better than chance. 27 While this explanation is appealing for its simplicity, on its own it cannot explain a recent finding that players can successfully predict the relative strength of the hot hand among their teammates (Miller and Sanjurjo 2014). 28 Further, bettors pattern of bets indicates that their success cannot be attributed solely to such an undiscerning heuristic. To illustrate why this is the case, we consider a model that fits bettors average tendency to bet on hit as a function of the preceding streak of like outcomes, e.g. three hits in a row or two misses in a row, and find that it yields bets that do not correlate with shot outcomes. In particular, first we fit a logistic model of P(bet t ) based on recent shot outcomes with bettor shooter fixed effects. Second, for each of the 44 sequences of shot outcomes, we generate the predicted ˆP(bet t ) using the fitted model (with fixed effects). Finally, we measure the fitted model s ability to predict hits by calculating the average of the estimate produced by performing 10,000 repetitions of the following procedure: (i) for each of the 44 sequences of shot outcomes, simulate a corresponding sequence of bets from the fitted model, i.e. the {ˆb t } are independent Bernoulli with P(ˆb t = 1) = ˆP(bet t ), (ii) estimate the increase in shooting percentage when the simulated bet predicts hit, rather than miss, i.e. the pooled analysis from Section To illustrate how this could occur, suppose that one were to bet hit every time a shooter hit 3 shots in a row, miss every time a shooter missed 3 shots in a row, and otherwise bet hit (miss) if the shooter s overall (unknown) field goal percentage is greater (less) than.5. With this heuristic, as one might anticipate given the findings of Miller and Sanjurjo (2018), on average, shooters would perform significantly better (+5pp) just after a hit bet than just after a miss bet. 28 For example, Miller and Sanjurjo (2014) find semi-professional players rankings to be highly correlated with teammates actual increases in performance when on hit streak in a shooting experiment that the rankers do not observe, yielding an average correlation of (p <.0001; where 1 is the rank of the shooter with the perceived largest percentage point increase) in this out-of-sample test. 15

16 For the estimated model, the (log-odds) linear index is given by X t β =1.31[MH] t [MHH] t [HHH] t [HM] t 0.58[HMM] t 0.99[MMM] t + fixed effects where [MH] t := miss t 2 hit t 1, [MHH] t := miss t 3 hit t 2 hit t 1, and so on. 29 Observe that bettors are more likely to bet on hit (miss) given a recent streak of hits (misses). In particular, the estimated coefficients indicate that relative to the no-streak baseline, the odds in favor of betting on hit increase by a factor of 3.7 e 1.31 upon the bettor observing exactly one hit, 8 e 2.09 upon observing two consecutive hits, 11 e 2.4 upon observing three consecutive hits (or more), and so on. To test whether these average betting tendencies yield a successful strategy for prediction we estimate prediction ability using the following linear probability model with bettor shooter fixed effects: hit t = β 0 +β 1ˆbt. Based on 10,000 repetitions, we find that the average estimated prediction ability is ˆβ 1 = (S.E. = ). 30,31 Thus, average betting tendencies based on recent streak patterns are not successful at predicting shot outcomes. 32 An alternative explanation for bettors observed prediction success is that beyond holding default beliefs in their teammates tendency to get the hot hand, basketball players can also recognize the hot hand as it occurs, and respond to it. 33 However, doing so could be difficult, as the ability to detect a shift in a shooter s underlying probability of success would require information beyond the outcomes of recent shots, given that a few observations of binary data is simply too weak a signal to confidently diagnose a shift in a player s probability of success. 34 Nevertheless, as experienced 29 If bettors condition their bets on recent shot outcomes then MLE assumptions are violated. In particular, because [MHH] t = 1 implies [MH] t 1 = 1, and so on, bet t 1 will be correlated with the regressors, rendering the errors non-i.i.d. Nevertheless, we find that the impact on coefficient estimates is negligible. We do this by repeatedly re-estimating the model with sequence-level fixed effects, after simulating bets based on logistic probabilities that are determined by the fitted parameters (with a zero-log-odds benchmark for all bettors). This yields coefficient estimates that are within (log-odds units) of the underlying data-generating process. 30 While correl(ˆb t, hit t 1) =.43, rendering the errors correlated, simulations reveal that the regression coefficient is only slightly (downward) biased, i.e. by approximately On the other hand, if betting is modeled without bettor shooter fixed effects, then the fitted (logistic) probabilities ˆP(bet t) produce bets ˆb t that do correlate with shot outcomes hit t (on average), though with an effect size much smaller than that found when using the real betting data. In particular, for the linear probability model with bettor shooter fixed effects, hit t = β 0 + β 1ˆbt, ˆβ1 = 0.01 (S.E. = , 10,000 repetitions). 32 A potential explanation for why actual bets correlate with shot outcomes but modeled bets do not is that modeled bets are constrained to use the same betting rule (heuristic) on every shot sequence, whereas actual bettors are not. Nevertheless, it is possible for certain consistently applied heuristics to be successful, as illustrated in footnote There is abundant evidence that basketball players and coaches react to recent shooting performance (Aharoni and Sarig 2011; Attali 2013; Bocskocsky et al. 2014; Cao 2011; Neiman and Loewenstein 2011; Rao 2009a). Whether they over (or under) react is more difficult to infer given the strategic complexity involved in game settings. See Green and Zwiebel (2017) for recent evidence in baseball. 34 To illustrate, suppose that a player s hit rate is.6 in the hot state,.4 in the normal state, and that the player is in the hot state on 20 percent of her shots. The likelihood of her hitting three in a row is (.6/.4) times higher when she is in the hot state. Thus, upon observing three hits in a row, the odds in favor of the player being in the hot state must increase by this factor. Nevertheless, because the prior odds are just 1:4 in favor, the posterior 16

17 teammates of the shooters, GVT s bettors could be uniquely positioned to learn how to associate ancillary informational cues, such as shooting technique or body language, with subsequent shooting success. C Appendix: GVT s survey on hot hand beliefs In addition to their betting task, GVT collected (unincentivized) survey data that indicated an almost unanimous belief in the hot hand by coaches, players, and fans. Though the survey questions were mostly qualitative, in light of GVT s finding that the hot hand did not exist, GVT could conclude that the hot hand was a powerful and widely shared cognitive illusion (Gilovich et al. 1985, p. 313). Specifically, in an interview of 8 NBA basketball players Gilovich et al. (1985) find that: Most of the players (six out of eight) reported that they have on occasion felt that after having made a few shots in a row they know they are going to make their next shot that they almost can t miss. Five players believed that a player has a better chance of making a shot after having just made his last two or three shots than he does after having just missed his last two or three shots. (Two players did not endorse this statement and one did not answer the question.) Seven of the eight players reported that after having made a series of shots in a row, they tend to take more shots than they normally would. All of the players believed that it is important for the players on a team to pass the ball to someone who has just made several (two, three, or four) shots in a row. Five players and the coach also made numerical estimates. Five of these six respondents estimated their field goal percentage for shots taken after a hit (mean: 62.5%) to be higher than their percentage for shots taken after a miss (mean: 49.5%). In light of recent findings of robust hot hand shooting across contexts (see Miller and Sanjurjo [2018]) such expressed beliefs can no longer be considered fallacious; instead they now appear reasonable. 35 odds become 3.38:4, indicating slightly less than fair odds of detecting a true hot hand. 35 For specific critiques of GVT s survey, and for discussion of the general issue of eliciting operational beliefs from unincentivized surveys, see Rao (2009a) and Green and Zwiebel (2017). 17