Additional Material for Harnessing Optimism: How Eliciting Goals Improves Performance (Sackett, Wu, White, & Markle, 2014)

Transcription

1 Additional Material for Harnessing Optimism: How Eliciting Goals Improves Performance (Sackett, Wu, White, & Markle, 2014) Overview This document provides methodological and analytical detail to supplement the main text of Sackett, Wu, White, and Markle (2014a). This is the most comprehensive and detailed version of this document available, but note that it has not undergone a formal peer-review process. A condensed, peer-reviewed version appears as the Supplemental Online Materials accompanying the main text of Sackett et al. (2014a). We endorse Simmons, Nelson, and Simonsohn s (2011) recommendation to present a full set of measures collected and experimental conditions (Requirements 3 & 4) and robustness analyses to address issues related to control variables, outliers, and data exclusions (see Requirements 5 & 6). We also support Simonsohn s (2013) proposal to publish experimental data and have made our dataset available for public download (Sackett, Wu, White & Markle, 2014b). However, the design of this study and the readily accessible nature of marathon finishing times create some serious challenges to full posting of raw data while also protecting participants privacy. It is therefore necessary to omit some data and to disguise some other data. See Sackett et al. (2014b) for details on the principles and procedures for omitting and disguising our data. Marathon Running The marathon is a long-distance running event that covers 26 miles and 385 yards, or 42,195 meters. In 2013, there were 541,000 finishing times for U.S. marathons, compared to

2 143,000 in 1980, 224,000 in 1990, 353,000 in 2000, and 507,000 in 2010 (Running USA, 2014). 1 The median finishing time across all U.S. marathons in 2013 was 4 hours, 41 minutes, and 38 seconds for women and 4 hours, 16 minutes, and 24 seconds for men (Running USA, 2014). World records at the beginning of 2014 were 2 hours, 3 minutes, and 23 seconds for men, and 2 hours, 15 minutes, and 25 seconds for women. Marathon running is popular throughout the world. In 2013, 38,690 runners completed the Paris (France) Marathon (WORLD S LARGEST MARATHONS, 2014), 35,308 runners completed the Tokyo (Japan) Marathon (WORLD S LARGEST MARATHONS, 2014), 7,275 runners completed the Buenos Aires (Argentina) Marathon (CLASIFICACION 42K NANDU 2013 V06, 2013), and 6,850 runners completed the Melbourne (Australia) Marathon (MarathonGuide.com, 2013). Additional Details on Methodology Recruitment We solicited participants through marathon organizers, organized running groups, marathon training programs, message boards, and athletic shops. The study was advertised as a study on the relationship between marathon performance and satisfaction. Over the course of three years ( ), we recruited prospective marathon runners for our targeted marathons, specifically Boston, 2008; Chicago, ; Grandma s, 2008; Los Angeles, 2008; Marine Corps, ; New York City, 2009; Portland (OR), 2007; Rock n Roll (San Diego), 2008; and Twin Cities, Each of these marathons was one of the 15 largest U.S. marathons of its year, ranging from 6,875 finishers for the 2008 Grandma s Marathon 1 Since some runners complete multiple marathons, the number of individuals who finished one or more marathons in 2013 is less than 541, We also recruited marathoners from the 2007 Honolulu, 2007 Philadelphia, and 2008 San Francisco Marathons. Because these three marathons yielded only 38 participants total (20 of which had usable data), we dropped these participants from our analyses. Including these participants does not meaningfully change the results presented in the main text. The mean difference in finishing times between conditions for experienced runners changes from 6.75 minutes to 6.35 minutes with no controls, t(964) = 1.92, p =.055, and from 8.13 minutes to 7.85 minutes with controls, t(933) = 2.80, p =

3 (MarathonGuide.com, 2009) to 38,557 finishers for the 2007 New York City Marathon (MarathonGuide.com, 2008). The number of total finishers and number of survey participants for each marathon is given in Table 1. Participants registered online for our study by providing their name, address, and the marathon that they were planning to run. They were then randomly assigned to one of 4 conditions in a 2 (pre-marathon survey: goal-not-asked vs. goal-asked) x 2 (post-marathon survey: early vs. late) design. Participants were entered into a random drawing for prizes. The prizes included one grand prize (a prize worth approximately $300 USD, such as a GPS watch, an ipod, or a Bose SoundDock), three second prizes (a prize worth approximately $100 such as a running jacket, an ipod Shuffle, or a heart rate monitor watch), and 10 third prizes (a prize worth approximately $10 such as running socks, running gloves, or a winter hat). A unique drawing was conducted for each marathon. Pre-Marathon Surveys We randomly assigned participants to one of two pre-marathon online survey conditions: a goal-not-asked condition and a goal-asked condition. Each of the marathons in our study Table 1. Basic statistics for all finishers as well as the study sample for each of the 15 targeted marathons. These numbers are taken from official results posted by the marathon organizers and sometimes differ from the numbers reported on some online compiled lists. 3

4 All#Finishers Study#Sample Total# Finishers ##Average# Finishing#Time# Female #Starting# Participants# Complete# Participants ##Average# Finishing#Time# Female Chicago#2007 ########## 25,523 ############### % ################ 170 ############### % Marine#Corps#2007 ########## 20,625 ############### % ################ 180 ############### % New#York#City#2007 ########## 38,623 ############### % ################## 31 ################# % Portland#2007 ############ 7,738 ############### % ################## 21 ################# % Twin#Cities#2007 ############ 7,154 ############### % ################ 248 ############### % Boston#2008 ########## 21,963 ############### % ################## 33 ################# % Grandma's#2008 ############ 6,876 ############### % ################## 20 ################# % Los#Angeles#2008 ########## 17,246 ############### % ################## 84 ################# % Rock#'n'#Roll#(San#Diego)#2008 ########## 16,760 ############### % ################## 82 ################# % Chicago#2008 ########## 31,344 ############### % ################ 157 ############### % Twin#Cities#2008 ############ 7,979 ############### % ################ 123 ############### % Marine#Corps#2008 ########## 18,237 ############### % ################ 141 ############### % Chicago#2009 ########## 33,703 ############### % ################ 276 ############### % Twin#Cities#2009 ############ 8,475 ############### % ################ 115 ############### % Marine#Corps#2009 ########## 21,405 ############### % ################ 440 ############### % Overall ######### 283,651 ############### % ############# 2,121 ############ 1, % survey, we ed participants links to the survey on the Friday that fell 16 days before the start of their marathon. 3 Participants were allowed to complete the survey anytime prior to the start of their marathon. We asked participants in the goal-asked condition to report the same demographic and experience information as the goal-not-asked participants. In addition, participants in the goalasked condition were asked about training, time goals, and other objectives. Specifically, participants were asked if they had a numeric time goal, and if so, they were asked to report it. Participants were also asked to estimate their likelihood of reaching that goal on a 0 to 100 probability scale, and to indicate their anticipated satisfaction on a 1 to 7 scale for finishing 1, 10, and (in the 2008 and 2009 marathon surveys) 20 minutes ahead or behind their time goal. In addition, they were asked to provide importance ratings on a 1 to 7 scale for a number of different objectives, including meeting their time goal, running a personal best, finishing the marathon, raising money, and having fun. We also asked for information about their training, 3 Our full design included three pre-marathon conditions. The additional condition was included for purposes not related to this investigation (see Markle, Wu, White, and Sackett, 2014). That condition also elicited goals from runners before the marathon, asking them to answer the questions in the goal-asked condition both 6 weeks and 2 weeks before the marathon. Our results are unaffected if we include this condition. For experienced runners, the difference in finishing times between the two goal-asked conditions and the goal-not-asked condition is a statistically significant 6.55 minutes, t(1162) = 2.13, p =.033 without controls and 6.70 minutes, t(1137) = 2.61, p =.009 with controls. We excluded this condition from our analysis because asking runners for goals 6 weeks prior to the marathon could influence athletes training in addition to the ambitiousness of time goals. 4

5 including the number of days they ran per week, their weekly mileage, and the length of their longest run. Appendix A contains the wording and order of all of the survey questions, while Table 2 provides summary statistics of the demographic and goal measures. Post-Marathon Survey For reasons unrelated to the present report, participants running marathons in 2007 and 2008 were randomly assigned to one of two post-marathon conditions. We dropped this 5

6 Table 2. Descriptive statistics on demographics, goals, satisfaction, and performance. Type of measure Demographics Running Background Training Goals Pre-marathon Objectives Performance Measure Average Median Std. Dev Minimum Maximum Age Female 58.02% Years running Previously run marathon 71.84% Number of completed marathons Seriousness of running Best 10 kilometers 0:50:25 0:50:00 9:33 30:31 2:01:29 Best half marathon 1:59:09 1:55:00 27:47 1:06:10 5:26:47 Best marathon 4:09:50 4:03:29 50:10 2:16:36 8:17:42 Last marathon 4:30:50 4:25:00 52:17 2:16:26 8:21:59 Days per week Miles per week Long run (miles) Have goal 86.19% Goal 4:12:41 4:05:00 41:54 2:19:59 8:10:00 Likelihood of reaching goal 73.84% 80.00% 18.67% 0.00% % Demonstrate self-confidence Demonstrate athletic ability Achieve feat See city Have fun Participate in a major event Run with friends/family Win cash Raise money Beat time goal Best best time Place high Finish Finishing time 4:36:01 4:32:59 51:37 2:17:21 8:25:26 Satisfaction with performance Satisfaction with effort Finishing time minus goal 0:16:41 0:10:43 24:17 2:43:02-1:00:10 Relative finishing time minus goal 73.66% 73.66% 73.66% 73.66% 73.66% Notes:'A'random'number'has'been'added'to'the'minimum'and'maximum'for'best'10'kilometers,'best'half'marathon,'best'marathon,'last' marathon,'and'finishing'time'so'that'participants'in'our'study'cannot'be'identified.''the'random'number'is'uniformly'distributed' between'@3%'of'the'actual'time'and'+3%'of'the'actual'time. manipulation for the 2009 marathons, with all 2009 participants receiving the post-marathon survey one day after the marathon. In the post-marathon survey, participants were asked to provide their marathon bib number. The bib number allowed us to link a participant s survey results to his or her official marathon results. Participants also indicated whether they finished the marathon, started but did not complete the marathon (in which case they were asked how many miles they completed, as well as why they did not finish), or did not run in the marathon at all (in which case they were 6

7 asked to explain why). We then asked participants to rate their satisfaction (on a 1 to 7 scale) with their performance and with their effort, followed by importance ratings for the same objectives asked in the pre-marathon surveys to participants in the goal-asked condition. 4 Finally, all participants were asked for their time goal if they had one. Note that participants in the goalnot-asked pre-marathon condition had not previously provided a time goal. The wording and order of the post-marathon survey questions is found in Appendix B. The number of participants in each of the four (pre-marathon x post-marathon) experimental conditions is presented in Table 3. As expected, the results reported in the main text did not differ by post-marathon condition, and we therefore collapsed the post-marathon conditions together. 5 Official Performance Data In addition to self-reported finishing times, we obtained official marathon results directly from the marathon websites and, in some cases, from marathon organizers. We also obtained intermediate split times when available, including the half-marathon time, and 5, 10, 15, 20, 25, 30, 35 and 40 kilometer times. 6 4 An analysis of the relationship between satisfaction, performance, and time goals is provided in Markle et al. (2014). We find that satisfaction is reference-dependent, consistent with prospect theory (Kahneman & Tversky, 1979) and a goals-as-reference-points account (Heath, Larrick, & Wu, 1999). 5 Our main result, that asking about goals improves performance for experienced runners, did not differ significantly across the two post-marathon conditions, Wald test, F(1, 931) = 1.06, p > All of our target marathons provided 10-kilometer and half-marathon splits. However, all other splits were inconsistently recorded. For example, we did not have 5-kilometer splits for the Portland, Grandma s, Los Angeles, and Rock n Roll Marathons, or 20-kilometer splits for the Portland, Twin Cities, Grandma s, Los Angeles, and Rock n Roll Marathons. In addition, splits for some runners were not posted in the official race results, even though the race recorded these splits for other runners. For example, we are missing half-marathon split time for 28 runners (19 of whom provided time goals) and 10-kilometer split times for 164 runners (111 of whom provided time goals). Thus, for analyses involving split times, we only analyzed results for the first half and the second half of the marathon. 7

8 Table 3. Number of participants across various conditions. Post-marathon condition Pre-marathon condition Early Late Goal-not-asked Goal-asked Marathoners are typically provided with two times for a race: a chip time and a clock time. 7 The difference between the two times reflects the large size of these marathon fields and how long it can take a runner to reach the starting line. For example, it took runners in the 2009 Chicago Marathon an average of minutes to cross the starting line. Marathon organizers employ a technological solution to deal with their large fields. Runners attach a radio frequency identification (RFID) chip to their running shoes or wear a race bib with an embedded RFID chip. This chip registers when a runner crosses the start line, when they pass pre-designated intermediate points on the course (e.g., the half marathon point) and when they cross the finish line. The clock time is the difference between when a runner finishes a race and when the race starts for all runners. The chip time is the difference between when a runner finishes a race and when that runner crosses the start line. The chip time is usually considered the official time and is always at least as fast as the clock time. 8 In our sample, the mean difference between the clock time and chip time is 7.66 minutes. We use the official chip time throughout as the measure of a marathoner s finishing time. 9 7 The chip time is also sometimes referred to as the net time and the clock time is also referred to as the gun time. 8 Finishers are typically ordered by chip time in results listings, but prize money is usually determined by clock time. 9 We have the official clock time for 1,589 of our 1,758 participants, with the missing clock times from the 2008 Boston and the 2007 Chicago Marathons. The results reported in the main text hold if we use clock time instead of chip time as our measure of performance. For example, the effect of our manipulation on clock time, measured by regression coefficients as calculated in the main text, for the 859 of 959 experienced runners with a clock time was minutes, t(857) = 2.70, p =.007 without controls and minutes, t(833) = 3.16, p =.002 with controls. The effect on chip time for the same group was 8.82 minutes, t(857) = 2.58, p =.011 without controls and 9.20 minutes with controls, t(833) = 3.10, p =

9 Overall, 94.4% of participants reported a finishing time within 1 minute of their official time, 97.7% reported a time within 5 minutes of their official time, and 23 participants did not report a finishing time. All of the deviations over 30 minutes were almost exactly 60 or 120 minutes off, suggesting input errors on the part of the participants. The results presented in the main text hold if we use self-reported finishing times instead of official chip times. 10 Additional Analyses Survey Completion Times Our survey was administered through Qualtrics.com, which recorded survey start and completion times. However, due to an error in data acquisition, 43.7% of pre-marathon survey results and 5.3% of post-marathon survey results were recorded as having the exact same start and completion times, indicating that the survey took zero seconds to complete, even though these surveys were completed. 5.7% of pre-marathon surveys and 5.4% of post-marathon surveys took longer than one hour, in which case participants most likely stopped and later resumed the survey. We omitted these outliers as well as responses with identical start and completion times from the summary statistics below but included these participants in our main analyses. Mean completion times for the pre-marathon survey were minutes in the goalasked condition and minutes in the goal-not-asked condition, reflecting the considerably longer instrument administered to participants in the goal-asked condition. The post-marathon surveys were identical for participants in both conditions and mean completion times were 9.31 minutes for the goal-asked condition and 9.77 minutes for the goal-not-asked condition. 10 The mean difference in self-reported finishing times across our two pre-marathon conditions was 8.33 minutes, t(779) = 2.28, p =.023 with no controls and minutes, t(754) = 3.17, p =.002 with controls. The sample size is smaller for this comparison than for other results because other analyses include participants who ran the marathon and did not complete the post-marathon survey. The effect on the official chip time for the same group was 8.08 minutes, t(779) = 2.21, p =.028 without controls and 9.78 minutes, t(754) = 3.06, p =.002 with controls. 9

10 On average, participants completed the pre-marathon survey days before the start of the marathon, with 85.3% of participants finishing the survey ten or more days prior to the marathon. Only 6.4% of participants completed the survey five or fewer days before the marathon, with six participants (0.4%) completing the survey two days before the marathon, and two participants (0.1%) completing the survey one day before the marathon. Survey Attrition Appendix C contains a flow chart of participant attrition. In addition to the information about survey attrition in the main text, one major reason why some participants did not either start or finish their marathon is that a few of the marathons were run in extremely hot weather. These marathons had a very high number of non-starters or non-finishers. For example, the high temperature for the 2007 Chicago Marathon was 88 degrees Fahrenheit (31 degrees Celsius), and organizers ultimately closed the marathon early due to dangerous heat (Tarm, 2007). 22.4% of our 2007 Chicago Marathon participants who completed both a pre- and post-marathon survey did not finish the race. The Twin Cities Marathon occurred on the same day with only slightly less oppressive heat, and 7.3% of the participants from this marathon who completed both a preand post-marathon survey did not start or finish the race (Associated Press, 2007). In addition, 29 of the 48 participants for whom we were unable to match their data to official results did not provide bib numbers. Some of these runners may have run as unofficial runners or bandits. We were unable to match official marathon results to the remaining 19 runners, because the bib numbers they provided were not found in the official results. Sample Characteristics We examined the representativeness of our sample relative to the overall population of the marathons used in our study. Although we do not have any data on goals or experience for marathon runners at large, our sample seems representative in other dimensions. We refer to the 10

11 population as the 283,651 marathon finishers in our 15 target marathons and the sample as the 1,758 runners described above. We created weighted averages by weighting the relevant statistics by the proportion of our sample in each marathon. For example, 113 (or 6.4%) of our 1,758 participants ran the 2007 Chicago Marathon. To compute the weighted average finishing time, we multiplied the finishing time for all runners in the 2007 Chicago Marathon ( minutes) by 6.4%, repeated this process for the other 14 marathons, and summed the 15 products. Our sample ran slightly faster than the population of marathoners. The mean finishing time in our sample was minutes (Median: minutes), while the weighted population mean was minutes (Median: minutes). The interquartile range for our sample, [238.92, ], was similar to the interquartile range for the population, [240.55, ]. Similarly, the 10 th and 90 th percentile times for our sample were and minutes, respectively, whereas the same percentiles for the population were and minutes, respectively. The median finishing times across all United States marathon finishers (including marathons not in our sample) in the three years of our study were: minutes in 2007 (MarathonGuide.com, 2008), minutes in 2008 (MarathonGuide.com, 2009), and minutes in 2009 (MarathonGuide.com, 2010). The mean age of our runners was 37.36, which was almost identical to the mean age of runners in the marathon population, The one dimension in which our sample was not representative is gender. Consistent with the finding that women are more likely to complete surveys than men (Curtin, Presser, & Singer, 2000), 58.0% of our participants were female, compared to 40.6% of runners in the marathon population The females in our sample (M = ) ran faster than the females in the population (M = ), as did the males in our sample (M = ) relative to the males in the population (M = ). 11

12 Goal Reporting Our manipulation did not change the likelihood that runners had time goals. In the postmarathon survey, 87.2% of runners in the goal-not-asked condition reported a time goal, compared to 85.1% of runners in the goal-asked condition, logit z = 1.19, p = This null result is important because it supports our claim that the experimental manipulation (i.e., whether the pre-marathon survey asked participants if they had a time goal for the marathon) did not lead participants to create a time goal when they otherwise would not have. Furthermore, a story in which our manipulation caused runners to create time goals would produce a difference in the opposite direction of what we observed. Instead, this evidence is consistent with our hypothesis that the observed effects on performance are due to the manipulation s effect of locking in ambitious goals. These comparisons use the post-marathon reports of time goals. 87.5% of participants in the goal-asked condition had time goals before the marathon, and this number dropped to 85.1% of runners after the marathon, t(724) = 1.97, p = runners in the goal-asked condition provided goals prior to the marathon but not after the marathon, and 29 runners in the goal-asked condition did not provide goals prior to the marathon but did report goals after the marathon. Participants in the goal-asked condition, who were asked to report their pre-marathon goals both before and after the marathon, exhibited a notable but small reporting bias, with runners reporting less ambitious goals after the marathon (M = ) than before it (M = ), t(587) = 3.05, p = % of participants reported less optimistic goals in the postmarathon survey than in the pre-marathon survey. Nevertheless, 48.8% of runners provided the same goal in the pre- and post-marathon surveys. 12 For experienced runners, 89.2% of runners in the goal-not-asked condition reported a time goal, compared to 85.1% of runners in the goal-asked condition, logit z = 1.76, p =

13 Although there are non-psychological reasons to change one s goal, including injury or unexpectedly hot weather, this difference could alternatively be interpreted as a self-enhancing or self-protective bias, since reporting less ambitious goals after the marathon would have closed the gap between goals and performance (76.0% of participants did not achieve their premarathon goals). 41.1% of runners who fell short of their pre-marathon goal increased their reported goal following the marathon, compared to 5.8% of the runners who met or bested their pre-marathon goal, χ 2 (1, 588) = p <.001. Of course, this correspondence could still reflect runners who injured themselves or learned that they would be running in unexpectedly hot weather, rather than a goal shift to reduce or eliminate a differential between goals and performance. Whether this shift occurred for psychological or non-psychological reasons, it is still small (1.29 minutes) relative to the mean difference in goals of 8.00 minutes (without controls; with controls, M = 8.48) between experienced marathoners in the goal-asked and goalnot-asked conditions. Moreover, this decrease in the ambitiousness of reported time goals works against our hypothesis about the effects of our manipulation on goal ambitiousness, since we expected (and observed) that asking about goals would produce more ambitious goals overall. To take a conservative approach to testing the effects of our manipulation on goal ambitiousness (and to provide the most comparable test between experimental conditions), we used goals reported in the post-marathon survey for both the goal-asked and goal-not-asked conditions even though we had pre-marathon goal data for the goal-asked condition runners. Thus, under-reporting of time goals in the post-marathon survey by participants in the goal-asked condition would make it more difficult to detect mediation of the effect of our manipulation on performance by goal ambitiousness. Indeed, we re-ran the mediation analyses reported in the main text with an alternative goal measure, using the pre-marathon goal for runners in the goalasked condition and the post-marathon goal for runners in the goal-not-asked condition. This 13

14 alternative measure yielded comparable Sobel test statistics (z = 2.66, p =.008 without controls, z = 3.24, p =.001 with controls) relative to the original goal measure that is used in Sackett et al. (2014a) (z = 2.62, p =.009 without controls, z = 3.28, p =.001 with controls). Bootstrap methods (with 25,000 iterations) revealed an equally strong mediation effect using the alternative goal measure (95% CI: [2.06, 13.78] without controls and [5.98, 16.22] with controls) than using the original goal measure (95% CI: [2.07, 14.53] without controls and [4.47, 14.87] with controls). It is noteworthy that our results become stronger if we eliminate runners who changed their goals substantially. For example, dropping the 10 runners who increased their goal by more than 30 minutes changed the mean difference in finishing time between the goal-not-asked and goal-asked condition runners from 6.75 to 7.79 minutes. In Table 4, we report how results change if we eliminate runners who changed their goals more than a certain threshold. The Time Course of Optimism We examined how goal ambitiousness (optimism) was related to the temporal proximity of pre-marathon survey completion to the marathon date. This analysis has some clear limitations. First, survey completion time was not randomly assigned, and thus we do not know whether completion time reflects some individual difference that might be related to optimism or skill level. Second, the precise time course of optimism is ambiguous and probably highly nonlinear. Previous studies show that optimism declines most precipitously just before performance or feedback (e.g., Shepperd, Ouellette, & Fernandez, 1996; Sweeny & Krizan, 2013). However, only five (0.3%) of our full sample of 1,758 participants completed a survey within two days of the marathon. This should be unsurprising, since most marathon runners are probably more interested in attending to final marathon preparation (e.g., diet, logistics, travel) than completing a survey in the last day or two before the marathon. Nevertheless, we examined the relationship 14

15 Table 4. The impact of dropping participants with large goal changes on the effect of asking for a goal on marathon finishing time. Regression3Model (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) Dependent3Variable Finishing3Time Finishing3Time Finishing3Time Finishing3Time Finishing3Time Finishing3Time Finishing3Time Finishing3Time Finishing3Time Finishing3Time GoalBasked3Condition B6.75* B6.93* B6.93* B6.95* B7.79* B7.92* B8.89** B6.90 B5.26 B4.31 (3.31) (3.30) (3.30) (3.31) (3.31) (3.32) (3.44) (3.60) (3.65) (3.71) Controls None None None None None None None None None None Goal3Change3 Exclusions None Over3603 minutes Over3503 minutes Over3403 minutes Over3303 minutes Over3203 minutes Over3103 minutes Over353 minutes Over minutes Observations Regression3Model (11) (12) (13) (14) (15) (16) (17) (18) (19) (20) Dependent3Variable Finishing3Time Finishing3Time Finishing3Time Finishing3Time Finishing3Time Finishing3Time Finishing3Time Finishing3Time Finishing3Time Finishing3Time GoalBasked3Condition B8.13** B8.28** B8.28** B8.20** B8.88** B9.02** B9.71** B8.09** B6.97* B5.95 (2.81) (2.81) (2.81) (2.81) (2.82) (2.83) (2.91) (3.04) (3.09) (3.14) Controls All All All All All All All All All All Goal3Change3 Exclusions None Over3603 minutes Over3503 minutes Over3403 minutes Over3303 minutes Over3203 minutes Over3103 minutes Over353 minutes Over minutes Observations Over3133 minute Over3133 minute Notes:3This3table3reports3regression3coefficients,3standard3errors,3and3tBstatistics3for3the3difference3in3finishing3time3(in3minutes)3across3the3two3preBmarathon3conditions3for3 experienced3runners.33a3negative3coefficient3for3the3goalbasked3condition3indicates3that3runners3in3the3goalbasked3condition3had3faster3average3finishing3times3than3runners3in3the3 goalbnotbasked3condition.33specifications3exclude3participants3in3the3goalbasked3condition3who3changed3their3goals3from3prebmarathon3to3postbmarathon3more3than3a3threshold3 amount.33regressions3113through3203include3all3controls3found3in3regression383of3table35a. *3Significant3at35%3level;3**3Significant3at31%3level between optimism and pre-marathon goals. A regression of pre-marathon goal on the number of days the survey was completed before the marathon shows no relation, b =.19, t(743) = 0.49, p >.250. When we control for best previous marathon, the relationship is in a direction consistent with the temporal optimism account but is not significant, b = -.12, t(504) = 0.42, p >.250. Goal Importance We asked participants to indicate, on a 1 to 7 scale, the importance of attaining their time goal. Runners in the goal-asked condition were asked before and after the marathon, while runners in the goal-not-asked condition were only asked to provide a rating in the post-marathon survey. Importantly, runners viewed time goals as less important in the goal-asked condition (M = 5.39) than in the goal-not-asked conditions (M = 5.62), t(1463) = 2.73, p = While the temporal-optimism hypothesis is agnostic about differences in goal commitment across 13 This effect holds for both novice marathon runners (M goal-asked = 5.30, M goal-not-asked = 5.56, t(673) = 2.22, p =.027) and experienced marathon runners (M goal-asked = 5.47, M goal-not-asked = 5.66, t(788) = 1.70, p =.089). 15

16 conditions, this data pattern, along with the statistically-equivalent rates of goal attainment, runs contrary to the alternative goal-commitment hypothesis. Experienced runners in the goal-asked condition provided statistically indistinguishable ratings in the pre-marathon survey (M = 5.50) and the post-marathon survey (M = 5.47), t(393) = 0.38, p >.250. Additionally, and contrary to a goal-commitment story, experienced participants in the goal-asked condition (M = 5.47) provided marginally lower ratings in the post-marathon survey than did participants in the goal-not-asked condition (M = 5.66), t(788) = 1.70, p =.089. Success rate of achieving goal Overall, 26.9% of participants met their time goal. The success rate was not significantly different between the goal-asked (M = 27.7%) and the goal-not-asked (M = 26.1%) conditions, logit z = 0.66, p >.250. In addition, 32.5% of runners ran faster than their best marathon, and 56.5% of runners performed better than their last marathon. The likelihood of beating one s personal best marathon was not significantly different between the goal-asked condition (M = 32.9%) and the goal-not-asked condition (M = 32.1%), logit z = 0.28, p >.250. However, runners in the goal-asked condition (M = 62.0%) were significantly more likely to outperform their most recent previous marathon time than were runners in the goal-not-asked condition (M = 51.1%), logit z = 3.64, p < % of runners who rated meeting their time goal as 7 out of 7 in terms of importance met their goal, compared to 20.7% of all remaining runners, logit z = 6.43, p <.001. There were no significant gender differences in the likelihood of achieving a time goal: 27.1% of males reached their time goal, compared to 26.7% of females, logit z = 0.18, p >.250. In addition, 24.2% of experienced marathoners (those who had completed more than one previous marathon) beat their time goal, compared to 30.0% of novice marathoners, logit z = 2.31, p =

17 We also asked runners in the post-marathon survey to estimate the likelihood of you reaching this time goal for this marathon. Runners in the goal-asked and goal-not-asked conditions gave nearly identical numbers (goal-asked M = 71.6% vs. goal-not-asked M = 71.7%) and thus were optimistic about their chances of achieving their time goal, compared to actual success rates. This difference between conditions was not significantly different, t(1264) = 0.08, p > 0.250, and is consistent with our hypothesis that our manipulation worked through the ambitiousness of goals, not goal commitment. In addition, we asked participants to indicate the extent to which they thought their goal was realistic on a 1 to 7 Likert scale. There was no significant difference between the goal-asked condition (M = 5.70) and the goal-not-asked condition (M = 5.75), t(1267) = 0.54, p >.250. Finishing times We illustrate our results by plotting the cumulative distribution of finishing times for runners in both the goal-asked and goal-not-asked condition (see Figure 1). Although this plot crosses (but approximately overlaps) for finishing times below 3:18 and above 5:25, the cumulative distribution of finishing times for times between 3:18 and 5:25 lies to the left of that for the goal-not-asked condition for most of the distribution. The cumulative distribution plot suggests that our main finding that eliciting a goal prior to the marathon leads to fast finishing times is not driven by outliers, a view further supported by quantile regression analysis (see Footnote 18). Marathon Experience Our sample represented a diverse range of marathon-running experience (see Figure 2). Figure 2 is a histogram of the number of marathons completed by our participants. As we noted in the main text, the effects of our goal-asking manipulation were moderated by experience. Experienced marathon runners ran significantly faster (M = 6.75 without controls, M =8.13 with 17

18 Cumulative distribution :30 3:00 3:30 4:00 4:30 5:00 5:30 6:00 6:30 7:00 7:30 Finishing time Goal-asked Condition Goal-not-asked Condition Figure 1. Cumulative distribution of finishing times for experienced participants in the goal-asked and goal-not-asked condition. For example, 40.0% of experienced runners in the goal-not-asked condition ran 4:11:21 or faster, whereas 40.0% of experienced runners in the goal-asked condition ran 4:03:23 or faster. Density > 50 Number of marathons completed Figure 2. Histogram of number of marathons completed. 18

19 controls) if they were in the goal-asked condition than if in the goal-not-asked condition. Our manipulation had no significant effect on novice marathon runners (M = -3.21, t(797) = 0.92, p >.250 without controls, M = -2.61, t(771) = 0.86, p >.250 with controls). We dichotomized the sample by experience level for illustrative purposes in the main paper (Sackett et al., 2014a). The dichotomization used in the main text yielded 799 novice marathoners (45.4%) and 959 experienced marathoners (54.6%). In this section, we provide a more detailed examination of the effect of marathon experience. We examined the interaction between eliciting a goal prior to the marathon and the level of marathon experience by regressing finishing time on: (i) a dummy variable for participants in the goal-asked condition; (ii) a fifth-order polynomial for level of experience (i.e., number of marathons completed, number of marathons completed squared,, number of marathons completed to the fifth power); and (iii) the interaction between (i) and (ii). 14 We then used the coefficients from these regressions to estimate how much faster or slower a runner with varying levels of marathon experience in the goal-asked condition ran. We obtained 95% confidence intervals by bootstrap methods (1,000 iterations). Figure 3 shows these results for number of marathons ranging from 0 to 15 (91.7% of our sample) without controls (Panel A) and with controls (Panel B) with negative values indicating that runners in the goal-asked condition with that level of experience ran faster than runners in the goal-not-asked condition. The panels support our dichotomization of experience into novices (0 or 1 previous marathon) and experts (2 or more previous marathons). One plausible reason why relatively experienced marathoners would be affected by our manipulation, while relatively inexperienced marathoners would not, is that inexperienced 14 Although a fifth-order polynomial for level of experience fits the data best, the results of this analysis look the same with third- and fourth-order polynomials as well. 19

20 Mean difference in finishing times Polynomial Regression with No Controls Number of marathons completed Error Bars: 95% Confidential Intervals Mean difference in finishing times Polynomial Regression with Controls Number of marathons completed Error Bars: 95% Confidential Intervals Figure 3. Interaction between goal-asked condition and marathon experience. Estimates are obtained from a fifth-order polynomial regression and indicate the estimated difference between finishing times for runners in the goal-asked and goal-not-asked conditions. Negative values indicate that runners in the goal-asked condition ran on average faster than runners in the goal-not-asked condition. Confidence intervals are obtained from bootstrapping with 1,000 iterations. marathon runners are more likely to focus on simply finishing the race than on a specific time goal. As runners gain more experience, finishing is more likely to be taken for granted, and time goals become relatively more important (of course, with experience comes age, and eventually merely finishing the marathon may be relatively more important than achieving a time goal). For support of this interpretation, we looked to the measures of goal importance and the importance of finishing the marathon. Figure 4 shows the mean ratings of these two measures by level of marathon experience. Clearly, rookie marathoners tended to put more importance on finishing than did more experienced marathoners, while achieving the time goal was relatively less important. For runners with 2 or more marathons of experience, finishing the marathon was seen as generally less important (though still important) and achieving the time goal was seen as more important. Interestingly, runners with 1 previous marathon finish were somewhat in-between, like more-experienced marathoners in goal importance and like less-experienced marathoners in importance of finishing. 20

21 Goal Importance Importance of Finishing Marathon Importance of achieving time goal (1 to 7) Number of marathons completed Error Bars: 95% Confidence Intervals Importance of finishing marathon (1 to 7) Number of marathons completed Error Bars: 95% Confidence Intervals Figure 4. Mean importance ratings conditional on marathon experience. The left panel shows the average rating of goal importance ( meeting the time goal you set for yourself) and the right panel shows the average rating of importance of finishing the marathon. Mean difference in finishing times No Controls Number of marathons completed for inclusion in experienced Error Bars: 95% Confidence Intervals Mean difference in finishing times Controls Number of marathons completed for inclusion in experienced Error Bars: 95% Confidence Intervals Figure 5. The average effect of the goal-asked manipulation on experienced runners for different definitions of experienced runners. For example, a threshold of 0 marathons defines all runners as experienced runners and thus reports the effect of the goal-asked condition on the whole sample. The main text uses a threshold of 2 marathons to define experienced runners. The plots show regression coefficients obtained from regressions without controls (left panel) and with controls (right panel). Finally, we tested the robustness of our dichotomization by investigating other breaks. We compare the performance of runners in the goal-asked and goal-not-asked conditions by considering different thresholds for distinguishing experienced and novice runners (see Figure 5). In the most inclusive analysis (threshold of 0), all runners were defined as 21

22 experienced. Figure 5 depicts the results of more exclusive definitions of experienced, plotting the effect of the goal-asked manipulation for different thresholds of experience. The left panel shows regression coefficients for specifications without controls, while the right panel shows results for specifications with controls. The threshold used in the main text takes 2 marathons as the definition of experienced. The large diamonds in Figure 5 at a threshold of 2 marathons show the results reported in the main text. This analysis shows that the basic results reported in the main text are reasonably robust to the precise definition of experienced and novice runners. Robustness Analysis Our regressions examined the effect of our goal manipulation on performance and reported time goals. A number of factors can lead a marathoner to run faster or slower. Even though our method is experimental, it is nevertheless possible that our primary findings could result from a correlation with one or more of these factors, combined with a failure of random assignment to experimental conditions. Thus, to investigate the robustness of our effect, we performed a variety of regressions including different control variables and different error specifications. We controlled for gender (in our sample, men ran an average of minutes faster than women) and the specific marathon run (in our sample, the mean finishing time ranged from 3:50:24 for runners in the 2008 Boston Marathon to 5:25:51 for runners in the 2008 Los Angeles Marathon). 15 We also controlled for age, dividing the sample into 5 categories, 0 to 29 (27.1%), 30 to 39 (34.0%), 40 to 49 (25.1%), 50 to 59 (11.1%), and 60 and over (2.7%), including dummy variables for each of these categories. For example, the mean finishing time for runners 29 and under was minutes, whereas the mean finishing time for runners 60 and over was All of the statistics reported in this section included only experienced runners. Thus, these numbers differ from those reported in Table 1. 22

23 minutes. We controlled for experience by creating 6 dummy categories: rookies (28.1%), 1 previous marathon (17.4%), 2 previous marathons (11.7%), 3 previous marathons (7.6%), 4 to 8 previous marathons (17.5%), and 9 or more previous marathons (17.8%). We also controlled for self-reported seriousness, creating a dummy variable for each of the seven Likert score values. 22.9% of participants viewed themselves as a casual runner (1 to 3 on the scale), whereas 53.0% of runners viewed themselves as a serious runner (5 to 7 on the scale). To control for the marathon run, we controlled for the marathon and the year (i.e., we included a dummy variable for running the Chicago Marathon in 2007, a dummy variable for running the Chicago Marathon in 2008, etc.). These fixed effects take into account differences across marathon courses (for example, some marathons, such as the Chicago Marathon, have fast and flat courses, while other marathons, such as the New York City Marathon, are considerably hillier and hence slower), differences in selection across marathons (for example, the Boston Marathon has qualifying times and thus only 17.2% of runners fit our definition of novices, while 72.9% of the runners in the 2008 Rock n Roll Marathon were novices), and variation across years (for example, the mean time for the 2007 Chicago Marathon, when the temperature reached 88 F/31 C, was minutes, while the mean time for the 2009 Chicago Marathon, run on a much cooler day, was minutes). Table 5a includes results for different combinations of controls and shows that our effect is robust to the inclusion of different control variables. We used the same set of controls to predict time goal (Table 5b), the time for the first half of the marathon (Table 5c), and the time 23

24 for the second half of the marathon (Table 5d). 16 All of these models indicate that asking for a goal prior to the marathon leads runners to run faster (in the first half of the marathon, the second half of the marathon, and overall), as well as to establish more ambitious goals. In addition, we estimated two alternative regression specifications. First, we log-transformed finishing time because the distribution of finishing times is skewed to the right. Second, we employed a quantile regression, estimating a conditional median instead of the conditional mean fit in Ordinary Least Squares (OLS) regression. Quantile regression (also known as median regression) is more robust to outliers than OLS regressions (Koenker & Hallock, 2001). 17 Results of these specifications are found in Tables 6a and 6b. These specifications produce results that are qualitatively similar to the results described above, producing significant differences for experienced runners after controlling when control variables are included. Mediation Analysis We performed mediation analyses in which we tested whether the effect of asking about goals prior to the marathon on finishing time was mediated by time goal ambitiousness. We tested this mediational pathway in two ways: using the Sobel test as well as the bootstrap method (Preacher & Hayes, 2004). We tested for mediation in regressions with no control variables, as well as regressions in which we included all the control variables described above (as shown in the main text). When these control variables were omitted, both the Sobel test (z = 2.62, p 16 The sample sizes for the regressions presented in Tables 5c and 5d differ slightly from those presented in Table 5a because we were missing half-marathon splits for 30 runners. The results in Table 5a are nearly identical if these 30 runners are omitted throughout. For example, the 6.75 minute difference in finishing times (8.13 with controls) between runners in the goal-asked and the goal-not-asked condition drops to 5.80 minutes (7.97 with controls) if we eliminate the 30 runners without half-marathon splits. Note that 7.97 minutes is the sum of the mean differences between the pre-marathon conditions, with controls, for the first half (M = 2.71 ) and the second half (M = 5.25) of the marathon reported in Sackett et al. (2014). Without controls, the mean differences in the first half (M = 1.81) and the second half (M = 3.99) sum to 5.80 minutes. 17 We also estimated the effect of our manipulation on quantiles other than the median and found significant results (at the p <.05 level) for quantiles ranging from.21 to.93 when we included no controls and for quantiles ranging from.37 to.99 when we included controls. 24