#60Tests#60Minutes #60Tests#60Minutes

Size: px

Start display at page:

Download "#60Tests#60Minutes #60Tests#60Minutes"

Tobias Caldwell
5 years ago
Views:

2 60 Tests. 60 Minutes. Yep Jessica Bosanko, M+R Debbie Schneiderman, Wildlife Conservation Society

3 About Wildlife Conservation Society #60Tests#60Minutes

4 About M+R #60Tests#60Minutes

5 Don t be too jealous.

6 A delicious test recipe If you re gonna do it, do it right!

7 This is what I mean: #60Tests#60Minutes

8 This is what I mean: Objective: Determine whether including pictures of dinosaurs lifts response #60Tests#60Minutes

9 This is what I mean: Objective: Determine whether including pictures of dinosaurs lifts response Hypothesis: They do!

10 This is what I mean: Objective: Determine whether including pictures of dinosaurs lifts response Hypothesis: They do! Quantity: Use a significance calculator to determine what size sample you need

11 This is what I mean: Objective: Determine whether including pictures of dinosaurs lifts response Hypothesis: They do! Quantity: Use a significance calculator to determine what size sample you need One variable: JUST the dino. JUST the picture. No different text.

Quantity: Use a significance calculator to determine what size sample you need One

12 This is what I mean: Objective: Determine whether including pictures of dinosaurs lifts response Hypothesis: They do! Quantity: Use a significance calculator to determine what size sample you need One variable: JUST the dino. JUST the picture. No different text. Usable results: Use that significance calculator: did it measurably lift response?

13 No Match vs. 2x vs. 3x Vote: No Match 2x Match 3x Match Tie Winner: 3x Match (38% lift over 2x match, 71% lift over no match) 1

14 $5 Ask to Prospects # Gifts Winner: Test 172% increase Revenue Winner: Test 97% increase 2

15 $5 Ask to Prospects (x10) # Gifts Winner: Test 10/10 Revenue Winner: Test 7/10 3

16 E-News Lift Note Winner: Test (39% lift in fundraising responses) 4

17 Donate Now vs. Become a Member Winner: Test (24% higher response rate) 5

18 Member info in the callout box? Winner: Test (28-59% Lift in Response Rates!) 6

19 Personalized Preheader Winner: Test (31% Lift in Response Rate) 7

20 Referencing demographics Winner: Test (25% Lift in Average Gift) 8

21 Action-Taker Reference Winner: Tie 9

22 Age-Related Content Winner: Tie 10

23 Remove the Dollar Signs on donation form Winner: Test (18% lift in average gift!) 11

24 Yellow Highlighting Winner: Tie 12

25 Preheader test Winner: Test (3.3% Higher Open Rate) 13

26 Most Supporters gave at this level Winner: Test (25% lift in average gift!) 14

27 Dogs vs. Cats vs. Pigs Vote: Dog Cat Pig Tie Winner: Cat (19% lift in conversions) 15

28 Vertical vs. Horizontal Ask String Winner: Control (25% lift in conversions) 16

29 Lift Note Length Winner: Tie 17

30 One-Step vs. Multi-Step Form Conversion Winner: Test (13% increase) Average Gift for prospects: Control (38% decrease) 18

31 Wolves vs. Polar Bears Winner: Control Marginally significant, 29% more revenue 19

32 Lapsed Donor Lift Note Response Rate Winner: Tie Average Gift Winner: Test (Marginally significant 30% lift) 20

33 Short vs. Long Winner: Tie (but 50% more gifts with long version!) 21

34 Short vs. Long Winner: Test (12% lift in response rate) 22

35 Short vs. Long Lapsed Donors: Test Everyone Else: Tie 23

36 Plain Old Design vs. Big Picture CTR Results: Control (100% lift) Response Results: Tie (but control had more gifts) 24

37 Testing two different signers Winner: Tie 25

38 Match Box vs. No Match Box Winner: Tie 26

39 Gift String Winner: Test 27

40 Donation Form Layout Winner: Test 28

41 Naked Links in Winner: Tie 29

42 Your Name on the Landing Page # of Gifts Winner: Tie Average Gift Winner: Test 30

43 Deadline vs. Urgent Winner: Tie 31

44 Prior Donor Test Winner: Test (13% more revenue) 32

45 Lapsed Donor Test Winner: Tie 33

46 Lapsed Donor Test Lapsed Donor Guilt-Inducing Content Winner: Test (6 times as much!) 35

47 Optimized Advocacy Form Vote: Control Test Winner: Test (3% lift - pretty close!) 36

48 Call-Out Box Button Winner: Tie 37

49 Thermometer in the Call-out Box Winner: Tie 38

50 P.S. vs. no P.S. Winner: Tie 39

51 Celebrity Sender Test Vote: Celebrity Celebrity FOR ORGANIZATION Winner: Celebrity 40

52 Our favorite time of the year Mozilla s 2013 End of Year Campaign 41-48

53 BREAKING IT DOWN Dissecting a Mozilla Supporter* Codes for fun Really freaking hates the CIA and NSA Loves wikipedia Wears hoodies, and loves Mozilla apparel Favors function over fashion Thinks the internet should be free *For brainstorm purposes only think of this as an aggregate of a Mozillian

54 NEW AND SHINY! AN OLDIE BUT A GOODIE Mission based copy Institutional reasons to give Reminder of tax-deductibility Deadline Driven ALL THAT STUFF + Update on fundraising total Average gift Assurance that if you give, we ll stop ing you. A reminder of this is what you told us via survey

55 SHOW AND TELL We work in the open, we fundraise in the open

56 $60, $50, test #1 Test: 0.15% RR Control: 0.12% RR #2 Test: 0.11% RR Control: 0.07% RR $40, $30, $20, control Statistically Significant! Statistically Significant! $10, $- #3 (NO TEST) Test: 0.07% RR Control: 0.07% RR #4 Test: 0.18% RR Control: 0.13% RR #5 Test: 0.08% RR Control: 0.06% RR Statistically Significant! Statistically Significant!

SUBJECT LINES We tested 3 sets of subject lines during the EOY campaign Email #2: Our big plans for 2014 vs.

57 SUBJECT LINES We tested 3 sets of subject lines during the EOY campaign #2: Our big plans for 2014 vs. How we win in 2014 NO STATISTICAL SIGNIFICANCE Test more drastically different subject lines Test subject lines that play into themes of transparency Skip subject line testing unless there is real value or variation #3: Do you trust the web with your data? vs. Trust the web with your data? vs. Trust in the web #4: Worth fighting for vs. We re in a fight for user privacy

58 TEST TOTAL: 0.11% Response Rate CONTROL TOTAL: 0.09% Response Rate THIS JUST IN! Transparency inspires! 26.3% more donors 21.6% increase in revenue Same average gift This is a big deal! People love transparency! Our supporters like and WANT to be looped in. The extra time it took to create the added layer of transparency was well worth the effort!

59 Facebook Re-targeting Test: Half got ads Winner: Tie 49

60 Animated GIFs in Winner: Tie 50

61 Icons vs Images Winner: Control 51

62 Icons vs Images Winner: Control 52

63 Image vs Status box Winner: Test 53

64 Happy vs Sad photos Control Test Sad photo in + Happy on landing page Happy photo in + Happy on landing page Winner: Test 54

FUNDRAISING PARTICIPANT GUIDE

FUNDRAISING PARTICIPANT GUIDE CONTACT INFORMATION Syracuse Heart Walk Website www.syracuseheartwalk.org American Heart Association Mailing Address American Heart Association Attn: Syracuse Heart Walk PO Box 3049, Syracuse, NY 13220-3049