BUSINESS STATISTICS FOR DECISION MAKING IN THE 21 ST CENTURY

Transcription

1 BUSINESS STATISTICS FOR DECISION MAKING IN THE 21 ST CENTURY Richard D. De Veaux Williams College Bordeaux, FRANCE June 8,

2 WILLIAMS COLLEGE Health expenditure-total-% of GDP-2013

3 WILLIAMS COLLEGE

4 WHERE IS MASSACHUSETTS? 4

5 WILLIAMSTOWN?

6 IMAGINE Thomasine lands her dream job analyst for H&M 6

7 7 FIRST PROJECT

8 DOWNLOADS THE DATA Yes No Yes Yes Yes No No No No No No No No No Yes No No No No No No No No No No Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes No Yes No Yes Municipality Location Route OwnerBuilt Date.Inspected SD.FO.Status Condition YearInspected AgeAtInspection Caroline Town 3.6 MI NW TIOGA CL;RTE NYSDOT /14/15 N Caroline Town.3 MI E JCT SH 79 & CR NYSDOT /19/15 N Caroline Town.5 MILE EAST OF BESEMER BANKS ROAD County /20/14 N Caroline Town.9 MI W SLATERVILLE SPNGS BOICEVILLE ROAD County /9/15 N Caroline Town IN SLATERVILLE SPRINGS BUFFALO ROAD County /13/15 N Caroline Town 2 MI N OF SPEEDSVILLE Blackman Hill Rd. County /20/14 N Caroline Town 4.9 MI SE JCT RTS. 330&79 CENTRAL CHAPEL RD County /14/15 N Caroline Town AT GUIDE BOARD CORNERS CENTRAL CHAPEL RD County /2/15 N Caroline Town 1 MI SE OF W.SLATERVILLE CENTRAL CHAPEL RD County /24/15 N Caroline Town AT BROOKTONDALE COOKS CORS-BRK RD County /17/14 N Caroline Town 1.6 MI SOUTH OF BESEMER CR113LOUNSBERRYRD County /17/14 N Caroline Town.4 MI W SLATERVILLE SPGS. CREAMERY ROAD County /21/15 N Caroline Town 2.8 MI W SLATERVLLE SPNGS HARFORD ROAD County /29/14 N Caroline Town 1 MI SOUTH OF BESEMER MIDDAUGH ROAD County /14/15 N Caroline Town.3 MILE S OF SPEEDSVILLE OLD SEVNTY SIX RD County /16/15 N Caroline Town 1.5 MI NW OF SPEEDSVILLE OLD SEVNTY SIX RD County /16/15 N Caroline Town IN SPEEDSVILLE OLD SEVNTY SIX RD County /6/15 N Caroline Town.5 MI S OF WEST SLATERVLE VALLEY ROAD County /14/15 N Danby Town 5.6 MI NW TIOGA CL-SH 96B 96B 96B NYSDOT /17/15 N Danby Town 3.3 mi NW Willseville 96B NYSDOT /3/15 N Danby Town 1.3 MI NORTH OF W DANBY BROWN ROAD Town /12/14 N Danby Town 1.8 MI S BUTTERMILK FALLS COMFORT ROAD County /7/14 N Danby Town 3.8 MILES NE OF NEWFIELD JERSEY HILL ROAD County /12/14 N Dryden Town.6 MI NW JCT SH 13 & SH NYSDOT /27/15 N Dryden Town 1.6 MI NE JCT RTS NYSDOT /18/15 FO Dryden Town 2.5 MI NE JCT SH 366 & SH NYSDOT /15/14 SD Dryden Town IN ETNA COUNTY ROAD 109 County /22/15 N Dryden Town IN ETNA COUNTY ROAD 109 County /17/15 N Dryden Town1 MI EAST OF ITHACA DODGE ROAD County /4/15 SD Dryden Town 3 MI SE DRYDEN-E LAKE RD EAST LAKE ROAD County /29/15 N Dryden Town.7 MI SW OF MCLEAN FALL CREEK ROAD County /12/15 N Dryden Town1 MI NE OF FREEVILLE FALL CREEK ROAD County /23/14 N Dryden Town AT VARNA FREESE ROAD County /22/15 SD Dryden Town1.3 MI E ITHACA CITY LMTS GAME FARM ROAD County /11/15 FO Dryden Town 1.8 MILES SE OF VARNA GENUNG ROAD County /26/15 N Dryden Town2.7 MI SE ITHACA CITY LMT GERMAN CROSS ROAD County /22/15 N Dryden Town0.7 MI W OF FREEVILLE MILL STREET County /3/15 FO Dryden Town 1.4 MI W JCT SH366 &SH355 PINCKNEY ROAD County /29/14 N Dryden Town 3.3 MI SE OF VARNA RINGWOOD ROADCounty /28/14 N Dryden Town 3.5 MI W OF DRYDEN RINGWOOD ROADCounty /21/15 N

9 9 AND THEN

10 HERE S WHAT KEEPS ME UP AT NIGHT Data Scientists teaching our course and calling it analytics Students thinking the course is irrelevant Students thinking that the world (or at least what Statistics can deal with) is univariate That we are teaching the same course we taught in

11 SO, WHAT S THE PROBLEM? We teach the wrong things We teach it in the wrong way We teach it in the wrong order I don t have all the answers but I ll keep asking the questions

12 WHAT DO I WANT? What do I want students to get out of the course That Statistics is relevant and essential actually cool empowering a powerful method for solving real business problems in today s world

13 HOW DO WE GET THERE? Get to more than two variables early on Introduce models early start with univariate, but quickly go to multivariate we don t need inference right away What to leave out probability models (great, but another course ) mathematics of sampling distributions, tests

14 PRODUCERS OR CONSUMERS? How to teach a lay up? Who s the audience? spectators referees players beginners pros

15 DIAMOND PRICES Carat Color Cut Clarity

16 CARAT

17 COLOR

18 PRICE VS. COLOR

19 THIS IS WHY

20 REGRESSION ON COLOR

21 ADD CARAT TO THE MODEL

22 SECOND ORDER MODEL

23 PROFILE FOR MULTIPLE REGRESSION

24 HOW MUCH IS A FIREPLACE WORTH? 10,700 houses collected from Saratoga NY public records by my student Candice Corvetti for her thesis A random sample of 1729 houses now in SaratogaHouses in library(mosaic) in R Problem: Test whether the prices of houses with and without fireplaces is the same and construct a 95% confidence interval for the difference

25 START BY LOOKING AT THE DATA Difference in means is $65,000 Contractor can add one for $20,000 good business decision?

26 LET S THINK STATISTICALLY H0: Means are equal t = , df = p-value < 2.2e percent confidence interval:

27 THAT SETTLES IT (!?) Courses typically end with A/B tests How do we get students to think multivariately?

28 DOES SIZE MATTER? Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) livingarea < 2e-16

29 DIFFERENT INTERCEPTS? Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) livingarea < 2e-16 FireplaceTRUE

30 WHAT NOW?

31 PROFILER

32 WHAT ABOUT BEDROOMS? 8 bedrooms 2 bedrooms Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) e-12 *** bedrooms < 2e-16 ***

33 AN EASY QUARTER MILLION $ If I chop each bedroom into 4, I ll have an 8 bedroom house worth $250,000 more!!!

34 TWO CORRELATED PREDICTORS Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) e-08 *** livingarea < 2e-16 *** bedrooms e-07 ***

35 MAKING COEFFICIENTS COME ALIVE

36 WHAT TO EMPHASIZE Data are messy and often wrong Inference is important but understanding data pedigree is more important most data analyses generate hypothesis, don t confirm The world is multivariate

37 Red Area (30% unemployed) WHAT ABOUT ETHICS? 65 Bay Street

38 WHAT ABOUT ETHICS? There are many important ethical issues when dealing with data and models. Some are not as obvious as this:

39 WHAT ARE OUR CHALLENGES Environment inertia Textbooks Support Background Proofiness

40 Introduce models early WHERE ARE WE? Motivate univariate questions from more complex models Not the other way around! Show that statistics is more than a collection of tools It s a way of thinking Should we ask them to produce so much? Or, maybe let the machines do the heavy lifting and we do the thinking?

41 THANK YOU!!