Deconstructing Data Science

Transcription

1 Deconstructing Data Science David Bamman, UC Berkele Info 29 Lecture 4: Regression overview Jan 26, 217

2 Regression A mapping from input data (drawn from instance space ) to a point in R (R = the set of real numbers) = the empire state building =

3 task predicting bo office revenue movie total bo office predicting stock movements predicting vote share $TWTR price at time t+1 Clinton 47%

4 Regression Supervised learning Given training data in the form of <, > pairs, learn ĥ()

5 Regression Can ou create (or find) labeled data that marks that value for a bunch of eamples? Can ou make that choice? Can ou create features that might help in distinguishing those classes?

6 Eperiment design training development testing size 8% 1% 1% purpose training models model selection evaluation; never look at it until the ver end

7 Metrics Measure difference between the prediction ŷ and the true Mean squared error 1 N (ŷ i i ) 2 (MSE) N i=1 Mean absolute error 1 N ŷ i i (MAE) N i=1

8 ŷ MAE MSE % of total MAE 98.6% of total MSE MSE error penalizes outliers more than MAE

9 Linear regression F ŷ = i β i i= β R F (F-dimensional vector of real numbers)

10 Polnomial regression F F ŷ = i β a,i + 2 i β b,i i=1 i=1 4 3 ^2 2 1 βa, βb R F (F-dimensional vector of real numbers)

11 Polnomial regression F F F ŷ = i β a,i + 2 i β b,i + 3 i β c,i i=1 i=1 i=1 5 ^3-5 βa, βb, βc R F (F-dimensional vector of real numbers)

12 Nonlinear regression Deep learning Decision trees Probabilistic graphical models Random forests Support vector machines (regression) Networks Neural networks

13 Number of Parameters order 1 (linear reg.) ŷ = F i=1 i β a,i F F order 2 ŷ = i β a,i + 2 i β b,i i=1 i=1 F F F order 3 ŷ = i β a,i + 2 i β b,i + 3 i β c,i i=1 i=1 i=1

14

15 instance space labeled data labeled data labeled data

16 degree 1, training MSE = 73.4

17 degree 2, training MSE = 71.9

18 degree 3, training MSE = 6.9

19 degree 4, training MSE = 6.6

20 degree 5, training MSE = 59.1

21 degree 6, training MSE = 5.2

22 degree 7, training MSE = 49.6

23 degree 8, training MSE = 46.8

24 degree 9, training MSE = 41.2

25 degree 1, training MSE = 35.8

26 degree 11, training MSE = 21.1

27 degree 12, training MSE = 18.4

28

29

30

31

32

33 Overfitting Memorizing the nuances (and noise) of the training data that prevents generalizing to unseen data

34 Sources of error Bias: Error due to mis-specifing the relationship between input and the output. [too few parameters, or the wrong kinds] Variance: Error due to sensitivit to random fluctuations in the training data. If ou train on different data, do ou get radicall different predictions? [too man parameters]

35 Low variance High variance Low bias High bias Image from Flach 212

36 Eample: High bias, low variance: Alwas predict Berkele geolocation on Twitter High bias, high variance: Predict most frequent cit in training data Low bias, high variance: man features, some of which capture true signal but capture random noise Low bias, low variance: enough features to capture the true signal

37 Ordinal regression In between classification and regression Y is categorical (e.g.,,, ) Elements of Y are ordered < < <

38 Ordinal regression task Y predicting star ratings movie {,, }

39 Computational Journalism Sarah Cohen, James T. Hamilton, and Fred Turner, Computational Journalism, Communications of the ACM (211) Slvain Parasie, Data-Driven Revelation? Epistemological tensions in investigative journalism in the age of big data, Digital Journalism (215)

40 Computational Journalism Changing how stories are discovered, presented, aggregated, monetized and archived (Cohen et al. 212) Draws on earlier tradition of computer-assisted reporting and precision journalism (Meer 1972)

41 Computational Journalism Database linking, e.g.: voting records to the deceased press releases from different members of congress indictments/settlements from U.S. attornes documents from SEC, Pentagon, defense contractors to note movement to industr (Cohen 212) DSA database of safet status of CA public schools + US seismic zones + school list from CA Dept of (Parasie 215)

42 Computational Journalism Information etraction: need to pull out people, places, organizations and their relationship from large (often sudden) dumps of documents. Analzing the relationship between entities

43 Computational Journalism Data-driven stories about large-scale trends Relationship between birth ear and political views NY Times (Jul 7, 214) Change in insured Americans under the ACA, NY Times (Oct 29, 214) 43

44 Computational Journalism Data-driven lead generation; the outliers in analsis that point to a stor

45 Computational Journalism Demands: High precision Fast turnaround Needs (Stra 216): Accurate document analsis Guided search Interactive methods

46 Project proposal, due 2/16 Collaborative project (involving up to 3 students), where the methods learned in class will be used to draw inferences about the world and criticall assess the qualit of those results. Proposal (2 pages): outline the work ou re going to undertake formulate a hpothesis to be eamined motivate its rationale as an interesting question worth asking assess its potential to contribute new knowledge b situating it within related literature in the scientific communit. (cite 5 relevant sources) who is the team and what are each of our responsibilities (everone gets the same grade)