DATA MINING SAMPLE RESEARCH: ACTIVITY RECOGNITION CLASSIFICATION IN ACTION

Transcription

1 DATA MINING SAMPLE RESEARCH: ACTIVITY RECOGNITION CLASSIFICATION IN ACTION 1

2 Mobile Activity Recognition Mobile devices like smartphones and smartwatches have many sensors Some sensors measure motion Tri-axial accelerometer Gyroscope GPS and other location sensors Activity Recognition is now pretty common but wasn t when this research started It took a year for our Fitbit order to be delivered 2

3 What is Activity Recognition? Identifying a user s activity based on data In our case the mobile sensor data from the accelerometer and gyroscope What type of data mining task is this? Classification How would you formulate this as a classification task? Not so obvious if you have not read the paper, since time dimension complicates things 3

4 More on Activity Recognition Examples of activities Walking, jogging, running, jumping, washing dishes, playing basketball, reading, partying, studying, eating, drinking, etc Why do we care? Context sensitive smart devices Fitness and health applications To track what we do for other purposes 4

5 The Data The data is collected at 20 Hz A timestamped sequence of numbers for each of 3 dimensions for both sensors 5

6 Walking Data Phone accelerometer Watch Gyroscope 6

7 Phone Jogging Accelerometer 7

8 Phone Accelerometer Standing Data 8

9 WISDM Activity Recognition Studies 2010 study using only smartphones Good results, but only 6 basic activities (29 subjects) More refined studies over next few years, including impact of personal models 2016 study: smartphones & smartwatches Good results over 18 activities (17 subjects) Hand-based activities including eating In progress Increasing test subjects to and more thorough evaluation of the four sensors Phone accel, phone gyro, watch accel, watch gyro, fusion 9

10 The 2016 Smartwatch Activities General Activities Walking* Jogging* Climbing Stairs* Sitting* Standing* Kicking Soccer Ball * These used in the 2010 smartphone study General Activities (hand-oriented) Dribbling Basketball Playing Catch with Tennis Ball Typing Handwriting Clapping Brushing Teeth Folding Clothes Eating Activities (hand-oriented) Eating Pasta Eating Soup Eating Sandwich Eating Chips Drinking from a Cup 10

11 Formulation as Classification Take the raw time series sensor data for nonoverlapping 10 second chunks and create one example Use higher level features to describe behavior over the 10 second period This is data transformation Mapping the data to a very different representation 11

12 High Level Features: 43 Total Average[3]: Average acceleration (per axis) Standard Deviation[3]: SD per axis Average Absolute difference[3]: per axis Average Resultant Acceleration[3]: average of square root of sum of squares of 3 values Time Between Peaks[3] Binned Distribution[30]: For each axis take max min value, create 10 equal sized bins, and record fraction in each bin 12

13 Types of Models Impersonal Models Generated using data from a panel of other users Personal Models Generated using data from the intended user. Must be separate from test data, as usual. 13

14 Results 14

15 Actual Class 2010 Study using Impersonal Model (IB3 Method) 72.4% Accuracy Predicted Class Walking Jogging Stairs Sitting Standing Lying Down Walking Jogging Stairs Sitting Standing Lying Down

16 Actual Class 2010 Study using Personal Model (IB3 Method) 98.4% accuracy Predicted Class Walking Jogging Stairs Sitting Standing Lying Down Walking Jogging Stairs Sitting Standing Lying Down

17 2010 Study Accuracy Results % of Records Correctly Classified Personal Universal Straw IB3 J48 NN IB3 J48 NN Man Walking Jogging Stairs Sitting Standing Lying Down Overall

18 2016 Study Universal Models Algorithm Phone accel (%) Watch accel (%) Watch gyro (%) RF J IB NB MLP Average Note: based on 18 activities 18

19 2016 Study Personal Models Algorithm Phone accel (%) Watch accel (%) Watch gyro (%) RF J IB NB MLP Average

20 2016 Detailed Summary Results Random Forest Impersonal (%) Personal (%) Activity Watch accel Phone accel Watch gyro Watch accel Phone accel Watch gyro Walking Jogging Stairs Sitting Standing Kicking Dribbling Catch Typing Handwriting Clapping Brush Teeth Fold Clothes Eat Pasta Eat Soup Eat Sandwich Eat Chips Drink Overall

21 Actitracker The phone-bases research was incorporated into a deployed app/system called Actitracker The development effort to handle real-time activity recognition was substantial Actitracker is no longer supported 21

22 New Directions My WISDM Lab is finishing work on the smartwatch activity recognition Beginning to consider data mining of static sensors since cheap Bluetooth sensors are now available Research related to Internet of Thing (IoT) 22

23 Data Collection Collecting the data is quite time intensive We are still collecting data for definitive set of AR experiements so if you want to volunteer, please me. Data collection usually at RH Will provide an Amazon gift card 23