Step Counting Investigation with Smartphone Sensors

Transcription

1 Rochester Institute of Technology Department Of Computer Science FALL 2016 Step Counting Investigation with Smartphone Sensors Author: Supervisor Dr. Leon Reznik December 12, 2016

2 Contents 1 Introduction 2 2 Background Research Data collection for the Step Counter Accuracy Study Results from the Step Counter Accuracy Study Inferences from the study Experimental Procedure Data Acquisition Approaches considered Step Counting Mechanism Results 11 5 Conclusion 13 1

3 Abstract Over the past 25 years, cellphones have gone from being devices without even a display which are only used for calling, to all-in-one devices capable of doing everything. They can not only do everything, but they are able to perform so well that they have singlehandedly eradicated product lines like IPods, Pagers, most digital cameras and for that matter, even reduced the need for a laptops. On the other hand, they are still lagging in terms of fitness tracking, more specifically step counters. Step Counters, in some form, have been around since at least the last 400 years [1]. They used to be and are still available as standalone devices whose sole or major responsibility was to track the number of steps taken by a user. Today s dedicated fitness trackers seem to have a really high accuracy rate of around 97% [8]. Smartphone sensors on the other hand, are still playing catch up. Even though the quality of the MEMS on smartphone sensors have improved dramatically over the past 10 years or so, the step counts from even the flagship phones on the market today still have really high error rates. There is a need to improve the quality of the step counters in smartphones and this is the problem being tackled in this project. 1 Introduction Step Counters have gained popularity in the past few years. While heath conscious people have always been interested in keeping a track of their step counts, calories they burn and the distances they cover during their workout sessions or during their daily life, even the average person now tracks the same metrics more regularly. This has been attributed to the rise of readily available and built in applications in their smartphones which facilitate this behavior. Smartphones on both Android and ios have had step counters since the past 6 years, and their accuracy as well as the general structure and purpose the applications has improved gradually over time. The have evolved into relatively more accurate step counters with a feature set that now also includes entire slew of fitness tracking metrics like distance walked, steps climbed and the new Samsung phones even having a heartrate sensor built right into the hardware of the phone. While these advancements improve the package as a whole, the step counting is still not accurate enough. There is a growing trend of people switching to FitBit s line of step counting wearables and even some lesser known brands. This shift of not relying on smartphone applications to dedicated fitness trackers is due to the perception that dedicated 2

4 fitness trackers generally are more accurate as compared to the smartphone ones. This is supported to by the studies conducted across the board like in the case of [2] and also in a background research conducted as part of preparation for this project. The goal of the project is to do an experimental study to try and improve the quality of smartphone sensors. The reason is that since we already depend on smartphones for pretty much everything we do, why not have a accurate step counter on it too.this can be accomplished by utilizing either one or multiple smartphone sensors. For this, the sensors such as the accelerometer, gyroscope and the magnetometer were considered. Another aspect of this project is to find out how the positioning of smartphones with respect to the body affects the step counting. 2 Background Research Before concluding on whether smartphone step counters are actually inaccurate and that the fitness trackers in dedicated wearables were better at step counting as discussed in [2] and [8], it made sense to actually conduct a study to verify whether this assertion. Hence, an analysis of smartphone sensor data was conducted, which included research about sensors like gyroscope, accelerometer and magnetometer, but mainly focused on a study to compare the quality of smartphone sensors in smartphones versus a dedicated fitness tracking wearable. 2.1 Data collection for the Step Counter Accuracy Study For this study[3], the smartphones considered were the Samsung Galaxy S7 and the Nexus 6P and the dedicated fitness tracking wearable chosen was the FitBit Alta. To have a broader analysis of the quality of sensors and the impact the final algorithm has on the quality of the step counting a combination of third party and built in applications were installed on the two smartphones. For the Samsung Galaxy S7, the Google Fit, Noom Walker, Pacer and Samsung s own S Health apps were installed. As for the Nexus 6P, the Google Fit, Pacer and Noom Walker apps were installed. Two subjects were each assigned a FitBit Alta and one of the smartphones and asked to use them daily whenever they were walking. Precaution was taken to ensure that both the subjects were using both the devices together always so that the data was as accurate as possible. Data for 20 days was collected from each of them. So, the step count data from various applications from both smartphones and the Fitbit Alta were collected. 3

5 2.2 Results from the Step Counter Accuracy Study Now, the step count data from the various FitBit devices has been known to be accurate within a 2-3% error rate margin. Hence, for this study, the step count data from them was considered as the baseline. The raw data from subject 1 using the Nexus 6P and subject 2 using the Samsung Galaxy S7 along with their respective Fitbit data is represented in Figures 1 and 2 [3] as tables. Also a graph representing the step data for each day is displayed in Figures 3 and 4[3]. Figure 1: Step count table for Samsung Galaxy S7 and the assigned FitBit Alta From the step data, it is clear that there is a large discrepancy between the data collected in the step counts from the smartphones. The percent error of the data collected from the smartphones when the step data from the Fitbit is considered absolute is represented in Figures 5 and 6[3]. 2.3 Inferences from the study Also, another interesting analysis can be made by deriving the cumulative error values when the data from both smartphones is used together. We can gauge the overall performance of the smartphone in general and also identify the error rates of each application. This is represented in Figure 7 [3]. It is clear from this, that the 4

6 Figure 2: Step count table for Nexus 6P and the assigned FitBit Alta step counter quality of the Google Fit application is really good. It ranges in the region of roughly 3-4% error rate. Another factor is that overall, the error rates on the flagship devices from Google and Samsung was still averaging between 7-10% error rate. This implies that on devices with not flagship like qualities, especially ones which don t even have a dedicated step counter MEMS, the quality will be somewhere in the range of 20% error rate. This is because, Noom Walker does not use the data from the step counter on a smartphone to calculate the number of steps. Hence, there is a need to develop a better method to get the step count on smartphones. 3 Experimental Procedure For this project, Android platform was chosen. While researching for the best approach to improve the accuracy of smartphone sensors I researched techniques from various papers which included using the various built in sensors in Android devices. [7], [4] were a few papers which provided techniques to use Accelerometer data for step counting. A few spoke about using magnetometer for step counting whereas [5] and [6] represented gyroscope based smartphone sensors. While all of them provided with compelling reasons to choose any of the above 5

7 Figure 3: Step count graph for Samsung Galaxy S7 and the assigned FitBit Alta mentioned sensors, based on the techniques mentioned in the papers and the given accuracy and techniques, the gyroscope based technique mentioned in [5] seemed to be the optimal choice. 3.1 Data Acquisition The data collection for this project involved creating an Android application. This application is responsible to collect the information from various smartphone sensors. In this case, since this was an experimental analysis and there was a possibility that the technique mentioned in [5] may not be as optimal as represented and if some modifications to that technique still did not yield any positive results, the accelerometer and gyroscope data both were collected. This application has a start and stop button. Figure 8 represents the layout of this application. The accelerometer and gyroscope sensors on smartphones are really sensitive. Even the smallest movements that you would not even notice, could trigger the accelerometer and gyroscope to record a change. The application was designed in such a way that all these movements were recorded. The procedure to do the data collection was carefully designed. Five subjects were 6

8 Figure 4: Daily step count for Nexus 6P and the assigned FitBit Alta chosen. Each of these subjects had a different height and body structure so that the data could be as diverse as possible. Each subject was given 2 smartphones, one to place in the pocket of their pants or jeans and the other one in their shirt pocket. Each of the 5 subjects had to walk 10 times. Starting with 50 steps, then 100 steps, then 150 steps, all the way to 500 steps at intervals of 50 steps. Two Google Pixel smartphones were used for each and every walk so that there is no issue of different hardware affecting the accelerometer and gyroscope data discrepancies. For each walk, the start time, end time, actual number of steps and all the accelerometer and gyroscope data was recorded. 3.2 Approaches considered One of the first approaches considered was one that was directly described in [5]. This approach was described in the paper as one which would even work when a person would be walking slowly. To achieve this, they used a sixth-degree low pass Butterworth filter on the gyroscope data. The data collection technique in the application needed to be changed for this testing. This is because the Butterworth filter required the gyroscope data to be collected at fixed intervals. When the filter was then applied to the gyroscope data, the number of steps that were finally calculated 7

9 Figure 5: Daily Error Percentage for the Samsung Galaxy S7 w.r.t to FitBit Alta was very low. This could be attributed to the fact that since the data was collected at fixed intervals to facilitate the possibility of using the Butterworth low pass filter, it left out some of the steps and hence the step counter was off. Another approach to counter this problem was to use a Gaussian low pass filter instead of the Butterworth filter, but even this approach yielded slightly better but nonetheless inaccurate results. 3.3 Step Counting Mechanism Out of the two approaches considered above one of them is used in [5]. The Butterworth filter and the Gaussian filter are both preprocessing techniques. These and many such approaches are sometimes used in the analysis of gyroscope data because the data from the gyroscope is usually very noisy. The reason for this apart from other reasons is that this sensor as well as most Android smartphone MEMS are very sensitive. This means that they are triggered by even the smallest movements. Hence, it is necessary to make sure that the data collected from these sensors is handled carefully and by keeping this into consideration. So, while none of the standard filtering techniques are not applied to the data, an alternate approach to 8

10 Figure 6: Daily Error Percentage for the Nexus 6P w.r.t to FitBit Alta identification of real significant data versus redundant data was devised. The data collection phase in its current phase has one major drawback. The application as displayed in Figure 8, has a start and stop button to help facilitate the step recording process. After clicking the start button, the phones need to be placed in the shirt and pant pockets. This means that during this time, the recording is already on, and that this may interfere with the step counting process. Hence, the initial and final few readings are discarded from the analysis. This is done by a metric which checks the total time, and accordingly discards the first and last 5 seconds of the data collected. Since, each reading from the sensor is tagged with a timestamp, this can be accomplished. This cleaning takes place every time a new walk session is started. Now, based on the method mentioned in [5], almost all smartphones in the pant pocket and the shirt pocket are usually placed vertically or slightly inclined. It is also mentioned that most humans walk in such a way that the motion of their body while walking is sinusoidal. This can be easily identified by the pattern in which ones leg moved while walking. Thus, according to this, the X axis motion on the gyroscope would be in sinusoidal motion. This is the basic principle used in this project s proposed technique too. So since the motion is sinusoidal, every time 9

11 Figure 7: Percentage error rate w.r.t to FitBit Alta there is a zero crossing, a step can be recorded and in that way, step count can be calculated. In reality, if we use this approach on the cleaned gyroscope data, we get more than double the actual number of steps, even triple in some cases. Hence another important factor to be able to count steps is that we need to be able to identify the real steps from huge amount of data collected from the sensors. The data collected from the walks is used for this purpose. The fact that raw gyroscope data estimates two to three times more steps than the actual number of steps indicates that there are a large number of false positives. This can be identified easily by observing Figure 9. This figure represents a graph of gyroscope data. As you can observe, there are a lot of places where there is a zero crossing, but it represents data points which are not conclusive enough to be steps. Hence, as opposed to filtering of such points in the form of a Butterworth or a Gaussian filter, a more granular approach of dealing with actual gyroscope cutoff values was determined. This was done, by analysis of actual steps versus the steps calculated when the cutoff values were modified. Since the data collected was significant in size, suitable lower and upper thresholds for cutoff values were selected. Obviously, since the data was collected in both the pant pocket and the shirt pocket, cumulative cutoff points were selected. Also individual cutoff values for both pant and shirts were calculated separately too. The values represent the distance between 10

12 Figure 8: Layout of data collection application the crest and trough values. The cumulative, pant pocket and shirt pocket cutoff values for the upper and lower thresholds are displayed in Figure 10. With the help of these cutoff values, the real steps can be determined. During the analysis of the gyroscope data after a walk, whenever there is a zero crossing, the corresponding crest and trough data is maintained and if the difference between these values is in between the upper and higher threshold values, a step is counted. During the span of a walk, every time this condition is satisfied, the number of steps are incremented. Hence, the total step count is calculated using this approach. 4 Results As mentioned in the previous section, 3 separate metrics were used to calculate the total number of steps taken. These are based on the cutoff values specified for shirt, pant and a common metric which is a cumulative cutoff value which should ideally work for both. The results from the experiments are specified in Figure

13 Figure 9: X-Axis Gyroscope data Figure 10: Cut Off Values It is evident that if the specific metrics for shirt and pant are used, the accuracy of the step counting is close to 87%. Whereas, if the common metric is used for step counting filtration, the accuracy is reduced to an average of around 79%. So clearly, this approach works best when the metrics specific to the position of the smartphone sensor are utilized. The data is also visualized in a graphical manner in Figure 12. Another interesting trend in the results, which can be observed in Figures 11 and 12 is that, based on the current metrics, the position of the phone impacts the final step count tremendously. There is a significant difference between the step counts observed for the same walk, where the step count from the phone placed in the pant pocket is on average higher than the original step count, and when placed in the shirt pocket, is lower than the step count. This can be attributed to the fact that the number of significant sinusoidal movements will be lower when the phone is in 12

14 Figure 11: Step Count Values for Different Positions and Metrics the shirt pocket, as compared to when the phone is in the pant pocket. A table representing the percentage error for different measures of actual steps is represented in Figure 13. It indicates that on an average, the accuracy of the step counting remains roughly the same, improving slightly as the number of steps increase. 5 Conclusion With a dedicated metric for positioning of the smartphone in both pant and shirt pocket, the technique used in this paper was able to achieve an accuracy of 87% on an average. If we consider, only the positioning of the smartphone in the pant pocket, which is the most common place, the accuracy is improved even further. Based on the results, it is clear that, as compared to other smartphone applications that don t use the built in step counter MEMS, this project provides an improvement in the step counting accuracy. While the technique specified in this project does improve the accuracy of step counters over applications that don t use the dedicated step counter MEMS, there is still an effort needed to improve on the ones that do. A few improvements like, a larger set of data samples for the analysis of the cut off values would improve the accuracy of the step counting even further. 13

15 Figure 12: Graphical Representation of Step Counts Figure 13: Percentage Errors of Step Counts for Different Positions and Metrics 14

16 The proposed technique could also be extended by working on technique for adaptive cutoff values. Essentially, if the person walking starts walking faster, lengthening their stride, the threshold value should be adapted accordingly to accommodate this change in walking pattern. Such improvements could improve this project even more providing us with a perfect step counting technique which could be used in even the most basic smartphone. 15

17 References [1] Who Invented the Pedometer, who-invented-the-pedometer/. [2] Meredith A Case, Holland A Burwick, Kevin G Volpp, and Mitesh S Patel. Accuracy of smartphone applications and wearable devices for tracking physical activity data. Jama, 313(6): , [3]. A Review of Mobile Sensors, unpublished. [4] W. Hongman, Z. Xiaocheng, and C. Jiangbo. Acceleration and orientation multisensor pedometer application design and implementation on the android platform. In 2011 First International Conference on Instrumentation, Measurement, Computer, Communication and Control, pages , Oct [5] S. Jayalath and N. Abhayasinghe. A gyroscopic data based pedometer algorithm. In th International Conference on Computer Science Education, pages , April [6] Y. P. Lim, I. T. Brown, and J. C. T. Khoo. An accurate and robust gyroscopebased pedometer. In th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages , Aug [7] Martin Mladenov and Michael Mock. A step counter service for java-enabled devices using a built-in accelerometer. In Proceedings of the 1st International Workshop on Context-Aware Middleware and Services: Affiliated with the 4th International Conference on Communication System Software and Middleware (COMSWARE 2009), CAMS 09, pages 1 5, New York, NY, USA, ACM. [8] Judit Takacs, Courtney L. Pollock, Jerrad R. Guenther, Mohammadreza Bahar, Christopher Napier, and Michael A. Hunt. Validation of the fitbit one activity monitor device during treadmill walking. Journal of Science and Medicine in Sport, 17(5): , Copyright - Copyright Copyright Agency Limited (Distributor) Sep 2014; Document feature - Tables; ; Graphs; Last updated