Gait Based Personal Identification System Using Rotation Sensor

Transcription

1 Gait Based Personal Identification System Using Rotation Sensor Soumik Mondal, Anup Nandy, Pavan Chakraborty, G. C. Nandi Robotics & AI Lab, Indian Institute of Information Technology Allahabad, India ABSTRACT Every individual has a unique walking pattern which may be used as a signature for identification. This paper describes the construction of a feasible human identification device using human gaits (Walking pattern). We have built a very low cost wearable suit mounted with eight rotation sensors, controller, software and power unit for measuring eight major joints of human body which are involved in locomotion. Different person s walking patterns can be captured with this suit which has been named IGOD (Intelligent Gait Oscillation Detector). IGOD meets an excellent standard of accuracy for capturing movements of major joint s oscillation during locomotion. A gait classifier has been designed using ANN (Artificial Neural Network) and LDA (Linear Discriminant Analysis) combined with a bottom up binary tree approach. The designed system is free from environment and projection uncertainties, which normally plague any vision based systems. The device has been tested with 30 subjects (Persons). We have achieved a high recognition accuracy of up to 100% over this limited sample of 30 persons. Keywords: Human Gait analysis for identification, Non Vision based approaches, Artificial Neural Networks, Linear Discriminant Analysis (LDA) with Binary Tree, Intelligent Gait Oscillation Detector (IGOD), Arm swing. 1. INTRODUCTION Research on human identification at a distance through video surveillance was started from 1999 and it has become popular since There were several DARPA (Defense Advanced Research Project Agency) projects involved in video surveillance based applications for criminal investigations and other asymmetric threats of national security. In gait biometric human walking pattern is an important signature for identification. The objectives are: To extract body biometric features. To identify the person from those features. Normally such work is done using vision based techniques. The advantages with such techniques are that they can be performed without the knowledge of the subject, and therefore no physical cooperation from the subjects is required. In recent years plenty of research papers have been published on vision-based human identification system. Although it attracts an enough impact on personal identification but there is no full proof methodology which can be applied on the real world. Gait pattern is considered to be the most promising attribute for human identification and video surveillance system. The accurate measurement of gait pattern and extraction of appropriate gait features provides an advantage for personal identification. There are several problems and complexities involved in vision based approaches. It attracts its inability to identify person accurately due to environmental changes, poor illumination and low resolution of camera, background changes variations of different cloths, speed variations, carrying with bags and changes of walking direction which are affected in person walking. Also the vision based techniques for gait recognition required extraction of silhouette image from the original image data which needs computationally very complex algorithm. The challenging and feasible system of wearable sensor based personal recognition has been introduced in this paper. The most common wearable sensors which are used in this recognition systems are Accelerometer, Gyro and Force sensors (from ) [1]. For the first, we introduced rotation sensor based personal recognition system which keeps the system low cost with better accuracy. A multi sensor based wearable biometric suit has been designed to capture human gait oscillations of eight major body joints simultaneously [2]. It is then followed by finding the suitable features which could be useful for human recognition purposes. ANN and LDA with bottom up binary tree technique have been used as a gait classifier separately for personal identification. The advantage with our method is the signal directly proportional to the rotation angle which we have already calibrated. This self made recognition device could be considered as a biometric tool for the research in Human Gait analysis and the extraction of gait signatures in medical diagnosis applications [2, 3]. The major drawback of our system is that it requires full cooperation from the subject. However, it offers much better accuracy in dynamic changes of the environment. We have tabulated different non-vision based techniques used in gait analysis in Table I. The methodology of this work has also been mentioned under the heading of techniques. 395

2 Table 1: Related work in vision based and non vision based approaches. Authors / References Gafurov et al. [4], [5] Nakajima et al. [6] Middleton et al. [7] Ottaviano et al. [8] Derawi et al. [9], [10] Rong et al. [11] Pan et al. [12] Moustakidis et al. [13] Technique Feature Data Data Non-Vision Biometrics set / No. of class Accelerometer Sensor based gait recognition. Acceleration in 3 Gait 21 directions Pressure sensor based personal recognition system Direction and Footprint 10 with Euclidean distance taken as a classifier. position Floor sensor based recognition system with stride Time on toe to time Gait 15 length, stride cadence and time on toe to time on heel on heel ratio ratio taken as a feature. Human walking characteristics analysis using Cassino Tracking System with force sensor. Accelerometer sensors Based Gait Authentication with Cross Comparison and Cyclic Rotation Metric taken as recognition methods. Identification of human by walking pattern using gait acceleration with the help of accelerometer sensor. Accelerometer sensor based gait recognition for human identification. Human Identification using Ground Reaction Force. Here SVM taken as a classifier. Position, Orientation and force. Gait cycle detection using DTW Gait cycles, gait feature code using DTW Acceleration in 5 different body parts Medial lateral, Anterior Posterior, and vertical force Human Walking NA Gait 60 Gait 35 Gait 30 Gait 40 Figure 1: (a) Rear and (b) Front view of IGOD suit. (c) The locations of the eight rotation sensors are shown. The sensors are placed at shoulder joints, elbow joints, hip joints and knee joints. In total eight rotation sensors. 2. OBJECTIVE OF THIS RESEARCH WORK The main objective of this research is to develop a low cost, portable, non-invasive wearable sensor based biometric suit which could be applied to different applications like, Human Identification Human Robot Interaction Gesture Classification Human Computer Interaction Diagnostic Gait Signature Detection Find the effect on gait with different age, body weight and different diseases. Analysis of abnormal gait signature Bio-inspired Bipedal robot controller [14] etc. In this paper several contributions have been shown which 396

3 periodically appeal their own success. The contributions are Introduce a rotation sensor based biometric wearable suit for human identification. Personal recognition system which can be applicable to real world situation. Simple feature selection procedure which could help us to recognize the person with up to 100% accuracy. Choose the real time pattern recognition algorithm compatible with our feature. Better accuracy has been achieved rather than existing approaches. Hand oscillation for human identification. 3. DATA COLLECTION AND FEATURE EXTRACTION In this work, we have used a self made wearable body suit for capturing the human gait data in real time. Those data are being represented in the form of gait angular movement. The following descriptions concentrate on data capturing technique followed by feature extraction process. 3.1 Body Suit Description Our self made multi-sensors based biometric suit is being made by rotation sensors (i.e. potentiometer taken as a transducer and the resolution of the potentiometer is degree) for measuring accurate joint angular movement. The sensors are mounted on the steel and aluminum plates for maintaining the rigidity. Those rigid links are placed on the different body joints (i.e. 2 knees, 2 hips, 2 shoulders and 2 elbows) for measuring the major body joins which are involved in human locomotion in the form of angular rotation [2]. Initially, the IGOD suit is used to get the raw data which is in the form of digital counts between Then we find out the zero corrected data from the raw data which eventually is to be converted in the corresponding angle values. The entire process is being done by the following formula given by: θθ iiii = φφ iiii ρρ jj , where θθ is the desired angle, φφ is the raw data captured from the suit, ρρ is the initial digital count, wwheeeeee ii = 1 to Total no. of samples and jj = 1 to 8 (corresponding joint). Here initial digital count ρρ implies all the joint angle values captured at the position of the person standing without any joint movement. (a)) and front (Figure 1 (b)) view. Figure 1 (c) clearly shows the locations of the eight rotation sensors. One of the major difficulties involved in such wearable sensor based system talks about the sensors cannot be attached at the same position over all the time or at the accurate pivot point of each joint angle. As a result the system is needed for proper calibration. At first the sensor has been mounted loosely around the joint pivot point. The joint is made to move the whole range of its oscillation. After a few oscillations the loosely mounted sensor navigates to the most preferred position of the joint pivot point in tune with the natural oscillation of the particular joint. Now the sensors are being tightly fastened. By doing this process we can achieve a good level of confidence in order to ensure the repeatability in the position of the sensors. 3.2 Data Collection The experimental data have been collected for our research work using this prototype. We have created the Gallery data bases for normal walking for 30 (25 male and 5 female) different humans with 30 sec time interval taken in 10 different sessions. The age of our volunteers is about years. According to our different volunteer s point of view the suit does not affect their natural walking. In each session we have taken 30 sec of walking data in order to offer some amount of freedom to their normal walking. It was observed that a person wearing this suit required at list few gait cycles to stabilize and naturalize his/her intrinsic gait movement. Therefore 30 sec of data was taken. 3.3 Feature Extraction Suitable feature extraction is one of the most challenging aspects of any pattern recognition problem. First we need to understand the different phases of a gait cycle. Figure 2 illustrates this different phases of the gait cycle. The left leg is represented in red, while the right leg represented in black. When right leg is in the stance phase the left leg goes through a swing phase and then roles are reversed forming a complete gait cycle. In our previous work [2] we had attempted to show that the theoretical concept of human locomotion and the practical data taken by this suit are equally be in similar. Figure 1 depicts a person wearing the IGOD suit with rear (Figure 1 Figure 2: Different phases of human locomotion within a gait cycle. 397

4 Having studied the human physiology we found that each and every gait cycle is independent on each other in human locomotion. It belongs to within the session when the walking pattern tends to be very straight forward and highly stable. Therefore, our objective is to find out the suitable features for recognition purposes within the gait cycle. The feature vectors are selected by calculating the mini mum and maximum angle values of each major joint within the gait cycle and the time propagated by that gait cycle. As the time plays a prominent role in an individual recognition using human gait so our total feature matrix is composed of 17 dimensions. Every gait cycle has been decomposed into substantial parts which are known as one sample of oscillation. The number of sample is extracted from the person s different gait which is dependent on the normal walking speed. The angular movement for each leg joint is based on the stride length. Figure 3 and Figure 4 explain the total feature extraction process for the captured gait data set. In those two figures we have shown only two consecutive gait cycles in the same gait. Where θ 0 is threshold angle, which we fix between the minimum and maximum. We then compute k th maxi mum t pk θθ kk5 = θθ ii5 ii, θθθθ kk5 = θθ ii5. tt ii5 tt pppp = θθ kk5 = θθ θθθθ ii5 θθ ii5. tt ii5 kk5 ii TT kk = tt pppp 1 tt pppp ii ii, ffffff θθ ii5 > θθ 0 The period T k, is the time gap between two consecutive peaks k and k-1. Hip joint oscillation of a human is close to a sinusoidal motion. Therefore the period of oscillation T could be determined from the time gap between two consecutive maxima. Any measured signal has some amount of noise therefore determining the peak time t pk for the maximum will not be straight forward. Fowling centroid method is used to calculate t pk. Figure 5 with graphically illustrates this centroiding technique and calculating the period T k. The shaded portion above the threshold θ 0 shown in the figure is used for weighted centroiding. Figure 5: Graphical illustration of centroiding technique to compute the peak time t pk using a threshold θ 0. The period of oscillation T k is calculated the peak time t pk. Figure 3: Oscillation curves of the Right Elbow, Shoulder, Hip and Knee. The features extracted are the maximum and minimum joint angle value and the gait cycle. These have been marked in the figure. 398

5 Figure 4: Oscillation curves of the Left Elbow, Shoulder, Hip and Knee. The features extracted are the maximum and minimum joint angle value and the gait cycle. These have been marked in the figure. (Please note the gait cycle shown in this and figure 3 should be the same) 4. METHODS USED FOR RECOGNITION The Figure 6 describes the steps for recognizing the gait pattern. The process begins by recoding the movement-joint angles values (First 3 blocks of Figure 6). Feature extraction matrix is created from the measured joint angles. Two pattern recognition algorithms ANN and LDA have been used for recognition. constituted in such a manner so that the each layers output could be propagated with minimum error rate. In our research work we have used ANN with Feed forward Networks and 30 hidden Neurons [15]. We have considered 60% of the total sample is for training and rest of the sample for testing. We have been achieved up to 100% recognition accuracy. 4.2 Linear Discriminant Analysis (LDA) We have used Linear Discriminant Analysis (LDA) [16]- [19] for several reasons. It helps us to provide a better understanding of the distribution of our feature data. In our research it is more efficient for data classification which offers us very good class separability among our classes of data. When we have done transformation of our original data sets to a 2D space it is observed that a decision region has been drawn effectively between our given classes. Figure 6: Recognition phases. We can find the LDA function from the Bayes' rule. If we ha ve n classes, the Bayes' rule is assigning the object x to class a, where PP(aa xx) > PP(bb xx), ffffff aa bb. We cannot get PP(aa xx) directly from the measurement but we can obtain PP(xx aa). So, the modified Bayes Rule is to assign the object x to class a, where 4.1 Artificial Neural Network (ANN) ANN is a very well known and very well acceptable pattern recognition tool applied in any machine learning applications. We have chosen ANN in gait identification for two reasons: It gives us the advantage to deal with our nonlinear data. The highly non linear structured neurons are PP xx aa. PP(aa) > PP xx bb.pp(bb), ffffff aa bb PP xx kk.pp(kk) PP xx kk.pp(kk) kk kk 399

6 That is PP(xx aa). PP(aa) > PP(xx bb). PP(bb),ffffff aa bb If we assume that, PP(xx aa) is Multivariate Normal distribution. So, we get 11 PP xx aa = (2222). CC aa ee[ (xx μμ aa )TT CC aa 11 (xx μμ aa ) ] llll( CC ) 2222 nn PP(aa) + xxcc 11 xx TT 22μμ aa CC 11 xx TT + μμ aa CC 11 μμ aa TT < llll( CC ) 2222 nn PP(bb) + xxcc 11 xx TT 22μμ bb CC 11 xx TT + μμ bb CC 11 μμ bb TT, ffffff aa bb So, we have μμ aa CC 11 xx TT μμ aacc 11 μμ aa TT + llll PP(aa) > μμ bb CC 11 xx TT μμ bbcc 11 μμ bb TT + llll PP(bb), ffffff aa bb Let ff ii = μμ ii CC 11 xx kk TT μμ iicc 11 μμ ii TT + llll (PP(ii)) Where, μμ aa is vector mean and CC aa is covariance matrix of class a. Now assign object with measurement x to class a if, PP(aa) (2222). CC aa ee[ PP(bb) > (2222). CC bb That is ee[ (xx μμ aa )TT CC aa 11 (xx μμ aa ) ] (xx μμ bb )TT CC bb 11 (xx μμbb ) ], ffffff aa bb PP(aa) 11 ee 22 (xx μμ aa )TT CC 11 aa (xx μμ aa ) CC aa > PP(bb) 11 CC bb ee[ 22 (xx μμ bb )TT CC 11 bb (xx μμbb ) ], ffffff aa bb We can write this equation as llll( CC aa ) 2222 nn PP(aa) + (xx μμ aa ) TT CC aa 11 (xx μμ aa ) > llll( CC bb ) 2222 nn PP(bb) + (xx μμ bb ) TT CC bb 11 (xx μμ bb ),ffffff aa bb If all covariance matrices are equal CC = CC aa = CC bb, then we can write llll( CC aa ) 2222 nn PP(aa) + (xx μμ aa ) TT CC 11 (xx μμ aa ) > llll( CC bb ) 2222 nn PP(bb) + (xx μμ bb ) TT CC 11 (xx μμ bb ),ffffff aa bb We can write (xx μμ aa ) TT CC 11 (xx μμ aa ) into xxcc 11 xx TT 22μμ aa CC 11 xx TT + μμ aa CC 11 μμ aa TT so, the inequality becomes So, Assign object with measurement x to group a if ff aa > ff bb, ffffff aa bb. The second term of this function that is μμ ii CC 11 μμ ii TT is actually Mahalanobis distance, which gives the distance to measure dissimilarity between several classes. That is Linear Discriminant functions. Thus, Linear Discriminant Analysis has assumption of Multivariate Normal distribution and all classes have the same covariance matrix. But, we can calculate same covariance matrix of all classes CC by the formula CC = 11 nn pp pp ii=11 iicc ii, where pp the total is number of sample, pp ii and cc ii is the number of sample and covariance matrices of ii th class. nn is the number of classes. In our case we have used this Linear Discriminant function combination with a bottom up Binary Tree structured classification rule to recognize different walking patterns. This technique of using the Binary Tree Structured was to arrest the exponential error growth rate. We will explain the working of our algorithm by considering 4 classes. These classes will have following 4 feature matrix R 1, R 2, R 3 and R 4. Person 1 P 1 => R 1, Person 2 P 2 => R 2, Person 3 P 3 => R 3 and Person 4 P 4 => R 4. The unknown test feature X is compared in binary tree method in one case with R 1 and R 2 and the other with R 3 and R 4. In both cases LDA maximum function is derived. These outputs in pairs (In this case one pair) are again compared with X in binary tree method. This will continue till all the classes are checked. Figure 7 shows the process for above example with 4 cases. In reality we make use of this technique with 30 classes instead of 4 classes. In this case it has been notified that the most dominating class which is actually being associated with test class X will reach to the last node of this tree. 400

7 Table 2: Performance Evaluation Table No. of Subjects Performance Gafurov et al. [4][5] 21 5%-13% EER Nakajima et al. [6] 10 85% RR Middleton et al. [7] 15 80% RR Derawi et al. [9], [10] %-20% EER Rong et al. [11] % EER Pan et al. [12] % RR Moustakidis et al. [13] % % RR Figure 7: Binary Tree structured based reorganization process with LDA. 5. RESULT ANALYSIS The gait recognition result deals with the feature selection technique and recognition technique applied in personal identification. In our work we have applied ANN and LDA separately. In this work we have taken 30 classes (30 different humans natural walking data). Both the methods give us up to 100% recognition result. In our work we randomly select the training set and testing set from our feature matrix in 60% and 40% ratio. We run this system for 30 times and for all the time we have achieved up to 100% recognition accuracy. It proves that our feature selection process and choosing those recognition techniques went through in positive directions. We have also attempted to apply those recognition methods with consideration of less number of features. As a result, the recognition rate gets considerably diminished. If we choose only the leg features then the average recognition rate by applying those methods are up to 87%. If we choose the leg feature with gait cycle time then the recognition method gives us about 93% recognition rate. Table 2 is the comparison between our adopted approach and others related approach in terms of presenting recognition performance. In this table EER means Equal Error Rate and RR means Recognition Rate. Compare to the other methods our result is extremely promising. It is hoped that a larger data set than 30 will also have a recognition rate close up to 100%. In This Paper 30 Up to 100% RR 6. CONCLUSION & DISCUSSION We are fully aware that current system will require full cooperation of the individual whose gait signature we would like to indentify. This individual should cooperate to wear our suit and lend his gait signature. There are also other biometric cues for human identification such as face iris, and finger printing. These biometric techniques have been long studied and tested on very large scale data set. It is therefore obvious to wonder why build new technique when other methods are functioning sufficiently well. The method we have proposed might seem irrelevant, but we believe that it has a far implication in understanding the health and structure of particular human being. Through further studies we would like to establish that every human being has a distinct gait signature. Most of gait analysis (Vision or non-vision) done by others some of which has been tabulated in Table-1, have only used the leg movements for identification. We strongly feel that the upper body movement (arm swings) has strong connection related to gait stability and locomotion which is strongly evident from our data. There is a strong correlation between the speed of locomotion and hand swing oscillation. As a future work we would like to investigate if this distinct human gait signature has a distinct correlation with the individual s body structure. Can aging effect be noticed in the gait signature? Does diseases such as Knee Osteoarthritis, Hip Osteoarthritis, Cerebral Palsy, Parkinson, and Hemiplegic Gait have distinct signature and can it be identified at the initial stages during the setting in of the diseases. Through our method we would like to construct a gait signature database with other biometric information such 401

8 as weight, height, sex and age of normal human beings and people with a known history of the above medical diseases. This will be a long drawn process. The present work is the first step in understanding the human gait signature. In this context Gait Based Personal Identification System will have grate relevance. ACKNOWLEDGEMENT This project was developed under the funding of Indian Institute of Information Technology Allahabad, India. Details of our designed suit were published in the Proceedings of the 3 rd International Conference on Contemporary Computing, Noida, India, 2010 [2]. REFERENCES [1] C. C. Yang and Y. L. Hsu A Review of Accelerometry- Based Wearable Motion Detectors for Physical Activity Monitoring, Sensors, Vol. 10, pp , [2] S. Mondal, A. Nandy, A. Chakrabarti, P. Chakraborty and G. C. Nandi, A Framework for Synthesis of Human Gait Oscillation Using Intelligent Gait Oscillation Detector (IGOD), Communications in Computer and Information Science, vol. 94, Contemporary Computing, Part 7, pp , [3] T. W Calvert, J. Chapman, and A. Patla, "Aspects of the Kinematic Simulation of Human Movement," Computer Graphics and Applications, IEEE, vol.2, no.9, pp.41-50, Nov [4] D. Gafurov, K. Helkala, and T. Søndrol, Biometric Gait Authentication using Accelerometer Sensor, Journal of Computers, vol. 1, no. 7, October/November [5] D. Gafurov, and E. Snekkenes, Gait Recognition Using Wearable Motion Recording Sensors, EURASIP Journal on Advances in Signal Processing - Special issue on recent advances in biometric systems: a signal processing perspective, vol. 2009, Article ID , January [6] K. Nakajima, Y. Mizukami, K. Tanaka, and T. Tamura, "Footprint-based personal recognition," IEEE Trans. Biomedical Engineering, vol. 47, no. 11, pp , Nov [7] L. Middleton, A. A. Buss, A. Bazin, and M. S. Nixon, "A floor sensor system for gait recognition," Fourth IEEE Workshop on Automatic Identification Advanced Technologies, 2005, pp , Oct [8] E. Ottaviano, M. Ceccarelli, and S. Grande, An Experimental Evaluation of Human Walking, 3rd International Congress Design and Modelling of Mechanical Systems CMSM [9] M. O. Derawi, P. Bours, and K. Holien, "Improved Cycle Detection for Accelerometer Based Gait Authentication," Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP), 2010, pp , Oct [10] M. O. Derawi, C. Nickel, P. Bours, and C. Busch, "Unobtrusive User-Authentication on Mobile Phones Using Biometric Gait Recognition," Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP), 2010, pp , Oct [11] L. Rong, D. Zhiguo, Z. Jianzhong, and L. Ming, "Identification of Individual Walking Patterns Using Gait Acceleration," The 1st International Conference on Bioinformatics and Biomedical Engineering, 2007, pp , 6-8 July [12] G. Pan, Y. Zhang, and Z. Wu, "Accelerometer-based gait recognition via voting by signature points," Electronics Letters, vol. 45, no. 22, pp , October [13] S. P. Moustakidis, J. B. Theocharis, and G. Giakas, "Subject Recognition Based on Ground Reaction Force Measurements of Gait Signals," IEEE Trans. Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 38, no. 6, pp , Dec [14] S. Mondal, A. Nandy, Chandrapal, P. Chakraborty and G. C. Nandi A Central Pattern Generator based Nonlinear Controller to Simulate Biped Locomotion with a Stable Human Gait Oscillation, International Journal of Robotics and Automation (IJRA), vol. 2, Issue 2, pp , [15] G. B. Huang, and H. A. Babri, "Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions," IEEE Trans. Neural Networks, vol. 9, no. 1, pp , Jan [16] T. K. Kim, and J. Kittler, Locally Linear Discriminant Analysis for Multimodally Distributed Classes for Face Recognition with a Single Model Image, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 3, pp , March [17] O. C. Hamsici, and A. M. Martinez, Bayes Optimality in Linear Discriminant Analysis, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 4, pp , April [18] A. M. Martinez, and A. C. Kak, PCA versus LDA, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp , February [19] A. Subasi, and M. I. Gursoy, EEG signal classification using PCA, ICA, LDA and support vector machines, Expert Systems with Applications, vol. 37, pp ,