Geometric moments for gait description C. Toxqui-Quitl, V. Morales-Batalla, A. Padilla-Vivanco, and C. Camacho-Bello. Universidad Politécnica de Tulancingo, Ingenierías 100, 43629 Hidalgo, México. ctoxqui@upt.edu.mx ABSTRACT The optical flow associated with a set of digital images of a moving individual is analyzed in order to extract a gait signature. For this, invariant Hu moments are obtained for image description. A Hu Moment History (HMH) is obtained from K frames to describe the gait signature of individuals in a video. The gait descriptors are subsequences of the HMH of variable width. Each subsequence is generated by means genetic algorithms and used for classification in a neuronal network. The database for algorithm evaluation is MoBo, and the gait classification results are above 90% for the cases of slow and fast walking and 100% for the cases of walking with a ball and inclined walking. Also an optical processor is implemented in order to obtain the descriptors of the human gait. 1. INTRODUCTION Gait recognition is the process of identifying a person by the manner in which a person walks. It means analyzes the pattern of shape and motion in video of a walking person. The gait of a person is determined by their underlying musculo-skeletal structure. 1 For this reason, it can be used for clinical identification of pathological conditions 2 and for the tracking of progress of patients under rehabilitation program 3. 4 In general the gait of a individual is affected by factors such as the choice of footwear, clothing, affliction of the legs, walking surface, etc. Typically, a model-based approach for human gait is required to estimate certain parameters such as gait frequency, phase, and center of mass coordinates 5. 6 However, this procedure requires a large number of operations for a single image. Although this might be a more accurate model is also computationally expensive. Other possibilities can use holistic measures as Zernike moments. 7 Hu described how the geometric moments of an image could be combined to produce the invariant moments that are theoretical independent of shift, rotation and scale. In the last years, various optical methods and hybrid processor have been presented to compute the moments, such as the processor to compute the moments in parallel by spatial convolution with a single fixed mask or by Fourier transformation. In section 2, a method for the description of human gait based on geometric moments and intensity moments is presented. Section 3 describes how the geometric moments are used to generate a moment history for each person and the genetic algorithms are used to generate the sequences of the subject. In the section 4 we present the results of gait classification using neural networks. Finally, the conclusions are presented in section 5. 2.1 Geometric moments The geometric moments are defined as, 8 2. A BRIEF REVIEW m p,q = M 1 x=0 N 1 y=0 x p y q I x,y, (1) where p and q are the (p + q) order and M N is the image size. The intensity centroid (x 0, y 0 ) of the image function I x,y is obtained as x 0 = m 1,0 m 0,0 and y 0 = m 0,1 m 0,0. From this, the central moments are defined as, µ p,q = M 1 x=0 N 1 (x x 0 ) p (y y 0 ) q I x,y. (2) y=0
This transformation makes the moment computation independent of position of the image reference. Normalized moments are computed as η p,q = µ p,q where γ = p+q µ γ 0,0 2 + 1. A set of seven invariant moments to rotations, translations and scale changes of an object was derived from the second and third moments by Hu. 8 Some of them are ϕ 1 = η 2,0 + η 0,2, ϕ 2 = [η 2,0 η 0,2 ] 2 + 4[η 1,1 ] 2 and so on. The Figure 1 shows the invariance description of Hu moments. Figure 1. (a) k Frames of two subjects under different scale, orientation and position. (b) Invariance descriptors based on the ϕ 5(k) Hu moment of (a). 2.2 Intensity moments Let a planar input transparency with amplitude transmittance I(x, y) be placed in front of a converging lens of focal length f. The input is illuminated by a monocromatic plane wave of amplitude A. The amplitude distribution in the back focal plane of the lens is, F (u, v) = A ifλ I(x, y)e i2π fλ (ux+vy) dxdy. (3) Measurement of the intensity distribution F (u, v) 2 in the focal plane yields knowledge of the power spectrum of the input. An example of two dimensional Fourier analysis is shown in the Figure 2. From this, intensity moments can obtained as, 9 m p,q = [( ) p ( ) q 1 ( 2iπ) p+q F (u, v)]. (4) u v u=v=0 If p = q = 0 then m 0,0 = F (0, 0) and represents the total image power associated with the irradiance distribution I(x, y). A CCD camera in the Fourier plane (u, v) measures the average of F (u, v) over an element s area. This detector array is equally spaced by h along orthogonal u, v grid lines, so the intensity moments are measured as,
Figure 2. (a) Optically obtained Fourier transform of (b) m p,q = F (u+h,v) F (u h,v) 1 ( 2iπ) p+q ( u) p ( v ) q F (u, v) u=v=0, (5) F (u,v+h) F (u,v h) where u F (u, v) = 2h and v F (u, v) = 2h. In the Figure 3 is shown a scheme of the optical system used and the distribution of the intensity moments up through third order. Figure 3. Irradiance distribution in the Fourier plane. 3.1 Data Base MoBo 3. HUMAN GAIT Carnegie Mellon University in Pittsburgh, Pennsylvania generates a database 10 of people walking on a threadmill at 6 different angles. Each camera capture 340 images per person, and the database has 25 people. The people
are walking at 4 different cases: walk slow, fast, inclined and with ball in hands. Figure 4 shows a schema of the acquisition system. Figure 4. Position of the cameras 3.2 Moment histories Gait is a coordinated and cyclic motion that engages the entired body especially the legs. The three main stages by which the gait is composed are shown in the Figure 5. The double support (DS) phase where weight is transferred to the leading foot, the mid-stance position (MS) where weight is only on one foot and the heel strike (HS) where the foot touches the ground. 11 The motion among consecutive heel strikes of opposite feet is a Step. The motion period is the time taken by a step and the gait frequency are the numbers of steps taken per second. Figure 5. (a) Phases of gait (b) Moment history related with one step in cyclic motion. The different phases of gait can be obtained by means Hu moments ϕ n or intensity moments m p,q. The Hu moment histories for two subjects are computed from the k-images in a sequence v for each subject s and shown in the Figure 6. Parameters such as velocities, and period of walking, can be determined through this method. As we can see, the moment histories profiles are very similar to the same subject but different between subjects. The starting
Figure 6. Hu moment history of (a) subject 1 and (b) subject 2. point z v and width v of gait sequences ϕ k,v,s n are different. The width v of the moment histories ϕ k,v,s n is related with a step of the subject, Figure 7, and this is determined with Genetic Algorithms (GA). 12 Figure 7. Three sequences in the moment history of variable width v Finally, each sequence v = 1, 2, 3 of variable width v of a each subject is used as vector descriptor in a neuronal network for gait classification.
4. GAIT RECOGNITION RESULTS 4.1 Gait description using intensity moments The optical system that has been implemented for image processing is shown in Figure 8, where SF is the spatial filter, S is the stop, C is a collimating lens, L is the lens that forms the Fourier transform, P is a polarizer, LCD is the liquid cristal display and CCD is the intensity detector. Figure 8. Optical image processor The intensity moment history is shown in Figure 9. As we can see, the profile of the intensity moment history shows the cyclic pattern of the gait. Figure 9. Optical calculation of intensity moments for gait description
4.2 Gait classification using Hu moments Figure 10 shows the Hu moment histories ϕ k,v,s 5 obtained from 6 different cameras for the case of walk fast. The curves on cameras 5 and 6 have a high periodicity. Figure 10. Moment histories based on Hu descriptors for the case of walk fast Figure 11 shows a comparison between different cameras. As cameras 1 and 4 has a opposite position, only one of them will be used. Also, the cameras 3 and 5 are highly correlated. In some cases, the same situation occurs for the cameras 2 and 6. Figure 11. Comparison of different moment histories for the case of walk with a ball.
A set of s = 15 images sequences of walking subjects were processed in order to extract a gait signature. Only the lower body silhouettes from MoBo database were used. From each HMH based on ϕ k,v,s 5, sequences v = 1, 2, 3 for each subject were obtained. The sequences have variable width k = v because the velocity of the gait and were used as gait descriptors in a neuronal network. In general, the classification results shows a good performance for the four cases of walking with only one moment and processing the cameras 1, 2 and 3. Slow Walking Fast Walking Ball Walking Incline Walking Camera 1 95.5% 91.1% 97.7% 91.1% Camera 2 86.6% 93.3% 97.7% 91.1% Camera 3 97.7% 91.1% 100% 100% 5. CONCLUSIONS A method of moments for gait description from sequences of lower body images was presented. Moment histories was obtained from Hu moments and intensity moments using and optical processor. Gait descriptors were generated from subsequences of moment histories. Each subsequence is generated by means genetic algorithms and used as descriptors for classification in a neuronal network. The database used for algorithm evaluation was MOBO database. From this, gait classification results are above 90% for the cases of slow and fast walking and 100% for the cases of walking with a ball and inclined walking. REFERENCES 1. A. Jain, P. Flynn, and A. Ross, Handbook of biometrics, Springer, (2010). 2. H. Lakany, Extracting a diagnostic gait signature, Pattern Recognition 41, 2008. 3. N. Mezghani, S. Husse, K. Boivin, K. Turcot et al., Automatic classification of asymptomatic and osteoarthritis knee gait patterns using kinematic data features and the nearest neighbor classifier, IEEE transactions on biomedical engineering 55 (3), 2008. 4. Y. Wu, and S. Krishnan, Statistical analysis of gait rhythm in patients with parkinsons disease, IEEE transactions on neural systems and rehabilitation engineering 18 (2), 2010. 5. Tafazzoli, F., Safabakhsh, R., Model-based human gait recognition using leg and arm movements, Engineering Applications of Artificial Intelligence 23(8), (2010). 6. M.Goffredo, Imed Bouchrika, J. N. Carter and M. Nixon, Self-Calibrating view-invariant gait biometrics, IEEE Trans. on Systems, Man and Cybernetics 40, (2010) 997-1008. 7. Shutler J.D and Nixon Mark S., Zernike Velocity Moments for sequence-based description of moving shapes, Image and Vision Computing 24 (2006) 343-356. 8. M.K. Hu, Visual pattern recognition by moment invariants, IRE. Trans. Inform. Theory 8, (1962) 179-187. 9. M. R. Teague, Optical calculation of irradiance moments, Applied Optics 19 (8), (1980) 1353-1356. 10. R.Gross and J. Shi, The CMU Motion of Body (MoBo) Database Tech. Report CMU-RI-TR-01-18 Robotics Institute, Carnegie Mellon University, 2001. 11. Grant, M. G., M. S. Nixon and P. H. Lewis, Extracting moving shapes by evidence gathering. Pattern Recognition 35(5), (2002) 1099-1114. 12. Z. Michalewicz., D. B. Fogel., How to solve It: Modern Heuristics, Second ed., Springer, 2004.