Continuous Identification of Gait Phase for Robotics and Rehabilitation Using Microsensors

Transcription

1 Continuous Identification of Gait Phase for Robotics and Rehabilitation Using Microsensors R. Héliot, R. Pissard-Gibollet, B. Espiau, F. Favre-Reguillon INRIA Rhône Alpes, 655 av. de l europe, Saint Ismier Cedex. rodolphe.heliot@inrialpes.fr CEA-LETI, 17 rue des Martyrs, Grenoble. francois.favre-reguillon@cea.fr Abstract Using microsensors for the robust and accurate analysis of human posture or gait is an interesting opportunity for rehabilitation and robotics applications. This paper describes a feasibility study in which the possibility of using a new type of embedded microsensors, based on the coupling of accelerometers and magnetometers, and developed by CEA/LETI is investigated. This study consists in identifying what part of the gait cycle is active by using a reconstruction of the knee joint angle by two microsensors fixed on tibia and thigh, during a steadystate sagittal walk. More than just an identification of a few gait states, this approach allows us to continuously extract the current position on the gait cycle. We compare the reconstructed knee joint angle with a stored reference taking into account uncertainties on the velocity and perturbations of the terrestrial magnetic field. To accurately identify the phase of the gait movement, we fuse different simple and complementary methods: morphomathematics, cyclogram analysis, wavelet transform, qualitative analysis, crosscorrelation. These results encourage us to extend this work to explore the possibility of recognition of a larger set of human movements using more sensors and improved algorithms of signal processing. I. INTRODUCTION Restoring motion functionalities in the disabled is one of the most challenging issues in current scientific research. Several domains are concerned when addressing this question, which remains difficult to solve in its whole: biomechanics, neurosciences, micro and nanoelectronics, robotics. Recent progresses in all these fields allow one to envision realistic so-called bionic approaches allowing to significantly improve the autonomy of patients in several cases: restoring walking for paraplegics, improving the gait of hemiplegics, making easier the walk of knee-amputees or generating arm motions using active protheses... Besides, non medical applications emerge through the concept of human augmentation, with, for example orthoses or exoskeletons allowing to amplify the force exerted by a human or to increase his walking range with a load. All these classes of applications share a basic problem: how to detect and understand the intention of the human, in order to optimize the man-machine interaction at the strategic level as well as at the tactic one? Considering the first level implies that it is necessary to know what kind of motion the human wants to achieve: staying in place, standing up,walking straight, turning, grasping, and so on among a large set of possibilities. The second concerns the values of the parameters and of the variables which are involved in the current motion, for example in order to control a FES (Functionnal Electrical Stimulation [5]) system or a motorized knee prothesis. It should be noticed that learning stages on the subject himself in living situation are generally difficult to perform, except in the case of human augmentation, since the desired motion can only be achieved when the assistance system itself is active. Obviously, a critical point in this approach is the problem of sensors. Direct measurement of cortical activity would certainly be a good way of detecting the human s intention at the strategic level. Nevertheless, the current state of the research in this area doesn t allow to discriminate in a reliable way between the candidate classes of motion, which may belong to a database with several tens or hundreds elements. EMG (Electromyogram) measurements, even surface ones, are interesting possibilities, since for example they can be used to detect anticipated postural adjusments. However, we have to keep in mind that extrinsic information about the kinematics of motion are also needed for the control, and that they are not easy to reconstruct from EMG signals. Furthermore, practical issues of robustness, reliability, size, easyness of installation, finally make them very difficult to use online and alone in the recovering motion objective. This is why we decided to explore the possibility of using a new type of embedded microsensors, based on the coupling of accelerometers and magnetometers, and developed by CEA/LETI. Each integrated sensor provides us with absolute 3D orientation of the part to which it is attached. The idea is then to install several sensors of that kind on the subject and to analyze their joint outputs in order to both determine what is the type of the current or starting motion and estimate the state variables which are required by the real-time controller. Our ultimate goals are to control a motorized prosthesis and to improve the control of a FES system for paraplegics. Before addressing this question in its generality, a first feasibility study was achieved, which is reported in this paper. We had indeed to verify that the new sensors were wellsuited to our application. We chose as a test bed the following problem: in a steady-state sagittal walk, can we identify what part of the gait cycle is active, and, more, know as accurately as possible what is the current position (called phase in the following, see fig. 1) on the gait cycle? To answer this question, the issues of the study are the following: a reference normalized gait cycle is stored; the considered variable is the knee angle; no assumption is made on the velocity of the current walk. However velocity changes have to be slow;

2 Fig. 2. Hardware Setup Fig. 1. At a given time in experiment, we look for the index of the reference sample that matches the current measurement. We call phase this index; it is continuous over time, excepted at the end of gait cycle, where it returns from 1000 to 0. the studied subject can be different from the one used in the reference recording. It is only assumed that his walking patterns are normal in some sense; the experimental facility includes two microsensors, the outputs of them can be used independently or for knee angle reconstruction, which are mounted on a rigid knee orthosis to which is added a position sensor. It should be noticed that using microsensors for human posture or gait analysis is a field that has already been explored. A first group of reported applications concerns posture analysis, the goal being to provide with an information such as The person is walking or The person is lying, for medical monitoring of patients or elderly (see for example [3], [7], [8], [9], [13]). Other works are dedicated to the characterization of the different phases of the gait (stance and swing for example). As already evoked, this problem is more difficult and various solutions have been proposed: Willemsen et al. [14] analyze a signal from an accelerometer, in order to detect events in the gait cycle using cross-correlation; Williamson et al. [15] classify accelerometers patterns with Rough Sets; Sekine et al. [11] use a wavelet decomposition of signals from a triaxial accelerometer to distinguish walking on level ground from walking on a stairway; Pappas et al. [10] combine informations from a gyroscope and three force sensitive resistors placed in a shoe sole in order to point out a state in the gait cycle; Auvinet et al. [1] monitor the autocorrelation coefficient to extract informations such as frequency and symmetry of the Fig. 3. Hardware on the Subject steps. Finally, compared to all these approaches, our work has the specificity to use sensor outputs continuously in the whole control system. The paper is organized as follows: after a description of the system (section II), we focus on the main difficulties of the problem, (section III). Then we present the method we adopted (section IV), followed by some results (section V). For an easier understanding, all mathematical considerations are not described in this paper; the reader is referred to [6] for more details. II. HARDWARE A. Microsensors and Processing The microsensors and their associated processing constitute the MOCAP system developed by the CEA (see [2], [9] for description of the system and application examples).

3 Since it was not necessary to modify this system for our application we do not describe it in detail. Embedded microsensors are gathered in small boxes called attitude sensors, each one containing three accelerometers and three magnetometers, organized as a trihedron. Accelerometers measure both the gravitational acceleration and the acceleration due to movement. Magnetometers measure the earth magnetic field, which has the interest of making artificial magnetic sources (which are expensive and limit the workspace) useless. In targeted applications, the earth magnetic field is generally locally uniform enough to be used directly (this assumption will be discussed in section III. B). Finally, each attitude sensor is able to compute an orientation of its box in the space, which is parametrized by Euler angles. B. Data Acquisition System During the experiments, we used the following items: an incremental shaft encoder (HENGSTLER 58-D) mounted on a Lennox Hill knee orthosis (which does not impair the movement). The goal of this encoder is to provide a reference measurement of the knee angle, in order to validate the angle computation performed by the attitude sensors; two attitude sensors, fixed on tibia and thigh; a link belt, that gather signals from all sensors, and then send them to the acquisition station; an acquisition and processing workstation. All signals are sampled at 100 Hz. There is no particular filtering system, but the analog anti-aliasing filter on the sensors output, before sampling. III. ENCOUNTERED DIFFICULTIES A. Uncertainties on the Velocity Recall that we have to recognize a movement from the informations provided by attitude sensors. More than the rough signals of accelerometers and magnetometers, we recover the orientation (Euler angles) of the sensors. So we are able to reconstruct a joint angle, simply by computing the difference of the two angles in the sagittal palne of two sensors. To recognize a movement, we have to compare it with a reference model. This model has to be a joint angle history which significantly describes the gait movement. We preferred the knee angle to hip or ankle ones, because of its low variability with respect to the speed of the gait and of the easyness of mounting of the sensors and of the shaft encoder. To compare the actual measurement with the stored reference, a natural idea would be to use the crosscorrelation function in order to determine the phase of the movement. However, since the movement can be achieved at different speeds, this method can t be used ; for illustration, let us consider that the reference contains about 100 samples (i.e. a one sec. movement sampled at 100 Hz). Assuming that the movements we observe may be achieved in a range from 0.5 to 1.5 second by cycle, they contain 50 to 150 samples; thus for a given observation period, the movement can appear either contracted or expanded (over time) compared with the reference. We have therefore to over- or under-sample the reference to make the correlation computation possible (because both measurement and reference arrays must have the same size). So, using correlation function would require to explore a large research space: several dimensions for the phase (crosscorrelation) and for the speed (contracted or dilated movement). Since correlation computation takes time, we therefore cannot use this method directly (see section IV-E). A consequence is that we have to prefer approaches which are invariant to (or that would not be too much affected by, or that would be able to adapt to) changes of the speed. For that purpose, we first implemented an adaptative filter estimating the velocity and tracking the movement phase. Despite some success, it nevertheless appeared not robust enough and we discarded this solution. B. Perturbations of Magnetic Field Magnetometers are sensitive to earth magnetic field, which was assumed to be locally almost uniform. But actually, electromagnetic perturbations lead to local variations of the magnetic field, and introduce a drift in the angle computed from the attitude sensors (which means that local variations of the angle remain unaffected, while the global trend can drastically change). Combining accelerometers and magnetometers outputs is a way to compensate for this drift. However, we have to keep in mind that the measurement made by the accelerometers is itself disturbed by the accelerations due to the movement. The problem can therefore be solved either by adding other sensors (like gyroscopes, for example; see [10]) or by using advanced signal processing methods, that would recognize the pattern of knee angle and remove the drift in an algorithmic way. We have preferred the last class of solutions. IV. SOLVING METHODS In order to overcome the two major difficulties emphasized above, we explored several methods of signal processing, some of them are described later. In fact, it appeared that their respective drawbacks and advantages were often complementary. This is why, finally, a fusion approach was required for extracting a reliable value of the phase. We give further information in the following, nevertheless without going deeply into details. A. Preprocessing with Top Hat Transform The Top Hat transform, which comes from morphomathematics [12] aims here at rejecting the drift which results from the disturbances in the terrestrial magnetic field. With respect to other methods, like high pass filtering, local variations of signal are not affected by the transform: the angle pattern is kept intact. Once this preprocessing is done, it is now possible to launch the different analysis methods. B. Cyclogram analysis The main interest of cyclograms is that this representation is globally independent of time, which allow to overcome the problems caused by the differences of velocities between the

4 Fig. 4. Top Hat Transform. Knee angle computation before (top) and after (bottom) Top Hat preprocessing. Angle pattern is not affected by transform. reference model and the current gait. For example, let us look at q 1 = f(q 2 ) instead of: { q1 (t) q 2 (t) Fig. 5. Cyclogram Analysis. The measurement points lying within the mask (dotted area) are assumed to be a part of movement. The gait cycle can be achieved at different speeds (movement 1 and movement 2 on the figure) without impairing the analysis. We represent in a 2D- space the tibia angle (X-axis) with respect to the thigh angle (Y-axis). Assuming a rather good invariance of the whole pattern through the considered range of motion velocities, the measurement points (tibia; thigh) follow the same path along gait cycle whatever the speed. Therefore, by defining an area, called mask around the reference path, we just have to count how many measurement points lie inside the mask to decide whether the observed movement is the gait - or not. If yes, we can look for the closest reference point from the latest measurement point: we then obtain the current value of the phase corresponding to the measurement (the same as the closest reference point). This method almost doesn t require computation, since we can prepare offline a look-up table which associates with every point (x;y) of the plan the associated phase φ. C. Wavelet Transform Wavelet transform provides us with a time/scale representation. In this space, we can search for frequential patterns, in a given time interval (see [11]). For example, we can look for a particular frequency and wait for specific activation patterns. Such an analysis can be used for example to detect heel strike on the ground without hit sensor: at landing time, heel strike on the ground leads to a significant increase of high frequencies (frequencies are high compared with normal frequencies observed in the gait movement). Thus we easily obtain a triggering information, i.e. the time when the heel strikes the ground, which can be stated as the beginning (0%) of the gait cycle. D. Qualitative Analysis In some cases, it can be interesting to conduct an analysis which is more qualitative than quantitative on the observed signals. It is often the case when working on biological signals, the numerical values of which are highly variable from an experiment to another, while their global shape remains unchanged. Furthermore, for example in bioreactors, the structure of dynamical models is only approximately known, while the parameters are not, and the measurements are sparsely achieved. Generally, qualitative analysis systems rely on the observation of the derivatives (see [4]). The information provided by such methods is not very accurate: they indicate what is the state of the observed system, without being able to give the exact situation of the system inside this state. But they easily provide with a robust information, that can then be completed by other methods. We implemented a simple qualitative analysis, combining a derivative observation with a state transition diagram. Gait cycle is divided in four states, two with a positive derivative, and two a with negative one; a stop state is also added. An advantage of this method is that it can detect the starting and stopping phases of the movement, during which we do not have enough stable data to derive an accurate variable estimation. Besides, it gives the possibility to easily compute the period of a gait cycle (the speed of the movement): it suffices to count how many samples occur between two identical transitions (from state 2 to state 3, for example, in our 4-states diagram). Obviously, the case considered in this paper is quite simple. Nevertheless, it is intended to use, in the future, several sensors distributed on the subject s body. In such a situation, because of the high dimension of the system, it is expected that the size of the related finite state machine will considerably increase and that a qualitative analysis will prove to be very efficient for identifying the type of the current motion. E. Crosscorrelation Finally, the gait speed information, that was previously not avaible, can now be used in the crosscorrelation computation.

5 Fig. 7. Fusion principle. The results of four complementary methods are fused to obtain a reliable phase. Fig. 6. Qualitative analysis. The knee angle history into the gait cycle is divided in four phases (1-4); a stop state is also present. Since, as previously mentionned, this processing is still very slow, it is started only every ten samples (i.e. at 10 Hz frequency). F. Fusion All methods presented above provide complementary informations: when qualitative analysis gives at every moment a state of the movement (robust but not accurate), wavelet analysis provides with a flag only one time per gait cycle, but in a very accurate way. As well, when crosscorrelation computation is very reliable but cannot be launched at each sample due to its computational cost, cyclogram analysis gives an estimation that is far more affected by noise, but which requires only very few operations. Since the different methods are complementary, it appears natural to fuse their results in order to obtain a fast and robust solution. The principle is to observe the results of each method, from the most reliable to the least; if an accurate information is useful and consistent, a phase is accepted and the fusion stops. More precisely, the fusion stage works as follows: 1) the state of the transition diagram is observed; if there is no movement ( stop state), fusion ends here 2) then, if the trigger signal issued from the wavelet analysis appears at this moment, the corresponding phase is assigned, and fusion ends 3) else, if at this moment a crosscorrelation estimation is available (which means that we launched the computation for this sample), its result is adopted 4) else, if the cyclogram estimation is consistent (the gap with the latest accepted phase is within a given bound), we accept it as a new phase value 5) else, we choose a linear progression, at the estimated speed of gait movement: φ(t) = φ(t 1) + V where V is proportional to the speed V. RESULTS We implemented this algorithm with Scilab 1 ; it took about one minute to process one minute of 100 Hz data. Of course, executing a binary code or using an implementation on a dedicated real-time system (e.g. DSP) would dramatically decrease this computation time, allowing then to use the method on-line. Among all obtained results, we select from [6] the few ones presented in figure 8: the final phase estimation is correct, rapidly achieved, and reliable. The overall approach is therefore proved to be useful for further studies in motion control (active prosthesis, FES). VI. CONCLUSION Our overall goal is to characterize the relationships between the measurements issued from embedded microsensors and the movements of a person. Precisely,in this study, we tried to accurately identify the phase in a gait movement. We implemented simple and complementary methods (morphomathematics, wavelet analysis, cyclogram analysis, qualitative analysis, crosscorrelation) and then fused their results. We finally extracted an accurate value of the phase, while preserving the rapidity and the robustness of the processing. 1 The Scilab Consortium;

6 This work can be extended along different ways. First, for the applications we aim at, we ll need an exhaustive recognition of all possible gait movements (climbing stairways for example) and not only of walking on level ground. This means that we have to use many more sensors, distributed on the body, and, possibly, to still improve the algorithms of signal processing. Beyond this, other types of systems could benefit from this approach for movement analysis, like in robotics. In some cases, such as deformable structures, or when the knowledge of an absolute orientation is needed (e.g. non horizontal ground), they still can operate when classical joint sensors are unefficient. Medical and Virtual Reality applications could also benefit from this system, that does not require heavy equipment, with regard to classical motion capture systems for. Finally, because the sensors are highly integrated and can be mounted on links rather than on joints, they may constitute an alternative or a complement to joint angle sensors in feedback control loops. REFERENCES [1] Auvinet, B., Chaleil, D., and Barrey, E., Accelerometric Gait Analysis for Use in Hospital Outpatients Revue du Rhumatisme [Ed. Fr.], pp , July-September 1999 [2] Bonnet, S., Couturier, P., Favre-Reguillon, F., Guillemaud, R. Evaluation Of Postural Stability by Means of a Single Inertial Sensor, EMBS, 2004 [3] Busser, H.J., Ott, J., van Lummel, R.C., Uiterwaal, M., and Blank, R. Ambulatory monitoring of children s activity Med. Eng. Phys., vol. 19, no. 5, pp , 1997 [4] Bernard, O., Gouzé, J.L.: Non-linear qualitative signal processing for biological systems: application to the algal growth in bioreactors Mathematical Biosciences, no 157, p , 1999 [5] Guiraud, D., Poignet, P., Wieber, P.B., El Makksoud, H., Pierrot, F., Brogliato, B., Fraisse, P., Dombre, P., Divoux, J.L., Rabischong, P. Modelling of the human paralysed lower limb under FES ICRA 2003 [6] Héliot, R.: Study of an Instrumented Orthosis, Master Thesis, [7] Morlock, M., Schneider, E., Bluhm, A., Vollmer, M., Bergmann, G., Muller, V. and Honl, M. Duration and frequency of every day activities in total hip patients, Journal of Biomechanics, vol. 34, pp , 2001 [8] Najafi, B; Aminian, K; Paraschiv-Ionescu, A; Loew, E; Bula, C J; Robert, P, Ambulatory System for Human Motion Analysis Using a Kinematic Sensor: Monitoring of Daily Physical Activity in the Elderly IEEE Trans. Biomed. Eng., vol. 50, no. 6, pp , June 2003 [9] Noury, N. et al. A MEMS based MicroSystem for the Monitoring of the Activity of Frail Elderly in their Daily Life: The ACTIDOM Project HealthCom2004, Odawara-Japan, June [10] Pappas, I.P.I.; Keller, T.; Mangold, S.: A reliable, gyroscope based gait phase detection sensor embedded in a shoe insole Proceedings of IEEE Sensors 002. First IEEE International Conference on Sensors, p vol.2, 2002 [11] Sekine, M., Tamura, T., Togawa, T., Fukui, Y.,: Classification of waistacceleration signals in a continuous walking record, Med. Eng. Phys., vol 22, no 4, p , 2000 [12] Serra J. Course on Mathematical Morphology - First Part: Morphological Operators. November 1999 [13] Veltink, P.H., Bussmann, H.B.J., de Vries, W., Martens, W.L.J. and Van Lummel, R.C.: Detection of static and dynamic activities using uniaxial accelerometers, IEEE Trans. Rehab. Eng., vol. 4, no. 4, pp , December 1996 [14] Willemsen, A.T.M.; Bloemhof, F.; Boom, H.B.K.: Automatic stanceswing phase detection from accelerometer data for peroneal nerve stimulation IEEE Transactions on Biomedical Engineering 37, no. 12, p , 1990 [15] Williamson, R.P.; Andrews, B.J.; Au, R.: Control of neural prostheses. II. Event detection using machine learning Proceedings of the RESNA 96 Annual Conference Exploring New Horizons... Pioneering the 21st Century, p. (291-3), 1996 Fig. 8. Results. From top to bottom: a) Rough accelerometer signal (V); b) Computed knee angle before Top Hat preprocessing (in degrees); c) States of the qualitative analysis: 0 value stands for stop state, 1-4 for the corresponding states; d) Estimated phase after fusion (from 0 to 1000 permil); e) Model knee angle corresponding to the estimated phase (degrees).