Optimal Gait Primitives for Dynamic Bipedal Locomotion

2012 IEEE/RSJ International Conference on Intelligent Robots and Systems October 7-12, 2012. Vilamoura, Algarve, Portugal Optimal Gait Primitives for Dynamic Bipedal Locomotion Bokman Lim, Jusuk Lee, Joohyung Kim, Minhyung Lee, Hoseong Kwak, Sunggu Kwon, Heekuk Lee, Woong Kwon, and Kyungshik Roh Abstract This paper presents a framework to generate dynamic walking for biped robots. A set of self-stable gait primitives is first constructed. It is done by 1) representing parametric gait primitives, 2) utilizing state-dependent torque control, and 3) doing numerical optimization that takes into account the complex multi-body dynamics with frictional contact forces. Dynamic walking to follow the arbitrary path including a curve is then generated online via sequentially composing primitive motions. Results show that dynamic gaits are humanlike and efficient compared to the conventional knee bent walkers. Our proposed method is applied to a torque-controlled, human-sized biped robot platform, Roboray which is cabledriven partially for joint compliance. Following a discussion on robot design and control, experimental results are also reported. I. INTRODUCTION Current walking robots are highly energy inefficient except recent passive walkers. Humanoid robots usually consume 10 times more energy than human while walking [1]. Generating energy efficient walking based on the dynamics models has received attention in robotics [2], [3]. We first briefly review two main approaches in research of biped locomotion: the zero moment point (ZMP) based walking and the dynamic (limit cycle) walking. The ZMP based walkers have the following common features [4]: 1) the stiff joints are position controlled with high gains, 2) ZMP criterion is used to guarantee biped walkers to not fall over. On the other hand, the dynamic walking (also known as energy efficient gait) has the following features 1) the compliant joints are force (torque) controlled, 2) stable walking is achieved by finding attractive stable orbits called limit cycles. In dynamic walking, we can also observe the periodic controlled falling motions like inverted pendulum [5] (notice that the force controlled compliant joints allow the inertial motion of the walker to be exploited rather than opposed). The ZMP based methods have disadvantages in energy efficiency. However, many ZMP based walkers [6], [7] have shown excellent performance in motion planning studies involving interaction with environment e.g., collision avoidance in 3-D space. On the contrary, most dynamic walking studies have been limited to straight walking [8]. Gregg et al. [5] showed curve walking but they used a simple model (4DOF and point feet). This is due to the difficulty in finding stable dynamic gaits for 3-D biped robots. In other Bokman Lim, Jusuk Lee, Joohyung Kim, Minhyung Lee, Hoseong Kwak, Sunggu Kwon, Heekuk Lee, Woong Kwon and Kyungshik Roh are with the Samsung Advanced Institute of Technology, Nongseo-dong, Giheung-gu, Yongin-si Gyeonggi-do, 446-712 Korea bokman.lim@samsung.com Fig. 1. Dynamic bipedal walking robot system (Roboray) words, the complex dynamics must be used to produce stable limit cycles. For example, curve walking has an asymmetric property compared to the straight walking which makes the problem much more difficult. Our main contribution is a development of a systematic framework for generating dynamic locomotion for humanoid robots. Various dynamic gait primitives as parametric forms (straight walking, in-place walking, backward walking, and curve walking) are first constructed off-line using optimization based method. Online motion is then generated by sequentially composing primitive motions. First of all, we choose dynamic walking scheme and generate also minimum torque motion for energy efficiency. To find stable limit cycles during walking, we minimize walking periodical errors using landing forces (i.e., minimizing the difference of stepto-step landing forces). A desired walking style is achieved by a constant step length, step velocity, and turning angle. For the optimal walking, a development of a reliable and practical dynamics-based movement optimization algorithm should be emphasized. For these problems, dynamics should be considered during optimization procedures. The complexities of the dynamic equations, however, often lead to intractable constrained nonlinear optimization problems, especially for high degree-of-freedom systems such as a humanoid type model. Previously many dynamics-based algorithms need super-computing power [9], [10], or are not applicable to a real system because of hardware limitations [11]. Based on our dynamics-based optimization algorithms for different types of motion [12], [13], [14], we developed a robust and optimal walking controller. This paper is organized as follows. In Section II we explain the dynamic walking generation framework, and describe movement optimization algorithms and balance control meth- 978-1-4673-1735-1/12/S31.00 2012 IEEE 4013

ods. Section III provides case studies of various walking motions using a humanoid robot, Roboray, and also includes simulation studies and hardware experiments. II. FRAMEWORK AND ALGORITHM Our general framework for dynamic biped walking generation is shown in Fig. 2. Given a high-level description for a desired gait, the task parser interprets each task, simultaneously sensing environment information (to avoid obstacles) and generates a corresponding sequence of desired walking motions in parametric forms. The motion generation then takes the sequence as input, draws upon the database of walking primitives, and plans an appropriate gait sequence. Walking styles are represented by step length, step velocity, and turning angle. By sequentially composing primitive walking motions we determine how to follow the given target path. Gait motion sequence is replanned periodically to avoid moving obstacles. Our framework is composed of three main modules: a finite state machine for control torque generation, a gait primitive database for various styled walking, and a real-time motion planner for interacting with environment. Each part is designed by systematic procedures as shown in Fig. 6. Fig. 3. Robot joint coordination in lateral (left) and frontal plane (right). Left leg is stance state and right leg is swing state. Stance and swing leg are alternatively changed. Fig. 4. Robot joint coordination in axial plane (Rectangles imply robot footprints). Hip yawing strategy for turning for curve walking. p 2 in Table 1 means a half turning angle. properties, we plan the next step by switching swing and stance leg motions. This strategy is applied to all pitch, roll, and yaw joints (motions in lateral, frontal, and axial plane). Therefore control parameters are k p, k d, and via points for target trajectory. Fig. 2. Walking motion generation framework A. Generation of State-dependent Control Torque The control torque is generated by simple spring-damper couples as in (1) and the desired target trajectory q d is planned by sensing current robot state i.e., left stance or right stance. Fig. 3 shows the robot joint coordination in lateral and frontal planes for the left stance state. τ = k p (q d q) k d q (1) Fig. 4 shows the robot joint coordination in axial plane and the hip yawing strategy for curve walking. If the robot s state is just changed from right stance to left stance, the q d for the current state is planned by interpolating via points using quintic spline with zero velocity and acceleration boundary conditions. The via points represent the predefined finite robot states (cyclic gait postures). For example we can obtain a hip pitch joint target trajectory as shown in Fig. 5. By using periodic and symmetric Fig. 5. Example of target trajectory for right hip joint. Table I shows all the parameters of the target trajectory for the leg and torso motions. Initial posture q 0 is determined by the last state joint information of the previous step including posture, velocity, and acceleration (i th step q 0 is equal to i-1 th step q f ). By adjusting q m and q f we determine current target joint trajectory. State changes from left to right stance occur by sensing normal contact forces. The contact forces are measured by FT sensor attached on the foot. We use 200N as the threshold value for the state change condition. 4014

TABLE I TARGET TRAJECTORY PARAMETERIZATION WITH VIA POINTS Description of joint motion q m q f torso pitch inclination p 1 p 1 swing hip yaw for turning p 2 p 2 swing hip roll p 3 0 swing hip pitch p 4 2deg p 4 swing knee pitch p 5 0 swing ankle pitch p 6 p 7 + p 4 swing ankle roll p 3 0 stance hip yaw for turning p 2 p 2 stance hip roll p 3 0 stance hip pitch N/A p 4 stance knee pitch 0 0 stance ankle pitch p 7 p 7 p 4 stance ankle roll p 3 0 Target arm swing trajectories are also parameterized with a similar method (using 1 or 2 via points). To reduce the parametric space of control, we minimize control parameters P = {p i }. As shown in Fig. 6, this iterative procedure includes experimental sensitivity analysis of the parameters. The six parameters below are sufficient to represent various straight walking with stable limit cycle for the given system. p r1 = p 3 = rolling angle of hip roll joint p r2 = p 1 = torso inclination angle p r3 = p 4 = sweeping angle of swing hip pitch (2) p r4 = p 5 = maximum bending angle of swing knee p r5 = p 7 = bending angle of stance ankle pitch p r6 = t f = step time By adding three additional parameters p r7 = p 3,asym, p r8 = p 4,asym, and p r9 = p 7,asym for asymmetric left/right hip rolling/sweeping and ankle bending motion, curve walking can be produced. B. Dynamics-based Motion Optimization Algorithm We first reduce the optimization search space by selecting optimal key coordinates as optimization variables. Upper body motions are not independent during optimization procedures. Both arm motions are generated synchronously by lower body motion. Optimization variables are equal to the reduced control parameters in (2) involving transition parameters from the initial posture to the steady-state walking. In dynamic walking, initial motion is very important to achieve stable limit cycle quickly. To determine the initial half step motion, we add transitional control parameters p r0,roll, p r0,sweep for initial hip rolling and pitching motion. The overall flow chart for the motion optimization algorithm is shown in Fig. 8. 1) Dynamic Modeling: To plan a dynamically welldefined gait, or a gait that is energy efficient and stable in motion, we should perform accurate dynamic modeling and analysis. Our biped robot is a floating-base system where Fig. 7. Hardware prototype, cad model and kinematic diagram of Roboray. Six virtual passive joints (three prismatic and three revolute) connect between the robot base and the fixed world frame. the base link is not fixed, and includes dynamically different contact phases depending on the number of supporting legs as shown in Fig. 7. Let {q, q base } be a set of coordinates describing the kinematic configuration of the robot, where q denotes the vector of actuated joints and q base denotes both the six virtual passive joints used to parameterize the position and orientation of the moving base link of the robot. Denote τ by the torque (or force) vectors associated with q. The floating base has no constraints in moving (applied force should be set zero). Accounting for contact forces, the equations of motion then assume the form M(q) ( qbase q ) + b(q, q) + J T c F c = ( 0 τ where M is the mass matrix, b represents Coriolis, centrifugal, and gravity terms, F c is the wrench corresponding to the contact force, and J c is the constraint Jacobians associated with the contact wrench. In the forward dynamics problem with floating base and frictional contact, the objective is to determine the output values { q base, q} with a prescribed command input torque of the system in the form of inputs {q, q, τ}. For this purpose, we use the Open Dynamics Engine (for general discussions of ODE algorithm and contact dynamics see [15]). 2) Objective Function and Constraints: Given the fixed gains k p, k d and the above governing equation of motion, our goal is to optimize the objective functions of the form min P w 1J 1 + w 2 J 2 + w 3 J 3 + w 4 J 4 + w 5 J 5 J 1 = t f t=0 τ(t) 2 J 2 = n i=1 x i x d i 2 J 3 = n i=1 F i 2 + n 3 i=3 ( F i 2 F i+1 2 + F i F i+1 2 + F i F i+3 2 ) J 4 = v v d 2 J 5 = (P P pred ) T W (P P pred ) where J 1 represents an applied torque summation to be minimized, J 2 is a foot position and orientation error for n-step, J 3 represent a summation of normal landing force ) (3) (4) 4015

Fig. 6. Development procedures for finite state machine, walking motion primitives, and realtime motion planner and periodicity error, J 4 is a walking velocity error, J 5 is a walking style difference comparing the predefined initial motion P pred, i means i th step, and w is a weighting factor. motions. Stable transition between two primitive motions is possible in bounded variation. Fig. 9. Target trajectory tracking performance (blue: desired target trajectory, green: measured joint trajectory, left: simulation, right: experiment). III. EXPERIMENT Fig. 8. Algorithm flowchart for optimizing walking controller. Fig. 10. Dynamic waking with our proposed method (walking speed = 2.1 km/h) C. Parametric Representation of Cyclic Gait Primitives We represent cyclic gait primitives as reduced control parameters p ri in (2). Dynamic walking for following given path is generated by sequentially composing primitive walking database. In Equation (4) J 5 is also used as a measure comparing the similarity between two primitive walking To validate the computational feasibility of our approach, we perform numerical and real experiments with a hardware prototype, Roboray. Roboray is a biped humanoid robot platform which is composed of 32 joints (except 40 finger joints). The robot itself is 1.5 meters tall and has a mass of about 62 kg. To find optimal control parameters, we use 4016

the Powell s conjugate gradient decent method which is an optimization algorithm for finding the local minimum of a function which may be not differentiable. The ODE (open dynamic engine [15]) library is used to calculate forward dynamics of a multi-body system with a floating base and frictional contact forces. As shown in Fig. 8, our optimization algorithm includes a procedure of learning by dynamics simulation. The motion optimization algorithm is developed with C++ language and executed on a Core 2 (2.5 GHz) personal computer. Each optimization problem took about three hour with 2 3000 iteration. A. Design and Control of the Hardware Prototype Fig. 7 shows our biped humanoid robot system. Each leg has six joints where the three pitch joints (hip, knee, and ankle pitch) are actuated by cable-driven modules and the remaining three joints (hip roll/yaw, ankle roll) are actuated by harmonic drive modules. The cable-driven modules are ball screw mechanisms which take up more space then the harmonic drive modules but they provide better joint compliance and back-drivability. In dynamic walking, this compliant joint with elastic elements is important to achieve energy efficient walking. Arm joints are actuated with gear driven modules. For the detailed specification for Roboray see [16]. The state dependent control torque in (1) is generated with low PD gains resulting in a poor tracking performance of the desired joint angular position (see Fig. 9). However, tracking performance of the desired torque is good except for the swing phase of each leg; during the swing phase, damping term prevents system oscillation at high speeds. As discussed in Section I, the accurate trajectory tracking performance in the ZMP based method leads to the energy inefficiency due to opposing (gravity-powered) natural dynamics. In our dynamic walking, we also observe the controlled falling motion like that of inverted pendulum. We can also observe periodic hip rising and falling motion like a natural human walking. B. Straight Walking Optimization with Target Step Length TABLE II OPTIMIZATION RESULT FOR A STRAIGHT WALKING (STEP LENGTH = 35CM) Initial motion Optimized motion J 1 1601 1429 J 2 6751 9.57 J 3 418 734 J 4 5457 8957 J 5 0 1877 Total 14048 13956 Straight walk is characterized with a zero turning angle (also involving in-place walk with zero step length and backward walk with opposite directional step length). Using our proposed movement optimization algorithm we can find stable straight walking primitives. Fig. 11. Changes in parameters with varying step lengths. We use different weighting factors for the cost functions. Listing the priority from the highest: J 2 (representing foot position error), J 1 (applied torque summation), J 3, J 4, and then lastly J 5. We set the weighting factor for J 2 the highest because we want to achieve a constant step length. As shown in Table II, J 2 is highly minimized and J 1 is minimized by 88%. Fig. 11 shows the results of the straight walk optimization with selected step lengths. We observe that p r3, p r0,sweep related to hip pitch sweeping are monotonically increasing with longer step lengths, indicating that these parameters are highly related to the step length. Our optimization problem is so highly nonlinear that we could not observe any other distinct parameter relations. C. Curve Walking Optimization with Target Turning Angle Unlike straight walking, curve walking has asymmetric motion in left and right swing. We can observe a limit cycle which repeats itself every two cycles as shown in Fig. 12(b). This means two successive stepping motions are not identical and two periodic gaits are stable. In straight walking, we can observe a limit cycle which repeats on every cycle as shown in Fig. 12(a).We accomplished turning walk with constant curvatures as shown in Fig. 13. D. Balance Control By using optimal control parameter values, we achieve self-stable walking with only torso balance control at the level ground. To handle the internal and external disturbances (e.g., gait transition, terrain unevenness), we use the gravity compensation control and the center of pressure control with virtual model. For the detailed explanations of the control algorithms see [17]. E. Gait Planning with Primitive Walking Motions To follow a given path, we first search the appropriate primitive motion as finding the closest primitive motion with desired step length, velocity and turning angle. We then globally plan by ordering primitives (also checking stable 4017

(a) Fig. 14. Gait planning experiment with primitive walking motions (b) Fig. 12. Phase portraits for (a) stable symmetric walk (straight walk with step length = 33cm, step time = 0.85s), (b) stable asymmetric walk (curve walk with turning angle = 20deg, step time = 9.5s) (a few orbital trajectories moves away from the limit cycle due to transition motions from starting posture to steady state walk or vice versa). Fig. 13. Curve walking experiment transition) with a number of steps. Using local planner, gait motion is replanned periodically to avoid moving obstacles. IV. CONCLUSION We proposed a framework to generate dynamic walking for biped robots. A set of self-stable gait primitives is constructed by dynamics-based movement optimization algorithm. Dynamic walking for following a given path involving curves is then generated online via sequentially composing primitive motions. Resulting dynamic gaits show natural walking motion compared to the conventional knee bent walkers. Although we exclusively dealt with the humanoid type robot, our framework is easily extendable to other robot mechanisms (e.g., industrial manipulators) for energy efficient and fast motions. [2] L. Roussel, C. Canudas de Wit, and A. Goswami, Generation of energy optimal complete gait cycles for biped robots, Proc. IEEE Int. Conf. Robotics and Automation, pp. 2036-2041, 1998. [3] G. Bessonnet, P. Seguin, and P. Sardain, A parametric optimization approach to walking pattern synthesis, Int. J. Robotics Research, vol. 24, no. 7, pp. 523-536, 2005. [4] B. Siciliano and K. Oussama, eds., Handbook of Robotics, Springer Verlag, Heidelberg, 2008. [5] R. D. Gregg, T. Bretl, and M. W. Spong, Asymptotically stable gait primitives for planning dynamic bipedal locomotion in three dimensions, Proc. IEEE Int. Conf. Robotics and Automation, pp. 1695-1702, 2010. [6] T. Takenaka, T. Matsumoto, and T. Yoshiike, Real time motion generation and control for biped robot - 1st report: walking gait pattern generation, Proc. IEEE Int. Conf. Robots and Systems, pp. 1084-1091, 2009. [7] http://www.toyota.co.jp/en/special/robot [8] D.J. Braun and M. Goldfarb, Control approach for actuated dynamic walking in biped robots, IEEE Transactions on Robotics, vol. 25, No. 6, pp. 1292-1303, 2009. [9] J. M. Wang, D.J. Fleet, A. Hertzmann, Optimizing walking controllers, ACM Transactions on Graphics, vol.28, No. 5, 2009. [10] K. Harada, K. Hauser, T. Bretl, J.-C. Latombe, Natural motion generation for humanoid robots, Proc. IEEE Int. Conf. Robots and Systems, pp. 833-839, 2006. [11] J. Laszlo, M. Panne, and E. Fiume, Limit cycle control and its application to the animation of balancing and walking, Proc. Int. Conf. on Computer Graphics and Interactive Techniques, pp. 155-162, 1996. [12] B. Lim, J. Babic, and F.C. Park, Optimal jumps for biarticular legged robots, Proc. IEEE Int. Conf. Robotics and Automation, pp. 226-231, 2008. [13] B. Lim, B. Kim, F.C. Park, and D.W. Hong, Movement primitives for three-legged locomotion over uneven terrain, Proc. IEEE Int. Conf. Robotics and Automation, pp. 2374-2379, 2009. [14] S. Yoo, C. Park, S. You, and B. Lim, A dynamics-based optimal trajectory generation for controlling an automated excavator, J. Mech. Eng. Sci., Vol. 224, Part C, pp. 0954-4062, 2010. [15] ODE. Open dynamics engine. http://www.ode.org. [16] J. Kim, Y. Lee, S. Kwon, K. Seo, H. Lee, H. Kwak, and K. Roh, Development of the lower limbs of a humanoid robot, Proc. IEEE Int. Conf. Robots and Systems, 2012. [17] B. Lim, M. Lee, J. Kim, J. Lee, J. Park, K. Seo, and K. Roh, Control Design to Achieve Dynamic Walking on a Bipedal Robot with Compliance, Proc. IEEE Int. Conf. Robotics and Automation, pp. 79-84, 2012. REFERENCES [1] S.H. Collins, A. Ruina, R. Tedrake, and M. Wisse, Efficient bipedal robots based on passive dynamic walkers, Science Magazine, vol. 307, pp. 1082-1085, 2005. 4018