Dynamically stepping over large obstacle utilizing PSO optimization in the B4LC system

1 Dynamically stepping over large obstacle utilizing PSO optimization in the B4LC system QI LIU, JIE ZHAO, KARSTEN BERNS Robotics Research Lab, University of Kaiserslautern, Kaiserslautern, 67655, Germany This paper proposes a control structure to resolve the issue of dynamically stepping over large obstacles in the B4LC control system. We reform the local control units LegSwing, LockHip and KneeF lexion respectively. The optimization module with Particle Swarm Optimization (PSO) method is employed to tune the parameters of those controllers by formulating locomotion stability. The optimization process and further validation are conducted on a 3-dimensional simulated bipedal robot. The simulation results reveal that the suggested approach enables robot to dynamically step over a large obstacle with 20cm height by 15cm width in a short time duration. Keywords: Bipedal locomotion, Particle Swarm Optimization 1. Introduction and related work To achieve human-like stable bipedal locomotion control in rough environment, Luksch proposed Bio-inspired Behavior-based Bipedal Locomotion Control (B4LC) system. 1 By transferring some of key findings in biomechanics and biology of human locomotion control to bipedal robot, it allows for various bipedal motions, e.g. changing speed locomotion, cyclic walking on even terrain and uneven terrain with small obstacles, as well as locomotion with external pushes. 2,3 However, compared to humans superior mobility in uneven ground, the B4LC system shows limited capability of large obstacle avoidance. Accordingly the state-of-the art on this topic is extensive and still growing. Michel et al. presented an approach that enables robot to navigate toward desired goal position while avoiding obstacles, whereas limited work is on the capability of traversing a large obstacle. 4 Zhou et al. introduced a control framework by exploiting the redundancy of pelvis rotation and generating foot trajectory when encountering large obstacles. 5 However, the static stepping over strategies reveal the limitation of redundant time

2 and energy consumption. Further researches have been conducted by Koch et al. that optimization method provides an efficient way to maintain stability during quasi-static overstepping motion. 6 Stasse et al. proposed an idea that using zero moment point (ZMP) criterion to prevent falling while dynamically crossing over an obstacle of 15cm. 7 The duration of obstacle stepping is significantly decreased compared to static strategies. Nevertheless, the challenge for those model-based approaches is complex dynamical models. To exploit the bipedal robot s potential capability in unstructured terrain, we concentrate on developing a control approach of dynamically stepping over large obstacle. We give a detailed introduction of the B4LC system and the proposed overstepping strategies. Furthermore, the optimization module in the B4LC system is utilized to find the optimized parameters of those local control units. 2. The concept of the B4LC system As described by Luksch, six classes of control units form a hierarchical B4LC control structure. 1 The highest layer locomotion mode evaluates the locomotion of robot, e.g. standing, cyclic walking and obstacle walking. Based on corresponding kinematic and kinetic events, five walking phases are further stimulated by SPG, which are weight acceptance, propulsion, stabilization, leg swing and heel strike respectively, as shown in Figure 1. The feed-forward and feedback control units at the local joints, namely motor pattern and reflex, will be activated to achieve required control behaviors. Fig. 1. 5 Phases of locomotion for left and right side. Each phase stimulates the corresponding motor patterns and reflexes at the local joint when the sensory event for phase transition is triggered. 1 Motor pattern produces uniform patterns of torque for one or more joints in feed-forward manner. According to target gait and speed, it is capable of forming the inherent dynamics or swinging upper and lower limb to achieve

3 certain body motions. Besides, the B4LC provides feedback control reflex at the local joints to generate reflexive motor actions based on sensory information. Some reflexes behave linear and nonlinear relation between sensor data and control output, in contrast, the others generate the control signals related to the occurrence of sensor event. 3. Overstepping control approach By studying on biomechanical behaviors, the hip and knee joints play an important role in obstacle avoidance during leg swing phase. Humans tend to bend knee joint when moving leg forward. Thus, the swing-leg is lifted up to reduce risk of hitting obstacles significantly. Additionally, hip joint of the swing-leg is locked with a large angle at the end of swing phase. This leads to a large step length assuring obstacle clearance. Inspired by those behaviors, we intend to develop the obstacle avoidance strategies in the leg swing phase of the B4LC system, whereas the control behaviors in the stance phases weight acceptance, propulsion and stabilization are kept constant. This enables robot to achieve smooth and fast transition from even ground locomotion to obstacle locomotion without suspending for redundant preparations. 3.1. Hip strategy To actively move swing-leg forward after ground clearance, motor pattern Leg Swing Obs generates desired torques with different amplitude and duration at hip joints in the sagittal plane, which are modulated as: ˆτ ls = A ls { 1 2 + 1 2 t sin( π( T 1 1 2 ) ) 0 t < T 1 1 T 1 t < T 2 1 2 1 2 t T2 sin( π( T 3 T 2 1 2 ) ) T 2 t < T 3 (1) where A ls, T 1, T 2 and T 3 are the parameters of Leg Swing Obs representing the maximum torque of torque command, the starting time of maximum torque, the ending of maximum torque and the total time of torque command respectively. Two different behaviors can be observed for the first and second moving leg during overstepping process. To prevent obstacle collision, biped is expected to achieve fast leg swinging and large hip flexion for the first step. Accordingly, intensive torques are generated at the hip joint. Whereas, as the distance from the second moving foot the obstacle is much less than the first one, there is limited time for knee-bending if large hip torque is produced at the second swing-leg. It is required that smaller torques are given to the rear leg s hip joint to achieve slow leg movement. Consequently,

4 we deploy two motor patterns Leg Swing Obs for the front and rear leg respectively with different parameters A f,ls and A r,ls, as described in Eq. 1. At the end of swinging phase, the swing-leg is expected to decelerate and then lock at a constant angle to prepare for a stable weight acceptance. Thus, local reflex Lock Hip Obs is designed to control the flexion of the hip joint at the end of swing phase. When the leg approaches a target angle, a damping torque which is proportional to the rotational velocity of the thigh is applied at the hip joint until the swinging phase come to the end. As soon as the thigh velocity become zero, the hip joint is locked with an target angle ˆα lh. The damping torque ˆτ lh and target angle ˆα lh are: ˆτ lh = ( α leg K torque ) P τ,lh (2) ˆα lh = (α lh,max + α pitch K pitch ) P α,lh (3) where α leg and α pitch stand for the velocity of swing-leg and the angular position of body pitch respectively. K torque and K pitch represent the parameters of torque factor and pitch factor, which are defined experimentally in. 1 α lh,max is a constant parameter denoting the maximum target hip joint angle. P τ,lh and P α,lh are the scale parameters which are defined as 1 during even ground locomotion. To achieving obstacle avoidance walking, the derived P τ,lh and P α,lh values are optimized in Section 4. 3.2. Knee strategy By studying on human gait, it is assumed that limited knee muscle behavior appears during leg swing phase of even ground locomotion. Knee joint tends to bend passively to guarantee ground clearance. However, to prevent large obstacle collision during overstepping phase, an active control of knee joint which can be conducted by reflex Knee Flexion Obs is required. As long as the first leg begins to swing forward, Knee Flexion Obs receives a target knee angle signal ˆα k,kf. As described in Algorithm 3.1, the reflex increases the activity with the equilibrium point setting to the target angle ˆα k,kf. The activity ranged from 0 to 1, which means that the higher value leads to a stronger influence of position control. After the target angle hip joint ˆα h,kf is reached, the activity of Knee Flexion Obs is formulated as zero. Afterwards, the lower leg moves forward freely to stretching position until the end of the leg swing. For the rear swing-leg, the reflex defines activity value autonomously as 1 with the target angle ˆα k,kf until the position of foot is higher than obstacle. As introduced in Section 3.1, smaller torques are generated at the hip joint to prevent obstacle collision. Thus, it is hard to achieve passive

5 rotation of knee joints for weight acceptance. Consequently, the reflex has to actively control the knee joint by gradually reducing the knee angle and activity value until knee is stretched. Algorithm 3.1 Knee Flexion Obs Algorithm 1: if forward leg swing then 2: if α hip < ˆα e,kf then 3: α knee = ˆα k,kf 4: activity = 1 5: else 6: activity = 0 7: end if 8: end if 9: if rear leg swing then 10: if swing foot not higher than obstacle then 11: α knee = ˆα k,kf 12: activity = 1 13: else 14: for knee stretching do 15: α knee α knee 0.1 16: activity activity 0.1 17: end for 18: end if 19: end if 4. Optimization-based obstacle obstacle locomotion 4.1. Particle swarm optimization algorithm Particle swarm optimization (PSO) method can be used to search the optimal parameters in the B4LC system as already demonstrated in. 3 The position of the particle in the searching space is decided by the value of the parameters in the B4LC system. The movement of a particle is guided in the direction of the position of each particle and the entire swarm with best fitness function values in the past iterations. The velocity of a particle in one iteration can be derived, as described in Eq. 4. v p = ωv p + c 1 Rand 1 ( B p x p ) + c 2 Rand 2 ( B g x p ) (4) where v p and x p represent the velocity and the position of a particle respectively. B p is the known best position of a particle in the past iterations, while B g the known best position of entire swarm in the past iterations. c 1 and c 2 mean the parameters of the acceleration constants respectively. Rand 1 and Rand 2 are the random values ranged within [0, 1]. The inertia weight ω controls the impact of the previous velocities on the current velocity. The larger inertia weight tends to favor global searching, while the smaller one leads to the local searching strategy. The fitness functions summarizing the distance of a given solution approaching to the objectives are calculated during each iteration. The best

6 position of particles B p and the best position of the entire swarm B g are updated by comparing the fitness values. The position of a particle x p is updated by summing the calculated particle velocity v p in Eq. 4. 4.2. Parameter optimization of obstacle locomotion An optimization module termed as Optimization Obstacle is integrated in the B4LC system by connecting the walking phase leg swing and the corresponding motor patterns and reflexes. A 8-dimensional vector P i,t that consists of the parameters of A f,ls, A r,ls, T 1, T 2, P τ,lh, P α,lh, ˆα k,kf and ˆα h,kf represents the position of a particle in the searching space. To increase the optimization efficiency, the searching space is constrained in a range based on the experimental studies of bipedal locomotion, as depicted in Eq. 5. A f,ls [0.0, 1.0] A r,ls [0.0, 0.6] T 1 [0.0, 0.4] T 2 [0.4, 0.9] P τ,lh [1.0, 2.0] P α,lh [0.0, 1.0] ˆα k,kf [0.0, 1.0] ˆα h,kf [1.0, 1.5] (5) The positions of each particles are initialized randomly in the searching space. The fitness function is formulated by considering optimization goals including robustness, stability and obstacle avoidance in form of: F = k 1 R d + k 2 R f k 3 P o (6) where k 1, k 2 and k 3 are the weightings of the fitness functions. R d is the reward for the stable locomotion, which is defined as linear to the walking distance that the robot achieves. R f indicates the final reward when the maximum walking steps reaches after successfully stepping over obstacle. P o represents the punishment when an obstacle collision is detected. 5. Simulation results The optimization scenario is set up on the 3D simulated robot with 21 degrees of freedom. The robot s height adds up to 1.8m and it weights 76kg. It is designed to start the optimization process with a stable walking on a flat ground. The obstacle is modeled as a block with dimensions of 20cm height by 15cm width. As long as the distance from stance foot to obstacle is less than 40cm, the obstacle locomotion is activated. Table 1. The parameters of controllers in obstacle locomotion Parameter A f,ls A r,ls T 1 T 2 P τ,lh P α,lh ˆα k,kf ˆα h,kf Value 0.98 0.38 0.24 0.8 0.06 1.24 0.96 1.15

The parameters generated by Optimization Obstacle known as the position of searching particles are fed into the motor pattern Leg Swing Obs, reflexes Lock Hip Obs and Knee Flexion Obs. The parameters are search in a 8-dimensional space until all searching particles acquire the optimized fitness function results. The optimized parameters are shown in Table 1. To validate the performance of the proposed control system, we conduct the obstacle locomotion transiting from even ground walking to obstacle avoidance phase. The angular rotation of hip and knee joint as well as foot height position are shown in Figure 2, which blue and red areas indicate the swing-phases of the first and second legs respectively. It is revealed that the first swing-leg generates large hip and knee angular rotation due to the intensive activation of Leg Swing Obs and Knee Flexion Obs. Thus, the biped can easily step over the obstacle in a high speed with the bending knee as shown in Figure 3. At the time of around 1.5s, the hip joint reaches the maximum angular position. The activity of Knee Flexion Obs is defined as zero again resulting in the forward passive rotation of the lower leg. 7 Fig. 2. The hip angle, knee angle and foot height position in the sagittal plane are shown. The blue and the red area indicate the overstepping phases of the first leg and the second leg respectively. A similar behavior is revealed from the second swing-leg. The knee joint is bended intensively after foot leaves ground. A smaller peek of hip joint angle can be observed due to less torques defined by parameter of A r,ls from Leg Swing Obs. This leads to a smooth transition between two locomotion modes. The robot is able to regain stable even ground walking after a successful obstacle avoidance. Besides, the dynamical overstepping duration is significantly reduced to 1.5s. The snapshots of successful stepping over obstacle are shown in Figure 3.

8 6. Conclusion In this paper we described novel obstacle avoidance control approach based on PSO optimization method in the B4LC system. The optimized parameters have been used to conduct torque and position control at the hip and knee joints during overstepping phases. Thus, the simulated biped can dynamically transit from even ground locomotion to obstacle overstepping locomotion freely. With optimized parameters, biped can easily step over large obstacle of 20cm height by 15cm in a short time duration. Fig. 3. Snapshots of successful stepping over large obstacle in the simulation. References 1. T. Luksch, Human-like control of dynamically walking bipedal robots, Verlag Dr. Hut, 2010. 2. J. Zhao, Biologically motivated push recovery strategies for a 3D bipedal robot walking in complex environments, Robotics and Biomimetics (RO- BIO), 2013 IEEE International Conference on. IEEE, 2013. 3. Q. Liu, Adaptive Motor Patterns and Reflexes for Bipedal Locomotion on Rough Terrain, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2015). 4. P. Michel, Vision-guided humanoid footstep planning for dynamic environments, IEEE/RAS International Conference on Humanoid Robots, 2005. 5. C. Zhou, Exploiting the Redundancy for Humanoid Robots to Dynamically Step Over a Large Obstacle, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2015). 6. K. Koch, Optimization based exploitation of the ankle elasticity of hrp-2 for overstepping large obstacles, IEEE/RAS International Conference on Humanoid Robots, 2014. 7. O. Stasse, strategies for humanoid robots to dynamically walk over large obstacles, IEEE Transactions on Robotics, 2006.