Hardware Implementation of a CPG-Based Locomotion Control for Quadruped Robots

Hardware Implementation of a CPG-Based Locomotion Control for Quadruped Robots Jose Hugo Barron-Zambrano 1, Cesar Torres-Huitzil 1, and Bernard Girau 2 1 Information Technology Laboratory, CINVESTAV Tamaulipas, Mexico {jhbarronz,ctorres}@tamps.cinvestav.mx 2 Cortex team, LORIA-INRIA Grand Est, Vandoeuvre-les-Nancy Cedex, France {Bernard.Girau}@loria.fr Abstract. This paper presents a hardware implementation of a controller to generate adaptive gait patterns for quadruped robots inspired by biological Central Pattern Generators (CPGs). The basic CPGs are modeled as non-linear oscillators which are connected one to each other through coupling parameters that can be modified for different gaits. The proposed implementation is based on an specific digital module for CPGs attached to a soft-core processor so as to provide an integrated and flexible embedded system. The system is implemented on a Field Programmable Gate Array (FPGA) device providing a compact and low power consumption solution for generating periodic rhythmic patterns in robot control applications. Experimental results show that the proposed implementation is able to generate suitable gait patterns, such as walking, trotting, and galloping. 1 Introduction The design of locomotion control systems of legged robots is a challenge that has been partially solved. In the literature, broadly, there are two main approaches to the design of locomotion control systems, the mathematical model-based and the biologically inspired approach. In the former, to move a leg in a desired trajectory, the joint angles are calculated in advance, by using a mathematical model that incorporates both robot and environment parameters, to produce a sequence of actions algorithmically scheduled [1]. The second approach uses CPGs which are supposed to play an important role in locomotion. CPGs are comprised of neural oscillators located in the spine of vertebrates and in the segmental ganglia of invertebrates [2]. CPGs are often modeled as oscillators that have mutually coupled excitatory and inhibitory neurons, following a regular structure. The CPGs have the ability to automatically generate complex control signals for the coordination of muscles during rhythmic movements, such as walking, running, swimming and flying [3]. The CPG-based approach for the design of locomotion control systems has several advantages. Due to the limit cycle behavior of neural oscillators, i.e. to produce stable rhythmic patterns, the system rapidly returns to its normal K. Diamantaras, W. Duch, L.S. Iliadis (Eds.): ICANN 2010, Part II, LNCS 6353, pp. 276 285, 2010. c Springer-Verlag Berlin Heidelberg 2010

Hardware Implementation of a CPG-Based Locomotion Control 277 rhythmic behavior after transient perturbations of the state variables. This provides robustness against perturbations. As a result of the natural synchronization and coordination of CPGs, the amount of computations is reduced. The synaptic plasticity of interconnections and feedback signals, used to integrate sensory information, allow CPGs to produce flexible locomotion in unknown environments [4,5]. However, one of the main disadvantages of CPGs is that their parameters have to be tailored for specific applications, and there are few methodologies to generate the rhythmic signals. The parameters are usually tuned either by trial and error method or by some optimization algorithms, genetic algorithms for example. These methodologies are still insufficient to tune the parameters for generating a periodic signal with a specific shape [6]. To address some of the future challenges for robotics, the miniaturization of walking, running and flying robots will be needed, so as to look for real-time adaptability of robots to the environment. These technologies will require small, low-cost, power efficient and adaptive controllers which might greatly benefit from custom bio-inspired hardware. Currently, some researches have used CPGbased locomotion control systems in robots. For example, CPG models have been used for controlling swimming robots, such as a salamander robot [7] and a turtle robot [8]. CPGs also have been used on quadrupeds, hexapods and octopods robots [9,10]. Control systems for quadruped robots using CPGs have been explored by Hiroshi Kimura et al. [11]. Authors have developed a quadruped walking robot capable of adapting to irregular terrain using the Matsuoka oscillator. Other works on CPGs in quadruped robots can be found in Billard et al. [12] and Shan et al. [13]. Many of these applications have been developed using dedicated hardware, both analog and digital [11,12,13]. On one hand, CPGs have been implemented using microprocessors providing high accuracy and flexibility but those systems consume high power and occupy a large area restricting their utility in embedded applications. On the other hand, analog circuits have been already proposed, being computation and power efficient but they usually lack flexibility and dynamics and they involve large design cycles. In this paper an FPGA-based hardware implementation to generate different gaits for quadruped robots is presented, based on established principles of locomotion that mimics the features of biological CPGs. A custom implementation of the Van Der Pol CPG attached to a Xilinx microblaze processor is presented and discussed. Potentially, this approach might provide modular control circuits that are adaptable and able to generate complex, coordinated movements. The goal of this implementation is to show the feasibility of self-contained locomotion solutions using modular, adaptable and compact modules with a higher degree of programmability to scale up to legged robots with high degrees of freedom. 2 CPG-Based Locomotion 2.1 Quadruped Gaits Animal locomotion employs different periodic patterns known as animal gaits. Researchers have established that gaits possess certain symmetries and have

278 J.H. Barron-Zambrano, C. Torres-Huitzil, and B. Girau Fig. 1. Typical gait patterns in quadruped locomotion and their relative phases between the limbs modeled the gaits of quadrupeds by a system of coupled cells where each cell is composed of a set of neurons directly responsible to synchronize the movement of their limbs. A simplified mathematical model of CPG-based locomotion consists of using one cell per limb and replacing each cell by a nonlinear oscillator. Thus, quadruped gaits are modeled by coupling four nonlinear oscillators, and by changing the coupling strength, it is possible to reproduce rhythmic locomotion patterns. In rhythmic movements of animals, a transition of the rhythmic movements is often observed. As a typical example, horses choose different locomotive patterns in accordance with their needs, locomotive speeds or the rate of energy consumption. In addition, each gait pattern is characterized by relative phase among the limbs [14]. Figure 1 shows the typical horse gait patterns and its relative phases between the limbs. Here, LF, LH, RF, and RH stand for left forelimb, left hindlimb, right forelimb, and right hindlimb, respectively. 2.2 Basic CPG Model There are several models for neural oscillators to model the basic CPG to control a limb, such as Amari-Hopfield model [15], Matsuoka model [12] and Van De Pol model [16]. In this work, the basic cell is modeled by a Van Der Pol (VDP) oscillator which is a relaxation oscillator governed by a second-order differential equation (equation 1): ẍ α(p 2 x 2 )ẋ + ω 2 x =0 (1) where x is the output signal from the oscillator, α, p and ω are the parameters that tune the properties of oscillators. In general terms, α affects the shape of the waveform, the amplitude of x depends largely on the parameter p. When the amplitude parameter p is fixed, the output frequency is highly dependent on the parameter ω. However, a variation of parameter p can slightly change the frequency of the signal, and α also can influence the output frequency. Actually, the VDP equation satisfies the Linard s theorem ensuring that there is a stable limit cycle in the phase space. Using the Linard s transformation, equation 1 can be rewritten as: ẋ = y ẏ = α(p 2 x 2 )y + ω 2 (2) x

Hardware Implementation of a CPG-Based Locomotion Control 279 2.3 Quadruped CPG Network In this work, the locomotion control system of a quadruped is modeled as a network of four VDP oscillators as shown in the figure 2a as suggested in most works reported in the literature [4,11]. Each oscillator controls the movement of a single limb. Within the CPG network, oscillators are mutually forced to oscillate in the same period and with a fixed phase difference. The mutual interaction among the VDP oscillators in the network produces a gait. By changing the phase difference between the oscillators, changing the coupling weights, it is posible to generate the three basic gaits. Figures 2b to 2d, present the configurations of the network that generate periodic rhythmic patterns corresponding to each gait (walk, trot, gallop). The dynamics of the ith coupled oscillator in the network is given by: ẍ c + α(p 2 c x2 cj ) x c ω 2 x cj =0 (3) For i =1,2,3,4,wherex c is the output signal from oscillator, x cj denotes the coupling contribution of its neighbors given by the equation 4: x cj = j λ cj x j (4) Where λ cj is the coupling weight that represents the strength of jth oscillator over the current oscillator. The generation of the respective gaits depends on the values of the system parameters. 3 Digital Hardware Implementation In this section, we describe the architecture of the CPG controller for interlimb coordination in quadruped locomotion. First, the design considerations for the implementation are presented. Next, the basic Van Der Pol Oscillator that constitute a part of the CPG network is given. Finally, the architecture of the complete system is described. (a) (b) (c) (d) Fig. 2. (a) General CPG network. (b)-(c) Functional configurations corresponding to the typical gaits patterns. Black and white dots represent excitatory and inhibitory connections, respectively.

280 J.H. Barron-Zambrano, C. Torres-Huitzil, and B. Girau 3.1 Design Considerations The Van Der Pol oscillator is suitable for CPG implementation as a digital circuit, however two main factors for an efficient and flexible FPGA-based implementation should be taken into account: a) arithmetic representation, CPG computations when implemented in general microprocessor-based systems use floating point arithmetic. An approach for embedded implementations is the use of 2s complement fixed point representation with a dedicated wordlength that better matches the FPGA computational resources and that saves further silicon area at the cost of precision, and b)efficiency and flexibility, embedded hard processor cores or configurable soft processors developed by FPGA vendors add the software programmability of optimized processors to the fine grain parallelism of custom logic on a single chip [17]. In the field of neural processing, several applications mix real-time or low-power constraints with a need for flexibility, so that FPGAs appear as a well-fitted implementation solution. Most of the previous hardware implementation of CPGs are capable of generating sustained oscillations similar to the biological CPGs, however, quite a few have addressed the problem of embedding several gaits and performing transitions between them. One important design consideration in this paper, is that the FPGA-based implementation should be a platform well suited to explore reconfigurable behavior and dynamics, i.e., the platform can be switched between multiple output patterns through the application of external inputs. 3.2 Module of Van Der Pol Oscillator From analysis of equation 2, three basic operations were used: addition, subtraction and multiplication. Thus, one block for each operation was implemented with 2 s complement fixed-point arithmetic representation. Figure 3a shows a simplified block diagram of the proposed digital architecture for the discretized VDP equation. In the first stage, the value of X ci is calculated: this value depends on the X c -neighbors and the coupling weight values. This stage uses four multipliers and one adder. The square values of p, X ci and ω are calculated in the second stage, it uses three multipliers. In the third stage, the values of α y c and p 2 X ci are calculated, one multiplier and a subtracter are used. The fourth stage computes the values of α y c (p 2 X ci )andω 2 X ci. This stage uses two multipliers. For the integration stage, the numerical method of Euler was implemented by using two shift registers and two adders. The integration factor is implemented by a shift register, which shifts six positions the values of y c and x c to provide an integration factor of 1/64. The block labeled as Reg stands for accumulators that hold the internal state of the VPD oscillators. Finally, the values y c and x c are obtained. The size word for each block was 18-bit fixed point representation with 11- bit for the integer part and 7-bit for the fractional part. Figure 3b shows the amplitude average error using different precisions for the fractional part. The errors were obtained from the hardware implementation. In figure 3b, it can be appreciated that the average error decreases as the resolution of the input

Hardware Implementation of a CPG-Based Locomotion Control 281 (a) (b) Fig. 3. (a) Digital hardware architecture for the Van Der Pol oscillator (b) Average error as a function of the bit precision used in the basic blocks variables is incremented. This reduction is not linear, and the graphic shows a point where such reduction is not significant. Seven bits were chosen as a good compromise for average error and implementation resources. 3.3 Quadruped Gait Network Architecture In the CPG model for quadruped locomotion all basic VDP oscillators are interconnected, as shown on figure 2a, through the connection weights (λ ij ). In order to overcome the partial lack of flexibility of the CPG digital architecture, it has been attached as a specialized coprocessor to a microblaze processor following an embedded system design approach so as to provide a high level interface layer for application development. A bank of registers is used to provide communication channels to an embedded processor. The bank has twenty-three registers and it receives the input parameters from microblaze, α, p 2, ω 2, λ ij and the initial values of each oscillator. The architecture sends output data to specific FPGA pins. Figure 4 shows a simplified block diagram of the VPD network interfacing scheme to the bank registers and the microblaze processor.

282 J.H. Barron-Zambrano, C. Torres-Huitzil, and B. Girau Fig. 4. Complete architecture for embedded implementation of a CPG-based quadruped locomotion controller 4 Implementation Results The CPG digital architecture has been modeled using the Very High Speed Integrated Circuits Hardware Description Language (VHDL) and Python was used for the implementation and simulation software. The CPG module has been attached as a slave coprocessor to the microblaze soft-processor using the PLB bus and a set of wrapping libraries according to the Xilinx design flow for embedded systems. The system has been synthesized using ISE Foundation and EDK tools from Xilinx targeted to a Spartan-3E device. To test the hardware implementation, a C-based application was developed on the microblaze to set the values of the parameters in the hardware digital implementation. The implementation was validated in two ways. The first one, the results were sent to the host computer through serial connection to visualize the waveforms generated by the module. Then, the hardware waveforms were compared with the software waveforms. In the second way, results were sent to digital-analog converter (DAC) and the output signal from DAC was visualized on a oscilloscope. Figure 5 shows, the periodic rhythmic patterns corresponding to the gaits (walk, trot, gallop) generated by hardware implementation. The values of weight matrix to configure the CPG network are shown in table 1. The initial values, x 0 =1,x 1 =1,x 2 =1, x 3 =1,y 0 = y 1 = y 2 = y 3 =0,α =1,p 2 =2,ω 2 = 20 were used. The values were calculated experimentally with a software implementation. Figure 5d shows the patterns for two gaits, walk and trot, and the transitions between them. The phase attractors for one VDP oscillator during walking and trotting, are shown in figures 5e and 5f. The phase attractor figures show the adaptability process until the stable cycle in the oscillator is reachieved. The time to reach the stable cycle is around 2 seconds. The system was synthesized to a Spartan-3E device using Xilinx ISE and EDK tools and tested in the Spartan-3E starter kit development board. Table 2 shows a summary of the FPGA resource utilization of the network architecture.

Hardware Implementation of a CPG-Based Locomotion Control 283 (a) Walk (b) Trot (c) Gallop (d) Transitions between gaits: walk to trot (e) Walk phase (f) Trot phase Fig. 5. Three basic gaits and transition between walking and trotting

284 J.H. Barron-Zambrano, C. Torres-Huitzil, and B. Girau Table 1. Weight matrix to configure the CPG network Gait Walk Trot Gallop 1.0 0.2 0.2 0.2 1.0 0.2 0.2 0.2 1.0 0.2 0.2 0.2 Weight 0.2 1.0 0.2 0.2 0.2 1.0 0.2 0.2 0.2 1.0 0.2 0.2 0.2 0.2 1.0 0.2 0.2 0.2 1.0 0.2 0.2 0.2 1.0 0.2 0.2 0.2 0.2 1.0 0.2 0.2 0.2 1.0 0.2 0.2 0.2 1.0 Table 2. Hardware utilization for implementation of the CPG control for a quadruped targeted to a Xilinx XC3S500e-5fg320 device Resource LTUs Flip-Flops Slices Embedded multipliers Maximum clock frequency Utilization 375 144 221 20 28 MHz 5 Conclusions and Future Work This work has presented a hardware digital implementation for Central Pattern Generator suitable for locomotion control of quadruped robots. The implementation takes advantage of the distributed processing of FPGA computational resources. The presented examples show that the measured waveforms from the FPGA-based implementation agree with the numerical simulations. The architecture of the elemental Van Der Pol oscillator was designed and attached as a co-processor to microblaze processor. The implementation provides flexibility to generate different rhythmic patterns, at runtime, suitable for adaptable locomotion and the implementation is scalable to larger networks. The microblaze, allow us to propose an strategy for both generation and control of the gaits, and it is suitable to explore the design with dynamic reconfiguration in the FPGA. Future work will focus on: (a) explore larger networks for a complete locomotion controller and embedding more diverse transitions (b) incorporate the feedback from the robot body to improve the generation of patterns, (c) integrate visual perception information to adapt the locomotion control in an unknown environment and (d) to scale up the present approach to legged robots with several degrees of freedom to generate complex rhythmic movements and behaviors. Acknowledgment Authors want to thank the partial support received from CONACyT through the research grant number 99912 and the INRIA associate team CorTexMex.

References Hardware Implementation of a CPG-Based Locomotion Control 285 1. Kimura, H., Shimoyama, I., Miura, H.: Dynamics in Dynamic Walk of Quadruped Robot. J. of the Society of Instrument and Control Engineers 29, 220 225 (1990) 2. Delcomyn, F.: Neural Basis of rhythmic behavior in animals. Science 210, 492 498 (1980) 3. Hooper, S.L.: Central Pattern Generator. Current Biology 10(5), 176 177 (2000) 4. Fujii, A., Saito, N., Nakahira, K., Ishiguro, A., Eggenberger, P.: Generation of an adaptive controller CPG for a quadruped robot with neuromodulation mechanism. In: IEEE Int. Conf. Intell. Robots Syst. (2002) 5. Ijspeerta, A.J.: Central pattern generators for locomotion control in animals and robots: A review. Neural Networks 21(4), 642 653 (2002) 6. Zielinska, T.: Coupled oscillators utilized as gait rhythm generators of a two-legged walking machine. Biol. Cybern. 74, 263 273 (1996) 7. Ijspeert, A.J., Crespi, A., Ryczko, D., Cabelguen, J.: From swimming to walking with a salamander robot driven by a spinal cord model. Science 9, 1416 1420 (2007) 8. Zhao, W., Hu, Y., Wang, L.: Construction and Central Pattern Generator- Based Control of a Flipper-Actuated Turtle-Like Underwater Robot. Advanced Robotics 23, 19 43 (2009) 9. Arena, P., Fortuna, L., Frasca, M., Sicurella, G.: n adaptive, self-organizing dynamical system for hierarchical control of bio-inspired locomotion. IEEE Transactions on Systems, Man and Cybernetics, Part B 34(4), 1823 1837 (2004) 10. Inagaki, S., Yuasa, H., Suzuki, T., Arai, T.: Wave CPG model for autonomous decentralized multi-legged robot: Gait generation and walking speed control. Robotics and Autonomous Systems 54(2), 118 126 (2006) 11. Fukuoka, Y., Kimura, H., Cohen, A.H.: Adaptive dynamic walking of a quadruped robot on irregular terrain based on biological concepts. The International Journal of Robotics Research 34, 187 202 (2003) 12. Billard, A., Ijspeert, A.J.: Biologically inspired neural controllers for motor control in a quadruped robot. In: Proceedings of the IEEE-INNS-ENNS international joint conference on neural networks, IJCNN 2000, vol. VI, pp. 637 641. IEEE Computer Society, Los Alamitos (2000) 13. Still, S., Tilden, M.W.: Controller for a four legged walking machine. In: Smith, S., Hamilton, A. (eds.) Neuromorphic Systems: Engineering Silicon from Neurobiology, pp. 138 148. World Scientific, Singapore (1998) 14. Dagg, A.I.: Gait in mammals. Mammal Rev. 3, 135 154 (1973) 15. Amari, S.: Characteristic of the random nets of analog neuron-like elements. IEEE Trans. Syst., Man, Cybern. SMC-2, 643 657 (1972) 16. Van Der Pol, B., Van der Mark, J.: The heartbeat considered as a relaxation oscillation, and an electrical model of the heart. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science Ser. 7(6), 763 775 (1928) 17. Torres-Huitzil, C., Girau, B., Gauffriau, A.: Hardware/software co-design for embedded implementation of neural networks. In: Diniz, P.C., Marques, E., Bertels, K., Fernandes, M.M., Cardoso, J.M.P. (eds.) ARCS 2007. LNCS, vol. 4419, pp. 167 178. Springer, Heidelberg (2007)