MASSIVELY DISTRIBUTED NEUROMORPHIC CONTROL FOR LEGGED ROBOTS MODELED AFTER INSECT STEPPING NICHOLAS STEPHEN SZCZECINSKI

Transcription

1 MASSIVELY DISTRIBUTED NEUROMORPHIC CONTROL FOR LEGGED ROBOTS MODELED AFTER INSECT STEPPING By NICHOLAS STEPHEN SZCZECINSKI Submitted in partial fulfillment of the requirements For the degree of Master of Science Thesis Adviser: Dr. Roger D. Quinn Department of Mechanical Engineering CASE WESTERN RESERVE UNIVERSITY January 2013

2 CASE WESTERN RESERVE UNIVERSITY SCHOOL OF GRADUATE STUDIES We hereby approve the thesis of Nicholas Stephen Szczecinski Candidate for the Master of Science degree*. (signed)roger D. Quinn (chair of committee) Roy E. Ritzmann Michael S. Branicky (date) 3 October 2012 *We also certify that written approval has been obtained for any proprietary material contained within.

3 Table of Contents List of Figures... 5 Chapter 1 Introduction Chapter 2 Literature Review Chapter 2.1 Insect Models Chapter 2.2 Robot Models Chapter 3 Simulation Environments and Models Chapter 3.1 Animatlab and Supplementary Environments Chapter Mechanical Simulations and Models Chapter Muscle Model Chapter Neuron and Synapse Models Chapter 4 Robust Robotic Stepping Chapter 4.1 Stepping Rules Chapter 4.2 Implementation of Stepping Rules Chapter Sensory Information Chapter Sensory Interneurons and Reflex Reversal Chapter Central Pattern Generators Chapter Muscle Control Units Chapter 4.3 Networks and Their Function Chapter Middle Leg Network Chapter Front Leg Network Chapter Hind Leg Network Chapter 4.4 Stepping Results Chapter Stepping Robustness Chapter Comparison to Blaberus Chapter 4.5 Robotic Implementation Chapter 5 Smooth Low Level Transitions Chapter 5.1 Implementing Behavior Changes via Reflex Reversals Chapter 5.2 Flexible Networks Capable of Changing Gait Chapter Gait Changes in the Middle Leg

4 Chapter Gait Changes in the Front Leg Chapter 5.3 Effect of CPGs on Gait Transitions Chapter 6 Smooth Behavioral Changes Chapter 6.1 Intermediate Level Coordination Chapter 6.2 Intermediate and Low-Level Gait Changes Chapter Changing Intermediate Gait Chapter Changing Low Level Gait Chapter 7 Conclusions and Future Work Chapter 7.1 Conclusions Chapter 7.2 Future Work Chapter Sensitivity Analysis and Parameter Tuning Chapter Actuator Types Chapter Intermediate Circuit Chapter Robotic Leg Appendix A Network Topologies Front Leg Middle Leg Hind Leg Intermediate Level Circuit Bibliography

5 List of Figures Table 1 - Table of stepping rules discovered in insects, how they are modified in SimROACH, and the original source Figure 2 Microscopic photographs of severed cockroach legs (left) for comparison to triangulated meshes (right) used in simulation Figure 3 Mechanical equivalent model to the linear Hill muscle model (top). Lengthtension relationship that limits tension output of the muscle (bottom left). Stimulustension relationship that determines the activation of the muscle (bottom right). Taken from (Shadmehr and Arbib 1992), accessed on Animatlab.com Figure 4 Plots showing the step response of neurons with different spiking threshold accommodation values. An accommodation of 0 makes the spiking frequency a function of stimulus current (top). An accommodation of 1 makes the spiking frequency a function of the derivative of the stimulus current (middle). A value between those will produce a response that is a combination of the two (bottom). The stimulus current (green) is 10 na in every picture Figure 5 Postsynaptic response of a neuron coupled to a tonically firing neuron via a synapse with a facilitation of 1 (top) and 0.5 (bottom). Facilitation values less than one cause the synapse to decay at a rate that is a function of presynaptic spiking frequency. 38 Figure 6 Plots that show the nonspiking neuron s step response without calcium currents (top), and with calcium currents during activation (middle) and deactivation (bottom). All stimuli (green) have a magnitude of 10 na Figure 7 Table of sensory triggers used to generate forward walking in the middle leg Figure 8 Table of sensory triggers used to generate forward walking in the front leg.. 45 Figure 9 Plots showing the measured angle (top), transduced current (middle), and resulting voltage of a neuron coding for the joint s rotation (bottom). The gray lines show that the integration lag between the current and the neuron s voltage is virtually nonexistent Figure 10 Plots showing the measured angle (top) and the current injected into the rest of the system to signal that an extreme position has been reached (bottom). There is no output until the joint reaches a certain limit, at which point it rapidly increases. The gray lines show what FTi angles signal to the rest of the network Figure 11 (Top) Unpublished results from the Zill lab showing how some populations in the cockroach respond to increasing load while others respond to decreasing load. (Bottom) Picture of network that processing loading information. Neuron A turns the signal D, the magnitude of the load on the foot, into a firing frequency (E.). Neuron B is inhibited by neuron A, and will fire when the load goes away. Neuron C is stimulated by 5

6 Neuron B with a decaying synapse, so its activity decays rapidly when the load ends (F.) Figure 12 Schematic of how a reflex reversal can be executed in this model. The Gait neuron can affect interneurons that relay sensory information, changing which neurons are affected by which sensors Figure 13 Voltage of one half-center of a CPG during oscillation. The oscillation reaches steady state after about 1 second Figure 14 Schematic of a CPG used in this model. Neurons 1 and 3 are the half-centers, communicating through interneurons 2 and Figure 15 Voltage of one half-center of a CPG during normal activity (top) and when the interneurons are inhibited by a current of na Figure 16 Voltage of one half-center of a CPG when the presynaptic neuron is strongly hyperpolarized at different points in the phase. A strong enough stimulus will reset the phase of the CPG at any point of the phase. The bars along the bottom show the bursting period before the stimulus was applied. A red circle is drawn around the perturbation, after which a normal looking period of activity is observed Figure 17 Behavior of the abstracted muscle pair system. If both stiffnesses (1,1) and set points (1,-1) are identical and the sin terms are removed, any initial condition will drive the system toward an angle between the two set points (A). Changing the stiffness of one set point (k1=5*k2), the system will move toward that set point (B). If the sin terms are included, the system can be tuned to oscillate with the desired frequency and amplitude (C). If the frequency is changed, the amplitude is decreased, since this system is a filter (D). The desired amplitude can be regained by increasing the stiffness of both set points (E). This is not something the current model is capable of. Keeping the original stiffnesses and instead increasing the set points will produce qualitatively similar behavior (F). This is the approach that the current model uses to increase the stiffness of a joint Figure 18 Plots of joint angles (blue) and extreme position neuron voltages (green). Note that flexion can be changed independently of extension (top) and vice versa (middle). They can also be changed together to change the mean angle (bottom) Figure 19 Schematic of the CPG and muscle control unit for the CTr joint. The CPG (red) only inhibits the Inter Pos neurons, which are interneurons between the error feedback control for each muscle and its motor neuron Figure 20 Control network for the middle leg of the cockroach with no particular gait active (top) and with the forward walking gait active (bottom). The inactive pathways have reduced fill

7 Figure 21 Control network for the front leg of the cockroach with no particular gait active (top) and with the forward walking gait active (bottom). The inactive pathways have reduced fill Figure 22 Control network for the hind leg of the cockroach. This network is much smaller than the others because no reflex reversals take place Figure 23 Three plots showing kinematics during walking in a middle leg without CPGs and under normal load (top), without CPGs during weighted walking (middle), and with CPGs during weighted walking (bottom). The leg is able to walk under normal conditions, but adding extra weight stops the reflex cascade. Adding CPGs to the model restores rhythmic behavior. The extra inertia causes high frequency noise in the kinematics that would otherwise be absent Figure 24 Plots showing kinematics during walking for a middle leg without feedback from one joint in a model without CPGs (top) and a model with CPGs (bottom). A CPG at every joint reduces the robot s reliance on sensory information in case of a malfunction Figure 25 Joint angles of the front leg during tripod walking. The kinematics of the animal (left) were recorded by Brown on an oiled plate. SimROACH s kinematics (right) are provided for comparison. The vertical axes are scaled to match biological data, and are the same in both graphs. Stance phase is indicated by gray shading Figure 26 Joint angles of the middle leg during tripod walking. The kinematics of the animal (left) were recorded by Brown on an oiled plate. SimROACH s kinematics (right) are provided for comparison. The vertical axes are scaled to match biological data, and are the same in both graphs. Stance phase is indicated by gray shading Figure 27 Plots comparing muscle activations with the onset of stance in Blaberus discoidalis (top) and SimROACH (bottom). In both systems the CTr joint is depressed to cause stance, which causes the extension of the FTi joint. The biological data was produced by the Zill lab. Stance is indicated in the bottom plot by gray shading Figure 28 Joint angles of the hind leg during walking. The kinematics of the animal (left) were recorded by Brown on an oiled plate. SimROACH s kinematics (right) are provided for comparison. The vertical axes are scaled to match biological data, and are the same in both graphs. Stance phase is indicated by gray shading Figure 29 Picture of the robotic leg used for hardware testing (A). It manages input and output through a NI CompactRIO (B) and outputs data to LabView (C) Figure 30 Joint Angles (top) and CPG activity (bottom) from a walking trial performed with the robotic leg. Stance is indicated by gray shading Figure 31 Diagrams that explain LegConNet when producing forward (left) and inside turning forward (right) behavior. Gait changes are generated by changing the connections 7

8 and thresholds between sensory influences and bistable CPGs. Taken with permission from (B L Rutter et al. 2011) Figure 32 Tables that show stepping rules for inside turning (top) and outside turning (bottom) implemented in the middle leg of this model. There is no one authoritative source for these turning rules, but they are based on literature and hypothesized transitions Figure 33 Control networks for inside turning (top) and outside turning (bottom) in the middle leg model. The sensory pathways are highlighted to match the rules listed in Figure 32. The behavior changes are the result of rerouting sensory information and turning CPGs off where necessary Figure 34 CPG output from the middle leg during the transition to inside turning (top) and outside turning (bottom). Stance is indicated by gray shading. Turning is indicated by pink shading Figure 35 Tables that show stepping rules for inside turning (top) and outside turning (bottom) implemented in the front leg of this model. There is no one authoritative source for these turning rules, but they are based on literature and hypothesized transitions Figure 36 Control network for the front leg configured to generate inside turning (top) and outside turning (bottom). The inactive pathways have been only partially filled. The rules for these networks are listed in Figure Figure 37 CPG output from the front leg during the transition to inside turning (top) and outside turning (bottom). Stance is indicated by gray shading. Turning is indicated by pink shading Figure 38 Plots showing how the command to flex the FTi joint (green) is only caused by loading (blue) in the model without CPGs (top), but can precede loading in the model with CPGs (bottom). Loading then reinforces this transition, making stepping even more robust Figure 39 Intermediate level circuit configured to produce a wave gait (A) and a tripod gait (B). Inactive pathways are shown with less fill. Synapses are color coded according to the key at the bottom Figure 40 Plots showing CPG activity in the three legs on one side while walking with a wave gait (top) and a tripod gait (bottom). The demonstrated patterns are consistent with gaits seen in insects Figure 41 CPG activity during the transition from a wave gait to tripod gait in ipsilateral (top) and contralateral (bottom) legs. The first trace is the same in each plot. Tripod walking and the transition are highlighted in pink

9 Figure 42 Picture of a segment of the intermediate circuit configured to turn right by stimulating the Turn Right neuron, which in turn stimulates the proper low level turning neurons Figure 43 Plots of CPG activity during the transition from forward walking to turning while using the wave gait (top) and the tripod gait (bottom). Turning is highlighted in pink. Dotted lines show that coordination is maintained during the transition Figure 44 Robot heading (top) during two typical turning trials. The robot is commanded to walk straight for 5 s (blue) and then turn (green). The paths were smoothed with a Gaussian kernel, and the curvature (bottom) for each trial was calculated as a function of path length. In the left turn trial, the RMS curvature was during forward walking and during turning. In the right turn trial, the RMS curvature was during walking and during turning Figure 45 Intermediate level circuit modified to require loading information to tell the ipsilateral leg to unload. This sensory information is only utilized during the metachronal wave gait

10 Acknowledgements This work, although it has my name on it, was a group effort. Two years ago I knew nothing about biology and was completely unfamiliar with the processes of research. Those around me have helped me learn and grow to make this project a reality. I must thank Dr. Quinn and Dr. Ritzmann for being talented and patient instructors and mentors. They have given me opportunities, criticisms, and encouragement that have improved my work and work ethic. I would also like to thank Dr. Branicky for his insight on this thesis and the future work on this project. I must thank everyone in the Biologically Inspired Robotics Lab, especially the Borg Cluster: Hunt for his methodical and thorough thought process, Lonsberry for his inspiring sense of creativity, BRTietz for his endless knowledge of everything, and Vickie for her fierce but friendly competition and motivation. Entering the lab with such a great support group has made this entire experience positive. I must thank my friends and family for being supportive during this particularly busy time of my life. 10

11 Abstract Massively Distributed Neuromorphic Control for Legged Robots Modeled After Insect Stepping By NICHOLAS STEPHEN SZCZECINSKI Simulated RObot exhibiting behavior CHanges (SimROACH) is a massively distributed control architecture for legged robots composed of simulated physiological neuron and synapse models. Its structure is based on insect neurobiology. Each joint uses a unique central pattern generator (CPG) to produce oscillation. The CPGs in each leg cannot directly communicate, but are coordinated by sensory influences, producing stepping motion. One CPG from each leg receives input from the same CPG in other legs, coordinating walking motion. The pathways that coordinate CPGs or legs can be modified by descending commands to change the way the joints flex or legs step with respect to one another, smoothly changing gait while in motion. SimROACH walks and changes gait in a simulated physics environment. SimROACH s middle leg network was further verified by successfully controlling a single robotic leg attached to a test stand. 11

12 Chapter 1 Introduction Insects are capable of producing flexible and robust walking motions. They negotiate different terrains by autonomously adapting their movements to suit their environment on the fly, something that robots often have difficulty doing correctly. This flexibility comes from the components and organization of their nervous systems, which are highly distributed and plastic. By stimulating or inhibiting part of the nervous system, the qualitative behavior of the entire insect changes, an approach that is rarely taken in robotics. This thesis presents Simulated RObot exhibiting behavior CHanges (SimROACH), a control architecture for legged robots composed of simulated physiological neuron and synapse models. As is hypothesized in insects, a few descending influences can reverse reflexes and modify interleg coordination pathways in SimROACH to change gait and other behaviors in mid-motion. To the author s knowledge, this work is unique in the fields of both biology and robotics. While impressive computational neuroscience models of legged locomotion control exist, no known computational neuroscience model incorporates the level of mechanical dynamics as in this simulation, Furthermore, there are no known robots or simulations of robots controlled with a computational neuroscience with this level of fidelity. This research has two primary goals and two secondary goals. The primary goals are to achieve robust robot walking through a control system based in insect neurobiology, as well as smooth behavior transitions with the same system. Cockroaches are some of the most agile hexapods, and a robot that could move like a cockroach would be extremely effective. SimROACH achieves robust walking by mimicking both the connections of the nervous system and mathematics of individual neurons. The entire system is controlled by computational neural models connected in ways suggested by 12

13 insect neurobiology literature. Each leg can walk while the body s height is changed, while stepping through holes, when weighted, and when some sensory information is eliminated. In addition, the front and middle legs can smoothly change between forward walking, inside turning, and outside turning motions, allowing SimROACH to change its heading while walking. SimROACH s behaviors are based on those observed in cockroaches. Direct comparison with cockroach movements is instructive because data for that model organism is available. Therefore SimROACH s secondary goal is to be a useful model of insect locomotion. Every piece of SimROACH s controls is taken directly from the literature or an educated guess based on biological hypotheses. SimROACH is able to produce some interesting behaviors despite being an incomplete model. Future work includes incorporating more control structures from neurobiology. Much research has been done to determine what causes coordinated stepping in insects (Akay et al. 2001; Bucher et al. 2003; Akay et al. 2004; Ridgel et al. 1999; Zill, Keller, and Duke 2009; Zill et al. 2011; Zill, Schmitz, and Büschges 2004). Chapter 2 reviews the literature on the subject and the following in this chapter is an introduction to that topic. Many ingenious experiments have revealed what sensory cues coordinate the motion of multiple joints (Akay et al. 2004). The middle leg is often the focus of these studies because its function is the most general of insect legs. The result of these experiments has been a more accurate understanding of how insect motor nervous systems function. One of the key findings is that each joint appears to be driven by its own central pattern generator (CPG) (Ryckebusch and Laurent 1993; Büschges, Schmitz, and Bässler 1995). Rather than any one central structure coordinating 13

14 in what fashion the different segments of the leg move, each can produce its own rhythmicity. In addition, many nonspiking neurons surrounding the rhythm generators help convert a rhythmic pattern into coordinated muscle contraction (Büschges 1995). Some interneurons receive drive from the CPG, while others show activity that is not affected or in anti-phase. Stimulating some of these neurons will cause the cycle phase to reset. The rate of oscillation can also be modulated, changing the overall frequency, changing the activity symmetry, or ceasing oscillation completely (Daun-Gruhn 2010). This massively distributed structure in insects is crucial to how SimROACH works. These CPGs are not coupled by direct connections (Büschges, Schmitz, and Bässler 1995), but rather by sensory information (Büschges et al. 2008; Akay et al. 2001). These influences are often described as reflexes that cause a change in a joint s timing (for example, from flexion to extension), but these influences likely affect CPG timing (Akay et al. 2004; Büschges et al. 2008). Other simulations have shown that this sensory coupling is an effective means by which to coordinate CPGs (Daun-Gruhn and Tóth 2010; Spardy et al. 2011; Daun-Gruhn 2010; Ekeberg, Blümel, and Büschges 2004). Chapter 3 Simulation Environments and Models describes the computational models used to simulate these neural populations, and Chapter 4 Robust Robotic Stepping explains how SimROACH uses these rules and structure to coordinate stepping. Since the joints in one leg are coordinated into stepping motion by sensory influences alone, changing where the sensory information goes can change the behavior of the leg. For instance, switching from forward to backward walking in the stick insect is the result of only changing what sensory cue causes raising and lowering of the leg (Akay et al. 2007). Other reflex reversals related to standing still (Akay and Büschges 2006) and 14

15 turning (Ekeberg, Blümel, and Büschges 2004) have been identified. In addition to neural connections, muscle activity (Mu and Ritzmann 2005) and joint kinematics (Brown 2011) are known to change when the cockroach Blaberus discoidalis produces sideways stepping motions. These rules have been successfully implemented in robotic models of insect legs capable of stepping according to walking rules (Lewinger and Rutter 2006) and changing these rules to produce turning behavior (Rutter et al. 2011). Chapter 5 Smooth Low Level Transitions explains how SimROACH makes low level network changes to change gait. In addition to low level activity and changes, intermediate level coordination has also been the focus of research. The Cruse rules provide observed rules for coordinating legs in various arthropods (Cruse 1990). Oil plate experiments with cockroaches and stick insects suggest that these coordination rules are enforced by neural connections between the legs rather than purely sensory or mechanical influences (Brown 2011; Gruhn, Zehl, and Büschges 2009). However, other work has shown that mechanical linking between legs also has an effect, perhaps reinforcing the neural pathways (Zill, Keller, and Duke 2009; Ridgel et al. 1999). Models that incorporate both means of coordination have been shown to successfully coordinate stepping in multi-leg models (Daun-Gruhn and Tóth 2010; Cruse et al. 1998), suggesting that intermediate coordination is indeed due to a combination of neural and sensory signals. Some robots have made use of more abstracted biological principles to produce effective interleg stepping. Robot II used a finite state machine implementation of generalized Cruse rules with additional sensory driven leg reflexes to produce robust stepping on irregular terrain (Espenschied et al. 1995). Other robots in the Biologically 15

16 Inspired Robotics Laboratory have sought to produce improved results with more accurate biological structures (Lewinger and Rutter 2006; Lewinger and Quinn 2010; Rutter, Taylor, and Bender 2011). More recently, the Buschges group has produced a robot Octavio controlled by artificial neural networks based on stick insect neurobiology. The neuron models used are abstracted, and the joints do no possess oscillating CPGs, but the network topologies are based on stick insect pathways (Von Twickel, Büschges, and Pasemann 2011; von Twickel et al. 2011). Not only are these robots excellent walkers, but they also replicate the results of stick insect walking experiments. None of these other systems, however, possess the behavioral flexibility of SimROACH, which is examined in Chapter 6 Smooth Behavioral Changes. Its motion is not identical to that of an insect, but it generates stepping and changes gaits in a similar manner. The massively distributed control architecture changes behavior through descending commands that change how sensory information affects each joint. While this system is unorthodox in the robotics community, initial tests with a robotic platform have shown that this system is indeed effective at generating stepping. The implementation in the robot model of a cockroach leg does not currently have the capability to change gait, this will be added in the near future. SimROACH is also novel in computational biology in that there is no simulation that is as complete. Not only does SimROACH simulate interjoint and interleg neural connections, it also simulates body dynamics and interaction with the environment. Even though some models use more sophisticated neuron models (Daun-Gruhn 2010; Daun- Gruhn and Tóth 2010), they and others do not simulate the kinetics of the animal (Cruse et al. 1998; Ekeberg, Blümel, and Büschges 2004). This trend is only now changing in 16

17 the field of computational neuroscience (Tóth, Knops, and Daun-Gruhn 2012). SimROACH is not a complete simulation of a cockroach, but the simulation is the most holistic to the author s knowledge. 17

18 Chapter 2 Literature Review Insect locomotion has been an active area of research for many decades. Much work has been done to understand how the multiple joints of each leg are coordinated into walking motion, how these joints change their motion to generate different gaits, and how legs communicate with one another. This research may be classified into behavioral, neural systems, or a combination of the two. The work has led to finite state machine and neural models of control systems that have sometimes been implemented in legged robots. Much of it has been conducted on stick insects in Europe and cockroaches in the United States. SimROACH s structure draws elements from both organisms since it is generally accepted that rules from one of these animals can be adapted to the other. SimROACH accomplishes its primary goals of robust walking and smooth behavior changes by mimicking what is known about insect locomotion. In addition, SimROACH s secondary goal is to be a useful model of insect locomotion. Therefore knowledge of insect neurobiology was crucial to its development. Most of the specific sensory-motor interactions incorporated into SimROACH come from results published after Many studies before this point revealed behavioral patterns, but the research described below identified specific neural systems and cause and effect relationships between sensors and muscles. Büschges, Schmitz, and Bässler (1995) found evidence that each joint in the stick insect leg is controlled by its own CPG. When deaffarented (i.e. all incoming connections were removed) and subjected to pilocarpine (i.e. an M-receptor agonist, mimicking acetylcholine), each joint s motor neurons fired rhythmically, but uncoordinated with other joints. This suggested that walking is due to the assembly of modular units, that is, independent joint controllers. These rhythms are coordinated in the intact animal by nonspiking sensory interneurons. 18

19 Hess and Büschges (1999) found that in the stick insect, the angle of the femurtibia (FTi) joint affected when the coxa-trochanter (CTr) joint extended or flexed in the stepping cycle. Bucher et al. (2003) found that these signals were more influential at extreme angles, and could cease CPG oscillation. Load sensors also contribute to coordinated stepping (Akay et al. 2001). Signals from the femoral campaniform sensilla (fcs) are important in generating motion in the FTi joint. Unloading the leg by flexing the CTr joint causes the FTi joint to extend, but loading the leg by extending the CTr joint does not cause the FTi joint to flex. Later work showed that the trochanteral campaniform sensilla (trcs) are more important for maintaining stepping coordination, while the fcs is important to controlling the FTi joint itself (Akay et al. 2004). Peg leg experiments in stick insects reinforced the observation that the trcs are most important for signaling leg loading. When the middle leg is deaffarented and deefferented distal from the middle of the femur, stick insects can walk in a normal fashion (Noah et al. 2004). This is consistent with the observations of Akay et al. (2001), which suggested that the other campaniform sensilla modulate FTi muscle strength, and thus were not important to peg leg walking. Further research on load sensors reaffirmed that loading modulates both muscle activity timing and strength throughout stepping (Zill, Schmitz, and Büschges 2004). CS are able to provide the rest of the nervous system with quite sophisticated input, including signals that respond to load magnitude, load direction, and the rates of increase and decrease of the load (Zill, Büschges, and Schmitz 2011). Also, some populations were active when the CS were unloaded (Ridgel et al. 1999). This means the lack of load 19

20 actively generates a sensory signal, which is a stronger influence on the network than the lack of a loading signal. These sensory influences were known to alter CPG timing among different joints, but how does CPG activity affect muscle contraction? Büschges et al. (2004) showed that when CPG activity was suppressed by hyperpolarizing current, motor neurons were more excited. When enough current to halt oscillation was applied to a CPG, the associated motor neurons depolarized to a constant value. This suggests that motor neurons are under a constant tonic drive, and CPGs suppress them rhythmically, rather than exciting them. Other work has sought to explain other types of leg behavior, such as standing still, walking backward, or turning. This has led to data on behavioral changes, as well as hypotheses as to how this occurs. (Mu and Ritzmann 2005) used kinematics and muscle activity to show that cockroaches change how they use their front four legs during turning, appearing to reverse key reflexes. Research in stick insects has produced similar results, showing that turning behavior is a low-level change (Hellekes et al. 2012). Further work with cockroaches has detailed the kinematic changes that occur in each joint of the organism while walking forward and turning (Brown 2011). Work in forward and backward walking stick insects revealed other reflex reversals and provided hypotheses about how these changes might occur in the nervous system (Akay et al. 2007). This work showed that altering the stepping motion of a single leg is a low level change, and can be produced by changing the order or direction of each joint s motion with respect to the others. The authors hypothesize this can be 20

21 accomplished by altering synaptic weights among parallel sensory influences, changing which sensory influences cause which joint to move. Interleg interactions have also been a topic of research. The Cruse rules are sufficient to coordinate multiple legs (Cruse 1990). These are based on behavioral experiments performed on various arthropods and have served as the basis of many animal models and robots (Lewinger and Quinn 2010; Cruse et al. 1998; Daun-Gruhn 2010; Beer et al. 1992; Espenschied et al. 1995; Nelson et al. 1997). These rules specify that loading a leg will promote unloading of the anterior leg, unloading a leg will prevent unloading of the anterior leg, leg stepping is targeted to track the successful placement of the anterior leg, legs will restep if they collide with the anterior leg, and loading the organism will increase the time spent in stance. The Cruse rules do not provide a mechanism, either neural or mechanical, for such coupling. Hypotheses and models based on proprioception and direct neural connections have been proposed. Experiments in stick insects suggest that the angle of FTi flexion caused transitions between stance and swing in adjacent legs (Bucher et al. 2003). Other experiments have shown that the onset of stance in one leg reduces the load in the anterior leg, promoting the transition to swing (Zill, Keller, and Duke 2009). Models of such coordination, however, typically use sensory information to modulate direct connections between CPGs to maintain coordination (Cruse et al. 1998; Daun-Gruhn 2010). 21

22 Table of Insect Stepping Rules and Their Adaptation to SimROACH Biological Observation Modifications in SimROACH Original Publication Each joint in the leg has its own CPG with no direct connections to the others None (Büschges, Schmitz, and Bässler 1995) FTi joint angle affects CTr motion Direction of FTi movement (Hess and Büschges 1999) reversed with respect to CTr in cockroach FTi joint angle more strongly affects CTr None (Bucher et al. 2003) motion at extreme angles fcs affects motion of the FTi joint All loading detected by tarsus (Akay et al. 2001) trcs affects timing of the FTi joint All loading detected by tarsus (Akay et al. 2004) trcs is the main input for determining All loading detected by tarsus (Noah et al. 2004) stance Loading modulates muscle activity throughout the leg during stance All loading detected by tarsus (Zill, Schmitz, and Büschges 2004) CS code for load, rate of increase of load, rate of decrease of load None (Zill, Büschges, and Schmitz 2011) CS code for unloaded None (Ridgel et al. 1999) Rhythmic muscle activity is due to CPG None (Büschges et al. 2004) suppression of otherwise excited motor neurons Strong hyperpolarizing stimulus to a CPG None (Büschges et al. 2004) will halt oscillation Muscle activity, joint angle ranges, and Amplitude of changes do not (Mu and Ritzmann 2005) joint angle phases change in the CTr and FTi joints of cockroaches while turning precisely match observations in cockroach Turning is the result of low level reflex None (Hellekes et al. 2011) changes only Turning behavior can be classified by None (Brown 2011) different joint kinematics than walking Gait changes (forward and backward walking) are due to reversing low level reflexes None (Akay et al. 2007) Loading a leg excites unloading the anterior leg; Unloading a leg inhibits unloading in the anterior leg; Loading a leg excites unloading in the contralateral leg; Unloading a leg inhibits unloading in the contralateral leg Legs may be coupled by sensory gated connections between one CPG in each leg TrF joint active in middle and hind legs during cockroach walking TrF joint only actively extended Direction of ipsilateral coupling reversed (that is, from front to back) Connections do not include sensory gating None TrF joint both actively flexed and extended (Cruse 1990) (Daun-Gruhn 2010) (Bender, Simpson, and Ritzmann 2010) (Carbonell 1947) Table 1 - Table of stepping rules discovered in insects, how they are modified in SimROACH, and the original source. Nearly all of these rules related to stepping have been implemented in SimROACH. Table 1 lists specific features, how they may be adapted to the cockroach, and the source from which each came. These features can be compared to prominent 22

23 computational biology projects: Walknet (Cruse et al. 1998), the Ekeberg leg simulation (Ekeberg, Blümel, and Büschges 2004), and the work of Silvia Daun-Gruhn (Daun- Gruhn 2010; Daun-Gruhn and Tóth 2010). Chapter 2.1 Insect Models The Walknet simulation is an artificial neural network that coordinates the joints and legs of a stick insect simulation into walking motions. Each of the legs can be in either of two states, stance and swing, at a time. These states select a positional controller that moves each joint toward extreme positions. These are implemented as feedforward perceptron networks that take in joint angles and output joint velocities. These networks can be modified to produce turning motions. In addition, the legs can be coordinated into a tripod or tetrapod gait in a kinematic simulation. These capabilities are impressive, but SimROACH differs in some key ways. SimROACH can produce the same types of behavior as Walknet (walking in tripod or tetrapod gaits and turning), but all mechanical dynamics are simulated. In addition, SimROACH uses dynamical neurons, not static neuroids. This allows SimROACH to use dynamical, naturally oscillating CPGs at each joint. Such a feature helps improve the stability of walking motion, particularly while changing gaits as discussed in Chapter 5.3 Effect of CPGs on Gait Transitions. The Ekeberg simulation is a finite state machine that uses rules from Akay et al. (2004), Akay et al. (2001), Bucher et al. (2003), Hess and Büschges (1997), and others to simulate walking in a dynamical simulation of all legs of the stick insect. Rules that generate stepping in the stick insect middle leg were adapted to the front and hind legs. Each joint s direction of motion was controlled by a bistable element whose state could be changed by the proper sensory cue. Each leg was simulated in software, but only one 23

24 was active at a time. The single legs were then used to repeat experiments performed on actual organisms. Results from both restricted stepping and single leg stepping were replicated, showing the merit of such modeling work. The model, however, was admittedly simple, and the authors hoped that muscle models and magnitude control of the muscles would produce a more realistic model. These are both features that SimROACH incorporates. In addition, SimROACH uses dynamical CPGs, neurons and synapses instead of bistable elements in a finite state machine and coordinates multiple legs stepping at the same time. The Ekeberg et al. simulation, however, was very useful to the development of SimROACH in that the stepping rules were able to be adapted to control the cockroach. A higher fidelity simulation of the stick insect nervous system is that presented by Silvia Daun-Gruhn in (Daun-Gruhn 2010; Daun-Gruhn and Tóth 2010). The simulation controls stepping in a full stick insect model with a network of Hodgkin-Huxley type neuron models configured to follow the rules from (Ekeberg, Blümel, and Büschges 2004). Instead of bistable units, this model is the first to simulate an independent dynamical CPG controlling every joint of the animal. Not only are these CPGs coupled within each leg via sensory influences, but they also communicate with those in other legs through sensory-gated connections between CPGs. All connections but a few are based on known neural connectivity in the stick insect. SimROACH, like the Daun- Gruhn model, has a unique CPG controlling the timing of each joint, coupled to the others only through shared sensory influences. In addition, legs are coordinated with one another by coupling the one CPG from each joint to the others. The Daun-Gruhn model, however, uses a more accurate neuron model. SimROACH uses a simpler model for 24

25 computational efficiency because it is intended to eventually run in real time on board a robot, whereas Daun-Gruhn s models are biological tools and do not need to simulate quickly. SimROACH, however, does have some advantages. For instance, it possesses both CPG timing control and sensory feedback magnitude control of muscles, rather than just CPG timing control as in the Daun-Gruhn model. This makes stepping more adaptable, something that is perhaps more important to a robot. In addition, the Daun- Gruhn model does not make low-level network changes to produce different motions like turning or reaching, although the authors of (Daun-Gruhn 2010) suggest that this would be possible given the network topology (something that SimROACH validates). Finally, SimROACH exists in a fully simulated dynamical environment, whereas mechanical dynamics are currently being integrated into the Daun-Gruhn model in pieces (Tóth, Knops, and Daun-Gruhn 2012). Chapter 2.2 Robot Models Although SimROACH can be classified as a biological model, its primary goal is to produce robust walking and smooth behavior transitions. Since it was developed for use in robotics, it should be compared to other legged robot control schemes. There have been many distributed control networks based in biology, so the goal is for SimROACH to offer advantages over these alternatives. Robot I (Beer et al. 1992) used a distributed neural system capable of generating a variety of gaits and stepping through a variety of environments. Robot I s control system was based on literature describing observed insect behavior (Beer et al. 1992). Robot II had a distributed control system with additional leg reflexes that made stepping extremely adaptable (Espenschied et al. 1996). These robots are exceptional walkers, and 25

26 SimROACH does not possess all of their capabilities. SimROACH currently cannot produce motions that are as adaptable as Robot II s because it lacks some of Robot s reflexes, but it uses actual pathways identified in insects to generate movement. Therefore, its success is encouraging in that future work implementing more details from animal nervous systems may lead to more animal like agility and robustness that surpasses current robot capabilities. The principles that made Robot I and Robot II successful have been used in other robots since then. Others have been developed that use either abstracted or more biological neural systems in addition to reflexes to coordinate movement. Tekken2 uses an artificial neural network to generate rhythms and reflexes for stable all-terrain quadruped walking (Kimura, Fukuoka, and Cohen 2007). Similarly, AMOS-WD06 uses an artificial neural network to coordinate walking, negotiate obstacles, and react to light stimulus (Manoonpong, Pasemann, and Florentin 2007). These robots show that even abstracted neural systems with the proper reflexes can successfully traverse difficult terrain. Further work in the Biologically Inspired Robotics Lab developed the controller as presented by Ekeberg et al. into a finite state machine called Sensory Coupled Action Switching Modules (SCASM) (Lewinger and Rutter 2006). This system was used to control a single leg on a static test stand and a pair of legs on a wheel set. It generated stepping by recreating much of the system from (Ekeberg, Blümel, and Büschges 2004). Bistable CPGs were coordinated by sensory information crossing thresholds, and were used to stimulate simulated muscles. Both platforms were capable of producing stepping motion, although improper transitions between FTi flexion and extension were observed. 26

27 Adding another sensory threshold resolved this issue, but increased the complexity of the system. Tests with SimROACH and a version of SimROACH built to mimic SCASM suggest that placing a rhythmic CPG at every joint improves stepping stability (See Chapter Stepping Robustness). In addition, a single leg robotic implementation of SimROACH did not produce any problems with stepping stability. SCASM was used to produce Leg Control Network (LegConNet), a system that could change gaits on the fly by reversing reflexes (Rutter et al. 2011). Higher command centers were used to modify the weights of various sensory pathways to the bistable CPG units at each joint. When applied instantaneously, the command to change gait successfully caused new behavior. If the sensory pathways or stepping rules were changed in a continuous way, the system was not so successful. The authors of (Rutter et al. 2011) hypothesized that implementing these rules as a neural simulation would resolve this problem, something that SimROACH validates. LegConNet was used to control all six legs in BILL-Ant-a (Biologically Inspired Legged Locomotion Ant autonomous) (Lewinger and Quinn 2010). Six such legs were coordinated by the most essential Cruse rules to generate coordinated stepping. The gait used by the robot depended on the initial position of each foot, but could exhibit a wave gait, tetrapod gait, and tripod gait as the speed was increased. LegConNet was modified to include reflexes to correct stepping, including an elevator reflex and a searching reflex. This allowed the robot to navigate obstructions. This system is perhaps the most similar to SimROACH in its goal to produce robust robotic behavior as strongly based on biology as possible. BILL-Ant-a used finite state machines, not a neural system, 27

28 but its ability to change speed and its stepping reflexes give it abilities that SimROACH does not yet possess. The stick insect rules from (Ekeberg, Blümel, and Büschges 2004) are being implemented in Octavio, a robot modeled after stick insect, to be used to study insect neurobiology (Von Twickel, Büschges, and Pasemann 2011; von Twickel et al. 2011). It has a distributed control system composed of sigmoidal activation neurons. Topologies based on experiments produced successful walking behavior, and then additional systems were developed with the help of genetic algorithms. These do not have connectivity based in experimental results, but produce behavior that is similar to that seen in the animal. These evolutionary algorithms included routines that could build structures in addition to tuning parameters, or focus on building local substructures. Algorithms like these may benefit SimROACH in the future. This related work has all influenced SimROACH. The biological data in the literature provided many rules and hypotheses about how motor systems are controlled. Computational models showed what kind of assumptions are acceptable and what level of detail is typically accounted for. Other robots highlight issues that need to be addressed in robotics and serve as performance benchmarks for this work. SimROACH has benefited from all of the work of these previous projects, and hopes to show that a more biomimetic control system can produce robust walking as well as smooth behavioral transitions. SimROACH can also be considered a biological model, so it is constructed of models used in computational neuroscience. These are detailed in the next chapter. 28

29 Chapter 3 Simulation Environments and Models Simulation was a crucial part of this work. Most of the actual model development was conducted in Animatlab, an open source neuromechanical simulator (Cofer et al. 2010). Mechanical models of cockroach appendages were developed in Blender (Stichting Blender Foundation, Amsterdam, Netherlands), an open source triangulated mesh editor. Neural models from Animatlab were reproduced in Matlab (Mathworks, Natick, Massachusetts) and XPP (G. Bard Ermentrout, University of Pittsburgh) when more rigorous mathematical analysis of particular portions of the system was needed. Chapter 3.1 Animatlab and Supplementary Environments Animatlab is a visual C++ Windows application developed by David Cofer of the University of Georgia as a part of his Ph.D. dissertation. It is a neuromechanical editor that simulates neural dynamics, mechanical dynamics, and interactions between them. A variety of neural models is available for use, and can be dragged and dropped into a connection network. Physics are simulated by the Vortex physics engine (CM Labs, Quebec, Canada). The two interact through muscles and sensors. Understanding how these models work is important for grasping the presented work, its capabilities, and its limitations. Chapter Mechanical Simulations and Models Animatlab can use triangulated meshes to simulate body inertia and collisions. The inertia properties of each body are the result of applying a uniform density to the mesh. This means the user can produce any shape and simulate its translational and rotational dynamics based on geometry. The legs of Blaberus discoidalis were dissected and measured for modeling. 29

30 Measurements were taken with the help of Al Pollack in the Ritzmann Lab. Female cockroaches judged to be normal in size and appearance were collected and sedated with carbon dioxide, decapitated, and frozen for approximately fifteen minutes. This allowed their internal fluids to congeal without reducing the body s flexibility. The bodies were pinned to petri dishes filled with silicone and examined under a microscope. Digital photography and markup were used to save and annotate images of the decapitated cockroach and its amputated legs in various configurations. Examples of these can be seen in Figure 2. These images were used to produce meshes of each of the legs segments. The distal segments (tarsus, tibia, femur, trochanter) were relatively straightforward to model since it is usually clear where one ends and the next begins. Modeling the coxa, however, was quite complicated. The front leg possesses three degrees of freedom between its coxa and thorax, and the other legs possess two. It is very hard to tell where the axis of each joint lies, its orientation, and its relative order of proximity to the thorax. Rotation matrices of bodies are order specific, which means that stacking the joints in the wrong order produces motions that are not comparable to what might be seen in the organism. 30

31 Figure 2 Microscopic photographs of severed cockroach legs (left) for comparison to triangulated meshes (right) used in simulation. 31

32 The joints were assigned in the order Thorax-coxa (ThC) 2, ThC1, ThC3, Coxatrochanter (CTr), Trochanter-femur (TrF), and Femur-tibia (FTi) distal from the thorax. The TrF joint is locked in the front leg, and the ThC3 joints are not included in the middle and hind legs (Bender, Simpson, and Ritzmann 2010). The Tibia-tarsus joint is locked in SimROACH. The tarsus is a compliant, actuated member that aids in proper foot placement and holding. This feature was omitted in favor of modeling simplicity and lower simulation run time (fewer actuators/dynamics to simulate). Cockroach posture reduces the need for foot pressure control, compared to more upright quadrupeds and bipeds that use foot pressure to maintain balance (Chou et al. 2009; Meyer, Oddsson, and De Luca 2004). Figure 2 shows the actuated joints labeled on each leg. The name and orientation of each joint are labeled. Joint angle measurements were recorded according to (Bender, Simpson, and Ritzmann 2010). To summarize, extension of each joint is a positive angle. The CTr and FTi joints would be 180 degrees if extended into straight lines, and flexing the joints reduces the angle. The TrF joint measures the angle between the plane defined by the coxa and the plane defined by the femur and tibia. The ThC1 joint is measured between the outside edge of the coxa and a vertical line pointing below the thorax. The ThC2 angle is measured between the axis of the ThC1 joint and a line horizontal to the thorax. Finally, the ThC3 joint is the rotation of the coxa about its own outside edge. These conventions will be used throughout the rest of this document. All joints are modeled as one degree of freedom pin joints. Mechanical stops exist at the limits as determined by dissections. The limits are modeled as damped spring buffers that are only active past the specified angle. 32

33 Chapter Muscle Model All of the muscles in SimROACH are a version of the linear Hill muscle model, which is built into Animatlab. The Hill muscle model is the result of force-displacement experiments performed on muscle tissue. The collected data were used to produce a mechanical equivalent of springs and dampers. That system can be seen in Figure 3. There are three passive components in the model: a series spring, a parallel spring, and a parallel damper. The Figure 3 Mechanical equivalent model to the linear Hill muscle model (top). Length-tension relationship that limits tension output of the muscle (bottom left). Stimulus-tension relationship that determines the activation of the muscle (bottom right). Taken from (Shadmehr and Arbib 1992), accessed on Animatlab.com. series spring s stiffness controls the strength of the muscle, the parallel spring s stiffness determines the amount of output energy that is stored per cycle, and the damper keeps the muscle from contracting too quickly. The tension in the muscle develops according to the differential equation: ( ( ) ) Where is the tension the muscle applies,,, and are the series stiffness, parallel stiffness, and damping, respectively, is the length of the muscle, and is the activation level of the muscle. 33

34 The activation level is determined by the voltage of the motor neuron pools in this model. Each muscle has a sigmoidal transfer function between the voltage of its motor neuron and the activation of the muscle. The activation level of the muscle is also a function of its length. Each muscle has a quadratic relationship relating its ability to apply tension to its length. It is modeled as parabolic because it is a close fit to data collected in frogs, cats, and humans. This particular model has also been used to model insect muscles (Cofer 2009). The heuristic for such tuning is that the muscle can apply full tension at its resting length and no tension at 133% and 67% of that length (Rassier, MacIntosh, and Herzog 1999). These three points define a parabola. Figure 3 shows plots of these two additional relationships. These relationships require proper tuning to produce useful tension. The lengthtension relationship in particular is difficult to manage. If one muscle of an antagonistic pair applies too much tension, it may pull the other muscle to a length at which it cannot apply tension and pull the limb back. The time dependent generation of muscle tension also makes the system more difficult to understand. All of the values in SimROACH have been set by examining the steady state tension of the muscles when each joint is at its extreme positions. This was then modified based on actual performance. Chapter Neuron and Synapse Models Many neuron models exist, and each highlights certain behaviors of neurons. The controls of SimROACH were constructed from neurons to more directly mimic findings in insects and hopefully produce more robust walking. However, if this system is to control a robot in real time, the neuron models must be simple and relatively easy to simulate. Therefore the integrate and fire (IF) model was selected for this work. It is perhaps the simplest model that simulates membrane dynamics rather than more 34

35 abstracted quantities such as firing frequency. This means neural data from SimROACH can potentially be directly compared to recordings in insects. Both spiking and nonspiking versions of this model exist, despite the contradictory nonspiking integrate and fire name. A quantitative analysis of the behavior and simulation speeds of different spiking neuron models is available in (Izhikevich 2004). Chapter Spiking Integrate and Fire Model Only the neurons that code for loading, unloading, and decreasing load are spiking neurons in SimROACH. This choice was made because it made for more direct comparisons between recordings from campaniform sensilla and SimROACH. This is also desirable because the spiking model, unlike the nonspiking model, can accommodate its spiking threshold and facilitate its synapses. This means properties change over time, encoding for derivatives or integrals of stimuli. These features were used to more closely mimic recordings from animals that reveal detection of load and the time derivative of load. Despite being simple, the IF model captures the basic characteristics of some neurons in a mathematically efficient way. The spiking IF model is a leaky integrator that will depolarize above a certain voltage threshold, then hyperpolarize below the resting voltage. The rate at which it spikes depends on the current stimulus applied or the rate at which it changes. These qualitative descriptions are consistent with the observations of Hodgkin and Huxley, who produced the first detailed description of neural behavior (Hodgkin, Huxley, and Katz 1952). The manner in which the IF model achieves these features is much simpler than the Hodgkin Huxley (HH) equations. The spiking neuron can receive current input from artificial stimuli (, synapses (, and what is called the after-hyperpolarization (AHP) current. The IF 35

36 only keeps track of the charge of the current, not particular ions like more sophisticated models. Therefore a spike is recorded when the voltage crosses the spiking threshold. Instead of the voltage increasing quickly due to ion channel dynamics, a cosmetic spike is applied and the AHP current is activated. Because the spike is cosmetic, the calculated membrane voltage does not change rapidly. This is advantageous because a simple integration scheme like forward Euler can be used to simulate membrane dynamics. The AHP current is applied by establishing an exponentially decaying conductance between the membrane voltage and the AHP voltage ( ), which hyperpolarizes the neuron and ensures there is time between spikes. This is different from the canonical integrate and fire neuron model, which sets the membrane voltage to a specified value in the time step after a spike. Applying the AHP current instead more closely mimics the shape of hyperpolarization seen after a neuron spikes and produces more flexible neural behavior. The IF membrane potential changes according to: where [( ( ) ] { The neuron leaks according to the membrane conductance ( between the membrane voltage ( and the resting voltage (. The AHP current is applied after, which is the time of the last spike, and decays with rate. Spikes are recorded when the voltage crosses the spiking threshold, which accommodates according to: ( ( ( 36

37 The spiking threshold accommodates from its initial value an amount proportional to the amount that the membrane voltage changes from its resting value. This proportionality is set by, and the threshold changes with time constant. Manipulating and can make the neuron emulate class III excitability, that is, activity that corresponds to the rate of change of the stimulus. Figure 4 shows the step response of the spiking neuron model with different values for threshold accommodation. Note how choosing an accommodation value between zero and one produces spiking frequency that is sensitive to both the stimulation level and the rate of stimulus. Figure 4 Plots showing the step response of neurons with different spiking threshold accommodation values. An accommodation of 0 makes the spiking frequency a function of stimulus current (top). An accommodation of 1 makes the spiking frequency a function of the derivative of the stimulus current (middle). A value between those will produce a response that is a combination of the two (bottom). The stimulus current (green) is 10 na in every picture. by: The synaptic current, { (, is defined can facilitate by: 37

38 ( ( ) is the static voltage of the synapse, is the conductance of the synapse at a given instant, is the user-specified synaptic strength, which decays at rate. This means each spike injects a decaying (roughly) exponential current into the postsynaptic neuron. The strength of the synapse can change from spike to spike according to, the synaptic facilitation. If, then each synapse s conductance is Figure 5 Postsynaptic response of a neuron coupled to a tonically firing neuron via a synapse with a facilitation of 1 (top) and 0.5 (bottom). Facilitation values less than one cause the synapse to decay at a rate that is a function of presynaptic spiking frequency. 38

39 lower than the maximum conductance by a factor of (. As the spikes continue, the cumulative effect of each is maintained, requiring that the synapse keep track of when the presynaptic neuron spikes for the duration of the simulation. This is used to make connections between neurons decay or strengthen. Figure 5 shows the postsynaptic response of a neuron coupled to a tonically firing neuron by a spiking synapse with different facilitation values. Much of this behavior can be compared to circuit components. A parallel resistorcapacitor circuit performs leaky integration when a voltage is applied across it. A spike is similar to a transistor becoming excited when the base-emitter voltage surpasses a threshold. These analogies may be useful to engineers in building models. Chapter Nonspiking Integrate and Fire Model The nonspiking IF model simulates single nonspiking neurons or the sum activity of populations of spiking neurons. This model was used for two reasons. First, many neurons in the stick insect motor control system are nonspiking, specifically those that make up CPGs, interneurons that regulate muscle activity, and interneurons from sensory inputs (Büschges, Kittmann, and Schmitz 1994; Büschges 1995; Büschges et al. 2004). Second, the bandwidth of a spiking neuron is fundamentally limited by its maximum firing rate (Trappenberg 2009). Single spiking neurons that encode sensory information can only update other parts of the system at its maximum firing rate or lower, which is lower than that at which their dynamics are simulated. Using multiple redundant spiking neurons would solve this problem, but this would slow simulation time. Nonspiking neuron models can communicate at the same rate at which they are simulated, meaning that more precise timing and better coordination can be obtained. 39

40 The nonspiking neuron model is a leaky integrator that integrates its input currents according to its membrane resistance and capacitance. This model can receive inputs from synapses ( or other stimuli (. It can also optionally include calcium channels, which add another current term to be integrated. In addition, two additional state variables must also be solved to calculate the calcium current. This current is due to a gated conductance between the calcium voltage (200 mv) and the resting voltage of the neuron. The dynamical equations are: where and and [( ( ] ( ( ( ( ( ( ( ( ) ( ( are voltage gated variables. They cause overshoot in the neuron s membrane voltage when stimulated by an external current. Figure 6 shows the step response of the nonspiking neuron with and without the calcium currents. 40

41 The nonspiking model communicates with other neurons via nonspiking synapses. The amount of current injected into the postsynaptic neuron ( is determined by the conductance between the postsynaptic neuron s membrane voltage ( and the voltage of the synapse (. The synapse s voltage is a static value. The conductance of the synapse ( is proportional to the presynaptic ( voltage s distance between a low and high threshold ( and, respectively). This can be mathematically described as: Figure 6 Plots that show the nonspiking neuron s step response without calcium currents (top), and with calcium currents during activation (bottom) and deactivation (bottom). All stimuli (green) have a magnitude of 10 na. 41

42 ( ) { This is much simpler than the spiking synapse model because it does not record presynaptic history. However, it is less flexible because it cannot facilitate. The synaptic strength cannot decay over time, meaning that phenomena that rely on synaptic facilitation cannot be replicated with this model. A decaying nonspiking synapse was written for use in the robotic leg controller: ( ) { ( ( ( ) ( ( The synapse conductance decays with a time constant of after the presynaptic neuron crosses from below to above the conductance threshold of the synapse. This is more computationally efficient than using the spiking neurons with facilitating synapses, which must keep track of presynaptic spikes. 42

43 Chapter 4 Robust Robotic Stepping SimROACH uses networks of these neuron and synapse models to excite muscles and generate movement. Each leg possesses its own distributed control network, and each uses the same types of sensory information (loading and joint angles) and actuators (simulated muscles). However, they are structured differently to suit the role of each leg, so they exhibit different motions. Each leg has a unique central pattern generator (CPG) governing each joint. These CPGs are only coupled through sensory signals, not direct connections among them. They act as gating clocks, changing the gain on muscular positional control units in an oscillatory fashion, producing periodic motion. Since the CPGs do not directly stimulate the motor neurons, the timing of the CPGs or the range of motion of the joints can be altered while affecting the other minimally. Comparisons with modified versions of SimROACH show that sensory coupled, dynamical CPGs make stepping more robust to perturbations than alternatives. Chapter 4.1 Stepping Rules Locomotion and the sensory cues that coordinate stepping have been the focus of research for some time (Bucher et al. 2003; Akay et al. 2001; Ridgel et al. 1999; Zill, Schmitz, and Büschges 2004; Zill, Keller, and Duke 2009; Zill et al. 2011). By examining which muscles are activated or deactivated when certain sensors are excited, sets of cause and effect rules have been uncovered and successfully modeled for stepping in the middle leg of the stick insect (Akay et al. 2004; Ekeberg, Blümel, and Büschges 2004). These rules are typically described as reflexes, although the actual interactions are more complicated. 43

44 These rules have been adapted for use in robotics, most notably in LegConNet (Rutter et al. 2011). SimROACH uses the sensory pathways outlined in LegConNet, but replaces the actuation of the ThC1 joint during walking with TrF actuation because of data presented in (Bender, Simpson, and Ritzmann 2010). In LegConNet, the ThC1 joint loaded and unloaded the leg while the CTr joint provided thrust (Rutter et al. 2011). In SimROACH the CTr and TrF joints fill slightly different roles at different parts of the stepping phase. Extending the CTr joint loads the leg, after which the TrF flexes (lowers). At the end of stance the TrF extends to raise the tarsus, unloading the leg and causing the CTr to flex in the return stroke. SimROACH actuates both extension and flexion of the TrF joint although in Blaberus, the TrF can only be actively raised by the reductor femoris and passively lowered by tendons (Carbonell 1947). Despite this difference, SimROACH s motion is more like that of a cockroach than with LegConNet because it Sensory Phenomenon Leg loaded Fully EXT/Fully DEP Leg load decreasing Leg unloaded Fully FLX Resulting Phase Changes FTi: Flex -> Extend TrF: Extend -> Flex TrF: Flex -> Extend CTr: Extend -> Flex FTi: Extend -> Flex CTr: Flex -> Extend Figure 7 Table of sensory triggers used to generate forward walking in the middle leg. actuates the same joints as Blaberus with similar timing. The kinematics of SimROACH are directly compared to Blaberus in Chapter Comparison to Blaberus. SimROACH s stepping rules, adapted from LegConNet, can be seen in Figure 7. Starting with the leg in swing, the CTr joint extends, loading the leg. This causes the FTi joint to extend. At the same time, the TrF flexes. Once the CTr or FTi joint have reached a critical angle, the TrF joint is extended to reduce the load on the leg. When the load is 44

45 decreasing, the CTr joint flexes, fully unloading the leg. At this point the FTi joint flexes, which triggers CTr depression to reload the leg. These rules were adapted to the other legs, which perform different tasks and exhibit different motion than the middle leg. The hind leg follows the same stepping rules as the middle leg while walking forward, but with joints sweeping different angles. The front leg exhibits a reaching type motion that includes actuation of the complicated three degree of freedom ThC joint as well as the CTr and FTi joints. The TrF joint is fused in the front leg of Blaberus (Bender, Simpson, and Ritzmann 2010) and is locked in SimROACH. Sensory Phenomenon Leg loaded Fully FLX/Fully DEP Leg load decreasing Leg unloaded Fully EXT Resulting Phase Changes FTi: Extend -> Flex ThC1: Flex -> Extend ThC3: Extend -> Flex CTr: Extend -> Flex ThC3: Flex -> Extend FTi: FLX -> EXT ThC1: Extend -> Flex CTr: Flex -> Extend Figure 8 Table of sensory triggers used to generate forward walking in the front leg. The front leg of the cockroach has not been studied to produce a set of stepping rules. Therefore the rules in the middle leg were modified to produce the coordination seen in the front leg of the cockroach. The primary difference is that the ThC1 joint is used to generate thrust and support, much like the CTr joint in the middle leg. Other differences are that the FTi joint extends when unloaded instead of flexing as in the middle leg, and the ThC3 joint, not the TrF, reduces the load to signal for swing to begin. These rules are summarized in Figure 8. This list of rules produces stepping that is coordinated and qualitatively resembles that observed in Blaberus. 45

46 Chapter 4.2 Implementation of Stepping Rules The stepping rules for each leg were implemented via a simulated network of neural population models. Even though each leg s network is different, the structure is the same in each one. At the top of the structure are various sensory influences, such as angle states and loading information. This information then passes through a network of interneurons that are excited or inhibited based on which gait is active. These neurons synapse onto interneurons in the CPG, and their excitation strengthens the reciprocal inhibition of the half-centers. The CPGs then change the gain on muscle positional controls, causing one antagonistic muscle or the other to contract. This causes motion that affects the sensory neurons. Chapter Sensory Information The sensory information used in SimROACH is not identical to that in the animal, but was designed to be analogous and provide the same types of information. The primary sensory influences used by these stepping rules are joint angles and loading information (Akay et al. 2004; Ekeberg, Blümel, and Büschges 2004). In stick insects information about FTi flexion is provided by the femoral chordotonal organ (fco), which codes for organ length and velocity. Experiments in which the organ was directly mechanically stimulated show that fco output affects motor activity around the CTr joint much like a switch, producing the strongest output at the extreme angles of normal FTi motion during walking (Bucher et al. 2003). This suggests that the fco s effect on interjoint coordination is step-like, even if the organ does code for stretch over the entire range of its motion. Instead of simulating the fco, SimROACH uses the actual joint angle as measured by the physics engine. The joint angle is transduced to a current by a linear 46

47 transfer function, which is injected into a neuron to code for FTi position. For consistency, the slope of all joint angle transductions is always +/- 10 na per radian. The line is then shifted up or down according to the desired mean angle. Figure 9 shows the angle, injected current, and neuron voltage for FTi extension of the middle leg when all other joints are locked and the FTi joint was given an arbitrary mechanical stimulus. The time constant of the position neurons was short (5 ms) so the integration lag between angle state and the neuron s state was negligible, as seen in the plots. To obtain performance like that observed in (Bucher et al. 2003), the FTi position neurons communicated with the rest of the circuit through synapses with relatively high thresholds for conductance (-47 mv). This produced behavior like that seen in stick insects preparations in which stretching the fco beyond a certain point changed the flexion or extension state of the CTr joint. Plots from the model demonstrating this can be seen in Figure 10. Figure 9 Plots showing the measured angle (top), transduced current (middle), and resulting voltage of a neuron coding for the joint s rotation (bottom). The gray lines show that the integration lag between the current and the neuron s voltage is virtually nonexistent. 47

48 Figure 10 Plots showing the measured angle (top) and the current injected into the rest of the system to signal that an extreme position has been reached (bottom). There is no output until the joint reaches a certain limit, at which point it rapidly increases. The gray lines show what FTi angles signal to the rest of the network. In addition to FTi excursion, stick insects and cockroaches use loading information to coordinate stepping (Akay et al. 2004; Zill, Schmitz, and Büschges 2004). Loading information breaks stepping up into two basic motions: stance, which supports and propels the animal while in contact with the ground, and swing, when the leg is being returned to an anterior position. Insects distinguish these states by measuring the strain of their legs through campaniform sensilla (CS). CS detect load amplitude, rate of load, whether the rate is positive or negative, and the lack of load (Zill, Schmitz, and Büschges 2004). SimROACH can detect the same signals. The population of CS on the trochanter are the most important for maintaining coordinated stepping (Akay et al. 2004). Because of modeling constraints, SimROACH does not calculate the strain on the trochanter, but instead uses the magnitude of the normal force acting on the tarsus to detect load. This signal is transduced to a current in a linear fashion and injected into the Load neuron, shown in Figure 11 (Neuron A). This neuron is a spiking neuron with a low spiking threshold (2 mv above rest) such that its firing frequency is a function of the load. In addition, its spiking threshold accommodates, making the firing frequency also depend on the rate of load increase. The 48

49 Figure 11 (Top) Unpublished results from the Zill lab showing how some populations in the cockroach respond to increasing load while others respond to decreasing load. (Bottom) Picture of network that processing loading information. Neuron A turns the signal D, the magnitude of the load on the foot, into a firing frequency (E.). Neuron B is inhibited by neuron A, and will fire when the load goes away. Neuron C is stimulated by Neuron B with a decaying synapse, so its activity decays rapidly when the load ends (F.). Load neuron strongly inhibits the Unload neuron (Figure 11 B), which has a tonic drive to make it fire when Load becomes less active at the end of stance. The Unload neuron excites the Load Decreasing neuron (Figure 11 C) with a facilitating synapse that decays over time, causing it to fire briefly as the load decreases to nothing. Figure 11 compares data from SimROACH to neural recordings from cockroaches. SimROACH can produce signals that encode leg loading similarly to cockroaches despite not measuring load in the same way. The robotic implementation of a single leg from SimROACH uses a strain gage on the trochanter to detect load, much 49

50 like CS. The robot was able to walk normally with this substitution, suggesting that detecting load from the tarsus is an acceptable substitution to help maintain stepping coordination. However, insects use CS signals from other locations on the leg to monitor muscle tension and leg orientation (Akay et al. 2004; Noah et al. 2004; Watson, Ritzmann, and Pollack 2002), something that could benefit SimROACH in the future. Chapter Sensory Interneurons and Reflex Reversal Sensory information in SimROACH affects CPG timing through a layer of interneurons. Different gaits in insects are generated by changing how only one or two sensory inputs are processed, changing the order in which joints are flexed or extended (Akay et al. 2007). SimROACH s primary goal is to produce flexible robotic behavior in this fashion, so this effect was replicated. Researchers in stick insect neurobiology have hypothesized that such changes could occur by changing the weight of parallel sensory pathways via descending commands (Akay et al. 2007). Such a method has been successfully implemented in both computational models (Daun-Gruhn 2010) and biologically inspired robots (Rutter et al. 2011). The specific rules that change in SimROACH s different behaviors will be discussed in Chapter 5 Smooth Low Level Transitions, but the mechanism that accomplishes this will be described here. Figure 12 Schematic of how a reflex reversal can be executed in this model. The Gait neuron can affect interneurons that relay sensory information, changing which neurons are affected by which sensors. SimROACH reverses reflexes by changing the excitation of interneurons 50

51 that conduct a sensory signal to a CPG or multiple CPGs. Figure 12 shows a simple circuit that illustrates this mechanism. Path A and Path B s membrane voltages will reflect the signal produced by Sensory Signal. When the Gait neuron is not excited, it does not affect these interneurons. Synapse 3 between Path A and the Flex neuron is configured such that in this case Path A relays information from Sensory Signal to the Flex neuron. Synapse 4 between Path B and the Extend neuron has a high conduction threshold, such that the voltage on Path B cannot affect the Extend neuron. When the Gait neuron is excited, however, the voltage of Path A is suppressed such that its signal remains below the conduction threshold for synapse 3. Conversely Path B is excited up to the conduction threshold of synapse 4, enabling it to conduct information from Sensory Signal to the Extend neuron. It is desirable to make smooth transitions between gaits rather than discontinuously changing behavior. The time constant of Gait neurons in this model, whether at the low or intermediate level, was 500 ms unless otherwise noted. This corresponds to reaching 99.7% of full excitation after 1500 ms of a step input, consistent with the observation that cockroaches change between forward walking and turning over the course of about 1500 ms (Brown 2011). Chapter Central Pattern Generators 51

52 The CPG model used in SimROACH is a nonspiking half-center oscillator. It is composed of four neurons, two that generate the rhythm and two that serve as interneurons between the rhythm generators. A schematic of this structure can be seen in Figure 13. Neurons 2 and 4 are the interneurons, and have properties similar to most others in this simulation. Figure 13 Schematic of a CPG used in this model. Neurons 1 and 3 are the half-centers, communicating through interneurons 2 and 4. Neurons 1 and 3 differ in that they utilize the optional calcium channels, which provide additional dynamics that lead to positive feedback in the membrane s response to input current as discussed in Chapter Nonspiking Integrate and Fire Model. Among all four neurons, each CPG has eight state variables: two neurons each with a membrane voltage and two each with a membrane voltage, a calcium channel activation state, and a calcium channel deactivation state. Figure 14 Voltage of one half-center of a CPG during oscillation. The oscillation reaches steady state after about 1 second. 52

53 The CPGs naturally oscillate without input, forming a limit cycle with all eight state variables. An example of the output of one of the half-centers is shown in Figure 14. Numerical simulations with XPP show that this limit cycle possesses exactly one Figure 15 Voltage of one half-center of a CPG during normal activity (top) and when the interneurons are inhibited by a current of na. 53

54 equilibrium point, which has four real negative, two negative complex, and two positive complex eigenvalues. The strongest set is, meaning the equilibrium point is unstable. The observed oscillation tells us that a limit cycle forms, so this is the only Figure 16 Voltage of one half-center of a CPG when the presynaptic neuron is strongly hyperpolarized at different points in the phase. A strong enough stimulus will reset the phase of the CPG at any point of the phase. The bars along the bottom show the bursting period before the stimulus was applied. A red circle is drawn around the perturbation, after which a normal looking period of activity is observed. 54

55 stable configuration of the system with this set of parameters and inputs. This CPG model replicates experiments performed on physiological CPGs in organisms. One of the defining characteristics of a CPG is that a strong hyperpolarization of one neuron will cause the oscillation to reset, with the inhibited neuron firing a full burst when released from inhibition. Experiments with this CPG model show that strongly hyperpolarizing one of the interneurons between the half-centers resets the oscillation regardless of the phase of stimulation, consistent with results from the heartbeat of the leech (Arbas and Calabrese 1987) and motor systems of stick insects (Büschges 1995). Figure 16 shows the voltage plots of one half-center when the presynaptic neuron is excited, hyperpolarizing the half-center (5 na pulse 100 ms long). The hyperpolarization swiftly ends the current phase of oscillation and causes an intact, full period of activity immediately following, shifting the phase of oscillation. This CPG model is also capable of producing a wide range of frequencies. Without external input, the CPG will oscillate at a frequency determined by the rates of calcium channel activation and deactivation in neurons 1 and 3. Inhibiting interneurons 2 and 4 weakens the inhibition between neurons 1 and 3, decreasing the CPG s oscillation rate as seen in stick insects (Büschges et al. 2004). Figure 15 shows output of the CPG both without external input to the interneurons and with na tonic drive. Such stimulus reduces the period by 80%. This means this CPG can potentially be used for a variety of gait speeds, although muscle properties and sensory information also influence the rate of stepping (Pearson 1993; Mackay-lyons 2002). Chapter Muscle Control Units SimROACH uses muscles as actuators. Muscles were chosen for two main reasons: compliance and biological consistency. Muscles are compliant actuators, and can 55

56 be modeled as a system of springs and dampers as described in Chapter Muscle Model. Complaint actuators interest engineers because they passively reject perturbation and require less precise control (Jindrich and Full 2002; Loeb, Brown, and Cheng 1999; Kingsley, Quinn, and Ritzmann 2006). Not needing to respond to every single disruption while stepping reduces both computational resources needed for control and energy consumption during stepping. The second reason SimROACH uses muscles is to more readily couple the neural control system with the motor output; they have evolved to work together. CPGs in stick insects do not excite muscles, but rather inhibit them from an excited state (Büschges et al. 2004). It is also known that sensory influences can affect muscle activation without affecting CPG timing (Zill, Schmitz, and Büschges 2004; Akay et al. 2001; Akay et al. 2004; Büschges 1995). In addition, nonspiking neurons modulate motor output based on CPG activity and sensory feedback (Büschges 1995). SimROACH generates controlled muscle tensions through an engineered system based on what is known about insect muscle control. Sasha Zill guided the development of this system. Each joint has one CPG that alternates between two antagonistic muscles. Each joint also has two equilibrium positions, fully flexed and fully extended, that are set as static values. Positional control systems built from neurons exist for each position, and both are active at all times. Damping is present in the muscles, so the system can be considered positional-derivative (PD) control. The CPGs modify the gain on each, causing one equilibrium position to be more attractive than the other. The error between the desired angle and the current angle is calculated for both the fully flexed and the fully extended position. These signals, one for each muscle, are then gated by the appropriate 56

57 CPG half-center. These positional errors are used to generate a force, which can lead to complicated dynamics. To better understand these dynamics, an abstracted version of this system was analyzed using XPP. The muscle control unit was initially simulated without CPGs by the following dynamical system: ( ( Where x is the position (angle), x 1 and x 2 are the desired flexion and extension equilibrium points, k 1 and k 2 are gain values on each positional error, and b is damping due to muscle dynamics. In this form, the system will move toward whichever term provides the highest drive. For instance, if x 1 equals -x 2 but k 1 is five times k 2, the system s attraction to x 1 will be five times as high and the position will settle at. An example of such behavior can be seen in Figure 17(B). The effect of CPGs can be examined by adding a sinusoidal component to the gain terms for each equilibrium point. If the gains are allowed to oscillate between 0 and 1 and are 180 degrees out of phase, the new dynamical system looks like: ( ( ) ( ( ( ) ( 57

58 A B C D E F Figure 17 Behavior of the abstracted muscle pair system. If both stiffnesses (1,1) and set points (1,-1) are identical and the sin terms are removed, any initial condition will drive the system toward an angle between the two set points (A). Changing the stiffness of one set point (k1=5*k2), the system will move toward that set point (B). If the sin terms are included, the system can be tuned to oscillate with the desired frequency and amplitude (C). If the frequency is changed, the amplitude is decreased, since this system is a filter (D). The desired amplitude can be regained by increasing the stiffness of both set points (E). This is not something the current model is capable of. Keeping the original stiffnesses and instead increasing the set points will produce qualitatively similar behavior (F). This is the approach that the current model uses to increase the stiffness of a joint. where omega is the frequency of the CPG. Using XPP, parameters in the system were varied and behavior changes were documented. Figure 17 shows some of these results. Initially, omega was set to 1, k 1 and k 2 were set to 1, and x 1 and x 2 were set to 1 and -1, respectively. In this configuration, the system oscillates at the desired frequency and 58

59 amplitude (Figure 17 (C)). If omega is increased to 2, however, the system no longer achieves the desired range of motion due to the transmissibility of the system (Figure 17 (D)). Increasing the stiffness of each positional controller, k, helps regain the desired amplitude by changing the band of the pass filter (Figure 17 (E)). Achieving this in SimROACH would require increasing the stiffness of the series element springs, something that the model cannot do. Cockroaches accelerate running speed by activating fast muscle fibers in each joint as it changes direction (Watson and Ritzmann 1998). SimROACH could be modified to do this, but it does not possess different muscle fiber types. Instead the equilibrium points are set further away from one another, producing larger errors and therefore larger forces (Figure 17 (F)). SimROACH uses this method to Figure 18 Plots of joint angles (blue) and extreme position neuron voltages (green). Note that flexion can be changed independently of extension (top) and vice versa (middle). They can also be changed together to change the mean angle (bottom). 59

60 increase reaction forces during stance and change kinematics during different gaits. This control method was tested in SimROACH in a reduced preparation. All joints on the mesothoracic leg were locked except the FTi joint. All sensory influences were removed from the FTi CPG, allowing it to oscillate at the CPG s natural rate. The desired position was then changed as the joint oscillated, changing the range of motion. Plots in Figure 18 show these changes. As long as the muscle properties are configured properly, one can change the desired flexed position and not affect the range of extension (Figure 18 (A)), and vice versa (Figure 18 (B)). In addition to increasing or reducing the range of motion of a joint, the mean position can also be shifted (Figure 18 (C)). These effects have been observed in some cockroach gait changes (Brown 2011). The structure of the muscle control subsystem is shown in Figure 19. This particular image is from the CTr joint of the middle leg, so it can depress (DEP) or levate (LEV). In the yellow box is the feedback control for the DEP muscle. The voltage of the DEP FB (depressor feedback) neuron codes for the angle of the CTr joint as discussed in Chapter Sensory Information. The Extreme POS Figure 19 Schematic of the CPG and muscle control unit for the CTr joint. The CPG (red) only inhibits the Inter Pos neurons, which are interneurons between the error feedback control for each muscle and its motor neuron. LEV (extreme position levation) and Extreme POS DEP (extreme position 60

61 depression) neurons code for the maximum angle for each direction. These can be changed by sensory influences or descending commands. The elev (levation error) and edep (depressor error) neurons are comparators between the actual angle and the extreme angles. These stimulate interneurons Inter Pos DEP (interneuron position depressor) and Inter Pos LEV (interneuron position levator), which receive inhibitory input from the CPG. The CPG reduces gain of the positional control units in a periodic manner, generating oscillatory motion. A bias was added to the error of both sides (+10 mv) to maintain a baseline level of stiffness. Chapter 4.3 Networks and Their Function Three networks were developed according to the structure described in Chapter 4.2 Implementation of Stepping Rules, one for each leg of the cockroach. The front leg is capable of the most agile motion including reaching motions by actuating its proximal joints, and brakes the animal s forward motion. The middle legs provide support and braking during the first half of stance and thrust in the second half. The hind legs provide support and most of the thrust during walking, and follow the same stepping rules as the middle leg (Full, Blickhan, and Ting 1991). Each leg has five actuated degrees of freedom. The rear two legs actuate the ThC2, ThC1, CTr, TrF, and FTi joints, while the front leg actuates the ThC3, ThC2, ThC1, CTr, and FTi joints. Figure 2 shows a photograph of a cockroach leg, as well as screenshots of each leg from the simulator with the degrees of freedom labeled. Chapter Middle Leg Network 61

62 Figure 20 Control network for the middle leg of the cockroach with no particular gait active (top) and with the forward walking gait active (bottom). The inactive pathways have reduced fill. The middle leg control network is shown in Figure 20. The neurons are color coded and arranged in a hierarchical fashion to make their purposes clear. The light blue neurons along the top are sensory neurons. These include information about loading and proprioception. The details can be found in Chapter Sensory Information. If this network could not produce different gaits, these neurons would directly synapse to the 62

63 top layer of the CPGs, shown in red. Instead these neurons synapse onto the dark blue sensory interneurons, which can be excited or inhibited by the green gait neurons along the sides. These neurons produce reflex reversals. The details are explained in Chapter Sensory Interneurons and Reflex Reversal. These neurons influence the interneurons of the CPGs (top red), which control the strength of inhibition, and thus frequency of the CPG half-centers (bottom red). The CPGs change the gain on muscle control units as discussed in Chapter Muscle Control Units. This network can be overwhelming to look at, so Figure 20 also shows the network with reduced fill on the neurons that are not used for walking. The rules that this network encapsulates are listed in Figure 7. In addition, the properties of every neuron and synapse are listed in Appendix A Network Topologies. The network was constructed from neurons and synapses with as few unique parameter sets as possible to simplify recreation on board a robot. More attention to detail might improve performance, something discussed in Chapter 7 Conclusions and Future Work. Chapter Front Leg Network The front leg network, shown in Figure 21, is noticeably larger than the middle leg network because of the larger group of dark blue sensory interneurons. Not only does the front leg utilize more joints than the other legs in most gaits, but it is also changes its behavior the most between them. The stepping rules used for forward walking are shown in Figure 8, and Figure 21 includes a picture of the network highlighting pathways that are active during forward walking. The color scheme of the neurons is the same as listed in Chapter Middle Leg Network, and a list of every neuron and synapse and its properties is provided in Appendix A Network Topologies. 63

64 Figure 21 Control network for the front leg of the cockroach with no particular gait active (top) and with the forward walking gait active (bottom). The inactive pathways have reduced fill. Chapter Hind Leg Network The network that generates stepping in the hind leg is shown in Figure 22. As noted previously, the hind leg does not change its behavior when Blaberus turns, so this network contains no dark blue sensory interneurons for reversing reflexes. As with the 64

65 Figure 22 Control network for the hind leg of the cockroach. This network is much smaller than the others because no reflex reversals take place. other legs, a table of all of the properties of the neurons and synapse is available in Appendix A Network Topologies. This network differs from the others in that the CTr and FTi joints exchange proprioceptive information in order to coordinate their motion. They should extend and flex at the same angles, so the difference between the angles is computed to determine how much the CTr joint is over flexed or over extended. The error is then used to stimulate the motor neuron for flexion or extension, respectively, of the FTi joint. This method was developed because there is no evidence that CPGs can influence one another directly to maintain coordination (Büschges, Schmitz, and Bässler 1995). However it is not perfect, and could be improved by using an actual controller to maintain this relative angle rather than a simple comparator. An engineering solution would be to coordinate the joints by sharing the same CPG. However such a decision would directly contradict the secondary goal of making as 65

66 accurate a biological model as possible, and could potentially limit SimROACH s behavioral flexibility as more functionality is added in the future. Chapter 4.4 Stepping Results The performance of such a biomimetic system can be judged in two ways: similarity to the animal and general engineering effectiveness. The intent of SimROACH is to make robots walk more robustly, so the primary metric was to produce effective stepping. However, cockroaches are some of the most agile hexapods and a robot that could move like a cockroach would be extremely effective. Further, direct comparison with cockroach movements is instructive because data for that model organism is available. SimROACH s motion while walking is not identical to that of the animal, but is similar enough to draw comparisons. In addition, some results from biology can be replicated by measuring neural or muscle activity. Chapter Stepping Robustness The goal of this section is to show that the stepping this system produces is robust when confronted with various challenges that a robot might face. All experiments were performed in simulation. The first experiments show that it can adjust to topographical changes, maintaining coordination as the body changes elevation or stepping in a hole. The second group of experiments shows how it can adjust to changes in its own form, such as extra load from an impediment or loss of communication from a sensor. In each of these cases coordination is maintained even if the walking pattern changes. Chapter Elevation Change Experiments Experiments were performed in which the simulated middle leg was attached to a test stand and made to walk on a frictionless surface, which models the robot experiment described later. The stand allowed the leg to lift the attachment point in stance, but had a 66

67 minimum height for the leg to simulate the support of the other leg. Two different tests were conducted. During the first, the attachment point of the leg was raised slowly as it walked, forcing the leg to reach lower and lower to make contact with the ground. The stepping rules state that the leg will depress the CTr joint until loading occurs and the FTi joint extends, so coordination is maintained as long as the leg can reach the ground. It could step in a coordinated fashion until lifted to a height of 1.7 cm, 170% of the height of the ThC1 joint. At this height it could not reach low enough to make ground contact and coordination disappeared. Similar experiments were performed in which the leg s minimum height was increased for only one stance phase, simulating a step in a hole. This is essentially no different from gradually changing the height as noted previously, except that the change is much faster. The middle leg could successfully step through a hole that was.73 cm deep, which is 56% of its standing height. After returning to normal stepping height, it continued to walk without disruption. Experiments were performed in which the stance phase was cut short by raising the leg during a step. Manually lifting the leg unloaded it, reducing propulsive forces in the muscles and causing the FTi joint to flex, which is a part of swing. Experiments in which swing was interrupted were also performed. The loading information caused the CPGs to transition to their stance states and then continue to step normally. These experiments show how SimROACH can adapt its stepping to unexpected obstructions. Chapter Body Manipulation Experiments The CPGs in SimROACH are intended to maintain stepping rhythm when the stepping motion is changed and normal sensory thresholds are not crossed. In order to show that CPGs help the leg continue stepping when the dynamics of the body change, 67

68 two versions of the middle leg were developed, one with CPGs and one without. That without CPGs needed sensory information to maintain rhythm, so restricted motion could halt stepping. Experiments were performed in which the foot segment s density was increased, mimicking situations when the environment might limit the leg s range of motion (e.g. stepping through mud, dragging along debris, etc.). In these experiments, the middle leg was attached to a simulated stand as in the previous section. Under normal conditions, both the leg with CPGs and that without CPGs were able to generate stepping motion. When the weight of the foot was increased, the simulation without CPGs ceased stepping because its range of motion was limited. This prevented it from reaching its normal sensory thresholds, and the reflex cascade halted. The model with CPGs, however, continued stepping despite the limited range of motion. Figure 23 shows the kinematic output of the leg during this trial. The version with CPG model steps with high frequency oscillation due to the extra mass, but maintains rhythm despite this. 68

69 CPGs cause stepping to continue in the absence of one sensory cue. This experiment simulates scenarios in which the robot s sensors are damaged, are removed, or malfunction in the field. This level of robustness would benefit a robot in a disaster zone or other dangerous environments. In these experiments, the middle leg was attached to a simulated stand as above. Both the leg with CPGs and without was able to walk forward with normal sensory input. However, when the positional signal from the FTi joint was eliminated, the version without CPGs stopped walking. This is because that feedback was necessary to drive the next joint Figure 23 Three plots showing kinematics during walking in a middle leg without CPGs and under normal load (top), without CPGs during weighted walking (middle), and with CPGs during weighted walking (bottom). The leg is able to walk under normal conditions, but adding extra weight stops the reflex cascade. Adding CPGs to the model restores rhythmic behavior. The extra inertia causes high frequency noise in the kinematics that would otherwise be absent. Gray shading indicates stance. transition in the reflex cascade. The model with CPGs, despite changed kinematics, was still able to move the joint in time with the other joints and 69

70 continue the walking motion. Plots of the kinematics from these trials are shown in Figure 24. Without CPGs, the simulation took a single step and then stopped. No FTi input means the leg would not extend its CTr joint, so the leg was artificially loaded around 2 s and 2.5 s, after which the leg took a step. However, it could not sustain the reflex cascade indefinitely without FTi information. This experiment highlights how CPGs can improve the rhythmicity of stepping, even when parts of the system are not intact. Figure 24 Plots showing kinematics during walking for a middle leg without feedback from one joint in a model without CPGs (top) and a model with CPGs (bottom). A CPG at every joint reduces the robot s reliance on sensory information in case of a malfunction. Chapter Comparison to Blaberus Kinematic data were collected from each leg during walking with the tripod gait. These data will be compared to joint angles on Blaberus during walking recorded by Amy Brown in the Ritzmann lab. 70

71 Figure 25 Joint angles of the front leg during tripod walking. The kinematics of the animal (left) were recorded by Brown on an oiled plate. SimROACH s kinematics (right) are provided for comparison. The vertical axes are scaled to match biological data, and are the same in both graphs. Stance phase is indicated by gray shading. The prothoracic leg possesses a complex three degree of freedom joint connecting its thorax and coxa. According to Brown s data the excursion each one makes during walking varies a fair amount, so general trends were used to produce SimROACH. During walking, the ThC2 joint is relatively inactive, and the ThC1 and ThC3 joints provide thrust and unload the leg. These two joints are highly active during walking, exhibiting average joint excursions of radians and radians, respectively. Figure 25 shows the kinematics the front legs of Blaberus and SimROACH while walking. In the cockroach the ThC3 and FTi joints extend and flex nearly in phase, something that SimROACH mimics. In addition, both flex the CTr joint during the stance phase, although SimROACH flexes at the end of phase rather than gradually throughout. SimROACH does not actuate the ThC1 and ThC2 joints properly. This is largely due to 71

72 Figure 26 Joint angles of the middle leg during tripod walking. The kinematics of the animal (left) were recorded by Brown on an oiled plate. SimROACH s kinematics (right) are provided for comparison. The vertical axes are scaled to match biological data, and are the same in both graphs. Stance phase is indicated by gray shading. the difficulty in tuning muscles to produce the desired range of motion. This issue is discussed in Chapter 7 Conclusions and Future Work. Figure 26 shows kinematic data from the middle leg of Blaberus and SimROACH. The middle leg matches biological data better than the front leg, producing similar ranges of motion and phase relationships. The CTr and FTi joints extend during stance, and the TrF joint extends during swing to position the leg for loading. In addition to kinematics, loading information, muscle activations, and the order in which they occur are similar to data recorded in the America cockroach Periplaneta americana. Figure 11 (See Chapter Sensory Information) shows the response of various neural populations in the cockroach to campaniform sensilum stimulation. As discussed in Chapter Sensory Information, SimROACH s load detection mimics that found in the cockroach. 72

73 Figure 28 Joint angles of the hind leg during walking. The kinematics of the animal (left) were recorded by Brown on an oiled plate. SimROACH s kinematics (right) are provided for comparison. The vertical axes are scaled to match biological data, and are the same in both graphs. Stance phase is indicated by gray shading. In addition to similar load processing, SimROACH activates leg muscles in the same order as cockroaches. In the American cockroach, the CTr extensor is active before the FTi extensor during stepping. SimROACH mimics this result because its stepping rules state that CTr extension causes loading, which causes FTi extension. Recordings from the cockroach and data from SimROACH are compared in Figure 27. This result suggests that the stepping rules that end swing and initiate stance are biologically accurate. The hind legs can be compared to data collected from Blaberus, shown in Figure 28. In the organism, the CTr and FTi joints are nearly locked in both phase and amplitude, a feature that was attempted in this model. This locking produces long propulsive strides. The mechanism that causes this is unknown, so SimROACH uses a comparator between the CTr and FTi joints detailed in Chapter Hind Leg 73

74 Figure 29 Plots comparing muscle activations with the onset of stance in Blaberus discoidalis (top) and SimROACH (bottom). In both systems the CTr joint is depressed to cause stance, which causes the extension of the FTi joint. The biological data was produced by the Zill lab. Stance is indicated in the bottom plot by gray shading. Top figure used with permission from Sasha Zill. Network. Figure 28 shows that this system locks the phase but not the amplitude of CTr and FTi motion. Chapter 4.5 Robotic Implementation A robotic model of the cockroach middle leg was built by Matt Klein (Figure 30) for experimentation on insect load sensing with Sasha Zill. The robot has five degrees of freedom actuated by Dynamixel AX-12+ smart servos. Sensors include potentiometers at each servo and strain gauges on the trochanter that mimic load sensors found on the cockroach (Zill, Schmitz, and Büschges 2004; Zill et al. 2011). Neural simulation was performed with LabVIEW (National Instruments, Austin, TX) and run on a laptop (2.0 74

75 GHz Intel Core2Duo). The laptop is connected wirelessly to a NI CompactRIO-9074 which handles all communication with the servos and sensors. The robotic leg used the circuit shown in Figure 20 to walk. Neurons not used for forward walking were removed for simplicity. This work proved the concept of using simulated nervous systems to control walking in a legged robot. The network ended up possessing Figure 30 Picture of the robotic leg used for hardware testing (A). It manages input and output through a NI CompactRIO (B) and outputs data to LabView (C). 22 neurons, including three CPGs. Only the three most important joints of the leg (FTi, TrF, and CTr) were rhythmically actuated. The muscle control units from SimROACH were adapted to the robot. Rather than performing the extra calculations needed to simulate the full muscle control unit, the entire system was abstracted. Each joint was assigned maximum and minimum joint excursion values. Half-center voltages were compared, and the more excited half-center s associated equilibrium point was sent to the servo. The servos only updated once every 50 neural timesteps, making this abstraction necessary for smooth motion. Smoothness was also increased by setting the servo compliance to its maximum value. This reduced the amount of torque the servos applied for a given positional error, decreasing the acceleration. 75

76 The robotic leg also differs from SimROACH in that its load detection is much more like in the animal. The robot has strain gages on the trochanter and tibia oriented in the same way as sets of campaniform sensilla in cockroaches. The readings from these were converted to neural activity in the same way as in SimROACH, by turning the load into a current to be injected into a neuron. In this implementation, only one strain gage on the trochanter fed into the coordinating circuit because it is most important to coordinating stepping (Akay et al. 2004). In the future, input from the others will be used to modify simulated muscle activity in other joints. Kinematic output of walking is shown in Figure 31. The joint excursions are nearly linear, which does not look organic. Figure 31 also shows CPG output, which is clearly coordinated in the desired fashion, with CTr extension loading the leg, FTi extension signaling for unloading, and TrF extension signaling for full unloading. All computation was performed on a laptop for these experiments, but an actual robot Figure 31 Joint Angles (top) and CPG activity (bottom) from a walking trial performed with the robotic leg. Stance is indicated by gray shading. 76

77 would need to perform calculations on board. A typical microcontroller would not be able to simulate the neural system in real time. Field Programmable Gate Arrays (FPGA), however, have been shown to perform this type of simulation in real time (Cheung et al. 2006). An FPGA uses a network of logic gates to physically create the circuit as interpreted by a compiler. This means an FPGA implementation of this system would actually build circuits that behaved like neurons, and then send information among them to simulate their interactions. As a proof of concept, one CPG (four neurons) was written to the FPGA built into the CompactRIO. The size of the network was limited by the storage capacity of the FPGA used. This FPGA properly simulated the CPG very rapidly, calculating a 1 ms integration step in only 9 µs. This is about twenty-five times faster than the 230 µs run time for an identical network on the laptop. Furthermore, the FPGA s parallel structure means it can simulate a network of any size in the same amount of time. Thus, as the network size increases, the FGPA s performance margin over traditional computers will also increase. 77

78 Chapter 5 Smooth Low Level Transitions SimROACH was designed not only to walk but also to smoothly and stably transition between gaits. It can produce inside and outside turning motions with its front and middle legs, allowing the body to turn while walking forward (Mu and Ritzmann 2005). This is accomplished by modifying the sensory pathways that couple the CPGs. Some of these changes are based on (Rutter et al. 2011), as seen in Figure 32. Rules for front leg gait changes were hypothesized based on kinematic data from (Brown 2011). Results from LegConNet presented in (Rutter et al. 2011) show that it could only change gait in a rapid, discontinuous way. If the command to turn were applied gradually, stepping often stopped. Gradually changing gait should not only produce smoother motion, but is also supported by work in cockroaches (Brown 2011). SimROACH exhibits gradual, smooth, and stable gait transitions due to its use of naturally rhythmic CPGs and gradual reflex reversals. Experiments show that removing the CPGs cause these transitions to fail. When transition timing is set to match observations in Blaberus, the presented model can change gait at any point in the stepping phase and smoothly change kinematics to produce the desired behavioral change. Figure 32 Diagrams that explain LegConNet when producing forward (left) and inside turning forward (right) behavior. Gait changes are generated by changing the connections and thresholds between sensory influences and bistable CPGs. Taken with permission from (B L Rutter et al. 2011) 78

79 Chapter 5.1 Implementing Behavior Changes via Reflex Reversals As described in Chapter Central Pattern Generators, each joint of each leg has its own CPG, which is not directly coupled to any other CPG. Instead, the CPGs are coordinated through sensory influences. These sensory influences are then modified by interneurons, allowing them to be reversed or rerouted by descending commands. In SimROACH such modifications are essentially bias changes to the interneurons, an idea familiar to perception networks. In typical neural nets, a neuron can be biased to change the threshold above which it can communicate with other neurons. Gait neurons (green in Figure 34) bias sensory interneurons in SimROACH to change which sensory pathways are active. The neural structure is detailed in Chapter Sensory Interneurons and Reflex Reversal. This biasing technique also has basis in findings from (Hellekes et al. 2012). The authors suggest that descending commands modify which sensory signals affect which joint. It is not known what part of the nervous system causes these changes, that is, stimulates the green Gait neurons in the control networks. Recordings in the central complex of cockroaches suggest that it may be the source of such reflex reversals (Guo and Ritzmann 2012). Insects may not change behaviors in exactly the same way that SimROACH does, but SimROACH is consistent with what is known about turning behaviors. In addition to reversing reflexes, CPGs must be able to be turned off (Daun-Gruhn 2010). It has been noted that different joints are actuated during different behaviors, so it is necessary to turn them off in a reversible way. The CPG model used will cease oscillating when sufficient inhibitory current is applied directly to the half-centers. When this occurs, the muscle control units receive no modulation from the CPGs, and both muscles are held taut as each control unit tries to reach its equilibrium point. 79

80 When examining biological joint angle data, a small range of motion for a joint in the organism may correspond to oscillatory actuation from a CPG or the passive reaction of the joint to the forces acting on the leg. The data compared to SimROACH come from oil plate experiments with Blaberus, during which these effects are minimal. For engineering simplicity, joints that are judged to traverse small angles are not actuated in SimROACH. Chapter 5.2 Flexible Networks Capable of Changing Gait As noted in Chapter 4.3 Networks and Their Function, separate control networks were developed for each leg. Each leg steps in a different manner, and gait changes are caused by reflex reversals specific to each leg. The front and middle legs of SimROACH can generate inside and outside turning behaviors, while the hind legs can only walk forward. These are all that is necessary for producing turning (Mu and Ritzmann 2005; Hellekes et al. 2012). The hind leg network, in its present state, cannot change gait and therefore is not presented in this section. 80

81 Chapter Gait Changes in the Middle Leg The middle leg can turn both inside and outside. Outside stepping is characterized by deactivation of the TrF joint and activation of the ThC2 to generate outside pushing motion. In MIDDLE LEG INSIDE TURNING Sensory Phenomenon Resulting Phase Changes Leg loaded FTi: Extend -> Flex TrF: Extend -> Flex ThC2: Extend -> Flex Fully FLX/Fully DEP TrF: Flex -> Extend Leg load decreasing Leg unloaded Fully EXT CTr: Extend -> Flex FTi: Flex -> Extend ThC2: Flex -> Extend CTr: Flex -> Extend MIDDLE LEG OUTSIDE TURNING Sensory Phenomenon Resulting Phase Changes addition, the CTr joint and FTi joint end stance further extended than usual (Brown 2011). Inside stepping is Leg loaded Leg load decreasing Leg unloaded Fully FLX FTi: Flex -> Extend ThC2: Flex -> Extend CTr: Extend -> Flex FTi: Extend -> Flex ThC2: Extend -> Flex CTr: Flex -> Extend characterized by reversing the role of FTi flexion and extension and activating the Figure 33 Tables that show stepping rules for inside turning (top) and outside turning (bottom) implemented in the middle leg of this model. There is no one authoritative source for these turning rules, but they are based on literature and hypothesized transitions. ThC2 joint to extend the leg s reach in swing. These rules are summarized in Figure 33, and are implemented in the middle leg control network, shown in the appropriate forms in Figure 34. Note that both features discussed previously, toggling CPGs and changing kinematics, are used to produce these behavioral changes. In addition, these changes occur when only one neuron in the circuit is stimulated, representing descending commands influence on the low level circuit. As noted before, this neuron takes 1500 ms to come to equilibrium, simulating the slow behavioral change observed in Blaberus. 81

82 Figure 34 Control networks for inside turning (top) and outside turning (bottom) in the middle leg model. The sensory pathways are highlighted to match the rules listed in Figure 33. The behavior changes are the result of rerouting sensory information and turning CPGs off where necessary. Evidence of these changes is most clearly seen in plots of CPG activity. Figure 35 shows the CPGs in the middle leg during the transitions from walking forward to turning in either direction. During walking, CTr extension slightly leads FTi extension, and full extension leads to CTr flexion followed by FTi flexion. Inside turning is more of a 82

83 reaching and pulling motion, not a pushing motion, so these joints switch: CTr flexion causes FTi extension, which causes CTr extension and FTi flexion. This puts the tarsus further from the thorax during swing and pulls inward during stance. The ThC2 joint is also activated during inside turning, extending to protract the leg further from the thorax during swing. The middle leg can produce outside stepping behavior by deactivating the TrF joint and activating the ThC2 joint. The ThC2 joint extends in stance, producing sideways pushing motion in stance. All other joints display similar motion to walking while turning. This is consistent with what is known about insects; outside turning motions Figure 35 CPG output from the middle leg during the transition to inside turning (top) and outside turning (bottom). Stance is indicated by gray shading. Turning is indicated by pink shading. usually show little difference from slow walking in cockroaches (Mu and Ritzmann 2005) or normal walking in stick insects (Hellekes et al. 2012). 83

84 Chapter Gait Changes in the Front Leg SimROACH can also produce turning motions with its front legs. The specific changes to joint activity that occur to cause such motion are presented in (Brown 2011). From these observations, the stepping rules for front leg turning in Figure 36 were developed. The networks in Chapter Front Leg FRONT LEG INSIDE TURNING Sensory Phenomenon Resulting Phase Changes Leg loaded FTi: Extend -> Flex ThC2: Flex -> Extend ThC3: Flex -> Extend Fully FLX/Fully DEP CTr: Extend -> Flex Leg load decreasing ThC3: Extend -> Flex FTi: Flex -> Extend Leg unloaded ThC2: Extend -> Flex Fully EXT CTr: Flex -> Extend FRONT LEG OUTSIDE TURNING Sensory Phenomenon Resulting Phase Changes Leg loaded FTi: Flex -> Extend ThC1: Flex -> Extend ThC3: Extend -> Flex Fully EXT/Fully DEP CTr: Extend -> Flex Leg load decreasing ThC3: Flex -> Extend FTi: Flex -> Extend Leg unloaded ThC1: Extend -> Flex Fully FLX CTr: Flex -> Extend Network encapsulate these rules. Adapting known rules to different legs has led to successful walking in robots (Rutter 2010) and biological models (Ekeberg, Blümel, and Büschges 2004). Figure 36 Tables that show stepping rules for inside turning (top) and outside turning (bottom) implemented in the front leg of this model. There is no one authoritative source for these turning rules, but they are based on literature and hypothesized transitions. Besides changes in joint excursion, outside turning is characterized by changing the phase of FTi actuation 180 degrees. Inside turning results from changing the phase of the ThC3 joint by 180 degrees and actuating the ThC2 joint to produce pulling motion. Figure 38 shows CPG activity during each of these changes. One can see that during outside turning the FTi joint extends rather than flexing in stance. In addition, the ThC3 joint extends in swing rather than stance to produce inside turning. Chapter 5.3 Effect of CPGs on Gait Transitions 84

85 Figure 37 Control network for the front leg configured to generate inside turning (top) and outside turning (bottom). The inactive pathways have been only partially filled. The rules for these networks are listed in Figure 36. The importance of CPGs was demonstrated by comparing gait transitions between two models, the middle leg of SimROACH and a version without CPGs, similar to 85

86 LegConNet. The designers of LegConNet showed that timing was important to changing gait properly (Rutter et al. 2011). Experiments with these simulations confirmed these results. The single legs were attached to a simulated cart as described previously. When made to walk and then transition to an inside turn, the version with CPGs successfully transitioned while the version without ceased stepping 50% of the time (6 trials). Examining network activity shows why this occurs. With no CPGs present, the FTi joint can only flex when the leg is loaded. In the model with CPGs, loading reinforces the signal to flex, but the CPG may cause the transition to occur slightly before load is detected. While turning, the leg does not load in the same Figure 38 CPG output from the front leg during the transition to inside turning (top) and outside turning (bottom). Stance is indicated by gray shading. Turning is indicated by pink shading. manner as during walking, breaking the reflex cascade. This effect can be seen in Figure 39. Without CPGs, flexion is only caused by load signals. However, CPGs may cause the FTi joint to flex before load is detected, making stepping more robust. 86

87 Figure 39 Plots showing how the command to flex the FTi joint (green) is only caused by loading (blue) in the model without CPGs (top), but can precede loading in the model with CPGs (bottom). Loading then reinforces this transition, making stepping even more robust. 87

88 Chapter 6 Smooth Behavioral Changes SimROACH also has an intermediate level network that coordinates its legs into walking gaits. Using Cruse rules (Cruse 1990) and hypothesized interleg pathways (Daun-Gruhn and Tóth 2010) it can switch between and lock into either a wave gait or a tripod gait. In addition, the low level networks can be changed by descending commands to produce turning gaits, much like in stick insects (Hellekes et al. 2012). These features enable SimROACH to smoothly change between behaviors, something that would benefit a legged robot. Chapter 6.1 Intermediate Level Coordination The most basic ipsilateral rules are that loading a leg excites unloading of the anterior leg, and unloading a leg prevents unloading the posterior leg. This rule is the same as contralateral coupling. These rules coordinate stepping in SimROACH, although insects use additional rules (See Chapter 2 Literature Review). SimROACH coordinates its legs only by coupling the CPG from the CTr joint of each leg to the others. SimROACH extends the CTr in stance and flexes it during swing in each leg, so coupling this one CPG keeps enough legs in stance at any time. This minimal coupling distributes control as much as possible, since each leg manages the details of its own stepping while sharing only minimal information (CTr state) with the other legs. This scheme is both flexible and adaptable; the individual legs can change their stepping motions while maintaining coordination with the others, and each leg can adapt to the terrain independently of the others. 88

89 Figure 40 Intermediate level circuit configured to produce a wave gait (A) and a tripod gait (B). Inactive pathways are shown with less fill. Synapses are color coded according to the key at the bottom. By changing the pathways that couple the legs, SimROACH can produce two different stepping patterns, a wave gait and a tripod gait. What distinguishes the two? All 89

90 Cruse rules apply in both, so there must be another factor. (Daun-Gruhn and Tóth 2010) hypothesized that these gaits change in the stick insect due to a modifiable connection between the hind and front legs. When using the wave gait, signals are passed forward from leg to leg and then looped around from the front leg to the hind leg to maintain the even spacing in stepping between legs. To generate a tripod gait, this connection is changed such that the inhibitory connections become excitatory and vice versa, locking the stepping phase of the front and hind legs. Cockroaches walk with a wave gait up to a certain walking speed, at which speed and above they utilize a tripod gait (Bender et al. 2011). SimROACH could explain how the tripod stepping relationship is maintained even as stepping speed increases. Gait pathways are switched by the same mechanism by which reflexes are reversed. Figure 40 (A) shows the circuit in the metachronal wave configuration and Figure 40B shows the circuit in the tripod configuration. Inhibiting the Metachronal neuron via descending commands causes the front and hind legs to mirror each other rather than staggering, producing a tripod gait. Chapter 6.2 Intermediate and Low-Level Gait Changes The wave gait is generated by extending the pattern of excitation and inhibition between legs to connect the front and hind legs. Figure 41 (A) shows CPG activity from the wave gait. The half-center of the CPG in each leg (front, middle, hind) associated with loading the leg (CTr extension) is shown. Each unloads as the leg behind it loads. This is unremarkable because the tripod gait is also a metachronal gait. What distinguishes the wave gait is that each leg steps only once before any other leg steps twice. If lines were drawn connecting the peak activity of each CPG, one could draw a 90

91 forward slanting or backward slanting line. But when an insect walks, it appears to have a forward traveling wave because each leg steps exactly once before any leg steps twice. Therefore drawing a forward slanting line in Figure 41 (top) makes the most sense for characterizing a wave gait. SimROACH can also produce a tripod gait by coupling the CPGs in the front and hind legs to cause simultaneous loading and Figure 41 Plots showing CPG activity in the three legs on one side while walking with a wave gait (top) and a tripod gait (bottom). The demonstrated patterns are consistent with gaits seen in insects. unloading. The middle leg steps 180 degrees out of phase of the others because of the Cruse rules. Figure 41 (bottom) shows CPG output from the model during the tripod gait. Again, only the half-centers that cause loading are shown. The front and hind legs are clearly in phase and the middle leg is exactly out of phase. One could draw diagonal lines connecting peak activity in each CPG, but in such a wave the front and hind legs would step twice per period, which an observer detects as a distinct pattern. Stable coordination required careful tuning of synaptic weights between legs. A numerical simulation performed with XPP revealed that a single CPG oscillates without 91

92 any equilibria besides one unstable spiral in the center of the limit cycle. Changing synaptic conductances between CPGs changes the rate of oscillation of the system. If the connections are too strong, the eigenvalues of the singular point all become negative and the point becomes stable, halting oscillation. Therefore SimROACH s legs are only weakly connected, but they very rapidly become coordinated, requiring no more than three or four steps from standstill. Chapter Changing Intermediate Gait The front to back connections can be stably changed without regard to stepping phase. Even though the transition momentarily disrupts the ipsilateral stepping pattern, contralateral leg coupling ensures SimROACH maintains support of its thorax. Figure 42 (top) shows CPG output for ipsilateral and contralateral CPGs during a gait change. The ipsilateral coordination smoothly changes by extending the period of front leg stepping during the transition. The other legs are unaffected because the connections between them do not change. The disruption of the ipsilateral stepping Figure 42 CPG activity during the transition from a wave gait to tripod gait in ipsilateral (top) and contralateral (bottom) legs. The first trace is the same in each plot. Tripod walking and the transition are highlighted in pink. pattern is remedied by the contralateral coupling, which ensures one leg of each pair is 92

93 always on the ground. Figure 41 (bottom) shows CPG output from both front legs during the same transition. Whenever the left leg extends its CTr joint, the right leg flexes its CTr joint. This coupling scheme ensures that SimROACH does not fall over while transitioning from a wave to a tripod gait. Chapter Changing Low Level Gait Chapter 5.2 Flexible Networks Capable of Changing Gait described how single legs of SimROACH can switch between walking and turning by reversing reflexes and deactivating joints. How do these changes affect the behavior of the entire system? SimROACH is a massively distributed control system, so while individual joints change during gait transitions, the rest of the system should be unaffected. Results show that this is true. SimROACH turns by stimulating the neurons that code for inside turn in the front and middle legs of one side and stimulating the neurons that code for outside turn in the front and middle legs of the other side. This is accomplished by stimulating all of the intended turning neurons by one neuron that codes for turning right or left, as shown in Figure 43. For example, Turn Right will excite the Inside Turn neurons in the right legs and the Outside Turn neurons in the left legs. The hind legs to not change their gait during turns, as in Blaberus (Mu and Ritzmann 2005). As intended, the effect of such changes on the intermediate level control system is slight. Figure 44 shows CPG output during turning while using the tripod and wave gaits. Since 93 Figure 43 Picture of a segment of the intermediate circuit configured to turn right by stimulating the Turn Right neuron, which in turn stimulates the proper low level turning neurons.

94 the intermediate level gait is not affected by turning behavior, coordination is maintained in both configurations. As noted previously, the legs are only coupled through the CPG that controls the CTr joints because they extend in stance and flex in swing in every leg during every gait. Therefore the phase relationship between the CTr in each leg should not change while turning. Maintaining coordination allows SimROACH to produce turning behavior. Experiments were Figure 44 Plots of CPG activity during the transition from forward walking to turning while using the wave gait (top) and the tripod gait (bottom). Turning is highlighted in pink. Dotted lines show that coordination is maintained during the transition. performed to quantify direction changes when the command to turn was given. Its path was recorded and the curvature was calculated as a function of path traveled. Results were gathered for both the wave and tripod gaits. SimROACH walked forward for 5 seconds and then turned for 5 seconds. Figure 45 shows results from two trials, one right and one left turn, showing clear changes in behavior as a result of the low level stepping rule changes. The RMS path curvature for all data using the tripod and wave gaits was m -1 and

95 m -1, respectively. This suggests that there was little difference between the performances when no sensory information was incorporated into intermediate level coordination. The radius of curvature varies greatly while turning, an undesirable trait for an engineered system. Videos of turning experiments reveal missteps in which a leg does not load properly, pulling at the air or brushing the ground. Another version of SimROACH s intermediate level circuit, shown in Figure 46, was developed in which pathways were gated by loading information. This gating did not make a noticeable difference in turning performance. Other models connect legs by allowing sensory information to modify activity of the low level circuits in adjacent legs (Daun-Gruhn 2010). Similar work with mammalian system modeling in the Biologically Inspired Robotics Lab has produced effective interleg coupling based on the same principle (Alex Hunt, Unpublished Figure 45 Robot heading (top) during two typical turning trials. The robot is commanded to walk straight for 5 s (blue) and then turn (green). The paths were smoothed with a Gaussian kernel, and the curvature (bottom) for each trial was calculated as a function of path length. In the left turn trial, the RMS curvature was during forward walking and during turning. In the right turn trial, the RMS curvature was during walking and during turning. 95

96 Figure 46 Intermediate level circuit modified to require loading information to tell the ipsilateral leg to unload. This sensory information is only utilized during the metachronal wave gait. Results). Implementing similar rules in the future might improve SimROACH s performance. Being able to change the radius of curvature would also be important for an actual robot. Currently SimROACH simply produces turning motions with each leg in an untargeted way. Perhaps ThC2 actuation, which controls abduction and adduction of each leg, could be modulated to produce turning motions that are more or less severe. 96

97 Chapter 7 Conclusions and Future Work Chapter 7.1 Conclusions This thesis presents a massively distributed control system based in insect neurobiology, SimROACH, which controls stepping in both software and hardware robot legs. The entire control network is assembled from physiological neuron and synapse models, meaning that sensory pathways and CPGs can be implemented in a biologically plausible way. This does not make SimROACH exactly like an animal, but since animals are much better locomotors than robots it is hoped this more accurate biomimicry will improve a robot s walking ability. SimROACH is able to change gaits smoothly and stably, something that remains a challenge for some robots today. Perhaps more biological accuracy in the future will further improve its performance. SimROACH also represents an alternative to traditional centralized robotic control methods. Like other distributed and neural network control systems, this network may solve the same problems in a more efficient way. Rather than performing complicated mathematical operations to set actuator torques, SimROACH only uses a computer to compute simple integration schemes, which are much less computationally expensive. The network connectivity determines the behavior and while more complicated behavior will require a larger network, the simulation method will not become more complicated. With further development and the addition of more advanced computation hardware like FPGAs, SimROACH and related systems may become an attractive method for controlling walking robots in the future. SimROACH was largely successful in accomplishing its goals. The first primary goal was to produce robust walking motions. SimROACH used a simulated nervous system to generate walking with structures discovered or hypothesized in stick insects and 97

98 cockroaches. The resulting motion is robust to perturbation and certainly carries SimROACH forward, although the kinematics do not precisely match all aspects of Blaberus, the primary model organism. Several steps in the future work outline how this could be improved. In addition walking, SimROACH can smoothly transition between walking and turning behaviors. It models hypothesized connections in insect nervous systems that allow them to make small changes to interjoint coordination and produce different stepping motions. This approach has been successful in simulation and will soon be applied to hardware. SimROACH was also moderately successful in becoming a useful model of insect locomotion control. SimROACH coordinates its legs in ways known or hypothesized to exist in insects, and captures a lot of what the animal does. The apparent motion is not identical, but this may be due to biological testing conditions or the simulation environment. In addition, such differences may lead to testable hypotheses for future biological research, such as finding pathways that appear to be necessary for proper motion in the model. In spite of this, the parameters of SimROACH s nervous system could be improved, and the future work is focused on resolving these issues. Better tuning could yield both more successful walking and more accurate biological models. Chapter 7.2 Future Work Chapter Sensitivity Analysis and Parameter Tuning SimROACH s locomotion, particularly the motion of its joints, does not precisely match that of the model organism Blaberus discoidalis. SimROACH simplifies many aspects of neurobiology, but proper joint range of motion is a straightforward comparison metric and should be obtainable despite simplification. Numerical tuning of muscle and neuron properties is largely responsible for these discrepancies. A formal sensitivity 98

99 analysis has not been performed on this system, but this will be crucial to direct any attempt to optimize SimROACH. Other work in the Biologically Inspired Robotics Laboratory has performed sensitivity analysis on muscle models, which could be leveraged in SimROACH. Earlier in this project a Matlab program was written that explored neural behavior by constructing a network, simulating its behavior for a short time, varying parameters of neurons and synapses, analyzing the output from each case, and fitting the results to a hypersurface for optimization. This approach ultimately failed because it explored parameters by generating every permutation of the system given ranges and resolutions for parameters, which both used too much memory and was time consuming. However, this experimentation made it clear that component and system behavior were more sensitive to some parameters than others, motivating a more formal sensitivity analysis of neural models in the future. What kind of tuning technique would be more appropriate for a system of this type and scale? Several methods exist for solving this type of problem, and fortunately this system has a reasonable seed value (SimROACH in its current form) and optimal data (kinematic data from the Ritzmann lab). Back propagation could be developed for these neuron and synapse models, although such a method may be slow for a system of this size. It may be useful to divide the system into subsystems and train each piece separately. Many sophisticated genetic algorithms exist, but finding a suitable parameter set for a system of this size may be very time consuming. More research and experimentation will have to be done to find a suitable method for tuning. 99

100 Chapter Actuator Types System performance may also be improved by using a different actuator than the simulated muscle used in SimROACH. SimROACH s two primary actuator issues are improper walking kinematics and no flexibility in walking speed. The range of motion depends on the length-tension relationship of the muscles, which has been very difficult to tune. An automated tuning method, if developed, could resolve this issue. However, the robot s servos produced the desired ranges of motion by adapting the muscle control units as described in Chapter 4.5 Robotic Implementation. This took very little time to implement, suggesting that eliminating muscles would accelerate the development of any future system based on SimROACH. Despite this success, the resulting motion was somewhat linear and inorganic. Rather than servos, a properly tuned muscle model with slow and fast muscle fibers could potentially produce motion closer to that seen in animals. Assuming tuning could produce the proper range of motion, the inclusion of additional fibers would enable SimROACH to generate more torque at its joints during transitions between stance and swing, increasing its walking speed like a cockroach (Watson and Ritzmann 1998). In addition, continuing to use simulated muscles would allow more direct comparisons between SimROACH and insects than servos would. Chapter Intermediate Circuit Besides muscles and parameter tuning, the completeness of the intermediate network could be improved. SimROACH uses a simple set of Cruse rules without any sensory feedback. SimROACH s walking has not been tested over rugged terrain, but one would expect it to struggle due to the lack of sensory feedback in the interleg connections. Simply adding more connections from sensors to CPGs caused the CPGs to 100

101 stop oscillating, similar to the phenomenon discussed in Chapter 6.2 Intermediate and Low-Level Gait Changes. Successfully implementing such changes would require a holistic design, that is, adding all connections simultaneously with low synaptic strength. This is very difficult to do properly by hand. Another alternative would be to couple the legs by allowing sensory information from one leg to modulate the muscle control units or sensory information in another. This could be used to make muscle positional or stiffness modifications, which are also important to coordinating multiple legs, rather than only step timing changes. This would be particularly useful when turning because different legs of SimROACH seem to fight each other when the simulation turns. Such additions should not halt CPG oscillation since the CPGs would not be directly affected. Features like these have led to successful interleg coupling in stick insect modeling (Daun-Gruhn 2010). Chapter Robotic Leg The final major improvement relates to the robotic leg. Currently the neural dynamics are calculated on a laptop nearby the test stand. Adding more neurons, whether for additional features or other legs, will increase the number of calculations performed every time step, slowing the system down. This method could not be used to control a mobile robot due to the power consumption and weight of the computer. Currently, field programmable gate arrays (FPGA) are being examined as an alternative. FPGAs are control chips that can be physically rewired by a computer. This builds the desired functionality into a circuit, allowing the neural system to be constructed in hardware and run with each neuron in parallel. Preliminary tests with the computing hardware in Chapter 4.5 Robotic Implementation suggest that the FPGA is over 25 times faster than a typical duo-core processor. In addition, building a larger network does not slow the 101

102 FPGA down, since it physically constructs a separate circuit for each neuron, then simulates them in parallel. This technology shows great promise for the simulation of physiological neural systems onboard robots. 102

103 Appendix A Network Topologies Since this work is ultimately for the development of an engineered device, the permutations of properties for neurons and synapses was kept to a minimum. Therefore a few stereotypical property combinations have been provided. Otherwise, the properties of each are listed next to their location on the maps. For nonspiking neurons, properties are listed in order resting voltage, time constant, membrane noise, tonic current, maximum calcium conductance, calcium activation midpoint voltage, calcium activation slope, calcium activation time constant, calcium deactivation midpoint voltage, calcium deactivation slope, and calcium deactivation time constant. Spiking neurons properties are listed as resting voltage, spiking threshold, membrane noise, tonic current, spiking threshold accommodation, and threshold accommodation time constant. Nonspiking synapses are listed as equilibrium potential, maximum conductance, low conductance threshold, and high conductance threshold. Finally, spiking synapses are listed as equilibrium potential, maximum conductance, time constant, facilitation, and facilitation time constant. Standard NSN -60 mv, 5 ms, 0 mv, 0 na Standard CaNSN -60 mv, 5 ms, 0.1 mv, 0 na, 5 us, -40 mv, 0.1, 2 ms, -100 mv, -0.1, 250 ms Standard Depolarizing -40 mv, 2 us, -60 mv, -40 mv Threshold Depolarizing -40 mv, 2 us, -47 mv, -45 mv Post Gate Depolarizing -40 mv, 2 us, -60 mv, -50 mv 103

104 Front Leg Location Properties F1-60 mv, -58 mv, 0 mv, 0 na, 0.5, 10 ms D2-60 mv, -55 mv E2-60 mv, -58 mv, 0 mv, 6 na 2 (G to I), 3, 4, 5 (All) Standard NSN 6 (All) Standard CaNSN 7 (All) Standard NSN 8 (All) -50 mv 9 (All) -100 mv, 20 ms 10 (All) -50 mv F1 to E2-70 mv, 1 us, 10 ms, 1, 50 ms E2 to D2-10 mv, 0.5 us, 20 ms, 0.5, 50 ms D2 to F2 Standard Depolarizing D2 to C3 Standard Depolarizing E2 to D2-10 mv, 0.5 us, 20 ms, 0.5, 50 ms E2 to A4 Standard Depolarizing F2 to I4 Standard Depolarizing F2 to J4 Standard Depolarizing G2 to D3 Standard Depolarizing G2 to E3 Standard Depolarizing G2 to F4 Standard Depolarizing G2 to H3 Standard Depolarizing G2 to I3 Standard Depolarizing H2 to G4 Standard Depolarizing 104

105 H2 to H4 Standard Depolarizing I2 to F3 Standard Depolarizing I2 to G3 Standard Depolarizing C3 to C4 Standard Depolarizing C3 to D4 Standard Depolarizing C3 to E5 Standard Depolarizing D3 to D5 Post Gate Depolarizing E3 to C5 Threshold Depolarizing F3 to H5 Post Gate Depolarizing G3 to G5 Threshold Depolarizing H3 to J5 Post Gate Depolarizing A4 to A5 Threshold Depolarizing B4 to B5 Threshold Depolarizing C4 to C5 Post Gate Depolarizing D4 to D5 Threshold Depolarizing E4 to E5 Post Gate Depolarizing F4 to F5 Post Gate Depolarizing G4 to G5 Post Gate Depolarizing H4 to H5 Threshold Depolarizing I4 to I5 Post Gate Depolarizing J4 to J5 Threshold Depolarizing 5 to 6 (All) -70 mv, 2 us, -60 mv, -40 mv 6 to 5 (All) Standard Depolarizing 8 to 7 (All) -80 mv, 2 us, -60 mv, -20 mv 7 to 9 (All) -10 mv, 1 us, -60 mv, -20 mv 8 to 8 (All) Standard Depolarizing 10 to 8 (All) -80 mv, 2 us, -60 mv, -20 mv Standing to 6 (All) -70 mv, 2 us, -60 mv, -40 mv Outside turning to F3, G4, H3, I4-90 mv, 3 us, -60 mv, -40 mv Outside turning to G3, H4, I3 Standard Depolarizing Inside turning to C4, D3, E4, F4-90 mv, 3 us, -60 mv, -40 mv Inside turning to A4, B4, D4, E3 Standard Depolarizing Walking 2 to A6, B6-80 mv, 1 us, 20 ms, 1, 50 ms 105

106 Middle Leg Location Properties D1-60 mv, -58 mv, 0 mv, 6 na F1-60 mv, -58 mv, 0 mv, 0 na, 0.5, 10 ms D2-60 mv, -55 mv, 0 mv, 0 na 2 (A to C, E to J), 3, 4 (All) Standard NSN 5 (All) Standard CaNSN 6 (All) Standard NSN 7 (All) -50 mv 8 (All) -100 mv, 20 ms 9 (All) -50 mv D1 to A2 Standard Depolarizing D1 to B3 Standard Depolarizing D1 to D2-10 mv, 0.5 us, 20 ms, 0.5, 50 ms D1 to J2 Standard Depolarizing F1 to D1-70 mv, 1 us, 10 ms, 1, 50 ms F1 to F2-10 mv, 1 us, 20 ms, 0.5, 50 ms F1 to G2-10 mv, 0.5 us, 3 ms, 1, 100 ms A2 to A4 Threshold Depolarizing B2 to B4 Threshold Depolarizing C2 to B3 Standard Depolarizing C2 to H4 Post Gate Depolarizing D2 to E2 Standard Depolarizing E2 to F4 Standard Depolarizing F2 to G4 Standard Depolarizing G2 to G3 Standard Depolarizing G2 to H3 Standard Depolarizing 106

107 H2 to C3 Standard Depolarizing H2 to D3 Standard Depolarizing I2 to E3 Standard Depolarizing I2 to F3 Standard Depolarizing J2 to I3 Standard Depolarizing J2 to J3 Standard Depolarizing A3 to A4 Threshold Depolarizing B3 to B4 Threshold Depolarizing C3 to H4 Post Gate Depolarizing D3 to E4 Threshold Depolarizing E3 to E4 Post Gate Depolarizing F3 to H4 Threshold Depolarizing G3 to I4 Post Gate Depolarizing H3 to J4 Threshold Depolarizing I3 to J4 Post Gate Depolarizing J3 to I4 Threshold Depolarizing 4 to 5 (All) -70 mv, 2 us, -60 mv, -40 mv 5 to 4 (All) Standard Depolarizing 5 to 6 (All) -70 mv, 2 us, -60 mv, -40 mv 6 to 8 (All) -10 mv, 1 us, -60 mv, -20 mv 7 to 6 (All) -10 mv, 1 us, -60 mv, -20 mv 7 to 7 (All) Standard Depolarizing 9 to 7 (All) -80 mv, 2 us, -60 mv, -40 mv Inside Turning to A2, B2, D3, F3, Standard Depolarizing H3, J3 Inside Turning to C3, E3, G3, I3-90 mv, 3 us, -60 mv, -40 mv Inside Turning to Walking 2-90 mv, 3 us, -60 mv, -40 mv Outside Turning to A3, B3 Standard Depolarizing Outside Turning to Walking 2-90 mv, 3 us, -60 mv, -40 mv Walking 2 to A5, B5-80 mv, 1 us, 20 ms, 1, 50 ms Walking 1 to C5, D5-80 mv, 1 us, 20 ms, 1, 50 ms Standing to 5 (All) -70 mv, 2 us, -60 mv, -40 mv 107

108 Hind Leg H1-60 mv, -58 mv, 0 mv, 0 na, 0.5, 10 ms A2-60 mv, -55 mv, 0 mv, 7 na C2-60 mv, -55 mv, 0 mv, 7 na F2-60 mv, -55 mv G2-60 mv, -58 mv, 0 mv, 6 na 2 (H, I), 3, 5 (All) Standard NSN 4 (All) Standard CaNSN 6 (All) -50 mv 7 (All) -100 mv, 20 ms 8 (All) -50 mv H1 to G2-70 mv, 1 us, 10 ms, 1, 50 ms G2 to F2-10 mv, 0.5 us, 20 ms, 0.5, 50 ms G2 to H3 Standard Depolarizing G2 to J3 Standard Depolarizing F2 to H2 Standard Depolarizing A2 to A4-70 mv, 1 us, 20 ms, 1, 50 ms A2 to B4-70 mv, 1 us, 20 ms, 1, 50 ms C2 to C4-70 mv, 1 us, 20 ms, 1, 50 ms C2 to D4-70 mv, 1 us, 20 ms, 1, 50 ms H2 to F3 Standard Depolarizing H2 to H3 Standard Depolarizing I2 to G3 Standard Depolarizing I2 to I3 Standard Depolarizing 3 to 4 (All) -70 mv, 2 us, -60 mv, -40 mv 4 to 3 (All) Standard Depolarizing 4 to 5 (All) -70 mv, 2 us, -60 mv, -40 mv 108

109 6 to 5 (All) -10 mv, 2 us, -60 mv, -20 mv 5 to 7 (All) -10 mv, 1 us, -60 mv, -20 mv 6 to 6 (All) Standard Depolarizing 8 to 6 (All) -80 mv, 2 us, -60 mv, -40 mv Standing to 4 (All) -70 mv, 2 us, -60 mv, -40 mv 109

110 Intermediate Level Circuit 110