Evaluating the Effect of a Flexible Spine on the Evolution of Quadrupedal Gaits

Animals demonstrate a level of agility currently unmatched in their robotic counterparts. The elasticity of muscles and tendons increase not only performance, but also the efficiency of movements. In contrast, robots are often constructed with rigid components connected by motors. However, recently compliant actuators and materials have been introduced to enhance robot designs, emulating the flexibility of natural organisms. In this paper, we incorporate passive flexibility into the spine of a quadruped animat and employ computational evolution to generate gaits. Results indicate that spine flexibility significantly increases both performance and efficiency of evolved individuals. Moreover, evolving the degree of spine flexibility along with artificial neural network controllers produces the highest performing solutions.


Introduction
Animals exhibit a diversity of behaviors that allow them to survive in dynamic environments, while having simultaneously evolved to be energetically efficient (Cavagna et al., 1977).Muscles perform the majority of work, but they are supported by the underlying skeletal and tendon systems.The inherent elasticity of muscles and tendons contributes to efficiency of movement throughout an animal's stride (Alexander, 1984).Beyond providing the basis for posture, the spine plays an important role in the energy efficiency of movements (Alexander, 1988).In particular, the spine can adopt very different roles.For example, in galloping horses, the spine acts similar to a stiff spring with minimal flexibility, while in sprinting cheetahs the spine moves actively, lengthening the stride (Hildebrand, 1959).
In contrast to their biological counterparts, robotic systems are considerably less flexible, typically comprising rigid components interconnected by motors.The addition of structures such as actuated spines can enhance the functionality of legged robots by increasing movement freedom (Leeser, 1996).Specifically, actuated spines expand the range of possible gaits (Berns et al., 1998) and increase maneuverability in constrained spaces (Park and Lee, 2007).In simulation, actuated spines have been shown to increase both the speed and efficiency of bounding gaits in a 2D quadruped, consistent with observations in biomechanics (Culha and Saranli, 2011).
Another possible approach to increasing the movement freedom of robotic systems is to incorporate passive compliance.Compliant designs exploit the intrinsic properties of materials, emulating the flexibility inherent in biological systems.Recently, other investigations have examined the integration of flexible materials into morphological components such as fish fins (Clark et al., 2012;Epstein et al., 2006;Clark et al., 2014), compliant actuators (Van Ham et al., 2009), and flexible arms (Sfakiotakis et al., 2014).Silva et al. (2005) described quadruped locomotion for a simulated animat with a flexible spine, but did not quantify the effect of that component on performance.
In this paper, we examine how a passively flexible spine influences the evolution of locomotive behaviors in a quadruped animat with an artificial neural network (ANN) controller.We first evolve gaits for an animat with a rigid spine.Next, we create a three-segmented spine with predefined flexibility.Finally, we allow the degree of flexibility to evolve along with the ANN controller.Results indicate that adding passive flexibility to a quadruped can significantly increase performance and efficiency of movement.These results complement earlier investigations in actively controlled spines, demonstrating that passive flexibility is beneficial, despite requiring no direct control from the ANN.Apparently, the additional degrees-of-freedom (DOF) allow the evolved controllers to express gaits that are not possible with a rigid spine.These results have practical implications for robotic systems by improving performance as well as efficiency.
cles (AUV).In addition, Krishnamurthy et al. (2010) presented an AUV design based on the morphology of an electric ray that employs flexible materials in the tail to increase efficiency.Flexibility has also been explored in terrestrial robotics.For example, modeling the flexibility of biological muscles has produced life-like bipedal gaits for simulated robots by allowing the body to absorb shock when contacting the ground (Geijtenbeek et al., 2013).Further, Ackerman and Seipel (2013) have shown that reducing the vertical movement of the center of mass during walking reduces energy consumption in legged robots.This reduction is similar to the dampening effect provided by tendons in biological organisms, reducing collision forces with the ground and increasing efficiency (Bertram and Hasaneini, 2013;Ruina et al., 2005).However, controlling flexible joints can be challenging as they react less predictably to controller input and interactions with the environment (Chaoui et al., 2009).
One possible approach to addressing the control problem is through evolutionary robotics (Sims, 1994;Nolfi and Floreano, 2000), where the controller evolves to take advantage of morphological characteristics.Evolutionary approaches have produced effective robotic gaits in quadrupeds (Clune et al., 2009;Doncieux and Mouret, 2013), salamanders (Ijspeert et al., 2005), and bipeds (Lessin et al., 2013).In addition to evolving active control strategies, co-evolving morphology and control can exploit relationships between brain and body (Bongard, 2011;Paul, 2006;Valsalam and Miikkulainen, 2008;Rieffel et al., 2010).Furthermore, computational evolution has proven effective at exploiting passive flexible materials (Moore and McKinley, 2012) and passive joints (Moore and McKinley, 2013) in robotic systems.In this paper, we evolve control and spinal flexibility in quadruped animats.

Methods
Simulation Environment Simulations are conducted with the Open Dynamics Engine (ODE) (Smith, 2013), a 3D physics simulation environment.Although designed to simulate rigid bodies, flexible components can be modeled by interconnecting multiple segments with spring-like joints.ODE also handles collisions between 3D bodies and forces such as friction and gravity.In this study, the environment is a flat, high-friction surface minimizing slippage.
Evolutionary runs are conducted with two quadruped animats.The first animat has a single, rigid torso with four legs.The second, shown in Figure 1, has a three segment torso connected by passively flexible, 2-DOF joints.This configuration emulates the flexibility of a spine in a natural organism, allowing for increased movement freedom over the single, rigid torso quadruped.The legs of both quadrupeds have two 2-DOF joints, a hip and knee, for a total of eight actively controlled joints.Movement of the legs can be away from, or along the long axis of the body.Treatments We conduct three treatments.The first (Rigid-Spine) uses the single, rigid-torso quadruped described above.The second treatment (Flex-Spine) has a quadruped with a three-segmented torso and a predefined (passive) flexibility.The third treatment (Evo-Flex) also has a threesegmented torso, but the level of flexibility in the passive spine evolves with the controller.
Passive Flexibility As noted above, we model a flexible spine by connecting rigid boxes with spring-like joints.When flexed, a joint attempts to return its two connected bodies back to their neutral position.Together, the three segments and two spring joints allow the torso to flex side-toside and up/down.Both joints in the animat have the same level of flexibility.Thus, the torso can contribute to movement, rather than serving a purely structural purpose.In the Evo-Flex treatment, spinal flexibility can range from very light springs (low spring coefficients), enabling the spine to move easily, to very stiff springs (high spring coefficients), providing minimal flexibility of the spine and returning the torso segments more rapidly to their neutral position.
Artificial Neural Network Controllers are evolved with the NEAT algorithm (Stanley and Miikkulainen, 2002), which uses a genetic algorithm to evolve recurrent artificial neural networks (ANNs).NEAT begins with a fully connected input-to-output network without hidden nodes, complexifying the ANN by adding nodes and links over evolutionary time.Speciation addresses the issue of two dissimilar network topologies acting as parents for an individual by determining compatibility between networks, thus defining which ANNs can be crossed over.A population of 120 individuals is evolved for 4000 generations.Fitness and Efficiency Evaluation Individuals are evaluated based on the Euclidean distance from their initial position to the final position after 10 seconds of simulation time.In order to reduce the bias on the evolutionary process, we do not constrain movement to a particular direction.During an evaluation, we record the forces exerted by each joint to determine an individual's efficiency.We calculate efficiency as the distance traveled per unit of power exerted by a robot.In previous work (Moore and McKinley, 2015), we found that evolved quadrupeds exhibit an inherent level of efficiency during movement, even when selecting for performance only.

Experiments & Results
Several distinct gaits evolved across the three treatments.They can be classified as bounding (rear legs provide power, front legs stability), shuffling (low posture, body often on the ground), and trotting (diagonally paired gait).Figures 2, 3,   and 4, respectively, illustrate sample gaits from each of the three treatments.High performing gaits evolve in every treatment.Videos of selected gaits from the three treatments are available at the following addresses: Rigid-Spine: http://youtu.be/laChVR5LWgEFlex-Spine: http://youtu.be/AynOi6tBg0AEvo-Flex: http://youtu.be/oW-tLQx5DSc Rigid Spine We first examine the evolution of quadrupedal gaits in the Rigid-Spine treatment.Each treatment comprises 20 replicate runs with unique random seeds.Figures 5 and 6 plot the evolutionary trajectories of distance and efficiency, respectively.Individuals evolve viable gaits with the best individual per replicate traveling 23.5 units (7.83 body lengths) on average during a simulation.Units of distance are a simulation based metric and do not correspond to a physical dimension.For reference, the main body is 3 units long, 1 unit wide, and 0.5 units tall.Moreover, the movements increase in efficiency over evolutionary time, although this is not selected for.Instead, it arises as distance traveled increases, apparently as the leg movements evolve coordination.Flexible Spine In the Flex-Spine treatment, the torso is divided into three segments with a fixed spring coefficient between segments.The evolutionary trajectory of distance traveled and efficiency are plotted in Figures 7 and 8, respectively.When compared to the Rigid-Spine treatment, the flexible spine significantly improves both distance traveled and efficiency (p < 0.001 Mann-Whitney-Wilcoxon Test for both).The best individuals across replicates move 32.5 units (10.83 body lengths) during a simulation.The addition of a passively flexible spine appears to enable both higher performing and more efficient gaits, as efficiency is more than double that of a fixed spine animat.
Evolvable Flexibility In the final treatment, we evolve the level of flexibility in the spine along with the ANN controller.Both spine joints in an animat have the same evolved level of flexibility.The evolutionary trajectories of distance traveled and efficiency are plotted in Figures 9 and 10   Compared to the Flex-Spine treatment, the evolvable flexibility significantly improves performance (p < 0.001 Mann-Whitney-Wilcoxon Test) with no significant difference in efficiency (p = 0.05589).The ability to evolve flexibility is beneficial, but we observe that the individuals do not converge to a specific level of flexibility.Instead, high performing individuals evolve different levels of spine flexibility.Thus, it appears that the co-evolutionary process of control and morphology leads to more diversity in solutions compared to a specific level of flexibility.Figures 12 and 13, respectively, plot results for two replicates (12 and 4) of the farthest traveling individual per generation for two replicates from the Evo-Flex treatment.These two are representative of the remaining replicates.Figure 12 shows a replicate that evolves moderate spinal flexibility crossing from highly flexible individuals through the gap in flexibility.Furthermore, Figure 13 shows a more moderate progression through the shaded rectangle.Although this pattern shows that individuals can evolve inside this range of flexibility, the best individual for this replicate evolves a stiffer spine.The evolutionary trajectories of the replicates may explain the split between the highest performing individuals (Spring Coefficient < 400) and those with moderate flexibility (Spring Coefficient > 750) who are not as high performing.Apparently, it is difficult for individuals within this flexibility range to travel far.In addition to performance gains, flexible spines (Flex-Spine, Evo-Flex) also result in a significant increase in efficiency when compared to the Rigid-Spine treatment (p < 0.001 Wilcoxon Test), see Figure 17.However, there is no  Body Position One way to compare gaits is to measure the height of the torso over the evaluation period.Figure 18 shows the mean torso height across replicates for each of the three treatments.The periodicity evident in the plot is due to the oscillating input signal provided to the evolved ANNs.Torso height for the flexible spine quadrupeds is measured as the position of the middle component of the torso, which corresponds to the measured position of the torso in the rigid spine animat.Two differences arise between flexible and rigid spines.First, the torso of individuals with flexible spines is generally higher than those with rigid spines.This suggests a posture with the legs under the robot as in walking or running, and less contact of the torso with the ground.Second, the amplitude of the rigid spine indi-  Further analysis of the difference in amplitude is provided in Figure 19.Here, we calculate the mean position of the torsos per treatment normalizing the data to be the absolute deviation from the mean.This figure shows the vertical movement of the torsos over time.As shown, the Rigid-Spine and Flex-Spine treatments have similar displacements from their respective means.Indeed, a Mann-Whitney-Wilcoxon Test indicates that there is no significant difference in the amplitudes between these two treatments (p = 0.06316).However, the displacement of the Evo-Flex treatment is significantly lower than that of both Rigid-Spine and Flex-Spine treatments (p < 0.001 for both).In short, individuals with both evolved control and flexibility have the least amount of vertical movement in their torso.This result is consistent with prior research showing that reducing the vertical movement of mass during locomotion can increase the energy efficiency of a legged robot (Ackerman and Seipel, 2013).In our study, the torso is the highest mass component of the quadruped, with the flexible spine acting to reduce its vertical movement.The result is greater distances traveled in a more energetically efficient gait.In biological organisms, minimizing collision force with the ground leads to efficient gaits (Bertram and Hasaneini, 2013).Collisions are typically dampened by minimizing the vertical displacement of the center of mass.After examining the body position over time, it appears that the evolved individuals with a flexible spine have gaits that minimize vertical displacement of the torso.

Conclusions
Biological organisms exhibit a level of agility and dexterity unparalleled in their robotic counterparts.In this paper, we have introduced a flexible spine to a quadruped animat.Results of evolutionary runs indicate that a flexible spine significantly increases performance in the evolved gaits.Furthermore, efficiency is also significantly improved, even without an explicit pressure to do so.It remains to be seen if additional benefits are to be gained by adding flexibility throughout the body, including legs and feet, which are rigid in this study.
In this preliminary investigation, we evolve individuals solely on a distance traveled metric.Efficiency is evaluated only after evolution.Future work will address incorporating efficiency as a metric for evolution, in addition to performance goals.Additionally, we plan to investigate passive components in other animats to help identify universal principles to apply to the robot design process.

Figure 1 :
Figure 1: Quadruped animat with a flexible spine.The torso is divided into three segments connected by passively flexible joints which move in reaction to external forces that arise from leg movement.
Figure2: An evolved quadruped gait with a single, rigid torso.This is a rear bounding gait taken from one of the highest performing individuals from the Rigid-Spine treatment.

Figure 3 :
Figure 3: Evolved quadruped gait with a three-segmented torso connected by passively flexible joints from the Flex-Spine treatment.The spine flexes throughout the gait providing increased compliance in the robot resulting in higher performance than the Rigid-Spine treatment.

Figure 4 :
Figure 4: An evolved gait from the Evo-Flex treatment.Here, the spine is highly flexible, contributing to the gait by lengthening the stride while the legs are in contact with the ground.riodic oscillating signal (1), two joint angle sensors per joint (16), touch sensors on each foot (4), and a bias (1).The 16 outputs specify the desired angles of the hinge joints, two per each 2-DOF joint.
Figure 5: The mean maximum and average fitness across 20 replicate runs for the Rigid-Spine treatment.The shaded areas in this and subsequent plots represent the 95% confidence intervals.

Figure 6 :
Figure 6: The mean maximum and average efficiency across 20 replicate runs for the Rigid-Spine treatment.Efficiency is measured as the distance traveled per unit of force exerted during a simulation.Efficiency rises slightly over evolutionary time although it is not selected for.
, respectively.This treatment produces the highest performing and most efficient individuals.The best individual per replicate averages 33.5 units traveled (11.16 body lengths).

Figure 7 :
Figure 7: The mean maximum and average fitnesses across 20 replicate runs for the Flex-Spine treatment.

Figure 8 :
Figure 8: Mean maximum and average efficiency across 20 replicate runs for the Flex-Spine treatment.

Figure 9 :
Figure 9: Mean maximum and average fitnesses across 20 replicate runs for the Evo-Flex treatment.

Figure 11
Figure 11 plots the performance of the farthest traveling individual from each of the 20 replicate runs in the Evo-Flex treatment.Evolved individuals fall into two categories of

Figure 11 :
Figure 11: The distance traveled versus spine flexibility for the best individual from each replicate in the Evo-Flex treatment.The farthest traveling individuals have highly flexible spines as indicated by the low spring coefficient.

Figure 12 :
Figure 12: The farthest traveling individual per generation for a single replicate from the Evo-Flex treatment.Brighter colors represent individuals who evolve in later generations.

Figure 13 :
Figure 13: The farthest traveling individual per generation for a single replicate from the Evo-Flex treatment.

Figure 14
Figure14plots the efficiency versus evolved spring coefficient of the farthest traveling individuals from each replicate in the Evo-Flex treatment.Although not a target of the selection process, there appears to be an inherent efficiency in evolved solutions.In contrast to the distance traveled results, efficiency scores are not biased toward highly

Figure 14 :
Figure 14: Efficiency of the farthest traveling individual from each replicate in the Evo-Flex treatment compared to their spine flexibility.In contrast to Figure 11, efficiency does not appear to depend on spine flexibility.Performance and Efficiency Comparisons Figure 15 plots efficiency versus distance traveled of the farthest traveling individuals from each replicate in the three treatments.The Evo-Flex treatment produces the farthest traveling individuals.Boxplot distributions for distance traveled are shown in Figure 16.Applying a pairwise Mann-Whitney-Wilcoxon Test, finds all three pairings to be significantly different (p < 0.001).The addition of a flexible spine, even one with a predetermined flexibility, results in a significant improvement in performance.

Figure 15 :
Figure 15: The farthest traveling individuals from each of the three treatments conducted in this study.Evolving the flexibility of the spine produces the highest performing, and most efficient individuals across all three treatments.In addition to performance gains, flexible spines (Flex-Spine, Evo-Flex) also result in a significant increase in efficiency when compared to the Rigid-Spine treatment (p < 0.001 Wilcoxon Test), see Figure17.However, there is no

Figure 16 :
Figure 16: The performance distribution of the farthest traveling individual from each replicate across the three treatments.significant difference in efficiency between the farthest traveling individuals from the Flex-Spine and Evo-Flex treatments (p = 0.05589).Still, evolving the flexibility of the spine results in the best individuals in terms of combined performance and efficiency.

Figure 17 :
Figure 17: Distribution of efficiency in the farthest traveling individual from each replicate across the three treatments.
Jared M.Moore, Craig P. McGowan, Philip K. McKinley (2015)  Evaluating the Effect of a Flexible Spine on the Evolution of Quadrupedal Gaits.Proceedings of the European Conference on Artificial Life 2015, pp.166-173 viduals is greater than those with flexible spines, indicating an increased vertical movement for those with rigid spines.

Figure 18 :
Figure 18: Mean torso height over simulation time across replicates for the three treatments.Error bars indicate the 95% confidence intervals.

Figure 19 :
Figure 19: Mean normalized torso displacement for 20 replicate runs across the three treatments.The average torso height was determined per treatment and displacements recorded.The Evo-Flex individuals generally have less vertical movement throughout their gait.