Evolutionary Progress in Heterogenous Cellular Automata ( HetCA )

Although very controversial in the field of evolutionary biology, the notion of evolutionary progress is nevertheless generally accepted in the field of Artificial Life. In this article we adopt the definition proposed by Shanahan (2012) to study the existence of evolutionary progress in an evolutionary simulation which we call HetCA. HetCA is a heterogeneous cellular automata characterized by its ability to generate open ended long-term evolution. In this study, we measure evolutionary progress on three criteria: the robustness, size and density of generated genotypes. Our results demonstrate that the oldest genotypes in terms of evolutionary time are frequently the most robust, and that phenotypic density is higher for genotypes collected later in the evolutionary process.


Introduction
Evolutionary progress (EP) hypothesizes that evolution tends toward a goal such as greater complexity of individuals.It's a truism to speak about EP in terms of Evolutionary Computation (EC) where evolution is used as an optimisation method to solve problems.But the existence of EP is much less obvious in biology which evolves in an open ended evolutionary process.HetCA, Medernach et al. (2013), is a heterogenous cellular automata (CA) evolutionary simulation which represents a version of open-ended evolution.
In this paper, we measure evolutionary progress in HetCA using three traits: the density, size and robustness of evolved individuals.To do so we compare the properties of genotypes taken at different stages of the evolutionary process.
The study of these three features allows us to analyze more precisely the nature of evolutionary strategies in HetCA.We hypothesize that size of the genotype together with robustness and density of its phenotype may serve as useful measures of the effectiveness of that genotype.We examine the existence of a correlation between these measures.
Our paper is organized as follows: firstly, in the Background section, we review EP and CA.Then, in the Methodology section, we present experimental settings and define the measures which we have chosen to use as evidence for the existence of EP.Following this, we present the simulation results in the Results section and finally, in the Discussion section, we discuss those results and explore some specific examples to discuss their behaviour qualitatively.

Background Evolutionary Progress
The popular belief that a form of evolutionary progress emerges naturally as a result of the process of natural selection may seem obvious, but as described by Shanahan (2000), it is a controversial view in the field of biology.While the notion has several supporters, including Richard Dawkins and Ernst Mayr, most evolutionary biologists, like for example, William Provine and Stephen Jay Gould strongly criticise this idea: "Progress is a noxious, culturally embedded, untestable, nonoperational, intractable idea that must be replaced if we wish to understand the patterns of history", Gould (1988).
There are various reasons for this rejection of the EP concept in biology.For one thing, the notion of EP is sometimes regarded as a remnant of a very anthropocentric view of the classification living beings (a legacy of Aristotle), which can also be seen in several predecessor of the Darwinian model of natural selection, such as theories proposed by Buffon or Lamarck.
Additionally, the concept of EP is usually linked to the existence of an increase in the complexity of a living being.And this idea is challenged in evolutionary biology for various reasons, summarized in McShea (1991).
Another argument, which could be seen to weaken the EP hypothesis, has been put forward by researchers such as Johnson et al. (2012) in which they highlight numerous examples of species losing previously acquired evolutionary traits, possibly induced by ecological changes.
On the other hand, in the field of Artificial Life, though the creation of artificial long-term EP in open ended simulation has not received consensus, the existence of evolutionary progress in the living is often taken for granted as illustrated by, for example, Ciliberti et al. (1999) and its association with the evolution of complexity is presented as one of the goals of Artificial Life by Bedau (2003).
This dissemblance in approach may be explained by the proximity of the field of Artificial Life with other evolutionary paradigms of optimization and problem solving which are similarly inspired by natural selection, such as Genetic Programming and Genetic Algorithms, where the existence of evolutionary progress is not disputed.
But the reason could simply be that the concept of evolutionary progress has not yet been fully formulated and developed.In that regard, Shanahan (2012) proposed a definition of evolutionary progress1 as "intergenerational directional change embodying improvement in the properties characterising a population of biological entities".A change should be considered directional if it "has a direction over a given time interval" and therefore "if the value of one of its properties increases during that time interval".It is important to note that, according to this definition, evolution acts on a multitude of populations themselves having multiple characteristics.It is, then, quite possible that evolutionary progress exists for a particular lineage of living beings while, in another lineages, for the same trait, there is no evolutionary progress or its direction is reversed.

Evolutionary computation and open ended simulations
One of the problems facing researchers who use natural selection as an optimization mechanism in EC, is a decrease in the diversity of individuals during evolution which may lead to premature convergence to a local optimum Hornby (2006).Several biologically inspired solutions for avoiding premature convergence have been successfully tested.For example, inspired by an evolutionary arms race, Hillis (1990) used the coevolution of two populations2 and Lessin et al. (2013) employed an intermediate step goal to generate behavioral complexity in evolved virtual creatures.In both of these studies the researchers examined evolution that was goal orientated but also in some sense open ended.Also, in Evolutionary robotics, Haasdijk et al. (2014), among others studied the ability to combine, in a single simulation, survival of the organism together with resolution of tasks.Thus, it may be useful to analyze the mechanisms that avoid or ameliorate premature convergence when evolution is open ended.
Figure 2: Specificities of states in HetCA.Here agemax = 7 iterations and decay time = 375 to 1875 iterations as specified in Table 1.Age of living cells increase each generation as long as it stay alive and is reboot to zero if the cell become Quiescent or Decay.
To the best of our knowledge, while CA models are widely used in Artificial Life for such things as artificial chemistry simulation Cole and Muthukrishna (2014) or to model various evolutionary processes Wolfram (2002) no attempt has yet been made to directly investigate EP in heterogeneous cellular automata.HetCA is an Artificial Life model designed to achieve some form of open-ended evolution3 .Its purpose is similar to simulations such as Tierra Ray (1993) or Avida Adami et al. (1995), systems where a competition between replicating computer programs occurs in a virtual machine.
As described by Medernach et al. (2013), HetCA is designed as a heterogenous cellular automata, using several cell categories: Living cells, decay cells and quiescent cells.Every living cell has, in addition to its state, a genotype that determines its transition rules.These genotypes mutate and may spread to neighboring cells as shown in Figure 1.Moreover, the application of mechanisms such as Aging, Quiescence and Decay as shown in Figure 2, demonstrate that even a minimal survival strategy of genotypes implies some form of cooperation between some cells4 as illustrated in Figure 3.The HetCA model achieves long term dynamics, some form of open-ended evolution and different ecosystems characterized by distinctive patterns.
Figure 3: Six steps survival strategy from one possible phenotype of a genotype extracted at iteration 300000 in HetCA, tested here in a randomly initialized homogenous CA.This phenotype does not provide cells with the opportunity to grow old enough to decay before an evolutionary step changes them into quiescent cells.In doing so the cells lose their own genotype to facilitate the survival of the genotypes of neighboring cells.The meaning of the colour coding is described in Figure 2.

Methodology
In this section we analyze three genotypic properties as potential indicators of evolutionary progress: robustness, size and density.

Collection of genotypes
In order to assess the existence of EP we created a collection of genotypes at various stages of the evolutionary process, in the following manner: We performed 30 simulations, each on 500000 iterations with the parameters5 listed in Table 1.The possible genotypes of an individual are its transition rules encoded with CA-LGP using the function set depicted in Table 2. Mutation of genotypes is enabled and we use the Micro/Marco-mutation of CA-LGP described in Medernach et al. (2013).For each simulation we saved the most common genotype (most frequently occurring) in iterations 5, 1000, 5000, 50000, 300000 and 500000.We have chosen to use iterations 5, 1000, 5000 and 50000 as they correspond to four distinct stages of the typical evolutionary process in HetCA and are characterized by four very differents environments as shown in Figure 4. Iteration 500000 was chosen because it is the final iteration of studied simulations, and iteration 300000 was selected as an intermediate step between iteration 50000 and iteration 500000.
The choice of the most common genotype may seem arbitrary, but it was not realistic to process all genotypes at each iteration of the simulation whereas the frequency is a naive, but nonetheless efficient criterion for assessing its representativeness and its success at any stage of the simulation.

Evaluation of genotype robustness
To assess the robustness of the individual we measure its ability to survive in different environments.One could compare this measurement with the definition of evolutionary progress proposed in Dawkins (1997): "a tendency for lineages to improve cumulatively their adaptive fit to their particular way of life, by increasing the numbers of features David Medernach, Jeannie Fitzgerald, Simon Carrignon, Conor Ryan (2015)   which combine together in adaptive complexes".The robustness of an individual is measured by comparing genotypes in a pairwise fashion in series of simulations where we disable mutations.We compare each genotype to every other genotype saved from a different run after a different number of iterations.We don't compare genotypes from the same runs because they have already competed in their evolutionary history, and making these comparisons could potentially skew results.Nor do we compare two genotypes collected after the same number of iterations of the cellular automata because this would not provide any information about EP.
Half of the cells, randomly selected, are initialized with the first genotype, the other half with the second one.All cells are initialized at a random living state.The simulation is stopped either after 50000 iterations6 , as illustrated in Figure 5, or when more than 99% of the living cells share the same genotype.After the simulation is stopped, the most frequent genotype is considered to be more robust than the other one due to it's dominance of the environment.To conduct this experiment in a reasonable amount of time we choose to use only genotypes from the first 10 simulations at the 6 previously chosen iterations, which represents 6 × 10 = 60 genotypes and 45 × 60/2 = 1350 simulations.
Figure 5: Formation of cells clusters in a robustness test.On the right column the cells are depicted with their current states (color coding is described in Figure 2), on the left column the repartition of the two genotypes is depicted.Cells that don't curently have a genotype because they are in a decay or quiescent state, are represented with their current states on both sides.

Evaluation of phenotype densities
The density of each genotype is evaluated by a simulation where mutations are disabled and each cell is initialized at a random living state with the tested genotype as transition rule, as illustrated in Figure 6.The simulation is run for 2000 iterations.The density measure ρ of a genotype is the average number of living cells during the simulation: × 100 and ρ i = n alive S Where ρ i is the density of the phenotype7 for the iteration i, n alive is the number of cells which current state is one of the alive states8 and S is the size of the grid of the cellular automata9 .The phenotypic densities of all the 30 × 6 = 180 genotypes are computed.

Evaluation of genotype sizes
In evolutionary biology, according to Lynch and Conery (2003) the increase of size of individual genotype is caused by genetic drift and linked to population size (N e )10 .Similarly, in evolutionary computation size is frequently studied, if only because of the potentially high computational cost associated and a potential correlation with overfitting Fitzgerald and Ryan (2014).More specifically, in evolutionary algorithms (EA), different studies show that size is not correlated with the ability of individuals to solve the presented task.We use the number of program statements (n prog ) as a measure of the genotype size.Sizes of all the 30 × 6 = 180 genotypes are computed.
Note that two of the three studied traits 11 are bounded and directly measurable while robustness is a relative criterion.

Robustness
In accordance with the hypothesis proposing the existence of evolutionary progress, Table 3 shows that during the pairwise comparison of the genotypes collected at different stages of the simulation, the oldest genotypes are frequently the most robust.The scores are higher than 89% in eleven out of fifteen cases, and even the smallest margin of 59% reported for the comparison of genotypes collected after 5000 and 1000 iterations, is the only non-significant using the binomial test at p = 0.05.Table 4 shows that not counting the tests that reach 50000 iterations slightly increases the win rate of the oldest genotypes.Not surprisingly, Table 5 indicates that robustness increases with increasing number of iterations.The inclusion or non-inclusion of simulations where no genotype had reached 99% dominance of living cells does not significantly impact the results, this shows that 50000 iterations are sufficient to perform this test.

Size
Results in Figure 7 do not show significant differences, between the size of the genotypes collected in iterations 1000, 5000 and 50000 using the Welsh test at p = 0.05.The genotypes collected at iterations 5 are significantly smaller while those taken at iterations 300000 and 500000 are significantly larger.The average size of the genotypes, selected in iterations 300000, is very close to the maximum of 50 and does not significantly increase at iteration 500000.
11 Size of the genotype and density of its phenotype.David Medernach, Jeannie Fitzgerald, Simon Carrignon, Conor Ryan (2015)  Table 5: Robustness of genotypes collected at different stages of evolution: Age is the number of iterations of the cellular automata after which collection of genotypes took place.The robustness is the proportion of the comparative tests where the tested genotype was the most prevalent.The ending iteration is the average number of iterations before the comparative tests terminated.

Density
Figure 8 shows that phenotypic density is higher for genotypes collected later in the evolutionary process.However there was no significant difference, using the Welsh test at p = 0.05, between iterations 1000 and 5000.Even if it is significant the difference in density, 47% against 48%, is very low between iterations 300000 and 500000 even thougth this is the second longest interval.It should be noted that, for genotypes extracted at iterations 1000, live cells have been completely extinguished before the simulation reaches 2000 iterations 12 .The average density of these same genotypes before extinction is 46% which would rank their density between those extracted at iterations 50000 and iterations 300000.

Discussion
The measures of evolutionary progress that we use here are pretty tough compared to the definition proposed by Shanahan (2012).Not only are the different genotypes collected within the same simulation not necessarily generated from 12 It has also occurred for two genotypes extracted at iteration 1000 and one extracted at iteration 5000.a single evolutionary lineage.But we also collect individuals from different simulations and thus from independent evolutionary processes.Nevertheless, at first sight we detect evolutionary progress in HetCA, between iteration 5 and 500000.There are several periods of stasis 13 during this process but the direction of progress is never reverses and therefore changes in the three traits studied here are directional.Weak or nonsignificant differences between the traits analyzed in iterations 1000 and 5000 could be explained by the fact that there are relatively few evolutionary steps between them.However, a significant difference exists between the genotypes selected at iteration 5 and those selected at iteration 1000 although the number of iterations between them is only 995 iterations.This is likely due to the differences between the typical environment in iterations 5 and 1000 as depicted in Figure 4.At iteration 5, the critical part of the selection is the ability to survive and reproduce as quickly as possible in the "primordial soup" of the early iterations of HetCA, whereas at iteration 1000 the greater part of the cells are in decay and selection is made on the ability of a small group of cells to survive without saturating a reduced space.
The increase in cell density seems quite logical.The density of the phenotype being an obvious evolutionary advantage in HetCA.It is interesting to note that while there maybe periods of stasis in the increase of density, between iterations 1000 and 5000 the values for robustness continue to increase.This demonstrates that possible evolutionary strategies are not limited to density increase.
However, if the three measures used here show directional changes, small modifications of the experimental protocol 13 For robustness and density.David Medernach, Jeannie Fitzgerald, Simon Carrignon, Conor Ryan (2015) Evolutionary Progress in Heterogenous Cellular Automata (HetCA).Proceedings of the European Conference on Artificial Life 2015, pp.512-519 could reverse this trend.Especially if the density measurement was stopped before the extinction of genotypes collected at iterations 5, in which case, they would have had a higher density than genotype collected at iteration 50000 as is shown in Figure 8.It is also quite difficult to assess whether the study period, 500000 iterations is sufficient to observe potential change in trend, and the EP seems to be slowing down in the last 200000 iterations.
The fast increase of genotype size between iteration 5 and 1000 could be the result of differences in size between random genotypes with which we initialize the simulation, those with viable strategies being maybe longer on average than others.We tested this hypothesis in the Table 6, it is checked but does not seem sufficient to explain this difference alone.An additional explication could be genetic drift, because as illustrated in Figure 4 the population, N e , is very small during this interval.

Average Size Proportion
Effective survival strategy 29 0.5% Ineffective survival strategy 25 99.5% Table 6: Survival strategy and size: We randomly generated genotypes then tested, one by one, their ability to survive on 40 × 30 size grids (in simulations without mutations and where all cells are initialized with the same genotype).Genotypes extinct before iteration 5 have not been taken into account; Genotypes extinct before iteration 100 are considered as having ineffective strategy; genotypes still having living cells at iteration 100 are considered using an effective strategy.We performed this test on 100000 genotypes.

Qualitative analysis
The pairwise comparison of the genotypes collected also facilitates some qualitative remarks on the nature of the interactions observed in HetCA.As can it be seen in Figure 5, groups of cells sharing the same genotype are formed very David Medernach, Jeannie Fitzgerald, Simon Carrignon, Conor Ryan (2015) Evolutionary Progress in Heterogenous Cellular Automata (HetCA).Proceedings of the European Conference on Artificial Life 2015, pp.512-519 quickly.This reinforces the hypothesis of cooperation between cells sharing the same genotype.Similarly, the emergence of cells in self-decay at the edge of two clusters is very visible in some simulations as shown in Figure 9.By doing this, those cells do not release space for cells sharing the same genotype and lose their own genotype, so this behavior is very rare and most likely usually counter selected.A hypothesis explaining the selection of such a strategy would be the creation of a barrier of cells in decay to block the progress of a hostile genotype.It is also interesting to note that during robustness testing, the genotype taking advantage early in the simulation is not necessarily the one that will dominate over the long term.This is probably explained by the progressive construction of patterns 14 , as illustrated in Figure 6, and it could make HetCA an interesting model of open-ended evolutionary developmental biology.This reinforces the hypothesis of the existence of complex strategies HetCA, and shows that the survival phenotype changes with development.Figure 10 shows an example of the diversity of evolved strategies where the density of the blue pictured genotype is very low but it appears to be efficiently competing against the genotype in orange by massively propagating inside an orange genotype cluster when contact occurs between these two genotypes.

Further work
We have analyzed the presence of evolutionary progress in a specific category of the population: the most common genotypes.It would be interesting to analyze the potential existence of evolutionary progress in other sub categories, or indeed, in the general population.For example it is possible to hypothesize that genotype lineages using a different strategy combining low density and high robustness have been completely ignored in this analysis.Yet, in natural evolution, so-called complex living beings such as mammals, are much less numerous than prokaryotes as Escherichia coli.Seeking EP in these population groups remains to be done in HetCA.Similarly it would now be interesting to assess the influence of criteria such as the introduction of environmental change 15 on the EP.

Figure 1 :
Figure 1: Genotype transfer at the start of a CA iteration, if cell A is not in Decay it will randomly receive a genotype from any cell shown here in green (Von Neumann (VN) neighboring cell A and cell A itself) that is neither in Decay nor Quiescent.

Figure 4 :
Figure 4: Typical HetCA grid at iterations 5, 1000, 5000 and 5000.Living cells are very frequent at iteration 5; at iteration 1000 most cells are in decay or in a quiescent state; and from there the population of living cells increases until iteration 50000.Color coding is described in Figure 2.

Figure 6 :
Figure 6: Density test.At each iteration the density rho i is measured as the proportion of living cells.It is the proportion of living cells among all the cells.(color coding is described in Figure 2)

Figure 7 :
Figure 7: Size of genotypes collected at different stages of evolution: Iterations is the number of iterations of the cellular automata that occurred before the collection of the genotype, The size is expressed as the number of program statements (n prog ) used in genotypes.

Figure 8 :
Figure 8: Density of genotypes collected at different stages of evolution.5* is the density computed at iteration 5 before extinction of living cells.

Figure 9 :
Figure9: Altruistic decay: The light gray cells are cells in auto-decay, they appear here in areas contested by the two genotypes, helping to create a barrier between them.On the top image, cells are depicted with their current states (color coding is described in Figure2), on the bottom image the repartition of the two genotypes is depicted.Cells that don't have a genotype (in decay or quiescent state) are represented with their current states on both sides.

Figure 10 :
Figure 10: Evolutionary strategy: In these images we can see that cells with blue genotype, despite their low density, are effectively able to eliminate cells with the orange genotype when they are in contact.

Table 3 :
Pairwise comparison of the robustness of genotypes collected at different stages of evolution: The robustness of genotypes is shown under the diagonal of the table, whereas average final iterations are shown above the diagonal.The robustness is the proportion of the comparative tests where the row genotype was more prevalent than the column genotype.

Table 4 :
Pairwise comparison of the robustness of genotypes collected at different stages of evolution without tests reaching 50000 iterations: The fields under the diagonal detail robustness of genotypes whereas, those on top of the table diagonal show the average final iteration.The robustness is the proportion of the comparative tests where the row genotype was more prevalent than the column genotype.Whereas Figure3simulations that reached 50000 iterations are not included.
Evolutionary Progress in Heterogenous Cellular Automata (HetCA).Proceedings of the European Conference on Artificial Life 2015, pp.512-519