Environmental bias forces parasitism in Tierra

Tierra is an iconic ALife system. It has three innovative features: self-optimisation, parasitism, and an uncrashable execution environment. We have identified four sources of bias in the evolutionary dynamics of Tierra. We have run two sets of simulations: one with the original configuration of Tierra 6.02, and one with three of the biases removed. We find that the innovations observed in the original Tierra are preserved, and the debiased configuration is more amenable for future experiments with open-ended evolution.


Introduction
A key part of any evolutionary algorithm is the fitness function.This is used to drive the evolution of the system to deliver a solution to a particular problem.However, if we want a mechanism to evolve the evolutionary process (Evo-Evo), we need to examine the internal function of the system through the lens of evolution.Our approach to this is to find ways to look at the intrinsic fitness of a system: the relative fitness of reproducing systems that exists apart from a specific fitness function.
Many ALife systems have hidden biases encoded within them.One such system is Tierra (Ray, 1991), an ALife system with a long pedigree, feted as an open-ended evolutionary system in 1991, and classified as an automata chemistry in Dittrich et al. (2001).Key innovations in Tierra are the optimisation of replication (self-optimisation) and the emergence of parasitism, within an uncrashable execution environment.Tierra was one of the first computational systems to exhibit increasing complexity, placing it beyond the critical level at which the change in complexity of a self-replicator is degenerative (von Neumann, 1966, p 79-80).Despite ever-more elaborate configurations of the core Tierra framework, however, no breakthrough open-ended Tierra systems have been reported.A networked version of Tierra (Ray, 1996) was intended to enlarge the search by providing a linked Tierran meta-population, but (although it is wrong to assume a negative result if this has not been reported) the system proved difficult to manage, and faltered before any new result was identified.Related work (Ray and Hart, 2000) demonstrated differentiation in a multithreaded version of Tierra, but this involved seeding the system with a hand-coded genome ten times longer than that used in earlier studies.Thus, unbounded evolution has not been reported in Tierra.The system seems to converge to an efficient replicator that tolerates various forms of parasitism.
We hypothesise that implementational biases in ALife environments can drive the populations that inhabit them toward particular biased solutions to the problem of survival, and that this limits their capability to demonstrate continual emergent behaviours.Since several researchers have found biases in the implementation of Tierra (see below), we have an opportunity to test the hypothesis in this system.
An artificial environment is constructed from parameters, variables and mechanisms.These set up a framework in which the ALife organisms must operate.The environment does not evolve, and it sets up a drive to phenotypes that exploit it best.Biases in the environment confer non-evolvable selective advantage on individuals and can reduce the effects of competition between individuals.
Here we explore the idea that the environmental biases in Tierra may play a part in limiting the open-endedness that is observed.This counters the view that the cascade of mutations that often arises in Tierra (parasitism, 'hyperparasitism', optimisation) represent the limits of what can evolve in automata chemistries (Channon and Damper, 1998).We have identified four biases reported in the literature.We name these as: dress 0 (Baugh and McMullin, 2012).
To reveal the effect of the biases on evolutionary dynamics, we examine the Quantitative Non-Neutral (QNN) evolutionary activity (Droop and Hickinbotham, 2012) between the original version 6.02 of Tierra and a version of the same system with all these the biases removed.We find that both parasitism and self-optimisation decrease in configurations of Tierra that have biases removed.

Tierra
For the original overview of Tierra, see Ray (1991).A Tierra simulation consists of a set of individual "programs" existing in a "soup" of memory.An individual "creature" in Tierra is a string of opcodes, a pointer, a set of registers and a stack.The opcodes specify a sequence of computational operations that shift values between the stack and the registers in a manner similar to conventional computers.The sequence of opcodes is called the 'genome' of the individual, and each unique genome is assigned a species identifier code (aaa0080 for example).The individuals compete for processor time, which is allocated via a control structure called a slicer.Another controller, the reaper, determines whether an individual should be deleted or not at each time step.The population of individuals exist together in an environment of memory with a particular fixed size (the 'soup').The system is seeded with a single individual whose genome encodes a program that, when run, creates a copy of the individual.New species that arise by mutation explore the functional space adjacent to their parents, and those that are able to persist do so by exploiting sequences of instructions belonging to neighbouring genomes.The shorter an individual's instruction set is, the more quickly it can be copied.Individuals cannot read-protect their genomes, and so cannot prevent neighbouring creatures from executing their code.Channon and Damper (1998) review a measure of evolutionary activity with respect to Tierra.They observe that the two main results of Tierra, namely parasitism and hyperparasitism, are the result of two bit-flip mutations.This implies that the seed Tierran configuration has a hair-trigger cascade of mutations built into it, and that the emergent properties are close to being 'designed in'.Channon and Damper (1998) are particularly scathing here: "it also shows the ecological results to be somewhat more trivial than we might have first hoped".Standish (2004) explores Tierra in the context of the neutral evolution theory (Kimura, 1983), and concludes that the co-evolutionary pressures in Tierra mean that neutrality is suppressed.Ray and Xu (2001) also measure evolutionary activity in Tierra.They use a variety of measures, but are not wholly satisfied with the ability of any of them to capture evolutionary activity.Most of their statistics are based on the concept of a 'shadow' model (Bedau and Brown, 1999), which is To address this, they have devised a version of Tierra that carries out random substitutions of entire genomes when organisms are created or destroyed, and claim that this results in a system with no evolution.However, the environmental biases in Tierra have not been removed, and thus still influence which creatures in the shadow system are selected for removal.

Biases
In order to clarify our view of biases in Tierra, we have abstracted a "platform model" (Andrews et al., 2010) from the implementation, shown in figure 1.This model provides our basis for arguing about the specific biases in the implementation.It captures the relationships between the components of the system described above.
There is no specification of whether the program of an individual is 'legal' or not, but unless a program is capable of self-reproduction, then it will eventually disappear from the system due to the action of the reaper.Biases occur when the implementation of the model introduces selection pressures that are not part of the model.We detail these below.
Reaper bias: When a new individual is created, a new entry is created at the back of the reaper queue.Individuals whose entry is near the front of the queue are deleted when there is insufficient space in the memory for a new individual.Thus the chance of death increases with age.There are two occasions when an individual's order in the queue is changed.First, when certain instructions are executed in a way not intended by the designers, an error condition is raised, which bumps the individual nearer to the front of the queue, penalising it.Second, two instructions are deemed so difficult to execute that the individual is moved back down the queue if these are executed without error, rewarding it.So the fitness function rewards individuals whose program is 'correct'.Indeed, Harada et al. (2012) have used the Tierra reaper queue as the basis of a genetic program.This emphasises that the reaper queue has bias hard-wired into it.Yet there is no reason why a self-replicating system should not survive in the absence of these pressures, since the speed Simon Hickinbotham, Susan Stepney (2015) Environmental bias forces parasitism in Tierra.Proceedings of the European Conference on Artificial Life 2015, pp.294-301 and accuracy of reproduction should be a sufficient selection pressure in itself.
Substitution bias: There are 32 instructions in Tierra, and each is represented on the genome by 5 bits.When an instruction is copied into a new genome, there is a small chance of mutation, which flips one bit, resulting in a different instruction.The bit-string encoding of instructions is arranged such that codes for similar functions are substituted for one another (Droop and Hickinbotham, 2011;Ray, 1996).Although bit-flips are common ways of carrying out mutation in genetic algorithms, in Tierra this approach introduces bias: there are only 5 opcodes that a single mutation can change a given instruction to in a single mutation (Ray, 1996).This has the effect of restricting the 'adjacent possible' of mutations.This facility is 'designed in' to Tierra (Ray, 1992, p.7), with the intention of emulating the threebase coding of amino acids in DNA.However, the replicators in Tierra are emulations of RNA world (Ray, 1992, p.5), in which translation of codons to peptides is absent.See also Baugh and McMullin (2013) for an alternative approach to implementing translation in Tierra.Since the manner of mutation is such a critical design issue, we argue that, in the absence of evidence to the contrary, it is essential that a single mutation can change one instruction on the genome to any other.This can be achieved in Tierra by setting parameter MutBitProp = .0.(In debiased Tierra, any other value of this parameter will have no effect.) Length bias: The number of mutations per opcode changes as a function of the average length of the individuals in the soup (Droop and Hickinbotham, 2012).Thus, the mutation rate of the individual is dependent not just on its own properties, but also on a global property of the population.When the average length of individuals in the population is lower, there is less mutation in the system, and so it is more likely to persist.It is difficult to predict analytically the dynamics that this bias engenders, but, since parasites tend to be shorter and more numerous than genuine self-replicating individuals, this bias may favour the emergence of populations of parasites.Parasite genomes can be as little as 12 opcodes long whereas the smallest self-replicator has a length of 23 opcodes.
Zero-address bias: Baugh and McMullin (2012) have implemented von Neumann's self-reproducing automata (von Neumann, 1966) within Tierra.The resulting distributed self-reproducing system exposes a bias in the implementation of two opcodes, 'jump' and 'return' that were not observed when using a population of self-reproducing 'RNA-world' genomes.Those authors have found that their system tends to collapse to a situation where a 'pathological constructor' exists at address 0, which spawns offspring whose function returns their instruction pointer to the program of the parent, thus effectively giving the parent multiple CPUs to execute its code.In order to preserve the selfreproducing automata architecture, they changed the func-tion of these opcodes so that their behaviour was not biased towards shifting the instruction pointer to address 0.

Measuring Evolutionary Activity
We are setting out to test the hypothesis that removing biases in Tierra has an influence on the evolutionary activity in the system.One problem with making this link is that the phenomenon of evolvability, also known as evolutionary activity, or the rate of evolution, is vague and difficult to measure quantitatively.Here we recap the measure of evolutionary activity first presented in Droop and Hickinbotham (2012), which gives a numeric summary of evolutionary activity and thus allows different configurations to be compared.
The reasoning behind the measure is as follows.It is assumed that a species demonstrating some new beneficial adaptation will exhibit a rapidly increasing cohort size.By contrast, a neutrally-drifting species will not show any rapid increase in cohort size.Therefore by creating a measure of cohorts that are rapidly increasing, we can detect evolutionary activity.
Let c i t be the number of molecules of species i a timestep t.The total cohort size at timestep t is: The proportion of species i at timestep t is: The expected proportion of species i at timestep t is the proportion observed at the previous timestep: The activity of species i at timestep t is defined to be the square of the excess of observed over expected proportion, scaled by cohort size: This definition emphasises large positive increases in cohort size for each species, particularly where this occurs within a large cohort.The total non-neutral activity A Q of the simulation is the sum of each species activity at each timestep: The measure A Q has two advantages.First, it is quantitative: it delivers a numerical measure which allows systems to be compared.Second, it measures non-neutral evolutionary activity without the need for an explicit model of neutral Simon Hickinbotham, Susan Stepney (2015) Environmental bias forces parasitism in Tierra.Proceedings of the European Conference on Artificial Life 2015, pp.294-301 evolution.The measure is called Quantitative, non-neutral (QNN) evolutionary activity.Note that QNN is applicable to any systems where changes in cohort over time can be measured.An implementation of the measure in the R programming language is available from the author's website 1 .

Debiasing Tierra
We used Tierra 6.02 as our reference implementation, with two changes to the source code.First, we removed a 'divide by zero' error that occurs when the program is run on a modern processor.This happens because the time units in the original version assume that one slice of time would take more than 1 second to execute.Second, we added functionality to output population dynamics data that was amenable to analysis with our QNN software library.Neither of these changes can influence the dynamics of the system since they are merely diagnostic constructs.
We created a second "debiased" implementation of Tierra, from this reference implementation, that removes three of the biases detailed above.This has been achieved as follows: Reaper bias: We removed all calls to the 'UpRprIf()' and 'DownReperIf()' functions, leaving the order of individuals in the reaper queue ranked solely in order of the time they were created.
Substitution bias: We changed the bit-flip mutation function 'mut site' to produce a random sequence of 5 bits whenever mutation is specified.This has the effect of randomly selecting an opcode, instead of selecting an opcode from one of the five operators with a Hamming distance of one from the instruction being copied. 2  Zero address bias: The authors of (Baugh and Mc-Mullin, 2013) supplied us with a patch to correct the zeroaddress biases in the 'jump' and 'return' instructions.If the 'jump' instruction has no template, it will move the instruction pointer to the address held in the Bx register.If a new creature has not yet written anything to Bx, then Bx will be zero, and the Instruction pointer will move to address 0. Similarly, the return instruction will also move the pointer to zero if the stack has not been used.In the debiased version, the 'jump' instruction acts like a 'nop' (no operation) instruction if there is no address template.The memory in Tierra is not circular, and Tierra does not allow individuals to span the boundary between the end of memory and the beginning.This means that it is impossible for a program to execute a set of functional instructions that would require a 'return' to address 0. Thus it is possible to specify that any call to return the instruction pointer to address 0 acts like a 'nop', simply incrementing the instruction pointer to the next address in the program.
The original configuration of Tierra uses a range of different mutation mechanisms.An investigation of their effect is 1 see http://www-users.cs.york.ac.uk/ ˜sjh/software 2 This could also have been achieved by setting MutBitProp = .0.
in Droop and Hickinbotham (2012); these are left at default values in this contribution.Removing the individual length bias from each of these will be the subject of future work.

Experiments
We used the original Tierra 6.02 'soup in' file to configure both systems, but made some changes to the reporting and termination parameters: BrkupSiz = 0; DiskOut = 1; GeneBnker = 1; alive = 10000.This ensured that we could determine the ability of each configuration to self-maintain for long periods.
We ran 100 simulations of both configurations of the system.The standard Tierra configuration completed the runs in two weeks on a cluster, whereas the debiased system completed the same number in only one week.Although it is not possible to check why there was such a large discrepancy, one explanation would be that the debiased system ran template-matching instructions (jump, call and adr, which are among the most processor-intensive instructions in the set) more quickly, since matches were generally found that were nearer to the instruction.
One of the features of Tierra is the massive diversity that the system generates, partly due to each creature's potential to read and execute (but not overwrite) the genomes of other individuals without any spatial or binding constraints (parameter MemModeProt = 2).This diversity is challenging when interpreting the results of the simulations.In the following, we have looked at the entire cohort of species in the soup, and at a subset of species whose population rose above 50 individuals in at least one of the time steps: c i t > 50 for some t.We label these groups all species and dominant species respectively.Having generated the data, we used R scripts to calculate the QNN activity for each simulation, along with other plots described below.

Results
In this section, we describe numerical outcomes of our analysis.We offer an interpretation of them in the next section.We found that in runs with the original configuration, seven of the 100 runs ceased replicating (i.e.making successful calls to the divide operator) and were terminated before they reached the 10 10 instruction limit, whereas in debiased Tierra this happened only three times.This demonstrates that debiased Tierra is no less stable than Tierra 6.02 for at least 10, 000 time steps.
Figure 2 shows the distribution of QNN activity for the two configurations of Tierra for all species, and for dominant species.The raw data is shown in red, with boxplots and beanplots (Kampstra, 2008) underlaid to emphasise the distribution of QNN.Looking at all species (left), Tierra 6.02 shows a non-uniform range of behaviours with three modes of QNN activity visible in the beanplot.Debiased Tierra shows less diverse behaviour, and overall less evolutionary activity.This picture is markedly different if we plot QNN Simon Hickinbotham, Susan Stepney (2015)   for the dominant species (right).Here the distribution of QNN for debiased Tierra is generally higher than for Tierra 6.02.The QNN score is generally higher for the dominant species because the species proportion p for each genome type is higher if only dominant species are considered (see equation 2).
We plot the number of different species present in four simulations for each configuration of Tierra in figure 3. A feature of most simulations is that there is a period of organisation and optimisation in the system up to around 1, 500 time-steps.During this period, the population moves from a monoculture of the initial seed species to an ecosystem of different species.In both simulations, the runs with the minimum QNN score (fig.3(a)) terminate early due to extinction of (self-) replicating entities.The medianand maximum-scoring runs are plotted in fig.3(b) to (d).Tierra 6.02 tends to maintain a population with more species overall.The number of dominant species diminishes from around t = 2000.The median-scoring run for Debiased Tierra shows more volatile dynamics than the other runs.It appears to show periods of more species at around 3, 000 and 5, 000 time-steps.
An alternative way of examining the species in Tierra is to group species by genome length, as shown in figure 4. Debiased Tierra tends to produce more lengthy genomes than does Tierra 6.02.There is a greater diversity of dominant species lengths in Tierra 6.02 than in Debiased Tierra.We have examined species present at the end of each run for both configurations of Tierra.A full analysis was beyond the scope of this study, but upon brief inspection we found examples of shorter genomes that tend to behave as parasites, co-existing with longer genomes that behave as selfreplicators.The organisation of genomes followed the patterns set out in Ray (1992).At the end of all runs, parasites are present in all configurations, but dominant species (shown in red) of parasites tend to be driven to extinction.
Our final analysis of the outputs of the simulation is to show changes in population sizes, figure 5.It is clear that debiased Tierra operates with fewer individuals overall, and fewer dominant species.Row (d) shows the maximumscoring runs for both configurations.Tierra 6.02 shows distinct epochs in the run, where the population size changes, due to a change in the distribution of genome lengths.

Discussion
Our interpretation of the results just described is as follows.
Genome Length: The distribution of genome lengths is different between the two configurations of Tierra under study.Genome lengths tend to be longer in debiased Tierra.This is shown most clearly in figure 4: there are many more genomes that are longer (grey regions), and the dominant species also tend to have longer genomes (red regions).Since both version include length bias, we suggest that the reaper queue bias works in favour of shorter genomes: since they will replicate more quickly, they will be moved down the reaper queue more often than longer genomes.
Fixed memory size: In both configurations examined here, the memory (or 'soup') could hold 60, 000 instructions.Since longer genomes require more memory, populations of longer genomes tend to be smaller than populations of smaller genomes.This is illustrated in figure 5, where the total populations (black) are lower for debiased Tierra, particularly for runs with higher QNN (rows (c) and (d)).
The effects of a fixed memory size are echoed in the QNN scores for all species in figure 2, where Tierra 6.02 displays more evolutionary activity than debiased Tierra.The lower total population sizes for debiased Tierra limit the evolutionary activity.However, if we look only at the dominant species, figure 2(b), we see that several debiased Tierra runs demonstrate more innovation than any Tierra 6.02 run even though there are fewer individuals in the system.
Number of species: Debiased Tierra tends to maintain a smaller number of dominant species.By contrast, many of the runs in Tierra 6.02 display a slow reduction in the num-Simon Hickinbotham, Susan Stepney ( 2015  Figure 5: Four selected runs for the two configurations of Tierra, plotting population size of all species (black) and dominant species (red) (y-axis), against Tierran time-steps, or 10 6 instruction executions (x-axis).
ber of dominant species after t = 2, 000.These results tally with the notion that genomes in Tierra 6.02 endure more parasitism, allowing mutants to emerge with low population sizes.The decline of dominant species (red lines) in figure 3 might simply be due to increasing diversity in all systems, indicating an environment where it is impossible for a particular species to dominate.Fault Tolerance: It can be argued that manipulating the reaper queue to remove genomes that are using instructions for the purpose which they were designed endows the system with a form of fault tolerance.Interestingly, parasitic genomes operate without raising any error flags.Reaper queue manipulations act as a fitness function, biasing the system towards using instructions in the manner for which they were designed.However, after removing reaper bias, fault tolerance has increased.Evidence for this is: fewer runs terminate early; longer, less efficient genomes persist; total population is stable.Investigations into changes in diversity in debiased Tierra will be the subject of future work.
Evolutionary activity: Despite the increase in QNN activity for debiased Tierra over Tierra 6.02, these experiments could be interpreted as a negative result for debiased Tierra, since no new emergent behaviour evolves.However, we note that the debiased configuration reduces the selection pressure for short genomes, allowing recovery from, or resistance to parasitism to emerge more frequently and more strongly.In addition, there is no reduction in survival of a self-replicating cohort.We suggest that the main result here is that the innovations observed in the original Tierra are preserved, and the debiased configuration is more amenable for future experiments with open-ended evolution.

Further work
These experiments have addressed some issues with Tierra that have been reported in the last quarter of a century.In this paper we have studied whether a debiased system would yield measurable differences: it does.Having established this, next we need to determine which of the biases has the most effect.Our analysis revealed that the biases placed selective advantage on parasitism and shorter genomes due to the fixed memory size of the environment.We have suggested mechanisms by which length bias and reaper bias could effect the dynamics of Tierra 6.02.We can offer no interpretation of the effect that the other two changes we made, namely removing substitution bias and zero-address bias.Further work is needed to investigate their effects.Indeed, the effects of these biases may only be revealed with a different self-replicating cohort of Tierran species, as shown in the work of Baugh and McMullin (2013).
Even though Tierra has some tools for analysis of runs, the analysis of Tierran genomes is a time-consuming affair, and it was not possible to complete an analysis of the interactions between all the dominant species in all 100 runs of each configuration for this study.Tools to automate this analysis offer the potential to help the search for innovation in automata chemistries generally.
Even in a debiased configuration, it might not be possible for longer genomes to generate as much QNN as shorter ones, simply because fewer of them can fit in the memory: they cannot possibly increase in numbers as much as shorter ones can.This is a problem with having a fixed carrying Simon Hickinbotham, Susan Stepney (2015) Environmental bias forces parasitism in Tierra.Proceedings of the European Conference on Artificial Life 2015, pp.294-301 capacity of the soup.
Restricting the carrying capacity ensures that there is never an unbounded population explosion in Tierra runs.There are alternative means of preventing such an explosion.For example, we have recently experimented with conservation of matter in the Stringmol automata chemistry (Hickinbotham and Stepney, 2015), and shown that it can limit population size whilst having a positive effect on evolutionary activity.
Finally, it may be possible to use Tierra in a framework where we use QNN as a fitness function to evolve a population of Tierran systems with increasing evolutionary activity.Tierra is particularly amenable to this, since it automatically records its terminal configuration in a text file called 'soup out' that can be used as the input to a new simulation.This approach has analogies with the work of Gutierrez et al. (2014), where an activity measure was applied to a chemical system to evolve novel configurations of catalysts.
The original Tierra tackled many problems at once, so it is unsurprising that there are a few artefacts in the design that merit improvement.It is difficult to determine what the best improvements are without a numerical measure of evolutionary activity.With QNN to hand, we have been able to test our hypothesis regarding biases in the original configuration.This approach opens the door to exploring and comparing the respective merits of a range of automata chemistries, with the goal of refining and consolidating the designs in order to address the problem of generating openended evolution.

Figure 1 :
Figure 1: Platform Model of Tierra

(
Environmental bias forces in Tierra.Proceedings of the European Conference on Artificial Life 2015, pp.294-301

Figure 2 :
Figure 2: Beanplots plus boxplots of the distribution of QNN values (y-axis) for 100 runs of Tierra 6.02 and debiased Tierra, for (a) all species and (b) dominant species.Note the different scaling of the y-axis for figures (a) and (b).
) Environmental bias forces parasitism in Tierra.Proceedings of the European Conference on Artificial Life 2015, pp.294

Figure 3 :Figure 4 :
Figure 3: Four selected runs for the two configurations of Tierra, plotting the total number species (black) and the number of dominant species (red) (y-axis), against Tierran time-steps, or 10 6 instruction executions (x-axis).(a) run with the minimum QNN activity score (9.1 for Tierra 6.02 and 8.5 for Debiased Tierra);(b) run with QNN ⇡ 46, the median score for the dominant species of Tierra 6.02; (c) run with QNN ⇡ 53, the median score for the dominant species of debiased Tierra; (d) run with the maximum QNN activity score (89.5 for Tierra 6.02 and 187.7 for Debiased Tierra).