Effects of public good properties on the evolution of cooperation

Cooperation is a still unsolved and ever-controversial topic in evolutionary biology. Why do organisms engage in activities with long-term communal benefits but short-term individual cost? A general answer remains elusive, suggesting many important factors must still be examined and better understood. Here we study cooperation based on the secretion of a public good molecule using Aevol, a digital platform inspired by microbial cooperation systems. Specifically, we focus on the environmental and physical properties of the public good itself, its mobility, durability, and cost. The intensity of cooperation that evolves in our digital populations, as measured by the amount of the public good molecule organisms secrete, strongly depends on the properties of such a molecule. Specifically, and somewhat counter intuitively, digital organisms evolve to secrete more when public good degrades or diffuses quickly. The evolution of secretion also depends on the interactions between the population structure and public good properties, not just their individual values. Environmental factors affecting population diversity have been extensively studied in the past, but here we show that physical aspects of the cooperation mechanism itself may be equally if not more important. Given the wide range of substrates and environments that support microbial cooperation in nature, our results highlight the need for careful consideration of public good properties when studying the evolution of cooperation in bacterial or computational models.


Introduction and Background
In recent years, such a complex and sophisticated array of collective behaviors has been observed in microbes that it has created and motivated a rapidly growing new field, "sociomicrobiology" (Parsek and Greenberg 2005;West et al. 2006).Arguably the most interesting among such behaviors is cooperation, which is frequently observed in nature, has been extensively studied theoretically, and in some cases, even experimentally (West et al. 2006).Cooperation in microbes can affect crucial cell processes, such as reproduction (Strassmann et al. 2000;Queller et al. 2003;Fiegna and Velicer 2005), resource sharing (MacLean and Gudelj 2006), biofilm formation (Brockhurst et al. 2006), and motility (Velicer and Yu 2003).Particularly interesting are instances when cooperation is maintained by public good, such as colicin toxins (Le Gac and Doebeli 2009), heavy metal detoxification (Ellis et al. 2007), quorum sensing (Dunny et al. 2008;Czaran and Hoekstra 2009), or triggering of host immune response via self-destruction (Ackermann et al. 2008).In all these cases, bacteria produce a public good, a molecule or a modification of the environment that is beneficial for the entire population but produced by individuals at a cost.Researchers have been especially interested in medical implications of cooperation via public good secretion, as in the case of Pseudomonas aeruginosa infections of cystic fibrosis patients (Paton 1996) because cooperation breakdown would decrease pathogen virulence and could be used as a treatment strategy.
The evolution and maintenance of cooperation has remained an important biological question because it appears to contradict basic principles of natural selection: organisms that help others at the cost of decreasing their own fitness should be selected against.In a mixed population, a nonproducing organism will have a higher fitness than a producing one because it does not pay any of the costs associated with public good creation and secretion.The majority of the currently accepted theories for the maintenance of public good production in microbes in spite of the direct non-producer advantage are a combination of spatial assortment (e.g.environment structure, limited dispersal, viscous environment) and kin selection (Griffin et al. 2004;Diggle et al. 2007), although other explanations are possible (Brockhurst et al. 2008;Kümmerli et al. 2009b;Ross-Gillespie et al. 2009).Simply put, the public good is expected to be maintained when it is preferentially benefiting its producers and their close relatives (Fletcher and Doebeli 2009).This theory has been experimentally tested in the past but with sometimes differing conclusions (Kümmerli et al. 2008;Kümmerli et al. 2009a).There has been significantly less work addressing durability of the public good and the environmental viscosity not just in terms of movement of the individuals but also in terms of the diffusion of the public good (however, see Brown and Taddei 2007;Kümmerli and Brown 2010).In our study we identify and quantify the effects and interactions of public good properties such as the rate of diffusion and degradation on the evolutionary trajectories of cooperative properties.
that enables tracking of large populations of digital organisms over thousands of generations.Aevol resembles other wellestablished in silico experimental systems, such as Avida (Lenski et al. 1999;Misevic et al. 2004;Ofria and Wilke 2004), but its main strength is in the greater attention that is given to the genome structure and encoding (Knibbe et al. 2006;Knibbe et al. 2008;Beslon et al. 2010).It is freely available at www.aevol.fr/downloadand we used the default parameters unless otherwise noted.Aevol has been described in great detail previously (Parsons et al. 2010) and here we highlight only its main properties and new features specifically implemented for the study of cooperation.

The Aevol experimental system
General properties.Aevol individuals mutate, interact with one another, have their fitness evaluated, and are reproduced in a typical genetic algorithm fashion.An individual is represented by its circular genome, a double stranded binary string that can be hundreds of thousands of digits long.There is a complex genotype to phenotype to fitness mapping which we briefly describe here (Figure 1).
Genotype to protein: Pre-determined binary motifs act as promoter and terminator sequences and specify the transcribed regions of the genome.Within these regions, start and stop codons mark the sequences that will be translated into proteins.Each protein's sequence is interpreted as three numerical values (m, w, h), for the mean, width and height of a triangle that represents the protein's phenotypic contribution.For more details on transcription and translation in Aevol, see (Parsons et al. 2010).
Proteins to phenotype: The phenotype is defined as the combination of all the expressed proteins, calculated by adding together the protein triangles.In practice, an organism's phenotype is typically a jagged, piecewise-linear function on the interval (0,1).

Phenotype to fitness:
We define the fitness of an organism as W = e -ag , where g is the geometric area between the organism's phenotype and the target phenotype, and a is a selection pressure constant.The typical target phenotype is the sum of several Gaussians and remains constant over the course of a single simulation.In more general terms, we consider an organism as a collection of traits.In order to represent these traits numerically in a limited number of dimensions, we simplify things by positioning all traits on the continuous trait axis.Each protein primarily affects a single trait (determined by m, the position of the triangle on the trait axis) at a specific level (determined by h, the height of the triangle), but also the neighboring traits (to the extent determined by w, the width of the triangle, representing the pleiotropy of the protein) at a lower level.It is important to note that the same protein can be encoded by different sequences and that neighboring traits are not necessarily encoded by sequences that are similar or close to one another on the genome.
Population spatial structure: A typical Aevol population resembles a well-mixed bacterial population, with organisms having no specific positions in space.However, under the spatial regime we implemented for this study, organisms reside on a rectangular grid with a periodic boundary condition, i.e. a torus.After their fitness is evaluated, Figure 1.Genotype to phenotype to fitness mapping in Aevol.A circular, double stranded genome is schematically represented with black and white squares, corresponding to zeros and ones in Aevol (for clarity, only 100 bases are represented here, much less than the typical genome size).The transcription and translation stages produce the proteins, represented as triangles, which are located on the phenotypic trait axis.The triangles are added together to form the organism's phenotype.Two regions of the trait axis are designated for different functions, metabolism (blue) or secretion (red), and each has a separate target phenotype, here the sum of two Gaussians, as specified in the main text.The gap g (shaded region) between the phenotype and the optimal phenotype is inversely proportional to a metabolic component of fitness (blue, metabolism) or to amount of the public good that the organism secretes (red, secretion).
organisms compete with one another to produce offspring that will populate the next generation.For each position in the grid, organisms in the classical 3x3 Moore neighborhood of the position have a probability (a -1) x a 9-R / a 9 -1 ) of reproducing into this position, where R is the organism's rank in the neighborhood, based on fitness, and a is a selection pressure constant.The organism that actually reproduces is then chosen using roulette selection.During replication, the genome of the new organism experiences a full range of different mutation types at user-set, per-base rates of 10 -5 (point mutation, and small insertions/deletions of up to 6 bases) and 10 -6 (duplications, large deletions, translocations and inversions).
In order to vary the strength of spatial structure in our experiments, at every generation we chose pairs of organisms at random and swapped their location.By increasing the migration parameter (mig, the number of swaps per generation), we can gradually vary the population structure from well mixed (high mig) to perfectly local (mig = 0).

Cooperation in Aevol.
During their lifetime, organisms may secrete a single type of molecule that accumulates in the environment, degrades and diffuses over time, and whose uptake directly affects the organism's fitness.This molecule is the public good that enables cooperation among individuals.To enable organisms to control the level of secretion, we split the axis of phenotypic traits into two sections, metabolism and secretion (Figure 1).The metabolic traits affect fitness directly and organisms evolve to match the metabolic target phenotype.The secretion traits determine the amount of public good an organism secretes into the environment, which is reversely proportional to the gap between the organism's phenotype and the predefined secretion target function.The total fitness of an organism depends on its metabolic fitness, the cost it pays for secreting the public good, and the benefit it gets from any public good molecules already present in the environment.Specifically, the fitness is equal to Wmet x (1 + PG -C x S) where Wmet is its metabolic fitness (calculated the same way as in experiments without secretion), PG is the amount of the public good molecule present in the grid cell that organisms inhabits, C is the cost of secreting a unit of public good, and S is the amount of public good molecule that the organism secretes.Additionally, Wmet = e -aGm and PG = e -aGs , where G m is the gap between the target phenotype and the organisms phenotype for metabolism, G s is the gap between the target phenotype and the organisms phenotype for secretion, and a is a selection pressure constant as before.It is important to note that an organism may not directly benefit from the molecule it secretes, as its amount is added to the environment only after the fitness of the organism is calculated.The benefit may only occur in the following generation making any selection for cooperation indirect.
Once the public good is secreted into the environment, at every generation it diffuses and degrades.We primarily think of diffusion and degradation as being dependent on the properties of the public good molecule itself (e.g. its size, hydrophobicity) but in nature they can also affected by the environmental properties (e.g.viscosity, solubility).The diffusion is controlled by the dif parameter, which specifies what percentage of the public good molecules present in a population grid location will diffuse into each of its eight neighbors in the 3x3 Moore neighborhood.Similarly, deg determines the percentage of the public good molecules present that is degraded at each generation, the public good durability.For example, if 2 units of public good are present, dif =0.05 and deg = 0.2, each of the 8 neighbors will receive 2 x 0.05 x (1 -0.2) = 0.08 units of public good, while the original location will have (2 -8 x 2 x 0.05) x (1 -0.2) = 1.28.Due to the multiplicative nature of diffusion, even at extremely high diffusion levels, its effects are local, less than 2% of the public good reaching grid locations at least three positions away.
To summarize, the Aevol system follows the genetic algorithm heuristic and each generation consists of the following steps: (1) evaluation of the organisms' fitness based on their metabolic proteins and on the amount of public good present in their local environment, (2) secretion of the public good molecule at levels determined by secretion proteins, (3) selection of the organisms that will reproduce, based on their fitness, (4) application of mutations to the new-born organisms, (5) diffusion and degradation of the public good, (6) organism migration, by swapping randomly chosen pairs of individuals.This setup enables us to study the evolution and maintenance of cooperation over thousands of generations.

Experimental design
Given the large number of parameters that could be varied in Aevol, it was computationally nonpermissive to examine all possible combinations of relevant public good properties.Instead, we first focused on establishing a medium level of the cost of public good secretion that enables the evolution and persistence of cooperation in our system.To do so, we performed experiments where the cost of secreting a unit of public good was 0, 0.01, 0.03, 0.1, or 0.3, while diffusion was set to 0.05 and degradation to 0.1.In a second set of experiments, we examined the effects of different levels of migration, diffusion and degradation on cooperation at a given cost level.The secretion cost was set to 0.03, while the other parameters were varied as follows: migration was 0, 100, 300, or 1000, diffusion was 0, 0.01, 0.05, or 0.1, and degradation was 0, 0.01, 0.1, or 0.3.In each set of experiments and for each combination of parameters, we evolved 10 replicate populations of 1024 individuals for 20,000 generations, starting with a different seed for the random number generator.In total, 680 experiments were run for a combined total of 13.6 million generations of evolution.All populations evolved in square toroidal grids and had the same phenotypic target function specified by four Gaussian functions of the form y = H exp( -(x -M) 2 / (2W 2 ) ), where (H, M, W) = {(0.2,0.3, 0.04), (0.3, 0.2, 0.02), (0.2, 0.7, 0.02), (0.3, 0.8, 0.04)}.At each generation, we recorded population averages of metabolic and total fitness, amount of public good secreted by an individual, and amount of public good present in the environment.The statistical analysis was performed using Matlab R2011b.

Secretion cost and the evolution of cooperation
We find that secretion, and thus cooperation, evolves to higher levels when the cost of secreting the public good is lower (Figure 2).In and of itself, this result is not surprising.Indeed, based on the way fitness is calculated in Aevol, a direct tradeoff exists between the cost and benefit of cooperation.However, there are several additional observations that can be made about the results of these experiments.Unlike gametheoretical simulations of cooperation, where cooperation is a discrete trait and each organism is either a "cooperator" or "cheater/defector", here we have a continuum of possible cooperation levels.One could consider that any organisms secreting less than the currently maximum is a cheater.However, given the constantly changing and often increasing maximal secreted amount, we would then have to frequently relabeled cooperators as cheaters, potentially creating much confusion.Instead, we altogether avoid such binary classification and focus on the average secretion level in the population.Although the amount of secretion has not stabilized in our experiments, when we examined individual population trajectories, rather than seeing large amplitude cycles of cooperation/defection, each type taking turns in invading and (possibly) taking over the population, the dominant pattern is one of steady, stepwise increase in public good secretion, with some low level variance.
We further analyzed the dynamics of evolution by examining the diversity of phenotypes within a population and did find multiple types coexisting, but no direct evidence of cycles (Figure 3).However, based purely on diversity data we cannot discern whether independent lineages of individuals with different levels of secretion are coexisting through time or if the lower level secretion repeatedly emerges via mutations.To distinguish between these two possibilities, we performed additional experiments in which we turned off all types of mutations after 10,000 generation of evolution, switching effectively to an "ecological mode", and recorded the amount of public good that was secreted.We analyzed 10 replicates of continued "ecological mode" for 5 different ancestral populations, such as the one in Figure 3.In all cases we obtained qualitatively identical results: within a few hundred generations, a single genotype, that with the highest secretion level, swept through the entire population (data not shown).We conclude that, at least at that stage of evolution, the diversity in the amount of public good secreted is due to a constant supply of mutations and not to ecological interactions.

Migration, diffusion and degradation
Overall effects and interactions.As described in the methods, we performed experiments at several different levels of migration, diffusion and degradation.The evolutionary Table 1.Three-way ANOVA of final average amount of public good secreted by an organism, with migration (mig), degradation (deg) and diffusion (dif) as independent variables.trajectories of the amount of public good secretion are shown in Figure 4.In order to avoid statistical problems of multiple testing, we did not conduct a large number of t-tests to compare different treatments.Such tests would also be inappropriate for comparing trajectories of secretion over time, since these are repeated measurements on the same populations.Instead, to analyze the effect of different public good properties, we conducted a 3-way ANOVA on the average amount of public good secreted by an organism at the end of our experiments, with migration, degradation and diffusion as independent variables (Table 1).We find highly significant effect of all three factors (p !0.0001), indicating that migration, diffusion and degradation all strongly affect the level of secretion and thus cooperation that evolves in our experiment.Additionally, there are statistically significant interactions of migration with both other factors (p = 0.0005 for degradation and p < 10 -4 for diffusion) indicating the existence of complex interaction between these different features of cooperation.We did not find significant interactions between diffusion and degradation, which reflects the generally similar pattern of secretion within a column repeated across different columns in Figure 4, albeit with some scaling.We continue the analysis by examining the effects of each of the cooperation properties in greater detail.
Effect of migration rate.In this set of experiments we used C = 0.03, the cost of public good secretion that results in medium levels of cooperation in previous experiments.We found that an increased migration rate generally leads to lower levels of secretion (Figure 4), in accordance with existing theories on evolution of cooperation (Hamilton 1964;Crespi 2001;Kümmerli et al. 2009a).Effectively, cooperation is maintained because the organisms are more likely to be surrounded by kin when there is little or no migration.
However, there are scenarios under which the migration rate had no effect on secretion.For example, when diffusion is Each column of panels in this figure shows data from experiments with the same degradation rate (0%, 1%, 10% or 30% of the public good present degrades at the end of each generation).Each row of panels shows data from experiments with the same diffusion rate (0%, 1%, 5%, or 10% of the public good present in a gird cell diffuses into each of the 8 neighboring cells).Each like is an average of 10 experiments using the same set of parameters, with different colors representing different migration rates (in each generation 0, 100, 300, or 1000 pairs of organisms were chosen at random and their positions were swapped).The lightly shaded area around the lines represents one standard error of the mean.4).It is not the case that individuals are just secreting at the optimal rate for this set of conditions as the rate is continuously increasing over time.Another possible intuitive explanation is that at these lower levels of degradation secreted public good remains for a longer time, lowering the potential negative effect of migrating away from the public good one just secreted.Further detailed studies of evolutionary paths under specific conditions would be necessary to fully explain this aspect of our results.
Effect and interplay of diffusion and degradation.Our most unexpected and interesting result is the increase in the amount of secreted public good with the increase in degradation rate, across many levels of diffusion and migration (Figure 4).One direct and simple explanation is that organisms are effectively compensating the decrease in the amount of public good at their location by increasing their secretion.In other words, organisms secrete more in conditions where more public good is needed to achieve the same level of benefit via cooperation.We can test this hypothesis by analyzing the data on the average amount of public good that is present in a grid cell over time (Figure 5).We find that indeed, all things being equal, there is less public good at higher rates of degradation, and that the highest secretion was unable to completely compensate and maintain equivalent contribution of cooperation to fitness.Although ultimately not fully successful in maintaining an unchanged amount of public good in the environment, such compensation is still significant and constitutes a novel evolutionary strategy.
A somewhat different pattern emerges when instead of degradation we examine the effect of diffusion on the evolution of cooperation, both in terms of the average amount of the public good that is secreted per generation and of the average amount of public good that is present in the environment at a given time.Secretion increases as the percentage of public good that diffuses to neighboring cells changes from 0 to 1% and 5%, but it seems to decrease (or at best remains unchanged) when we compare the secretion curves for 5% and 10% secretion in Figure 4. Diffusion has a somewhat similar effect to that of degradation in so far as it decreases the amount of public good available locally.However, in case of diffusion, not only do the organisms compensate for the decrease by secreting more, but there is actually a greater amount of secreted compound present.We interpret the initial rise in diffusion as being beneficial to the evolution of cooperation because it modifies the surrounding locations in the population to be more suitable for being inhabited by cooperators -the public good diffusing there will be able to offset at least some of the secretion cost born by the organisms that inhabit these locations in the future.When smaller percentage of the public good degrades, a secreting organisms living in that exact location may compensate for the cost of secretion and survive to reproduce for another generation.In contrast, when public good diffuses to neighboring cells, it is enabling the spread of cooperators into those cells as well.Of course, at high diffusion rates (e.g.5%), the public good may spread faster than the secretor genotype, benefiting unrelated and potentially non-secreting individuals, which in turn lowers the selection for cooperation.
Cooperation without diffusion or degradation.We included two limit cases in our experiments: scenarios where there is either no diffusion or no degradation of public good.While unlikely to occur in nature, these scenarios can be informative about relevant situations beyond the microbial world (e.g.organisms cooperating in producing fixed, large structures, such as termite nests of beaver dams).We find that organisms secrete less or not at all when degradation equals zero, even though their offspring remaining in the same location would receive a greater benefit from its parent's secretion, strengthening the (still indirect) selection for cooperation (Figure 4).Especially interesting is the case where there is neither diffusion nor degradation and little secretion goes a long way: while the amount of secreted public good in the population is higher than in many other cases (Figure 5, note the different scale for the y-axes), by the end of the experiment, there is no significant number of secreting organisms in any of the populations.

Degradation v. consumption of public good.
In Aevol there is no explicit consumption of public good or any cost associated to it.However, given that the organisms occupy every single location in the population grid, we could consider the degradation rate to encompass both the actual decomposition of the molecules and their consumption by the individuals.This interpretation implicitly assumes that the rate of consumption is dependent on the amount of public good molecule present.While it would be interesting to examine the effect of these additional properties of public good systems, due to additional computational resources necessary, they remain outside of the scope of this study.We do not expect that the results would be qualitatively different, but separately considering different decreases in the amount of public good may uncover additional complexities of the evolution of cooperation.

Conclusions
In our study we introduced several new features of the in silico experimental system Aevol that enable the detailed study of the evolution and maintenance of cooperation.We implemented cooperation in spatially structured populations of digital organisms via a public good molecule that can be secreted (at a cost) and that degrades and diffuses over time.
In our experiments we tested the effects of different levels of diffusion and degradation of the public good, cost of secretion, and organism migration on the evolved level of cooperation.We found a complex pattern of interactions between the degree of spatial organization and the physical properties of the cooperation mechanism.Most interestingly, we observed an increase in the secretion of public good at higher rates of diffusion and degradation.In nature many different cooperation systems can be observed, with much research being focused on microbial ones, based on producing and sharing a public good.There are equally many, if not more ways of modeling cooperation dynamics in silico.The main message of our work is that we must carefully consider a greater range of physical properties that characterize both our digital and in vitro/vivo model systems to truly test hypotheses about the evolution of cooperation.Generic models of public good cooperation run the risk of remaining in a specific corner of parameter space, difficult to generalize and apply to natural systems.In our ongoing research we expand and improve the implementation of cooperation in Aevol, by introducing additional features such as multiple public good molecules, evolvable mechanisms for public good consumption, and mobile genetic elements, similar to bacterial plasmids.Each improvement in Aevol and related models continues to bridge the gap between complexity we observe in nature and the complexity we can capture with our computers, allowing us to make the continual advancements in understanding cooperative processes.
Acknowledgments.We wish to thank DevoLab and Lenski Lab at Michigan State University as well as the French National Institute for Nuclear and Particle Physics, Lyon, who provided the computational resources used for the experiments included here.We would also like to thank Guillaume Beslon, Carole Knibbe, the members of the Taddei team and INSERM Unit 1001 for many useful discussions, comments, and advice.We appreciated the careful reading and well-thought-out comments of the three anonymous reviewers who helped us improve the manuscript and provided a number of interesting directions for future research.Our work was financially supported by ANR COOP-INFO grant and EURYI Award to FT.
Contributions.DM conceived and designed the study, performed the experiments and wrote the paper.AF and DP contributed to necessary modifications of the Aevol code and helped with interpreting the findings.FT provided input on the study design, result analysis and interpretation.

Figure 2 .
Figure 2. Trajectories for fitness, metabolic fitness and the amount of public good secreted during evolution at different costs of secretion.Each curve is the average of 10 different replicate experiments and the shaded area represents one standard error of the mean.

Figure 3 .
Figure 3. Frequency of different amounts of secretion in a single population over time.We binned organisms by the amount secreted into 100 equally sized bins between 0 and maximal secretion throughout the experiment and represented secretion by color.For clarity, any bin with less than 10 organisms is shown in white.

Figure 4 .
Figure 4. Effects of migration, diffusion and degradation on the amount of secreted public good over time.Each column of panels in this figure shows data from experiments with the same degradation rate (0%, 1%, 10% or 30% of the public good present degrades at the end of each generation).Each row of panels shows data from experiments with the same diffusion rate (0%, 1%, 5%, or 10% of the public good present in a gird cell diffuses into each of the 8 neighboring cells).Each like is an average of 10 experiments using the same set of parameters, with different colors representing different migration rates (in each generation 0, 100, 300, or 1000 pairs of organisms were chosen at random and their positions were swapped).The lightly shaded area around the lines represents one standard error of the mean.

Figure 5 .
Figure 5. Effects of migration, diffusion and degradation on the amount of public good present in a grid cell over time.As in Figure 4, different colors represent different mutation rates, while columns and rows show data with the same degradation or diffusion rate, respectively.The light shading area around the curves represents one standard error of the mean.Note the different scales on the y-axes in different panels.For clarity, the legend for the entire figure is split between two panels.