Exhaustive exploration of Prisoner's Dilemma Parameter space in one-dimensional cellular automata

Exhaustive exploration of Prisoner's Dilemma Parameter space in one-dimensional cellular automata

Marcelo Alves Pereira^I; Alexandre Souto Martinez^II; Aquino Lauri Espíndola^III

^IDepartamento de Física e Matemática - FFCLRP - USP Av. Bandeirantes, 3900, 14040-901 Ribeirão Preto, SP, Brazil, [e-mail]

^IIDepartamento de Física e Matemática - FFCLRP - USP Av. Bandeirantes, 3900, 14040-901 Ribeirão Preto, SP, Brazil, [e-mail]

^IIIDepartamento de Medicina Social - FMRP - USP Departamento de Física e Matemática - FFCLRP - USP Av. Bandeirantes, 3900, 14040-901 Ribeirão Preto, SP, Brazil [e-mail]

ABSTRACT

The Prisoner's Dilemma (PD) is one of the most popular games of the Game Theory due to the emergence of cooperation among competitive rational players. In this paper, we present the PD played in cells of one-dimension cellular automata, where the number of possible neighbors that each cell interacts, z, can vary. This makes possible to retrieve results obtained previously in regular lattices. Exhaustive exploration of the parameters space is presented. We show that the final state of the system is governed mainly by the number of neighbors z and there is a drastic difference if it is even or odd.

Keywords: Prisoner Dilemma; Emergence of Cooperation; Game Theory; One-dimensional cellular automata; Nonequilibrium phase transition

I. INTRODUCTION

Due to the emergence of cooperation between competitive rational players [1-4], the Prisoner's Dilemma (PD) [5, 6] is one of the most popular games of the Game Theory [7]. When it is played repeatedly, one has the Iterated Prisoner's Dilemma [8, 9]. If the PD is played in a group of players with spatial structure, this version is known as Spatial Prisoner's Dilemma (SPD) [10]. These spatial structures may generate chaotically changing spatio-temporal patterns. Cooperators and defectors coexist, and cooperator proportion oscillates indefinitely. This occurs when each player interacts with the nearest neighbors, for instance, in a square lattice. Moreover, adding the interaction with the next nearest neighbors (corresponding to the chess king possible moves) the spatial patterns are smoother. During the game, cooperators and defectors organize themselves in clusters. The most interesting dynamics occurs on the borders of these clusters, causing the oscillating behavior of the proportion of cooperators.

The final proportion of cooperators and defectors in the chaotic phase depends on the initial configuration and the magnitude of the parameter T (temptation). Moreover, the connectivity among players also plays an important role in the dynamics of the clusters [11]. Studies about PD had been carried out in different topologies such as square lattice [10], graphs [12] and also in complex networks as random graphs [11], scale-free networks [13], small-word networks [14]. We have used the simplest lattice topology, i.e. one-dimensional lattice to represent regular lattices at any dimensionality [15]. The computational implementation of PD in the one-dimensional case is simpler than in other topologies, and it requires less computational time to run the numerical codes. In one-dimensional cellular automata, it is simpler to understand the way that oscillations in the cooperator proportion take place [15]. Beyond the topologies, it is also possible to consider the mobility of players [16].

In this paper we present an exhaustive exploration of the parameter space for the IPD in the one-dimensional cellular automata with a variable number of interacting neighbors. After introducing the model in Section , we show the results in Section . Final remarks are presented in the Section IV.

II. THE MODEL

Consider a cellular automaton in a one-dimensional lattice, with L cells, where each cell represents one player, who has two possible states: q = 1 (q = 0) for cooperator (defector). The automaton has no empty cells, so the cooperator proportions, r_c(t), and defectors, r_d(t), leads to r_c(t) + r_d(t) = 1. The initial proportion of cooperators, r_c(0) = r₀, 0 < r₀< 1, is an important parameter. The state of Lr₀ players, which are chosen randomly by a uniform distribution, are set as cooperators and the remaining ones are set as defectors. The neighborhood of the i-th player is defined by z = (1, 2, ..., L). If z is even, there are a = z/2 adjacent interacting players to the right and to the left hand side of this player. If z is odd, each side has a = (z-1)/2 players and player i interacts with his/her own state (self-interaction) [15, 19, 20]. In addition to r₀ and z the other free parameter in this model is the temptation T in the conflict range 1 < T < 2.

Consider two players i and j playing the PD. The payoff of player i due to interaction with player j is given by = q_i q_j + T(1-q_i q_j)q_j, where q_j is the state of player j, with j = (1, 2, ..., L). The total payoff, P_i, of player i is: P_i = , , where is the neighborhood of the i-th agent. Since the payoff of each player depends on z, the macroscopic regime, r_c, also depends on it. Player i will compare P_i to P_k, where P_k is the payoff of k = (1, 2, ..., z) set of players. If P_i< P_k, player i copies the state of the player with the highest payoff, otherwise player i does not change his/her current state. The dynamics of the model is totally deterministic. This strategy of copying the state of the neighbor that had the highest payoff is the Darwinian Evolutionary Strategy. Others evolutionary strategies can be adopted, like the Pavlovian one [21]. The states of the players are updated synchronously and they play until the system reaches a stationary or dynamical equilibrium regime.

The cooperator proportion, r_c(t,T,r₀,z), depends on time, temptation, initial proportion of cooperators, and the number of interacting players. The dependence of r_c as a function of r₀ and z is commonly neglected, possibly due to the fixed lattice restriction in a d-dimensional space.

The asymptotic cooperator proportion, r_¥(T,r₀,z), is obtained when the system reaches the steady state, which represents the final phase for the set of parameters (T,r₀,z). The dependence of r_¥ on z can be understood due to the number of interacting cooperators, c, with 0 < c < z, in the neighborhood of each player. When player i interacts with c_i cooperators out of z neighbors, his/her payoff is [11, 19]: (q_i) = [T-(T-1)q_i]c_i. Some useful relations follow immediately: for a cooperator (1) = c, while for a defector (0) = cT. For T > 1, (0) > (1) and (q) > (q). Transitions in r_¥(T) occur when temptation crosses threshold values. In the conflict range, 1 < T < 2, these transitions are controlled by [11]: T_c(n,m) = (z-n)/(z-n-m), where 0 < n < z and 1 < m < int[(z-n-1)/2)] are integers.

III. RESULTS

We have used a one-dimensional cellular automaton with L = 1,000 cells, with L r₀ cells set as cooperators and the remaining ones as defectors. The asymptotic cooperator proportion, r_¥, is obtained from the mean values of an ensemble of 1,000 configurations for the same initial parameters. The parameter T increases in steps DT = 0.01 in the range 1 < T < 2 and r₀ increases in steps Dr₀ = 0.1 and the intermediate values are linearly interpolated.

It could seem meaningless to consider T = 1.00, as the cooperators and defectors have the same payoff, when one plays against the other. However, the total payoff of each player depends on the neighborhood, then, if the player belongs to a cooperative cluster he/she has a higher payoff than the player from a defective one. In the cooperative/defective clusters border, the differences among payoffs are essential to determine the system dynamics [15].

Results for r₀ = 0 and r₀ = 1 are the trivial cases due to the Darwinian Evolutionary Strategy. In a population of cooperators (defectors) it is not possible to emerge a defector (cooperator), because the players only can copy the states of their neighbors. Mutations are not allowed in our model, i.e. the noise of the system is null [22].

Our results are equivalent to those obtained in the square lattices, which are briefly reviewed in the following. Consider four scenarios. First, defectors can dominate the system and determine the complete extinction of cooperators, leading the system to a defective phase (r_¥ = 0). Second, defectors can increase and domain the system, but cooperators are not extinguished, resulting in a defective phase as well (0 < r_¥ < 0.5). Third, cooperators may domain the system forming a cooperative phase (0.5 < r_¥ < 1.0). And finally, cooperators can extinguish the defectors determining a cooperative phase (r_¥= 1.0).

Figures 1a and 1b show the surface of r_¥ plotted as a function of T and r₀, for z = 8 (without self-interaction) and z = 9 (with self-interaction), respectively. Differences between the presence/absence of self-interaction are clear. The region of low values of T and high values of r₀ is a cooperative phase. The region of high values of T and low values of r₀ is a defective phase. The other two regions, low T and low r₀ or high T and high r₀, the value of r_¥ is different and depends strongly whether z is even or odd. The valleys for r₀ ~ 0.9 are due to the system dynamics. For higher values of r₀, the defective clusters formed are tiny in comparison to the cooperative ones. These defectors exploit theirs cooperator neighbors, but they neither do not invade the cooperative cluster nor are extinguished by the cooperative neighborhood during time evolution.

Another visualization of r_¥ for z = 8 is given in Figures 2a and 2b, and for z = 9 in Figures 2c and 2d. It is equivalent to observe the phase diagram plotted as surface in Figures 1a and 1b from the top view. The images 2b and 2d are the standard deviation of r_¥ due to statistics to avoid the initial configuration dependence. Figures 2b and 2d, show very high values of standard deviation. In these regions, small changes in the initial configuration drastically modify r_¥ from a cooperative phase, r_¥ > 0.5, to a defective phase, r_¥< 0.5. Thus, in this region, it is not possible to define the system as cooperative or defective, and this region is considered as the coexistence of cooperative/defective phases. In other words, the chaotic phase. The inclusion of self-interaction implies in larger cooperation area in the phase diagram as shown in Figures 2a and 2c. This means that cooperation prevails when self-interaction is included. In Figure 2a, r_¥ drops abruptly for T > 1.7, this rapid decay does not occur in Figure 2c, because self-interaction shifts T_c to higher values. In Figure 2d, the higher values of standard deviation fulfill a larger area, especially for T > 1.7, for the same reasons. When T_c is shifted, a region that should be cooperative, when the self-interaction is present, becomes defective in the absence of the self-interaction.

The slice r_¥ r₀ of Figure 1 shows r_¥ as a function of r₀. The curves are plotted for T = (2.0, 1.8, 1.6, 1.4, 1.2, 1.0), in Figures 3a for z = 8, and Figure 3b for z = 9. The value of r_¥ increases in presence of self-interaction in the region 0 < r₀ < 0.4, for 1.0 < T < 1.4. Self-interaction also shifts the emergence of cooperation to lower values of r₀, when compared to a system without self-interaction. In Table I, one sees the values of r₀, where r_¥ > 0.5 occurs for the first time for different values of T. Notice the strong difference concerning the parity of z.

The r_¥non-monotonous behavior for intermediate values of r₀ presented in Figures 1 and 3, in the region 1.3 < T < 1.5 and 0 < r₀ < 0.5 are due to the coexistence phases. In this region, the standard deviation of r_¥ is higher than in the remaining regions.

To observe the behavior of r_¥, when z increases, see the surfaces of r_¥ for z = 20, in Figures 4a, and z = 19, Figure 4b. Comparing Figures 1a and 1b, one observes that if z is increased, the surfaces become more similar.

Figures 5a and 5c are the top view of the r_¥ surfaces for z = 20 and z = 19, respectively. They show the convergence of r_¥ for even and odd z. Figures 5b and 5d are the r_¥ standard deviation for z = 20 and z = 19, respectively. A relevant difference between even and odd z is that the cooperative phase persists for T < 1.1 in the range of 0.1 < r₀ < 0.5 in the presence of self-interaction (see Figure 5c). If self-interaction is present the shift in T_c to higher values remains in higher values of z.

A slice of the plane T r₀ of the r_¥ surface in Figures 1a and 1b at r_¥ = 0.5, may represent a phase diagram. In Figures 6a and 6c, the contours separate the cooperative/defective phases, i.e. the phase-diagram. Figures 6b and 6d are the contours that take into account the standard deviation. Since there is the phase coexistence, these contours separate the cooperative/coexistence/defective phases.

In Figures 7a and 7b, there are the contours of the cooperative/defective phase for different z values. When z increases the contours converge to the same pattern independently if z is even or odd as shown in Figure 6c and 6d. For small z values, the z parity generates remarkable differences in the contours, if z increases, the contours converge and present a similar form and the phase coexistence region is narrower than for small z values.

IV. CONCLUSION

The Prisoner's Dilemma in the one-dimensional cellular automata yields results according to the results obtained previously for regular lattices in d dimensions. The exhaustive exploration of the parameter space allows us to observe that the parameter z plays the main role in the dynamics. For low z values, the influence of self-interaction is remarkable. Some studies about the PD with variable coordination number, i.e. the neighborhood size z, have been carried out. However, these studies adopt lattice topologies that are different from the one-dimensional lattice used here, e.g. square lattice [25], complex networks as random graphs [11], scale-free networks [25, 26], small-world networks [14]. Another difference in comparison to these studies is that the state update of the players is asynchronous [25, 26], but in our case is synchronous. Despite these differences, the main features due to the z variation remain, such as the dependence on the asymptotic cooperator proportion on the neighborhood size. Our results are similar to those obtained by Durán and Mulet [11] considering the neighboorhood with self-interaction (odd z). Comparing our results to those found in the literature, it is possible to see that the way the connection among the players is settled plays another important role in this problem independently of the space dimensionality or network structure.

For intermediate values of T and r₀ the chaotic phase occurs. In the chaotic phase the outcome r_¥ can belong to the cooperative or defective phase due to only a small change in the initial distribution of the cooperators.

Acknowledgments

M. A. P. would like to thank CAPES for the fellowship. A. S. M. acknowledges the agencies CNPq (305527/2004-5) and FAPESP (2005/02408-0) for support. A. L. E. would like to thank CNPq for the fellowship and FAPESP and MCT/CNPq Fundo Setorial de Infra-Estrutura (06/60333-0) for the financial support.

Received on 29 October, 2007

• [1] D. Stauffer Physica A 336, 1 (2004).
• [2] J. P. Bouchaud Physica A 313, 238 (2002).
• [3] C. Anteneodo, C. Tsallis, and A. S. Martinez Europhys. Lett. 59 5, 635 (2002).
• [4] P. E. Turner, L. Chao Nature 398, 441 (1999).
• [5] M. Dresher, The Mathematics of Games of Strategy: Theory and Applications (Prentice-Hall, Englewood Cliffs, NJ, 1961).
• [6] W. Poundstone, Prisoner's Dilemma (Doubleday, New York, 1992).
• [7] J. Neumann and O. Morgenstern, Theory of Games and Economic Behavior (Princeton University Press, New York, 1947).
• [8] R. Axelrod and W. D. Hamilton, Science 211, 1390 (1981).
• [9] R. Axelrod, The evolution of cooperation (Basic Books, New York, 1984).
• [10] M. A. Nowak and R. M. May, Nature 359, 826 (1992).
• [11] O. Durán and R. Mulet, Physica D 208, 257 (2005).
• [12] E. Lieberman, C. Hauert, and M. A. Nowak, Nature 433, 312 (2005).
• [13] Z. Wu, J. Guan, X. Xu, and Y. Wang, Physica A 379, 672 (2007).
• [14] G. Abramson and M. Kuperman, Phys. Rev. E 63, 030901 (2001).
• [15] M. A. Pereira, A. S. Martinez, and A. L. Espíndola, arXiv:0708.3520v1 [physics.comp-ph] to appear in Int. J. of Modern Phys. C.
• [16] M. H. Vainstein, A. T. C. Silva, and J. J. Arenzon J. Theor. Biol. 244 (4), 722 (2007)
• [17] S. Wolfram, Science 9, 2 (1983).
• [18] M. A. Nowak and R. M. May, Int. J. Bifurcat. Chaos 3, 35 (1993).
• [19] R. O. S. Soares and A. S. Martinez, Physica A 369, 823 (2006).
• [20] D. Alves, P. R. A. Campos, A. T. C. Silva, and J. F. Fontanari, Phys. Rev. E 63, 011911 (2000)
• [21] H. Fort, S. Viola J. Stat. Mech. 1, P01010 (2005).
• [22] A. Mukherji, V. Rajan, and J. R. Slagle, Nature 379, 125 (1996).
• [23] G. Hardin Science 162, 1243 (1968).
• [24] M. Gardner Scientific American 223, 120 (1970).
• [25] M. Ifti, T. Killingback, and M. Doebeli J. Theor. Biol. 231, 97 (2004).
• [26] F. C. Santos and J. M. Pacheco Phys. Rev. Lett. 95, 098104 (2005).

Publication Dates

• Publication in this collection
  04 Apr 2008
• Date of issue
  Mar 2008

History

• Received
  29 Oct 2007