Cooperate or Not Cooperate in Predictable but Periodically Varying Situations? Cooperation in Fast Oscillating Environment

Abstract In this work, the cooperation problem between two populations in a periodically varying environment is discussed. In particular, the two‐population prisoner's dilemma game with periodically oscillating payoffs is discussed, such that the time‐average of these oscillations over the period of environmental variations vanishes. The possible overlaps of these oscillations generate completely new dynamical effects that drastically change the phase space structure of the two‐population evolutionary dynamics. Due to these effects, the emergence of some level of cooperators in both populations is possible under certain conditions on the environmental variations. In the domain of stable coexistence the dynamics of cooperators in each population form stable cycles. Thus, the cooperators in each population promote the existence of cooperators in the other population. However, the survival of cooperators in both populations is not guaranteed by a large initial fraction of them.

where p i , i = 1, .., N (q j , j = 1, .., M) represents the frequency of type i (j) in the first (second) population, which consist of N (M ) different types. The matrix elements a il (b jk ) represent the pairwise interaction outcome between the types i (j) from the first(second) population and l (k) from the second (first) population. Success of a given type i is governed by the fitness of that type: this is the first term in the right hand-side of (20) and (21). The frequency of a given type will increase if the fitness of that type (or expected payoff of the given strategy) is higher than the mean fitness of the population to which it corresponds. Note that the fitness of a given type from one population is defined by the interaction with the other population, i.e. different types from the same population don't interact with each other. Thus, a given type from a population interacts with the members of other population, but the selection is through its own population.
Furthermore, it follows from (20) and (21) that if a given type is absent from the population initially, i.e. p i = 0 (or q j = 0), then this type will not occur during the time evolution of the system.
It follows from the system of equations (20) and (21) that the sum of the frequencies P N i p i = 1 ( P M j q j = 1) remains invariant during the time evolution. Thus, the dynamical system (20)- (21) is defined on the the second population), and this simplex and its faces are invariant. Hereon, we will focus on the two-population, two-type case, i.e. N = M = 2.

Two-population, two-type replicator dynamics in fixed environment
We recall here the dynamical equations for the fixed environment case: dp dt = p(1 p)(a 12 (a 12 + a 21 )q), under the conditions
We next see that the boundary can admit no further rest points: consider a point on the p = 0 boundary that is not a corner point. Then we have dp dt = 0, but dq dt = q(1 q)b 21 < 0, so the point is not a rest point. Points on the other edges of the boundary can be considered similarly, and shown to be not rest points by using the conditions (24) and (25).
Finally, we see that dynamics admits no rest points in the interior of S 1 ⇥ S 1 , since for all (p, q) 2 S 1 ⇥ S 1 we have that a 12 (a 12 + a 21 )q = a 12 (1 q) a 21 q < 0 (26) by condition (24), so for all (p, q) 2 (0, 1) 2 we have that dp dt < 0 by the dynamical equation (22).

(0,0) is the only stable rest point
which at a corner point (p(1 p) = q(1 q) = 0) becomes We have established in (26) that a 12 (a 12 + a 21 )q < 0, and can show similarly that b 21 (b 21 + b 12 )p < 0. Hence we have that the signs of the eigenvalues of the Jacobian matrix depend only on the signs of (1 2p), (1 2q), i.e. which quadrant the corner point is in. Thus: • At (0, 0): 1 2p, 1 2q > 0 so both eigenvalues are real and negative, and hence we have a stable rest point.
• At (0, 1) and (1, 0): 1 2p, 1 2q are of opposite sign, and thus the two eigenvalues are real and of opposite sign. Hence we have a saddle point.

Hessian of Hamiltonian in the interior
It can be verified that is a Hamiltonian for the dynamics (for fixed environment) in the interior of S 1 ⇥ S 1 . From this, we can compute the first partial derivatives We can then compute the elements of the Hessian matrix: This gives us the Hessian matrix,

Derivation of dynamical equations for slow-varying terms
In the presence of a varying environment that is incorporated into the system as oscillating payoffs, replicator dynamics gives us the equations (Note that we have assumed a 11 (⌧ ) = a 22 (⌧ ) = b 11 (⌧ ) = b 22 (⌧ ) = 0. We can do this because, as per the discussion for the fixed environment case, we can at every moment in time add a constant to the columns of the matrix a ij and the rows of the matrix b ij so that the diagonal terms of the payoff matrix are 0 without changing the dynamics of the system.) Let us now, for the sake of convenience, define the functions Furthermore we define the function so that we can rewrite (36) and (37) as Our aim now is to split the components of equation (40) (and similarly, (41)) into a slow-varying part and a fast-varying part. For the left-hand side of (40), we do this by taking the total derivative of ✏ p(t), q(t), ⌧ with respect to t, from which we obtain dp dt = dp dt + @✏ @p dp dt + @✏ @q where @ ⌧ ✏ is the partial derivative of ✏(p, q, ⌧ ) taken with respect to ⌧ .
For the right-hand side of equation (40), we simply take the Taylor expansion of G about (p, q) to get where A ⌘ a 12 , e A(⌧ ) ⌘ e a 12 (⌧ ) + e a 21 (⌧ ).
Now, equating expressions (42) and (43) and taking the fast-varying, O(1) terms, we get which we can integrate to get Similarly we get We can then substitute the expressions (46) and (48) into equation (43) to get Now take the time-average over a period to get the slow-time variation: since from (46) we have that and Thus (50) becomes and similarly we get that the slow-time variation of q is Now we observe that so by defining the parameters and dropping the O 1 ! terms, we get the dynamical equations which written out fully give Now we input the further simplification that a 12 = a 21 = a < 0 and b 21 = b 12 = b < 0, which translate to This gives us the final form of the dynamical equations,

Vertices remain rest points, no new rest points created on the boundary
From the dynamical equations (63) and (64), it is easy to see that the vertices of S 1 ⇥ S 1 remain rest points.
To see that no new rest points are created on the boundary, consider a point on the p = 0 boundary that is not a vertex. By (64) we have that dq dt = q(1 q)b < 0, so this point is not a rest point. Similarly, we can show that the other edges admit no rest points apart from the vertices of

Stability of the corner rest points unchanged by environmental oscillation
Let us compare the case with environmental variation to the case with fixed environment. To do this, substitute the parameters for the fixed environment ( e A(⌧ ) = e B(⌧ ) = e C(⌧ ) = e D(⌧ ) = 0) into the equations (40), (41). This gives the dynamical equations dp dt = G(A, B, p, q), dq dt = G(C, D, q, p).
We note that the dynamical equations for the oscillating environment, written in the form of (58) and (59), are very similar to these equations for the fixed environment, just modified by a term that is very small at the corner points. Thus, we should expect the dynamics near the corner points to be the same as that in the fixed environment. (Indeed, because the extra term is "doubly zero" at each corner point, it will not contribute anything to the Jacobian, i.e. the Jacobian at the corner points would be the same with or without environmental oscillation.) Nevertheless, we present below rigorous computation of the Jacobian to find the stability of the corner points: From the dynamical equations (63) and (64), and noting that at the corner points we have we can compute the entries of the Jacobian matrix at the corner points: and similarly @q @p = 0, @ṗ @p = (1 2q)b.
(Here,ṗ denotes dp dt .) This gives us the Jacobian matrix at the corner points, @ṗ @p @ṗ @q @q @p @q @q Since we have that a, b < 0, we have that the first and second eigenvalues of J are negative iff 1 2p > 0 and 1 2q > 0 respectively. Thus: • At ( Thus we see that the stability of the corner rest points are unchanged by the oscillations in the environment.

Computation of Jacobian at possible interior rest point
At an interior rest point (p ⇤ , q ⇤ ), we have fromṗ =q = 0 that Using this, we can compute the elements of the Jacobian matrix: and similarly @q @p @q @q (Note the minus sign in @q @q because the sign of the ↵ term in the expression forq is the opposite of the sign of the ↵ term in the expression forṗ.) This gives us the Jacobian at the possible interior rest point, @ṗ @p @ṗ @q @q @p @q @q We see that the Jacobian is traceless, so any interior rest point must be either a center or a saddle.

Hessian matrix of Hamiltonian in interior
From the Hamiltonian H(p, q) = a ln q 1 q b ln p 1 p ( q p) + ↵pq, we can compute the first partial derivatives @H @p = b p(1 p) @H @q = a q(1 q) + ↵p.
This gives us the Hessian matrix, Thus, for the ↵ = 0 case, the anti-diagonal components vanish and we have that: • for the interior rest point (p ⇤ , q ⇤ ) with p ⇤ > 1 2 , q ⇤ < 1 2 , the Hessian matrix is positive definite and so the rest point is a local minimum of the Hamiltonian function, and • for the interior rest point (p ⇤ , q ⇤ ) with p ⇤ < 1 2 , q ⇤ > 1 2 , the Hessian matrix is negative definite and so the rest point is a local maximum of the Hamiltonian function.