Doubly heterogeneous networks facilitate the emergence of collective cooperation

First, we explore how the heterogeneous strategy updating affects the fate of cooperators on four commonly-studied population structures: lattice, small-world, Erdös-Rényi, and scale-free networks [Fig. 2]. Under the traditional scenario of identical update rates ($\lambda_{i}=1$ for all $i$), scale-free networks demand the largest critical benefit-to-cost ratio $\mathcal{C}^{*}$, above which cooperation is favored among all the four structures, and the lattice structure the smallest [Fig. 2(a)], consistent with previous findings [4, 5]. But surprisingly, when a node’s update rate varies inversely with its number of neighbors ($\lambda_{i}=1/k_{i}$), we find that this trend is reversed [Fig. 2(b)]. Here, scale-free network becomes the most amenable to cooperation, and lattice the least. Interestingly, we find that heterogeneous update rates can even improve upon the canonical threshold $b/c>\langle k\rangle$ (namely, $\mathcal{C}^{*}=\langle k\rangle$) for homogeneous populations [4], allowing cooperation to emerge even when $b/c<\langle k\rangle$ (namely, $\mathcal{C}^{*}<\langle k\rangle$). Furthermore, we find that this pattern is strengthened when the update rate is inversely proportional to higher powers of $k_{i}$ [Fig. 2(d)]. In contrast, when $\lambda_{i}$ is positively related to $k_{i}$, the ordering of $\mathcal{C}^{*}$ over different structures matches the identical-rate case, but with the inhibition of cooperation fixation by heterogeneous networks amplified [Fig. 2(c)].

We further shed light on our numerical findings by deriving a closed-form expression for the critical benefit-to-cost ratio $\mathcal{C}^{*}$ as a function of the network structure (see Appendix A)

Here, $k_{i}=\sum_{j}e_{ij}$ defines the number of neighbors (degree) of individual $i$, and $e_{ij}=e_{ji}=1$ indicates that there is an edge between nodes $i$ and $j$ ($e_{ij}=e_{ji}=0$ otherwise). The probability of a $1$-step ($n$-step) random walk from $i$ to $j$ is denoted by $p_{ij}$ ($p_{ij}^{(n)}$), and $\eta_{ij}$ is the coalescence time [30]—the expected time for two random walks starting from nodes $i$ and $j$ to meet at a common node. As shown in Fig. 2(e), all numerical results in Figs. 2(a)–2(d) are in good agreement with the theoretical prediction of Eq. (1).

To intuitively understand why heterogeneous update rates can improve the fixation of cooperation in heterogeneous networks, we first consider how the evolutionary dynamics play out on a simple double star structure [Fig. 3]. When the fixation of cooperation occurs in this highly heterogeneous structure, it usually does so in four stages: (i) occupation of one of the hubs; (ii) formation of a stable cluster of cooperators among that hub and its neighbors; (iii) occupation of the other hub; and finally (iv) spread to the remaining nodes. As such, the ultimate triumph of cooperators can be thwarted if a hub imitates defection from even one of its (many) neighbors before stages (ii) and (iv) are complete [Fig. 3(c)]. There are ample opportunities for this to occur under the traditional setting of identical update rates ($\lambda_{i}=1$), as illustrated in Fig. 3(b). When $\lambda_{i}=1/k_{i}$ however [Fig. 3(a)], hubs update relatively infrequently. As such, once a hub becomes a cooperator, it is effectively “locked in”, giving time for its strategy to spread to the hub’s neighbors. By the same logic, the preferential updating of hubs ($\lambda_{i}=k_{i}$) usually leads to the extinction of cooperation, as the formation of stable clusters of cooperators and the spread of cooperation is even harder than the traditional scenario of identical updating [Fig. 3(c)].

In Fig. 4, we illustrate the fundamental mechanism explaining why infrequent updates of hubs can facilitate cooperation. If an individual [grey node in Fig. 4(a)] decides to update its strategy, it will imitate the strategy of its neighbors according to their payoffs. The neighboring cooperator obtains an average payoff $P_{\text{C}}=bq_{\text{C}|\text{C}}(\langle k\rangle-1)/\langle k\rangle-c$ and the neighboring defector obtains $P_{\text{D}}=bq_{\text{C}|\text{D}}(\langle k\rangle-1)/\langle k\rangle$, where $q_{\text{C}|\text{C}}$ ($q_{\text{C}|\text{D}}$) represents the conditional probability to find a cooperative neighbor for a given cooperator (defector). The contribution to the neighboring cooperator and defector from the updating individual is excluded since they are equal. Thus the cooperator is favored compared to the defector to disperse its strategy if $P_{\text{C}}>P_{\text{D}}$, namely

For the canonical setting with identical update rates ($\lambda_{i}=1$), we know $(q_{\text{C}|\text{C}}-q_{\text{C}|\text{D}})(\langle k\rangle-1)=1$ according to pair approximation (see Supplemental Material Sec. II [29]), namely a cooperator has on average one more cooperative neighbor than a defector [Fig. 4(a)]. This leads to the conclusion that cooperation is favored when $b/c>\langle k\rangle$ (namely, $\mathcal{C}^{*}=\langle k\rangle$), which also degenerates to the simple rule [4] for homogeneous networks where $k_{i}=\langle k\rangle$.

Next we show how heterogeneous update rate alters the local dispersal of cooperation on heterogeneous networks. When $\lambda_{i}=1/k_{i}$, we find that $(q_{\text{C}|\text{C}}-q_{\text{C}|\text{D}})(\langle k\rangle-1)>1$ (see Supplemental Material Sec. II [29]), indicating that the number of cooperative neighbors of a cooperator exceeds that of a defector by more than one [Fig. 4(b)]. This implies that the net payoff of cooperators relative to defectors is further increased, giving cooperators more advantage in competition and dispersal. Therefore, the critical ratio for $\lambda_{i}=1/k_{i}$ is smaller than the average degree $\langle k\rangle$ for a wide range of heterogeneous networks ($\mathcal{C}^{*}<\langle k\rangle$). In contrast, when $\lambda_{i}=k_{i}$, the hubs update frequently and $(q_{\text{C}|\text{C}}-q_{\text{C}|\text{D}})(\langle k\rangle-1)<1$ (see Supplemental Material Sec. II [29]), indicating that on average, the number of cooperative neighbors of a cooperator exceeds that of a defector by less than one [Fig. 4(c)]. This leads to a larger critical ratio ($\mathcal{C}^{*}>\langle k\rangle$) for promoting cooperation compared to the scenario with identical update rates shown in Fig. 4(a).

We have numerically confirmed the above mechanism on larger scale-free networks. Figures 4(d)–4(f) show the state of the hub, and the fraction of cooperators among the hub’s neighbors over the course of the game dynamics. For $\lambda_{i}=1/k_{i}$, we observe long-lasting periods of cooperation on the hub [Fig. 4(e)], with infrequent strategy switches from cooperation to defection, which results in the highest $q_{\text{C}|\text{C}}$ for the hub [Fig. 4(g)] and in turn the highest $q_{\text{C}|\text{C}}-q_{\text{C}|\text{D}}$ over all nodes with different degrees compared to other settings [Fig. 4(h)]. In contrast, fast-updating hubs ($\lambda_{i}=k_{i}$) have the lowest average fraction of cooperators among their neighbors [Fig. 4(g)], leading to a low fraction of cooperative neighbors for the cooperators relative to defectors over the whole network [Fig. 4(h)]. This confirms that degree-inverse update rates promote cooperation on heterogeneous networks because a hub with a low update rate is more conducive to driving its neighbors to cooperation, which further enhances the local dispersal of cooperation among nodes with different degrees.

We next explore how different distributions of $\lambda_{i}$ affect $\mathcal{C}^{*}$ over four different synthetic networks: random regular, Erdös-Rényi, small-world, and scale-free. For a given network structure, we theoretically predict $\mathcal{C}^{*}$ via Eq. (1) for uniform, normal, exponential and power-law distributions of the update rate. We find that the critical threshold of a typical homogeneous network—such as a lattice or random regular network—is almost unaffected by the choice of update rate distribution [Fig. 5(a)]. In contrast, heterogeneous structures are quite sensitive, with scale-free networks presenting the most drastic variations in $\mathcal{C}^{*}$ among the different update-rate distributions we consider. This malleability of $\mathcal{C}^{*}$ in heterogeneous networks suggests the possibility of deliberately tuning the update rates to lower the barrier for the emergence of cooperation in a particular network. But to put this into practice, we must first overcome a computational hurdle.

In order to calculate $\mathcal{C}^{*}$ using Eq. (1), one needs to solve a system of $N(N-1)/2$ linear equations for the recurrence relations between the $\eta_{ij}$ (see Eq. (9) in Appendix A). Unfortunately, this requires an overall complexity of $\mathcal{O}(N^{6})$, rendering the problem intractable for large networks. To circumvent this, we offer an efficient approximation $\mathcal{C}^{*}$ as

This expression obviates the need to solve large systems of linear equations and reduces the computational complexity to $\mathcal{O}(N^{3})$. Here $\langle k^{2}\rangle$ is the second moment of the degree distribution. We have $\zeta=\sum_{i,j}\frac{k_{i}k_{j}\Lambda}{NK^{2}(\lambda_{i}+\lambda_{j})}$, where $\Lambda=\sum_{i}\lambda_{i}$ defines the total rate of update events and $K=\sum_{i}k_{i}$ is the summation of all nodes’ degrees. Finally, $\Delta_{\lambda^{(1)}}$, $\Delta_{\lambda^{(2)}}$, $\Delta_{\widetilde{\eta}_{n}}$ and $\Delta_{\widetilde{\eta}_{d}}$ are constants related to the heterogeneity of update rates and coalescence times, the expressions for which are given in Appendix B. When the update rates are identical, we have $\Delta_{\lambda^{(1)}}=\Delta_{\lambda^{(2)}}=\Delta_{\widetilde{\eta}_{n}}=\Delta_{\widetilde{\eta}_{d}}=0$, and Eq. (3) recovers the previous results [21, 4].

Figure 5(b) compares the value of $\mathcal{C^{*}}$ predicted by the approximation in Eq. (3) with that of numerical simulation on two empirical social networks [31, 32]. We see that our approximation is remarkably accurate in both networks, regardless of the distribution of the update rates. Moreover, Eq. (3) offers intuition behind our previous observation that homogeneous structures are robust to different update rates [Fig. 5(a)]. The high symmetry present in these networks means that heterogeneous update rates affect only a limited number of nodes. For such networks, we have $\Delta_{\widetilde{\eta}_{n}}\approx\Delta_{\widetilde{\eta}_{d}}\approx 0$, meaning that $\mathcal{C}^{*}\rightarrow\langle k\rangle$ in the limit of large $N$. This coincides with the classical result ($\mathcal{C}^{*}=\langle k\rangle$) [4] regardless of the distribution of update rates.

Starting from Eq. (3) (see Appendix B), we have the critical benefit-to-cost ratio for large heterogeneous networks

where $\langle k\rangle$ is the average degree and $\Delta_{\widetilde{\eta}^{(\infty)}}\approx\frac{\overline{\eta}}{K^{2}}\sum_{i<j}(k_{i}-k_{j})(\lambda_{i}-\lambda_{j})e_{ij}/(\lambda_{i}+\lambda_{j})$. Note that $\Delta_{\widetilde{\eta}^{(\infty)}}<0$ when any pair of nodes $i$ and $j$ satisfies the rule $(k_{i}-k_{j})(\lambda_{i}-\lambda_{j})<0$. When the update rates are identical, we have $\Delta_{\widetilde{\eta}^{(\infty)}}=0$ and hence $\mathcal{C}^{*}\approx\langle k\rangle$ as expected. In contrast, $\mathcal{C}^{*}$ is smaller (larger) than $\langle k\rangle$ when $\Delta_{\widetilde{\eta}^{(\infty)}}<0$ ($\Delta_{\widetilde{\eta}^{(\infty)}}>0$) (see Supplemental Material Sec. III [29]). Table 1 summarizes the values of $\mathcal{C}^{*}$ predicted by Eq. (4) for the combinations of network structure/update-rate settings.

Taken together, we have theoretically motivated an efficient rule of thumb for lowering the threshold for the emergence of cooperation on large heterogeneous structures. Put simply, the order of nodes’ update rates (for example, $\lambda_{i}>\lambda_{j}$) should be reversed from the order of the nodes’ degrees (for example, $k_{i}<k_{j}$). That is, nodes with larger degree should have smaller update rates and vice versa, as is demonstrated in Fig. 5(c). A simple but general realization of this rule is $\lambda_{i}=1/k_{i}^{\gamma}~{}(\gamma>0)$ which we study numerically in Fig. 2(d) for different values of $\gamma$. This rule can achieve a lower critical ratio $\mathcal{C}^{*}$ than identical update rates ($\gamma=0$) on both synthetic heterogeneous [Fig. 2(b)] and empirical networks ($\gamma=1$) [Table S1, Figs. 5(b), S2 and S3 [29]]. Meanwhile, the contrary configuration of $\lambda_{i}=k_{i}^{\gamma}$ leads to increases in $\mathcal{C}^{*}$ on heterogeneous networks [Figs. 2(c), 5(b), S4 and S5 [29]].

Can we improve upon the simple heuristic to favor cooperation in the previous section? Specifically, can we find an optimal set of node update rates for a given network? To answer this question, we employ a protocol based on the RMSprop [35] algorithm to search for a set of $\lambda_{i}$ that minimizes $\mathcal{C}^{*}$, via iterative gradient descent (see Supplemental Material Sec. IV and Fig. S6 [29]). Consistent with our rule, Fig. 5(d) shows scale-free networks are more flexible and attain a much smaller threshold at its optimal rate than lattice. Moreover, the update rates of higher-degree nodes tend to decrease during the optimization process, while those of smaller-degree nodes increase [Figs. 5(e) and S7 [29]]. Interestingly, we find that even on homogeneous structures such like lattices, a policy of identical update rates is not the best choice for promoting cooperation. Indeed, the final update rates deviate significantly from the initial conditions [Figs. 5(f) and S8 [29]]. Figure 5(g) shows that the optimal update rates for different network structures are consistent with our rules shown in Fig. 5(c)—namely that a node $i$’s update rate $\lambda_{i}$ should vary inversely with its degree $k_{i}$.