Conditional investment strategy in evolutionary trust games with repeated group interactions

Linjie Liu Xiaojie Chen xiaojiechen@uestc.edu.cn College of Science, Northwest A & F University, Yangling 712100, China School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu 611731, China

Abstract

It has a long tradition to study trust behavior among humans or artificial agents by investigating the trust game. Although previous studies based on evolutionary game theory have revealed that trust and trustworthiness can be promoted if network structure or reputation is considered, they often assume that interactions among agents are one-shot and investors do not consider the investment environment before making decisions, which collide with many realistic situations. In this paper, we introduce the conditional investment strategy into the repeated $N$ -player trust game, in which conditional investors decide to invest or not depending on their assessment of the trustworthiness level of the group. By using the approach of the Markov decision process, we study the evolutionary dynamics of trust in repeated group interactions with the conditional investment strategy. We find that conditional investors can form an effective alliance with trustworthy trustees, hence they can sweep out untrustworthy trustees. Moreover, we verify that such alliance can exist in a wide range of model parameters. These results may explain why trusting in others and reciprocating them with trustworthy actions can be sustained in game interactions among intelligent agents.

keywords:

Evolutionary game theory , the

N-

player trust game , trustworthiness , conditional investment strategy , Markov decision process

^†^†journal: Information Sciences

1 Introduction

Trust is a value-laden concept in Distributed Artificial Intelligence (DAI), and it is also fundamental for cooperative behavior and other forms of prosocial behavior [17, 27, 32, 42, 45]. However, a basic fact accepted by the public is that trust is often accompanied by risks. As Deutsch defined, entering a trust relationship is to choose an ambiguous path that can lead to a beneficial event or a harmful event depending on the behavior of the other person [13]. In other words, there are risks in the interactions with potentially untrustworthy agents, while the interactions with trusted agents will lead to benefits [39]. Thus understanding how trust and trustworthiness evolve in Multi-agent Systems (MAS) and exploring the conditions in which they will emerge have always been great challenges [19].

Evolutionary game theory provides a theoretical framework to study the above problems [15, 25, 29, 31, 41, 48], and the trust game (TG) has been widely used to study trust and trustworthiness as a typical paradigm [1, 22]. The classical TG model involves interactions between an investor and a trustee where the investor first decides whether he/she is willing to invest his/her funds to the trustee, and then the trustee decides how much to give back to the investor (or not at all) [4, 23]. Previous theoretical work has proved that the subgame perfect equilibrium of TG is that the investor invests zero and the trustee returns zero [18]. The above two-player TG model perfectly characterizes pairwise interactions between an investor and a trustee and has recently been extended to group interactions of multiple players [1, 8, 22, 24]. A representative $N$ -player trust game (NTG) framework was developed by Abbass et al. [1], and then introduced to a population of agents playing TG concurrently in a well-mixed environment. Along this line, many researchers have considered network structure [8, 9, 24], punishment strategy [16], asymmetric demographic parameters [26], and reputation [22], into the NTG model to explore the evolution of trust.

Although there were attempts for enhancing the level of trust from different aspects, we notice that these studies are usually carried out in the framework of one-shot interaction, to our knowledge [8, 9, 16, 24, 26]. Indeed, real interactions are repeated rather than the one-shot interaction assumed above [11, 10, 30, 33, 28]. For example, we are likely to interact frequently with friends, co-workers, and economic partners in our daily life. Second, agents’ behaviors may be affected by noise (such as imitation error and behavioral mutation) in the process of social learning [37, 38, 47], which is usually ignored by previous theoretical models. Furthermore, in repeated interactions, investors have the opportunity to adjust their actions according to the investment environment they encountered [30, 34, 28]. Furthermore, it is very common for investors to choose conditional investment behavior in their daily economic activities when interacting with other agents [2, 5]. Concretely, the decision to invest or not will depend on one’s own assessment of the abundance of trustworthy trustees in the group. However, it is still unclear what the effects of conditional investment strategy on the evolution of trust in the repeated NTG are.

In order to answer the above question, we construct an evolutionary game model based on the NTG where the interactions among agents are repeated (see Fig.1). In our model, we consider three strategists: investor, untrustworthy trustee, and trustworthy trustee. Different from previous works [22, 24], we assume investors will invest deterministically in the first round of the game, but in the subsequent game rounds their decision to invest or not depends on the number of the trustworthy trustee in the group. Using the approach of the Markov decision process (MDP), we study the stochastic evolutionary dynamics in finite populations. We find that the introduction of the conditional investment strategy can lead to the emergence of high trust and high trustworthiness in repeated group interactions. Furthermore, we verify that such evolutionary outcomes are robust against model parameters.

2 Related Works

The formalization of the NTG that we follow here was proposed in Ref. [1], where Abbass et al. investigated the evolutionary dynamics of trust in an infinite well-mixed population. By analyzing the replicator dynamics, they found that both the whole society and all agents can obtain the maximal wealth when the initial population contains no untrustworthy agents. While the initial population consists of one single untrustworthy agent, untrustworthy agents would spread rapidly to the whole population. Furthermore, they found that the system will eventually converge to a stable state, in which untrustworthy agents will not completely occupy the whole population, because a fraction of the population would always remain trustworthy even if there are few or no investors.

Nevertheless, the analysis of replicator dynamics in the previous literature requires that the network structure is a complete graph, and thus this approach cannot be used to analyze the evolutionary dynamics when agents interact with local neighborhoods in social networks. Subsequently, Chica et al. studied the evolution of trust in different social networks including regular lattices, scale-free, and random networks [8]. By performing simulations, they found that trust can be promoted when agents interact on a social network even if there are untrustworthy agents in the population initially. In addition, the level of trust is influenced by network structure, the temptation to defect, and the initial number of untrustworthy trustees in the population. Along this line, the effects of different evolutionary update rules on promoting the evolution of trust were investigated [9], and simulation results showed that updating rules play an important role in promoting trust and improving global net wealth.

However, some other studies have found that social networks cannot produce a high level of trust [22, 24]. Recently, Kumar et al. investigated the trust game where the trustor’s investment and the trustee’s return of the investment are two important parameters on different social networks [24]. By performing Monte Carlo simulations, they found that the network structure has little effect on the evolution of trust and trustworthiness. In particular, trust cannot evolve in well-mixed populations, lattices, random, or scale-free networks. Considering that in the real society, the investment behavior of investors is often related to the reputation of the trustees, and agents with good reputation are more likely to attract investors’ investment, Hu et al. studied networked NTG with an adaptive reputation based on the third-party moral assessment system [22]. They showed that the frequency of untrustworthy trustees will decrease when rational investors can get the reputation score of the trustee.

The literature mentioned above does not consider the case in which agents’ decision-making may be affected by random factors, including imitation error and behavioral mutation. Considering this, Lim explored the evolutionary dynamics of trust in finite populations and found that the combination of strong selection in the population of investors and weak selection in the population of trustees can promote the emergence of high trustworthiness and high trust [26]. It is worth noting that they analyzed the stationary distribution of the system in the assumption of weak-mutation limit. The above assumption has been justified in population genetics [12, 14], but this approximate does not seem to be suitable for modeling social learning [47].

Different from previous works both in infinite populations [1] and structure populations [8], we study the evolutionary dynamics of trust in a finite population where imitation error and behavioral mutation are both exist. Besides, we release the assumptions that the mutation rate is significantly small [26] and the interactions between agents are one-shot [1, 8, 24, 26], and investigate the stochastic dynamics of trust in finite populations with repeated group interactions when mutation rates are arbitrarily large. Accordingly, we are committed to solving the following questions: can potentially richer evolutionary dynamics be produced? Can a high level of trust and trustworthiness be reached in the NTG with repeated group interactions? To answer these questions, we establish an NTG model where the conditional investment strategy is involved in repeated group interactions. Concretely, conditional investors decide to invest only when the number of trustworthy trustees in the game group reaches their expected threshold. We study the evolutionary dynamics of strategies in a finite population by using the approach of MDP. Our concrete theoretical model and methods are presented in the following section.

Refer to caption — Figure 1: Illustration of two different interactions scenarios in repeated $N$ -player trust games where the repeated probability is set as $w$ . $N$ agents are randomly sampled to form a group for playing the repeated trust game where a conditional investor will act as an unconditional investor in the first round, and then can agree or refuse to invest depending on whether the number of trustworthy agents in the group exceeds the expected threshold in subsequent game rounds.

3 Theoretical Model and Methods

3.1 Repeated NTG

Let us assume a finite well-mixed population with $Z$ agents who play a repeated NTG [1, 8]. At every time step, $N$ agents are chosen and offered the opportunity to participate in the repeated NTG. As considered in previous work [28], the trust game will be repeated with probability $w$ with $0<w<1$ (also called the discount factor in previous work [21]), resulting in an average number of $r=1/w$ rounds. In this game, each agent needs to make two decisions in advance. First, agent needs to decide whether to act as a trustee or an investor. The second is to decide whether to be trustworthy or not. Here, we consider three baseline strategists in this NTG:

1.

Conditional Investor ( $CI$ ): Pays an observation cost $\sigma$ before the game to gather information including the number of trustworthy agents and untrustworthy agents in the group, and always chooses to invest $tv$ in the first round. Subsequently, $CI$ agents invest only when the number of trustworthy agents in the group is not less than their expected threshold $M$ .
2.

Trustworthy Agent ( $T$ ): Returns the received fund obtained in each round multiplied by $R_{T}$ to the investors.
3.

Untrustworthy Agent ( $U$ ): Returns nothing to the investors in all game rounds.

We shall first notice that, by adopting $CI$ , an agent will always invest in the first round and subsequently either act as an investor or as an outsider, depending on whether the number of trustworthy trustees in the group has reached the expected level $M$ (see Fig.1). If the expected level is reached, $CI$ agents are willing to pay $tv$ to trustees, and each trustee receives the same amount of benefit $\frac{N_{CI}tv}{N_{T}+N_{U}}$ , where $N_{CI},N_{T}$ , and $N_{U}$ respectively denote the number of $CI,T$ , and $U$ agents in the group. Then each $T$ agent returns $\frac{R_{T}tv}{N_{T}+N_{U}}$ to every $CI$ agent and reserves $\frac{R_{T}N_{CI}tv}{N_{T}+N_{U}}$ for himself/herself. While each $U$ agent returns nothing to $CI$ agents, but keeps $\frac{R_{U}N_{CI}tv}{N_{T}+N_{U}}$ for himself/herself. The parameters $R_{T}$ and $R_{U}$ respectively denote the multiply factor of $T$ agents and $U$ agents. If the number of $T$ agents in the group does not reach the expected level, every agent in the group can obtain nothing from the subsequent game rounds. Thus the payoffs of $CI,T$ , and $U$ agents obtained from the game are depicted by

$\displaystyle\Pi_{CI}$	$\displaystyle=\left\{\begin{aligned} &\frac{R_{T}N_{T}tv}{N-N_{CI}-1}-tv+(\frac{R_{T}N_{T}tv}{N-N_{CI}-1}-tv)(r-1)\Theta(N_{T}-M)-\sigma,\quad\text{if}\ N_{CI}\neq N-1\ ;\\ &0,\quad\text{otherwise.}\end{aligned}\right.$	(1)
$\displaystyle\Pi_{T}$	$\displaystyle=\left\{\begin{aligned} &\frac{R_{T}N_{CI}tv}{N-N_{CI}}+\frac{R_{T}N_{CI}tv}{N-N_{CI}}(r-1)\Theta(N_{T}+1-M),\quad\text{if}\ N_{CI}\neq N\ ;\\ &0,\quad\text{otherwise.}\end{aligned}\right.$	(2)
$\displaystyle\Pi_{U}$	$\displaystyle=\left\{\begin{aligned} &\frac{R_{U}N_{CI}tv}{N-N_{CI}}+\frac{R_{U}N_{CI}tv}{N-N_{CI}}(r-1)\Theta(N_{T}-M),\quad\text{if}\ N_{CI}\neq N\ ;\\ &0,\quad\text{otherwise,}\end{aligned}\right.$	(3)

where $\Theta(k)$ is the Heaviside function (that is, $\Theta(k)=1$ whenever $k\geq 0$ , being zero otherwise). The threshold values $M$ range from 1 to $N-1$ . If $M<1$ all $CI$ agents will choose to invest unconditionally and $M>N-1$ means always choosing not to invest fund.

In a finite population with $Z$ agents, $N$ agents are randomly sampled to form a game group and accumulate their payoffs by interacting with other group agents according with the payoff functions of NTG defined above. Concretely, when there are $i_{CI}$ agents choosing to adopt the $CI$ strategy, $i_{T}$ agents choosing to adopt the $T$ strategy, $i_{U}=Z-i_{CI}-i_{T}$ agents choosing to adopt the $U$ strategy in a finite population, the probability of finding $j_{CI}$ $CI$ agents, $j_{T}$ $T$ agents, and $j_{U}$ $U$ agents in a sample game group can be depicted by the multiple hypergeometric distribution, given as

\displaystyle H(j_{CI},j_{T},N,i_{CI},i_{T},Z)=\frac{\tbinom{i_{CI}}{j_{CI}}\tbinom{i_{T}}{j_{T}}\tbinom{Z-i_{CI}-i_{T}}{N-j_{CI}-j_{T}}}{\tbinom{Z}{N}},

which describes the configuration of the population for a given time. Accordingly, for a given configuration $\textbf{i}=({i_{CI},i_{T}})$ , the average payoffs of $CI,T$ , and $U$ agents can be respectively computed as

$\displaystyle f_{CI}$	$\displaystyle=$	$\displaystyle\sum\limits_{j_{CI}=0}^{N-1}\sum\limits_{j_{T}=0}^{N-1-j_{CI}}\frac{\tbinom{i_{CI}-1}{j_{CI}}\tbinom{i_{T}}{j_{T}}\tbinom{Z-i_{CI}-i_{T}}{N-1-j_{CI}-j_{T}}}{\tbinom{Z-1}{N-1}}$
	$\displaystyle\times$	$\displaystyle\Pi_{CI}(j_{CI}+1,j_{T}),$
$\displaystyle f_{T}$	$\displaystyle=$	$\displaystyle\sum\limits_{j_{CI}=0}^{N-1}\sum\limits_{j_{T}=0}^{N-1-j_{CI}}\frac{\tbinom{i_{CI}}{j_{CI}}\tbinom{i_{T}-1}{j_{T}}\tbinom{Z-i_{CI}-i_{T}}{N-1-j_{CI}-j_{T}}}{\tbinom{Z-1}{N-1}}$
	$\displaystyle\times$	$\displaystyle\Pi_{T}(j_{CI},j_{T}+1),$
$\displaystyle f_{U}$	$\displaystyle=$	$\displaystyle\sum\limits_{j_{CI}=0}^{N-1}\sum\limits_{j_{T}=0}^{N-1-j_{CI}}\frac{\tbinom{i_{CI}}{j_{CI}}\tbinom{i_{T}}{j_{T}}\tbinom{Z-i_{CI}-i_{T}-1}{N-1-j_{CI}-j_{T}}}{\tbinom{Z-1}{N-1}}$
	$\displaystyle\times$	$\displaystyle\Pi_{U}(j_{CI},j_{T}),$

where $\Pi_{CI},\Pi_{T},$ and $\Pi_{U}$ are respectively shown in Eqs. (LABEL:eq1)-(LABEL:eq3).

3.2 Social Learning

The average payoffs of agents obtained above stand for their social success, then we can analyze the evolutionary dynamics of strategy adopting by using social learning [35, 36, 37], which means that the most successful strategy will more often tend to be adopted by other agents. Generally, the above social learning can be characterized by the pairwise comparison rule. Concretely, the probability that an agent adopting strategy $A$ with the payoff $f_{A}$ imitates another agent’s strategy $B$ with payoff $f_{B}$ is given by the Fermi function [40]

\displaystyle P(f_{B}-f_{A})=\frac{1}{1+\exp(-\beta(f_{B}-f_{A}))},

(4)

where $\beta$ characterizes the intensity of selection, controlling how the imitation process depends on the difference between the payoffs of two agents. For $\beta\rightarrow\infty$ (strong imitation), any difference in the payoffs will produce a sizeable effect on imitation probability. For $\beta\rightarrow 0$ (weak selection), the strategy is imitated randomly. In between these extremes, the difference of the payoffs and stochastic effects associated with errors can both impact imitation probability.

Furthermore, we introduce the exploration term: with probability $1-\mu$ , an agent with strategy $A$ adopts another agent’s strategy $B$ according to the social learning rule described above, and with probability $\mu$ , he/she randomly selects a strategy different from the current one from the strategy space. Thus, the probability that an agent with strategy $A$ adopts strategy $B$ according to the mutation-selection process can be written as

\displaystyle T_{A\rightarrow B}

\displaystyle=

\displaystyle(1-\mu)[\frac{i_{A}}{Z}\frac{i_{B}}{Z-1}\frac{1}{1+\exp({-\beta(f_{B}-f_{A})})}]+\mu\frac{i_{A}}{2Z}.

(5)

The existence of behavioral mutation makes the population never fixate in any of the three possible monomorphic configurations. This fact renders the stationary distribution and the gradient of selection as the appropriate quantities to analyze the behavior of the population.

3.3 Markov Decision Process

We know that the update process only relies on the current state $\textbf{i}(t)$ of the system, so that $\textbf{i}(t)=\{i_{CI},i_{T}\}$ has Markov property. Thus the evolutionary dynamics of CI, T, and U can be described by the Markov chain in two-dimensional space. Then the evolutionary dynamics of the system can be analyzed by investigating the probability distribution function $p_{\textbf{i}}(t)$ providing information about the pervasiveness of each configuration at time t, and it satisfies the following discrete time Master Equation [46]

\displaystyle p_{\textbf{i}}(t+\tau)-p_{\textbf{i}}(t)=\sum_{\textbf{i}^{{}^{\prime}}}\left\{T_{\textbf{i}\textbf{i}^{{}^{\prime}}}p_{\textbf{i}^{{}^{\prime}}}(t)-T_{\textbf{i}^{{}^{\prime}}\textbf{i}}p_{\textbf{i}}(t)\right\},

where $T_{\textbf{i}\textbf{i}^{{}^{\prime}}}$ and $T_{\textbf{i}^{{}^{\prime}}\textbf{i}}$ denote the transition probabilities between configurations $\textbf{i}^{\prime}$ and i. Technically, we can obtain the so-called stationary distribution $\bar{p}_{\textbf{i}}$ , by searching the eigenvector associated with the eigenvalue 1 of the transition matrix $\Lambda=[T_{\textbf{ij}}]^{T}$ with dimension $\frac{(Z+1)(Z+2)}{2}\times\frac{(Z+1)(Z+2)}{2}$ . The transition probability between two adjacent states can be computed as

$\displaystyle T_{\textbf{i}(i_{CI},i_{T})\rightarrow\textbf{i}^{{}^{\prime}}(i_{CI}+1,i_{T})}$	$\displaystyle=$	$\displaystyle T_{U\rightarrow CI},$
$\displaystyle T_{\textbf{i}(i_{CI},i_{T})\rightarrow\textbf{i}^{{}^{\prime}}(i_{CI}-1,i_{T})}$	$\displaystyle=$	$\displaystyle T_{CI\rightarrow U},$
$\displaystyle T_{\textbf{i}(i_{CI},i_{T})\rightarrow\textbf{i}^{{}^{\prime}}(i_{CI},i_{T}+1)}$	$\displaystyle=$	$\displaystyle T_{U\rightarrow T},$
$\displaystyle T_{\textbf{i}(i_{CI},i_{T})\rightarrow\textbf{i}^{{}^{\prime}}(i_{CI},i_{T}-1)}$	$\displaystyle=$	$\displaystyle T_{T\rightarrow U},$
$\displaystyle T_{\textbf{i}(i_{CI},i_{T})\rightarrow\textbf{i}^{{}^{\prime}}(i_{CI}-1,i_{T}+1)}$	$\displaystyle=$	$\displaystyle T_{CI\rightarrow T},$
$\displaystyle T_{\textbf{i}(i_{CI},i_{T})\rightarrow\textbf{i}^{{}^{\prime}}(i_{CI}+1,i_{T}-1)}$	$\displaystyle=$	$\displaystyle T_{T\rightarrow CI}.$

The transition probability between two nonadjacent states i and v is $T_{\textbf{i}\textbf{v}}=0$ . Thus the probability of the system staying in the current state is

\displaystyle T_{\textbf{i}(i_{CI},i_{T})\rightarrow\textbf{i}(i_{CI},i_{T})}

\displaystyle=

\displaystyle 1-\sum_{\textbf{i}\neq\textbf{i}^{{}^{\prime}}}(T_{\textbf{i}\textbf{i}^{{}^{\prime}}}+T_{\textbf{i}^{{}^{\prime}}\textbf{i}}).

In one discrete time step, the probability of the system transferring from one state to another can be calculated by equation (5).

In addition to the analysis of the stationary distribution of the system, another important quantity for studying the evolutionary dynamics in finite populations is the gradient of selection, which indicates the most likely evolutionary path when the system leaves the current configuration [47]. The gradient of selection is described as

\displaystyle\nabla_{\textbf{i}}

\displaystyle=(T_{\textbf{i}}^{CI+}-T_{\textbf{i}}^{CI-})\textbf{u}_{\textbf{CI}}+(T_{\textbf{i}}^{T+}-T_{\textbf{i}}^{T-})\textbf{u}_{\textbf{T}},

(6)

where $\textbf{u}_{\textbf{CI}}$ and $\textbf{u}_{\textbf{T}}$ are a set of standard orthogonal bases, and we set $\textbf{u}_{\textbf{CI}}=(1,0)^{T}$ and $\textbf{u}_{\textbf{T}}=(0,1)^{T}$ in this work. $T_{\textbf{i}}^{CI+}(T_{\textbf{i}}^{CI-})$ and $T_{\textbf{i}}^{T+}(T_{\textbf{i}}^{T-})$ respectively denote the probabilities that the numbers of $CI$ agents and $T$ agents increase (decrease) one, which read

$\displaystyle T_{\textbf{i}}^{CI+}$	$\displaystyle=$	$\displaystyle T_{U\rightarrow CI}+T_{T\rightarrow CI},$
$\displaystyle T_{\textbf{i}}^{CI-}$	$\displaystyle=$	$\displaystyle T_{CI\rightarrow U}+T_{CI\rightarrow T},$
$\displaystyle T_{\textbf{i}}^{T+}$	$\displaystyle=$	$\displaystyle T_{U\rightarrow T}+T_{CI\rightarrow T},$
$\displaystyle T_{\textbf{i}}^{T-}$	$\displaystyle=$	$\displaystyle T_{T\rightarrow U}+T_{T\rightarrow CI}.$

Furthermore, we provide an important index to describe the average level of each strategy. Concretely, the average levels of $CI,T,$ and $U$ strategies, averaging over all possible states i, weighted with the corresponding stationary distribution $\bar{p_{\textbf{i}}}$ , are computed as

$\displaystyle\bar{\rho}_{CI}$	$\displaystyle=$	$\displaystyle\sum_{\textbf{i}}\frac{\textbf{i}_{i_{CI}}\bar{p_{\textbf{i}}}}{Z},$
$\displaystyle\bar{\rho}_{T}$	$\displaystyle=$	$\displaystyle\sum_{\textbf{i}}\frac{\textbf{i}_{i_{T}}\bar{p_{\textbf{i}}}}{Z},$
$\displaystyle\bar{\rho}_{U}$	$\displaystyle=$	$\displaystyle\sum_{\textbf{i}}\frac{\textbf{i}_{i_{U}}\bar{p_{\textbf{i}}}}{Z},$

where $\textbf{i}_{i_{CI}},\textbf{i}_{i_{T}}$ , and $\textbf{i}_{i_{U}}$ denote the number of $CI,T$ , and $U$ agents in the configuration i, respectively.

Subsequently, by using $f_{S}(\textbf{i})$ combined with the stationary distribution $\bar{p}_{\textbf{i}}$ , we can calculate the average payoff of one agent with $S$ strategy as

\displaystyle\bar{f}_{S}=\sum_{\textbf{i}}\bar{p_{\textbf{i}}}f_{S}(\textbf{i}),

where $S=CI,T,$ or $U$ .

In the following, we investigate the gradient of selection and the stationary distribution to study the evolutionary dynamics of $CI$ , $T$ , and $U$ strategies in finite well-mixed populations.

4 Results

We first present the results of evolutionary dynamics in finite populations with the conditional investment strategy for different values of $M$ . In Fig. 2, we investigate the stationary distribution and the gradient of selection to study what roles conditional investment plays in the evolution of trust. Clearly, if the investment tendency is strong enough ( $M=0$ ), $CI$ agents will become unconditional investors. We can notice that the population will spend a significant time on configurations near the $TU$ -edge ( $T$ means trustworthy agents and $U$ means untrustworthy agents) of the triangle simplex, as shown in Fig. 2(a). Besides, most of the arrows flow to the coexistence states of $T$ and $U$ where $U$ is highly prevalent. The weakening of investment tendency will lead the population to spending most of the time in states with a high prevalence of $T$ and $CI$ (see Fig. 2(b)). If $M$ further increases to $M=N$ , $CI$ agents invest in the first round but refuse to invest during the remaining rounds. In this case, each agent only retains the benefits of the first round of the game, and no one can obtain non-zero payoffs from the remaining rounds. The evolutionary outcomes in finite populations are similar to the result reported in Fig. 2(a), that is, the system spends most of the time in states where $T$ and $U$ agents coexist (see Fig. 2(c)).

Furthermore, we investigate how the investment threshold influences the stationary frequency of strategies $CI,T,$ and $U$ , as shown in Fig. 3. We find that when the threshold $M$ is zero, $U$ can be more prevalent than $CI$ and $T$ . With the increase of threshold, $CI$ and $T$ agents can form a strong alliance, preventing the invasion of $U$ agents. If $M$ further increases to $N$ , $CI$ agents only invest in the first round, and thus all agents can only obtain benefit from the first round. In this case, the advantage of $U$ agents is greater than that of $T$ and $CI$ agents.

To probe deeper into the underlying mechanisms responsible for such a significant improvement of trust stemming from the investment threshold, we show in Fig. 4 how the overall average payoffs of $CI,T$ , and $U$ agents change with $M$ . It can be seen that the average payoffs of all agents increase first and then decrease. Particularly, when $M=0$ or $M=N$ , the average payoffs of $CI$ agents are negative (see the inset of Fig. 4). In general, an intermediate investment threshold can ensure a more evident advantage of alliance formed by $CI$ and $T$ agents over $U$ agents, even if this advantage can shrink with the increase of $M$ .

In order to illustrate the robustness of all results obtained in this paper, we first present how evolutionary dynamics of $CI,T$ , and $U$ change with the multiply factor of $U$ agents, $R_{U}$ (see Fig. 5). Concretely, we show how the average levels of three strategies vary with $R_{U}$ in Fig. 5(a), and we find the average levels of $CI,T$ , and $U$ remain almost constant with the increase of $R_{U}$ . In addition, we show the evolutionary outcomes of the gradient of selection and the stationary distribution, which allow visualization of the dynamics, for three different values $R_{U}$ in Fig. 5(b)-(d). It shows that the population spends a significant time near the $CI$ - $T$ edge of the simplex $S_{3}$ , and most of the arrows in the simplex flow to the intermediate region of $CI$ - $T$ edge, indicating that conditional investors can form an alliance with trustworthy trustees. It is worth pointing out that we set $R_{U}=6.66$ , 7.98, and $9.96$ , which can respectively correspond to the mild, moderate, and harsh interaction environment in the trust game (see Ref. [8]). Different from previous findings in [8], our results show that trust can be maintained no matter whether the environment of the trust game is mild or harsh.

In what follows, we investigate the impact of the investment cost of investors on the evolutionary dynamics (see Fig. 6). We find that with the increase of $tv$ value, the average levels of three strategies are almost unchanged (see Fig. 6(a)). In addition, we find that changing the $tv$ values does not influence the gradient of selection and the stationary distribution of the system (see Fig. 6(b)-(d)).

In Fig. 7, we further present how observation cost affects the evolutionary dynamics of $CI$ , $T$ , and $U$ strategies. We find that the increase of observation cost will not change the evolutionary advantage of $CI$ and $T$ alliance over $U$ agents. Concretely, the average level of $CI$ decreases with the observation cost, the average level of $T$ increases, but the average level of $U$ remains almost unchanged (see Fig. 7 (a)). Furthermore, we find that the entire population will spend most time near configurations in which $CI$ and $T$ agents coexist. In addition, with the increase of observation cost, the background shadow area is gradually moving towards the vertex $T$ (see Fig. 7(b)-(d)).

In Fig. 8, we investigate the role of discount factors $w$ in the evolutionary dynamics of $CI$ , $T$ , and $U$ strategies. Our results show that the average levels of these three strategies remain almost unchanged with the increase of $w$ (see Fig. 8 (a)). Besides, for three different $w$ values, we find that our results are robust, that is, the population will spend most of the time in the intermediate region of $CI-T$ edge, and most arrows flow to the coexistence states of $CI$ and $T$ (see Fig. 8(b)-(d)).

Furthermore, the effects of the mutation rate $\mu$ and the intensity of selection $\beta$ on the evolutionary dynamics are investigated (Figs. 9 and 10). When $\beta$ is small ( $\beta<10^{-3}$ ) or when $\mu$ is significantly large (close to 1), the evolutionary process is mainly affected by imitation error or behavioral mutation, which leads to the average levels of all strategies close to $1/3$ (see Figs. 9(a) and 10(a)). As $\beta$ increases or $\mu$ decreases, a strategy’s performance becomes increasingly important for the strategy’s survival, eventually favoring the emerging alliance of the $CI$ and $T$ strategies, which crowds out the most $U$ agents (see Fig. 9 (b)-(d) and Fig. 10 (b)-(d)).

In what follows, we investigate how evolutionary dynamics of $CI,T$ , and $U$ strategies change with the group size $N$ (Fig. 11). Here we set $M/N=0.5$ , and we find that with the increase of $N$ , the average levels of $CI,T$ , and $U$ remain almost unchanged (see Fig. 11(a)). In particular, the average level of $T$ is the highest, the second largest frequency is formed by $CI$ , while $U$ makes up the smallest fraction. In addition, the dynamic visualization results show that the population spends most time in the intermediate region of $CI-T$ edge, and most arrows point to the states where $CI$ and $T$ agents coexist when $N$ changes appropriately (see Fig. 11(b)-(d)).

At the end of this section, we would like to point out that the work done here can have wider implications on other areas such as cognitive science, behavioral science, artificial intelligence, economics, and management science. Particularly, in the management system, developing and maintaining trust can promote the emergence of social exchange and economic transactions, which is significantly important for the effectiveness of management and organization. By investigating the evolutionary dynamics of trust in repeated group interactions, we show that the introduction of the conditional investment strategy can lead to the emergence of high trust and high trustworthiness in repeated group interactions, which can explain why trusting in others and reciprocating them with trustworthy actions can be sustained in social and economic interactions.

5 Conclusions and Discussion

In this work, we study the evolutionary dynamics of the conditional investment strategy in the NTG in which agents engage in repeated group interactions. Different from the setting of NTG considered in previous works [1, 22], we consider that investors can adjust their investment decisions according to the investment environment in repeated group interactions framework. Concretely, we use the tolerance threshold as an important parameter to describe the investment tendency of investors. We find that an intermediate threshold can lead to the evolution of high trust and high trustworthiness, while too low threshold level or too high threshold level cannot promote the evolution of trust. In summary, as the answers to the initially proposed questions, we can conclude that the introduction of the conditional investment strategy provides an avenue for trust to thrive in repeated group interactions.

As we have emphasized above, our model setup is well aligned with reality in which investors generally make decisions based on the current investment environment rather than investing blindly. A good example where our model could apply is the trust relationship between consumers and suppliers [20]. Consumers will refuse to consume if they find more suppliers selling inferior products during daily shopping on Amazon. Therefore, the collection of information is particularly important. The key assumption of conditional investment is that investors pay a permanent observation cost $\sigma$ to collect information about the trustworthiness of the trustee, and then make decisions in the subsequent possible group interactions based on what they observe. We have shown that increasing $\sigma$ leads to the decrease of the frequency of conditional investors and the increase of the frequency of trustworthy agents, while has little effect on the frequency of untrustworthy agents. Therefore, the result that the alliance formed by $CI$ and $T$ agents can effectively resist the invasion of $U$ agents is robust to the observation cost.

Lastly, it is worth emphasizing that in addition to the observation cost, our results are more robust to the change of other model parameters. Previous work on networked NTG has revealed that the level of trust is correlated with how “difficult” the game is [8]. In our model, even though the social dilemma is extremely difficult (i.e. the ratio of temptation to defect $\frac{R_{U}}{R_{T}}>1.66$ ), a high level of trust can always be achieved (see Fig. 5(a)). Besides, for the sake of mathematical convenience, analysis of evolutionary dynamics of NTG has been mostly dealt with either in the limit of rare mutations [26] or in the limit of weak selection [44]. Here our approach can be applied to arbitrary mutation rate and arbitrary intensity of selection values. We find that appropriate changes in mutation rate and intensity of selection will not affect our main results (see Fig. 9 and Fig. 10).

Future work could explore the evolutionary dynamics of the repeated NTG in structured populations where the interactions among agents are typically not random but rather limited to a subset of the population [6, 43, 49]. Indeed, the network where agents interact with others has different structures, and the MDP approach adopted in our work is suitable for complete graph networks and can also be extended to other network structures [7]. Besides, we can use theoretical approximation and agent-based simulations to study the evolutionary dynamics of trust on any population structure [3].

CRediT authorship contribution statement

Linjie Liu: Conceptualization, Methodology, Writing - original draft. Xiaojie Chen: Conceptualization, Formal analysis, Writing - review & editing, Supervision.

Declaration of Competing Interest

The authors declare that they have no competing financial interests.

Acknowledgment

This work was supported by the National Natural Science Foundation of China (Grants Nos. 61976048 and 62036002) and the Fundamental Research Funds of the Central Universities of China. L.L. acknowledges the support from Special Project of Scientific and Technological Innovation (Grant No. 2452022107).

References

[1] H. Abbass, G. Greenwood, E. Petraki, $N$ -player trust game and its replicator dynamics, IEEE Trans. Evol. Comput. 20(3) (2016) 470-474.
[2] A. Ahmed, Conditional reciprocity in the investment game, Soc. Sci. J. 48(2) (2011) 404-408.
[3] B. Allen, G. Lippner, Y. T. Chen, B. Fotouhi, N. Momeni, S. T. Yau, M. A. Nowak, Evolutionary dynamics on any population structure, Nature 544(7649) (2017) 227-230.
[4] J. Berg, J. Dickhaut, K. McCabe, Trust, reciprocity, and social history, Games Econ. Behav. 10(1) (1995) 122-142.
[5] S. R. Bond, M. Söderbom, Conditional investment–cash flow sensitivities and financing constraints, J. Eur. Econ. Assoc. 11(1) (2013) 112-136.
[6] X. Chen, F. Fu, L. Wang, Interaction stochasticity supports cooperation in spatial prisoner’s dilemma, Phys. Rev. E 78(5) (2008) 051120.
[7] Y. Chen, K. R. Liu, Indirect reciprocity game modelling for cooperation stimulation in cognitive networks, IEEE Trans. Commun. 59(1) (2011) 159-168.
[8] M. Chica, R. Chiong, M. Kirley, H. Ishibuchi, A networked $N$ -player trust game and its evolutionary dynamics, IEEE Trans. Evol. Comput. 22(6) (2018) 866-878.
[9] M. Chica, R. Chiong, J. Ramasco, and H. Abbass, Effects of update rules on networked n-player trust game dynamics, Commun. Nonlinear Sci. Numer. Simul. 79(2019) 104870.
[10] R. Chiong, M. Kirley, Effects of iterated interactions in multiplayer spatial evolutionary games, IEEE Trans. Evol. Comput. 16(4) (2012) 537-555.
[11] S. Y. Chong, X. Yao, Behavioral diversity, choices and noise in the iterated prisoner’s dilemma, IEEE Trans. Evol. Comput. 9(6) (2005) 540-551.
[12] J. F. Crow, An Introduction to Population Genetics Theory, (Harper and Row New York 1970).
[13] M. Deutsch, Cooperation and trust: Some theoretical notes, Nebr. Sym. Motiv. (1962) 275–320.
[14] W. J. Ewens, Mathematical Population Genetics, (Springer New York 2004).
[15] L. Fang, G. Shi, L. Wang, Y. Li, S. Xu, Y. Guo, Incentive mechanism for cooperative authentication: An evolutionary game approach. Inf. Sci. 527 (2020) 369-381
[16] X. Fang, X. Chen, Evolutionary dynamics of trust in the N-player trust game with individual reward and punishment, Eur. Phys. J. B 94 (2021) 176. .
[17] P. García-Victoria, M. Cavaliere, M. A. Gutiérrez-Naranjo, M. Cárdenas-Montes, Evolutionary game theory in a cell: A membrane computing approach. Inf. Sci. 589 (2022) 580-594.
[18] W. Güth, P. Ockenfels, M. Wendel, Cooperation based on trust. An experimental investigation, J. Econ. Psychol. 18(1) (1997) 15-43.
[19] T. A. Han, C. Perret, S. T. Powers, When to (or not to) trust intelligent machines: Insights from an evolutionary game theory analysis of trust in repeated games, Cogn. Syst. Res. 68 (2021) 111-124.
[20] F. Hawlitschek, T. Teubner, C. Weinhardt, Trust in the sharing economy, Die Unternehmung 70(1) (2016) 26-44.
[21] C. Hilbe, S̆. S̆imsa, K. Chatterjee, M. A. Nowak, Evolution of cooperation in stochastic games, Nature 559(7713) (2018) 246-249.
[22] Z. Hu, X. Li, J. Wang, C. Xia, Z. Wang, M. Perc, Adaptive reputation promotes trust in social networks, IEEE Trans. Net. Sci. Eng. 40(1) (2021) 2327-4697.
[23] B. King-Casas, D. Tomlin, C. Anen, C. F. Camerer, S. R. Quartz, P. R. Montague, Getting to know you: reputation and trust in a two-person economic exchange, Science 308(5718) (2005) 78–83.
[24] A. Kumar, V. Capraro, and M. Perc, The evolution of trust and trustworthiness, J. R. Soc. Interface 17(169) (2020) 20200491.
[25] H. Liang, Y. Cui, X. Ren, X. Wang, Almost sure exponential stability of two-strategy evolutionary games with multiplicative noise. Inf. Sci. 579 (2021) 888-903.
[26] I. S. Lim, Stochastic evolutionary dynamics of trust games with asymmetric parameters, Phys. Rev. E 102(6) (2020) 062419.
[27] L. Liu, X. Chen, Evolutionary game dynamics in multiagent systems with prosocial and antisocial exclusion strategies, Knowl-Based Syst. 188 (2020) 104835.
[28] L. Liu, Z. Xiao, X. Chen, A. Szolnoki, Early exclusion leads to cyclical cooperation in repeated group interactions. J. R. Soc. Interface 19(188) (2022) 20210755.
[29] G. Martínez-Cánovas, E. Del Val, V. Botti, P. Hernández, M. Rebollo, A formal model based on game theory for the analysis of cooperation in distributed service discovery. Inf. Sci. 326 (2016) 59-70.
[30] L. A. Martinez-Vaquero, F. C. Santos, V. Trianni, Signalling boosts the evolution of cooperation in repeated group interactions, J. R. Soc. Interface 17(172) (2020) 20200635.
[31] M. Perc, J. Gómez-Gardenes, A. Szolnoki, L. M. Floría, Y. Moreno, Evolutionary dynamics of group interactions on structured populations: a review, J. R. Soc. Interface, 10(80) (2013) 20120997.
[32] M. Perc, J. J. Jordan, D. G. Rand, Z. Wang, S. Boccaletti, and A. Szolnoki, The evolution of trust and cooperation between strangers: A computational model, Phys. Rep. 687 (2017) 1-51.
[33] J. Quan, Y. Zhou, X. Ma, X. Wang, J. B. Yang, Integrating emotion-imitating into strategy learning improves cooperation in social dilemmas with extortion, Knowl-Based Syst. 233 (2021) 107550.
[34] J. Quan, X. Li, X. Wang, The evolution of cooperation in spatial public goods game with conditional peer exclusion, Chaos 29(10) (2019) 103137.
[35] J. Quan, W. Liu, Y. Chu, X. Wang, Stochastic evolutionary voluntary public goods game with punishment in a Quasi-birth-and-death process. Sci. Rep. 7 (2017) 16110.
[36] J. Quan, J. Zheng, X. Wang, X. Yang, Benefits of asynchronous exclusion for the evolution of cooperation in stochastic evolutionary optional public goods games. Sci. Rep. 9 (2019) 8208.
[37] K. Sigmund, D. H. Silva, A. Traulsen, C. Hauert, Social learning promotes institutions for governing the commons, Nature 466(7308) (2010) 861-863.
[38] W. Sun, L. Liu, X. Chen, A. Szolnoki, V. V. Vasconcelos, Combination of institutional incentives for cooperative governance of risky commons, iScience 24(8) (2021) 102844.
[39] R. L. Swinth, The establishment of the trust relationship, J. Conflict Resolut 11(3) (1967) 335–344.
[40] G. Szabó, C. Tőke, Evolutionary prisoner’s dilemma game on a square lattice, Phys. Rev. E 58(1) (1998) 69.
[41] A. Szolnoki, M. Perc, Correlation of positive and negative reciprocity fails to confer an evolutionary advantage: Phase transitions to elementary strategies, Phys. Rev. X, 3(4) (2013) 041021.
[42] A. Szolnoki, M. Mobilia, L. L. Jiang, B. Szczesny, A. M. Rucklidge, M. Perc, Cyclic dominance in evolutionary games: a review, J. R. Soc. Interface, 11(100) (2014) 20140735.
[43] A. Szolnoki, M. Perc, Conformity enhances network reciprocity in evolutionary social dilemmas, J. R. Soc. Interface 12(103) (2015) 20141299.
[44] C. E. Tarnita, Fairness and trust in structured populations, Games 6(3) (2015) 214-230.
[45] R. Urena, G. Kou, Y. Dong, F. Chiclana, E. Herrera-Viedma, A review on trust propagation and opinion dynamics in social networks and group decision making frameworks. Inf. Sci. 478 (2019) 461-475.
[46] K. Van, G. Nicolaas, Stochastic Processes in Physics and Chemistry, (Elsevier 1992).
[47] V. V. Vasconcelos, F. C. Santos, J. M. Pacheco, A bottom-up institutional approach to cooperative governance of risky commons, Nat. Clim. Change 3(9) (2013) 797–801.
[48] C. Zhang, S. Liu, Z. Wang, F. J. Weissing, J. Zhang, The “self-bad, partner-worse” strategy inhibits cooperation in networked populations, Inf. Sci. 585 (2022) 58-69.
[49] J. Zhang, J. Lou, J. Qiu, J. Lu, Dynamics and convergence of hyper-networked evolutionary games with time delay in strategies, Inf. Sci. 563 (2021) 166-182.