Black-Box Strategies and Equilibrium for Games with Cumulative Prospect Theoretic Players

There is a large amount of evidence that human agents as decision-makers do not conform to the independence axiom of expected utility theory (EUT). (See, for example, Allais (1953); Weber and Camerer (1987) and Machina (1992).) This has led to the study of several alternate theories that do away with the independence axiom (Machina, 2014). Amongst these, the cumulative prospect theory (CPT) of Tversky and Kahneman (1992) stands out since it accommodates many of the empirically observed behavioral features from human experiments without losing much analytical tractability (Wakker, 2010). Further, it includes EUT as a special case.

The independence axiom says that if lottery $L_{1}$ is weakly preferred over lottery $L_{2}$ by an agent (i.e. the agent wants lottery $L_{1}$ at least as much as lottery $L_{2}$), and $L$ is some other lottery, then, for $0\leq\alpha\leq 1$, the combined lottery $\alpha L_{1}+(1-\alpha)L$ is weakly preferred over the combined lottery $\alpha L_{2}+(1-\alpha)L$ by that agent. A weakened form of the independence axiom, called betweenness, says that if lottery $L_{1}$ is weakly preferred over lottery $L_{2}$ (by an agent), then, for any $0\leq\alpha\leq 1$, the mixed lottery $L=\alpha L_{1}+(1-\alpha)L_{2}$ must lie between the lotteries $L_{1}$ and $L_{2}$ in preference. Betweenness implies that if an agent is indifferent between $L_{1}$ and $L_{2}$, then she is indifferent between any mixtures of them too. It is known that independence implies betweenness, but betweenness does not imply independence (Chew, 1989). As a result, EUT preferences, which are known to satisfy the independence axiom, also satisfy betweenness. CPT preferences, on the other hand, do not satisfy betweenness in general (see example 2.2). In fact, in theorem 2.3, we show that CPT preferences satisfy betweenness if and only if they are EUT preferences (recall that EUT preferences are a special case of CPT preferences). Several empirical studies show systematic violations of betweenness (Camerer and Ho, 1994; Agranov and Ortoleva, 2017; Dwenger et al., 2012; Sopher and Narramore, 2000), and this makes the use of CPT more attractive than EUT for modeling human preferences. Further evidence comes from Camerer and Ho (1994), where the authors fit data from nine studies using three non-EUT models, one of them being CPT, to find that, compared to the EUT model, the non-EUT models perform better.

Suppose in a non-cooperative game that given her beliefs about the other players, a player is indifferent between two of her actions. Then according to EUT, she should be indifferent between any of the mixtures of these two actions. This facilitates the proof of the existence of a Nash equilibrium in mixed actions for such games. However, with CPT preferences, the player could either prefer some mixture of these two actions over the individual actions or vice versa.

As a result, it is important to make a distinction in CPT regarding whether the players can actively randomize over their actions or not. One way to enable active randomization is by assuming that each player has access to a randomizing device and the player can “commit” to the outcome of this randomization. The commitment assumption is necessary, as is evident from the following scenario (the gambles presented below appear in Prelec (1990)). Alice needs to choose between the following two actions:

1. 1.

   Action $1$ results in a lottery $L_{1}=\{(0.34,\$20{,}000);(0.66,\$0)\}$, i.e. she receives $\$20{,}000$ with probability $0.34$ and nothing with probability $0.66$.

2. 2.

   Action $2$ results in a lottery $L_{2}=\{(0.17,\$30{,}000);(0.83,\$0)\}$.

(See example 3.7 for an instance of a $2$-player game with Alice and Bob, where Alice has two actions that result in the above two lotteries.) Note that $L_{1}$ is a less risky gamble with a lower reward and $L_{2}$ is a more risky gamble with a higher reward. Now consider a compound lottery $L=16/17L_{1}+1/17L_{2}$. Substituting for the lotteries $L_{1}$ and $L_{2}$ we get $L$ in its reduced form to be

In example 2.1, we provide a CPT model for Alice’s preferences that result in lottery $L_{1}$ being preferred over lottery $L_{2}$, whereas lottery $L$ is preferred over lotteries $L_{1}$ and $L_{2}$. Roughly speaking, the underlying intuition is that Alice is risk-averse in general, and she prefers lottery $L_{1}$ over lottery $L_{2}$. However, she overweights the small $1\%$ chance of getting $\$30{,}000$ in $L$ and finds it lucrative enough to make her prefer lottery $L$ over both the lotteries $L_{1}$ and $L_{2}$. Let us say Alice has a biased coin that she can use to implement the randomized strategy. Now, if Alice tossed the coin, and the outcome was to play action 2, then in the absence of commitment, she will switch to action 1, since she prefers lottery $L_{1}$ over lottery $L_{2}$. Commitment can be achieved, for example, by asking a trusted party to implement the randomized strategy for her or use a device that would carry out the randomization and implement the outcome without further consultation with Alice. Regardless of the implementation mechanism, we will call such randomized strategies black-box strategies. The above problem of commitment is closely related to the problem of using non-EUT models in dynamic decisions. For an interesting discussion on this topic, see Wakker (2010, Appendix C) and the references therein.

Traditionally, mixed actions have been considered from two viewpoints, especially in the context of mixed action Nash equilibrium. According to the first viewpoint, these are conscious randomizations by the players – each player only knows her mixed action and not its pure realization. The notion of black-box strategies captures this interpretation of mixed actions. According to the other viewpoint, players do not randomize, and each player chooses some definite action, but the other players need not know which one, and the mixture represents their uncertainty, i.e. their conjecture about her choice. Aumann and Brandenburger (1995) establish mixed action Nash equilibrium as an equilibrium in conjectures provided they satisfy certain epistemic conditions regarding the common knowledge amongst the players.

In the absence of the betweenness condition, these two viewpoints give rise to different notions of Nash equilibria. Throughout we assume that the player set and their corresponding action sets and payoff functions, as well as the rationality of each player, are common knowledge. A player is said to be rational if, given her beliefs and her preferences, she does not play any suboptimal strategy. Suppose each player plays a fixed action, and these fixed actions are common knowledge, then we get back the notion of pure Nash equilibrium (see definition 3.2). If each player plays a fixed action, but the other players have mixed conjectures over her action, and these conjectures are common knowledge, then this gives us mixed action Nash equilibrium (see definition 3.4). This coincides with the notion of Nash equilibrium as defined in Keskin (2016) and studied further in Phade and Anantharam (2019). Now suppose each player can randomize over her actions and hence implement a black-box strategy. If each player plays a fixed black-box strategy and these black-box strategies are common knowledge, then this gives rise to a new notion of equilibrium. We call it black-box strategy Nash equilibrium (see definition 3.8). If each player plays a fixed black-box strategy and the other players have mixed conjectures over her black-box strategy, and these conjectures are common knowledge, then we get the notion of mixed black-box strategy Nash equilibrium (see definition 3.10).

In the setting of an $n$-player normal form game with real valued payoff functions, the pure Nash equilibria do not depend on the specific CPT features of the players, i.e. the reference point, the value function and the two probability weighting functions, one for gains and one for losses. Hence the traditional result on the lack of guarantee for the existence of a pure Nash equilibrium continues to hold when players have CPT preferences. Keskin (2016) proves the existence of a mixed action Nash equilibrium for any finite game when players have CPT preferences. In example 3.9, we show that a finite game may not have any black-box strategy Nash equilibrium. On the other hand, in theorem 3.12, we prove our main result that for any finite game with players having CPT preferences, there exists a mixed black-box strategy Nash equilibrium. If the players have EUT preferences, then the notions of black-box strategy Nash equilibrium and mixed black-box strategy Nash equilibrium are equivalent to the notion of mixed action Nash equilibrium (when interpreted appropriately; see the remark before proposition 3.14; see also figure 6).

The paper is organized as follows. In section 2, we describe the CPT setup and establish that under this setup betweenness is equivalent to independence (theorem 2.3). In section 3, we describe an $n$-player non-cooperative game setup and define various notions of Nash equilibrium in the absence of betweenness, in particular with CPT preferences. We discuss the questions concerning their existence and how these different notions of equilibria compare with each other. In section 4, we conclude with a table that summarizes the results.

To close this section, we introduce some notational conventions that will be used in the document. If $Z$ is a Polish space (complete separable metric space), let $\mathscr{P}(Z)$ denote the set of all probability measures on $(Z,\mathscr{F})$, where $\mathscr{F}$ is the Borel sigma-algebra of $Z$. Let $\mathop{\rm supp}(p)$ denote the support of a distribution $p\in\mathscr{P}(Z)$, i.e. the smallest closed subset of $Z$ such that $p(Z)=1$. Let $\Delta_{f}(Z)\subset\mathscr{P}(Z)$ denote the set of all probability distributions that have a finite support. For any element $p\in\Delta_{f}(Z)$, let $p[z]$ denote the probability of $z\in Z$ assigned by $p$. For $z\in Z$, let $\textbf{1}\{z\}\in\Delta_{f}(Z)$ denote the probability distribution such that $p[z]=1$. If $Z$ is finite (and hence a Polish space with respect to the discrete topology), let $\Delta(Z)$ denote the set of all probability distributions on the set $Z$, viz.

with the usual topology. Let $\Delta^{m-1}$ denote the standard $(m-1)$-simplex, i.e. $\Delta(\{1,\dots,m\})$. If $Z$ is a subset of a Euclidean space, then let $co(Z)$ denote the convex hull of $Z$, and let $\overline{co}(Z)$ denote the closed convex hull of $Z$.

We first describe the setup for CPT (for more details see Wakker (2010)). Each person is associated with a reference point $r\in\mathbb{R}$, a value function $v:\mathbb{R}\to\mathbb{R}$, and two probability weighting functions $w^{\pm}:[0,1]\to[0,1]$, $w^{+}$ for gains and $w^{-}$ for losses. The function $v(x)$ satisfies: (i) it is continuous in $x$, (ii) $v(r)=0$, (iii) it is strictly increasing in $x$. The value function is generally assumed to be convex in the losses frame ($x<r$) and concave in the gains frame ($x\geq r$), and to be steeper in the losses frame than in the gains frame in the sense that $v(r-z)\leq-v(r+z)$ for all $z\geq 0$. However, these assumptions are not needed for the results in this paper to hold. The probability weighting functions $w^{\pm}:[0,1]\to[0,1]$ satisfy: (i) they are continuous, (ii) they are strictly increasing, (iii) $w^{\pm}(0)=0$and $w^{\pm}(1)=1$. We say that $(r,v,w^{\pm})$ are the CPT features of that person.

Suppose a person faces a lottery (or prospect) $L:=\{(p_{k},z_{k})\}_{1\leq k\leq m}$, where $z_{k}\in\mathbb{R},1\leq k\leq m$, denotes an outcome and $p_{k},1\leq k\leq m$, is the probability with which outcome $z_{k}$ occurs. We assume that $\sum_{k=1}^{m}p_{k}=1$. (Note that we are allowed to have $p_{k}=0$ for some values of $k$, and we can have $z_{k}=z_{k^{\prime}}$ even when $k\neq k^{\prime}$.) Let $z:=(z_{k})_{1\leq k\leq m}$ and $p:=(p_{k})_{1\leq k\leq m}$. We denote $L$ as $(p,z)$ and refer to the vector $z$ as an outcome profile and $p$ as a probability vector.

Let $\alpha:=(\alpha_{1},\dots,\alpha_{m})$ be a permutation of $(1,\dots,m)$ such that

Let $0\leq k_{r}\leq m$ be such that $z_{\alpha_{k}}\geq r$ for $1\leq k\leq k_{r}$ and $z_{\alpha_{k}}<r$ for $k_{r}<k\leq m$. (Here $k_{r}=0$ when $z_{\alpha_{k}}<r$ for all $1\leq k\leq m$.) The CPT value $V(L)$ of the prospect $L$ is evaluated using the value function $v(\cdot)$ and the probability weighting functions $w^{\pm}(\cdot)$ as follows:

where $\pi^{+}_{k}(p,\alpha),1\leq k\leq k_{r},$ and $\pi^{-}_{k}(p,\alpha),k_{r}<k\leq m,$ are decision weights defined via:

Although the expression on the right in equation (2.2) depends on the permutation $\alpha$, one can check that the formula evaluates to the same value $V(L)$ as long as the permutation $\alpha$ satisfies (2.1). The CPT value in equation (2.2) can equivalently be written as:

A person is said to have CPT preferences if, given a choice between prospect $L_{1}$ and prospect $L_{2}$, she chooses the one with higher CPT value.

We now define some axioms for preferences over lotteries. We are interested in “mixtures” of lotteries, i.e. lotteries with other lotteries as outcomes. Consider a (two-stage) compound lottery $K:=\{(q^{j},L^{j})\}_{1\leq j\leq t}$, where $L^{j}=(p^{j},z^{j}),1\leq j\leq t$, are lotteries over real outcomes and $q^{j}$ is the chance of lottery $L^{j}$. We assume that $\sum_{j=1}^{t}q^{j}=1$. A two-stage compound lottery can be reduced to a single-stage lottery by multiplying the probability vector $p^{j}$ corresponding to the lottery $L^{j}$ by $q^{j}$ for each $j,1\leq j\leq t$, and then adding the probabilities of identical outcomes across all the lotteries $L^{j},1\leq j\leq t$. Let $\sum_{j=1}^{t}q^{j}L^{j}$ denote the reduced lottery corresponding to the compound lottery $K$.

Let $\preceq$ denote a preference relation over single-stage lotteries. We assume $\preceq$ to be a weak order, i.e. $\preceq$ is transitive (if $L_{1}\preceq L_{2}$ and $L_{2}\preceq L_{3}$, then $L_{1}\preceq L_{3}$) and complete (for all $L_{1},L_{2}$, we have $L_{1}\preceq L_{2}$ or $L_{2}\preceq L_{1}$, where possibly both preferences hold). The additional binary relations $\succeq,\sim,\prec$ and $\succ$ are derived from $\preceq$ in the usual manner. A preference relation $\preceq$ is a CPT preference relation if there exist CPT features $(r,v,w^{\pm})$ such that $L_{1}\preceq L_{2}$ iff $V(L_{1})\leq V(L_{2})$. Note that a CPT preference relation is a weak order. A preference relation $\preceq$ satisfies independence if for any lotteries $L_{1},L_{2}$ and $L$, and any constant $0\leq\alpha\leq 1$, $L_{1}\preceq L_{2}$ implies $\alpha L_{1}+(1-\alpha)L\preceq\alpha L_{2}+(1-\alpha)L$. A preference relation $\preceq$ satisfies betweenness if for any lotteries $L_{1}\preceq L_{2}$, we have $L_{1}\preceq\alpha L_{1}+(1-\alpha)L_{2}\preceq L_{2}$, for all $0\leq\alpha\leq 1$. A preference relation $\preceq$ satisfies weak betweenness if for any lotteries $L_{1}\sim L_{2}$, we have $L_{1}\sim\alpha L_{1}+(1-\alpha)L_{2}$, for all $0\leq\alpha\leq 1$.

Suppose a preference relation $\preceq$ satisfies independence. Then $L_{1}\preceq L_{2}$ implies

Thus, if a preference relation satisfies independence, then it satisfies betweenness. Also, if a preference relation satisfies betweenness, then it satifies weak betweenness.

In the following example, we will provide CPT features for Alice so that her preferences agree with those described in section 1. This example also shows that cumulative prospect theory can give rise to preferences that do not satisfy betweenness.

The following example shows that CPT can give rise to preferences that do not satisfy weak betweenness (the lotteries and the CPT features presented below appear in Keskin (2016)).

Given a utility function $u:\mathbb{R}\to\mathbb{R}$ (assumed to be continuous and strictly increasing), the expected utility of a lottery $L=\{(p_{k},z_{k})\}_{1\leq k\leq m}$ is defined as $U(L):=\sum_{k=1}^{m}p_{k}u(z_{k})$. A preference relation $\preceq$ is said to be an EUT preference relation if there exists a utility function $u$ such that $L_{1}\preceq L_{2}$ iff $U(L_{1})\leq U(L_{2})$. Note that if the CPT probability weighting functions are linear, i.e. $w^{\pm}(p)=p$ for $0\leq p\leq 1$, then the CPT value of a lottery coincides with the expected utility of that lottery with respect to the utility function $u=v$. It is well known that EUT preference relations satisfy independence and hence betweenness. Several generalizations of EUT have been obtained by weakening the independence axiom and assuming only betweenness, for example, weighted utility theory (Chew and MacCrimmon, 1979; Chew, 1983), skew-symmetric bilinear utility (Fishburn, 1988; Bordley and Hazen, 1991), implicit expected utility (Dekel, 1986; Chew, 1989) and disappointment aversion theory (Gul, 1991; Bordley, 1992). The following theorem shows that in the restricted setting of CPT preferences, betweenness and independence are equivalent.

We now consider an $n$-player non-cooperative game where the players have CPT preferences. We will discuss several notions of equilibrium for such a game and will contrast them.

Let $\Gamma:=(N,(A_{i})_{i\in N},(x_{i})_{i\in N})$ denote a game, where $N:=\{1,\dots,n\}$ is the set of players, $A_{i}$ is the finite action set of player $i$, and $x_{i}:A\to\mathbb{R}$ is the payoff function for player $i$. Here $A:=\prod_{i}A_{i}$ denotes the set of all action profiles $a:=(a_{1},\dots,a_{n})$. Let $A_{-i}:=\prod_{i\neq j}A_{j}$ denote the set of action profiles $a_{-i}$ of all players except player $i$.

The notion of pure Nash equilibrium is the same whether the players have CPT preferences or EUT preferences because only deterministic lotteries, comprised of being offered one outcome with probability $1$, are considered in the framework of this notion. It is well known that for any given game $\Gamma$, a pure Nash equilibrium need not exist.

Let $\mu_{-i}\in\Delta(A_{-i})$ denote a belief of player $i$ on the action profiles of her opponents. Given the belief $\mu_{-i}$ of player $i$, if she decides to play action $a_{i}$, then she will face the lottery $\{(\mu_{-i}[a_{-i}],x_{i}(a_{i},a_{-i}))\}_{a_{-i}\in A_{-i}}.$

Note that this definition is consistent with the definition of the best response action set that takes an action profile $a_{-i}$ of the opponents as its input (definition 3.1), if we interpret $a_{-i}$ as the belief $\textbf{1}\{a_{-i}\}\in\Delta(A_{-i})$, since $\mathscr{A}_{i}(\textbf{1}\{a_{-i}\})=\mathscr{A}_{i}(a_{-i})$.

Let $\sigma_{i}\in\Delta(A_{i})$ denote a conjecture over the action of player $i$. Let $\sigma:=(\sigma_{1},\dots,\sigma_{n})$ denote a profile of conjectures, and let $\sigma_{-i}:=(\sigma_{j})_{j\neq i}$ denote the profile of conjectures for all players except player $i$. Let $\mu_{-i}(\sigma_{-i})\in\Delta(A_{-i})$ be the belief induced by conjectures $\sigma_{j},j\neq i$, given by

which is nothing but the product distribution induced by $\sigma_{-i}$.

In other words, the conjecture $\sigma_{i}$ over the action of player $i$ should assign positive probabilities to only optimal actions of player $i$, given her belief $\mu_{-i}(\sigma_{-i})$.

It is well known that a mixed Nash equilibrium exists for every game with EUT players, see Nash (1951). Keskin (2016) generalizes the result of Nash (1951) on the existence of a mixed action Nash equilibrium to the case when players have CPT preferences.

Let $B_{i}:=\Delta(A_{i})$ denote the set of all black-box strategies for player $i$ with a typical element denoted by $b_{i}\in B_{i}$. Recall that if player $i$ implements a black-box strategy $b_{i}$, then we interpret this as a trusted party other than the player sampling an action $a_{i}\in A_{i}$ from the distribution $b_{i}$ and playing action $a_{i}$ on behalf of player $i$. We assume the usual topology on $B_{i}$. Let $B:=\prod_{i}B_{i}$ and $B_{-i}:=\prod_{j\neq i}B_{j}$ with typical elements denoted by $b$ and $b_{-i}$, respectively.

Note that, although a conjecture $\sigma_{i}$ and a black-box strategy $b_{i}$ are mathematically equivalent, viz. they are elements of the same set $B_{i}=\Delta(A_{i})$, they have different interpretations. We will call $s_{i}\in\Delta(A_{i})$ a mixture of actions of player $i$ when we want to be agnostic to which interpretation is being imposed. Let $S_{i}:=\Delta(A_{i}),S:=\prod_{i}\Delta(A_{i})$ and $S_{-i}:=\prod_{j\neq i}S_{i}$ with typical elements denoted by $s_{i},s$ and $s_{-i}$, respectively. (Note that $S\neq\Delta(A)$ unless all but one player have singleton action sets.)

For any belief $\mu_{-i}\in\Delta(A_{-i})$ and any black-box strategy $b_{i}$ of player $i$, let $\mu(b_{i},\mu_{-i})\in\Delta(A)$ denote the product distribution given by

Given the belief $\mu_{-i}$ of player $i$, if she decides to implement the black-box strategy $b_{i}$, then she will face the lottery $\{\mu(b_{i},\mu_{-i})[a],x_{i}(a))\}_{a\in A}$.

Let us compare the two concepts: the best response action set (definition 3.3) and the best response black-box strategy set (definition 3.5). Even though both of them take the belief $\mu_{-i}$ of player $i$ as input, the best response action set $\mathscr{A}_{i}(\mu_{-i})$ outputs a collection of actions of player $i$, whereas the best response black-box strategy set $\mathscr{B}_{i}(\mu_{-i})$ outputs a collection of black-box strategies of player $i$, which are probability distributions over the set of actions $a_{i}\in A_{i}$. If we interpret an action $a_{i}$ as the mixture $\textbf{1}\{a_{i}\}\in S_{i}=\Delta(A_{i})$, and a black-box strategy $b_{i}$ as a mixture as well, then we can compare the two sets $\mathscr{A}(\mu_{-i})$ and $\mathscr{B}(\mu_{-i})$ as subsets of $S_{i}$. The following example shows that, in general, the two sets can be disjoint, and hence quite distinct.

For any black-box strategy profile $b_{-i}$ of the opponents, let $\mu_{-i}(b_{-i})\in\Delta(A_{-i})$ be the induced belief given by

If the players have EUT preferences, a conjecture profile $\sigma=(\sigma_{1},\dots,\sigma_{n})$ is a mixed action Nash equilibrium if and only if the black-box strategy profile $b=(b_{1},\dots,b_{n})$, where $b_{i}=\sigma_{i}$, for all $i\in N$, is a black-box strategy Nash equilibrium. Thus, under EUT, the notion of a black-box strategy Nash equilibrium is equivalent to the notion of a mixed action Nash equilibrium, although there is still a conceptual difference between these two notions based on the interpretations for the mixtures of actions. Further, we have the existence of a black-box strategy Nash equilibrium for any game when players have EUT preferences from the well-known result about the existence of a mixed action Nash equilibrium. The following example shows that, in general, a black-box strategy Nash equilibrium may not exist when players have CPT preferences.