Incentive Compatibility in Two-Stage Repeated Stochastic Games

On each day $l,$ each player $i$ is required to report its supertype to the social planner in the first stage so that the latter can compute the optimal first-stage outcome. Since the players’ supertypes are assumed to remain the same on all days, it suffices for the players to bid their supertypes just once, namely, in the first stage of the game on day $1.$ Owing to strategic reasons that will be clear shortly, the players may not bid their supertypes truthfully, and so we denote by ${\widehat{\theta}_{i}}$ the supertype bid of player $i$ and by $\sigma_{i}\mathrel{\mathop{\mathchar 58\relax}}\Theta\to\Theta$ the first-stage strategy according to which player $i$ constructs its supertype bid. Therefore, $\widehat{\theta}_{i}=\sigma_{i}(\theta_{i}).$ Once all players submit their supertype bids, the social planner computes the first-stage outcome as $g_{1}^{*}(\boldsymbol{\sigma}(\boldsymbol{\theta})),$ where $\boldsymbol{\sigma}(\boldsymbol{\theta})\coloneqq[\sigma_{1}(\theta_{1}),\ldots,\sigma_{n}(\theta_{n})].$ The game then proceeds to the second stage.

In the second stage on each day $l$, each player $i$ observes the realization of $\delta_{i}(l)$ which it is supposed to report to the social planner. However, owing to strategic reasons that will become clear shortly, the players may not bid their type realizations truthfully, and so we denote by $\widehat{\delta}_{i}(l)$ player $i$’s type bid on day $l.$ We allow for the player to construct its type bid on any day $l$ using all information available to it until day $l$, and in accordance with any randomized, history-dependent policy of its choosing. Specifically, a second-stage bidding policy $\mu$ of player $i$ is a rule which specifies for each $o_{1}\in\mathcal{O}_{1}$ and each $l\in\mathbb{Z}_{+},$ a probability transition kernel $\mathbb{P}_{\mu}(\widehat{\delta}_{i}(l)\big{|}\delta_{i}^{l},\widehat{\delta}_{i}^{l-1},o_{2}^{l-1};o_{1})$ according to which player $i$ constructs its second-stage bid $\widehat{\delta}_{i}(l)$ on day $l$ if the first-stage outcome is $o_{1}$. We denote by $\Pi_{i}$ the set of all second-stage bidding policies available to player $i.$

Note that the second stage bidding policy is a rule that maps the history of observations available to a player to its second-stage bid. While the outcome of the rule is random owing to the types and second-stage outcomes being random, there is nothing random about the rule itself. Consequently, a player without any loss of generality can choose its second-stage bidding policy right on day $1$ as a function of its supertype. This leads to the notion of the second-stage strategy which is described next.

A second-stage strategy of player $i$ is a function $\pi_{i}\mathrel{\mathop{\mathchar 58\relax}}\Theta\to\Pi_{i}$ which specifies the second-stage bidding policy that it employs as a function of its private supertype $\theta_{i}.$ Therefore, $\pi_{i}(\theta_{i})$ is the second-stage bidding policy employed by player $i.$

Once all players submit their type bids, the social planner computes the second-stage outcome for day $l$ as $o_{2}(l)=g_{2}^{*}(\boldsymbol{\sigma}(\boldsymbol{\theta}),\widehat{\boldsymbol{\delta}}(l))$. Note that once the players’ first-stage and second-stage bidding policies are fixed, a functional relationship is established between the types and the type bids, and all random variables become well-defined.

We refer to the composition of the first- and second-stage strategies simply as a strategy. I.e, $S_{i}\coloneqq(\sigma_{i},{\pi}_{i})$ is referred to as the strategy of player $i.$ We denote by $\Lambda_{i}$ the set of strategies available to player $i.$ Finally, we refer to $\boldsymbol{S}\coloneqq(S_{1},\ldots,S_{n})$ as the strategy profile of the players and denote by $\Lambda$ the set of strategy profiles $\Lambda_{1}\times\ldots\times\Lambda_{n}$.

The stochasticity of the player types necessitates the definition of truthful strategy to be weaker than requiring a player to bid its type truthfully on all days.

The social planner collects a payment from each player at the end of each day that is determined as a function of the bids that they submit until that day. We denote by $p_{i,l}\mathrel{\mathop{\mathchar 58\relax}}\Theta_{1}\times\ldots\times\Theta_{n}\times\Delta_{1}^{l}\times\ldots\times\Delta_{n}^{l}\to\mathbb{R}$ the payment rule so that $p_{i,l}(\boldsymbol{\widehat{\theta}},\widehat{\boldsymbol{\delta}}^{l})$ specifies the amount that player $i$ should pay on day $l$. The utility accrued by player $i$ is defined as

Note that a player’s utility is a random variable that depends on the realization of the type sequence $\boldsymbol{\delta}^{\infty}.$

As mentioned in Section I, a “mild” behavioral assumption, one that is quite likely to hold in practice, is that no player behaves in a manner that might result in its own bankruptcy. This is captured by the notion of a non-bankrupting strategy.

We are now ready to introduce a notion of equilibrium that is “slightly” weaker than dominant strategy equilibrium.

It is perhaps instructive to contrast DNBE with Dominant Strategy Equilibrium (DSE) and Nash Equilibrium (NE) to gain a better appreciation of the notion. Note that for a strategy profile $\boldsymbol{S}$ to form a Nash equilibrium, it must hold for every $i\in\{1,\ldots,n\}$ that $S_{i}$ is a best response to $\boldsymbol{S}_{-i}.$ On the other hand, for the strategy profile $\boldsymbol{S}$ to form a DNBE, we must have for all $i\in\{1,\ldots,n\}$ that $S_{i}$ is a best response not only to $\boldsymbol{S}_{-i}$, but also to all $\boldsymbol{S}^{\prime}_{-i}\in\mathcal{NB}_{-i}.$ It follows that any dominant strategy non-bankrupting equilibrium is also a Nash equilibrium but not vice-versa. The stronger notion of dominant strategy equilibrium requires for all $i\in\{1,\ldots,n\}$ that $S_{i}$ is a best response to every $\boldsymbol{S}^{\prime}_{-i}\in\Lambda_{-i},$ and not just to those in $\mathcal{NB}_{-i}$ as required by DNBE. Hence, any dominant strategy equilibrium is also a dominant strategy non-bankrupting equilibrium. Fig. 1 illustrates the hierarchy formed by these equilibrium notions.

Arbitrarily fix the strategy profile $\boldsymbol{S}$ of the players. The long-term average social welfare that results from the game is

The objective of the social planner is to ensure that the average social welfare $q(\boldsymbol{S},\boldsymbol{\theta},\boldsymbol{\delta}^{\infty})$ equals the optimal value $W^{*}(\boldsymbol{\theta})$ that would result almost surely if all players employ a truthful strategy. However, the objective of each player $i$ is to maximize its own utility given by (2), and so it may not employ a truthful strategy if there is a possibility for it to accrue a higher utility by doing so than by employing a truthful strategy. This brings us to the mechanism design problem. We wish to design a payment rule $\{p_{i,l}\mathrel{\mathop{\mathchar 58\relax}}(i,l)\in\{1,\ldots,n\}\times\mathbb{Z}_{+}\}$ such that each player employing a truthful strategy is a Dominant Strategy Non-Bankrupting Equilibrium. The next section develops the mechanism and establishes the incentive and efficiency properties guaranteed by it.

The first-stage payment $p_{i}^{F}$ is a function of only the first-stage bids of the players. Since these quantities remain the same on all days, so do the first-stage payments. The first-stage payment is simply the VCG payment and is defined as

where $\widehat{\boldsymbol{\theta}}_{-i}$ denotes the supertype bids of all players other than player $i$.

At a high level, the first functionality of the second-stage payment is to bind the first-stage and the second-stage strategies of the players. To achieve this, the second-stage payment rule compares the empirical frequencies of the players’ type bids with their supertype bids and penalizes discrepancies between them. To elaborate, denote by $\widehat{\theta}_{i}(t)$ the probability that a random variable distributed according to $\widehat{\theta}_{i}$ takes the value $t,$ $t\in\Delta.$ On each day $l$ and for each player $i,$ $(l,i)\in\mathbb{Z}_{+}\times\{1,\ldots,n\},$ the second-stage payment rule computes the discrepancy

for every $t\in\Delta$, and imposes a penalty of $J_{p}(l)$ on player $i$ if $\widehat{f}_{i,t}(l)$ falls outside a window of size $r(l)$ for some $t\in\Delta,$ i.e., if

for some $t\in\Delta.$

In a setting of repeated playing, the sequence of second-stage outcomes serves as a source of common randomness which the players can potentially use to correlate their second-stage bids if there is a possibility for them to accrue a higher utility by doing so than by fabricating their bids independently of the other players’ bids. The second functionality of the second-stage payment is to disincentivize such strategies. Towards this end, on each day $l$ and for each player $i$, $(l,i)\in\mathbb{Z}_{+}\times\{1,\ldots,n\},$ the second-stage payment rule computes

for every $\mathbf{d}\in\Delta^{n},$ and imposes a penalty of $J_{p}(l)$ on player $i$ if it falls outside a window of size $r(l)$ for some $\mathbf{d}\in\Delta^{n},$ i.e., if

for some $\mathbf{d}\in\Delta^{n}.$

How should the window size sequence $\{r\}$ be chosen? On the one hand, the window size $r(l)$ must tend to zero as $l$ tends to infinity for otherwise, the set of sequences $\{\boldsymbol{\widehat{\delta}}\}$ that satisfy (7) and (9) would be “large,” thereby violating incentive compatibility. On the other hand, if $\{r\}$ decays too quickly, then even truthful type bids would violate (7) and (9) infinitely often, thereby incurring a large penalty and violating individual rationality. Hence, the sequence $\{r\}$ should be chosen in a manner that balances the two objectives. This is achieved by choosing $\{r\}$ such that

and for some $\gamma>0,$

for all $l\in\mathbb{Z}_{+}.$¹¹1It suffices that (11) holds not for all $l$ but only for all sufficiently large $l$.

To obtain an intuition for condition (11), note that the empirical frequency $\frac{1}{l}\sum_{l^{\prime}=1}^{l}\mathds{1}_{\{\delta_{i}(l^{\prime})=t\}}$ resulting from the true type sequence of player $i$ is a random variable with mean $\theta_{i}(t)$ and standard deviation that scales as ${{1}/\sqrt{l}}.$ Therefore, if the window size decays at the same rate, then the probability of the empirical frequency falling outside the window would remain at a constant value. This suggests that the window size must scale slower than at least ${{1}/\sqrt{l}}.$ By scaling the window size only slightly slower than ${{1}/\sqrt{l}}$, namely the rate specified by condition (11), truthful bids are guaranteed to almost surely satisfy (7) and (9) for all but finitely many values of $l$. This is established in Lemma 1.

How should the penalty sequence $\{J_{p}\}$ be chosen? As shown in Lemma 1, truthful players incur a penalty only finitely often almost surely, and so the long-term average penalty that they incur is almost surely zero regardless of how the sequence $\{J_{p}\}$ is chosen. Therefore, the only objective in the design of $\{J_{p}\}$ is for every non-truthful strategy to incur a sufficiently high penalty. This is accomplished by choosing $\{J_{p}\}$ to be any nonnegative sequence such that

We now have the necessary quantities to define the second-stage payment function. Define the event

which denotes the occurrence of at least one of (7) and (9). The second-stage payment of player $i$ on day $l$ is defined as

A negative value of the above quantity implies a transfer from the social planner to player $i$ on day $l.$ Note that if all players employ a truthful strategy, then the long-term average second-stage payment almost surely equals zero for every player.

The total payment $p_{i,l}(\boldsymbol{\widehat{\theta}},\boldsymbol{\widehat{\delta}}^{l})$ that player $i$ transfers to the social planner on day $l$ is

The following theorem establishes the incentive and optimality guarantees of the mechanism.