Mechanism design with multi-armed bandit

Multi-agent systems can be made efficient by mediators who make system-wide (social) decisions in a way that maximizes social welfare. For example, in a trading network where firms sell goods to each other or to external markets (Hatfield et al.,, 2013), the mediator can ensure that those traded goods are produced by the firms with the lowest costs and purchased by those with the highest needs (Osogami et al.,, 2023). While such mediators could maximize their own profit by charging participants (e.g., today’s tech giants who operate consumer marketplaces or ads platforms), they would typically take most of the profit, leaving the participants with only a small portion.

We instead envision an open platform whose purpose is to provide maximal benefits to the participants in multi-agent systems. This is similar to the purpose of designing an auction mechanism for public resources, which should be given to those who need them most, and there should be neither budget deficits nor surpluses on the mediator (Bailey,, 1997; Cavallo,, 2006; Dufton et al.,, 2021; Gujar and Narahari,, 2011; Guo,, 2012; Guo and Conitzer,, 2009; Manisha et al.,, 2018; Tacchetti et al.,, 2022). However, such mechanisms exploit the particular structure of single-sided auctions, where all participants are buyers, and it may be impossible to achieve desired properties in other multi-agent systems such as double-sided auctions (Hobbs et al.,, 2000; Zou,, 2009; Widmer and Leukel,, 2016; Stößer et al.,, 2010; Kumar et al.,, 2018; Chichin et al.,, 2017), matching markets (Zhang and Zhu,, 2020), and trading networks (Osogami et al.,, 2023; Wasserkrug and Osogami,, 2023) even if those properties could be guaranteed for single-sided auctions.

Here, we design mechanisms for general environments that include all these multi-agent systems, with the primary objectives of efficiency and strong budget balance (SBB). Specifically, we require that the mediator chooses a social decision such that its total value to the players (or participants) is maximized (decision efficiency; DE) and that the expected revenue of the mediator is equal to a target value, $\rho\in\mathbb{R}$ (SBB when $\rho=0$). As is standard in Bayesian mechanism design (Shoham and Leyton-Brown,, 2009), we assume that the value of each social decision to a player depends on the player’s type that is only known to that player, but a joint probability distribution of the players’ types is a common knowledge. Since it would be difficult to achieve DE without knowing the types, we also require that the optimal strategy of each player is to truthfully declare its type regardless of the strategies of the others (dominant strategy incentive compatibility; DSIC). To promote participation, we also require that the expected utility of each player is no less than a target value, $\theta(t_{n})\in\mathbb{R}$, that can depend on the type $t_{n}$ of the player (individual rationality or IR when $\theta\equiv 0$).

Although these properties are standard in the literature of mechanism design (Shoham and Leyton-Brown,, 2009), here we have introduced parameters, $\rho$ and $\theta$, to generalize the standard definitions of SBB ($\rho=0$) and IR ($\theta\equiv 0$) with three motivations. First, it is not always possible to achieve the four desired properties with the standard definitions of IR and SBB (Green and Laffont,, 1977; Myerson and Satterthwaite,, 1983; Osogami et al.,, 2023), while the generalization will enable the exact characterization of when those properties can be satisfied. Second, this generalization will allow us to develop a principled approach of learning a mechanism, where some of the quantities are estimated from samples, with theoretical guarantees. The mechanism designed with our learning approach can be shown to satisfy the four desired properties with high probability. Finally, the additional parameters will allow us to model practical requirements. For example, the mediator might need positive revenue to cover the cost of maintaining the platform or might want to guarantee positive expected utility to players to encourage participation in this platform rather than others.

We require SBB and IR in expectation (ex ante or interim) with respect to the distribution of types, while DE and DSIC are satisfied for any realization of types (ex post). While these assumptions are similar to those in Osogami et al., (2023); Wasserkrug and Osogami, (2023) for trading networks, the in expectation properties are certainly weaker than the ex post properties usually assumed for auctions (Bailey,, 1997; Cavallo,, 2006; Dufton et al.,, 2021; Gujar and Narahari,, 2011; Guo,, 2012; Guo and Conitzer,, 2009; Manisha et al.,, 2018; Tacchetti et al.,, 2022). With the weaker properties, however, we derive analytical solutions of the mechanisms that satisfy all the four desired properties in the general environments (and characterize when those properties can be satisfied). This is in stark contrast to the prior work, where mechanisms are analytically derived only for auctions with a single type of goods (Bailey,, 1997; Cavallo,, 2006; Guo,, 2011; Guo and Conitzer,, 2007, 2009; Moulin,, 2009) or with unit demand (Gujar and Narahari,, 2011; Guo,, 2012). For more complex auctions (Dufton et al.,, 2021; Manisha et al.,, 2018; Tacchetti et al.,, 2022) or trading networks (Osogami et al.,, 2023; Wasserkrug and Osogami,, 2023), mechanisms are computed by numerically solving optimization problems, whose size often grows exponentially with the number of players.

While the numerical approaches proposed in Osogami et al., (2023) have been applied only to the trading networks with two players, our analytical solutions can be evaluated numerically with $\sim 10$ players, depending on the number of types. The key bottleneck in our analytical solutions lies in the evaluation of the minimum expected value over possible types of each player. Exact evaluation of this quantity would require computing an efficient social decision for all the $K^{N}$ combinations of types, where $K$ is the number of possible types of each player, and $N$ is the number of players.

To overcome this bottleneck, we model the problem of evaluating this minimum expected value in a multi-armed bandit (MAB) approach (Lattimore and Szepesvári,, 2020) and propose an asymptotically optimal learning algorithm for this problem. While the standard objectives of MAB are regret minimization (Auer et al.,, 1995, 2002) and best arm identification (Audibert et al.,, 2010; Maron and Moore,, 1993; Mnih et al.,, 2008; Bubeck et al.,, 2009), our objective is to estimate the mean reward of the best arm. We propose a probably approximately correct (PAC) algorithm, which approximately (with error at most $\varepsilon$) estimates the best mean with high probability (at least $1-\delta$), and proves that its sample complexity, $O((K/\varepsilon^{2})\,\log(1/\delta))$, matches the lower bound that we derive. This learning approach substantially reduces the number of computing efficient social decisions from $K^{N}$ to $O(K\,N\,\log N)$, enabling us to numerically find mechanisms for $\sim 100$ players, depending on $K$.

Our contributions thus revolve around the optimization problem whose solution gives the mechanism that satisfies DE, SBB, DSIC, and IR (see Section 3-4). First, we establish a sufficient condition that ensures the optimization problem has feasible solutions, and prove that this sufficient condition is also necessary when the players have independent types (see Section 5). Second, for cases where this sufficient condition holds, we analytically derive a class of optimal solutions to this optimization problem, which in turn gives mechanisms that satisfy DE, SBB, DSIC, and IR for general environments including auctions and trading networks (see Section 5). Third, we model the problem of evaluating a quantity in the above analytical expressions as best mean estimation in MAB, propose a PAC algorithm for this problem, and prove its asymptotic optimality (see Section 6). Finally, we empirically validate the effectiveness of the proposed approach (see Section 7). In Section 2, we start by positioning our contributions to the prior work.

The prior work most related to ours is Osogami et al., (2023), which formulates and numerically solves the LP whose solution gives the mechanism for a trading network that satisfies DE, DSIC, IR, and weak budget balance (WBB; expected revenue of the mediator is nonnegative). While the objective of the LP is rather arbitrary and SBB is given just as an example in Osogami et al., (2023), we focus on SBB and drive analytical solutions for this particular objective. Our formulation extends Osogami et al., (2023) with additional parameters and notations that cover environments beyond trading networks, but these extensions are relatively straightforward.

In the rest of this section, we discuss related work on mechanism design, with a focus on those aiming to achieve SBB, as well as related work on MAB with a focus on PAC algorithms. In particular, we will see that our learning approach is unique in that we estimate a particular quantity in the optimal solution that we derive analytically, and this leads us to propose a new PAC algorithm and establish its optimality for an underexplored objective of best mean estimation in MAB.

In single-sided auctions where only buyers make strategic decisions, Vickrey–Clarke–Groves (VCG) mechanisms with Clark’s pivot rule (also called VCG auction) satisfy ex post DE (called allocative efficiency in auctions), DSIC, IR, and WBB (Nisan,, 2007). However, the Green-Laffont Impossibility Theorem implies that no mechanism can guarantee DE, DSIC, and SBB simultaneously for all environments (Green and Laffont,, 1977, 1979). This has led to a series of work on redistributing the revenue of the mediator to the players as much as possible (i.e., to make budget balance as strong as possible), while satisfying DSIC, DE, IR, and WBB. For auctions with single or homogeneous goods (Bailey,, 1997; Cavallo,, 2006; Guo,, 2011; Guo and Conitzer,, 2007, 2009; Moulin,, 2009) or for auctions where players have unit demand (Gujar and Narahari,, 2011; Guo,, 2012), researchers have derived analytical solutions that optimally redistribute the payment to the players. For auctions with multi-unit demands on heterogeneous goods, the prior work has proposed numerical approaches that seek to find the piecewise linear functions (Dufton et al.,, 2021) or neural networks (Manisha et al.,, 2018; Tacchetti et al.,, 2022) that best approximate the optimal redistribution functions.

We consider the environments that not only allow heterogeneous goods and multi-unit demands but also are more general than single-sided auctions. In particular, our players may have negative valuation on a social decision. The Myerson-Satterthwaite Impossibility Theorem (Myerson and Satterthwaite,, 1983) thus implies that, unlike VCG auctions, no mechanism can guarantee ex post DE, DSIC, IR, and WBB simultaneously for all the environments that we consider. We thus derive mechanisms that achieve DE, DSIC, IR, and SBB in the best possible manner. A limitation of our results is that IR and SBB are satisfied only in expectation. Such a guarantee in expectation can however be justified for risk-neutral mediator and players (Osogami et al.,, 2023). Our model can also guarantee strictly positive expected utility, which in turn can ensure nonnegative utility with high probability when the player repeatedly participate in the mechanism.

For auctions, there also exists a large body of the literature on maximizing the revenue of the mediator (Myerson,, 1981) with recent focus on automated mechanism design (AMD) with machine learning (Duetting et al.,, 2019; Rahme et al.,, 2021; Ivanov et al.,, 2022; Curry et al.,, 2020) and analysis of its sample complexity (Balcan et al.,, 2016; Morgenstern and Roughgarden,, 2015; Syrgkanis,, 2017). Similar to these and other studies of AMD (Sandholm,, 2003; Conitzer and Sandholm,, 2002), we formulate an optimization problem whose solution gives the mechanism with desired properties. However, instead of solving it numerically, we analytically derive optimal solutions. Also, while the prior work analyzes the sample complexity for finding the mechanism that maximizes the expected revenue, we analytically find optimal mechanisms and characterize the sample complexity of evaluating a particular expression in the analytically designed mechanisms.

We evaluate our analytical expression through best mean estimation (BME) in MAB, where the standard objectives are regret minimization (Auer et al.,, 1995, 2002) and best arm identification (BAI) (Audibert et al.,, 2010; Maron and Moore,, 1993; Mnih et al.,, 2008; Bubeck et al.,, 2009). The prior work on MAB that is most relevant to ours is PAC learning for BAI and analysis of its sample complexity. We reduce the problem of BME to BAI and prove the lower bound on the sample complexity of BME using a technique known for BAI (Even-Dar et al.,, 2002). However, while this technique does not give tight lower bound for BAI (Mannor and Tsitsiklis,, 2004), we show that it gives tight lower bound for BME. Notice that the problem of estimating the best mean frequently appears in reinforcement learning (van Hasselt,, 2010) and machine learning (Kajino et al.,, 2023), but there the focus is on how best to estimate the best mean with a given set of samples (van Hasselt,, 2013), while our focus is on how best to collect samples to estimate the best mean.

The goal of mechanism design is to specify the rules of a game in a way that an outcome desired by the mechanism designer is achieved when rational players (i.e., players whose goal it is to maximize their individual utility) participate in that game (Jackson,, 2014; Shoham and Leyton-Brown,, 2009). Formally, let $\mathcal{N}\coloneqq\{1,2,\ldots,N\}$ be the set of players and $\mathcal{O}$ be the set of possible outcomes. For each player $n\in\mathcal{N}$, let $\mathcal{A}_{n}$ be the set of available actions and $\mathcal{T}_{n}$ be the set of possible types. Let $\mathcal{A}\coloneqq\mathcal{A}_{1}\times\ldots\times\mathcal{A}_{N}$ and $\mathcal{T}\coloneqq\mathcal{T}_{1}\times\ldots\times\mathcal{T}_{N}$ be the corresponding product spaces. A mechanism $\mu:\mathcal{A}\to\mathcal{O}$ determines an outcome depending of the actions taken by the players. Let $u_{n}:\mathcal{O}\times\mathcal{T}_{n}\to\mathbb{R}$ be the utility function of each player $n\in\mathcal{N}$.

We consider Bayesian games where the players’ types follow a probability distribution that is known to all players and the mediator. Before selecting actions, the players know their own types but not the types of the other players. A strategy of each player $n\in\mathcal{N}$ is thus a function from $\mathcal{T}_{n}$ to $\mathcal{A}_{n}$.

We assume that an outcome is determined by a social decision and payment; hence, a mechanism $\mu$ consists of a decision rule and a payment rule. Let $\mathcal{D}$ be the set of possible social decisions. Given the actions of the players, the decision rule $\phi:\mathcal{A}\to\mathcal{D}$ determines a social decision, and the payment rule $\tau:\mathcal{A}\to\mathbb{R}^{\mathcal{N}}$ determines the amount of (possibly negative) payment to the mediator from each player. Let $v:\mathcal{D}\times(\mathcal{T}_{1}\cup\ldots\cup\mathcal{T}_{N})\to\mathbb{R}$ specify the value of a given social decision to the player of a given type. Then the utility of player $i$ when players take actions $a\in\mathcal{A}$ is

Throughout, we assume that $\mathcal{N}$, $\mathcal{D}$, and $\mathcal{T}_{n},\forall n\in\mathcal{N}$ are finite sets.

Without loss of generality by the revelation principle (Shoham and Leyton-Brown,, 2009), we consider only direct mechanisms, where the action available to each player is to declare which type the player belongs to from the set of possible types (i.e., $\mathcal{A}_{n}=\mathcal{T}_{n},\forall n\in\mathcal{N}$). We will thus use $\mathcal{T}_{n}$ for $\mathcal{A}_{n}$.

Then we seek to achieve the following four properties with our mechanisms. The first property is Dominant Strategy Incentive Compatibility (DSIC), which ensures that the optimal strategy of each player is to truthfully reveal its type regardless of the strategies of the other players. Formally, {fleqn}

where the left-hand side represents the utility of the player having type $t_{n}$ when it declares the same $t_{n}$, and the other players declare arbitrary types $t_{-n}^{\prime}$.

The second property is Decision Efficiency (DE), which requires that the mediator chooses the social decision that maximizes the total value to the players. With DSIC, we can assume that the players declare true types, and hence we can write DE as a condition on the decision rule: {fleqn}[10pt]

As the third property, we generalize individual rationality and require that the expected utility of each player is at least as large as a target value that can depend on its type. We refer to this property as $\theta$-IR. Again, assuming that players declare true types due to DSIC, we can write $\theta$-IR as follows: {fleqn}[10pt]

where $\theta:\mathcal{T}_{1}\cup\ldots\cup\mathcal{T}_{N}\to\mathbb{R}$ determines the target expected utility for each type. Throughout (except in Section 6, where we discuss general MAB models), $\mathbb{E}$ denotes the expectation with respect to the probability distribution $\mathbb{P}$ of types, which is the only probability that appears in our mechanisms.

The last property is a generalization of Budget Balance (BB), which we refer to as $\rho$-WBB and $\rho$-SBB. Specifically, $\rho$-WBB requires that the expected revenue of the mediator is no less than a given constant $\rho\in\mathbb{R}$, and $\rho$-SBB requires that it is equal to $\rho$. Again, assuming that the players declare true types due to DSIC, these properties can be written as follows: {fleqn}[10pt]

While $\rho$-SBB is stronger than $\rho$-WBB, we will see that $\rho$-SBB is satisfiable iff $\rho$-WBB is satisfiable.

Following Osogami et al., (2023), we seek to find optimal mechanisms in the class of VCG mechanisms. A VCG mechanism is specified by a pair $(\phi^{\star},h)$. Specifically, after letting player take the actions of declaring their types $t\in\mathcal{T}$, the mechanism first finds a social decision $\phi^{\star}(t)$ using a decision rule $\phi=\phi^{\star}$ that satisfies DE (3). It then determines the amount of payment from each player $n\in\mathcal{N}$ to the mediator by

where we define $\mathcal{N}_{-n}\coloneqq\mathcal{N}\setminus\{n\}$, and $h_{n}:\mathcal{T}_{-n}\to\mathbb{R}$ is an arbitrary function of the types of the players other than $n$ and referred to as a pivot rule. The decision rule $\phi^{\star}$ guarantees DE (3) by construction, and the payment rule (6) then guarantees DSIC (2) (Nisan,, 2007).

Our problem is now reduced to find the pivot rule, $h=\{h_{n}\}_{n\in\mathcal{N}}$, that minimizes the expected revenue of the mediator, while satisfying $\theta$-IR and $\rho$-WBB. This may lead to satisfying $\rho$-SBB if the revenue is maximally reduced. To represent this reduced problem, let

be the total value of the efficient social decision when the players have types $t$. Then we can rewrite $\theta$-IR (for the player having type $t_{n}$) and $\rho$-WBB as follows (see Appendix A.1 for full derivation):

Therefore, we arrive at the following linear program (LP):

The approach of Osogami et al., (2023) is to numerically solve this LP possibly with approximations. Since $h_{n}(t_{-n})$ is a variable for each $t_{-n}\in\mathcal{T}_{-n}$ and each $n\in\mathcal{N}$, the LP has $N\,K^{N-1}$ variables and $N\,K+1$ constraints, when each player has $K$ possible types. When this LP is feasible, let $h^{\star}$ be its optimal solution; then the VCG mechanism $(\phi^{\star},h^{\star})$ guarantees DSIC, DE, $\theta$-IR, and $\rho$-WBB (formalized as Proposition 4 in Appendix A.1). Otherwise, no VCG mechanisms can guarantee them all. In the next section, we characterize exactly when the LP is feasible and provide analytical solutions to the LP.

We first establish a sufficient condition and a necessary condition for the LP to have feasible solutions. Note that complete proofs for all theoretical statements are provided in Appendix B.

These two lemmas establish the following necessary and sufficient condition.

While the LP is not necessarily feasible, one may choose $\theta$ and $\rho$ in a way that it is guaranteed to be feasible. For example, the feasibility is guaranteed (i.e., (13) is satisfied) by setting

where $[x]^{-}\coloneqq\min\{x,0\}$ for $x\in\mathbb{R}$. When $\rho<0$, the mediator might get negative expected revenue, but the expected loss of the mediator is at most $|\rho|$. Appendix A.2 provides alternative $\rho$ and $\theta$ that guarantee feasibility, but some of the players might incur negative expected utility.

Finally, we derive a class of optimal solutions to the LP when it is feasible.

When types are independent, the condition (13) is necessary for the existence of a feasible solution; hence, we do not lose optimality by considering only the solutions in $\mathcal{H}$. On the other hand, when types are dependent, the condition (13) may still be satisfied, where the solutions in $\mathcal{H}$ remain optimal. However, dependent types no longer necessitate (13) to satisfy (11)-(12); hence the space of constant pivot rules may not suffice to find an optimal solution if (13) does not hold.

In deriving the optimal solutions, we have substantially reduced the essential number of variables (from $N\,T^{N-1}$ to $N$ when each player has $T$ types). Our approach can therefore not only find but also represent or store solutions exponentially more efficiently than Osogami et al., (2023).

It turns out that the solutions in $\mathcal{H}$ not only satisfy $\rho$-WBB but also $\rho$-SBB (in addition to DE, DSIC, and $\theta$-IR) regardless of whether the types are independent or not:

When the LP is infeasible, we can construct a mechanism that satisfies one of $\rho$-SBB and $\theta$-IR (in addition to DE and DSIC) regardless of whether the types are independent or not:

For example, for any $t_{n}\in\mathcal{T}_{n}$ and $n\in\mathcal{N}$, the following pivot rule always satisfies $\theta$-IR:

where we define (see also (58) in the appendix)

So far, we have analytically derived the class of optimal solutions for the LP. Although these analytical solutions substantially reduce the computational cost of solving the LP compared to the numerical solutions in Osogami et al., (2023), they still need to evaluate

for each $n\in\mathcal{N}$. Since $\mathbb{E}$ is the expectation with respect to the distribution $\mathbb{P}$ over $\mathcal{T}$, this would require evaluating $w^{\star}(t)$ for all $t\in\mathcal{T}$. Recall that $w^{\star}(t)$ defined with (7) is the total value of the efficient decision $\phi^{\star}(t)$, which is given by a solution to an optimization problem (3). Without any structure in $\mathcal{D}$ and $v$, this would require evaluating the total value for all decisions in $\mathcal{D}$.

To alleviate the computational complexity associated with (20), we propose a learning approach. A key observation is that the problem of estimating (20) can be considered as a variant of a multi-armed bandit (MAB) problem whose objective is to estimate the mean reward of the best arm. Specifically, the MAB gives the reward $\theta(t_{n})-w^{\star}(t)$ when we pull an arm $t_{n}\in\mathcal{T}_{n}$, where $t$ is a sample from the conditional distribution $\mathbb{P}[\cdot\mid t_{n}]$. Following the relevant prior work on MAB (Even-Dar et al.,, 2002, 2006; Hassidim et al.,, 2020; Mannor and Tsitsiklis,, 2004), we maximize reward in this section.

Since we assume that $\mathcal{N}$, $\mathcal{D}$, and $\mathcal{T}_{n},\forall n\in\mathcal{N}$ are finite, there exist constants, $\bar{\theta}$ and $\bar{v}$, such that $|\theta(t^{\prime})|\leq\bar{\theta}$ and $|v(d;t^{\prime})|\leq\bar{v},\forall d\in\mathcal{D},\forall t^{\prime}\in\cup_{n\in\mathcal{N}}\mathcal{T}_{n}$. Then we can also bound $|\theta(t_{n})-w^{\star}(t)|\leq\bar{\theta}+N\,\bar{v},\forall t\in\mathcal{T},\forall n\in\mathcal{N}$. Namely, the reward in the MAB is bounded. We assume that we know the bounds on the reward and can scale it such that the scaled reward is in $[0,1]$ surely.

We also assume that we have access to an arbitrary size of the sample that is independent and identically distributed (i.i.d.) according to $\mathbb{P}[\cdot\mid t_{n}]$ for any $t_{n}\in\mathcal{T}_{n},n\in\mathcal{N}$. When players have independent types, such sample can be easily constructed as long as we have access to i.i.d. sample $\{t^{(i)}\}_{i=1,2,\ldots}$ from $\mathcal{T}$, because $\{(t_{n},t_{-n}^{(i)})\}_{i}$ is the sample from $\mathbb{P}[\cdot\mid t_{n}]$ for any $t_{n}\in\mathcal{T}_{n},n\in\mathcal{N}$.

Consider the general $K$-armed bandit where the reward of each arm is bounded in $[0,1]$. For each $k\in\{1,2,\ldots,K\}$, let $\mu_{k}$ be the true mean of arm $k$. Let $\mu_{\star}\coloneqq\max_{k}\mu_{k}$ be the best mean-reward, which we want to estimate. We say that the sample complexity of an algorithm for a MAB is $T$ if the sample size needed by the algorithm is bounded by $T$ (i.e., arms are pulled at most $T$ times).

A standard Probably Approximately Correct (PAC) algorithm for MAB returns an $\varepsilon$-optimal arm with probability at least $1-\delta$ for given $\varepsilon,\delta$ (Even-Dar et al.,, 2006; Hassidim et al.,, 2020). We will use the following definition:

Definition 1 is different from what we need to evaluate (20). Formally, we need

BAI and BME are related but different. For example, consider the case where the best arm has large variance and $\mu_{\star}=1/2$, and the other arms always give zero reward $\mu_{n}=0,\forall n\neq\star$. Then relatively small sample would suffice for BAI due to the large gap $\mu_{\star}-\mu_{n}=1/2,\forall n\neq\star$, while BME would require relatively large sample due to the large variance of the best arm $\star$. As another example, consider the case where we have many arms whose rewards follow Bernoulli distributions. Suppose that half of the arms have an expected value of 1, and the other half have an expected value of $1-(3/2)\,\varepsilon$. Then by pulling sufficiently many arms (once for each arm), we can estimate that the best mean is at least $1-\varepsilon$ with high probability (by Hoeffding’s inequality), but BAI would require pulling arms sufficiently many times until we pull one of the best arms sufficiently many times (to be able to say that this particular arm has an expected value at least $1-\varepsilon$).

For ($\varepsilon,\delta$)-PAC BAI, the following lower and upper bounds on the sample complexity are known:

We establish the same lower and upper bounds for ($\varepsilon,\delta$)-PAC BME:

Our upper bound is established by reducing BME to BAI. Suppose that we have access to an arbitrary $(\varepsilon,\delta)$-PAC BAI with sample complexity $M$ (Algorithm 2 in Appendix A.3). We can construct a $(\frac{3}{2}\varepsilon,2\delta)$-PAC BME by first running the $(\varepsilon,\delta)$-PAC BAI and then sampling from the arm $\hat{I}$ that is identified as the best until we have a sufficient number, $m^{\star}$, of samples from $\hat{I}$ (Algorithm 3 in Appendix A.3). When $m^{\star}$ is appropriately selected, the following lemma holds:

By Proposition 2, this establishes the upper bound on the sample complexity in Theorem 1.

Our lower bound is established by showing that an arbitrary $(\varepsilon,\delta)$-PAC BME can be used to solve the problem of identifying whether a given coin is negatively or positively biased (precisely, $\varepsilon$-Biased Coin Problem of Definition 3 in Appendix A.3), for which any algorithm is known to require expected sample complexity at least $\Omega((1/\varepsilon^{2})\,\log(1/\delta))$ to solve correctly with probability at least $1-\delta$ (Chernoff,, 1972; Even-Dar et al.,, 2002) (see Lemma 7 in Appendix A.3). The following lemma, together with Lemma 7, establishes the lower bound on the sample complexity in Theorem 1:

We prove Lemma 5 by reducing the $\varepsilon$-Biased Coin Problem to BME. This proof technique was also used in Even-Dar et al., (2002) to prove a lower bound on the sample complexity of BAI. What is interesting, however, is that the lower bound in Even-Dar et al., (2002) is not tight when $\delta<1/K$, and a tight lower bound is established by a different technique in Mannor and Tsitsiklis, (2004). On the other hand, our proof gives a tight lower bound on the sample complexity of BME. In Appendix A.3, we further discuss where this difference in the derived lower bounds comes from.

Finally, we connect the results on BME back to our mechanism in Section 5 and provide a guarantee on the properties of the mechanism when the term (20) is estimated with BME. To this end, we define the terms involving expectations in Lemma 3 as follows:

Then, recalling that $N$ is the number of players, we have the following lemma:

Notice that the sufficient condition of Lemma 1 states that the simplex in Lemma 3 is nonempty. Analogously, when the simplex in Lemma 6 is empty, we cannot provide the solution that guarantees the properties stated in Lemma 6. Since DSIC and DE remain satisfied regardless of whether $\kappa_{n}$ for $n\in\mathcal{N}$ and $\lambda$ are estimated or exactly computed, Lemma 6 immediately establishes the following theorem by appropriately choosing the parameters:

Recall that the exact computation of our mechanism in Lemma 3 requires evaluating $w^{\star}(t)$ for all $t\in\mathcal{T}$, whose size grows exponentially with the number of players, $N$. Our BME algorithm reduces this requirement to $O(N\,\log N)$, as is formally proved in the following proposition:

We conduct several numerical experiments to validate the effectiveness of the proposed approach and to understand its limitations. Specifically, we design our experiments to investigate the following questions. i) BME studied in Section 6 has asymptotically optimal sample complexity, but how well can we estimate the best mean when there are only a moderate number of arms? ii) How much can BME reduce the number of times $w^{\star}(t)$ given by (7) is evaluated? iii) How well $\theta$-IR and $\rho$-SBB are satisfied when (20) is estimated by BME rather than calculated exactly? In this section, we only provide a brief overview of our experiments; for full details, see Appendix A.4. In particular, we find the key empirical property of BME that it can reduce the computational complexity by many orders of magnitude when the number of players is moderate ($\sim 10$) to large but is relatively less effective against the increased number of types.

For question i), we compare Successive Elimination for BAI (SE-BAI), which is known to perform well for a moderate number of arms (Hassidim et al.,, 2020; Even-Dar et al.,, 2006), against the corresponding algorithm for BME (SE-BME) and summarize the results in Appendix A.4.1. Overall, we find that SE-BME generally requires smaller sample size than SE-BAI, except when there are only a few arms and their means have large gaps. The efficiency of SE-BAI for this case makes intuitive sense, since the best arm can be identified without estimating the means with high accuracy.

For question ii), we quantitatively validate the effectiveness of SE-BME in reducing the number of times $w^{\star}(t)$ is computed when we evaluate (20) in a setting of mechanism design, as is detailed in Appendix A.4.2. Figure 1 summarizes the results. Panel (a) shows that the means close to the minimum value are estimated with high accuracy, while others are eliminated by SE-BME after a small number of samples. Panels (b)-(c) show that SE-BME (solid curves) evaluates $w^{\star}(t)$ by orders of magnitude smaller number of times than what is required by exact computation (dashed curves). The effectiveness of SE-BME is particularly striking when $N$ (the number of players) grows. While exact computation evaluates $w^{\star}(t)$ exponentially many times as $N$ grows, SE-BME has only linear dependency on $N$. This insensitivity of the sample complexity of SE-BME to $N$ makes intuitive sense, since $N$ only affects the distribution of the reward and keeps the number of arms unchanged.

For question iii), we quantitatively evaluate how well $\theta$-IR and $\rho$-SBB are guaranteed when we estimate (20) with SE-BME, where we continue to use the same setting of mechanism design. In Figure 2, we study two analytical solutions: one guarantees to satisfy $\theta$-IR (with $\theta\equiv 0$), and the other guarantees to satisfy $\rho$-SBB (with $\rho=0$). The red dots show the expected utility of the players in (a) and (c) as well as the expected revenue of the mediator in (b) and (d), when (20) is evaluated either exactly (horizontal axes) or estimated with SE-BME (vertical axes). Overall, the expected revenue of the mediator is more sensitive than the expected utility of the players to the error in the estimation of (20), because (12) involves the summation $\sum_{n\in\mathcal{N}}\eta_{n}$, while (11) only involves $\eta_{n}$ for a single $n\in\mathcal{N}$. We can however ensure that $\rho$-WBB (instead of SBB) is satisfied with high probability by replacing the $\rho$ with a $\rho^{\prime}>\rho$ when we compute $\eta_{n}$ after (20) is estimated. As we show and explain with Figure 8 in Appendix A.4.3, this will simply shift the dots in the figure, and we can guarantee $\rho$-WBB with high probability with an appropriate selection of $\rho^{\prime}$.

We have run all of the experiments on a single core with at most 66 GB memory without GPUs in a cloud environment, as is detailed in Appendix A.4.4. The associated source code is submitted as a supplementary material and will be open-sourced upon acceptance.

We have analytically derived optimal solutions for the LP that gives mechanisms that guarantee the desired properties of DE, DSIC, $\rho$-SBB, and $\theta$-IR. When there are $N$ players, each with $T$ possible types, the LP involves $N\,T^{N-1}$ variables, while our analytical solutions are represented by only $N$ essential variables. While Osogami et al., (2023) numerically solves this LP only for $N=T=2$, we have exactly evaluated our analytical solutions for $N=T=8$ (see Figure 1). The analytical solution, however, involves a term whose exact evaluation requires finding efficient social decisions $T^{N}$ times. We have modeled the problem of evaluating this term as best mean estimation in multi-armed bandit, proposed a PAC estimator, and proved its asymptotic optimality by establishing a lower bound on the sample complexity. Our experiments show that our PAC estimator enables finding mechanism for $N=128$ and $T=8$ with a guarantee on the desired properties.

The proposed approach makes a major advancement in the field and can positively impact society by providing guarantees on desired properties for large environments, which existing approaches are unable to handle. However, it presents certain limitations and potential challenges, which inspire several directions for further research. Regarding limitations, one should keep in mind that our mechanisms guarantee $\rho$-SBB and $\theta$-IR only in expectation, and that the our approaches have only been applied to the environments up to 128 players and up to 16 types in our experiments. Regarding societal impacts, our approach might result in the mechanisms that are unfair to some of the players, because DE does not mean that all of the players are treated fairly. These limitations and societal impacts are further discussed in Appendix A.5.