Distributed Computation of Stochastic GNE with Partial Information: An Augmented Best-Response Approach

In the Nash equilibrium problem (NEP), a set of self-interested players aim to optimize their individual payoffs which depend not only on their own decisions but also on the decisions of others [2]. The generalized Nash equilibrium problem (GNEP) extends the NEP by considering additional global resource constraints that these players should collectively satisfy [3, 4]. In numerous practical applications, such as strategic behaviors in power markets [5, 6], engagement of multiple humanitarian organizations in disaster relief [7], and the traffic assignment of strategic risk-averse users [8], in addition to couplings in objectives and global constraints, there is also uncertainty involved in the objective functions. For example, in the target-rate power management problem for wireless networks, the power of battery-driven devices should be regulated in a real-time manner in the presence of inherent stochastic fluctuations of the underlying network [9]. These applications motivate us to consider an extension to the GNEP, called the stochastic generalized Nash equilibrium problem (SGNEP). In the SGNEP, instead of deterministic objective functions, players optimize the expected values of uncertain objective functions which are dependent on some random variables. Although the SGNEP can capture a wider range of applications, computing its Nash equilibria becomes a much more challenging problem compared to the GNEP, due to the absence of closed-form expressions of the objective functions. Fortunately, as has been shown in [10, Sec. 1.4], many SGNEPs can be formulated as stochastic variational inequalities (SVIs) and solved by leveraging existing results from this field, e.g. [11, 12, 13, 14, 15].

Our aim in this paper is to develop a distributed algorithm under the partial-decision information setting for solving SGNEPs over a network of players. In the context of non-cooperative games on networks, participants are self-interested and make rational decisions that maximize their own payoffs. It is unrealistic that these participants will adopt any centralized methods that require cooperation among them. Because of this, there is an enduring research interest in distributing the computation of Nash equilibria [16, 17], especially through the avenue of operator splitting technique [18, 19]. In addition to the distributed computation, under most circumstances, participants can only have access to local information and decisions of their neighbors, which constitutes a partial-decision information setting [20, 21, 22]. One reason is that these players are reluctant to send their local information and decisions to the general public out of privacy concerns while being willing to share these with their trusted neighbors on the whole network. Although there may exist some central node that collects and distributes the necessary information to each player, this central node is subject to performance limitations, such as single point of failure, and limited flexibility and scalability [23]. The limited capacity of communication channels also constraints information sharing over the network.

Significant efforts have been devoted to designing algorithms to solve SGNEPs distributedly under the full-decision information setting where each player has access to all other players’ decisions. The authors of [11] consider a Cartesian stochastic variational inequality problem with a monotone map. They propose a stochastic iterative Tikhonov regularization method and a stochastic iterative proximal-point method to solve it, which let players update their regularization parameters and centering term properly after each iteration. The authors of [24] propose a solution based on the preconditioned forward-backward (FB) operator splitting with the expected-value pseudogradient assumed to be restricted co-coercive and approximated via the stochastic approximation (SA) scheme. To accelerate game dynamics and relax the co-coercivity assumption, [25] adopts a forward-backward-forward framework. These works are mostly gradient-based which enjoy low complexity in implementation and updating per player step. Nevertheless, rational players would take best-response actions given others’ strategies and deviate from gradient-based schemes unless forced by some external authority. The work in [26] provides an inexact generalization of the proximal best-response (BR) schemes to the SNEP whose corresponding proximal BR map admits a contractive property. The authors of [27] further consider SNEPs with composite objectives and design a variable sample-size proximal BR scheme, under a contractive property on the proximal BR map. Yet, far less has been studied when it comes to the distributed solution to SGNEPs with merely partial information. The only existing work to our best knowledge is [28], which also relies on the FB framework along with the SA method. The convergence of the proposed algorithms has been analyzed under the assumption that the preconditioned forward operator is restricted co-coercive, which only allows comparatively small step sizes.

Our contributions can be summarized in the following aspects. First, we propose a distributed algorithm to solve the SGNEP under the partial-decision information setting based on the Douglas-Rachford splitting and the proximal mapping. In the proposed algorithm, the involved players are asked to update their decision vectors in two separate steps: solving the augmented best-response subproblems, and projecting onto the local feasible sets after some linear transformations. The updates of their local estimates and dual variables only require some trivial linear transformations. This algorithm can deal with cases where the scenario-based objectives of players are nonsmooth, and relaxes some commonly-made assumptions such as the $\alpha-$cocoercivity with $\alpha>1/2$ in the FB splitting. Second, we establish the convergence of the proposed algorithm under assumptions concerning the properties of the pseudogradient, the extended pseudogradient, and the stochastic subgradients of the objectives. Without relying on the contractive property, the proof in this paper is based on the Robbins-Siegmund theorem and extends the convergence results discussed in [26]. Drawing tools and techniques from stochastic approximation and convex analysis, we then construct a feasible inexact solver for the augmented best-response subproblems based on the projected stochastic subgradient method and discuss the prescribed accuracy within which the inexact solver should achieve such that the algorithm convergence is ensured. The proposed inexact solver based on the projected stochastic subgradient method requires the projection onto some bounded box sets rather than the (potentially complicated) local feasible sets of the original problem, which considerably improves the computational efficiency.

The remainder of this paper is organized as follows. In Sec. II, we formally formulate the SGNEP on networks and provide some basic definitions as well as assumptions. In this section, we recast the SGNEP as the zero-finding problem of a certain operator and prove that the solution of the latter is a "variational" solution of the former. In Sec. III, a distributed algorithm in a partial-decision information setting is proposed. Sec. IV focuses on the convergence analysis of the proposed algorithm. In this section, we also construct an inexact solver based on the projected stochastic subgradient method. In Sec. V, to demonstrate the theoretical findings and the proposed algorithm in practical applications, we include two numerical examples: a stochastic Nash-Cournot distribution game and a multi-product assembly problem with the two-stage model. Sec. VI concludes the paper and highlights potential extensions and applications.

Basic Notations: For a set of matrices $\{V_{i}\}_{i\in S}$, we let $\text{blkd}(V_{1},\ldots,V_{|S|})$ or $\text{blkd}(V_{i})_{i\in S}$ denote the diagonal concatenation of these matrices, $[V_{1},\ldots,V_{|S|}]$ their horizontal stack, and $[V_{1};\cdots;V_{|S|}]$ their vertical stack. For a set of vectors $\{v_{i}\}_{i\in S}$, $[v_{i}]_{i\in S}$ or $[v_{1};\cdots;v_{|S|}]$ denotes their vertical stack. For a matrix $V$ and a pair of positive integers $(i,j)$, $[V]_{(i,j)}$ denotes the entry on the $i$-th row and the $j$-th column of $V$. For a vector $v$ and a positive integer $i$, $[v]_{i}$ denotes the $i$-th entry of $v$. Denote $\overline{\mathbb{R}}\coloneqq\mathbb{R}\cup\{+\infty\}$, $\mathbb{R}_{+}\coloneqq[0,+\infty)$, and $\mathbb{R}_{++}\coloneqq(0,+\infty)$. $\mathbb{S}^{n}_{+}$ (resp. $S^{n}_{++}$) represents the set of all $n\times n$ symmetric positive semi-definite (resp. definite) matrices. $\iota_{\mathcal{S}}(x)$ is defined to be the indicator function of a set $\mathcal{S}$, i.e., if $x\in\mathcal{S}$, then $\iota_{\mathcal{S}}(x)=0$; otherwise, $\iota_{\mathcal{S}}(x)=+\infty$. $N_{S}(x)$ denotes the normal cone to the set $S\subseteq\mathbb{R}^{n}$ at the point $x$: if $x\in S$, then $N_{S}(x)\coloneqq\{u\in\mathbb{R}^{n}\mid\sup_{z\in S}\langle u,z-x\rangle\leq 0\}$; otherwise, $N_{S}(x)\coloneqq\varnothing$. If $S\in\mathbb{R}^{n}$ is a closed and convex set, the map $\operatorname{Pj}_{S}:\mathbb{R}^{n}\to S$ denotes the projection onto $S$, i.e., $\operatorname{Pj}_{S}(x)\coloneqq\operatorname{argmin}_{v\in S}\lVert v-x\rVert_{2}$. We use $\rightrightarrows$ to indicate a point-to-set map. For an operator $T:\mathbb{R}^{n}\rightrightarrows\mathbb{R}^{n}$, $\text{Zer}(T)\coloneqq\{x\in\mathbb{R}^{n}\mid Tx\ni\boldsymbol{0}\}$ and $\text{Fix}(T)\coloneqq\{x\in\mathbb{R}^{n}\mid Tx\ni x\}$ denote its zero set and fixed point set, respectively. We denote $\operatorname{dom}(T)$ the domain of the operator $T$ and $\text{gra}(T)$ the graph of it. The resolvent and reflected resolvent of $T$ are defined as $J_{T}\coloneqq(I+T)^{-1}$ and $R_{T}\coloneqq 2J_{T}-I$, respectively.

To compute the zeros of the operator $\operatorname{\mathbb{T}}$ given in the previous section, we leverage the Douglas-Rachford (DR) splitting method which combines operator splitting and the Krasnoselskii-Mann (K-M) schemes. Given a nonexpansive operator $Q$ with a nonempty fixed point set $\text{Fix}(Q)$, the K-M scheme [33, Sec. 5.2] suggests the following iteration:

where $(\gamma^{(k)})_{k\in\mathbb{N}}$ is a sequence such that $\gamma^{(k)}\in[0,1]$ for all $k\in\mathbb{N}$ and $\sum_{k\in\mathbb{N}}\gamma^{(k)}(1-\gamma^{(k)})=\infty$. Here, we introduce a set of local bounded box constraints $\{\mathcal{X}^{B}_{i}\}$ which can be chosen manually as long as it satisfies $\mathcal{X}_{i}\subseteq\mathcal{X}^{B}_{i}$ for all $i\in\mathcal{N}$. We similarly define the extended box set $\hat{\mathcal{X}}^{B}\coloneqq\hat{\mathcal{X}}^{B}_{1}\times\hat{\mathcal{X}}^{B}_{2}\times\cdots\hat{\mathcal{X}}^{B}_{N}$ where the extended box set of each player $i$ is defined as $\hat{\mathcal{X}}^{B}_{i}\coloneqq\mathbb{R}^{n_{<i}}\times\mathcal{X}^{B}_{i}\times\mathbb{R}^{n_{>i}}$. It is easy to see that the normal cones of $\hat{\mathcal{X}}^{B}$ and $\hat{\mathcal{X}}$ satisfy $N_{\hat{\mathcal{X}}^{B}}+N_{\hat{\mathcal{X}}}=N_{\hat{\mathcal{X}}}$. The motivation for introducing these box sets is to simplify the computation while maintaining boundedness for the convergence analysis as we will show later in this paper. We split the operator $\operatorname{\mathbb{T}}$ into the following operators $\mathbb{A}$ and $\mathcal{B}$:

with $D$, $\mathcal{A}_{y}$, and $\mathcal{B}_{y}$ defined by

Furthermore, we introduce the following design matrix $\Phi$ for distributedly computing the resolvents $J_{\Phi^{-1}\operatorname{\mathbb{A}}}$ and $J_{\Phi^{-1}\operatorname{\mathcal{B}}}$:

where $\boldsymbol{\tau}_{1}\coloneqq\text{blkd}(\tau_{11}I_{n},\ldots,\tau_{1N}I_{n})$ with $\tau_{11}\in\mathbb{R}_{++},\ldots,\tau_{1N}\in\mathbb{R}_{++}$; similarly for $\boldsymbol{\tau}_{2}$, $\boldsymbol{\tau}_{3}$ and $\boldsymbol{\tau}_{4}$. Notice that these step sizes $\boldsymbol{\tau}_{1},\ldots,\boldsymbol{\tau}_{4}$ should be small enough to guarantee that $\Phi$ is positive definite. Conservative upper bounds for these step sizes can be derived using the Gershgorin circle theorem [34].

Here, $d_{i}$ denotes the degree of node/player $i$. In general, determining the above step sizes requires some global information acquired through coordination among players such as a proper $\rho_{\mu}$. After the incorporation of the design matrix $\Phi$, we now work in the inner product space $\mathcal{K}$ which is a real vector space endowed with the inner product $\langle\psi_{1},\psi_{2}\rangle_{\mathcal{K}}=\psi_{1}^{T}\Phi\psi_{2}$. For brevity, let $\operatorname{\bar{\mathbb{A}}}\coloneqq\Phi^{-1}\operatorname{\mathbb{A}}$ and $\operatorname{\bar{\mathcal{B}}}\coloneqq\Phi^{-1}\operatorname{\mathcal{B}}$. In the DR splitting scheme, the general operator $Q$ in (12) is given by $\mathscr{R}_{*}\coloneqq R_{\operatorname{\bar{\mathcal{B}}}}\circ R_{\operatorname{\bar{\mathbb{A}}}}$ and it suggests the following exact iteration:

Given a generic single-valued operator $Q$, we say that $Q$ is restricted nonexpansive w.r.t. a set $S$ if, for all $\psi\in\operatorname{dom}{Q}$ and $\psi^{*}\in S$, $\lVert Q\psi-Q\psi^{*}\rVert\leq\lVert\psi-\psi^{*}\rVert$ [20]; if, in addition, $S=\text{Fix}(Q)$, then $Q$ is quasinonexpansive [33, Def. 4.1(v)]. From the main convergence results in [32, Thm. 2&3], if Assumptions 1 to 6 hold, even though $\mathscr{R}_{*}$ is not nonexpansive in a general sense, it possesses quasinonexpansiveness in the inner-product space $\mathcal{K}$, and hence the sequence $(y_{i}^{(k)})_{k\in\mathbb{N}}$ generated by the exact iteration above (see [32, Algorithm 1] for detailed implementations) will converge to a v-SGNE of the original problem defined in (3).

However, unlike the problem setting in [32] where each player has a closed-form objective function, here the objective function is expected-value, and all too often its closed-form expression may be unavailable. Consequently, the $\operatorname{argmin}$ operation in the first player loop of [32, Algorithm 1] can not be carried out exactly. In this case, we need a desirable inexact solver such that, although at each iteration step, it can only get an approximate solution, the computed sequence can still eventually converge to a v-SGNE. We let $R_{\operatorname{\bar{\mathcal{A}}}}$ denote the (scenario-based) approximate operator to the exact reflected resolvent $R_{\operatorname{\bar{\mathbb{A}}}}$, and $\mathscr{R}$ denote the corresponding composite $R_{\operatorname{\bar{\mathcal{B}}}}\circ R_{\operatorname{\bar{\mathcal{A}}}}$. Substituting the operator $\mathscr{R}_{*}$ with $\mathscr{R}$ in [32, Algorithm 1] gives rise to the following approximate iteration:

The updating steps of (18) are presented in Algorithm 1. For brevity, let ${\tilde{y}}^{-i(k)}_{iL}\coloneqq\sum_{\scriptscriptstyle j\in\mathcal{N}_{i}}({\tilde{y}}^{-i(k)}_{i}-{\tilde{y}}^{-i(k)}_{j})$, and similarly for ${\tilde{y}}^{i(k)}_{iL}$, ${\tilde{\lambda}}^{(k)}_{iL}$, ${\hat{y}}^{i(k+1)}_{iL}$, and ${\hat{\lambda}}^{(k+1)}_{iL}$; let ${\tilde{\mu}}^{-i(k)}_{iB}\coloneqq\sum_{\scriptscriptstyle j\in\mathcal{N}^{+}_{i}}{\tilde{\mu}}^{-i(k)}_{ji}-\sum_{\scriptscriptstyle j\in\mathcal{N}^{-}_{i}}{\tilde{\mu}}^{-i(k)}_{ij}$, and similarly for ${\tilde{\mu}}^{i(k)}_{iB}$, ${\tilde{z}}^{(k)}_{iB}$, ${\hat{\mu}}^{(k+1)}_{iB}$, and ${\hat{z}}^{(k+1)}_{iB}$; let ${\hat{y}}^{(k+1)}_{ji}\coloneqq{\hat{y}}^{(k+1)}_{i}-{\hat{y}}^{(k+1)}_{j}$, and similarly for ${\hat{\lambda}}^{(k+1)}_{ji}$, ${\bar{y}}^{(k+1)}_{ji}$, and ${\bar{\lambda}}^{(k+1)}_{ji}$.

Depending on the inexact solver adopted, $R_{\operatorname{\bar{\mathcal{A}}}}$ usually admits no explicit formulas. Yet, as will be shown later in the next section, we can still establish the convergence of Algorithm 1 based on some specific properties of $R_{\operatorname{\bar{\mathcal{A}}}}$.