DOT and DOP: Linearly Convergent Algorithms for Finding Fixed Points of Multi-Agent Operators ^††thanks:

Fixed point theory in real Hilbert spaces is known as a powerful tool in a variety of domains such as optimization, engineering, economics, game theory, and nonlinear numerical analysis [1, 2]. Generally speaking, the main goal is to devise algorithms for computing a fixed point of an operator.

Up to now, plenty of research has extensively addressed centralized algorithms for finding a fixed point of nonexpansive or quasi-nonexpansive operators in the literature [3, 4, 5, 6], where a central/global coordinator or computing unit is able to access all information of the studied problem. It is known that the typical Picard iteration in general does not converge for nonexpansive operators (e.g., a simple example is the operator $-Id$ with nonzero initial points, where $Id$ is the identity operator), although it usually performs well for contractive operators. For nonexpansive operators, one prominent algorithm is the so-called Krasnosel’skiĭ-Mann (KM) iteration [7, 8], and it is shown to converge weakly to a fixed point of a nonexpansive operator in real Hilbert spaces under mild conditions [9].

In recent few decades, distributed (or decentralized) algorithms have been an active topic in a wide range of domains, including fixed point theory, computer science, game theory, and control theory, and so on, mostly inspired by the fact that distributed algorithms, in contrast with centralized ones, possess a host of fascinating advantages, such as low cost, robustness to failures or antagonistic attacks, privacy preservation, low computational complexity, and so on. Distributed algorithms do not assume global/central coordinators or computing units and instead a finite group of agents (e.g., computing units and robots, and so on.), who may be spatially separated, aim to solve a global problem in a collaborative manner by local information exchanges. Wherein, local information exchanges are often depicted by a simple graph, connoting that every agent can interact with only a subset of agents, instead of all agents. Along this line, distributed algorithms have thus far been investigated extensively under both fixed and time-varying communication graphs in distributed optimization [10, 11, 12], game theory [13], and multi-agent systems/networks [14, 15], to just name a few.

In more recent years, distributed algorithms have received a growing attention in the fixed point finding problem [16, 17, 18, 19, 20, 21]. For instance, a synchronous distributed algorithm was proposed in [16] for computing a common fixed point of a collection of paracontraction operators, and for the same problem, an asynchronous distributed algorithm was developed in [17]. Meanwhile, different from paracontraction operators, another type of operators (i.e., strongly quasi-nonexpansive operators) was addressed for the common fixed point seeking problem in [18] by designing a distributed algorithm in the presence of time-varying delays under the assumption that the communication graph is repeatedly jointly strongly connected. It should be noted that many interesting problems can boil down to the common fixed point finding problem, such as convex feasibility problems [22, 23] and the problem of solving linear algebraic equations in a distributed fashion [24, 25, 26], and so forth. For example, the linear algebraic equation solving problem was formulated such that the distribution of random communication graphs is not required, which include asynchronous updates and/or unreliable interconnection protocols. Notice that all the aforementioned works are in the Euclidean space. Regarding the Hilbert space, the authors in [19] investigated distributed optimization under random and directed interconnection graphs, where a distributed algorithm was proposed and shown to be convergent in both almost surely and mean square sense along with the introduction of a novel convex minimization problem over the fixed-value point set of a nonexpansive random operator. In addition, the authors in [20] took into account the common fixed point finding problem for a finite collection of nonexpansive operators, where two distributed algorithms were proposed with a full coordinate updating and a random block-coordinate updating, respectively, and compared with [16, 17, 18], the contributions of [20] lie in the study of real Hilbert spaces, the consideration of operator errors, and the establishment of a sublinear convergence speed. Furthermore, a more general scenario, where no common fixed points are assumed for all local operators, was investigated in [21], where two distributed algorithms were devised to resolve the problem. It is noteworthy that, to our best knowledge, [21] is the first to investigate the fixed point finding problem of a global operator in real Hilbert spaces, where the global operator is an average of local operators over a multi-agent network. Nevertheless, the convergence rate is not analyzed for the proposed algorithms in [21].

Motivated by the above facts, the purpose of this paper is to further investigate the fixed point finding problem of a quasi-nonexpansive global operator over a time-invariant, directed, and unbalanced communication graph. Two scenarios are taken into account, that is, the global operator is sum separable and block separable. In the first case, the global operator is composed of a sum of local operators, and in the second case, the global operator is comprised of a family of local block operators. That is, in both cases the global operator is separable and consists of local operators, which are assumed to be Lipschitz and are only privately accessible to each individual agent, thereby needing all agents to tackle the global problem in a collaborative manner. The contributions of this paper are threefold as follows.

1. 1.

   For the first case, a distributed algorithm, called distributed quasi-averaged operator tracking algorithm (DOT), is developed and shown to be convergent to a fixed point of the global operator at a linear rate under a linear regularity condition. Compared with the closely related work [21], where no convergence speed is provided, a different algorithm is developed here and shown to be convergent at a linear rate. It should be noted that the problem here is more general than the common fixed point seeking problem [17, 16, 18, 20], where all local operators are assumed to have at least one common fixed point, while this assumption is dropped here. As a special case, linear convergence can also be ensured for the common fixed point seeking problem. In contrast, [20] only provides a sublinear rate for nonexpansive operators.

2. 2.

   For the second case, a distributed algorithm, called distributed quasi-averaged operator playing algorithm (DOP), is proposed, which is shown to be linearly convergent to a fixed point of the global operator under the linear regularity condition. To our best knowledge, this is the first to study the block separable case in a decentralized manner.

3. 3.

   The studied setups in this paper provide a unified framework for a host of interesting problems. For example, the proposed DOT and DOP algorithms can be leveraged to resolve distributed optimization and multi-player games under partial-decision information.

A preliminary version of the paper was presented at a conference [27]. The present paper extends the result of [27] in several ways. [27] only considers the sum separable case without providing detailed proofs of the main result. In comparison, a full proof of the main result (i.e., Theorem 1) for the sum separable case is provided. Besides, one more scenario is investigated in this paper, i.e., the block separable case (see Theorem 2). Also, one more applications is presented here, i.e., multi-player games under partial-decision information, along with one more numerical example.

The structure of this paper is as follows. Some basic knowledge and the problem formulation are introduced in Section II, and the first case with the global operator being sum separable is addressed in Section III, followed by the second case with the global operator being block-coordinate separable in Section IV. Several applications are provided in Section V. Some numerical examples are presented in Section VI, and the conclusion is drawn in Section VII.

This section aims to develop a distributed algorithm for tackling problem (9) which can converge at a linear rate. Without loss of generality, the vectors in $\mathcal{H}$ are viewed as column vectors in this section.

For problem (9), if $F$ can be known by a global/central computing unit (or coordinator), then a famous centralized algorithm, called the KM iteration [35], can be exploited, i.e.,

where $\{\alpha_{k}\}_{k\in\mathbb{N}}$ is a sequence of relaxation parameters with $\alpha_{k}\in[0,1]$. Note that the KM iteration usually applies to nonexpansive operators, but it still works for quasi-nonexpansive operators here under the linear regularity condition in Assumption 3. However, the centralized iteration (12) is not realistic here since no global/central computing unit (or coordinator) exists in our setting, which hence motivates us to devise distributed (or decentralized) algorithms based on only local information exchanges among all agents.

Motivated by the classical KM iteration and the tracking techniques such as those in [36, 34, 37], a distributed quasi-averaged operator tracking algorithm (DOT) is proposed as

where $x_{i,k}$ is an estimate of a fixed point of the global operator $F$ by agent $i$ at time $k\geq 0$ for all $i\in[N]$, and $\alpha\in(0,1)$ is the stepsize to be determined. Set the initial conditions as: arbitrary $x_{i,0}\in\mathcal{H}$, $y_{i,0}=F_{i}(x_{i,0})$, and $w_{i,0}=1$ for all $i\in[N]$. It is noteworthy that neighboring agents are only involved in (13) for each agent due to $a_{ij}=0$ and $b_{ij}=0$ when agent $j$ is not a neighbor of agent $i$.

Roughly speaking, $y_{i,k}$ is employed to track the weighted global operator $\nu_{i}\sum_{i=1}^{N}F_{i}(x_{i,k})$, and meanwhile $w_{i,k}$ is a scalar used to track $\nu_{i}N$ in order to counteract the imbalance of the matrix $B$ in (13b).

For (13c), it is easy to verify that each $w_{i,k}$ will exponentially converge to $N\nu_{i}$. However, invoking the method in [38], it can be concluded that the final value $N\nu_{i}$ can be evaluated for each agent $i$ in finite time in a distributed manner. Because of this, without loss of generality, algorithm (13) can be rewritten by replacing $w_{i,k}$ with $N\nu_{i}$ as in Algorithm 1.

To facilitate the following analysis, Algorithm 1 can be written in a compact form

where $x_{k},y_{k}$ are concatenated vectors of $x_{i,k},y_{i,k}$, respectively, $\mathbf{A}:=A\otimes Id$, $\mathbf{B}:=B\otimes Id$, $D_{\nu}:=diag\{\nu_{1},\ldots,\nu_{N}\}$, and $\mathbf{F}(z):=col(F_{1}(z_{1}),\ldots,F_{N}(z_{N}))$ for a vector $z=col(z_{1},\ldots,z_{N})\in\mathcal{H}^{N}:=\mathcal{H}\times\cdots\times\mathcal{H}$ (the $N$-fold Cartesian product of $\mathcal{H}$).

To proceed, it is helpful to introduce two new weighted norms for the Cartesian product $\mathcal{H}^{N}$, i.e.,

for any vector $z=col(z_{1},\ldots,z_{N})\in\mathcal{H}^{N}$. Let $\|\cdot\|$ be the natural norm in $\mathcal{H}^{N}$, i.e., $\|z\|:=\sqrt{\sum_{i=1}^{N}\|z_{i}\|^{2}}$. Additionally, it is also necessary to introduce two weighted norms in $\mathbb{R}^{N}$ [34], i.e., for any $x=col(x_{1},\ldots,x_{N})\in\mathbb{R}^{N}$,

Please note that the notations $\|\cdot\|_{\pi}$ and $\|\cdot\|_{\nu}$ in (17) and (18) should be easily distinguished by the context. Accordingly, let us denote by $\|M\|_{\pi}$ and $\|M\|_{\nu}$ (resp. $\|M\otimes T\|_{\pi}$ and $\|M\otimes T\|_{\nu}$) the norms for a matrix $M\in\mathbb{R}^{N\times N}$ (resp. a matrix $M\in\mathbb{R}^{N\times N}$ and an operator $T$ in $\mathcal{H}$) induced by $\|\cdot\|_{\pi}$ and $\|\cdot\|_{\nu}$ in (18) (resp. (17)), respectively.

It is easy to see that the natural norm $\|\cdot\|$ is equivalent to $\|\cdot\|_{\pi}$, $\|\cdot\|_{\nu}$ in (17), (18), and thus to the induced matrix norms, that is, there are positive constants $c_{i},i\in[4]$ such that

Then the following results can be obtained.

To move forward, an important result for the convergence analysis is first given below.

We are now ready to give the main result of this section.

As a special case, when all local operators have at least one common fixed point, problem (9) will reduce to the common fixed point seeking problem due to $Fix(F)=\cap_{i=1}^{N}Fix(F_{i})$ in this case (e.g., Proposition 4.47 in [1]). Therefore, we have the following result.

Note that the convergence speed is also analyzed for the common fixed point seeking problem in [20] (i.e., the DO algorithm), where the main difference between DOT and DO is that an estimate is introduced for each agent to track the global operator $F$ here, while each agent does not perform this tracking in the DO algorithm. However, the rate is sublinear and all operators are assumed to be nonexpansive in [20], while a linear rate is provided here in Corollary 1 and less conservative operators, i.e., quasi-nonexpansive operators, are considered. Note that time-varying communication graphs were considered with non-identical stepsizes for the DO algorithm in [20], while this paper is concerned with static communication graphs with an identical stepsize for all agents. Along this line, it is interesting to further address the case with non-identical stepsizes for different agents and time-varying communication graphs in future.

This section is concerned with solving problem (10). Without loss of generality, the vectors in $\mathcal{H}$ are viewed as row vectors in this section for convenient analysis.

For problem (10), each agent $i\in[N]$ can only privately access $\texttt{F}_{i}$ with its own vector $x_{i}$ for a whole vector $x=(x_{1},\ldots,x_{N})\in\mathcal{H}$ over a network of $N$ agents, where $x_{i}$ is privately known by agent $i$ itself, as commonly encountered in multi-player games, and so on. To handle this problem, each agent $i\in[N]$ maintains a vector $x_{k}^{i}=(x_{1,k}^{i},\ldots,x_{N,k}^{i})\in\mathcal{H}$ at time step $k\geq 0$ as an estimate of a fixed point of $F$, where $x_{j,k}^{i}$ is an estimate of $x_{j,k}$ (i.e., the vector of agent $j$ at time $k$) by agent $i$ at time $k$ with $x_{i,k}^{i}=x_{i,k}$. That is, each agent $i$ updates its own vector $x_{i,k}$ at time slot $k$ without access to the vectors of all other agents $j\neq i$, and thus each agent $i$ needs to estimate other agents’ vectors $x_{j,k}$ denoted as $x_{k}^{i}$ at each time $k\geq 0$ over the communication graph $\mathcal{G}$ satisfying Assumption 1.

Now, a distributed algorithm is proposed as in Algorithm 2, where $A=(a_{ij})\in\mathbb{R}^{N\times N}$ is the communication matrix introduced after Assumption 1, which is only row-stochastic, and $x_{-i,k}^{j}:=(x_{1,k}^{j},\ldots,x_{i-1,k}^{j},x_{i+1,k}^{j},\ldots,x_{N,k}^{j})$ for all $i,j\in[N]$, i.e., $x_{-i,k}^{j}$ is the agent $j$’s estimate of all agents’ vectors except the $i$-th agent. We recall that $\pi$ is the left stochastic eigenvector of $A$ associated with the eigenvalue $1$ as introduced in Assumption 1.2. It should be noted that the $i$-th entry $\pi_{i}$ of $\pi$ can be evaluated by agent $i$ in finite time in a distributed fashion using the approach in [38]. Thus, Algorithm 2 is distributed.

To ease the upcoming analysis, let us define $x_{k}:=col(x_{k}^{1},\ldots,x_{k}^{N})\in\mathcal{H}^{N}$, $\hat{x}_{i,k}:=\sum_{j=1}^{N}a_{ij}x_{i,k}^{j}$, and $\bar{F}:=diag\{(\texttt{F}_{1}(x_{k}^{1})-\hat{x}_{1,k})/\pi_{1},\ldots,(\texttt{F}_{N}(x_{k}^{N})-\hat{x}_{N,k})/\pi_{N}\}$. Then algorithm (35) can be written in a compact form

Multiplying $\pi^{\top}$ on both sides of (36) yields that

where $\tilde{x}_{k}=(\tilde{x}_{1,k},\ldots,\tilde{x}_{N,k}):=\sum_{i=1}^{N}\pi_{i}x_{k}^{i}$ and $\tilde{F}:=(\texttt{F}_{1}(x_{k}^{1})-\hat{x}_{1,k},\ldots,\texttt{F}_{N}(x_{k}^{N})-\hat{x}_{N,k})$.

To move forward, it is useful to recall the weighted norm $\|z\|_{\pi}:=\sqrt{\sum_{i=1}^{N}\pi_{i}\|z_{i}\|^{2}}$ for a vector $z=col(z_{1},\ldots,z_{N})\in\mathcal{H}^{N}$, as defined in (17). And let $\|\cdot\|$ be a norm in $\mathcal{H}^{N}$ defined by $\|z\|:=\sqrt{\sum_{i=1}^{N}\|z_{i}\|^{2}}$. Remember that the vectors in $\mathcal{H}$ are seen as row vectors in this section. Then similar to (25) in Lemma 2, it is easy to obtain the following result.

To ensure the linear convergence, Assumptions 2 and 3 are still imposed in this section, but $F_{i}$ in Assumption 2 is replaced with $\texttt{F}_{i}$, i.e., $\|\texttt{F}_{i}(x)-\texttt{F}_{i}(y)\|\leq L_{i}\|x-y\|,~{}\forall x,y\in\mathcal{H}$ for $i\in[N]$.

With the above preparations, we are now ready to give the main result of this section.