Decentralized Optimization On Time-Varying Directed Graphs Under Communication Constraints

In recent years, decentralized optimization has attracted considerable interest from the machine learning, signal processing, and control communities [13, 14, 11, 9]. We consider the setting where a collection of agents attempts to minimize an objective that consists of functions distributed among the agents; each agent evaluates one of the functions on its local data. Formally, this optimization task can be stated as

where $n$ is the number of agents and $f_{i}:\mathbb{R}^{d}\to\mathbb{R}$ is the function assigned to the $i\textsuperscript{th}$ node, $i\in[n]:=\left\{1,...,n\right\}$. The agents collaborate by exchanging information over a network modeled by a time-varying directed graph ${\mathcal{G}}(t)=(|n|,{\mathcal{E}}(t))$, where ${\mathcal{E}}(t)$ denotes the set of edges at time $t$; agent $i$ can send a message to agent $j$ at time $t$ if there exist an edge from $i$ to $j$ at $t$, i.e., if $\left\{i,j\right\}\in{\mathcal{E}}(t)$.

The described setting has been a subject of extensive studies over the last decade, leading to a number of seminal results [12, 4, 17, 3, 10, 15, 7]. Majority of prior work assumes symmetry in the agents’ communication capabilities, i.e., models the problem using undirected graphs. However, the assumption of symmetry is often violated and the graph that captures properties of the communication network should be directed. Providing provably convergent decentralized convex optimization schemes over directed graphs is challenging; technically, this stems from the fact that unlike in undirected graphs, the so-called mixing matrix of a directed graph is not doubly stochastic. The existing prior work in the directed graph settings includes the grad-push algorithm [5, 11], which compensates for the imbalance in a column-stochastic mixing matrix by relying on local normalization scalars, and the directed distributed gradient descent (D-DGD) scheme [19] which carefully tracks link changes over time and their impact on the mixing matrices. Assuming convex local function, both of these methods achieve $\mathcal{O}(\frac{\mathrm{ln}T}{\sqrt{T}})$ convergence rate.

In practice, communication bandwidth is often limited and thus the amount of information that can be exchanged between the agents is restricted. This motivates design of decentralized optimization schemes capable of operating under communication constraints; none of the aforementioned methods considers such settings. Recently, techniques that address communication constraints in decentralized optimization by quantizing or sparsifying messages exchanged between participating agents have been proposed in literature [18, 20, 15]. Such schemes have been deployed in the context of decentralized convex optimization over undirected networks [7] as well as in fixed directed networks [16]. However, there has been no prior work on communication-constrained decentralized learning over time-varying directed networks.

In this paper we propose, to our knowledge the first, communication-sparsifying scheme for decentralized convex optimization over directed networks, and provide formal guarantees of its convergence; in particular, we show that the proposed method achieves $\mathcal{O}(\frac{\mathrm{ln}T}{\sqrt{T}})$ convergence rate. Experiments demonstrate efficacy of the proposed scheme.

Assume that a collection of agents aims to collaboratively find the unique solution to decentralized convex optimization (1); let us denote this solution by ${\mathbf{x}}^{*}$ and assume, for simplicity, that $\mathcal{X}=\mathbb{R}^{d}$. The agents, represented by nodes of a directed time-varying graph, are allowed to exchange sparsified messages. In the following, we do not assume smoothness or strong convexity of the objective; however, our analysis can be extended to such settings.

Let $W_{in}^{t}$ (row-stochastic) and $W_{out}^{t}$ (column-stochastic) denote the in-neighbor and out-neighbor connectivity matrix at time $t$, respectively. Moreover, let $\mathcal{N}_{{in},i}^{t}$ be the set of nodes that can send information to node $i$ (including $i$), and $\mathcal{N}_{out,j}^{t}$ the set of nodes that can receive information from node $j$ (including $j$) at time $t$. We assume that both $\mathcal{N}_{{in},i}^{t}$ and $\mathcal{N}_{out,i}^{t}$ are known to node $i$. A simple policy for designing $W^{t}_{in}$ and $W^{t}_{out}$ is to set

We assume that the constructed mixing matrices have non-zero spectral gaps; this is readily satisfied in a variety of settings including when the union graph is jointly-connected. Matrices $W_{in}^{t}$ and $W_{out}^{t}$ can be used to synthesize the mixing matrix, as formally stated in Section 3 (see Definition 1).

To reduce the size of the messages exchanged between agents in a network, we perform sparsification. In particular, each node uniformly at random selects and communicates $k$ out of $d$ entries of a $d$-dimensional message. To formalize this, we introduce a sparsification operator $Q:\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$. The operator $Q$ is biased, i.e., $\mathbb{E}[Q({\mathbf{x}})]\neq{\mathbf{x}}$, and has variance that depends on the norm of its argument, $\mathbb{E}[\|Q({\mathbf{x}})-{\mathbf{x}}\|^{2}]\propto\|{\mathbf{x}}\|^{2}$. Biased compression operators have previously been considered in the context of time-invariant networks [15, 7, 6, 16] but are not encountered in time-varying network settings.

A common strategy to solving decentralized optimization problems is to orchestrate exchange of messages between agents such that each message consists of a combination of compressed messages from neighboring nodes and a gradient noise term. The gradient term is rendered vanishing by adopting a decreasing stepsize scheme; this ensures that the agents in the network reach a consensus state which is the optimal solution to the optimization problem.

To meet communication constraints, messages may be sparsified; however, simplistic introduction of sparsification to the existing methods, e.g., [5, 1, 2, 11], may have adverse effect on their convergence – indeed, modified schemes may only converge to a neighborhood of the optimal solution or even end up diverging. This is caused by the non-vanishing error due to the bias and variance of the sparsification operator. We note that the impact of sparsification on the entries of a state vector in the network can be interpreted as that of link failures; this motivates us to account for it in the structure of the connectivity matrices. Specifically, we split the vector-valued decentralized problem into $d$ individual scalar-valued sub-problems with the coordinate in-neighbor and out-neighbor connectivity matrices, $\{W_{in,m}^{t}\}_{m=1}^{d}$ and $\{W_{out,m}^{t}\}_{m=1}^{d}$, specified for each time $t$. If an entry is sparsified at time $t$ (i.e., set to zero and not communicated), the corresponding coordinate connectivity matrices are no longer stochastic. To handle this issue, we re-normalize the connectivity matrices $\{W_{in,m}^{t}\}_{m=1}^{d}$ and $\{W_{out,m}^{t}\}_{m=1}^{d}$, ensuring their row stochasticity and column stochasticity, respectively; node $i$ performs re-normalization of the $i\textsuperscript{th}$ row of $\{W_{in,m}^{t}\}_{m=1}^{d}$ and $i\textsuperscript{th}$ column of $\{W_{out,m}^{t}\}_{m=1}^{d}$ locally. We denote by $\{A_{m}^{t}\}_{m=1}^{d}$ and $\{B_{m}^{t}\}_{m=1}^{d}$ the weight matrices resulting from the re-normalization of $\{W_{in,m}^{t}\}_{m=1}^{d}$ and $\{W_{out,m}^{t}\}_{m=1}^{d}$, respectively.

Following the work of [1] on average consensus, we introduce an auxiliary vector $\mathbf{y}_{i}\in\mathbb{R}^{d}$ for each node. Referred to as the surplus vector, $\mathbf{y}_{i}\in\mathbb{R}^{d}$ records variations of the state vectors over time and is used to help ensure the state vectors approach the consensus state. At time step $t$, node $i$ compresses $\mathbf{x}_{i}^{t}$ and $\mathbf{y}_{i}^{t}$ and sends both to the current out-neighbors. To allow succinct expression of the update rule, we introduce $\mathbf{z}_{i}^{t}\in\mathbb{R}^{d}$ defined as

The sparsification operator $Q(\cdot)$ is applied to $\mathbf{z}_{i}^{t}$, resulting in $Q(\mathbf{z}_{i}^{t})$; we denote the $m\textsuperscript{th}$ entry of the sparsified vector by $[Q({{\mathbf{z}}}_{i}^{t})]_{m}$. The aforementioned weight matrix $A^{t}_{m}$ is formed as

where $\mathcal{S}_{m}^{t}(i,j):=\{j|j\in\mathcal{N}^{t}_{in,i},[Q({{\mathbf{z}}}_{j}^{t})]_{m}\neq 0\}\cup\{i\}$. Likewise, $B^{t}_{m}$ is defined as

where $\mathcal{T}_{m}^{t}(i,j):=\{i|i\in\mathcal{N}^{t}_{out,j},[Q({{\mathbf{z}}}_{i}^{t})]_{m}\neq 0\}\cup\{j\}$.

To obtain the update rule for the optimization algorithm, we first need to define the mixing matrix of a directed network with sparsified messages.

With $\mathbf{z}_{i}^{t}$ and $\bar{M}_{m}^{t}$ defined in (3) and (6), respectively, node $i$ updates the $m\textsuperscript{th}$ component of its message according to

where $g_{im}^{t}$ denotes the $m\textsuperscript{th}$ entry of the gradient vector $\mathbf{g}_{i}^{t}$ constructed as

Moreover, $F=\left[\begin{matrix}\mathbf{0}&I\\ \mathbf{0}&-I\end{matrix}\right]$, and $\alpha_{t}$ is the stepsize at time $t$.

In (7), the update of vectors ${\mathbf{z}}_{i}^{t}$ consists of a mixture of the compressed state vectors and surplus vectors, and includes a vanishing gradient computed from history. The mixture of compressed messages can be interpreted as obtained by sparsification and multiplication with the mixing matrix from the previous time steps, except for the times when

When $t$ satisfies (9), the update of ${\mathbf{z}}_{i}^{t}$ incorporates stored vectors ${\mathbf{z}}_{i}^{\mathcal{B}\lfloor t/\mathcal{B}\rfloor}$. Note that ${\mathbf{z}}_{i}^{\mathcal{B}\lfloor t/\mathcal{B}\rfloor}$ is multiplied by $\epsilon F$, where the perturbation parameter $\epsilon$ determines the extent $F$ affects the update. One can show that $\epsilon F$, in combination with the mixing matrix $\bar{M}_{m}^{t}$, guarantees non-zero spectral gap of the product matrix over $\mathcal{B}$ consecutive time steps starting from $t=k\mathcal{B}$. Similarly, gradient term $\alpha_{\lfloor t/\mathcal{B}\rfloor}g_{im}^{\mathcal{B}\lfloor t/\mathcal{B}\rfloor}$, computed using state vectors ${\mathbf{x}}_{i}^{t-(\mathcal{B}-1)}$, participates in the update when (9) holds. We formalize the proposed procedure as Algorithm 1.

Remark. It is worth pointing out that in Algorithm 1 each node needs to store local messages of size $4d$ (four $d$-dimensional vectors: the current state and surplus vectors, past surplus vector, and local gradient vector). Only the two current vectors may be communicated to the neighboring nodes while the other two vectors are used locally when (9) holds. Note that $\bar{M}_{m}^{t}$ has column sum equal to one but it is not column-stochastic due to having negative entries. Finally, note that when $\mathcal{B}=1$, the network is strongly connected at all times.

We test Algorithm 1 in applications to linear and logistic regression, and compare the results to Q-Grad-Push, obtained by applying simple quantization to the push-sum scheme [11], and Q-De-DGD [16]. Neither of these two schemes was developed with communication-constrained optimization over time-varying directed networks in mind – the former was originally proposed for unconstrainted communication, while the latter is concerned with static networks. However, since there is no prior work on decentralized optimization over time-varying directed networks under communication constraints, we adopt them for the purpose of benchmarking.

We use Erdős–Rényi model to generate strongly connected instances of a graph with $10$ nodes and edge appearance probability $0.9$. Two uni-directional edges are dropped randomly from each such graph while still preserving strong connectivity. We then remove in-going and out-going edges of randomly selected nodes to create a scenario where an almost-surely strongly connected network is formed only after taking a union of graphs over $\mathcal{B}$ time instances (see Assumption 1). Finally, recall that $q$ denotes the fraction of entries that nodes communicate to their neighbors (small $q$ implies high compression).

Decentralized linear regression. First, consider the optimization problem $\min_{{\mathbf{x}}}\frac{1}{n}\sum_{i=1}^{n}\|\mathbf{y}_{i}-D_{i}\mathbf{x}_{i}\|^{2},$ where $D_{i}\in\mathbb{R}^{200\times 128}$ is a local data matrix with $200$ data points of size $d=128$ at node $i$, and $\mathbf{y}_{i}\in\mathbb{R}^{200}$ represents the local measurement vector at node $i$. We generate $\mathbf{x}^{*}$ from a normal distribution, and set up the measurement model as $\mathbf{y}_{i}=M_{i}\mathbf{x}^{*}+\eta_{i}$, where $M_{i}$ is randomly generated from the standard normal distribution; finally, the rows of the data matrix are normalized to sum to one. The local additive noise $\eta_{i}$ is generated from a zero-mean Gaussian distribution with variance $0.01$. In Algorithm 1 and Q-Grad-Push, local vectors are initialized randomly to $\mathbf{x}_{i}^{0}$; Q-De-DGD is initialized with an all-zero vector. The quantization level of the benchmarking algorithms is selected to ensure that the number of bits those algorithms communicate is equal to that of Algorithm 1 when $q=0.09$. All algorithms are run with stepsize $\alpha_{t}=\frac{0.2}{t}$. The performance of different schemes is quantified by the residuals $\frac{\|\mathbf{x}^{t}-\bar{\mathbf{x}}\|}{\|\mathbf{x}^{0}-\bar{\mathbf{x}}\|}$. The results are shown in Fig.1 (a), (b). As shown in the subplots, for all the considered sparsification rates Algorithm 1 converges with rate proportional to $q$, while the benchmarking algorithms do not converge to the optimal solution.

Decentralized logistic regression. Next, we consider a multi-class classification task on the MNIST dataset [8]. The logistic regression problem is formulated as

The data is distributed across the network such that each node $i$ has access to $N=120$ training samples $(\mathbf{m}_{ij},y_{ij})\in\mathbb{R}^{64}\times\{0,\cdots,9\}$, where $\mathbf{m}_{ij}$ denotes a vectorized image with size $d=64$ and $y_{ij}$ denotes the corresponding digit label. Performance of Algorithm 1 is again compared with Q-Grad-Push and Q-De-DGD; all algorithms are initialized with zero vectors. The quantization level of the benchmarking algorithms is selected such that the number of bits they communicate is equal to that of Algorithm 1 for $q=0.07$. The experiment is run using the stepsize $\alpha_{t}=\frac{0.02}{t}$; we set $\mu=10^{-5}$. Fig. 1 (c), (d) show the classification correct rate of Algorithm 1 for different sparsification and connectivity levels. As can be seen there, all sparsified schemes achieve the same level of the classification correct rate. The schemes communicating fewer messages in less connected networks converge slower, while the two benchmarking algorithms converge only to a neighborhood of the optimal solution.

We considered the problem of decentralized learning over time-varying directed graphs where, due to communication constraints, nodes communicate sparsified messages. We proposed a communication-efficient algorithm that achieves $\mathcal{O}(\frac{\mathrm{ln}T}{\sqrt{T}})$ convergence rate for general decentralized convex optimization tasks. As part of the future work, it is of interest to study reduction of the computational cost of the optimization procedure by extending the results to the setting where network agents rely on stochastic gradients.