Distributed gradient-based optimization in the presence of dependent aperiodic communication

One of the most common models in the distributed optimization literature is a time-varying network model that is represented by a time-varying graph (Definition 5). For this graph, the most common assumption is that there is a constant $M$ such that the union graph associated with all time intervals $[n,n+M]$ is strongly connected [32, 1, 34, 15]. A network with this property is typically called uniformly strongly connected [21], $M$-strongly connected [22, 28] or jointly strongly connected [33]. This model implies guaranteed periodic communication. Another common model is to assume a time-varying network graph whose expected union graph is strongly connected, where the events that describe the success of communication across network edges are independent across time [3, 16, 14, 25].

In ref. [16, 22, 32, 28, 34, 14, 3, 15] the objective is that agents come to a consensus on one global optimization variable to minimize the sum of real-valued functions, each of which associated with one of the local agents. Although such consensus-type problems might appear quite different to (1.1), it turns out that an algorithm for (1.1) can also find a solution for consensus problems after a minor reformulation at the cost of additional communication, which we discuss in [27]. In contrast to the consensus type problems, ref. [33, 1, 25] and this work consider distributed optimization problems where each agent has to select a local optimization variable, such that the combination of all local variables solves a global optimization problem.

Observe that literature exclusively considers network models that either guarantee periodic communication or require communication based on independent events. We believe that these are restrictive assumptions that do not represent real-world communication networks well. To close this gap, we present a less restrictive network model and verifiable network conditions that guarantee that an SGD algorithm finds a solution to problems of the form (1.1). We also show the aforementioned typical network assumptions from the literature are stronger versions of our new set of network assumptions (Assumptions 5 and 6). Our assumptions only require a stochastic form of strong connectivity and a dependency decay (mixing) property. To the best of our knowledge, ours is the first work that guarantees asymptotic convergence of a distributed optimization scheme under such mildly restrictive conditions, connecting an abstract optimization theory with a wide range verifiable network conditions. However, it must be noted that other papers (such as those discussed above) can provide rate of convergence results while we merely give an almost sure convergence analysis.

Our work contributes to the literature of network conditions that guarantee asymptotic convergence to the set of stationary points of a distributed stochastic optimization problems with potentially non-convex objective function. Our work, builds on our previous work on SGD for time-varying networks [25]. However, whereas in [25] the focus was on the optimization iteration, with a strong and restrictive i.i.d. network assumption, this work focuses on guaranteeing convergence under significantly weaker network conditions. Most importantly, our network conditions cover time-varying network topologies, unbounded communication delays, non-independent aperiodic communication, asynchronous local updates and event-driven communication.

As the first step, we describe a distributed stochastic gradient descend algorithm (Algorithm 1) that instead of true local variables uses aged variables as a consequence of network communication. The AoI variables $\tau_{ij}(n)$ therefore induce gradient errors when comparing Algorithm 1 with and without AoI. As our first major contribution, we show in Lemma 2 that the aforementioned gradient errors vanish asymptotically under an asymptotic growth conditions for the AoI variables. Specifically, we require that all $\tau_{ij}(n)$ for all $n\in\mathbb{N}_{0}$ are stochastically dominated by a non-negative integer-valued random variable with at least finite first moment. This provides a significant weakening of traditional assumptions from the stochastic approximation literature in the present setting, since traditionally a dominating random variable with at least a bounded moment greater than one was required. With Lemma 2 we then show the convergence of Algorithm 1 in Theorem 1.

Our second contribution is a universally applicable time-varying network model and associated assumptions to generally verify the existence of dominating random variables with an arbitrary required moment conditions. Our time-varying network model is formulated using events $A^{n}_{ij}$, each of which represents successful information exchange from some agent $i$ to another agent $j$ during some time slot $n$. We then introduce the notion of $(\varepsilon,\kappa)$-stochastically strongly connected (SSC) network with $\varepsilon\in(0,1)$ and $\kappa\in\mathbb{N}_{0}$. This notion requires that there is a set of network edges that form a strongly connected graph for which $\mathbb{P}\left(\cup_{n}^{n+\kappa}A_{ij}^{n})\right)\geq\varepsilon$ for all $n\in N_{0}$. In other words, for those edges communication is successful at least ones over every interval of length $\kappa$ with at least probability $\varepsilon$. We then present a general recipe to validate stochastic dominance properties with required moment conditions. Afterwards, Theorem 2 presents our main result: Fix any $p\geq 0$ and consider a $(\varepsilon,\kappa)$-SSC network. If there exists some $\eta\in\mathbb{N}_{0}$, such that the processes $\mathbbm{1}_{\bigcup_{k=n}^{n+\eta}A^{n}_{ij}}$ are $\alpha$-mixing with $\sum_{n=0}^{\infty}n^{p-1}\alpha(n)<\infty$, then all AoI variables $\tau_{ij}(n)$ are stochastically dominated by non-negative integer-valued random variable $\overline{\tau}$ with $\mathbb{E}\left[\overline{\tau}^{p}\right]<\infty$. This result, together with Theorem 1, will therefore imply our final convergence result for Algorithm 1 under the minimal requirement of a SSC network with summable $\alpha$-mixing coefficients.

The rest of the paper is structured as follows: In Section 2 we state notation and preliminaries from probability and graph theory. In Section 3 we discuss the problem formulation and our distributed SGD algorithm. Afterwards we prove the almost sure convergence of Algorithm 1 in Section 4 under asymptotic growth conditions for the AoI variables. We then introduce our time-varying network model and associated assumptions in Section 5. Section 6 then presents our construction to validate stochastic dominance properties and our main results. Finally, we discuss the verifiability of our network assumptions and future work in Section 7.

We consider that the agents iteratively refine their local variables using the partial derivatives $\nabla_{x_{j}}f(\cdot,\xi)$. We assume that the agents do not know the distribution of $\xi$, but during any time slot $n$ an agent can observe an i.i.d. realisation $\xi^{n}$ of $\xi$. For simplicity, we assume that all agents are affected by the same realisation of the random variable $\xi$. In other words, when agent $i$ and agent $j$ calculate their partial derivatives during some time slot $n$, they use the same realisation $\xi^{n}$ of $\xi$, i.e. $\nabla_{x_{i}}f(\cdotp,\xi^{n})$ and $\nabla_{x_{j}}f(\cdotp,\xi^{n})$. The extension to agent-specific realisations of $\xi$ is merely a technical reformulation that was already described in [24].

To evaluate the partial derivatives $\nabla_{x_{i}}f(\cdotp,\xi^{n})$, agent $i$ uses the most recent available version of the optimization variable $x^{n}_{j}$ of agent $j$ for all $j\not=i$, i.e. it calculates $\nabla_{x_{i}}f(\hat{X}^{n}_{i},\xi^{n})$ instead of $\nabla{x_{i}}f(x^{n},\xi^{n})$. The following SGD iteration is used by each agent to update its local variable:

where $\{a(n)\}_{n\in\mathbb{N}_{0}}$ is a given step-size sequence and $\lambda_{i}^{n}$ is a local stochastic additive error that may arise during the calculation of $\nabla_{x_{i}}f$. Algorithm 1 summarizes the protocol that runs on every agent locally. For now, we assume that the agents use some communication protocol to exchange their local believe vectors $\hat{X}^{n}_{i}$ over a network. The protocol and the network properties therefore induce the distribution of the AoI variables $\tau_{ij}(n)$. In the next section, we will proof the convergence of Algorithm 1 under an abstract growth conditions for the AoI variables. Section 5 then formulates a communication network model and associated assumptions to satisfy these growth conditions.

Recall that Algorithm 1 requires that the agents exchange their local variables $x_{i}^{n}$ over a network. The network and an associated communication protocol should allow that local variables $x_{i}^{n}$ can frequently spread across the network and reach every agent. We will now introduce a network model were the agents try to exchange their local belief vectors $\hat{X}_{i}^{n}$. The agents therefore try to share their latest available version of all other agents local variable with other agents. This might potentially flood the network with data, however, there well known protocols to reduce the number of possibly redundant transmissions [17].

We assume a time-varying network (Definition 5)

which is a sequence of directed graphs. Each agent is in one-to-one correspondence with one node in the graph. For every time step $n\in\mathbb{N}_{0}$, an edge $(i,j)\in\mathcal{E}^{n}$ represents the event that agent $i$ successfully exchanges its local believe vector $\hat{X}_{i}^{n}$ during time slot $n$ with agent $j$. We denote this event by $A^{n}_{ij}$. Therefore, the sequence of directed graphs and the sequences of events $\{A^{n}_{ij}\}_{n\in\mathbb{N}_{0}}$ are in one to one correspondence: An edge $(i,j)\in\mathcal{E}^{n}$ if and only if the event $A^{n}_{ij}$ occurs. An edge therefore does not represent the possibility for communication, but the actual event of communication.

One may add additional complexity to the model, e.g. using a graph that represents the possibility for communication. Additionally, the model may be extended to scenarios where multiple successive events $A_{ij}^{n}$ need to occur to guarantee the exchange of a single realization of a believe vector $\hat{X}^{n}_{i}$. This might be necessary if the dimension of $\hat{X}^{n}_{i}$ is very large and/or the network bandwidth is small.

Note that although we defined the events $A^{n}_{ij}$ for all $(i,j)\in\mathcal{V}\times\mathcal{V}$, some of those events might never occur over the whole time horizon. We will especially do not require that all agents communicate directly! However, at least some of the events $A^{n}_{ij}$ should occur “frequently” enough such that the time-varying network satisfies certain connectivity properties. This will be formulated in Section 5.2 with Assumption 5.

The formulation of the time-varying communication network using the edge events $A^{n}_{ij}$ has several advantages. The model allows for an underlying time-varying graph that may be the consequence of an network scheduling algorithm or the physical dynamics of the agents themselves. Each event $A_{ij}^{n}$ can be represented as a multistage process. For example, (i) the availability of a channel, (ii) the use of an access protocol given the availability of a channel, (iii) the success of the transmission given the successful channel access. In general, the event-based formulation appears to be very convenient for analysis.

In the next two subsection, we will formulate our assumptions for the time-varying network $\{(\mathcal{V},\mathcal{E}^{n})\}_{n\in\mathbb{N}_{0}}$ using the events $A_{ij}^{n}$.

The following assumption formalizes our required network connectivity property.

Using the events $A^{n}_{ij}$, a $(\varepsilon,\kappa)$-SSC network requires there exists a strongly connected graph $(\mathcal{V},\mathcal{E})$, such that for all $n\in\mathbb{N}_{0}$ and for all $(i,j)\in\mathcal{E}$, we have

A $(\varepsilon,\kappa)$-SSC network therefore requires that there are some pairs of agents $(i,j)\in\mathcal{E}$ that can communicate directly at least ones in every time-interval of the form $[n,n+\kappa]$ with positive probability $\varepsilon$. Notice that SSC does not require direct communication between every pair of agents. The only agents that do communicate are those given in the set $\mathcal{E}$. A SSC network reflects our intuition of a non-degenerate communication network. Some agents can “frequently” exchange information with positive probability and information can spread across the network since $\mathcal{E}$ is strongly connected.

Note that a network that is SSC does not imply guaranteed transmissions periodically. We will see shortly that SSC is significantly weaker that plain guaranteed periodic communication. With stochastic strong connectivity we can not draw any conclusions about the dependency of events in the network. On the other hand, assuming guaranteed periodic communication does imply a strong form of dependency decay as shown in Section 5.4. Recall that our objective is to verify Assumption 1.1. However, using SSC alone is not sufficient to even guarantee the existence of a dominating random variable as required in Assumption 1.1. The next subsection therefore formulates dependency decay conditions using strong mixing (Definition 2).

Recall that our time-varying network is given by a sequence of directed graphs $\{(\mathcal{V},\mathcal{E}^{n})\}_{n\in\mathbb{N}_{0}}$. The sequence is in one-to-one correspondences with events $A^{n}_{ij}$ that represent the presence of an edge at time $n$. We will now formulate a dependency decay assumption based on the notion of strongly mixing processes. We can then show that the AoI variables $\tau_{ij}(n)$ associated with a $(\varepsilon,\kappa)$-SSC network satisfies specific moment conditions depending on the assumed rate at which dependency decays in the network.

With this assumption we do not require that the dependency of subsequent events $A_{ij}^{n}$ does decay at any specific rate. However, there should be an interval size $\eta>0$, such that the dependency of subsequent union events $\bigcup_{k=n}^{n+\eta}A^{n}_{ij}$ decays sufficiently fast. Notice that Assumption 6 is a dependency decay assumption for the network processes $\mathbbm{1}_{\bigcup_{k=n}^{n+N}A^{n}_{ij}}$ associated with all network edges $(i,j)\in\mathcal{V}$. However, we actually only require the assumption for those edges $(i,j)\in\mathcal{E}$ in an edge set $\mathcal{E}$ according to Assumption 5. Additionally, notice that we do not require any form of independence or dependency decay between transmissions over different edges. The reason for this is Lemma 5. The lemma will show that the existence of a dominating random variable for the AoI variables is in a natural way a transitive property of the network.

In this work we don’t give a recipe to verify Assumption 6. However, we will see in the next subsection that the standard assumptions in the distributed optimization literature all imply Assumption 6. Another set of examples where Assumption 6 is also directly satisfied are scenarios where the network events $A^{n}_{ij}$ are driven by a geometrically ergodic Markov process [10, 8]. Of course, it can be comparatively difficult to verify this in practice. However, traditionally and also more recently it has been quite common to model network fading channels by finite Markov chains [31, 4, 23, 19, 5]. We further discuss the verifiability of Assumption 6 in Section 7.

In this subsection we show that the typical network assumptions in the literature imply Assumptions 5 and 6.

First, consider the network models in [25, 3, 16, 14]. It is easy to check that network models imply the following properties:

1. 1.

   There is a strongly connected graph $(\mathcal{V},\mathcal{E})$ and some $\varepsilon>0$, such that $\mathbb{P}\left(A_{ij}^{n}\right)>\varepsilon$ for all $n\in\mathbb{N}_{0}$ and for all $(i,j)\in\mathcal{E}$.

2. 2.

   The events $A_{ij}^{n}$ are independent for different time-steps or different edges.

Independence is particularly unrealistic for wireless communication systems, since transmission that occur close in time, space, frequency or code can be highly correlated. Notably, this assumptions do not show any trade off between the choice of the step size sequence $a(n)$ and some network related property. Hence, there is no trade of between the growth of the AoI variables and the choice of the step size sequence. In fact, it is easy to show that under this assumptions all moments of all $\tau_{ij}(n)$ are bounded, see Section 6.1 Example 1.

We can now show that the above properties imply Assumptions 5 and 6. Assumption 5 is directly satisfied for $\kappa=1$. Define the $\sigma$-algebras

Then Assumption 6 holds trivially, since the independence of the events $A_{ij}^{n}$ implies that

for $A\in\mathcal{F}_{0}^{l}$ and $B\in\mathcal{F}_{l+n}^{\infty}$ for all $l,n\in\mathbb{N}$. Hence, the mixing coefficients $\alpha_{ij}(n)$ for each process $A_{ij}^{n}$ satisfies $\alpha_{ij}(n)=0$ for every $n\geq 0$.

Second, consider the time-varying network in [33, 22, 32, 28, 1, 34, 15]. The authors assume that their network is $M$-strongly connected. Hence, they assume guaranteed periodic communication. Assumption 5 is therefore directly satisfied by choosing $\kappa=M$. Then $\mathbb{P}\left(\bigcup_{k=n}^{n+\kappa}A^{k}_{ij}\right)=1$ for all $n\in\mathbb{N}_{0}$. Assumption 6 is also directly satisfied by choosing $\eta=M$. To see this, fix any $n,m\geq 0$ with $m\not=n$. Then

since the intersection of almost sure events is an almost sure event. Therefore,

and Assumption 6 follows.

We have therefore shown that the network models in the literature satisfy Assumptions 5 and 6. Moreover, Assumptions 5 and 6 are significantly weaker, since they do not require independent communication or guaranteed periodic communication, but merely asymptotic independence.

We now describe a general construction to establish the stochastic dominance properties with some finite $p$-th moment (Definition 7) for an AoI variable $\tau_{ij}(n)$. The idea is to find a uniform upper bound $u:\mathbb{N}_{0}\to\mathbb{R}_{\geq 0}$, such that

for all $m\geq N$ independent of $n\in\mathbb{N}_{0}$ for some $N\in\mathbb{N}_{0}$ and $\lim\limits_{m\to\infty}u(m)=0$. We can now use this bound to define the CDF of a new random variable. Since $\lim\limits_{m\to\infty}u(m)=0$ there is some $M\in\mathbb{N}_{0}$, such that $u(m)\leq 1$ for all $m\geq M\geq N$. Now define a non-negative integer-valued random variable $\overline{\tau}_{ij}$ by describing its CDF (more precisely its complementary CDF) as follows:

By definition $\overline{\tau}_{ij}$ stochastically dominates all $\tau_{ij}(n)$ for all $n\in\mathbb{N}_{0}$. Moreover, if

for some $p>0$, then it will follow from Proposition 1 that $\tau_{ij}(n)$ is stochastically dominated with finite $p$-th moment.

As the next step, we describe how we can find a function $u(m)$ for the above construction. Consider a $(\varepsilon,\kappa)$-SSC network. Let $(\mathcal{V},\mathcal{E})$ be the strongly connected graph associated with the $(\varepsilon,\kappa)$-SSC network and fix an edge $(i,j)\in\mathcal{E}$. Let $\Delta(m)$ be an increasing sequence in $\mathbb{N}$, with $\lim\limits_{m\to\infty}\Delta(m)=\infty$. Now for each $n,m\in\mathbb{N}_{0}$ use this sequence to define time indices

as long as $n_{k}\leq n$. Let $L(m)$ be the number of constructed time indices and observe that

This follows since $\tau_{ij}(n)>m$ implies $\tau_{ij}(n_{k})>\Delta(m)$ for all $k\in\{1,\ldots,L(m)\}$ by the very construction of the time indices $n_{k}$. In general, we can now derive $u(m)$ as an upper bound to the right-hand side in (6.7), which we illustrate immediately for case of independent network communication, i.e. were the events $A_{ij}^{n}$ are independent. For the case of dependent network communication, this will be formulated in Lemma 4 in the next section.