Detection and Mitigation of
Byzantine Attacks in Distributed Training

All work in this area (including ours) assumes a reliable parameter server that possesses the global data set and can assign specific subsets of it to workers. Robust aggregation methods have also been proposed for federated learning [27, 28]; however, as we make no assumption of privacy, our work, as well as the methods we compare with do not apply to federated learning.

One category of defenses splits the data set into $K$ batches and assigns one to each worker with the ultimate goal of suitably aggregating the results from the workers. Early work in the area [12] established that no linear aggregation method (such as averaging) can be robust even to a single adversarial worker. This has inspired alternative methods collectively known as robust aggregation. Majority voting, geometric median, and squared-distance-based techniques fall into this category [8, 9, 10, 11, 12, 13].

One of the most popular robust aggregation techniques is known as mean-around-median or trimmed mean [11, 10]. It handles each dimension of the gradient separately and returns the average of a subset of the values that are closest to the median. Auror [25] is a variant of trimmed mean which partitions the values of each dimension into two clusters using k-means and discards the smaller cluster if the distance between the two exceeds a threshold; the values of the larger cluster are then averaged. signSGD in [26] transmits only the sign of the gradient vectors from the workers to the PS and exploits majority voting to decide the overall update; this practice reduces the communication time and denies any individual worker too much effect on the update.

Krum in [12] chooses a single honest worker for the next model update, discarding the data from the rest of them. The chosen gradient is the one closest to its $k\in\mathbb{N}$ nearest neighbors. In later work [24], the authors recognized that Krum may converge to an ineffectual model in the landscape of non-convex high dimensional problems, such as in neural networks. They showed that a large adversarial change to a single parameter with a minor impact on the $L^{p}$ norm can make the model ineffective. In the same work, they present an alternative defense called Bulyan to oppose such attacks. The algorithm works in two stages. In the first part, a selection set of potentially benign values is iteratively constructed. In the second part, a variant of trimmed mean is applied to the selection set. Nevertheless, if $K$ machines are used, Bulyan is designed to defend only up to $(K-3)/4$ fraction of corrupted workers.

Another category of defenses is based on redundancy and seeks resilience to Byzantines by replicating the gradient computations such that each of them is processed by more than one machine [15, 16, 17, 18]. Even though this approach requires more computation load, it comes with stronger guarantees of correcting the erroneous gradients. Existing redundancy-based techniques are sometimes combined with robust aggregation [16]. The main drawback of recent work in this category is that the training can be easily disrupted by a powerful, omniscient adversary that has full control of a subset of the nodes and can mount judicious attacks [14].

Redundancy-based method DRACO in [17] uses a simple Fractional Repetition Code (FRC) (that operates by grouping workers) and the cyclic repetition code introduced in [29, 30] to ensure robustness; majority voting and Fourier decoders try to alleviate the adversarial effects. Their work ensures exact recovery (as if the system had no adversaries) with $q$ Byzantine nodes, when each task is replicated $r\geq 2q+1$ times; the bound is information-theoretic minimum, and DRACO is not applicable if it is violated. Nonetheless, this requirement is very restrictive for the typical assumption that up to half of the workers can be Byzantine.

DETOX in [16] extends DRACO and uses a simple grouping strategy to assign the gradients. It performs multiple stages of aggregation to gradually filter the adversarial values. The first stage involves majority voting, while the following stages perform robust aggregation. Unlike DRACO, the authors do not seek exact recovery; hence the minimum requirement in $r$ is small. However, the theoretical resilience guarantees that DETOX provides depend heavily on a “random choice” of the adversarial workers. In fact, we have crafted simple attacks [14] to make this aggregator fail under a more careful choice of adversaries. Furthermore, their theoretical results hold when the fraction of Byzantines is less than $1/40$.

A third category focuses on ranking and/or detection [19, 17, 20]; the objective is to rank workers using a reputation score to identify suspicious machines and exclude them or give them lower weight in the model update. This is achieved by means of computing reputation scores for each machine or by using ideas from coding theory to assign tasks to workers (encoding) and to detect the adversaries (decoding). Zeno in [20] ranks each worker using a score that depends on the estimated loss and the magnitude of the update. Zeno requires strict assumptions on the smoothness of the loss function and the gradient estimates’ variance to tolerate an adversarial majority in the cluster. Similarly, ByGARS [19] computes reputation scores for the nodes based on an auxiliary data set; these scores are used to weigh the contribution of each gradient to the model update.

In this paper, we propose novel techniques which combine redundancy, detection, and robust aggregation for Byzantine resilience under a range of attack models and assumptions on the dataset/loss function.

Our first scheme Aspis is a subset-based assignment method for allocating tasks to workers in strong adversarial settings: up to $q$ omniscient, colluding adversaries that can change at each iteration. We also consider weaker attacks: adversaries chosen randomly with limited collusion abilities, changing only after a few iterations at a time. It is conceivable that Aspis should continue to perform well with weaker attacks. However, as discussed later (Section V-B), Aspis requires large batch sizes (for the mini-batch SGD). It is well-recognized that large batch sizes often cause performance degradation in training [31]. Accordingly, for this class of attacks, we present a different algorithm called Aspis+ that can work with much smaller batch sizes. Both Aspis and Aspis+ use combinatorial ideas to assign the tasks to the worker nodes. Our work builds on our initial work in [22] and makes the following contributions.

• •

  We demonstrate a worst-case upper bound (under any possible attack) on the fraction of corrupted gradients when Aspis is used. Even in this adverse scenario, our method enjoys a reduction in the fraction of corrupted gradients of more than 90% compared with DETOX [16]. A weaker variation of this attack is where the adversaries do not collude and act randomly. In this case, we demonstrate that the Aspis protocol allows for detecting all the adversaries. In both scenarios, we provide theoretical guarantees on the fraction of corrupted gradients.

• •

  In the setting where the dataset is distributed i.i.d. and the loss function is strongly convex and other technical conditions hold, we demonstrate a proof of convergence for Aspis. We demonstrate numerical results on the linear regression problem in this part; these show the advantage of Aspis over competing methods such as DETOX.

• •

  For weaker attacks (discussed above), our experimental results indicate that Aspis+ detects all adversaries within approximately 5 iterations.

• •

  We present top-1 classification accuracy experiments on the CIFAR-10 [32] data set for various gradient distortion attacks coupled with choice/behavior patterns of the adversarial nodes. Under the most sophisticated distortion methods [23], the performance gap between Aspis/Aspis+ and other state-of-the-art methods is substantial, e.g., for Aspis it is 43% in the strong scenario (cf. Figure 7(a)), and for Aspis+ 19% in the weak scenario (cf. Figure 13).

Let $\mathcal{U}$ be the set of workers. Our scheme has $|\mathcal{U}|\leq f$ (i.e., fewer workers than files). Our assignment of files to worker nodes is specified by a bipartite graph $\mathbf{G}_{task}$ where the left vertices correspond to the workers, and the right vertices correspond to the files. An edge in $\mathbf{G}_{task}$ between worker $U_{i}$ and a file $B_{t,j}$ indicates that the $U_{i}$ is responsible for processing file $B_{t,j}$.

We first consider a weak attack, denoted ATT-1, where the Byzantine nodes operate independently (i.e., do not collude) and attempt to distort the gradient on any file they participate in. For instance, a node may try to return arbitrary gradients on all its assigned files. For this attack, the identity of the workers may be arbitrary at each iteration as long as there are at most $q$ of them.

Our second scenario, named ATT-2, is the strongest one we consider. We assume that the adversaries have full knowledge of the task assignment at each iteration and the detection strategies employed by the PS. The adversaries can collude in the “best” possible way to corrupt as many gradients as possible. Moreover, the set of adversaries can also change from iteration to iteration as long as there are at most $q$ of them.

This attack is similar to ATT-1 and will be called ATT-3. On the one hand, it is weaker in the sense that the set of Byzantines (denoted $A$) does not change in every iteration. Instead, we will assume that there is a “Byzantine window” of $T_{b}$ iterations in which the set $A$ remains fixed. Also, the set $A$ will be a randomly chosen set of $q$ workers from $\mathcal{U}$, i.e., it will not be chosen systematically. A new set will be chosen at random at all iterations $t$, where $t\equiv 0$ (mod $T_{b}$). Conversely, it is stronger than ATT-1 since we allow for limited collusion amongst the adversarial nodes. In particular, the Byzantines simulated by ATT-3 will distort only the files for which a Byzantine majority exists.

For each of the attacks considered above, the adversaries can distort the gradient in specific ways. Several such techniques have been considered in the literature and our numerical experiments use these methods for comparing different methods. For instance, ALIE [23] involves communication among the Byzantines in which they jointly estimate the mean $\mu_{i}$ and standard deviation $\sigma_{i}$ of the batch’s gradient for each dimension $i$ and subsequently use them to construct a distorted gradient that attempts to distort the median of the results. Another powerful attack is Fall of Empires (FoE) [34] which performs “inner product manipulation” to make the inner product between the true gradient and the robust estimator to be negative even when their distance is upper bounded by a small value. Reversed gradient distortion returns $-c\boldsymbol{\mathbf{g}}$ for $c>0$, to the PS instead of the true gradient $\boldsymbol{\mathbf{g}}$. The constant attack involves the Byzantine workers sending a constant gradient with all elements equal to a fixed value. To our best knowledge, the ALIE algorithm is the most sophisticated attack in literature for deep learning techniques.

In what follows, we suppress the iteration index $t$ since the Aspis algorithm is the same for each iteration. For the Aspis task assignment (cf. Section III-A1), any two workers, $U_{j_{1}}$ and $U_{j_{2}}$, have ${{K-2}\choose{r-2}}$ common files.

Let us index the $q$ adversaries in $A=\{A_{1},A_{2},\dots,A_{q}\}$ and the honest workers in $H$. We say that two workers $U_{j_{1}}$ and $U_{j_{2}}$ disagree if there is no edge between them in $\mathbf{G}$. The non-existence of an edge between $U_{j_{1}}$ and $U_{j_{2}}$ only means that they disagree in at least one of the $\binom{K-2}{r-2}$ files that they jointly participate in. For corrupting the gradients, each adversary has to disagree on the computations with a subset of the honest workers. An adversary may also disagree with other adversaries.

A clique in an undirected graph is defined as a subset of vertices with an edge between any pair of them. A maximal clique is one that cannot be enlarged by adding additional vertices to it. A maximum clique is one such that there is no clique with more vertices in the given graph. We note that the set of honest workers $H$ will pair-wise agree on all common tasks. Thus, $H$ forms a clique (of size $K-q$) within $\mathbf{G}$. The clique containing the honest workers may not be maximal. However, it will have a size of at least $K-q$. Let the maximum clique on $\mathbf{G}$ be $M_{\mathbf{G}}$. Any worker $U_{j}$ with $\deg(U_{j})<K-q-1$ will not belong to a maximum clique and can right away be eliminated as a “detected” adversary.

The essential idea of our detection is to run a clique-finding algorithm on $\mathbf{G}$ (summarized in Algorithm 1). The detection may be successful or unsuccessful depending on which attack is used; we discuss this in more detail shortly.

We note that clique-finding is well-known to be an NP-complete problem [35]. Nevertheless, there are fast, practical algorithms with excellent performance on graphs even up to hundreds of nodes [36, 37]. Specifically, the authors of [37] have shown that their proposed algorithm, which enumerates all maximal cliques, has similar complexity as other methods [38, 39], which are used to find a single maximum clique. We utilize this algorithm. Our extensive experimental evidence suggests that clique-finding is not a computation bottleneck for the size and structure of the graphs that Aspis uses. We have experimented with clique-finding on a graph of $K=100$ workers and $r=5$ for different values of $q$; in all cases, enumerating all maximal cliques took no more than 15 milliseconds. These experiments and the asymptotic complexity of the entire protocol are addressed in Supplement Section XI-A.

During aggregation (see Algorithm 2), the PS will perform a majority vote across the computations of each file (implementation details in Supplement Section XI-B). Recall that $r$ workers have processed each file. For each such file $B_{t,i}$, the PS decides a majority value $\mathbf{m}_{i}$

Assume that $r$ is odd and let $r^{\prime}=\frac{r+1}{2}$. Under the rule in Eq. (6), the gradient on a file is distorted only if at least $r^{\prime}$ of the computations are performed by Byzantines. Following the majority vote, we will further filter the gradients using a robust estimator $\widehat{\mathrm{med}}$ (see Algorithm 2, line 25). This robust estimator is either the coordinate-wise median or the geometric median; a similar setup was considered in [14, 16]. For example, in Figure 1, all returned values for the red file will be evaluated by a majority vote function on the PS, which decides a single output value; a similar voting is done for the other 3 files. After the voting process, Aspis applies the robust estimator $\widehat{\mathrm{med}}$ on the “winning” gradients $\mathbf{m}_{i}$, $i=0,1,\dots,f-1$.

Our motivation for proposing Aspis+ originates in the limitations of the subset assignment of Aspis. It is evident from the experimental results in Section VIII-B that Aspis is more suitable to worst-case attacks where the adversaries collude and distort the maximum number of tasks in an undetected fashion; in this case, the accuracy gap between Aspis and prior methods is maximal. Aspis does not perform as well under weaker attacks such as the reversed gradient attack (cf. Figures 8(a), 8(b), 8(c) even though it achieves a much smaller distortion fraction $\epsilon$, as discussed in Section VII. This can be attributed to the fact that the number of tasks is $K\choose r$ and even for the considered cluster of $K=15$, $r=3$, it would require splitting the batch into 455 files; hence, the batch size must be a multiple of 455. There is significant evidence that large batch sizes can hurt generalization and make the model converge slowly [31, 40, 41]. Some workarounds have been proposed to solve this problem. For instance, the work of [41] uses layer-wise adaptive rate scaling to update each layer using a different learning rate. The authors of [42] perform implicit regularization using warmup and cosine annealing to tune the learning rate as well as gradient clipping. However, these methods require training for a significantly larger number of epochs. For the above reasons, we have extended our work and proposed Aspis+ to handle weaker Byzantine failures (cf. ATT-3) while requiring a much smaller batch size.

The principal intuition of the Aspis+ detection approach (used for ATT-3) is to iteratively keep refining the graph $\mathbf{G}$ in which the edges encode the agreements of workers during consecutive and non-overlapping windows of $T_{d}$ iterations. At the beginning of each such window, the PS will reset $\mathbf{G}$ to be a complete graph, i.e., as if all workers pairwise agree with other. Then, it will gradually remove edges from $\mathbf{G}$ as disagreements between the workers are observed; hence, the graph will be updated at each of the $T_{d}$ iterations of the window, and the PS will assume that the Byzantine set does not change within a detection window. In practice, as we do not know the “Byzantine window,” we will not assume an alignment between the two kinds of windows, and we will set $T_{d}\neq T_{b}$ for our experiments. The detection method will be the same for all detection windows; thus, we will analyze the process in one window of $T_{d}$ steps.

For a detection window, let us encode the agreement of workers $U_{j_{1}},U_{j_{2}}$ on common file $i$ during the current iteration $t$ of the window $t=1,2,\dots,T_{d}$ as

Across all files, the total number of agreements between a pair of workers $U_{j_{1}},U_{j_{2}}$ during the $t^{\mathrm{th}}$ iteration is denoted by

Assume that the current iteration of the window is indexed with $t^{\prime}\in\{1,2,\dots,T_{d}\}$. The PS will collect all agreements for each pair of workers $U_{j_{1}},U_{j_{2}}$ up until the current iteration as

Since the placement is known, the PS can always perform the above computation. Next, it will examine the agreements and update $\mathbf{G}$ as necessary.

Based on the task placement (cf. Section III-A2), an edge $(U_{j_{1}},U_{j_{2}})$ exists in $\mathbf{G}$ only if the computed gradients of $U_{j_{1}}$ and $U_{j_{2}}$ match in all their $\lambda$ common groups in all iterations up to the current one indexed with $t^{\prime}$, i.e., a pair $U_{j_{1}},U_{j_{2}}$ needs to have $\alpha^{(j_{1},j_{2})}=\lambda\times t^{\prime}$ for an edge $(U_{j_{1}},U_{j_{2}})$ to be in $\mathbf{G}$. If this is not the case, the edge $(U_{j_{1}},U_{j_{2}})$ will be removed from $\mathbf{G}$. After all such edges are examined, detection is done using degree counting. Given that there are $q$ Byzantines in the cluster, after examining all pairs of workers and determining the form of $\mathbf{G}$, a worker $U_{j}$ will be flagged as Byzantine if $deg(U_{j})<K-q-1$. Based on Eq. (10), it is not hard to see that such workers can be eliminated and their gradients will not be considered again until the last iteration of the current window. The only exception to this is if the Byzantine set changes before the end of the current detection window. This is possible due to a potential misalignment between the “Byzantine window” and the detection window (recall that $T_{d}\neq T_{b}$ is assumed to avoid trivialities). In this case, more than $q$ workers may be detected as Byzantines; the PS will, by convention, choose $\hat{A}$ to be the most recently detected Byzantines. Algorithm 3 discusses the detection protocol. Following detection, the PS will act as follows. If at least one Byzantine has been detected, it will ignore the votes of detected Byzantines, and for each group, if there is at least one “honest” vote, it will use this as the output of the majority voting group; also, if a group consists merely of detected Byzantines, it will completely ignore the group. The remaining groups will go through robust aggregation (as in Section V-A). In our experiments in Section VIII-C, all Byzantines are detected successfully in at most 5 iterations. Example 3 showcases the utility of permutations in our detection algorithm using $K=7$ workers.

We use the GD algorithm (1) with the initial randomly chosen parameter vector $\mathbf{w_{0}}\sim\mathcal{N}(\mathbf{0}_{d},I_{d})$. We partition the data matrix $X$ row-wise into $f$ submatrices $X_{1},X_{2},\dots,X_{f}$, and correspondingly the label vector $\mathbf{y}$ into $f$ sub-vectors $\textbf{y}_{1},\textbf{y}_{2},\dots,\textbf{y}_{f}$, where $f$ is the number of files of the distributed algorithm. A file $B_{t,i}$ consists of a pair $(X_{i},\textbf{y}_{i})$. For each of its assigned files $B_{t_{i}}=(X_{i},\textbf{y}_{i})\in\mathcal{N}^{w}(U_{i})$, worker $U_{i}$ either computes the honest partial gradient or a distorted value and returns it to the PS. Using the formulation of Section IV, the gradient in Eq. (3) for linear regression is the product $\textbf{g}_{t,i}=X_{i}^{T}X_{i}\textbf{w}-X_{i}^{T}\textbf{y}_{i}$.

Metrics: For each scheme and value of $q$ we run multiple Carlo simulations, and calculated the average least-square loss that each algorithm converges to across the Monte Carlo simulations. For each simulation we declare convergence if the final empirical loss is less than 0.1 We record the fraction of experiments that converged and the rate of convergence. In computing the average loss, the experiments that did not converged are not taken into account (for more details, please see Supplement Section XI-C).