PriRoAgg: Achieving Robust Model Aggregation with Minimum Privacy Leakage for Federated Learning

In a vanilla federated learning protocol, the early concept involves conducting model training locally to ensure the privacy of users’ datasets. [1, 40]. Training is concurrently performed across all the users through an iterative process in which the users interact with the center server to jointly update the global model. A standard FL pipeline operates as follows: at $t$-th iteration, the server shares the current state of the global model $\mathbf{w}^{(t)}$ with all the users. Upon receiving the global model, each user $i\in[N]$ trains a local model $\mathbf{w}_{i}^{(t)}$ on its dataset, where the corresponding local update is $\mathbf{x}_{i}^{(t)}\triangleq\mathbf{w}_{i}^{(t)}-\mathbf{w}^{(t)}$. This is typically done by employing a minibatch stochastic gradient descent rule [41] to minimize the local objective function $\mathcal{L}_{i}(\mathbf{w})$. Each user then uploads the local update $\mathbf{x}_{i}^{(t)}$ to the server. After receiving the model updates, the server aggregates them to update the global model as follows.

where $\eta^{(t)}$ is the learning rate. The updated global model $\mathbf{w}^{(t+1)}$ is distributed to all the users to start the next iteration.

Contradicting the early concept, studies have revealed that conventional federated learning poses risks of leaking users’ privacy. Works such as [2, 4, 42, 5] have shown that by launching model inversion attacks, it is possible to reveal information about the local training data of individual users from their local updates or gradients. To address this issue, secure aggregation is introduced to resolve the aggregation problem without leaking any information except for the final aggregation result [43, 30, 19, 20, 18, 21, 22, 23, 24].

The main approaches in secure aggregation include Secret sharing (SS) [32, 33, 30, 31], Differential privacy (DP) [43, 18, 21, 22] and Homomorphic encryption (HE) [25, 26, 27]. Secret sharing enables a user to split its local update (secret) into multiple secret shares, where the secret remains theoretically private as long as no entity holds more shares than a specified threshold. Shareholders (other users) can collectively aggregate these secret shares to compute the aggregated secret. The server collects the aggregation of shares and reconstructs the actual aggregation of local updates. The differential privacy technique is to add controlled noise or randomness to updates before aggregation. This ensures that the aggregated results do not disclose sensitive information about local training data. Homomorphic encryption is a powerful cryptographic technique that allows computations to be performed on encrypted data without decrypting it. The server can exploit such a feature to perform a secure aggregation and decrypt the real aggregation without privacy leakage.

Aside from concerns for privacy, the issue of model robustness has been pervasively discussed in FL. The defense strategies and attack methods have been competitively growing in recent years. Generally, there are two types of attacks of interest. Some malicious users can launch untargeted attack, during which they upload arbitrary false models to impede the server from completing the model aggregation correctly [44, 45, 5]. There are also omniscient untargeted attacks where the adversary is aware of other benign updates, so it can design the update to launch stronger attacks [46]. The other type is targeted attack, where the malicious users deliberately implant the global model with a backdoor without affecting the main learning task  [47, 48, 49, 50, 51, 52, 53].

To detect such fraudulent model updates, robust FL aggregation has been introduced by many studies. At the onset of the introduction of model poisoning, a few noise-filtering-based algorithms are put forward to select honest ones across users [9, 10, 11, 12, 13]. The statistical traits of local updates are exploited to capture the malformed ones and compute a final aggregation without them. However, malicious users can manipulate the updates to bypass the defense so that get selected for the final aggregation [54, 46]. Competitively, Some recent robust aggregation protocols against poisoning attacks have been proposed to address this issue [14, 15, 16, 17].

In addition, various backdoor defense methods have been proposed to counter deliberate manipulations on local updates, as evidenced by studies such as [55, 56, 57, 52, 58, 59, 60, 61, 62, 63, 59, 64]. In [62], the coordinate-wise median and trimmed mean techniques are employed to achieve promising performance in robust aggregation. Several studies also investigate the impact of the sign of updated gradients [63, 59, 64]. Particularly, [63] is the first to utilize majority voting on gradient signs during final aggregation to enhance robustness and convergence. Subsequent works investigate how to use the coordinate-wise sign information to guide the final aggregation; for instance, [59] uses the sign to adjust the learning rate in each round to defend against backdoor attacks, while SignGuard  [64] clusters the signs to filter malicious gradients.

Recent works attempt to simultaneously address both aspects of privacy and robustness [32, 43, 21, 33, 30, 65, 35, 31]. BREA [32] is the first FL protocol known to consider privacy-preserving and robustness simultaneously. It employs the statistical algorithm multi-Krum [14] to protect the final aggregation from Byzantine local updates, and uses secret sharing to hide the updates from the adversary. A state-of-the-art protocol, EIFFeL  [30], provides an exemplary solution that addresses robustness and privacy simultaneously. It significantly improves the privacy guarantees to the privacy using new cryptography preliminaries. Regarding robustness, EIFFeL shows that users can use any function to validate the integrity of their individual updates. EIFFeL does not require users to upload the local updates for integrity validation, instead, a zk-proof for correctly performing the evaluation on updates. RoFL [31] further extends the norm bounding validation in EIFFeL and demonstrates different norm constraints also mitigate the effect of backdoor attacks. It also boosts the efficiency of the protocol in addition. A recent work, ByITFL, proposes a theoretically secure protocol [28] that provides robustness in update aggregation. It builds upon the Byzantine-resilient protocol FLTrust [66] to further provide privacy, especially in scenarios involving colluding users alongside a curious server. Other protocols have also been proposed to offer privacy and robustness by exploiting multiple-server architectures, such as [34, 35, 36].

Current robust and secure protocols face limitations in either privacy-preserving or robust aggregation performance. The primary issue of BREA [32] is the incomplete privacy left with security breaches. The multi-Krum algorithm [14] used in the protocol reveals the pairwise distances between the local updates to the server. Such pairwise distances potentially provide information to a curious server to reconstruct users’ local updates. On the other hand, for works that provide provable privacy, the defensive measurements are insufficient to advance adversary attacks. EIFFeL [30] and RoFL [31] perform the validation function on user updates individually; hence, their defense cannot leverage the overall statistical information across users. For example, malicious users can easily bypass detection by keeping their updates below the norm threshold while still implanting a backdoor.

For general robust aggregations, cryptography techniques mentioned in Section II-B have limitations in different aspects. Though HE provably addresses privacy concerns, a critical limitation is its impractical complexity [25, 26, 27, 34, 28]. In practice, the computational cost is $10^{3}$ to $10^{4}$ times higher than the non-encrypted vanilla FL, even when the models used are as simple as logistic regression and multi-layer perceptron. Hence, many HE protocols suffer from high complexity and become highly impractical when sophisticated models are involved. DP alleviates the high complexity associated with HE, providing better efficiency along with provably secure solution [43, 18, 21, 22]. However, a major issue with DP is that its random masking hinders the detection of malicious updates. A malicious update potentially becomes indistinguishable to many robust algorithms by masquerading as a benign one using a masked update. How to evaluate and provide robustness against deliberately manipulated updates still remains an open problem in DP [29, 22].

To overcome these limitations, we are motivated to introduce a framework for general robust algorithms while minimizing information leakage with provable security. Specifically, we consider the strongest threat model used in single-server FL protocols, where Byzantine users can collude with the server to compromise the privacy of the victim users. Under this threat model, for a general class of robust algorithms against various attacks, we envision that the best scenario of privacy guarantee is the same as the secure aggregation, where only the final model aggregation is revealed to the server. To this end, we ask: For the colluding threat model, what is the minimum amount of information that can be revealed, to accomplish a robust and secure aggregation protocol? To answer this question, we first characterize the minimum privacy leakage through formalizing a novel privacy notion called aggregated privacy, which allows nothing but the aggregation of certain statistics of users’ updates (including the aggregated update) to be revealed. Then, we develop PriRoAgg, a general privacy-preserving framework for robust aggregation that satisfies aggregated privacy.

We use the standard Key agreement (KA) and Authenticated encryption (AE) for user communications’ privacy and integrity. KA involves three algorithms and AE involves two algorithms.

• •

  $pp\leftarrow$ KA.param($\kappa$). It generates public parameters $pp$ from security parameter $\kappa$.

• •

  $[pk_{i},sk_{i}]\leftarrow$ KA.gen($pp$). The key generation algorithm creates a public-secret key pair.

• •

  $sk_{i,j}\leftarrow$ KA.agree($pk_{j},sk_{i}$). User $i$ combines two keys to generate a symmetric session key $sk_{i,j}$.

• •

  $c\leftarrow$AE.enc($k,m$). The encryption algorithm uses a message $m$ with key $k$ and returns the ciphertext $c$.

• •

  $m\leftarrow$AE.dec($k,c$). The decryption algorithm takes a ciphertext $c$ with key $k$ and returns the original text $m$, or error symbol $\perp$ when decryption fails.

The proposed protocol relies on a secret sharing scheme with encoding, which is known as Lagrange coded computation (LCC) [37]. We extend it to the verifiable LCC with constant-size commitment in the next subsection.

Initially, the server and all users agree on $K+T+N$ distinct elements $\{\alpha_{i},\beta_{k}:i\in[N],k\in[K+T]\}$ from $\mathbb{F}_{q}$ in advance. User $i$ holds a vector $\mathbf{x}_{i}\in\mathbb{F}_{q}^{d}$, where $d$ is the dimension of the vector. Then, user $i$ partitions its message ${\mathbf{x}}_{i}$ into $K$ equal-size pieces, labeled by ${\mathbf{x}}_{i,1},{\mathbf{x}}_{i,2},\ldots,{\mathbf{x}}_{i,K}\in\mathbb{F}_{q}^{\frac{d}{K}}$. To provide $T$-colluding privacy protection, the user $i$ independently and uniformly generates $T$ random noises $\mathbf{z}_{i,K+1},\ldots,\mathbf{z}_{i,K+T}$ from $\mathbb{F}_{q}^{\frac{d}{K}}$. These random noises are used for masking the partitioned pieces, which are achieved by creating a polynomial $f_{i}:\mathbb{F}_{q}^{\frac{d}{K}}\rightarrow\mathbb{F}_{q}^{\frac{d}{K}}$ of degree $K+T-1$ such that¹¹1Unless otherwise specified, all operations are taken modulo $q$.

The Lagrange interpolation rules and the degree restriction guarantee the existence and uniqueness of the polynomial $f_{i}(\alpha)$, which has the form of

Next, user $i$ evaluates $f_{i}(\alpha)$ at point $\alpha=\alpha_{j}$ to obtain an encoded piece for user $j$, by

We use $[\cdot]$ to distinguish the original data and its encoded shares ²²2When user indices appear on both sides of $[\cdots_{i}]_{j}$, it indicates the user $i$’s secret share sent to user $j$. We abuse $[\cdot]$ to present vectors without such indices.. Taking $[\mathbf{x}_{i}]_{j}$ in (8) for an example, $\mathbf{x}_{i}$ is the user $i$’s secret, and $j$ stands for the share holder’s index, therefore, $[\mathbf{x}_{i}]_{j}$ is what user $j$ holds for secret $\mathbf{x}_{i}$. Here, we allow $i=j$ for protocol simplicity. Finally, we wrap LCC as a primitive module that includes two algorithms:

• •

  $[[\mathbf{x}_{i}]_{1},\ldots,[\mathbf{x}_{i}]_{N}]\leftarrow$ LCC.share($\mathbf{x}_{i}$). Given the threshold $T$ and parameter $K$, where $T+K<N$, it generates shares for a secret vector $\mathbf{x}_{i}$. The algorithm partitions a vector into $K$ pieces, uses (7) to encode them, and then generates shares using (8).

• •

  $\mathbf{x}_{i}\leftarrow$ LCC.recon($[[\mathbf{x}_{i}]_{1},\ldots,[\mathbf{x}_{i}]_{N}]$). The LCC generates a $[N,K+T,N-K-T+1]$ Reed-Solomon code. It can be decoded by the Gao’s decoding algorithm [76].

We combine LCC and constant-size commitments  [33, 39] to leverage threshold privacy and encoding efficiency, which further reduces the communication overhead.

Let $g$ be a generator of the cyclic group $\mathbb{G}$ of order $\lambda$, where $\lambda$ should be some enough large prime such that $q$ divides $\lambda-1$. To verify that the piece $\mathbf{x}_{i,k}$ is correctly constructed by using the Lagrange polynomial from (7), user $i$ broadcasts the commitments $[{c}_{i,1},\ldots,{c}_{i,K+T}]$ to all other parties, by

where $\mathbf{x}_{i,k}[p]$ is the $p$-th entry of $\mathbf{x}_{i,k}$. A trusted third party generates $\gamma\in\mathbf{F}_{q}$ and computes $\mathbf{B}\triangleq\left[g^{\gamma^{0}},g^{\gamma^{1}},\ldots,g^{\gamma^{\frac{d}{K}-1}}\right]$, where $\mathbf{B}$ is public to all users.

Upon receiving the share (8) and the public commitments (9) from user $i$, the user $j$ can verify the share by checking

The equality ensures that share $[\mathbf{x}_{i}]_{j}$ is generated correctly by the polynomial in (7). Moreover, assuming the intractability of computing the discrete logarithm for all users and the server, i.e., they cannot compute the discrete logarithm $\log_{g}({c}_{i,k})$ for any $k\in[K]$, therefore, there is no information leakage about the user’s message. Following (9) and (10), we define two more algorithms for verifiability of LCC:

• •

  $[c_{i,1},\ldots,c_{i,K+T}]\leftarrow$ LCC.commit($\mathbf{x}_{i}$). Given $\mathbf{x}_{i}$ and LCC encoding as described in (7), the algorithm generates a constant-size vector commitment with dimension $K+T$, denoted as $\mathbf{c}_{\mathbf{x}_{i}}=[c_{i,1},\ldots,c_{i,K+T}]$ as in (9).

• •

  $\{\perp,1\}\leftarrow$ LCC.verify($[\mathbf{x}_{j}]_{i},\mathbf{c}_{\mathbf{x}_{j}}$). This algorithm allows user $i$ to verify whether the received share is consistent with the commitments opened by user $j$, as defined in (10). The function returns a Boolean value of $1$ to indicate success in verification, or the error symbol $\perp$ if verification fails.

We introduce the secret-shared non-interactive proof (SNIP) as a primitive [38]. In the FL setting, SNIP can be used to prove that an arithmetic circuit is evaluated on certain local data that is secret shared. SNIP is originally proposed in additive secret sharing, EIFFeL extends it to the Shamir secret sharing [30] and we further extend it to LCC for PriRoAgg.

SNIP follows the LCC module’s setting. Given an arithmetic circuit agreed by all parties, denoted by $f(\cdot)$, the goal is for a user (prover) to prove it has faithfully performed $f(\mathbf{x}_{i})=\mathbf{e}_{i}$ without revealing its inputs $\mathbf{x}_{i}$ and outputs $\mathbf{e}_{i}$, Without loss of generality, we take a user to act as the prover while all the other users act as verifiers. Assume that the arithmetic circuit of $f$ has $P$ multiplication gates, where each gate has two input wires and one output wire. The topological order of the multiplication gates is given by $[1,2,\ldots,P]$ and agreed upon by all users. The SNIP works in the following steps.

1. 1.

   A prover (user $i$) evaluates function $f$, $f(\mathbf{x}_{i})=\mathbf{e}_{i}$. For the circuit multiplication gates, denote two input wires for gates $1$ to $P$ as $\{u_{1}^{i},\ldots,u_{P}^{i}\}$ and $\{v_{1}^{i},\ldots,v_{P}^{i}\}$. Use each side of input wires to interpolate polynomials, resulting in two polynomials of degree-$(P-1)$, $f_{u}^{i}(t)$ and $f_{v}^{i}(t)$, such that $f_{u}^{i}(p)=u_{p}^{i},f_{v}^{i}(p)=v_{p}^{i},\forall p\in[P]$. The product of two polynomials $f_{h}^{i}(t)\triangleq f_{u}^{i}(t)\cdot f_{v}^{i}(t)$ is exactly the output wire of the $p$-th gate at the evaluation $t=p$. Denote the coefficients of polynomial $f_{h}^{i}(t)$ by $\mathbf{h}_{i}\triangleq[h_{1}^{i},\ldots,h_{2P-2}^{i}]$. The prover then generates secret shares for $\mathbf{h}_{i}$ by LCC.share($\mathbf{h}_{i}$) with $K=1$, which is equivalent to Shamir’s secret sharing, where shares are denoted by $[\mathbf{h}_{i}]_{j},\forall j\in[N]$.

2. 2.

   A verifier (user $j$) recovers the computation of the function by the shares and generates a share for the Schwartz-Zippel polynomial test  [77, 78]. Specifically, with $[\mathbf{h}_{i}]_{j}$, the verifier interpolates a polynomial up to the degree of $2P-2$, $[f_{h}^{i}]_{j}(t)$, which corresponds to all output wires. Thus, the share of the circuit output, $[\mathbf{e}_{i}]_{j}$, can be evaluated on the polynomial. Next, with $[\mathbf{x}_{i}]_{j}$ and $[f_{h}^{i}]_{j}(p),p\in[P]$, it can recover all intermediate results for input wires of all gates as additions and scalar multiplications can be directly applied. Since they are all computed from the share of polynomials, we denote the input wire values by $[u_{p}^{i}]_{j},[v_{p}^{i}]_{j}$. SNIP interpolates two polynomials for two ways of input wires from $1$ to $P$, $[f_{u}^{i}]_{j}(t)$, $[f_{v}^{i}]_{j}(t)$. Beaver’s protocol [79] is used to compute $[f_{u}^{i}(t)*f_{v}^{i}(t)]_{j}$ from $[f_{u}^{i}]_{j}(t)$ and $[f_{v}^{i}]_{j}(t)$. All verifiers agree on one or multiple random $r$ for the Schwartz-Zippel polynomial test. Recall that the output wire’s polynomial is given by $[f_{h}^{i}]_{j}(t)$. Given an $r$, a share of polynomial test is generated by computing $[\sigma_{i}]_{j}=[f_{u}^{i}(r)*f_{v}^{i}(r)]_{j}-[f_{h}^{i}(r)]_{j}$.

3. 3.

   The server collects verifiers’ $[\sigma_{i}]_{j}$ to reconstruct the result of the polynomial identity test. For a prover (user $i$), the server decodes $\sigma_{i}$ from $\{[\sigma_{i}]_{j}|{j}\in[N]$ by LCC.recon. If the user honestly performs the circuit, the reconstruction is $0$. Otherwise, user $i$ fails in the proof.

We summarize the described scheme as a module SNIP with two functions for prover and verifier correspondingly.

• •

  $\mathbf{h}_{i}\leftarrow$ SNIP.prove$\left(f(\mathbf{x}_{i})=\mathbf{e}_{i}\right)$. $f$ is the given arithmetic circuit a prover (user $i$) wants to prove with input $\mathbf{x}_{i}$ and output $\mathbf{e}_{i}$. ³³3The Beaver’s triplet is generated and broadcast by each user insetup phase. Algorithm output $\mathbf{h}_{i}$ is the coefficients given at the end of step 1.

• •

  $[\sigma_{j}]_{i},[\mathbf{e}_{j}]_{i}\leftarrow$ SNIP.recon($f,[\mathbf{x}_{j}]_{i},[\mathbf{h}_{j}]_{i}$). This algorithm constructs the share of the polynomial test and the share of the circuit output as in step 2.

PriRoAgg starts with a setup phase to prepare all parties with some public parameters, which includes the security parameter $\kappa$ for key initializations, the finite field $\mathbb{F}_{q}$ with sufficiently large $q$ and amplifier $p\in\mathbb{F}_{q}$ for update computations, threshold $T$ for the maximum number of malicious users, partitioning parameter $K$ for LCC, vector $\mathbf{B}$ for constant-size commitment scheme, a set of points $\{\alpha_{i},\beta_{k}\in\mathbb{F}_{q}|i\in[N],k\in[K+T]\}$ for LCC evaluations.

Leveraging the proposed framework PriRoAgg, we construct two concrete $SRA$ protocols for two specific robust aggregation algorithms. One is Robust Learning Rate (RLR) for backdoor attack, and the other is Robust Federated Aggregation (RFA) for model poisoning attack. For each robust aggregation algorithm, we first explain how the robust algorithms operate in plaintext, and then present how the robust algorithms apply to PriRoAgg to achieve the $SRA$ protocols. Additionally, a special feature of repeated pattern circuits is used in both instantiations to reduce SNIP’s computational overhead. Detail is provided in Appendix A.

Applying PriRoAgg with RLR, we let $\phi$ be:

In this case, the evaluation circuit to prove is simply $f_{\phi}(\mathbf{x}_{i})\triangleq sign(\mathbf{x}_{i})=\mathbf{e}_{i}$. As a result, the server reconstructs $\sum_{i\in[N]}\mathbf{e}_{i}$ and $\sum_{i\in[N]}\mathbf{x}_{i}$ in the last round, with no additional information revealed. Next, it recovers the binary mask $\mathbf{v}$ from $\sum_{i\in[N]}\mathbf{e}_{i}$ to correctly compute final robust aggregation as in RLR.

In RFA instantiation, we choose $\phi$ to be :

where $||\mathbf{x}_{i}||_{2}\neq 0$. Since the arithmetic circuit does not allow division, the circuit to be verified is designed as $f_{\phi}({\mathbf{x}}_{i})\triangleq(1-\omega_{i}||\mathbf{x}_{i}||_{2})^{2}=\mathbf{e}_{i}$, where $\mathbf{e}_{i}=0$. In round $2$, user $i$ computes $[\omega_{j}\mathbf{x}_{j}]_{i}$ from $[\omega_{j}]_{i}$ and $[\mathbf{x}_{j}]_{i}$ by the Beaver’s protocol [79]. At the end of round 2, user $i$ substitutes $\sum_{j\in[N]}[\mathbf{x}_{j}]_{i}$ by concatenating $\sum_{j\in[N]}[\omega_{j}\mathbf{x}_{j}]_{i}$ and $\sum_{j\in[N]}[\omega_{j}]_{i}$ as message $m_{4}$. In the final round, the server reconstructs and checks if $\sum_{i\in[N]}\mathbf{e}_{i}=0$. Together with the fact check of polynomial identity $\sigma_{i}=0$, the server ensures the circuit $f_{\phi}(\mathbf{x}_{i})$ is faithfully evaluated. For the final aggregation, the server reconstructs and decomposes $m_{4}$ to compute the weighted average $\frac{1}{\sum_{i\in[N]}\omega_{i}}\sum\limits_{i\in[N]}\omega_{i}\mathbf{x}_{i}$ as in RFA.

In this subsection, we provide the security proofs for the privacy guarantees of two protocols in PriRoAgg.

The computational overhead and communication overhead per iteration are summarized in Table I, and the comparison with EIFFeL is presented. Recall that there are $N$ users with up to $T$ malicious users; the update dimension is $d$; $K$ is the partitioning parameter for LCC satisfying $K+T<N-1$; the number of circuit $f_{\phi}$’s multiplication gates is $\mathcal{O}(d)$.

We choose $N=10$ for all Fashion-MNIST experiments and $N=40$ for all CIFAR-10 experiments, with $10\%-30\%$ malicious users. The security threshold is set to $T=\left\lceil 0.3N\right\rceil$ and $K=\left\lfloor 0.7N\right\rfloor-1$ to maximize efficiency while ensuring compliance with privacy guarantee. Each experiment is run for $10$ times and the average results are reported.

For backdoor attacks in (g)-(l) on Fig. 2, we evaluate the main task on the validation dataset and test the backdoor task on the poisoned validation dataset. As shown in (i) and (k), although RoFL and F&C manage to slow down the implantation process on Fashion-MNIST, both ultimately succumb to the backdoor. Compared to them, PriRoAgg with RLR performs significantly well against backdoor implantation on both datasets. It greatly supports the discussion in Section II-E about validation on the sole update not sufficiently supporting sophisticated model aggregation. The EIFFeL with cosine similarity has nearly no defensive capability against backdoor attacks, while RoFL with norm bounding performs slightly better. We consider multiple parameter settings for EPPRFL with F&C and find that F&C highly relies on its early-stage updates’ history; that is, if it cannot detect the malicious users in the first several rounds, it fails in the backdoor attack. Hence, F&C exhibits notable instability. As PriRoAgg allows the overall statistics of the updates, PriRoAgg with RLR robustly defends against the backdoor in both datasets, as well as achieving the best performance in the main task.

As shown in Fig. 3, the time against the number of users and the dimension of update follows the complexity analysis in Table  I. The time cost increases almost linearly as it is primarily dominated by variables $d$ and $N$. For time against $T$ with fixed $N$, the LCC parameter $K$ is chosen as the maximum, $N-T-1$, inducing a quadratic relation between time and $T$. In Fig.4, compared with EIFFeL, PriRoAgg benefits from higher encoding efficiency of LCC and repeated pattern circuit in SNIP. Consequently, PriRoAgg is superior in end-to-end efficiency, especially when $N$ is large since it allows for a larger $K$ for LCC. This advantage is particularly evident in Fig.4.