Privacy-Preserving Analytics on Decentralized Social Graphs: The Case of Eigendecomposition

Graphs can characterize the complex inter-dependency among entities, and are used in various applications, such as social networks [14] and webpage networks [20]. As one popular and fundamental graph analytics task, eigendecomposition works on the adjacency matrix to yield eigenvalues/eigenvectors, and can benefit various applications [19, 10, 9, 26, 20, 21, 22]. For example, eigendecomposition-based graph analytics can greatly benefit community detection through the following ways: 1) finding the community structure based on the eigenvectors [19]; 2) identifying and characterizing nodes importance to the community according to the relative change in the eigenvalues after removing them [10]; 3) partitioning a social graph based on its top-2 eigenvectors [9]. Another important application is PageRank [26, 20, 21, 22], which is one of the best-known ranking algorithms in web search. PageRank measures the importance of website pages by computing the principal eigenvector of the matrix describing the hyperlinks of the website pages. In addition, the second eigenvector can be used to detect a certain type of link spamming [23]. However, all of them consider the execution of eigendecomposition in the plaintext domain without privacy protection.

There exist a variety of designs that aim to securely perform certain graph analytics tasks. Some works [27, 28, 29] focus on privacy-preserving training of graph neural networks (GNN) based on the federated learning paradigm [30], which aim to train GNN models across multiple clients holding local datasets (e.g., spatio-temporal datasets [27] or graphs [28, 29]) in such a way that the datasets are kept local. The work [27] focuses on GNNs over decentralized spatio-temporal data, and has the clients exchange model updates with the server in cleartext. In contrast, the works [28, 29] focus on graph datasets and design privacy-preserving mechanisms to protect the individual model updates.

There has been a line of work [31, 32, 33, 34, 35] aimed at the support for graph analytics with cryptographic methods like secure multi-party computation techniques and searchable encryption. The main focus of this line of work has been on supporting different kinds of graph queries under different scenarios in a secure manner. Wu et al. [31] propose a protocol for privacy-preserving shorted path query in a two-party setting, where a client holding a query and a server holding a plaintext graph, based on cryptographic techniques including private information retrieval, garbled circuits, and oblivious transfer. In contrast, the works [34, 35] focus on the support for privacy-preserving shortest path queries in an outsourcing setting, where private queries need to be executed over encrypted graphs outsourced to the cloud. The work [32] designs protocols that can support privacy-preserving shortest distance queries and maximum flow queries over outsourced encrypted graphs. These works rely on the combination of searchable encryption (e.g., order-revealing encryption), homomorphic encryption, and/or garbled circuits. In [33], Araki et al. consider a scenario where all nodes and edges of a graph are secret-shared between three servers and devise protocols for breadth-first search and maximal independent set queries, based on secret sharing and secure shuffling. The above works all target graph analytics tasks that are substantially different from the one considered in this paper.

There is another line of work [11, 12] focuses on publishing graph matrices with differential privacy while preserving their eigenvalues/eigenvectors. They work under the setting of centralized social graphs where the social graph is held by a single entity and processed in the plaintext domain. Some works consider the scenario of decentralized social graphs, and focus on the privacy-preserving support for different tasks with differential privacy, such as estimating subgraph counts [3] and generating representative synthetic social graphs [4].

The state-of-the-art design that is most related to ours is PrivateGraph [8], which is also aimed at eigendecomposition analytics over decentralized social graphs with privacy protection. However, as mentioned above, PrivateGraph is subject to several crucial downsides in terms of functionality and security, which greatly limit its practical usability. In light of this gap, we present a new system design PrivGED for privacy-preserving eigendecomposition analytics on decentralized social graphs. Compared to PrivateGraph, PrivGED is much advantageous in that it (i) supports both directed and undirected social graphs (via secure realizations of both the Arnoldi method and the Lanczos method), (ii) does not reveal any users’ exact degree information, and (iii) fully exploits the cloud to free the analyst from staying online for active and frequent interactions and conducting a large amount of local intermediate and post processing. In particular, as reported in [8], to obtain the eigenvalues/eigenvectors, the analyst must spend 0.2 hours as well as communicate 10 GB with the cloud. In contrast, PrivGED allows the analyst to directly receive the final eigenvalues/eigenvectors. Table I summarizes the prominent advantages of our PrivGED over PrivateGraph.

A graph comprises a set of nodes with a corresponding set of edges which connect the nodes. The edges may be directed or undirected and may have weights associated with them as well. Eigendecomposition works on the adjacency matrix associated with a graph to yield eigenvalues/eigenvectors. A complete eigendecomposition on an $N*N$ matrix poses a considerable time complexity of $O(N^{3})$, which indeed also results in unnecessary cost for a large $N$ since only top-$k$ ($k\ll N$) eigenvalues/eigenvectors are used in most eigendecomposition-based graph analysis tasks [19, 10, 9, 20, 21, 23, 22]. In practice, given a large-scale adjacency matrix $\mathbf{A}$, to calculate its top-$k$ eigenvalues/eigenvectors, the first step is to reduce its dimension from $N*N$ to $M*M$ ($M$ is usually slightly larger than $k$), producing a new matrix $\overline{\mathbf{A}}$ for further processing. The most popular dimension reduction methods are the Arnoldi method (Algorithm 1) [24] and the Lanczos method (Algorithm 2) [13], which work on general (possibly non-symmetric) matrices and symmetric matrices, respectively. After dimension reduction, the QR algorithm [25] is usually used to efficiently calculate the complete eigenvalues/eigenvectors of $\overline{\mathbf{A}}$. Finally, the top-$k$ eigenvalues of $\overline{\mathbf{A}}$ are used to represent the top-$k$ eigenvalues of $\mathbf{A}$, and the corresponding eigenvectors $\overline{\mathbf{V}}$ can be transformed to the eigenvectors $\mathbf{V}$ of $\mathbf{A}$ by $\mathbf{V}=\mathbf{P}\overline{\mathbf{V}}$ where $\mathbf{P}$ is determined by line 11, Algorithm 1 or line 11, Algorithm 2.

Compared to the traditional differential privacy model [36] that assumes a trusted data collector which can collect and see raw data, the recently emerging LDP model [37] considers the data collector to be untrusted, in which each user only reports perturbed data with calibrated noises added. The formal definition of $(\epsilon,\delta)$-LDP is as follows.

Laplace distribution is a widely popular choice to draw the noises, which is formally defined as follows.

Given a private value $x\in\mathbb{Z}_{2^{k}}$, ASS in a two-party setting works by splitting it into two secret shares $\langle x\rangle_{1}$ and $\langle x\rangle_{2}$ such that $x=\langle x\rangle_{1}+\langle x\rangle_{2}$ [16]. Each share alone reveals no information about $x$. We denote by $\llbracket x\rrbracket$ the ASS of $x$ for short. It is noted that if $k=1$, we say the secret sharing is binary sharing, denoted as $\llbracket x\rrbracket^{B}$, and otherwise arithmetic sharing, denoted as $\llbracket x\rrbracket^{A}$. Given a public constant $c$, and the secret sharings $\llbracket x\rrbracket$ and $\llbracket y\rrbracket$, addition/subtraction $\llbracket x\pm y\rrbracket=\llbracket x\rrbracket\pm\llbracket y\rrbracket$ and scalar multiplication $\llbracket c\cdot x\rrbracket=c\cdot\llbracket x\rrbracket$ can be performed without interaction among the two parties that hold the shares respectively, while multiplication $\llbracket x\cdot y\rrbracket=\llbracket x\rrbracket\cdot\llbracket y\rrbracket$ requires the two parties to have one round of online communication with the use of Beaver triples which can be prepared offline. It is noted that addition and multiplication over $\mathbb{Z}_{2}$ are equivalent to XOR and AND respectively.

FSS [39] is a low-interaction secret sharing for secure computation, presenting prominent advantages in online communication and round complexity compared to other alternative techniques, such as garbled circuits [40] or ASS. From a high-level point of view, a two-party FSS-based approach to a private function $f$ consists of a pair of probabilistic polynomial time (PPT) algorithms: (i) $(k_{1},k_{2})\leftarrow\mathsf{Gen}(1^{\lambda},f)$: given a security parameter $\lambda$ and a function description $f$, output two succinct FSS keys $k_{1},k_{2}$, each for one party. (ii) $\langle f(x)\rangle_{i}\leftarrow\mathsf{Eval}(k_{i},x)$: given an FSS key $k_{i}$ and an evaluation point $x\in\{0,1\}^{n}$, output the share $\langle f(x)\rangle_{i}$ of the function evaluation result. The security of FSS ensures that an adversary learning only one FSS key $k_{i},i\in\{1,2\}$ learns no information about the function $f$ and output $f(x)$.

Fig. 1 illustrates PrivGED’s system architecture. There are three kinds of entities: the users $\mathcal{U}_{i}(i\in[1,N]$), the cloud, and the analyst. The users and the social relationship (friendship for simplicity) among them constitute a (decentralized) social graph, where each $\mathcal{U}_{i}$ represents a graph node and the friendship between any two users indicates the existence of an edge, and the number of each $\mathcal{U}_{i}$’ associated friends is the degree of the corresponding graph node. Consider as a concrete example a phone network [4], where each $\mathcal{U}_{i}$ has a limited phone numbers in his contact list and thus a limited local view about the social graph.

The decentralized social graph can be characterized by an adjacency matrix $\mathbf{A}$ of size $N*N$, where each row $\mathbf{A}[i,:](i\in[1,N]$) indicates $\mathcal{U}_{i}$’s local view. For example, in an unweighted social graph, $\mathbf{A}[i,j]=1$ may indicate that $\mathcal{U}_{i}$ and $\mathcal{U}_{j}$ are friends; in a weighted social graph, $\mathbf{A}[i,j]=v$ may indicate the degree of intimacy between $\mathcal{U}_{i}$ and $\mathcal{U}_{j}$ by a value $v$. The users are willing to allow analytics over their federated data of local views to produce analytical results for the analyst. Our focus in this paper is the analytics task of eigendecomposition, which plays a vital role in graph analytics and has many applications as aforementioned. However, due to privacy concerns, each $\mathcal{U}_{i}$ is not willing to disclose his private social relationship throughout the analytical process over the data federation, so the enforcement of data protection is demanded.

The graph analytics service is empowered by the cloud for well understood benefits. In PrivGED, the power of the cloud is split into two cloud servers from different trust domains which can be hosted by different service providers in practice. Such multi-server model is getting increasingly popular in recent years for security designs in various domains, including both academia [32, 41, 42, 43, 44, 45, 46, 47] and industry [48, 49]. The adoption of such model in PrivGED follows this trend. The two cloud servers in PrivGED, denoted as $\mathcal{CS}_{\{1,2\}}$, collaboratively perform the analytics task of eigendecomposition without seeing each $\mathcal{U}_{i}$’s data, and produce encrypted results (the top-$k$ eigenvalues/eigenvectors) which are delivered to the analyst on demand. At a high level, PrivGED proceeds through two phases: (i) Secure collection of decentralized social graph data. In this phase, each user $\mathcal{U}_{i}$’s local view data $\mathbf{A}[i,:]$ regarding the social graph is collected by the cloud servers in encrypted form so as to form the matrix for eigendecomposition, through sparsity-aware and privacy-preserving techniques developed in PrivGED. (ii) Secure eigendecomposition. After the encrypted adjacency matrix is adequately formed at the cloud, the cloud servers collaboratively perform secure eigendecomposition, through a suite of customized secure protocols developed in PrivGED.

Along the system workflow in PrivGED, we consider that the primary threats come from the cloud entity empowering the eigendecomposition-based graph analytics service. Similar to most prior security designs in the two-server setting [50, 51, 41], we assume a semi-honest and non-colluding adversary model where $\mathcal{CS}_{\{1,2\}}$ honestly follow the protocol specification of PrivGED, but may individually attempt to learn private information about individual users’ social connections in the decentralized social graph and the analytics result. Note that each user’s private social connections with other entities in the graph are reflected by the non-zero elements in her local view vector, as introduced above. So, for the privacy of individual users, PrivGED aims to conceal the sensitive information regarding the non-zero elements in each user $\mathcal{U}_{i}$’s local view vector $\mathbf{A}[i,:]$, which includes the positions, values, and number.

In this phase, each user $\mathcal{U}_{i}$ provides his local view data $\mathbf{A}[i,:]$ in protected form to the service. For high efficiency, PrivGED resorts to the lightweight ASS technique for encryption of the elements in $\mathbf{A}[i,:]$. A simple method is to have each $\mathcal{U}_{i}$ directly apply ASS over his complete $\mathbf{A}[i,:]$ of length $N$. However, such simple method is clearly inefficient due to the sparsity of the (decentralized) social graph. According to Facebook’s statistics [14], on average a user has 130 friends in a social network, which is far less than $N$ (e.g., hundreds of thousands). This leads to high sparsity, indicating that the complete $\mathbf{A}[i,:]$ will be filled with many zeros. So the simple method will incur significant cost on the user side as well as pose unnecessary workload in the subsequent secure eigendecomposition process.

To remedy this, a plausible approach is to leverage sparse representation. Specifically, $\mathcal{U}_{i}$ applies ASS only over each nonzero element at location $j$ (denoted as $\{(i,j,\mathbf{A}[i,j])\}$), and sends $\{(i,j,\llbracket\mathbf{A}[i,j]\rrbracket^{A})\}$ to $\mathcal{CS}_{\{1,2\}}$. Such approach yet leads to prominent privacy leakages: (i) The number of nonzero elements (i.e., the degree) indicates the number of $\mathcal{U}_{i}$’s friends, which can be used in inferring $\mathcal{U}_{i}$’s privacy [15]. (ii) The presence of a nonzero element $(i,j,\llbracket\mathbf{A}[i,j]\rrbracket^{A})$ implies the presence of the relationship between $\mathcal{U}_{i}$ and $\mathcal{U}_{j}$, which can also be used in inference attacks [15]. (iii) If the social graph is unweighted (i.e., each element is 0 or 1), the presence of a nonzero element $(i,j,\llbracket\mathbf{A}[i,j]\rrbracket^{A})$ implies $\mathbf{A}[i,j]=1$, revealing the data to $\mathcal{CS}_{\{1,2\}}$.

Therefore, the challenge here is how to preserve the benefits of sparsity while protecting the privacy of individual user’s social relationships. Meanwhile, the effectiveness of subsequent eigendecomposition should not be affected.

Our key insight is to delicately trade off (node degree) privacy for efficiency, by having $\mathcal{U}_{i}$ blend in some dummy edges with zero weights at random empty locations in $\mathbf{A}[i,:]$, and apply ASS over the weights of both true and dummy edges, inspired by [8]. Under ASS, even encrypting the same (zero) value multiple times will result in different shares (ciphertexts) indistinguishable from uniformly random values. Therefore, the dummy edges cannot be distinguished from the true edges, as well as do not affect the effectiveness of the subsequent secure eigendecomposition process.

What remains challenging here is how to appropriately set the number of dummy edges so as to delicately balance the trade-off between efficiency and privacy. Too many dummy edges will impair the sparsity and increase the system overhead, while too few dummy edges will result in weaker privacy protection. Therefore, a tailored security design is demanded to provide a theoretically sound approach by which $\mathcal{U}_{i}$ can select the adequate number of dummy edges to achieve a balance between efficiency and privacy. Our main idea is to rely on LDP so as to make the leakage about the node degrees differentially private. In what follows, we start with some basic approaches and discuss their limitations. Then we present our tailored solution.

A basic approach based on LDP can work as follows. As aforementioned, each $\mathcal{U}_{i}$ draws a noise $n_{i}$ from the discrete Laplace distribution, and then blends $n_{i}$ dummy edges with zero values at random empty locations in $\mathbf{A}[i,:]$, followed by encrypting the weights of both true and dummy edges by ASS. In this way, differential privacy guarantee on the node degree can be achieved, and the presence of edges is disguised as well [8]. Here, the sensitivity $\Delta$ needs to be set to $\Delta=(d_{max}-d_{min})$, where $d_{max}$ and $d_{min}$ are the possible maximum and minimum degree in the decentralized social graph, respectively. The above basic approach, however, would result in a very large $\Delta$ (up to $N$ in theory) to be applied, leading to a large number of dummy edges to be added and heavily impairing the sparsity. To illustrate why this is the case, Fig. 2 shows the probability density function of the discrete Laplace distribution with different $\Delta$. It is revealed that a larger $\Delta$ leads to the shape of the density function being more uniform. This indicates that the larger the sensitivity $\Delta$ is, the larger probability that $\mathcal{U}_{i}$ draws a large $|n_{i}|$ will be. In contrast, a small $\Delta$ will make the probability density function concentrated (e.g., $\Delta=50$ in Fig. 2), which means that $\mathcal{U}_{i}$ will draw a small $|n_{i}|$ with a large probability.

In order to achieve better efficiency, an alternative approach as proposed by [8] is to use a bin-based mechanism that provides bin-wise differential privacy rather than graph-wise differential privacy. The main idea is to partition users into several bins, each of which contains an approximately equal number of users whose degrees are within a small interval $[d_{p},~{}d_{q}]$. As such, all users in the same bin can use a much smaller sensitivity $\Delta=d_{q}-d_{p}$. We note that while the idea proposed by Sagar et al. [8] is useful, their design to instantiate such idea is not satisfactory and has crucial downsides. Firstly, it requires some (sampled) users to open their degrees to estimate the degree histogram of the social graph so as to split users into bins, harming the privacy of the sampled users. Secondly, to avoid edge deletion when the drawn noise is negative, it lets all users add a large offset to the drawn noise, making a notable impact on the sparsity. In addition, no formal analysis regarding the differential privacy guarantee is provided.

Based on the aforementioned idea of bin-based LDP, we design a new protocol for secure collection of decentralized social graph data, which does not expose any users’ exact node degree information, as opposed to PrivateGraph [8]. Meanwhile, our protocol does not require users to add a large offset to the noises, and thus is much more advantageous in maintaining the sparsity. We also later provide formal analysis on the differential privacy guarantee. Our protocol for secure collection of decentralized social graph data is comprised of the following ingredients: (i) secure degree histogram estimation, (ii) secure binning map generation, and (iii) local view data encryption.

The encrypted data collected from the users forms the encrypted adjacency matrix under sparse encoding for eigendecomposition at the cloud. We denote it by $\llbracket\mathbf{A}^{*}\rrbracket^{A}:=\{(i,j,\llbracket\mathbf{A}[i,j]\rrbracket^{A})\}_{i\in[1,N],j\in\mathcal{L}^{*}}$, where $\mathcal{L}^{*}$ is a multi-set that contains the all locations from $\mathcal{L}_{1}$, $\mathcal{L}_{2}$, $\cdots$, $\mathcal{L}_{N}$. Upon deriving $\llbracket\mathbf{A}^{*}\rrbracket^{A}$, PrivGED needs to enable $\mathcal{CS}_{\{1,2\}}$ to obliviously perform eigendecomposition on $\llbracket\mathbf{A}^{*}\rrbracket^{A}$, producing the encrypted top-$k$ eigenvalues/eigenvectors.

Firstly, PrivGED introduces techniques (Section 6.2) to enable $\mathcal{CS}_{\{1,2\}}$ to obliviously transform $\llbracket\mathbf{A}^{*}\rrbracket^{A}$ to $\llbracket\mathbf{\overline{A}}\rrbracket^{A}$ (in dense encoding) of a much smaller size $M*M$, with respect to the Arnoldi method (for general (possibly non-symmetric) matrices) and Lanczos method (for symmetric matrices) as introduced before in Section 3.1. Then, PrivGED provides secure realizations of the QR algorithm so as to achieve secure eigendecomposition over the $\llbracket\mathbf{\overline{A}}\rrbracket^{A}$ to produce encrypted top-$k$ eigenvalues/eigenvectors. We give a basic design (Section 6.3) for the secure QR algorithm as a starting point, followed by a delicate optimized design (Section 6.4) that further achieves a performance boost.

We first consider how to enable $\mathcal{CS}_{\{1,2\}}$ to obliviously execute the Arnoldi method on $\llbracket\mathbf{A}^{*}\rrbracket^{A}$ to output $\llbracket\mathbf{\overline{A}}\rrbracket^{A}$. Looking into the operations in Algorithm 1, we note that the operations in line 1-7, which are comprised of addition and multiplication, can be naturally and securely realized in the arithmetic secret sharing domain by $\mathcal{CS}_{\{1,2\}}$. However, it remains challenging for $\mathcal{CS}_{\{1,2\}}$ to obliviously perform the operations in lines 8, 9, because square root and division are not directly supported in the secret sharing domain.

Our solution is to approximate the square root and division operations using basic operations (i.e., $+,\times$), so that they can be securely supported in the secret sharing domain. For secure square root $\llbracket\sqrt{x}\rrbracket^{A}$, inspired by the very recent work [53], PrivGED utilizes a roundabout strategy to approximate the inverse square root $\frac{1}{\sqrt{x}}$ by the iterative Newton-Raphson algorithm [54]:

which will converge to $y_{n}\approx\frac{1}{\sqrt{x}}$. Obviously, both subtraction and multiplication operations are naturally supported in the secret sharing domain. After that, PrivGED lets $\mathcal{CS}_{\{1,2\}}$ multiply $\llbracket y_{n}\rrbracket^{A}$ by $\llbracket x\rrbracket^{A}$ to derive $\llbracket\sqrt{x}\rrbracket^{A}$. For secure division $\llbracket\frac{y}{x}\rrbracket^{A}$ in the secret sharing domain, we note that the main challenge is to compute the reciprocal $\llbracket\frac{1}{x}\rrbracket^{A}$. However, we also observe that the reciprocal of division in Algorithm 1 is $\frac{1}{\overline{\mathbf{A}}[k,k-1]}$, which is exactly the inverse square root computed in line 8. Therefore, PrivGED lets $\mathcal{CS}_{\{1,2\}}$ directly perform the operation in line 9 by multiplying the inverse square root computed in line 8 by $\llbracket\mathbf{q}_{k}\rrbracket^{A}$.

We give our construction for securely realizing the Arnoldi method in Algorithm 6. Regarding the Lanczos method, we note that the operations required to be securely supported are identical to the Arnoldi method, so we omit the algorithm description for the secure Lanczos method. It is noted that the encrypted adjacency matrix $\llbracket\mathbf{A}^{*}\rrbracket^{A}:=\{(i,j,\llbracket\mathbf{A}[i,j]\rrbracket^{A})\}_{i\in[1,N],j\in\mathcal{L}^{*}}$ under sparse encoding can significantly save the cloud-side cost. For example, the secure matrix-vector product between $\llbracket\mathbf{A}^{*}\rrbracket^{A}$ and $\llbracket\mathbf{v}\rrbracket^{A}$ can be efficiently performed by only securely multiplying the elements $\llbracket\mathbf{A}[i,j]\rrbracket^{A}$ and the corresponding $\llbracket\mathbf{v}[j]\rrbracket^{A}$. Obliviously, the number of multiplications in this example is independent of the number of columns in the original complete $\mathbf{A}$ and only depends on the number of rows and the number of nonzero elements in $\mathbf{A}$.

We now introduce, as a basic design, how to enable $\mathcal{CS}_{\{1,2\}}$ to obliviously calculate the complete encrypted eigenvalues/eigenvectors of $\llbracket\overline{\mathbf{A}}\rrbracket^{A}$, through a protocol for securely realizing the widely used QR algorithm. The QR algorithm works in an iterative manner, consisting of a series of QR decomposition. Formally, given matrix $\mathbf{L}$ and $\mathbf{T}_{0}=\mathbf{L}$, in the $k$-th ($k\in[1,K]$) iteration, on input $\mathbf{T}_{k-1}$, we compute QR decomposition $\mathbf{T}_{k-1}=\mathbf{Q}_{k-1}\mathbf{R}_{k-1}$, where $\mathbf{Q}_{k-1}$ is an orthogonal matrix (i.e., $\mathbf{Q}^{T}=\mathbf{Q}^{-1}$), $\mathbf{R}_{k-1}$ is an upper Hessenberg matrix, and then output $\mathbf{T}_{k}=\mathbf{R}_{k-1}\mathbf{Q}_{k-1}$. At the end of QR algorithm, the diagonal elements of $\mathbf{T}_{K}$ are $\mathbf{L}$’s eigenvalues and $\mathbf{S}=\mathbf{Q}_{1}\cdots\mathbf{Q}_{K}$ are $\mathbf{L}$’s eigenvectors.

In PrivGED, we perform the QR decomposition utilizing the Givens rotations [55]. Formally, given an $M*M$ upper Hessenberg matrix $\mathbf{T}_{k-1}$, we create the orthogonal Givens rotation matrix $\mathbf{G}_{i},i\in[1,M-1]$:

where

and $\mathbf{H}(i)=\mathbf{G}^{T}_{i-1}\mathbf{H}(i-1),\mathbf{H}(1)=\mathbf{T}_{k-1}$. At the end of this iteration,

and $\mathbf{Q}_{k-1}=\mathbf{G}_{1}\cdots\mathbf{G}_{M-1}$.

Next, we consider how to securely and efficiently perform the above process in secret sharing domain. The main challenging part of the secure QR algorithm is to let $\mathcal{CS}_{\{1,2\}}$ obliviously calculate square root and division in secret sharing. This can be effectively addressed based on the techniques introduced in Section 6.2. In addition, we note that securely realizing the QR Algorithm consists mainly of matrix multiplications in the secret sharing domain.

A straightforward method is to perform secret-shared value-wise multiplication, which would require $M^{3}$ multiplications and online communication of $2M^{3}$ ring elements. We note that a better choice is to work with vectorization [16], where the Beaver triples needed for secure multiplication are represented in a vectorized form, i.e., $(\mathbf{X},\mathbf{Y},\mathbf{Z})$, where $\mathbf{Z}=\mathbf{X}\mathbf{Y}$; and $\mathbf{X}$ and $\mathbf{Y}$ play the role in masking the input matrices during secure multiplication. Based on such vectorization trick, the online communication is reduced to only $2M^{2}$ ring elements. PrivGED chooses to work with vectorization for secret-shared matrix multiplication.

Algorithm 7 presents our basic design for securely realizing the QR algorithm. With $\llbracket\overline{\mathbf{A}}\rrbracket^{A}$ as input, it outputs $\llbracket\mathbf{T}_{K}\rrbracket^{A}$ and $\llbracket\mathbf{S}\rrbracket^{A}$. Note that the top-$k$ diagonal elements of $\mathbf{T}_{K}$ represent the desired top-$k$ eigenvalues of $\mathbf{A}^{*}$. For the secret-shared eigenvectors $\llbracket\mathbf{S}\rrbracket^{A}$ of $\llbracket\mathbf{\overline{A}}\rrbracket^{A}$, they can be obliviously transformed to the corresponding eigenvectors of $\llbracket\mathbf{A}^{*}\rrbracket^{A}$ by $\llbracket\mathbf{V}\rrbracket^{A}=\llbracket\mathbf{P}\rrbracket^{A}\llbracket\mathbf{S}\rrbracket^{A}$, where $\llbracket\mathbf{P}\rrbracket^{A}$ is the output matrix of our secure Arnoldi method (or Lanczos method).