Adversarial for Social Privacy: A Poisoning Strategy to Degrade User Identity Linkage

Network alignment aims to search related nodes across different networks and has been widely applied in diverse research fields [6], including chemistry [24], bioinformatics [25], knowledge graph [6], and social computing [3]. In this paper, attentions are paid to network alignment models in social fields. Sometimes, it is also known as user identity linkage [26] or anchor link prediction [3].

Most previous works utilize the “topology consistency” in such kind of matching issues. Inspired by PageRank [27], IsoRank [25] and FINAL [10] algorithm iteratively propagate the pairwise node similarity in graph. Additionally, BigAlign [28] uses the alternating projected gradient descent to solve the bipartite network alignment. Then, with the development of graph representation, topology consistency can be also addressed by specific embedding. PALE [8] and IONE [7] learn cross-network mapping via embedding that captures the specific structural regularities. DeepLink [9] uses embeddings inspired by word embedding technologies. Based on heterogeneous networks, TALP [29] design a type-aware embedding method to better model matching objectives.

Generally, graph adversarial attack methods can be divided into three categories according to specific tasks, i.e., node-relevant, link-relevant, and graph-relevant tasks. Node-relevant models focus on node-level tasks such as node classification and node embedding. Zügner et al. [13] first propose NETTACK to fool the node classification models. RL-S2V [14] utilizes a reinforcement learning method to learn the attack policy. Additionally, Bojchevski et al. [30] analyze adversarial vulnerability by random walks and propose approaches to worsen the quality of node embeddings. Most link-relevant models attack link prediction tasks. Zhou et al. [16] try to minimize the similarity metrics to lower the accuracy of link prediction. FPTA [17] designs heuristic rules to attack target models. Moreover, graph-relevant models concentrate on graph-level tasks such as graph classification. Xi et al. [18] propose the backdoor attack on graphs to worsen the model.

However, most attack models concentrate on single network tasks while UIL is related to multiple networks, thus existing models may be invalid in degrading the performance of UIL. Although the work like GMA [19] can perform its effectiveness on graph matching, the requirement on cooperative perturbations cross networks limits the real application on privacy protection.

In this paper, a graph is defined as a tuple $G=(V,E,\boldsymbol{X})$, where $V$ and $E$ denote the set of nodes and edges respectively. Symbol $v$ denotes a node and $e_{ij}$ represents the edge with node $v_{i}$ and $v_{j}$ as its endpoints. Let $|V|$ and $|E|$ denote the total number of nodes and edges. Each node $v\in V$ has $d$-dimensional attributes which are represented by a row in $\boldsymbol{X}\in\mathbb{R}^{|V|\times d}$. The adjacency matrix of $G$ is represented by $\boldsymbol{A}$.

For a node $v\in V$, the $k$-hop neighborhood of $v$ contains nodes at a distance less than or equal to $k$ from $v$. The $k$-ego network of node $v$ is a subgraph of $G$ induced by the $k$-hop neighborhood of $v$ and $v$ itself, and is denoted by $G_{k}(v)$.

Kernel methods are commonly used to measure similarity between two objects with a kernel function which corresponds to an inner product in reproducing kernel Hilbert space (RKHS). Kernels on graphs are generally defined on R-Convolution framework [31]. The main idea of R-Convolution is to decompose a graph into sub-structures and kernel value is a combination of sub-structure similarities. In this way, the kernel $\mathcal{K}(\cdot,\cdot):\mathbb{G}\times\mathbb{G}\rightarrow\mathbb{R}$ defined on graphs $G,G^{*}\in\mathbb{G}$ is given by:

where $g=g_{1},\ldots,g_{D}$ are decompositions of graph $G$, $\mathcal{R}$ denotes the relation from sub-structures $g$ to graph $G$, and $\mathcal{R}^{-1}(G)$ is a set of sub-structures in graph $G$. The $\mathcal{K}_{d}(\cdot,\cdot):\mathbb{R}^{n}\times\mathbb{R}^{n}\rightarrow\mathbb{R}$ is a general kernel and satisfies:

Here, function $\phi$ maps a vector to a high dimensional space, and $<\cdot,\cdot>$ denotes the inner product.

Let $\mathcal{X}$ be a $d$-dimensional compact set and $\text{Prob}(\mathcal{X})$ denotes space of probability measures defined on $\mathcal{X}$. Earth mover’s distance (EMD) defines the discrepancy between two distribution $P_{1},P_{2}\in\text{Prob}(\mathcal{X})$:

where $\Gamma(P_{1},P_{2})$ is the set of all joint distributions whose marginals are $P_{1}$ and $P_{2}$. The $\gamma(x,y)$ can be intuitively interpreted as “mass” transportation from $x$ to $y$ so that EMD is the minimum cost for transforming one distribution $P_{1}$ into another distribution $P_{2}$. The $c(x,y)$ determines transportation cost from $x$ to $y$, which is generally derived from distance measures. The discrete form of EMD is as follows:

Variational graph auto-encoder (VGAE) [23] is a commonly used network embedding framework, which encodes both graph structures and node attributes into the embedding. Given adjacency matrix $\boldsymbol{A}$ and attribute $\boldsymbol{X}$ for a graph, the encoder part of VGAE computes latent variables $\boldsymbol{\mu}$ and $log\boldsymbol{\sigma}$ with GCNs [32]: $\boldsymbol{\mu}=GCN_{\mu}(\boldsymbol{A},\boldsymbol{X})$, $log\boldsymbol{\sigma}=GCN_{\sigma}(\boldsymbol{A},\boldsymbol{X})$, and latent embedding $\boldsymbol{z_{i}}$ is given by a normal distribution $\mathcal{N}$:

The decoder part reconstructs the graph via inner product of latent embeddings. In this paper, we use VGAE to generate node embeddings that combine structural and attribute features.

Given source network $G^{s}=(V^{s},E^{s},\boldsymbol{X}^{s})$ and target network $G^{t}=(V^{t},E^{t},\boldsymbol{X}^{t})$. Let $\boldsymbol{H}\in[0,1]^{|V^{s}|\times|V^{t}|}$ represents the prior knowledge matrix. The goal of UIL algorithm is to find the matching matrix:

The matrix $\boldsymbol{M}$ has size $|V^{s}|\times|V^{t}|$ and the entry $\boldsymbol{M}_{ij}$ reflect the probability that node $v_{i}$ (in source networ) and node $v_{j}$ (in target network) are linked. Typically, matrix $\boldsymbol{H}$ denotes labeled data for supervised model. Otherwise, $\boldsymbol{H}$ can also be pairwise node similarity. If no prior knowledge is known, all entries of $\boldsymbol{H}$ are set to 0.

In this paper, the goal of adversarial attacks on UIL algorithms is to remove a set of edges $E^{-}\subset E^{t}$ to maximize changes in matching matrix. Let $\boldsymbol{M^{*}}$ denotes the matching matrix between source network and poisoned target network $G^{t*}=(V^{t},E^{t}-E^{-},\boldsymbol{X}^{t})$, the objective is:

where $K$ is an allowed threshold of removed edges. Here, we only consider the removing edge attack because graphs in real applications are usually sparse, thus the solution spaces are relatively small compared with other kinds of attack. Meanwhile, removing edge is easy to implement on social platforms.

Previous studies on UIL have reached a consensus [10, 11] that, among different networks, nodes that have similar local structures are more likely to be linked, which is known as the “topology consistency”. Accordingly, instead of assuming one specific UIL algorithm as the threat model, we propose a unified framework to re-formalize the UIL problem based on the widely used topology consistency, which is conducive to devising an effective and universal strategy to degrade diverse UIL models.

The local structure of a node $v$ can be well captured by the $k$-ego network of $v$ and the similarity across different structures is measured by graph kernels. Thus, consistency between node $v_{i}$ in graph $G^{s}$ and node $v_{j}$ in graph $G^{t}$ can be measured as: $\mathcal{K}(G_{k}^{s}(v_{i}),G_{k}^{t}(v_{j}))$. The UIL task can be uniformly expressed as the following form:

The above equation means nodes with higher topology consistency should have a higher probability to be aligned and, at the same time, the final alignment matrix $\boldsymbol{M}$ needs to be consistent with prior knowledge matrix $\boldsymbol{H}$. Here, $\alpha$ is a hyperparameter that controls the importance of prior knowledge.

Let $\boldsymbol{\Phi}$ denote the Kernel Matrix and $\boldsymbol{\Phi}_{ij}=\mathcal{K}(G_{k}^{s}(v_{i}),G_{k}^{t}(v_{j}))$. Eq (1) can be written in the matrix form as follows:

The closed-form solution is obtained by setting its derivative to be zero:

and finally we can get

Eq. (2) gives a unified view of UIL tasks based on the topology consistency. With such a perspective, the matching matrix is a linear combination of the kernel matrix and the prior knowledge matrix. Although most existing UIL models do not use kernel-based modeling explicitly, the pairwise similarity comparison in them is closely related to kernel methods. For example, the matching results of FINAL [10] are relevant to random walk kernels and the similarity comparison in DeepLink [9] can be viewed as a variant of path-based kernels.

Next, we will display our attack strategy based on the proposed unified kernel-based framework on UIL.

With the unified kernel-based framework on UIL, the matching matrix $\boldsymbol{M^{*}}$ between source network $G^{s}$ and poisoned target network $G^{t*}$ is calculated as:

where $\boldsymbol{\Phi}^{*}$ denote the Kernel Matrix for source network and poisoned target network so that $\boldsymbol{\Phi}^{*}_{ij}=\mathcal{K}(G_{k}^{s}(v_{i}),G_{k}^{t*}(v_{j}))$. Integrating Eq (2) and Eq (3), the original objective of the attack problem can be transformed into the following form:

Here, $k$-ego network $G_{k}^{t*}(v)$ changes as the removing edge set $E^{-}$ changes and the goal is to maximize the Eq (IV-C). It can be noticed that the source network $G^{s}$ and the target network $G^{t}$ are both involved in the objective function which indicates the optimization process needs information about two networks. However, collecting details across different networks is usually difficult in real applications. Therefore, the following theorem is utilized to simplify the objective function:

The proof is referred to in Appendix A-A. Theorem 1 indicates that maximizing the differences on kernel function values between source network and target network before and after the attack as in Eq. (IV-C) is equivalent to minimizing the kernel values for the clean and poisoned target network. With this transformation, only the target network information is required and the objective is converted to:

In this section, we will define a specific graph kernel applied in TOAK (shown in Fig. 2). Following the R-Convolution framework mentioned in Section III-B, a naive decomposition treats the graph as a bag-of-edges, where pre-image $\mathcal{R}^{-1}(G)$ corresponds to the set of edges in graph $G$. In this way, the R-Convolution kernel related to $G$ and $G^{*}$ can be reduced to:

where $D=\{(e_{ij},e^{*}_{ij})|\ e_{ij}\in E,\ e^{*}_{ij}\in E^{*}\}$ refers to the set of comparable edge pairs between $G$ and $G^{*}$, and $\vec{e}_{ij},\ \vec{e^{*}}_{ij}$ denotes representations of edges $e_{ij},\ e^{*}_{ij}$ respectively.

Generally, edge representation can be well interpreted by its endpoints’ representations [7]. With consideration of the decomposition insensitive to topology variance in Eq. (6), we apply VGAE to generate powerful node representations that fuses the structural and attribute information:

where $\boldsymbol{V_{emb}}$ is node representation matrix and row vector $\vec{v}_{i}$ denotes corresponding node representation, With the calculated node representation, the edge representation is obtained with concatenation and normalization of endpoints’ representations:

Here, $\oplus$ represents the concatenation operation of two vectors and $\parallel\cdot\parallel$ denotes the norm of a vector.

Based on edge representations, kernel $\mathcal{K}_{d}$ is formalized as a radial basis function, i.e.,

where $\gamma_{d}$ is a shape parameter to scale the distance between edges. Substitute Eq. (8) into Eq. (6), the kernel $\mathcal{K}$ can be formulated as

Then, we formalize the unresolved parameter $\gamma_{d}$. We regard a graph as a distribution on its edge embedding space and measures the discrepancy between two distributions with Earth mover’s distance as follows:

Here $P_{G}=\{(\vec{e}_{ij},p(e_{ij}))\}_{e_{ij}\in E}$ and $P_{G^{*}}=\{(\vec{e^{*}}_{ij},p(e^{*}_{ij}))\}_{e^{*}_{ij}\in E^{*}}$ are the converted distribution of $G$ and $G^{*}$ respectively. Parameter $\gamma_{d}$ is exactly defined as the joint probability. With this definition, the kernel reflects not only the similarity between two graphs but also the importance of different edges in a graph. The effect of removing an edge can be calculated conveniently, which is essential in the attack task.

The marginal probability $p({e}_{ij})$ in Eq. (10) is defined as follows:

where $RW(e)$ is a function counting the frequency that edge $e$ appears in random walk process. The $\boldsymbol{H}_{i:}$ and $\boldsymbol{H}_{j:}$ represent the sum of $i$-th row and $j$-th row in prior knowledge matrix $\boldsymbol{H}$ respectively. Function $\mathbb{I}(x)$ is an indicator, it equals 1 when $x\geqslant 1$ and equals 0 otherwise. Note that, random walk and prior knowledge indicate the edge importance in graphs and UIL task respectively.

Overall, the proposed kernel can be formulated as follows:

The literature [33] and [34] have proved kernels in this form are positive definite, thus ensuring the correctness of the problem-solving process in TOAK.

Directly applying the proposed kernel to objective Eq. (5) will calculate EMD with two $k$-ego networks. The time complexity of computing EMD in Eq. (10) is $O(n^{3}log(n))$, where $n$ represents the number of nodes in $k$-ego network. The high cost of computing makes it hard to be applied to dense or large-scale graphs. In this way, we estimate EMD by its lower bound:

Proof and discussion on the lower bound can be found in Appendix A-B. Using the lower bound in Theorem 2 to replace the EMD in Eq. (12) and we have:

This form of kernel can be viewed as a variant of typical Gaussian kernels which is positive definite. Otherwise, the complexity is reduced to $O(n)$, which is applicable to large-scale datasets.

With the defined kernel functions, we can measure the impact of removing one edge within the object Eq (5):

Here, poisoned graph $G^{t*}$ is obtained via removing edge $e$ in clean graph $G^{t}$. The graph $G_{k}^{t*}(v_{j})$ is the $k$-ego network of node $v_{j}$ on poisoned graph. Note that, if edge $e$ is not in the edge set of $G_{k}^{t}(v_{j})$, which means removing edge $e$ will not affect the $k$-ego network of $v_{j}$, thus the kernel value equals 1. Usually, $k$-ego network contains only a small amount of edges. Therefore, most terms in score(e) equal 1 and only a small part of kernels needs to be calculated.

After computing scores, the edge with the minimum score is removed. By repeating the computing-removing process for $K$ times, the set $E^{-}$ is finally obtained. However, recalculating the score for $K$ times is time-consuming. Therefore, we utilize the greedy selecting strategy. According to the score, we choose $K$ edges at once with $K$-smallest values to compose the removing edge set $E^{-}$. The overall algorithm is shown in Algorithm 1.

The time complexity of the random walk process is $O(|V|RL)$, in which $R$ denotes random walk amount and $L$ denotes length of random walk. The score calculating has a complexity of $O(|E|nm)$, where the number $n$ represents the average nodes among the $k$-ego networks and the number $m$ denotes average times an edge appears in $k$-ego networks. The total complexity of TOAK is $O(|V|RL+|E|nm)$. As $|V|\gg R,|V|\gg L$ and $|E|\gg nm$, the algorithm is almost linear about node numbers and edge numbers.

Note that the proposed score calculating method in Eq. (14) can also compute the effect of non-existing edges, which represents the adding-edge attack. However, due to the sparsity of graphs, the possible candidates of adding-edge attack are huge, causing nearly $O(|V|^{2}mn)$ time complexity of score calculating process. As a result, in this paper, we only consider removing-edges attacks. Efforts on designing other kinds of attacks will be our future task.

In this section, we describe the datasets, baselines, UIL models, metrics, and other details of experiment¹¹1The code and data of TOAK will be available at Github after acceptance..

Datasets We conduct experiments on three real-world datasets to evaluate the performance of proposed TOAK model. Basic information of datasets is listed in Table I. The Douban and TF datasets come from famous social networks while the ARXIV dataset contains co-author networks. By reversing the source and target network in datasets, six user identity linkage datasets can be built and we denote them as: Douban (Online$\rightarrow$Offline), Douban (Offline$\rightarrow$ Online), TF (Twitter$\rightarrow$Foursqure), TF (Foursquare$\rightarrow$Twitter), ARXIV (Mat$\rightarrow$DM), and ARXIV (CS$\rightarrow$DM), where the former in brackets is the source network and the later is the target network.

Baselines Our TOAK model is compared with 6 baselines including state-of-the-art methods. Random attack randomly removes edges to generate perturbed graphs. DEG attack removes edges that connect to nodes with the largest degree and PR attack deletes edges connected to nodes with the highest pagerank [27] score. DW3 [30] perturbs the random walk based node embedding via analysis on the PPMI matrix of the graph. FPTA [17] is a heuristic approach that relies on hand-crafted rules, which remove edges connected with known linked users. GMA [19] push nodes to dense regions in graphs via the density estimation approach and achieves state-of-the-art performance. Also, we compare TOAK with its variants TOAK_-, which runs without the EMD acceleration technique. To ensure the fairness of comparison, baseline models only remove edges on the target network as TOAK does.

Target UIL models The effectiveness of the proposed attack method is validated with kinds of UIL models, and differences among them are displayed in Table II. FINAL [10] is a network alignment model which finds aligned nodes through iterations on the Kronecker product of adjacency matrix. REGAL [38] leverages the power of automatically learned node representations to match nodes across different graphs. PALE [8] employs network embedding with awareness of observed anchor links as supervised information. DeepLink [9] utilize more powerful embedding approaches inspired by word embedding technologies. IONE [7] learns multiple node embeddings and tries to model followers/followees as different context vectors. Note that, FINAL and REGAL are not deep models, thus the GMA method can not be applied to them.

Metrics The accuracy index is the most popular measure in UIL tasks, which calculates the ratio of correctly linked user pairs among all linked pairs. To directly reflect the attacking quality, we use a variant of Accuracy named Mismatching Rate (MR) as the metric. MR can be calculated as follows:

As for the measurement of imperceptibility of attacks, it is generally weighted by the total removed edges. However, this index can only reflect the imperceptibility of the whole graph. To go a further step, we consider the imperceptibility at a more precise node level, and computes the ratio of unpoisoned edges on each node, which is called Node Imperceptibility Score ($NIS$):

The higher the $NIS$, the harder the attack is to be perceived at the node $v$.

Experimental Details In this part, we describe the implementation details of TOAK and the experimental settings.

TOAK utilizes typical VGAE²²2https://github.com/dmlc/dgl/tree/master/examples/pytorch/vgae to embed the network. The latent variable of VGAE is calculated with 2-layers GCN with hidden dimensions of 32 and 16 respectively. The learning rate is set to 0.001 and the total running epochs are 1000. The prior knowledge matrix $\boldsymbol{H}$ in unsupervised UIL algorithms is the node degree similarity matrix and for supervised UIL models, $\boldsymbol{H}_{ij}=1$ for training node pairs $(v_{i},v_{j})$ and $\boldsymbol{H}_{ij}=0$ otherwise. For all of the datasets, the random walk length (hyperparameter $L$) is set to 5 and repeated 1000 times (hyperparameter $R$). The hyperparameter $\lambda=1.0$ in Douban (online $\rightarrow$ offline) and $\lambda=5.0$ for Douban (offline $\rightarrow$ online). For TF and ARXIV datasets, $\lambda$ are all set to $2.0$.

As for the details of UIL algorithms. FINAL and REGAL are unsupervised algorithms, and all the linked users are used for testing. The supervised UIL algorithms including PALE, DeepLink, and IONE randomly select 20% of linked user node pairs as training set and remains 80% of linked user pairs for testing. The hyperparameters of UIL models are adjusted to the best. We repeat the experiment 5 times and report the average mismatching rate.

All of the experiments are conducted on a compute server running on Linux with 2 CPUs of Intel Xeon E5-2640 v4 (at 2.40 GHz), 2 GPUs of NVIDIA Telsa K80 (11GB memory), and 128GB of RAM.

Firstly, we make an overall comparison of the attack performance among all the models when removing 10% edges in target network. The results are shown in Table III.

We can observe that the proposed model outperforms other baselines on all datasets. Among different experimental settings over datasets and UIL algorithms, the proposed TOAK improves mismatching rate with an average of 4.58% when compared with best baselines, and gains at most 22.28% increasing on PALE in Douban (Offline$\rightarrow$Online), indicating the effectiveness of TOAK. Meanwhile, results show that attacks from baselines perform unsteady over UIL models. Besides, with a constraint on gradient descent method, GMA cannot be applied to FINAL and REGAL. Meanwhile, TOAK has no such limits and gains the best performance, reflecting better transferability than compared attack methods.

It is worth noting that, on ARXIV datasets, barely removing edges in DM network can dramatically confused the linkage on both the Mat$\rightarrow$DM and CS$\rightarrow$DM. Such a result demonstrates the potential application of TOAK in real-world, where only the target network can be modified. Interestingly, TOAK always obtains more MR promotions on supervised deep UIL models. The phenomenon may be attributed to the vulnerability of deep linkage methods. Finally, despite approximated EMD used in TOAK, it can still achieve similar mismatching rate as TOAK_- (which applies exact calculation on EMD). The comparison on TOAK and TOAK_- demonstrates the correctness of the accelerating process.

Fig. 3 presents mismatching rate on various UIL models by varying the ratios of removed edges from 2% to 10%. As it illustrates, TOAK model always has higher mismatching rates under different ratios of removed edges when compared with other baselines. Meanwhile, mismatching rates can be consistently increased along with higher removing ratios resulting from TOAK.

Next, we discuss the influence of hyper-parameters in the proposed method, including random walk length $L$, random walk amount $R$, and coefficient $\lambda$. Comparisons are conducted with 10% edges in the target network removed. The experimental results on TF (Foursquare$\rightarrow$Twitter) dataset are illustrated in Figure 4. Besides, the influence on other datasets has similar trends and we show them in Appendix A-C. As for random walk length, we can notice that using a longer walk length can improve the performance, and the mismatching rate becomes stable when the length is extended. The phenomenon can be explained that the longer walk length can reflect more comprehensive local structures around nodes thus leading a better performance. In addition, random walk amount has no prominent influence on TOAK, where mismatching rate keeps similar as random walk amount varies from 600 to 1000. The last hyper-parameter $\lambda$ plays an important role in TOAK, which controls the ratio of the prior knowledge in edge distributions. TOAK has almost the worst performance when $\lambda=0$, which means no prior knowledge to refer to, and the mismatching rate starts rising as $\lambda$ becomes larger. Results on $\lambda$ indicate that prior knowledge plays an important role in attacking.

Additionally, the effect of network embedding methods in TOAK is also verified and the results are shown in Fig. 5. Here TOAK is the proposed model with VGAE embedding. The TOAK_node2vec and TOAK_TADW uses node2vec [39] and TADW [40] embedding respectively. The mismatching rate is similar on TOAK and TOAK_TADW while TOAK_node2vec has worse performance. It is because node2vec considers only the graph structure features and ignores the node attributes.

In addition to the effectiveness of the model, the imperceptibility of attack and efficiency are also of great significance. Generally, the imperceptibility of the attack is weighted by the total removed edges, which can only reflect the attack imperceptibility on the whole graph. In practice, users are more sensitive to changes that appear closely around them. As a result, we consider the average node imperceptibility score to better reflect the attack imperceptibility of a single user. The results are shown in Fig 6. It is obvious that our model has the best mismatching rate performance and has higher imperceptibility than the state-of-the-art baseline GMA. Another baseline like DEG or PR has higher $NIS$ but their performances are limited. Overall, the proposed TOAK model can reach a balance between attack effectiveness and attack imperceptibility.

Fig 7 shows the time cost of GMA, TOAK, and TOAK_- on different datasets for removing 10% edges in the target network. The TOAK_-, which computes earth mover’s distance directly, spends much more running time than TOAK and GMA, especially for large target networks. With acceleration techniques, TOAK shows more than 10 times speedup compared with TOAK_- among different datasets. The acceleration process becomes necessary when the graph is large. The TOAK and TOAK_- have almost the same mismatching rate whereas TOAK has a lower time cost. At the same time, TOAK performs better than GMA and is less time-consuming. Comparison of running times shows the proposed TOAK is superior on not only the effectiveness but also the efficiency.