A Comprehensive Survey on
Graph Anomaly Detection with Deep Learning

Anomalies were first defined by Grubbs in 1969 [grubbs1969procedures] as “one that appears to deviate markedly from other members of the sample in which it occurs” and the studies on anomaly detection were initiated by the statistics community in the 19th century. To us, anomalies might appear as social spammers or misinformation in social media; fraudsters, bot users or sexual predators in social networks; network intruders or malware in computer networks and broken devices or malfunctioning blocks in industry systems, and they often introduce huge damage to the real-world systems they appear in. According to FBI’s 2014 Internet Crime Report¹¹1https://www.fbi.gov/file-repository/2014_ic3report.pdf/view, the financial loss due to crime on social media reached more than $60 million in the second half of the year alone and a more up-to-date report²²2https://www.zdnet.com/article/online-fake-news-costing-us-78-billion-globally-each-year/ indicates that the global economic cost of online fake news reached around $78 billion a year in 2020.

In computer science, the research on anomaly detection dates back to the 1980s, and detecting anomalies on graph data has been an important data mining paradigm since the beginning. However, the extensive presence of connections between real-world objects and advances in graph data mining in the last decade have revolutionized our understanding of the graph anomaly detection problems such that this research field has received a dramatic increase in interest over the past five years. One of the most significant changes is that graph anomaly detection has evolved from relying heavily on human experts’ domain knowledge into machine learning techniques that eliminate human intervention, and more recently, to various deep learning technologies. These deep learning techniques are not only capable of identifying potential anomalies in graphs far more accurately than ever before, but they can also do so in real-time.

For our purposes today, anomalies, which are also known as outliers, exceptions, peculiarities, rarities, novelties, etc., in different application fields, refer to abnormal objects that are significantly different from the standard, normal, or expected. Although these objects rarely occur in real-world, they contain critical information to support downstream applications. For example, the behaviors of fraudsters provide evidences for anti-fraud detection and abnormal network traffics reveal signals for network intrusion protection. Anomalies, in many cases, may also have real and adverse impacts, for instance, fake news in social media can create panic and chaos with misleading beliefs [hooi2016birdnest, ahmed2019combining, nguyen2020fang, tam2019anomaly], untrustworthy reviews in online review systems can affect customers’ shopping choices [yu2016survey, benamira2019semi, kumar2018rev2], network intrusions might leak private personal information to hackers [mongiovi2013netspot, miller2013efficient, DBLP:journals/compsec/MiaoSZ20, perozzi2016scalable], and financial frauds can cause huge damage to economic systems [xuexiong2022, 10.1145/3394486.3403361, 10.1145/3394486.3403354, DBLP:conf/www/GuoLAHHZZ19].

Anomaly detection is the data mining process that aims to identify the unusual patterns that deviate from the majorities in a dataset [iglewicz1993detect, chandola2009anomaly, Fraud2020]. In order to detect anomalies, conventional techniques typically represent real-world objects as feature vectors (e.g., news in social media are represented as bag-of-words [sun2018detecting], and images in web pages are represented as color histograms [DBLP:conf/icdm/WuHPZCZ14]), and then detect outlying data points in the vector space [DBLP:conf/ijcai/WangL20, DBLP:conf/kdd/PangCCL18, DBLP:journals/corr/abs-2007-02500], as shown in Fig. 1(a). Although these techniques have shown power in locating deviating data points under tabulated data format, they inherently discard the complex relationships between objects [akoglu2015graph].

Yet, in reality, many objects have rich relationships with each other, which can provide valuable complementary information for anomaly detection. Take online social networks as an example, fake users can be created using valid information from normal users or they can camouflage themselves by mimicking benign users’ attributes [hooi2017graph, CARE-GNN]. In such situations, fake users and benign users would have near-identical features, and conventional anomaly detection techniques might not be able to identify them using feature information only. Meanwhile, fake users always build relationships with a large number of benign users to increase their reputation and influence so they can get unexpected benefits, whereas benign users rarely exhibit such activities [pandit2007netprobe, Densealert]. Hence, these dense and unexpected connections formed by fake users denote their deviations to the benigns and more comprehensive detection techniques should take these structural information into account to pinpoint the deviating patterns of anomalies.

To represent the structural information, Graphs, in which nodes/vertices denote real objects, and the edges denote their relationships, have been prevalently used in a range of application fields [liu2008spotting, wu2014multi, gao2017collaborative, aggarwal2011outlier, wu2017multiple, NIPS20201], including social activities, e-commerce, biology, academia and communication. With the structural information contained in graphs, detecting anomalies in graphs raises a more complex anomaly detection problem in non-Euclidean space - graph anomaly detection (GAD) that aims to identify anomalous graph objects (i.e., nodes, edges or sub-graphs) in a single graph as well as anomalous graphs among a set/database of graphs [akoglu2015graph, DBLP:journals/jiis/ChenHS12, GBGP]. As a toy example shown in Fig. 1(b), given an online social network, graph anomaly detection aims to identify anomalous nodes (i.e., malicious users), anomalous edges (i.e., abnormal relations) and anomalous sub-graphs (i.e., malicious user groups). But, because the copious types of graph anomalies cannot be directly represented in Euclidean feature space, it is not feasible to directly apply traditional anomaly detection techniques to graph anomaly detection, and researchers have intensified their efforts to GAD recently.

Amongst earlier works in this area, the detection methods relied heavily on handcrafted feature engineering or statistical models built by domain experts [akoglu2010oddball, eswaran2018spotlight, li2014probabilistic]. This inherently limits these techniques’ capability to detect unknown anomalies, and the exercise tended to be very labor-intensive. Many machine learning techniques, such as matrix factorization [li2017radar, DBLP:journals/pnas/MahoneyD09] and SVM [DBLP:journals/pr/ErfaniRKL16], have also been applied to detect graph anomalies. However, real-world networks often contain millions of nodes and edges that result in extremely high dimensional and large-scale data, and these techniques do not easily scale up to such data efficiently. Practically, they exhibit high computational overhead in both the storage and execution time [DBLP:journals/jbd/ThudumuBJS20]. These general challenges associated with graph data are significant for the detection techniques, and we categorize them as data-specific challenges (Data-CHs) in this survey. A summary of them is provided in Appendix A.

Non-deep learning based techniques also lack the capability to capture the non-linear properties of real objects [DBLP:journals/corr/abs-2007-02500]. Hence, the representations of objects learned by them are not expressive enough to fully support graph anomaly detection. To tackle these problems, more recent studies seek the potential of adopting deep learning techniques to identify anomalous graph objects. As a powerful tool for data mining, deep learning has achieved great success in data representation and pattern recognition [DBLP:conf/wsdm/WangNWYL20, zhang2019unsupervised, DBLP:conf/icdm/WangZ00ZX20]. Its deep architecture with layers of parameters and transformations appear to suit the aforementioned problems well. The more recent studies, such as deep graph representation learning and graph neural networks (GNNs), further enrich the capability of deep learning for graph data mining [ijcai2020-693, wu2020comprehensive, cui2018survey, NIPS20202, su2021comprehensive]. By extracting expressive representations such that graph anomalies and normal objects can be easily separated, or the deviating patterns of anomalies can be learned directly through deep learning techniques, graph anomaly detection with deep learning (GADL) is starting to take the lead in the forefront of anomaly detection. As a frontier technology, graph anomaly detection with deep learning, hence, is expected to generate more fruitful results on detecting anomalies and secure a more convenient life for the society.

In this section, we provide definitions of different types of graphs mostly used in node/edge/sub-graph-level anomaly detection (Section 3 to Section 7). For consistency, we have followed the conventional categorization of graphs as in existing works [akoglu2015graph, ranshous2015anomaly, kwon2019survey] and categorize them as static graphs, dynamic graphs, and graph databases. Unless otherwise specified, all graphs mentioned in the following sections are static. Meanwhile, as graph-level anomaly detection is discussed far away on page 13, to enhance readability, the definition for the graph database is given closer to the material in Section 8.

Definition 1 (Plain Graph). A static plain graph $G=\{V,E\}$ comprises a node set $V=\{v_{i}\}_{1}^{n}$ and an edge set $E=\{e_{i,j}\}$ where $n$ is the number of nodes and $e_{i,j}=(v_{i},v_{j})$ denotes an edge between nodes $v_{i}$ and $v_{j}$. The adjacency matrix $A=[a_{i,j}]_{n\times n}$ restores the graph structure, where $a_{i,j}=1$ if node $v_{i}$ and $v_{j}$ is connected, otherwise $a_{i,j}=0$.

Definition 2 (Attributed Graph). A static attributed graph $G=\{V,E,X\}$ comprises a node set $V$, an edge set $E$ and an attribute set $X$. In an attributed graph, the graph structure follows the definition in Definition 1. The attribute matrix $X=[\mathbf{x}_{i}]_{n\times k}$ consists of nodes’ attribute vectors, where $\mathbf{x}_{i}$ is the attribute vector associated with node $v_{i}$ and $k$ is the vector’s dimension. Hereafter, the terms attribute and feature are used interchangeably.

Definition 3 (Dynamic Graph). A dynamic graph $G(t)=\{V(t),E(t),X_{v}(t),X_{e}(t)\}$ comprises nodes and edges changing overtime. $V(t)$ is the nodes set in the graph at a specific time step $t$, $E(t)$ is the corresponding edge set, $X_{v}(t)$ and $X_{e}(t)$ are the node attribute matrix and edge attribute matrix at time step $t$ in the graph if existed.

In reality, the nodes or edges might also be associated with numerical or categorical labels to indicate their classes (e.g., normal or abnormal). When label information is available/partially-available, supervised/semi-supervised detection models could be effectively trained.

Anomalous nodes are commonly recognized as individual nodes that are significantly different from others. In real-world applications, these nodes often represent abnormal objects that appear individually, such as a single network intruder in computer networks, an independent fraudulent user in online social networks or a specific fake news on social media. In this section, we specifically focus on anomalous node detection in static graphs. The reviews on dynamic graphs can be found in Section 4. Table II at the end of Section 4 provides a summary of techniques reviewed for ANOS ND.

When detecting anomalous nodes in static graphs, the differences between anomalies and regular nodes are mainly drawn from the graph structural information and nodes/edges’ attributes [li2017radar, bojchevski2018bayesian, zhu2020mixedad, perozzi2014focused]. Given prior knowledge (i.e., community structure, attributes) about a static graph, anomalous nodes can be further categorized into the following three types:

• •

  Global anomalies only consider the node attributes. They are nodes that have attributes significantly different from all other nodes in the graph.

• •

  Structural anomalies only consider the graph structural information. They are abnormal nodes that have different connection patterns (e.g., connecting different communities, forming dense links with others).

• •

  Community anomalies consider both node attributes and graph structural information. They are defined as nodes that have different attribute values compared to other nodes in the same community.

In Fig. 3, node 14 is a global anomaly because its 4th feature value is 1 while all other nodes in the graph have the value of 0 for the corresponding feature. Nodes 5, 6, and 11 are identified as structural anomalies because they have links with other communities while other nodes in their community do not form cross-community links. Nodes 2 and 7 are community anomalies because their feature values are different from others in the communities they belong to.

Real-world networks can be modeled as dynamic graphs to represent evolving objects and the relationships among them. In addition to structural information and node attributes, dynamic graphs also contain rich temporal signals [DBLP:conf/wsdm/RossiGNH13], e.g., the evolving patterns of the graph structure and node attributes. On the one hand, these information inherently makes anomalous node detection on dynamic graphs more challenging. This is because dynamic graphs usually introduce large volume of data and temporal signals should also be captured for anomaly detection. But, on the other hand, they could provide more details about anomalies [ranshous2015anomaly, akoglu2015graph, wang2019detecting]. In fact, some anomalies might appear to be normal in the graph snapshot at each time stamp, and, only when the changes in a graph’s structure are considered, do they become noticeable.

In this section, we review the network representation based techniques and GAN based techniques as follows. Relevant techniques from traditional non-deep learning approaches are reviewed in Appendix D.

In contrast to anomalous node detection, which targets individual nodes, ANOS ED aims to identify abnormal links. These links often inform the unexpected or unusual relationships between real objects [chang2021f], such as the abnormal interactions between fraudsters and benign users shown in Fig. 1, or suspicious interactions between attacker nodes and benign user machines in computer networks. Following the previous taxonomy, in this section, we review the state-of-the-art ANOS ED methods for static graphs, and Section 6 summarizes the techniques for dynamic graphs. A summary is provided in Table III. This section includes methods based on deep NNs, GCNs and network representations. The non-deep learning techniques are reviewed in Appendix E.

Dynamic graphs are powerful in reflecting the appearance/disappearance of edges over time [ranshous2016scalable]. Anomalous edges can be distinguished by modeling the changes in graph structure and capturing the edge distributions at each time step. Recent approaches to ANOS ED on dynamic graphs are reviewed in this section.

In real life, anomalies might also collude and behave collectively with others to garner benefits. For instance, fraudulent user groups in an online review network, as shown in Fig. 1, may post misleading reviews to promote or besmirch certain merchandise. When these data are represented as graphs, anomalies and their interactions usually form suspicious sub-graphs, and ANOS SGD is proposed to distinguish them from the benign.

Unlike individual and independent graph anomalies, i.e., single nodes or edges, each node and edge in a suspicious sub-graph might be normal. However, when considered as a collection, they turn out to be anomalous. Moreover, these sub-graphs also vary in size and inner structure, making anomalous sub-graph detection more challenging than ANOS ND/ED [dGraphScan]. Although extensive effort has been placed on circumventing this problem, deep-learning techniques have only begun to address this problem in the last five years. For reference, traditional non-deep learning based techniques are briefly introduced in Appendix F, and a summary of techniques reviewed for ANOS SGD is provided in Table III at the end of Section 9.

Due to the flexibility of heterogeneous graphs in representing the complex relationships between different kinds of real objects, several recent works have taken advantage of deep network representation techniques to detect real-world anomalies through ANOS SGD. For instance, Wang et al. [wang2018deep] represented online shopping networks as bipartite graphs (a specific type of heterogeneous graph that has two types of nodes and one type of edge), in which users are source nodes and items are sink nodes. Fraudulent groups are then detected based on suspicious dense blocks that form in these graphs.

Wang et al. [wang2018deep] aimed to learn anomaly-aware representations of users such that suspicious users in the same group will be located closely in the vector space, while benign users will be far away (as shown in the embedding space in Fig. 8). According to the observation that user nodes belonging to one fraudulent group are more likely to connect with the same item nodes, the developed model, DeepFD, measures similarities in the behavior of two users, $sim_{ij}$, as the percentage of items shared among all the items they have reviewed. User representations are then generated through a traditional autoencoder, which is trained using three losses and follows the encoding-decoding process. The first loss is the reconstruction loss $\mathcal{L}_{res}$ that ensures the bipartite graph structure can be reconstructed properly using the learned user representations and item representations. The second term $\mathcal{L}_{sim}$ preserves the user similarity information in the learned user representations. That is, if two users have similar behaviors, their representations should also be similar. This loss is formulated as:

where $m$ is the number of user nodes, $\widehat{sim}_{ij}$ measures the similarity of user $i$ and $j$’s representations using an RBF kernel or other alternative. The third loss $\mathcal{L}_{reg}$ regularizes all trainable parameters. Finally, the suspicious dense blocks, which are expected to form dense regions in the vector space, are detected using DBSCAN [DBSCAN].

Another work, FraudNE [FraudNE], also models online review networks as bipartite graphs and further detects both malicious users and associated manipulated items following the dense block detection principle. Unlike DeepFD, FraudNE aspires to encode both types of nodes into a shared latent space where suspicious users and items belonging to the same dense block are very close to each other while others distribute uniformly (as shown in Fig. 8). FraudNE adopts two traditional autoencoders, namely, a source node autoencoder and a sink node autoencoder, to learn user representations and item representations, respectively. Both autoencoders are trained to jointly minimize their corresponding reconstruction losses and a shared loss function, and the total loss can be formulated as:

where $\alpha$ and $\eta$ are hyperparameters, and $\mathcal{L}_{reg}$ regularizes all trainable parameters. Specifically, the reconstruction losses (i.e., $\mathcal{L}_{res}^{source}$ and $\mathcal{L}_{res}^{sink}$) measure the gap between the input user/item features (extracted from the graph structure) and their decoded features. The shared loss function is proposed to restrict the representation learning process such that each linked pair of users and items get similar representations. As the DBSCAN [DBSCAN] algorithm is convenient to apply for dense region detection, FraudNE also uses it to distinguish the dense sub-graphs formed by suspicious users and items.

To date, only a few works have put their efforts into using deep learning techniques for ANOS SGD. However, with intensifying research interest in sub-graph representation learning, we encourage more studies on ANOS SGD and highlight this as a potential future in Section 11.1.

Beyond anomalous node, edge, and sub-graph, graph anomalies might also appear as abnormal graphs in a set/database of graphs. Typically, a graph database is defined as:

Definition 4 (Graph Database). A graph database $\mathcal{G}=\{G_{i}=(V_{i},E_{i},X_{v}(i),X_{e}(i))\}^{N}_{i=1}$ contains $N$ individual graphs. Here, each graph $G_{i}$ is comprised of a node set $V_{i}$ and an edge set $E_{i}$. $X_{v}(i)$ and $X_{e}(i)$ are the node attribute matrix and edge attribute matrix of $G_{i}$ if it is an attributed graph.

This graph-level ANOS GD aims to detect individual graphs that deviate significantly from the others. A concrete example of ANOS GD is unusual molecule detection. When chemical compounds are represented as molecular/chemical graphs where the atoms and bonds are represented as nodes and edges [6702420, sun2021sugar], unusual molecules can be identified because their corresponding graphs have structures and/or features that deviate from the others. Brain disorders detection is another example. A brain disorder can be diagnosed by analyzing the dynamics of brain graphs at different stages of aging in sequence and finding an inconsistent snapshot at a specific time stamp.

The prior reviewed techniques (i.e., ANOS ND/ED/SGD) are not compatible with ANOS GD because they are dedicated to detecting anomalies in a single graph, whereas ANOS GD is directed at detecting graph-level anomalies. This problem is commonly approached by: 1) measuring the pairwise proximities of graphs using graph kernels [manzoor2016fast]; 2) detecting the appearance of anomalous graph signals created by abnormal groups of nodes [hooi2018changedar]; or 3) encoding graphs using frequent motifs [noble2003graph]. However, none of these methods are deep learning-based. As the time of writing, very few studies in ANOS GD with deep learning have been undertaken. As such, this is highlighted as a potential future direction in Section 11.1.

For dynamic graph environments, graph-level anomaly detection endeavors to identify abnormal graph snapshots/temporal graphs. Similar to ANOS ND and ED on dynamic graphs, given a sequence of graphs, anomalous graphs can be distinguished regarding their unusual evolving patterns, abnormal graph-level features, or other characteristics.

In order to derive each graph snapshot/temporal graph’s characteristics, the commonly used GNN, LSTM and autoencoder are feasible to apply. For instance, Teng et al. [teng2018deep] applied a LSTM-autoencoder to detect abnormal graph snapshots, as shown in Fig. 10. In their proposed model, DeepSphere, a dynamic graph is described as a collection of three-order tensors, $\{\mathcal{X}_{k},k=1,2...\}$ where each $\mathcal{X}\in\mathcal{R}^{N\times N\times T}$, and the slices along the time dimension are the adjacency matrices of graph snapshots. To identify abnormal tensors, DeepSphere first embeds each graph snapshot into a latent space using an LSTM autoencoder, and then leverages a one-class classification objective [ruff2018deep] that learns a hypersphere such that normal snapshots are covered, and anomalous snapshots lay outside. The LSTM autoencoder takes the adjacency matrices as input sequentially and attempts to reconstruct these input matrices through training. The hypersphere is learned through a single neural network layer and its objective function is formulated as: