Rumor Detection with Hierarchical Representation on Bipartite Adhoc Event Trees

Researchers in different disciplines have investigated rumor detection, including philosophy and humanities [23], social psychology [24, 25], computer science and artificial intelligence [26, 27]. It is difficult to identify rumors early on because a large number of news investigations are required to verify suspicious tweets. With the increased usage of social media, there is a massive information overload, making rumor detection a time-consuming and labor-intensive task. Early studies focused on using manual linguistic features extracted from rumor textual contents (source/root and responsive posts) to decide rumor labels (i.e., true or false) via traditional machine learning methods [28, 29, 30]. For example, Zhao et al. [28] believed that people exposed to a rumor claim would seek more information or express skepticism before deciding whether to believe it or not. Thus, he clustered the claim and its successor texts which contain doubtful expressions such as “Is this true?” and “Really?”, and then used classical classification models such as SVM and decision tree to conduct the classification task. Ma et al. [17] aggregated all of the responses to the root claim to form a rumor propagation tree and then identified the claim using SVM with different kernel numbers.

These methods rely on manually created features, which can not guarantee the accuracy of rumor classification. To achieve real-time tracking and identification of rumors, researchers must shift their focus to automatic rumor detection methods. The initial research on the automatic rumor detection method is based on text mining with supervised models built on feature engineering. For instance, Ma et al. [31] proposed a dynamic time series structure (DSTS) that can capture a variety of social context text features based on time series changes, as well as the various features generated by rumors along with time, allowing rumors to be detected in different periods. Rath et al. [32] proposed a new trustworthiness concept based on responsive post characteristics to identify authors who spread rumors automatically. Because of the large amount of noise information, such feature engineering-based methods extract more single or coarse-grained features, and the accuracy of the classification task cannot be guaranteed.

Deep learning research solves the automatic feature extraction problem while achieving fine granularity and diversity. Researchers are no longer limited to the single processing and use of features because of richer and more accurate feature information. However, they are instead exploring the fusion of multiple features and structural modeling.

Serialized propagation topology representation. Some methods place less emphasis on the propagation topology and instead focus on propagation time sequence properties. These researchers hoped to explore relevant clues generated over time to characterize the different characteristics of the rumor at different times by modeling the propagation process of information with time characteristics. Yu et al. [33] designed a CNN-based model called CAMI to extract flexible content sequence between key characteristics, formation of high-level interactions, and aids in the effective classification. Extensive mixed features, such as time series and content-related features, are increasingly important to research. Thus Ma et al. [14] used an RNN network to process each time step in the rumor propagation sequentially. Chen et al. [15] proposed an attention mechanism to capture longer-term dependencies and enhance the long-term temporal features [34]. In addition, attention networks [20] and Transformers [35] were adopted to better mine the posts’ features with time sequences as a clue to identify rumors.

Author traits, on the other hand, are frequently used to define authors’ personality information, such as authors’ profile information and social relationships, which are used to vividly describe the authors’ involvement in propagation and their behavior [16]. Shu et al. [36] argued that fake news could imitate real news, which means that the dissemination of authors’ information must be thoroughly researched. Ruchansky et al. [37] developed a tri-relationship embedding framework TriFN that can learn the characteristics of forwarded texts and author profiles while also generating credibility scores to help with rumor detection. Shu et al. [38] built five novel metrics to interactively model the relationships between news and users (publishers, social participants, etc.). Recently, Wu et al. [21] constructed a decision tree mechanism to address rumor interpretability issues, using users’ credibility scores as evidence to help identify rumors. Lu et al. [22] built a graph-aware joint attention network to detect fake news by taking into account the root article’s text content (claim), forwarding author sequence (sequential propagation) and author information (author profile). These methods, however, cannot accurately reflect the characteristics of actual rumor propagation and spreading because they focus on mining the characteristics of authors and posts changing over time via simple temporality while ignoring the internal topology of rumor propagation.

Structured propagation topology representation. The rumor propagation topology representation is concerned with modeling the rumor propagation structure, gathering various information such as stances and sentiments in the topology, and mining the interactive information of multiple features of posts and authors to enrich the rumor representation. Simultaneously, understanding the process and mode of rumor propagation aids in identifying rumors at an early stage of rumor propagation and achieves a good explanatory effect in detecting the authenticity of rumors. Rumor propagation topology enables us to better abstract rumor propagation patterns and further excavate the correlations between hybrid features via propagation aggregation [39, 40, 41], resulting in good interpretability on the veracity of rumors. Wu et al. [16] calculated the substructural similarity of two propagation trees on Sina Weibo datasets for detection. Similarly, Ma et al. [17] employed the tree kernel to compute similar substructures and identify various types of rumors in Twitter datasets. The above two methods rely on the similarity of the two trees. But they cannot distinguish between individual trees and are still based on feature engineering rather than deep learning. Moreover, considering that RvNN [42, 43] were originally used to learn phrase or sentence representations for semantic parsing, Wong et al. [18] proposed two types of recursive tree-structured neural network models to learn representations from their structure and contents. Ma et al. [44] recently developed an improved rumor detection model based on Wong et al. [18] with designed discriminative attention mechanisms for model optimization.

Bian et al. [19] believed that Wong et al. [18] only considered propagation depth, ignoring spread. Two graph convolutional networks (GCN) were used to learn the top-down propagation depth and the down-top disperse breadth simultaneously. Wei et al. [20] is also based on GCN networks and considers the reliability of the rumor propagation relationship when optimizing the propagation model for a better rumor classification effect. Graph attention networks (GAT) are also used in various research fields. Lin et al. [45] proposed a new ClaHi-GAT model to represent the action graph between the post content and the author and learn the characteristics of rumors by the multi-layer attention mechanism of the root post layer and event layer. In addition, Liu et al. [46] proposed a novel rumor detection framework based on structure-aware retweeting GNNs based on a converted tractable tree. Wei et al. [47] further considered the uncertainty of interactions caused by users’ various subjective factors and proposed fuzzy graph convolutional networks to model uncertain interactions in the information cascade of a tweet graph. Although such methods are constantly improving the modeling of rumor propagation topological structure to obtain better rumor representations, most ignore author-level information, including simple user personality characteristics (author profiles) and the structured interaction relationship formed between users during the spread process.

We employ the node-level representation module in both the post tree and author tree to learn a more informative representation of each node from the perspectives of semantics (content) and interactive (propagation) correlations in the bipartite tree. Specifically, this module consists of two sub-modules: Tree Nodes Process (TNP) and Root-aware Attention Learning (RAL). TNP is introduced to perform embeddings of post nodes and author nodes. The embeddings are then fed into the RAL model to learn the correlations between words/author features within each post/author and the semantic/interactive correlations between each responsive post/author and the claim (root) post/author.

(1) Tree Node Process (TNP). We first introduce Tree Node Process in the post tree to capture the text information of all posts in an adhoc event. The topological post tree obtained in Fig. 2 is processed along the tree structure, where all post text information and topological structure in the tree are recorded. Notably, the topological structure in the tree mainly consists of root nodes, non-leaf nodes, and leaf nodes. We denote the maximum length of the post text as $L_{m}$ and collect all words in all post texts as a corpus. All words are then tokenized such that each word can be represented by a corpus index. Formally, all the tokenized texts are initialized as $\{\widetilde{\mathbf{t}}_{0},\widetilde{\mathbf{t}}_{1},\widetilde{\mathbf{t}}_{2},...,\widetilde{\mathbf{t}}_{n}\}$ with $\widetilde{\mathbf{t}}_{i}\in\mathbb{R}^{L_{m}\times 1}$ and mapped into a low-dimensional space through an embedding layer as follows:

where $\widetilde{\mathbf{E}}_{i}\in\mathbb{R}^{L_{m}\times d},i\in[0,n]$, and $d$ is the dimension of embedding. We further count the frequency of each word appearing in its post and obtain the frequency vectors of all posts as $\{\mathbf{q}_{0},\mathbf{q}_{1},\mathbf{q}_{2},...,\mathbf{q}_{n}\}$ with the word frequencies of $i$-th post $\mathbf{q}_{i}\in\mathbb{R}^{1\times L_{m}}$ and $i\in[0,n]$. Then, we get the frequency-enhanced embeddings of posts as below:

where $*$ denotes the element-wise product, and $\mathbf{E}_{i}\in\mathbb{R}^{L_{m}\times d}$. Accordingly, we obtain the embeddings of all the posts in the post tree: $\{\mathbf{E}_{0},\mathbf{E}_{1},...,\mathbf{E}_{n}\}$.

Regarding the author nodes, we have two types of author features, i.e., the author’s basic features and the authors’ writing habits, as shown in Table II. Specifically, “similarity to claim” is calculated using Jaccard similarity, expressing the relevance between the repost and the claim post, “post text length” records the number of words in the post, “max post text length” is the maximum of text length in the tree, and “average word length” reveals the author’s habits in terms of the average length of words in his/her posts.

We perform feature encoding for each type of feature separately and then adopt one-hot encoding to transform categorical features such as “verified”, “geo-enabled”, and “time-zone enabled”. In addition, the timestamp features (i.e., “time of posting” and “time of creating an account” are replaced with timestamp intervals calculated as follows:

where $s_{0}$ and $s_{i}$ correspond to the timestamps of the root post and the $i$-th responsive post, respectively. Accordingly, we obtain two matrices of raw basic features and writing-habit features for all authors. The two matrices are then normalized to $[0,1]$ using the min-max normalization separately, resulting in two normalized feature matrices $\mathbf{F}^{b}=[\mathbf{f}^{b}_{0},\mathbf{f}^{b}_{1},...,\mathbf{f}^{b}_{n}]$ and $\mathbf{F}^{h}=[\mathbf{f}^{h}_{0},\mathbf{f}^{h}_{1},...,\mathbf{f}^{h}_{n}]$ where $\mathbf{f}^{b}_{i}$ and $\mathbf{f}^{h}_{i}$ denote the normalized feature vectors of the basic features and writing-habit features of the $i$-th author (i.e., $a_{i}$), respectively. Finally, we can obtain the feature representation for each author calculated as follows:

where $\bm{F}_{i}\in\mathbb{R}^{(|\mathbf{f}^{b}|+|\mathbf{f}^{h}|)\times d}$ denotes the representation embedding of author $a_{i}$, $\bm{W}^{b}\in\mathbb{R}^{1\times d}$ and $\bm{W}^{h}\in\mathbb{R}^{1\times d}$ denote the transform matrices, and $\oplus$ denotes the concatenation operation. Finally, we obtain the feature representations of all authors $\{\bm{F}_{0},\bm{F}_{1},...,\bm{F}_{n}\}$.

(2) Root-aware Attention Learning (RAL). We propose a root-aware attention learning module consisting of two attention modules to learn comprehensive node representations for both the post tree and the author tree. As shown in Fig. 3 (a), we use $\{\mathbf{V}_{0},\mathbf{V}_{1},...,\mathbf{V}_{n}\}$ to denote the representation matrices obtained from either the post tree or author tree via the tree nodes process module, i.e., $\mathbf{V}_{i}=\mathbf{E}_{i}$ or $\mathbf{V}_{i}=\mathbf{F}_{i}$.

First, to learn the correlations between words (features) within a post (author) node, we leverage the self-attention mechanisms to enhance the representations obtained through the Tree Nodes Process module to capture the semantics or features correlations within each post/author node:

where $\bm{W}^{Q},\bm{W}^{K},\bm{W}^{V}\in\mathbb{R}^{d\times d}$ are projection weight parameters, $\mathcal{A}(\mathbf{Q},\mathbf{K},\mathbf{V})=\operatorname{Softmax}(\frac{\mathbf{Q}\mathbf{K}^{T}}{\sqrt{d_{k}}})\mathbf{V}$ denotes the calculation of self-attention, and $\widetilde{V}_{i}$ denotes the output of self-attention.

Secondly, with the increase in the number of participants in actual rumor/non-rumor propagation, the claim post information and its author’s characteristic information in the root node may be weakened or forgotten in circulation. To address this, we introduce another root-aware attention module to learn the interactive correlations between the root node and each responsive node:

where $\bm{W}^{a}\in\mathbb{R}^{2d\times d}$ are projection weights, $\mathbf{b}^{a}\in\mathbb{R}^{d}$ is the bias term, $\widetilde{\mathbf{V}}_{0}$ and $\widetilde{\mathbf{V}}_{i}$ are the self-attention representations of the root node and the $i$-th responsive node, respectively, and $\widehat{\mathbf{V}}_{i}$ is the attention output attending to both the two nodes.

Finally, we fuse the outputs of the two attention modules:

where $\mathbf{U}_{0}$ and $\mathbf{U}_{i}$ denote the fused representations of the root node and the $i$-th responsive node, respectively. In addition, $\mu$ is a hyper-parameter used to determine the extent of how much interactive correlation between the root node and responsive node should be considered. Subsequently, substituting $\mathbf{V}_{i}=\mathbf{E}_{i}$ or $\mathbf{V}_{i}=\mathbf{F}_{i}$ to the above Equations 5-8 separately, we can obtain the representations of all post nodes and author nodes in the bipartite trees which are denoted as $\{\mathbf{S}_{0},\mathbf{S}_{1},...,\mathbf{S}_{n}\}$ and $\{\mathbf{O}_{0},\mathbf{O}_{1},...,\mathbf{O}_{n}\}$, respectively.

All node-level representations learned above are fed into the Structural-level Representation module to capture a comprehensive representation of both the post tree and author tree. The module consists of two sub-modules: Tree Variant RNN (TRvNN) and Tree-aware Attention Learning (TAL). TRvNN is a structure-aware GRU layer that simulates the hidden transitions according to the connections in the tree, while TAL aggregates the output hidden states from TRvNN to generate the representation vectors via a tree-aware attention mechanism.

(1) Tree Variant RNN (TRvNN). We incorporate the propagation tree topology to RNN to form TRvNN, a tree-variant RNN inspired by [18]. To guarantee computation efficiency in modeling the long propagation tree, we choose GRU cells with one node corresponding to one GRU cell.

Specifically, $\{\mathbf{S}_{0},\mathbf{S}_{1},...,\mathbf{S}_{n}\}$ and $\{\mathbf{O}_{0},\mathbf{O}_{1},...,\mathbf{O}_{n}\}$ are served as inputs to TRvNN. We assume that all information from a non-leaf node in the post tree or author tree can be passed to all of its descendant nodes. Therefore, the hidden state of the GRU cell is calculated by combining the representations of the current node and the hidden state of its direct parent node. Formally, given all node representations of the post tree or author tree $\{\mathbf{U}_{0},\mathbf{U}_{1},...,\mathbf{U}_{n}\}$, i.e., $\mathbf{U}_{i}=\mathbf{S}_{i}$ or $\mathbf{U}_{i}=\mathbf{O}_{i}$, the GRU cell in our TRvNN is calculated as follows:

where $P(i)$ is the parent of the $i$-th node in either the post tree or author tree, $\sigma_{r},\sigma_{z}$ are the activation functions (precisely $\operatorname{Sigmoid}$ used in the experiments), $\bm{W}^{U}\in\mathbb{R}^{d\times d}$ and $b^{X}\in\mathbb{R}^{d}$ denote the transformation weights and the bias term, respectively, and $\mathbf{W}^{r},\mathbf{W}^{z},\mathbf{W}^{H}\in\mathbb{R}^{2d\times d}$ are the projection weights for calculating the reset gate $\bm{r}_{i}$, the update gate $\bm{z}_{i}$ and the candidate activation $\widetilde{\mathbf{h}}_{i}$, respectively. Accordingly, we can obtain all output hidden state $\{\bm{h}_{i}\}_{i=0}^{n}$ from TRvNN, resulting in $\{\bm{h}^{p}_{i}\}_{i=0}^{n}$ and $\{\bm{h}^{a}_{i}\}_{i=0}^{n}$ corresponding to all post nodes and author nodes, respectively.

(1) Tree-aware Attention Learning (TAL). To aggregate the output hidden state from TRvNN and capture structural information from the post/author tree, we propose a tree-aware attention learning module. This module leverages attention mechanisms and adopts $\{\bm{h}^{p}_{i}\}_{i=0}^{n}$ and $\{\bm{h}^{a}_{i}\}_{i=0}^{n}$ as inputs to obtain the final structural representations of the post tree and author tree. For simplicity, we introduce the module using $\{\bm{h}_{i}\}_{i=0}^{n}$, as shown in Fig. 3(b).

First, based on the topological structure, we divide the obtained hidden state representations into three categories: the hidden states of the root node, leaf nodes, and other nodes, which are denoted by dotted boxes of diffident colors as shown in Fig. 3(b). The treatment aims to show that different kinds of nodes in the post tree or author tree contribute differently to spreading rumors. That is, we distribute different weights to different kinds of nodes to learn tree-aware representations of the post/author tree:

(a) The leaf nodes reflect the information aggregation process from the root node to the current leaf nodes along with the tree structure. Intuitively, the emotional intensity and propagation flow in the path is all gathered in the leaf nodes.

(b) The other nodes act on the paths from the root node to the leaf nodes. Their node representations in the current path may influence the other paths. For example, the negative emotion in the current node may affect the expression of the nodes on other paths.

(c) The root node contains the most important information in the propagation. However, during recursive learning, the information in the root node may be weakened or forgotten as the path length increases.

Following the above node division, all leaf nodes are fed into a max-pooling layer to obtain the maximum value of each vector element of all leaf nodes, thereby representing the representation of all leaf nodes. Intuitively, this treatment can account for and capture the most attractive and indicative representation of all propagation paths:

where $\mathbf{h}_{l1},\mathbf{h}_{l2},\mathbf{h}_{l3},\cdots$ denote the hidden states of the leaf nodes, the operator $\operatorname{MaxPooling}$ returns element-wise maximum values on $\{\mathbf{h}_{l1},\mathbf{h}_{l2},\mathbf{h}_{l3},\cdots\}$ over $\mathbb{R}^{d}$, and the resultant $\mathbf{h}_{L}\in\mathbb{R}^{d}$ denotes the maximum representation on all the leaf nodes. Accordingly, we calculate the correlation between this maximum representation and the root node representation and obtain an enhanced representation for the roof node $\mathbf{h}_{E}$:

where $\mathbf{h}_{0}$ denote the hidden state representation of the root node, $\sigma_{E}$ is the activation function ($\operatorname{tanh}$ used in the experiments), $\bm{W}^{E}\in\mathbb{R}^{2d\times d}$ and $\mathbf{b}^{E}\in\mathbb{R}^{d}$ are the weight parameters and bias term, respectively. On the one hand, root node information can be fused with indicative maximum leaf node information to perform a strong combination via Equation 11. On the other hand, it can compensate for the loss of root node information due to a long propagation.

Subsequently, the representation $\mathbf{h}_{E}$, along with other node hidden states $\{\bm{h}_{i}\}_{i=1}^{n}$, will be fed into a self-attention network. We expect to learn their correlations and capture the contribution of each node to the final tree structure representation. Let $\mathbf{h}_{i}$ be the $i$-th node hidden states, where $i\in[0,n]$ and $\mathbf{h}_{0}=\mathbf{h}_{E}$. We calculate the attentive weight $\alpha_{i}$ for each $\mathbf{h}_{i}$:

where the parameter vector $\mathbf{w}_{c}\in\mathbb{R}^{d}$ can be treated as the learnable weight vector of a class token for rumor identification. Accordingly, we obtain the representation for the post tree or author tree via the attentive weighted summation:

Accordingly, we can obtain $\mathbf{h}_{post}$ and $\mathbf{h}_{author}$ according to Equation 13 for the post tree and author tree, respectively.

To obtain comprehensive representations, we concatenate the representations of the post tree $\mathbf{h}_{post}$ and author tree $\mathbf{h}_{author}$, fusing the shared features of the two trees:

where $\mathbf{h}_{o}\in\mathbb{R}^{2d}$ denotes the output representation for prediction. Since we define the rumor detection task as a binary classification problem, we utilize a binary label $y\in[0,1]$ as the ground truth. To predict the label of the claim (the root node of the post tree), we feed $\mathbf{h}_{o}$ into a full-connection layer with the Softmax activation as follows:

where $\bm{W}^{y}\in\mathbb{R}^{2d\times 2},\mathbf{b}^{y}\in\mathbb{R}^{2}$ are parameter weights and the bias term. Our model is trained to minimize the cross-entropy loss between each prediction $\hat{y}$ and the ground truth $y$ as below:

where $\mathcal{L}$ denotes the cross-entropy loss, $\Theta=\{\{\mathbf{W}\},\{\mathbf{b}\}\}$ denotes the set of trainable weight parameters, and $\Omega=\{d,\mu\}$ is the set of hyperparameters.

Datasets. We adopt two publicly available datasets, i.e., PHEME¹¹1https://figshare.com/articles/dataset/PHEME_dataset_of_rumours_and_non-rumours/4010619 [48] and RumorEval²²2https://alt.qcri.org/semeval2017/task8/index.php?id=data-and-tools [49], for evaluation. Detailed statistics of these two datasets are provided in Table III. Both datasets are real-world data collected from the social platform Twitter, and the RumorEval dataset was developed for the SemEval-2017 Task 8 competition. The PHEME dataset contains five breaking news events, while RumorEval involves eight breaking news events. Both datasets contain post texts, author information, timestamps, and propagation information. Each claim is labeled with a binary class (i.e., True Rumor or False Rumor). In addition, during data preprocessing of the PHEME data, we removed noisy samples that have less than 3 retweets or do not have author information associated with the authors (IDs) of the claim post or retweet post. All comparative models are evaluated on the two datasets to provide an extensive and fair comparison.

Baselines. We conduct a comprehensive comparison of our model with state-of-the-art rumor detection baselines.

• •

  DTR [28]: A decision tree-based ranking model proposed to identify trending rumors through searching query phrases.

• •

  CNN [30]: A convolutional neural network with multiple filter sizes that is used to detect the stance of posts and determine the veracity of specific rumors.

• •

  SVM [31]: A linear SVM classifier that is used to identify rumors by leveraging temporal sequences to model the chronological variation of social context features.

• •

  TE [50]: A content-based model that leverages tensor decomposition to derive concise article embeddings, capture contextual information and create an article-by-article graph on limited labels.

• •

  RvNN [18]: A top-down (down-top) model based on tree-structured RvNN proposed to detect rumors through integrating structure and content. We choose the well-performed top-down model as our baseline.

• •

  DTCA [21]: A decision tree-based co-attention model developed to detect rumors by discovering evidence for explainable verification, in which the evidence scores combine both authors’ and texts’ information.

• •

  RvNN-GA [44]: An enhanced rumor detection model proposed with designed discriminative attention mechanisms based on TD-RvNN/BU-RvNN. In this work, we select the top-down global attention model as a baseline.

• •

  Bi-GCN [19]: A bi-directional graph model based on two-layer GCNs to detect rumors through integrating structural features of both propagation depth (top-down) and dispersion (down-top).

• •

  ClaHi-GAT [45]: A graph attention network model is proposed to represent posts and authors as an undirected graph and then augmented by a multi-layer attention mechanism to learn the rumor indicator characteristics.

• •

  EBGCN [20]: An edge-enhanced rumor detection model proposed with multiple graph conventional layers to iteratively aggregate features of structural neighbors and consider the uncertainty issue.

• •

  RGNN [46]: A rumor detection framework based on structure-aware retweeting graph neural networks to integrate content, users, and propagation patterns from a converted tractable binary tree to enhance performance.

• •

  FGCN [47]: A neuro-fuzzy method with fuzzy graph convolutional networks to capture uncertain interactions in the information cascade in a fuzzy perspective based on the tree or tree-like structure of source tweets.

These representative and state-of-the-art baselines are chosen carefully for extensive comparisons. 1) DTR, CNN, and SVM are representative baselines based on handcrafted features, and TE is a content-based detection model. Those baselines are compared to investigate the effectiveness of propagation information in the detection. 2) The other state-of-the-art baselines include RNN-, attention-, and GNN-based models that elaborately leverage propagation information to enhance detection. Those baselines are selected to investigate the benefits of our proposed bipartite adhoc trees, especially involving the propagation information from the author’s perspective. In addition, multi-modality detection models are not considered in the experiments since those methods based on multi-modality information of text and image inputs do not belong to the family of propagation-based models.

Experimental settings. (1) Parameter settings: In the experiments, all parameters are initialized with a normal distribution, and we perform a grid search of the learning rate over $\{1e^{-3},5e^{-3},1e^{-4},5e^{-4},1e^{-5}\}$ and L2 regularization rate over $\{0,1e^{-1},1e^{-2},1e^{-3},1e^{-4}\}$ to guarantee the best performance of our proposed BAET. For hyperparameters $\Omega={d,\mu}$, we tune the dimension of the embedding and hidden units $d$ over $\{16,32,64,128,256\}$ and $\mu$ over $\{0.0,0.2,0.4,...,1.0\}$ to choose the values with the best performance for BAET. Finally, we set i) the embedding dimension $d=128$; ii) a balance weight $\mu=0.6$; iii) the ADAM optimizer [51] with an initial learning rate of $0.005$; iv) a batch size of $16$; v) L2 regularization rate of $1e^{-3}$; vi) a dropout rate of $0.5$. (2) Baseline settings: Regarding the baselines, we implement RvNN, RvNN-GA, BiGCN, and EBGCN according to their published source codes, while other baselines are implemented by referring to the corresponding papers. To ensure fair comparisons, the best parameters for all comparative models are chosen by carefully tuning the author-recommended parameter ranges based on their recommended settings. Specifically, for the parameters in each baseline, we first initialize them with the values reported in the original paper and then carefully tune them on our datasets for best performance. (3) Metrics: We adopt four metrics accuracy, precision, recall, and F1-score to evaluate all comparative rumor detection models. We conducted the experiments using five-fold cross-validation and report the average evaluation results. (4) Platforms: All experiments were implemented in PyTorch and were conducted on two NVIDIA 1080 12G GPUs.

In Table IV, we present the performance of different baseline models in comparison with our model based four evaluative metrics: accuracy, precision, recall and F1-score. The best results are highlighted in bold, and the best baseline results are underlined. From the table, we can make the following observations.

(1) DTR, CNN, and SVM are rumor detection baselines based on handcrafted features. SVM can achieve superior results among methods based on handcrafted features. However, we attribute these results to the fact that SVM integrates temporal with textual information and, in doing so, simulates information transmission to a certain extent, which increases the relevance between texts.

(2) TE aims to mine rumor text features more thoroughly using social context. However, the table shows that the accuracy of the TE model is significantly lower (nearly 7%) than that of the SVM model, indicating that multi-type feature fusion and information transmission topology may still be considered to achieve better performance.

(4) The proposed RvNN architecture was shown to provide a more convincing propagation tree structure for rumor detection. In this sense, RvNNs can better combine rumor propagation features and improve classification performance. On the other hand, the DTCA architecture achieved good performance by relying on a decision tree mechanism to detect rumors by establishing evidence scores while ignoring propagation information. RvNN-GA, with its unique attention mechanism, also achieves significant performance improvements compared with RvNN. However, these two models (RvNN, RvNN-GA) model the propagation topology solely on text contents, ignoring the characteristics of author groups involved in rumor spreading. Meanwhile, the exploration of interactive features between texts is insufficient.

(5) The proposed propagation tree structure broadens the scope of rumor detection research, and researchers focused on modeling the spread of the topological structure in more ways. BiGCN and EBGCN model the depth and breadth of propagation using graph convolution networks, which has yielded desirable results. Meanwhile, the EBGCN model proposes to investigate the uncertainty of propagation relations, which improves the model’s interpretability.

(6) The baselines RGNN and FGCN also achieve desirable performance (the best two baselines). This is reasonable since RGNN and FGCN introduce GNN to capture the post text, author information, and propagation information. However, unlike RGNN and FGCN, BAET introducing bipartite adhoc event trees in terms of both posts and authors achieves better performance. The results demonstrate the effectiveness of introducing the author tree and modeling more comprehensive authors’ information for rumor detection.

(7) On these datasets, we can see that our proposed BAET outperforms other benchmarks, proving that author information is effective for judging the veracity of the corresponding posts ”Birds of a feather propagate together” and our model structure is valid. Simultaneously, it is demonstrated that modeling propagation structure from the perspectives of texts and authors can mine the features gathered through propagation relations, and novel interactive learning modules are introduced to generate a more comprehensive representation of the claim to be detected.

To verify the contribution of each component of BAET, we perform ablation studies on PHEME and RumorEval, respectively, by discarding certain important components of BAET. Note that our BAET consists of two parallel components for the post tree and the author tree, respectively. Consequently, we perform the ablation study on the post-based and author-based components separately, denoted as post-based ablation and author-based ablation, respectively.

Both post-based and author-based components consist of two modules for node-level representation (including two successive sub-modules, TNP and RAL) and structural-level representation (including two successive sub-modules, TRvNN and TAL), which are necessary to learn to represent the bipartite adhoc event trees for detecting rumors. To ensure effective ablation studies, all variants of BAET keep the two-level structure, that is, at least one sub-module at either node-level or structure-level. Specifically, we compare BAET with its seven variants in the post-based and author-based ablation studies, respectively. We illustrate the variants in the post-based (author-based) ablation study as follows:

• •

  w/o AT(PT): We discard the author (post) tree from BAET, namely using $\mathbf{h}_{post}$ ($\mathbf{h}_{author}$) only for detection.

• •

  w/o TNP&TRvNN: We remove TNP and TRVNN from the node-level representation and structural-level representation modules of BAET, respectively. Accordingly, only the two attention sub-modules, i.e., RAL and TAL, are maintained to learn node/structural representations.

• •

  w/o RAL&TAL: We remove the two attention sub-modules, i.e., RAL and TAL, from the node-level representation and structural-level representation modules of BAET, respectively, and feed the node embeddings from TNP to the tree-structural TRvNN to learn to detect.

• •

  w/o TNP: We skip the sub-module TNP from the node-level representation module of BAET and feed the raw word embedding embeddings (author feature embeddings) into RAL to the node representations.

• •

  w/o RAL: We skip the sub-module RAL from the node-level representation module of BAET and feed the node embeddings from TNP directly into the structural-level representation module to detect.

• •

  w/o TRvNN: We remove TRvNN from the structural-level representation module of BAET and feed the node representations into TAL to learn structural representations for detection.

• •

  w/o TAL: We remove TAL from the structural-level representation module of BAET and output the last hidden state from TRvNN for detection.

We report the results of the post-based and author-based ablation studies in terms of accuracy in Fig. 4 and Fig. 5, respectively. From the two tables, we observe that the proposed BAET outperforms all other variants in both datasets, indicating the contributions of each module to rumor detection and verifying the rationality and effectiveness of our model design in BAET. Specifically, we have the following observations.

(1) The accuracy of the variants w/o AT and w/o PT is inferior to that of BAET, demonstrating that modeling propagation structure using only posts or authors is insufficient. The results verify that the bipartite adhoc event trees comprehensively summarize the rumor propagation characteristics and that the circulation of post texts and author features contribute to modeling rumor propagation.

(2) The variants w/o TNP&TRvNN and w/o RAL&TAL fall behind BAET, indicating that TNP&TRvNN and RAL&TAL complement each other and contribute to modeling the propagation information in both the post tree and author tree. The attention modules (RAL&TAL) help discover post (author) correlations in circulation and learn more informative representations, while TNP and TRvNN involve post/author features and reveal/embed the topological structure, respectively. In addition, the variant w/o TNP&TRvNN performs worse than w/o RAL&TAL, reflecting the significance of the propagation structure for rumor detection.

(3) The performance degradation of the remaining four variants compared with BAET, reflecting that the four sub-modules play different effects and complement each other for rumor detection. Specifically, TNP aims to model post text embeddings or author feature embeddings, RAL learns node representations attending to the root node, and TRvNN and TAL capture the topological structure and post (author) correlations, respectively. These designs effectively enhance the representations of rumor propagation from the node and structural levels.

(4) Finally, the performance of the variants in the Fig. 5 is comparable with (slightly worse than) that of the variants in Fig. 4. The results indicate that the author characteristics in the author tree are beneficial for the task of rumor detection. In particular, the users’ social attributes and writing habits are involved in learning author node representations. Empirically, these author features facilitate capturing authors’ influence (such as reputation and reliability) on rumor propagation and significantly contribute to judging the veracity of rumors.

(1) First, we investigate the degree and difference to which information from the two interactive modes need to be considered in the Node-level Representation Module in Fig. 3 (a) using hyper-parameters $\bm{\mu}$. To complete it, we use Equation 7 and Equation 8. Fig. 6 plots the Accuracy and F1 scores under different $\bm{\mu}$ values in Root-aware Attention Learning (RAL). From the figure, we can conclude that $\bm{\mu}$ of $0.4$ on PHEME and $0.6$ on RumorEval yield the best performance. According to the dataset introductions in Table III, we speculate that because the PHEME dataset contains a greater number of responsive posts, more propagation information can be captured, and thus the dependency degree of claim information from the claim is slightly lower than that of RumorEval. This demonstrates that the effects of responsive information and claim information on rumor detection results are complementary, and our model also effectively handles the fusion of the two types of them. Furthermore, when ignoring the root-aware attention layer, i.e., $\bm{\mu}=0.0$, the performance of BAET gets worse, indicating that ignoring the correlations between the claim node and its successors yields a poor performance of BAET. The results verify the effectiveness and necessity of our proposed root-aware attention layer. Moreover, as $\bm{\mu}$ increases, the performance initially improves, while this does not mean that a bigger value of $\bm{\mu}$ is always better. If $\bm{\mu}$ exceeds a certain value, the model performance exhibits a downward trend, indicating that the correlation information obtained by two correlation modes may conflict with one another.

(2) Moreover, Fig. 7 shows the performance of our model under different dimension size $d$. As shown, our model performance improves as dimension size $d$ increases. We hypothesize that increasing the embedding and hidden unit dimensions will result in more multidimensional and comprehensive representations of both post texts and author features and that these representations will provide richer information for model training. At the same time, the multi-hierarchical representation learning model that we developed is capable of capturing more interactive information between dimensions. Because our model achieves the best performance on both datasets when $d=128$, we choose 128 as the dimension of embedding and hidden unit. When the dimension size, on the other hand, continues to grow, the model performance gradually improves or worsens. We believe that indefinitely increasing the representation dimension is ineffective. With the increasing dimensions, the model relies on more interactive dimension calculation information, which may increase the possibility of noise information and thus affect the model’s detection effect.

To further investigate the importance of the propagation process for rumor detection, we evaluate BAET with five propagation-based baselines (i.e., FGCN, RGNN, EBGCN, BiGCN, and RvNN) under the different number of retweets or forwarding (i.e., responsive post counts) to which a claim post. We report the accuracy of the comparative models on the PHEME dataset in Fig. 8. As the number of posts increases from $0$ to $150$, our proposed BAET still outperforms all the baselines, indicating the superiority of BAET even for early rumor detection (i.e., few posts). When the number of posts is low, all models perform poorly, which is attributed to a lack of relevant information. The veracity of a claim cannot be determined immediately after it is sent out. During the propagation process, responding posts may express approval or disapproval of the claim, as well as provide some relevant evidence in the reply. The pieces of information are gathered through the propagation structure, which provides a solid foundation for identifying the rumor.

As a result, as the number of posts grows, the performance of all models improves. Because multi-perspective modeling of rumor propagation and mining interactive information are taken into account, our model BAET outperforms all others in rumor detection. Second, RvNN proposed an investigation into the rumor detection propagation structure, but its detection capability is inferior to that of other models. Furthermore, while EBGCN and BiGCN have comparable performance, EBGCN has a better detection effect, and the BiGCN model has a more consistent performance increase. We hypothesize that the uncertainty of the propagation relation (edges in the tree) considered in the EBGCN model has an auxiliary effect on rumor detection, but because the complexity of the propagation relation increases with the number of posts, the EBGCN model’s performance fluctuates slightly.

To get a deep insight into the importance of authors who participate in rumors and non-rumors in the author propagation tree, we visualize the attention weights (Tree-aware Attention Learning of claims randomly sampled from the PHEME datasets. The result is shown in Fig. 9 with the author propagation tree structure of the first True Claim in the left part. Three observations can be made from Fig. 9:

(1) Most authors have high weights, indicating that the author’s propagation tree structure facilitates the modeling of claim veracity.

(2) For true claims, authors closer to the root author yield a higher weight, i.e., these authors contain more features similar to the root author and ”flock” around him, while authors form small-scale flocks during the propagation of false claims. This observation is confirmed by  [52] that most people forward true rumors from a centralized source (claim), while rumors are transmitted by people forwarding to others.

(3) Hence, we can infer that to judge a true claim, we could better check the traits of the early forwarding and closer authors. This is not absolute because the experiment in Section V-E shows that the number of responsive posts is also very important for rumor detection; thus, our model needs to be able to integrate multiple factors.