Learning Hierarchical Review Graph Representations for Recommendation

Traditional review-based recommendation methods employ topic modeling techniques on the review data to learn latent feature distributions of users and items. For example, McAuley and Leskovec [10] propose the HFT model that learns the latent factors of users and items from review texts by a LDA model. Guang et al. [12] propose the RMR model that combines the topic model with a mixture of Gaussian models to improve the recommendation accuracy. In [11], Bao et al. employ non-negative matrix factorization to derive topics of each review, and a transform function to align the topic distributions with corresponding latent user/item factors. In [13], Tan et al. propose a rating-boosted method that combines textural reviews with users’ sentiment orientations to learn more accurate topic distributions.

The quick development of deep learning motivates the application of deep learning techniques to extract semantic information from the review data and build the end-to-end recommendation models. For instance, Wang et al. [15] propose the pioneer work that employs stacked denoising autoencoders (SDAE) to learn deep representations from review contents, and probabilistic matrix factorization to model users’ rating behaviors. Zheng et al. [16] propose the DeepCoNN model employing CNN to extract semantic information from the user reviews, and factorization machine to capture the interactions between the user and item features. Recently, the attention mechanism is applied to further improve the recommendation accuracy. In [17], Chen et al. extract the review representation using CNN, and scored each review through an attention mechanism to learn informative user and item representations. In [22], Wu et al. utilize CNN with attention mechanism to highlight relevant semantic information by jointly considering the reviews written by a user and the reviews written to an item. Moreover, Lu et al. [23] propose to jointly learn the user and item information from ratings and reviews by optimizing matrix factorization and an attention-based GRU (i.e., Gated Recurrent Unit) network. Liu et al. [18] propose the dual attention mutual learning between ratings and reviews for recommendation. They use both local and mutual attention of the CNN to jointly learn review representations for improving the model interpretability. Wu et al. [24] utilize three attention networks to learn representations of sentence, review, and user/item in turn, and a graph attention network to model user-item interactions. To incorporate the user/item characteristics, Liu et al. [19] apply a personalized attention model to learn personalized representations of users and items from the review data. In addition, there are some aspect-based recommendation models [20, 21] that apply aspect-based representation learning to automatically detect the most important parts of a review, and jointly learn the aspect, user, and item representations.

The graph neural networks (GNNs) extend deep learning techniques to process the graph data. Two main categories of GNN methods are recurrent graph neural networks and graph convolutional neural networks [25].

The pioneer GNN methods are about recurrent graph neural networks, which learn the representation of a node by propagating its neighborhood information in an iterative manner until convergence. For example, in [26], the node’s states are updated by exchanging its neighborhood information recurrently before satisfying the convergence criterion. Gallicchio and Micheli [27] improve the training efficiency of [26] by generalizing the echo state network to graph domains. Moreover, Li et al. [28] propose the gate graph neural networks (GGNN) which employs a gated recurrent unit as the recurrent function. This method no longer needs parameter constraints to ensure convergence.

The graph convolutional neural networks (GCNs) generalize the convolution operation from grid data to graph data. Existing GCN methods usually fall into two groups, i.e., spectral-based methods and spatial-based methods. The spectral-based methods define convolution from the perspective of graph signal processing. For example, Bruna et al. [29] employs the eigen decomposition of the graph Laplacian and defines the convolution in the Fourier domain. To reduce computational complexity, Defferrard et al. [30] approximate the convolutional filters by Chebyshey polynomials of the diagonal matrix of eigenvalues. Then, Kipf and Welling [31] develop a simpler convolutional architecture by the first-order approximation of the CheNet [30]. The spatial-based methods directly operate on the graph and propagate node information along edges. For instance, Micheli [32] proposes to directly sum up a node’s neighborhood information and applies residual connection to memorize information over each layer. Atwood and Towsley [33] treat the graph convolution as a diffusion process, and they assume there is a certain transition probability from a node to its neighbor. As the number of neighbors of nodes varies, Hamilton et al. [34] propose the GraphSage model which samples a fixed number of neighbors for each node and utilizes three different aggregators (e.g., LSTM) to aggregate the neighbors’ feature information. In [35], Veličković et al. propose the graph attention networks (GAT) that assumes the contributions of neighbor nodes to the central node are different. The multi-head attention mechanism is performed to calculate the importance of different neighbors. Moreover, Zhang et al. [36] employ a self-attention mechanism to compute an additional attention score for each attention head. In the recent work [37], Xu et al. identify the graph structures that cannot be distinguished by popular GNN methods, and propose the Graph Isomorphism Network (GIN) which uses a learnable parameter to adjust the weight of the central node.

The pooling operation usually helps provide better performance in graph representation tasks. Earlier pooling methods are mainly based on the deterministic graph clustering algorithms [38, 39], which are less flexible. The recent graph pooling methods can be divided into two groups: spectral and non-spectral methods. The spectral pooling methods [40, 41] employ laplace matrix and eigen-decomposition to capture the graph topology. However, these methods are not scalable to large graph, due to high computation complexity. Thus, a lot of research attentions have been attracted to develop the non-spectral pooling methods, which can be further divided into global and hierarchical pooling methods. The global pooling methods [42, 43] pool the representations of all nodes at one step. For example, Vinyals et al. [42] propose the Set2Set model that aggregates nodes information by combining LSTM with a content-based attention mechanism. Zhang et al. [43] develop the SortPool model that first sorts nodes according to their features and then concatenates the embedding of sorted nodes to summarize the graph. Differing from the global pooling methods, the hierarchical pooling methods aim to learn hierarchical graph representations. For example, Ying et al. [44] propose the first end-to-end differentiable pooling operator, named DiffPool, which learns a dense soft clustering assignment matrix of nodes by GNN. The gPool [45, 46] and SAGPool [47] pooling operators utilize a node-scoring procedure to select a set of top ranked nodes to form an induced graph from original graph. Moreover, Diehl [48] proposes a hard pooling method called EdgePool, which is developed based on edge contraction. In [49], Ranjan et al. utilize a self-attention mechanism to capture the importance of each node. This method learns a sparse soft cluster assignment for nodes to pool the graph. In [50], Zhang et al. propose a non-parametric pooling operation using a structure learning mechanism with sparse attention to learn a refined graph structure that preserves the key substructure in original graph.

Various graph based methods have been proposed to process the text data [52], which aim to capture the inherent dependence between words. Here, we utilize the graph-of-words method [53] to construct the review graph with preserving the word orders. For the reviews $\mathcal{S}_{u}$ of the user $u$, we firstly apply the phrase pre-processing techniques such as tokenization and text cleaning to select keywords of each review in $\mathcal{S}_{u}$. Then, all reviews in $\mathcal{S}_{u}$ can be encoded into an un-weighted and directed graph, where nodes represent words and edges describe the co-occurrences between words within a sliding window with fixed size $\omega$. Note that we define the word windows considering the positions of selected keywords in the original text.

Intuitively, the order information of words in the text may help capture the sentiment information [54]. For example, the phrases “not really good” and “really not good” convey different levels of negative sentiment information, which is only caused by the word orders. To preserve the orders of words in the review, we define three types of edges in the review graph: forward type, backward type, and self-loop type (respectively denoted by $e_{f}$, $e_{b}$, and $e_{s}$). We denote the set of edge type by $\mathcal{R}=\{e_{f},e_{b},e_{s}\}$. In a review $s_{u}^{t}\in\mathcal{S}_{u}$, if the selected keyword (i.e., node) $x_{2}$ appears before the selected keyword $x_{1}$ and the distance between $x_{1}$ and $x_{2}$ in the original text of $s_{u}^{t}$ is less than $\omega$, we construct an edge $E(x_{2},x_{1})$ from $x_{2}$ to $x_{1}$ and set the type of edge $E(x_{2},x_{1})$ as $e_{f}$. Moreover, we also construct an edge $E(x_{1},x_{2})$ from $x_{1}$ to $x_{2}$ and set the type of $E(x_{1},x_{2})$ as $e_{b}$. In the rare situation where the word $x_{2}$ may appear both before and after the word $x_{1}$, we randomly set the type of edge $E(x_{1},x_{2})$ as $e_{f}$ or $e_{b}$. In the experiments, we empirically set $\omega$ to 3. To consider the information of the word itself (e.g., $x_{1}$), we add a self-loop edge at each word (e.g., $E(x_{1},x_{1})$) and set the edge type as $e_{s}$. This trick also helps define the attention score in Eq. (1).

Figure 2 shows an example about constructing the review graph for the review “My son likes this beautiful looking football”, where the words “son”, “likes”, “beautiful”, “looking”, and “football” are selected as keywords to build the review graph. In the original review text, the distance between “son” and “likes” is 1 which is less than $\omega$. Thus, we construct an edge from “son” to “likes” and set the edge type as $e_{f}$. Moreover, we also construct an edge from “likes” to “son” and set the edge type as $e_{b}$. However, the distance between “son” and “beautiful” in original text is 3, which is not smaller than the windows size ($\omega=3$). Therefore, as shown in Figure 2, there is no connection between the selected keywords “son” and “beautiful”.

For each user $u$, we denote her review graph by $\mathcal{G}_{u}=\{\mathcal{X}_{u},\mathcal{E}_{u}\}$, where $\mathcal{X}_{u}$ denotes the set of nodes (i.e., selected keywords) and $\mathcal{E}_{u}$ denotes the set of node-edge-node triples $(x_{h},e_{r},x_{t})$. Here, $e_{r}$ is type of the edge connecting from node $x_{t}$ to $x_{h}$, which is one of the three edge types defined above. Similarly, we can build the review graph $\mathcal{G}_{i}=\{\mathcal{X}_{i},\mathcal{E}_{i}\}$ for each item $i$.

After completing the construction of review graph $\mathcal{G}_{u}$, we employ the representation learning procedure as shown in Figure 3 to learn hierarchical representations of $\mathcal{G}_{u}$, by repeating the TGAT and PGP operations multiple times. In this section, we introduce the details of the operations in the $\ell$-th representation learning layer, where the input graph is $\mathcal{G}_{u}^{\ell}=\{\mathcal{X}_{u}^{\ell},\mathcal{E}_{u}^{\ell}\}$. Suppose $\mathcal{G}_{u}^{\ell}$ has $K_{u}^{\ell}$ nodes. Then, we also use an adjacency matrix $\mathbf{A}^{\ell}_{u}\in\mathbb{R}^{K_{u}^{\ell}\times K_{u}^{\ell}}$ and a feature matrix $\mathbf{X}_{u}^{\ell}\in\mathbb{R}^{K_{u}^{\ell}\times d_{2}^{\ell}}$ to describe $\mathcal{G}_{u}^{\ell}$, where each row in $\mathbf{X}_{u}^{\ell}(x_{1},:)$ denotes the input feature of the node $x_{1}$, and $d_{2}^{\ell}$ denotes the dimensionality of the input feature. Node that the features of nodes in the review graph are initialized by the embeddings of corresponding words. Thus, at the first layer, $d_{2}^{\ell}$ is equal to $d_{0}$. In $\mathcal{G}_{u}^{\ell}$, if there exists an edge between two nodes $x_{1}$ and $x_{2}$, we set $\mathbf{A}^{\ell}_{u}[x_{1},x_{2}]$ to 1, and set to 0, otherwise.

After repeating the TGAT and PGP operations $L$ times, we can obtain multiple pooled sub-graphs with different size, i.e., $\mathcal{G}_{u}^{2},\mathcal{G}_{u}^{3},\cdots,\mathcal{G}_{u}^{L+1}$. This hierarchical pooling operation extracts the coarser-grained information of review graph layer-by-layer, which can be considered as different level representations of reviews, e.g., word-level, sentence-level, and review-level. To integrate these sub-graph information, we first use the max-pooling and MLP to generate representation of each sub-graph $\mathcal{G}_{u}^{\ell}$, where $2\leq\ell\leq L+1$, as follows:

where $\mathbf{W}_{6}^{\ell}\in\mathbb{R}^{d_{2}\times d_{1}}$ and $\mathbf{b}_{6}^{\ell}\in\mathbb{R}^{1\times d_{1}}$ are the weight matrix and bias vector. $\mathbf{g}_{u}^{\ell}$ denotes the representation of the sub-graph generated at the $\ell$-th layer. Then, we concatenate the non-linear transform of the user embedding after a MLP layer and the representations of sub-graphs at all layers to generate the semantic representation $\mathbf{p}_{u}$ of the user $u$ as follows,

where $\oplus$ is the concatenating operation, $\mathbf{W}_{7}\in\mathbb{R}^{d_{1}\times d_{1}}$ and $\mathbf{b}_{7}\in\mathbb{R}^{1\times d_{1}}$ are the weight matrix and bias vector. Similarly, we can get the semantic representation of the item $i$ as $\mathbf{q}_{i}$.

A factorization machine (FM) layer is used to predict the user’s preference on an item, considering the higher order interactions between the user and item semantic features. Specifically, we first concatenate the user and item semantic representations as $\mathbf{z}=\mathbf{p}_{u}\oplus\mathbf{q}_{i}$, and the prediction $\hat{y}_{ui}$ of the user’s preference can be defined as follows,

where $b_{0}$, $b_{u}$ and $b_{i}$ are the global bias, user bias, and item bias. $\mathbf{w}\in\mathbb{R}^{1\times d^{\prime}}$ is the coefficient vector, and $d^{\prime}=2*(L+1)*d_{1}$. $\mathbf{v}_{i},\mathbf{v}_{j}\in\mathbb{R}^{1\times k}$ are the latent factors associated with $i$-th and $j$-th dimension of $\mathbf{z}$. Empirically, we set $k=d_{2}$ in this work. $z_{i}$ is the value of the $i$-th dimension of $\mathbf{z}$. In this work, we focus on the rating prediction problem. Thus, the model parameters $\mathbf{\Theta}$ of RGNN can be learned by solving the following optimization problem,

where $\lambda$ is the regularization parameter, $\mathcal{D}$ denotes the set of user-item pairs used to update the model parameters. The entire framework can be effectively trained by the end-to-end back propagation algorithm.

In this work, we build a review graph to describe all the reviews of a user/item. If a word appears in multiple review sentences, its neighbors in review graph aggregate all its neighboring words (within a word window) in these sentences. Then, TGAT is proposed to learn the word embedding, considering its global context words among all the relevant reviews. However, the CNN/RNN-based methods separately aggregate the local neighboring words of a word in different review sentences. Moreover, TGAT employs attention mechanism to automatically assign different importance weights to neighboring words. By performing multiple times of PGP operations, the proposed model can capture the long-term dependency between selected words.

Suppose the average number of selected keywords for each user/item is $N_{w}$. The time complexity of constructing the review graph is $O\big{(}\omega N_{w}\big{)}$. In practice, the review graph construction can be performed offline. The time complexity for RGNN to obtain the hierarchical review representation is $O\big{(}\sum_{\ell=1}^{L}\alpha^{\ell}N_{w}d_{0}^{2}\big{)}$, where $L$ denotes the number of layers, $\alpha$ and $d_{0}$ denote the pooling ratio and word embedding dimension. The total training time complexity of RGNN is $O\big{(}LN_{w}d_{0}^{2}\big{)}$. Moreover, the time complexity of using CNNs to obtain the review representation is $O\big{(}MSN_{w}d_{0}\big{)}$, where $M$ and $S$ denote the number of convolutional kernels and the convolutional kernel size, respectively. Thus, compared with the CNN based methods, the proposed RGNN model is not complicate or can be even more efficient.

Table II summarizes the results of different recommendation methods. We can make the following observations.

• •

  The review-based recommendation methods significantly outperforms PMF, which demonstrates the importance of review data for improving recommendation accuracy.

• •

  The neural methods using CNNs and attention mechanism (e.g., DeepCoNN and NARRE) achieve better performances than CDL. The potential reason is that CDL employs bag-of-words model to process the review data, thus ignores the word order and local context information.

• •

  The review-based methods based on attention mechanism (e.g., NARRE, NRPA, RMG) outperform DeepCoNN, by learning the importance of words or reviews for better understanding of users’ rating behaviours. DAML usually performs best among baseline methods. It utilizes mutual attention of the CNN to study the correlation between the user reviews and item reviews, thus can capture more information about the user preferences and item properties.

• •

  RGNN consistently achieves the best performances on all datasets. The improvements over baselines are statistically significant with $p<0.05$ using student t-test. In RGNN, the user/item reviews are described by a directed review graph with preserving the word orders, which can capture the global and non-consecutive topology and dependency information between words in reviews. Moreover, the PGP operator extracts hierarchical and coarser-grained representations of reviews while considering the user/item-specific properties, thus can further help improve the recommendation performance.

Moreover, Table III summarizes the performances of RGNN with respect to different settings of $L$. On Music, Office, Toys datasets, the best performances of RGNN are achieved by setting $L$ to 2. On Games, Beauty, and Yelp datasets, RGNN achieves the best performances, when $L$ is set to 3, 4, and 3, respectively. This demonstrates the effectiveness of the hierarchical graph representation learning.

To investigate the importance of each component of RGNN, we firstly study the contributions of the review graph and the factorization machine layer. We consider the following variants of RGNN for experiments:

• •

  RGNN-RG: In this variant, we remove both the user and item review graphs. We use the max pooling of the embeddings of the keywords extracted from $\mathcal{S}_{u}$ as the review representation of the user $u$, which is denoted by $\mathbf{r}_{u}$. Then, in Eq. (10), we concatenate $\mathbf{m}_{u}$ with $\mathbf{r}_{u}$ to form the user representation $\mathbf{p}_{u}$. Similar operation is performed on the reviews of the item $i$. Note that FM layer is also used for rating prediction in this variant.

• •

  RGNN-FM: For this variant, we remove the FM layer and replace the rating prediction function in Eq. (11) by the product of $\mathbf{p}_{u}$ and $\mathbf{q}_{i}$ as $\hat{y}_{ui}=\mathbf{p}_{u}\mathbf{q}_{i}^{\top}$.

Table IV summarizes the performances achieved by DAML, RGNN-RG, RGNN-FM, and RGNN on the Music, Office, and Toys datasets. From Table IV, we can note that RGNN outperforms RGNN-RG by 5.01% and 1.48% on Music and Toys dataset. Compared with the best baseline method DAML that employs FM layer for prediction, RGNN-FM outperforms DAML on the Office and Toys datasets, achieves comparable results with DAML on the Music dataset. These two observations demonstrate the effectiveness of the proposed review graph based representation learning strategy in capturing the users’ preferences on items. Moreover, RGNN outperforms RGNN-FM on Music and Toys datasets. This indicates the FM layer can help improve the rating prediction accuracy. As shown in Table IV, RGNN-FM achieves better results than RGNN-RG on three datasets. This demonstrates that the main improvements achieved by RGNN are caused by the proposed hierarchical review graph representation learning strategy.

Moreover, we perform experiments to analyze the contributions of different components in the hierarchical graph representation learning procedure. Specifically, we evaluate the performances of the following RGNN variants:

• •

  RGNN-T: RGNN without considering the word orders (i.e., removing the edge type embedding in Eq. (2)).

• •

  RGNN-P: RGNN without considering personalized properties of users/items (i.e., using the same $\bm{\theta}^{\ell}$ for all users/items in Eq. (5)).

• •

  RGNN-S: RGNN without considering the feature diversity of a node’s neighborhood in estimating the node importance (i.e., removing the second part in Eq. (5)).

• •

  RGNN-PL: RGNN without considering the graph pooling (i.e., removing the PGP operators in Figure 3).

The experimental results on Music, Office, and Toys datasets are summarized in Figure 4. As shown in Figure 4, RGNN outperforms RGNN-T on all three datasets. This indicates that the word order is important in extracting semantic information from reviews for recommendation. Moreover, RGNN achieves better performances than RGNN-P, RGNN-S and RGNN-PL in terms of MSE. This result demonstrates that the proposed PGP operator can help improve the recommendation accuracy via capturing more informative review words for a user/item. When estimating the importance of a node (i.e., word), the proposed PGP operator considers the user/item-specific properties, as well as the feature diversity of a node’s neighborhood in the review graph. From Figure 4, we can also note that GNN-PL achieves better MSE than RGNN-P on Office and Toys datasets. RGNN-P adopts the same pooling parameter for all users/items. It tends to select a set of common words to form the review representations for all the users/items. The specific properties of a user/item in the review data may not be exploited. Therefore, RGNN-P may perform poorer than RGNN-PL that employs max pooling on the embeddings of all nodes in the review graph of each individual user/item.

In addition, we also conduct experiments to analyze the impacts of different edge types. Specifically, we study the performances of RGNN with the following settings:

• •

  RGNN-f: RGNN without considering the forward edge type $e_{f}$.

• •

  RGNN-b: RGNN without considering the backward edge type $e_{b}$.

• •

  RGNN-s: RGNN without considering the self-loop edge type.

Figure 5 summarizes the MSE values achieved by RGNN with different settings of edge types on Music, Office, and Toys datasets. As shown in Figure 5, RGNN-b outperforms RGNN-f on all three datasets. This indicates that the forward type of edges are more important than the backward type of edges in capturing users’ rating behaviors for recommendation. Moreover, RGNN-s achieves the worst MSE values on Music and Toys datasets. This indicates the self-loop type of edges play an important role in the graph representation learning procedure. This observation is consistent with the definition of the attention mechanism in Eq. (3). By removing the self-loop type of edges, for each node $x_{h}$ in the graph at the $\ell$-th layer, it is not included in its first-hop neighbors $\mathcal{N}_{h}^{\ell}$ in the graph. According to Eq. (3), the new embedding of $x_{h}$ (i.e., $\bar{\mathbf{x}}_{h}^{\ell}$) is updated by only considering the input embeddings of other neighboring nodes at the $\ell$-th layer, without considering the input embedding of $x_{h}$ at the $\ell$-th layer. In addition, RGNN achieves the best results on Music and Office datasets. This indicates that the combination of all three types of edges usually provides a better description about the review data, thus achieves more accurate rating prediction results.

In this section, we perform experiments to evaluate the sensitivity to hyper-parameters for RGNN. The dimensionality of the semantic space $d_{0}$, the dimensionality of the rating space $d_{1}$, and the dimensionality of the hidden space of the graph representation learning layers $d_{2}$, are varied in $\{4,8,16,32,48,64,128\}$. The graph pooling ratio $\alpha$ is adjusted in $\{0.1,0.3,0.5,0.7,0.9\}$. The size of the word window $\omega$ is chosen from {2, 3, 4, 5, 6, 7, 8, 9, 10}. The results on Music dataset are summarized in Figure 6. We can note that the best MSE can be achieved by setting $d_{0}$, $d_{1}$ to 16, and $d_{2}$ to 8. Larger dimensionality of the latent semantic space and rating space cause more computation cost, however may not help improve the rating prediction accuracy. For the pooling ratio, we can notice that the best setting for $\alpha$ on Music dataset is 0.5. Moreover, RGNN usually achieves better performances by setting $\alpha<0.5$, compared by setting $\alpha>0.5$. One potential reason is that when $\alpha>0.5$ more nodes would be kept by the pooling operation, thus the “noise nodes” are more likely to be involved in learning the hierarchical review graph representations. In addition, as shown in Figure 6 (c), RGNN achieves the best MSE value when $\omega$ is set to 5 on Music dataset. Better performances can be usually achieved when $\omega$ is set in the range between 3 and 6.

In addition, we also conduct a case study to further explore whether RGNN can select informative words in the graph pooling layer. We sample a user-item pair in test data and extract the review documents of the user and item. Figure 7 visualizes the heat maps of these review documents. We highlight the words extracted from the first PGP layer of RGNN. The words with dark yellow background have larger importance scores than the words with yellow background (refer to Eq. (5)). As shown in Figure 7, the words “playing”, “sound”, “money”, and “long” are highlighted in user reviews. These words indicate that this user may be more interested in the quality and price of an item. Moreover, in the item reviews, the words “high”, “quality”, “solid”, “sound”, and “cheap” which describe the item properties are selected in the first pooling layer. In the review given by the user on the item, from the words “price”, “quality”, “natural”, and “sound”, we can argue that the user’s high rating on this item is indeed, because this item meets the concerns of the user. Hence, this case again demonstrates that the PGP layer can select informative words from the review graphs, thus can help improve the recommendation accuracy.