LFGCF: Light Folksonomy Graph Collaborative Filtering for Tag-Aware Recommendation

Collaborative tagging recommendation allows users to assign tags freely on all kinds of items and then utilizes them to provide better-personalized recommendations. However, just like collaborative filtering recommendation, collaborative tagging recommendation suffers from the sparsity of interactions. Nevertheless, the redundancy and the ambiguity greatly compromise the performance of recommendations. By incorporating tagging information into collaborative filtering, Zhen et al. [14] came up with TagiCofi, which managed to characterize users’ similarities better and achieved better recommendations. To tackle the redundancy of tags, Shepisten et al. [11] used hierarchical clustering on social tagging systems. Peng et al. [15] further integrated the relationship between users, items, and tags and then came up with a Joint Item-Tag recommendation. Zhang et al. [16, 17] incorporated a user-tag-item tripartite graph, which turned out effective for improving the accuracy, diversity, and novelty of recommendations. FolkRank++ was brought up by Zhao et al. [18] to dig out the similarities among users and items fully. Focusing on personalized ranking recommendations in a collaborative tagging system, Rendle et al. [3] introduced matrix factorization and brought up RTF, which optimized ranking in personalized recommendations. Later, enlightened by BPR [19], Li et al. [6] came up with BPR-T for tag-aware recommendations. With the development of deep learning, Zuo et al. [5] used deep neural networks to extract tag information, which helped alleviate the sparsity, redundancy, and ambiguity of tags and generate accurate user profiles.

Despite the fact that much effort has been devoted to tag-aware recommendations, the implicit pattern under user tagging behavior are not fully extracted. We introduce the modified knowledge graph algorithm to tackle this problem.

Recent years have witnessed rapid development in graph neural networks, which perform excellently in node classifications and edges predictions. Berg et al. [20] brought up GCMC, in which autoencoder was used in a user-item bipartite graph to generate expressive embeddings. Ying et al. [21] focused on web-scale recommendation systems and came up with a GCN-based algorithm: PinSage. Results showed that Pinsage had excellent robustness and could generate high-quality recommendations. Wang et al. [22] incorporated GCN and came up with NGCF. Benefited by the multi-hop neighborhood connectivities, NGCF achieved good recommendations performance. Later in the research of He et al. [13], it is proven that some designs in NGCF are burdensome and compromising recommendation performance. So LightGCN was brought up to simplify the model design, achieving better recommendations. Focusing on click-through rate (CTR) predictions, Li et al. [23] came up with Fi-GNN, in which the gated recurrent units (GRU) was used for information aggregation. DG-ENN was brought up by Guo et al. [24] for the same task. By incorporating the attribute graph and the collaborative graph, DG-ENN was able to alleviate the feature and behavior sparsity problem. For certain factors in CTR tasks, Zheng et al. [25] taken price into account when coming up with a GCN-based model. Furthermore, Su et al. [26] used L0 regularization via GNN approach to distinguish valuable factors automatically. Focusing on the re-ranking task in recommendations, Liu et al. [27] developed IRGPR, which introduced an intent embedding network to embed user intents, and it is proven effective for re-ranking. Lately, several researchers have focused on applying graph neural networks to collaborative tagging systems. Chen et al. [4] used graph convolutional networks for tag-aware recommendations, and their TGCN model outperformed other state-of-the-art models. Huang et al. [10] used two graph attention networks for embeddings aggregation and achieved high-quality recommendations as well.

Few researchers use graph neural networks on tag-aware recommendations, while the model structure is quite complicated, making the training rather tricky. Our proposed method uses a relatively light and straightforward graph structure which suppose lowers the training cost and improves the performance.

Folksonomy also named user tagging behavior, is the fundamental factor of the TRS. It is defined as a series of tags assigned by some users when interacting with certain items they are interested in. Generally, users interacting with certain items by operations such as clicking, tagging, or commenting could be viewed as folksonomy records. It is aggregated into a triplet, i.e, $a=(u,t,i)$, which represents user $u$ assigned tag $t$ to item $i$. These personalized tags reflect users’ subjective preferences and characteristics of items. This tagging procedure is rich in collaborative information. By exploring this collaborative information, TRS can further infer users’ preferences, summarize features of items and understand the connotation of tags, which hopefully improve the quality of recommendation systems.

Suppose the number of elements in user set $\mathcal{U}$, item set $\mathcal{I}$, tag set $\mathcal{T}$ are $N_{u}$, $N_{i}$ and $N_{t}$, respectively. The folksonomy is a tuple $\mathcal{F=(\mathcal{U},\mathcal{T},\mathcal{I},\mathcal{A})}$, where $\mathcal{A}\subseteq\mathcal{U}\times\mathcal{I}\times\mathcal{T}$ is a record in a typical TRS. During the tagging procedure, user $u$ interacts with the target item $i$ through an explicit tagging feedback by tag $t$, i.e., the user watching the movie ’Transformers’ because of the tag ’sci-fi’.But few researchers have addressed the problem of information leaks on folksonomy. We argue that directly modeling explicit tagging feedback as implicit feedback may leak information because a part of tagging behavior is negative feedback, i,e, user tagging movie as ’boring movies’. The leak is supposed to hinder the recommendation performance in the test dataset. Our approach to tackling the problem is to model user-tag tagging interactions and item-tag tagged interactions in $\mathcal{A}$ separately, rather than model user-item interactions directly.

This paper focuses on recommending the personalized ranking list of items for each user $u$ on the TRS. By exploring the implicit feedback in user tagging assignments, leveraging collaborative information, and training a model to refine embeddings, we can generate the Top-K list of items for each user.

To prevent the information leak from happening, we construct two sets of edges $\mathcal{E}_{u,t}$ and $\mathcal{E}_{i,t}$ among the folksonomies records $a=(u,t,i)\in\mathcal{A}$. Set $\mathcal{E}_{u,t}$ reflects the assignments between users and tags. On the one hand, set $\mathcal{E}_{i,t}$ indicates the passive tagged interactions between items and tags. In TRS scenario, each kind of edge $e$ from $\mathcal{E}_{u,t}$ and $\mathcal{E}_{i,t}$ is respectively defined as:

To make it easier to follow, the matrix form of folksonomy is shown in Fig. 1. Therefore, the FG is constructed based on user-tag tagging and item-tag tagged interactions according to the assignments set $\mathcal{A}$. Each set of edges can be respectively regarded as the set of edges in the bipartite graph $G_{tagging}=(\mathcal{U},\mathcal{T},\mathcal{E}_{u,t})$, $G_{tagged}=(\mathcal{I},\mathcal{T},\mathcal{E}_{i,t})$.

Mainstream GNN models, such as GCN[28] and GAT[29] were originally proposed for node or graph representation learning on attributed graphs. Specifically, each node has existing embeddings as input features, which are firstly transformed into a uniform shape by features transformation and then aggregated with its neighbors on the graph. In the end, embeddings are updated by nonlinear activations. Whereas from the view of the bipartite graph for Collaborative Filtering, each node (user or item) is only identified by a unique token without any concrete semantics as input features. In such a case, given unique token embeddings as input, performing multiple layers of feature transformation and nonlinear activations are keys to the success of modern neural networks[30]. However, these operations not only make little benefits to the performance of recommendation tasks but also increase the difficulties of representation training. Inspired by the ideas of LightGCN[13], we propose a light yet effective model by including the essential ingredients of GCN for a recommendation.

Knowledge graph embedding effectively parameterizes nodes as vector representations while keeping the graph’s topology. In this paper, we propose a new method for embedding a knowledge graph based on the idea of transformers. Here we propose a new regularization function, which is based on TransR[35], a widely used method in a knowledge graph. To be specific, it learns the embedding of each node by optimizing the translation principle $e_{u}+e_{t}\approx e_{t}$, if a triplet $(u,t,i)\in\mathcal{A}$. Herein, $e_{u},e_{i},e_{t}\in\mathbf{R}^{d}$ is the final embedding of user $u$, item $i$ and tag $t$, respectively, and $e_{u},e_{i}$ are the projected representations of $e_{u}$ and $e_{i}$ in the tag’s space. Hence, for a given folksonomy record $(u,t,i)$, its plausibility score could be defined as follows:

where $e_{u},e_{i},e_{t}$ are in the same d-dimension space, but not the same semantics space. A lower score of $g(u,t,i)$ suggests that the folksonomy score is more likely to be reasonable and vice versa.

The trainable parameters of LFGCF are only the embeddings of the 0-th layer, which combine with users, items, and tags in FG. In other words, the model complexity is the same as the standard matrix factorization (MF). To optain better ranking, we employ the Bayesian Personalized Ranking (BPR) loss[19], which is a pairwise loss that encourages the prediction of an observed record to be higher than its unobserved sampled counterpart. The BPR loss is defined as follows:

where $\mathcal{O}=\{(u,i,i^{\prime})|(u,i)\in\mathcal{A},(u,i^{\prime})\notin\mathcal{A}\}$ indicates pairwsie $(u,i)$ observed in folksonomy records, and pairwise $(u,i^{\prime})$ means that the user $u$ and item $i^{\prime}$ not observed in record, but uniformly sampled from the unobserved pairs.

To train TransRT, we minimize the plausibility score:

where $\alpha$ controls the strength of the knowledge graph regularization, and $g(u,t,i)$ is the plausibility score of the record $(u,t,i)$.

To effective learning parameters for recommendation and preserve the regularization relationship among folksonomy records, we integrate the Top-K recommendation task and the TransRT by a jointly learning framework. Finally, the total objective function of LFGCF is defined as follows:

where $\gamma$ controls the strength of regularization. We employ the Adam[36] optimize $\mathcal{L}_{LFGCN}$ and $\mathcal{L}_{TranRT}$ and use it in a mini-batch manner. Besides, an early stopping strategy is also applied to avoid overfitting during training.

In order to answer the questions above, a series of experiments are designed and carried out. We first show that LFGCF outperforms other state-of-the-art models, then elaborate on its expressiveness and explainability. Meanwhile, according to the investigation from [37], experiments of some other models are carried out with the different processes, which brings difficulties to repeating them. Hence, all the experiments in this paper are all under the uniformed experimental framework Recbole[38] to make fair comparisons.

The performance of TRS is directly related to the quality of Top-K recommendations, which are evaluated by the following metrics: Recall@N, Precision@N, Hit Ratio@N, NDCG@N, and MRR@N[40]. Empirically, higher metrics mean better performances. Each metric is elaborated:

• $\bullet$

  Recall@N measures the percentage of the number of items in Top-K recommendations to the actual item set that user interact with.

  $$Recall@N=\frac{|R^{N}(u)\bigcap T(u)|}{|T(u)|}$$

  (14)

  where $R^{N}(u)$ denotes the Top-K recommendations and $T(u)$ denotes the ground truth item set.

• $\bullet$

  Precision@N measures the fraction of items the users would interact with among the Top-K recommendations.

  $$Precision@N=\frac{|R^{N}(u)\bigcap T(u)|}{K}$$

  (15)

• $\bullet$

  Hit Ratio@N measures the percentage of users who interact with the recommendations for at least one time.

  $$HR@N=\frac{1}{\mathcal{U}}\sum_{u\in\mathcal{U}}I(|R^{N}(u)\bigcap T(u)|>0)$$

  (16)

  where $I(\cdot)$ is the indicator function.

• $\bullet$

  NDCG@N reflects the quality of ranking by distinguishing the contributions of the accurately recommended items.

  $$NDCG@N=\frac{1}{\mathcal{U}}\sum_{u\in\mathcal{U}}\frac{\sum^{N}_{n=1}\frac{I(R^{N}_{n}(u)\in T(u))}{log(n+1)}}{\sum^{N}_{n=1}\frac{1}{log(n+1)}}$$

  (17)

  where $R^{N}_{n}(u)$ means the $n^{th}$ item in Top-K recommendations $R^{N}(u)$.

• $\bullet$

  MRR@N computes the reciprocal rank of the first relevant item found by an rank algorithm.

  $$MRR@N=\frac{1}{\mathcal{U}}\sum_{u\in\mathcal{U}}\frac{1}{rank_{u}^{*}}$$

  (18)

  where $rank_{u}^{*}$ means the rank position of the first relevant item in recommendations for a user.

To fairly evaluate the performance and effectiveness of LFGCF, we adopt some classic or state-of-the-art TRS models as baselines.

• $\bullet$

  DSPR[7] leverages deep neural networks to learn tag-based features by mapping user and item profiles to deep latent space.

• $\bullet$

  CFA[5] is a user-based collaborative filtering model which adopts a sparse autoencoder to extract latent features.

• $\bullet$

  BPR-T[6] is a collaborative filtering model which incorporates tagging information and the Bayesian ranking optimization.

• $\bullet$

  TGCN[4] is a collaborative filtering model which incorporates tagging information into GCN along with an attention mechanism.

In order to make impartial comparisons, each model is optimized with mini-batch Adam, while the batch size is set as 2048. The learning rate of each model is searched from {0.0001, 0.0005, 0.001, 0.005, 0.01, 0.05} and the regression weight is searched from {1e-5, 1e-4, 1e-3, 1e-2}. For the autoencoder in CFA and the graph structure in TGCN and LFGCF, the number of layers is searched from {1, 2, 3, 4}. For BPR-T, its three regression weight is searched from {1e-5, 1e-4, 1e-3, 1e-2, 1e-1}. Additionally, by further searching the coefficients $\alpha,\beta$ from {1e-5, 1e-4}, we can analyze the sensibility of TransRT. The hyperparameter experiments are conducted under 10 random seeds. By conducting these experiments, all of the models are at their optimal performances, ensuring the fairness of the following comparisons.

The experiment results in this section are all achieved with the optimal hyperparameters. Best performance is in boldface, best baseline proformance is in underline and imp. means the improvement of LFGCF over the state-of-the-art baseline.

Table.LABEL:performance_comparison shows the Top-K recommendation performance metrics of LFGCF and other baselines in three datasets when $N=\{10,20\}$. Fig.3 shows the Top-K recommendation performance of our LFGCF and other baselines in terms of Recall@N, Precision@N and MRR@N while N ranges from 5 to 30 with the interval of 5.

Fig.3 shows the Top-K recommendation performance of our LFGCF and other baselines in Recall@N, Precision@N, and MRR@N, while N ranges from 5 to 30 with an interval of 5.

The performance comparisons of our model and other baselines shows that LFGCF achieved the state-of-the-art performance while succcessfully alleviating the training difficulty with the help of light designed GCN. Among baseline models, TGCN and BPR-T outperform DSPR and CFA by a large margin.

We summarize why our LFGCF outperforms other baselines for the following reasons: (1) More effective GCN. Two LFGCN implemented for feature learning show better performances than deep neural networks in models CFA and DSPR. Effective representation propagation and aggregation benefit LFGCF by lifting performances and reducing training difficulties; (2) TransRT assists with extracting more expressive user and item representations. Recent years witnessed efforts devoted to the loss function based on BPR loss to better fit recommendation tasks. Experiments indicate that using the improved knowledge graph algorithm TransRT allows more efficient uses of collaborative tagging information.

To verify our summaries of the superior performance of LFGCF above, we conduct a series of ablation studies on LFGCF.