Neural News Recommendation with Negative Feedback

In this section, we introduce how to model user interests from their clicked news by considering negative feedback. Usually, the interests of users on the news they read carefully are very different from those they closed quickly after having a glance at the body. For example, as shown in Fig. 1, we can infer that the user may be indeed interested in the second news since she reads this news for about 220 seconds. However, she is probably not interested in or unsatisfied with the first news since she reads this news only for 3 seconds after click. Thus, incorporating short reading dwell time as the implicit negative feedback of users has the potential to enhance the modeling of user interest. To characterize the differences of news in representing the real user interests, we use their reading dwell times $t$ to distinguish positive news clicks from the negative ones. We regard the clicked news with $t<T$ as negative news clicks and those with $t\geq T$ as positive news clicks, where $T$ is a time threshold. Since simply aggregating all news together will ignore the differences of positive and negative news, in the user encoder module we propose to use a multi-view learning framework to incorporate positive and negative news clicks as different views of users. Their details are introduced as follows.

The positive news click view is used to learn a positive user embedding from positive clicked news. We denote the $P$ positive clicked news of a user as $[D_{1},D_{2},...,D_{P}]$. We first use a news encoder to obtain their representations, which are denoted as $\mathbf{R}^{p}=[\mathbf{r}_{1},\mathbf{r}_{2},...,\mathbf{r}_{P}]$. Then, we use a positive feedback encoder to learn a unified user embedding from these news representations. It uses a similar architecture with the user encoder used by Wu et al. wu2019neuralnrms . Since different clicked news usually have some interactions with each other, we use a news-level Transformer vaswani2017attention to capture the interactions among them³³3We add position embeddings to the news embeddings to capture behavior orders., and the output sequence is computed as:

where the three parameters in $Transformer(\cdot)$ respectively denote the input query, key and value. In addition, since different clicked news have different importance in modeling user interests, we use an additive attention yang2016hierarchical network to select important news to learn more informative user representations. The final positive user embedding $\mathbf{u}_{p}$ learned from this view is computed as:

where $\mathbf{q}^{p}$ is a query parameter vector, $\mathbf{D}^{p}$ is regarded as the key and value. It summarizes user interests inferred from the positive user feedback.

The negative news click view is used to learn representations of users from their negative clicked news. Similar to positive news, we use a combination of Transformer and additive attention network to learn the negative user embedding from the representations of the negative clicked news. We denote the negative user embedding learned from this view as $\textbf{u}_{n}$, which condenses the information of negative user interest inferred from the negative clicked news.

In this section, we introduce the details of click prediction in our approach. Usually, if a candidate news is similar to the news that a user reads carefully, it may have a high probability to be clicked by this user. However, if it is very similar to the news that are previously closed by a user very quickly, it will probably not be clicked by this user. Motivated by the observations above, we propose to calculate a positive click score $\hat{y}_{p}$ and a negative click score $\hat{y}_{n}$ respectively based on the relevance of candidate news to the positive and negative user interest. We denote the embedding of a candidate news $D^{c}$ as $\mathbf{r}_{c}$. Following okura2017embedding , $\hat{y}_{p}$ and $\hat{y}_{n}$ are respectively computed by the inner product of the candidate news embedding and the user embeddings learned from the positive and negative views, which are formulated as:

The final click probability score $\hat{y}$ is a weighted summation of the positive score $\hat{y}_{p}$ and the negative score $\hat{y}_{n}$, which is formulated as:

where $w_{p}$ and $w_{n}$ are learnable parameters. A positive parameter means that the relevance between candidate news and the corresponding user embedding has a positive correlation with the final click score, and this correlation is negative if the parameter is negative.

In this section, we introduce the architectures of the news encoder in our approach for news modeling. It is used to learn representations of news from their titles and bodies. Since news titles and bodies usually have very different characteristics, simply concatenating title and body together may not be optimal. Thus, in this module we use a multi-view learning framework to incorporate the information of title and body as different views of news. In addition, since there is usually inherent relatedness between news title and body, we propose to use an interactive attention network to capture their relations and form the unified news embedding. The details of each module in the news encoder are introduced as follows.

The first module in the news encoder is a title transformer, which is used to capture the contexts of words in a news title. We first use a word and position embedding layer to convert the sequence of words in a news title into a sequence of low-dimensional word embeddings with their position information. Denote a news title with $T$ words as $[w^{t}_{1},w^{t}_{2},...,w^{t}_{T}]$. It is converted into an embedding sequence $\mathbf{E}^{t}=[\mathbf{e}^{t}_{1},\mathbf{e}^{t}_{2},...,\mathbf{e}^{t}_{T}]$. Then, we use a Transformer to learn the contextual representation of each word, which is computed as follows:

The second module in the news encoder is a body transformer, which aims to capture the contexts of words in a news body. Similar to news title, we also use a word and position embedding layer to obtain the embeddings of the $M$ words in a news body, which are denoted as $\mathbf{E}^{b}=[\mathbf{e}^{b}_{1},\mathbf{e}^{b}_{2},...,\mathbf{e}^{b}_{M}]$. Then, we use a Transformer to capture the contexts of words, and its output $\mathbf{H}^{b}$ is computed as

The third module is an interactive attention network, which is used to capture the relations between news title and body when forming the unified news embedding. We first use two additive attention network to respectively attend to the important words in news title and body to form the title and body representations(denoted as $\mathbf{r}^{t}$ and $\mathbf{r}^{b}$), which are formulated as

where $\mathbf{q}^{t}$ and $\mathbf{q}^{b}$ are learnable query vectors. Then, we use the title representation as the query of the body attention network to select words in news body according to their relevance to the content of news title, and the output title-aware body representation $\mathbf{c}^{b}$ is computed as follows:

We learn a body-aware title representation $\mathbf{c}^{t}$ in a similar way by using the body representation as the title attention query, which is formulated as:

We finally aggregate all title and body representations into a unified news representation $\mathbf{r}$, i.e., $\mathbf{r}=\mathbf{r}^{t}+\mathbf{r}^{b}+\mathbf{c}^{t}+\mathbf{c}^{b}$.

Following huang2013learning ; wu2019npa , we use negative sampling techniques to construct labeled data from the raw news impression logs for model training. For each news clicked by a user, we randomly sample $Q$ news that are displayed in the same impression but not clicked by this user. We denote the click probability scores of the clicked news sample as $\hat{y}^{+}$ and the $Q$ unclicked news samples as $[\hat{y}_{1}^{-},\hat{y}_{2}^{-},...,\hat{y}_{Q}^{-}]$. we re-formulate the news click prediction problem as a pseudo $Q+1$-way classification task to jointly predict these scores. We normalize these scores via the softmax function to compute the posterior click probability of the clicked news, which is formulated as follows:

The final loss function we used is the negative log-likelihood of all clicked news samples, which is formulated as:

where $\mathcal{S}$ is the set of clicked news samples for training.

Our experiments were conducted on a real-world news recommendation dataset collected from MSN News⁴⁴4https://www.msn.com/en-us/news logs from Oct. 31, 2018 to Jan. 29, 2019. The detailed statistics are shown in Table 1. We sorted all sessions by time, and we used the first 80% of sessions for training, the next 10% for validation and the rest 10% for test.

In our experiments, the word embeddings were 300-dimensional and initialized by Glove pennington2014glove The Transformers had 8 attention heads, whose output dimension was 32. The maximum lengths of title and body were respectively set to 30 and 200, respectively. The numbers of positive and negative clicked news were respectively limited to 50 and 20. Following huang2013learning , the negative sampling ratio $Q$ was set to 4. The threshold $T$ was set to 10 seconds. Adam kingma2014adam was used as the optimizer, and the batch size was 30. To mitigate overfitting, we applied dropout srivastava2014dropout techniques to the word embeddings and Transformer networks, and the dropout ratio was set to 0.2. These hyperparameters were tuned on the validation set. We independently repeated each experiment 10 times and reported the average results in terms of AUC, MRR, nDCG@5 and nDCG@10.

We evaluate the performance of our approach by comparing it with many baseline methods, including:

• •

  LibFM rendle2012factorization , a factorization machine based method for recommendation. We extracted the TF-IDF features from the titles and bodies of news clicked by users to build their representations.

• •

  DSSM huang2013learning , deep structured semantic model. We regarded the concatenation of the news clicked by a user as the query and candidate news as documents, and we used the same feature with LibFM for news and user representation.

• •

  Wide&Deep cheng2016wide , a famous neural recommendation method which consists of a wide linear channel and a deep neural network channel. We used the same features with LibFM as the input of both channels.

• •

  DeepFM guo2017deepfm , another widely used neural recommendation method which uses a combination of factorization machines and deep neural networks. We used the same TF-IDF features to feed for both components.

• •

  DFM lian2018towards , a deep fusion model for news recommendation, which combines fully connected layers with different depths. We used TF-IDF features and word embeddings as the input.

• •

  EBNR okura2017embedding , an embedding based neural news recommendation method which uses denoising autoencoders to learn news representations and a GRU network to learn user representations.

• •

  DKN wang2018dkn , a neural news recommendation method which learns news representations via knowledge-aware CNNs and learns user representations via an attention network based on the relevance between clicked news and candidate news.

• •

  DAN zhu2019dan , a neural news recommendation method which learns news representations via two parallel CNNs and learns user representations using a combination of attentional LSTM and CNN;

• •

  NAML wu2019neuralnaml , a neural news recommendation approach with attentive multi-view learning.

• •

  NRMS wu2019neuralnrms , a neural news recommendation method that uses multi-head self-attention for news and user modeling.

• •

  NRNF-basic, a variant of our NRNF approach that does not consider negative feedback.

• •

  NRNF, our news recommendation approach with negative feedback.

For fair comparison, in GRU, DKN, DAN and NRMS we used the concatenation of news title and news body to learn news representations. The results of these methods are summarized in Table 2.

From Table 2, we have several observations. First, compared with baseline methods that use handcrafted features to represent news and users (e.g., LibFM, DSSM and DeepFM), the models using neural networks for learning news and user representations (e.g., EBNR, NAML and NRNF) achieve better recommendation performance. This is probably because neural networks can learn more informative news and user representations than hand-crafted features. Second, compared with other neural news recommender like DAN, NAML and NRMS, NRNF and its variant NRNF-basic achieve better performance. This is because the latter methods can capture the relatedness between news title and body to enhance news modeling, while other methods cannot. Third, NRNF outperforms other baseline methods, and further hypothesis tests show that the improvement is significant. This is probably because incorporating the implicit negative user feedback can help learn more accurate user interest representations.

Next, we explore the influence of incorporating different news information for news modeling. We compare the performance of our approach with its variants with different combinations of news information, and the results are shown in Fig. 4. From Fig. 4, we have several observations. First, both news title and body are useful for news recommendation. This is because users usually decide which news to click based on its title, and read news body for detailed information. Thus, both news title and body can provide important information for news recommendation. Second, news body is more useful than news title in representing news. This is intuitive because news body usually contains the details of news, which can provide richer information than news title. Besides, combining both title and body of news can further improve the performance of our approach. These results validate the effectiveness of the multi-view learning framework in our approach.

Then, we explore the influence of different attention networks on the performance of our approach. The results are shown in Fig. 5. From Fig. 5, we find that word-level attention networks are very useful for our approach. This is because different words in news titles and bodies usually have different importance in representing news. Thus, selecting important words can help learn more informative news representations. In addition, news-level attention network is also important. This is because different news usually have different informativeness in representing the preferences of users, and selecting informative news can benefit user representation learning. Besides, the interactive attention network can also boost the performance. This may be because it can help capture the relations between news title and body to enhance news modeling. Moreover, combining all these kinds of attention networks can further improve the performance of our approach. These results verify the effectiveness of each kind of attention mechanism in our approach.

In this section, we study the influence of an important hyperparameter on our approach, i.e., the dwell time threshold $T$. We compare the performance of our approach w.r.t different values of $T$. The results are illustrated in Fig. 6. We find the performance of our approach improves when the threshold $T$ increases. This is probably because when $T$ is too small, many negative news clicks cannot be distinguished from the positive ones, and the modeling of user interest is not accurate enough. In addition, the performance of our approach declines when $T$ goes too large. This is also intuitive because when $T$ is too large, many positive news clicks will be mistakenly regarded as negative ones and the errors will be encoded into the negative user embedding, which leads to the sub-optimal performance. Therefore, a moderate selection of $T$, e.g., 10 seconds may be more appropriate for our approach, which is also consistent with our news browsing experiences.

In this section, we conducted several studies on the results of other two key parameters learned by our model, i.e., $w_{p}$ and $w_{n}$. The results in 10 independent experiments are illustrated in Fig. 7. According to these results, we find $w_{p}$ is consistently positive. It is intuitive because the news read by users carefully can usually represent the preferences of users. Thus, the similarities between the candidate news and the positive clicked news should have a positive impact on the click probability. In addition, we find $w_{n}$ is consistently negative. This may be because if a user reads a news article very quickly, we can infer that he/she is probably not interested in this news. Thus, the click probability should have a negative correlation with the similarities between candidate news and negative clicked news.

In this section, we will conduct several qualitative analysis on the characteristics of the negative clicked news. First, we want to compare the topic distributions of positive and negative news, and the results are shown in Fig. 8. According to Fig. 8, we find the ratios of negative news in different topic categories have some differences. For example, it is interesting that the ratio of negative news is the highest in the “video” category. This may be because many users are attracted by the titles of the news in this category, but they are not interested in the content of these news. Thus, incorporating the information of negative feedback may also implicitly encode topical information into user representations, which is useful for more accurate news recommendation.

In addition, we aim to discover some common patterns of negative news, e.g., the contents and writing styles of their titles. We calculate the ratio of negative feedback (NF) for each news⁵⁵5We ignore the news with less than 10 clicks to filter some possible noisy news., and the titles of the top 10 news with the highest NF ratios are listed in Table 3. From Table 3, we find many news with top NF ratios are clickbaits, and they have very similar writing styles, i.e., describing something in a sensationalized way but no details are provided. Thus, many users may be attracted by the news titles and click these news. However, after taking a glance at the news body, many users may be disappointed and close these news quickly. Thus, it is harmful for news platforms to improve user experiences if clickbaits are frequently recommended to users. Fortunately, our approach may have the potential to avoid recommending too many clickbaits to user by incorporating the negative feedback of users, which can improve the reading experiences of users.

In this section, we conducted several case studies to show the effectiveness of our approach. First, we want to visually explore the effectiveness of incorporating the negative feedback of users. The clicked news and candidate news of a randomly selected user are listed in Table 4. We calculate the click probability score respectively using our approach and its variant without negative feedback. According to Table 4, the second candidate news is assigned the highest click score when negative feedback is not considered. This is probably because it is very similar to the second and third clicked news, since all of them are about Oscars. However, we can infer that this user is probably not very interested in this topic since she closes the second and third clicked news very quickly. Thus, it may be ineffective to recommend Oscars related news to this user. Fortunately, our approach can assign the second candidate news a low click score. This is because our approach can take the implicit negative user feedback into consideration, which is beneficial for modeling user preferences more accurately.