Bilateral Self-unbiased Learning from Biased Implicit Feedback

Collaborative filtering (CF) (Ricci et al., 2015; Choi et al., 2021; Lee et al., 2013; Adomavicius and Tuzhilin, 2005) is the most prevalent technique for building commercial recommender systems. CF typically utilizes two types of user feedback: explicit and implicit feedback. Explicit feedback provides richer information about user preferences than implicit feedback as users explicitly rate how much they like or dislike the items. However, it is difficult to collect explicit feedback from various real-world applications because only a few users provide feedback after experiencing the items. On the other hand, implicit feedback is easily collected by recording various users’ behaviors, e.g., clicking a link, purchasing a product, or browsing a web page.

There are several challenges in using implicit feedback in CF. (i) Existing studies (Hu et al., 2008; He et al., 2017; Xue et al., 2017; He et al., 2018; Tang and Wang, 2018; Wu et al., 2016; Yao et al., 2019; Shenbin et al., 2020; Prathama et al., 2021; Choi et al., 2022) regard the observed user interactions solely as positive feedback. However, some observed interactions, such as clicking an item or viewing a page, do not necessarily indicate whether the user likes the item; there is a semantic gap between the true relevance and the observed interactions. (ii) User feedback is observed at uniformly not random. For instance, users tend to interact more with popular items, so the more popular the items are, the more they are collected in the training dataset. Because of the inherent nature of implicit user-item interactions, recommender models are biased toward ranking popular items with high priority.

Existing studies have developed unbiased recommender learning methods (Saito et al., 2020; Saito, 2020b; Zhu et al., 2020; Qin et al., 2020) to estimate true user preferences from implicit feedback under the missing-not-at-random (MNAR) assumption (Marlin et al., 2007; Steck, 2010; Zheng et al., 2021). They formulated a new loss function to eliminate the bias of items by using inverse propensity weighting (IPW), which has been widely established in causal inference (Joachims et al., 2017; Wang et al., 2018a; Ai et al., 2018; Hu et al., 2019; Wang et al., 2021a, 2016; Ovaisi et al., 2020; Schnabel et al., 2016; Wang et al., 2021b, 2019, 2019; Saito et al., 2020; Saito, 2020b; Zhu et al., 2020; Qin et al., 2020). In addition, recent studies (Zhang et al., 2021; Wei et al., 2021) introduced a causal graph that represents a cause effect relationship for recommendations and removes the effect of item popularity.

Specifically, they are categorized in two directions. The first approach exploits a heuristic function to eliminate the popularity bias of items. Although item popularity is a critical factor in the bias of training datasets, there are other vital factors, such as exposure bias, in the recommended models. To overcome this issue, the second approach develops a learning-based method that accounts for various bias factors. Joint learning methods (Saito, 2020a; Zhu et al., 2020) first suggested utilizing a pseudo-label or propensity score inferred from an additional model. Zhu et al. (2020) employed multiple models for different subsets of a training dataset to infer the propensity scores. Saito (2020a) adopted two pre-trained models with other parameter initializations and generated a pseudo-label as the difference between the two model predictions. They then made use of consistent predictions by training multiple models. However, as multiple models converge to a similar output, it leads to an estimation overlap issue. Recently, causal graph-based training methods (Zhang et al., 2021; Wei et al., 2021) were proposed to overcome the sensitivity of IPW strategies. Wei et al. (2021) modeled a causal graph using item popularity and user conformity to predict true relevance. Zhang et al. (2021) analyzed the negative effects of item popularity through a causal graph and removed bias through causal intervention. However, they did not address the exposure bias caused by recommender models.

To eliminate exposure bias, we propose a novel unbiased recommender learning model, namely the BIlateral SElf-unbiased Recommender (BISER) with two key components: self-inverse propensity weighting (SIPW) and bilateral unbiased learning (BU). Motivated by self-distillation (Mobahi et al., 2020; Yang et al., 2019), we first devise a self-inverse propensity weighting (SIPW) to iteratively mitigate the exposure bias of items. Specifically, we reuse the model prediction from the previous training iteration, enabling us to gradually eliminate the exposure bias of items as the training evolves. Notably, SIPW has two key advantages: (i) it effectively handles the exposure bias caused by recommender models and (ii) it does not require an additional model for propensity score estimation.

IPW usually suffers from the high variance problem, as reported in the literature (Zhang et al., 2021; Wei et al., 2021). To resolve this issue, we first assume that the true user preference should be consistent with the predictions of different models. We then design bilateral unbiased learning (BU) using two recommender models. Specifically, we utilize user- and item-based autoencoders (Sedhain et al., 2015). Because they capture different hidden patterns on the user and item sides, it does not require us to split the training and estimation subsets from an entire training set (Zhu et al., 2020) or to utilize multiple models with different parameter initializations (Saito, 2020a). We exploit the predicted value of one model as a pseudo-label for another model. As a result, we resolve the high-variance issue in estimating the SIPW.

To summarize, the key contributions of this paper are as follows:

• •

  We review existing studies on unbiased recommender learning and analyze their limitations to eliminate the exposure bias of items caused by recommender models (Section 2).

• •

  We propose a novel unbiased recommender learning model, namely BIlateral SElf-unbiased Recommender (BISER), utilizing (i) a learning-based propensity score estimation method, i.e., self-inverse propensity weighting (SIPW), and (ii) bilateral unbiased learning (BU) using user- and item-based autoencoders with complementary relationships (Section 3).

• •

  We demonstrate that the BISER outperforms state-of-the-art unbiased recommender models, including RelMF (Saito et al., 2020), AT (Saito, 2020a), CJMF (Zhu et al., 2020), PD (Zhang et al., 2021), and MACR (Wei et al., 2021), on both unbiased evaluation datasets (e.g., Coat and Yahoo! R3) and conventional datasets (e.g., MovieLens and CiteULike) (Sections 4–5).

Notations. Formally, we denote $\mathcal{U}$ as a set of $m$ users and $\mathcal{I}$ as a set of $n$ items. We are given a user-item click matrix $\textbf{Y}\in\{0,1\}^{m\times n}$,

We model $y_{ui}$ as a Bernoulli random variable, indicating the interaction of a user $u$ on an item $i$. For implicit feedback data, the user interaction is a result of observation and preference. That is, a user may click on an item if (i) the item is exposed to the user and the user is aware of the item (exposure) and (ii) the user is actually interested in the item (relevance). Existing studies (Schnabel et al., 2016; Saito, 2020b; Saito et al., 2020; Chen et al., 2020; Zhu et al., 2020; Qin et al., 2020) formulate this idea as follows:

where $o_{ui}$ is an element of the observation matrix $\textbf{O}\in\{0,1\}^{m\times n}$ that represents whether the user $u$ has observed the item $i$ ($o_{ui}=1$) or not ($o_{ui}=0$), and $r_{ui}$ is an element of the relevance matrix $\textbf{R}\in\{0,1\}^{m\times n}$, representing true relevance regardless of observance. If the user $u$ likes the item $i$, $r_{ui}=1$, and $r_{ui}=0$ otherwise. For simplicity, we denote $P(o_{ui}=1)$ and $P(r_{ui}=1)$ with $\omega_{ui}$ and $\rho_{ui}$, respectively. The interaction matrix Y with biased user behavior is decomposed into element-wise multiplication of the observation and relevance components.

Similarly, the ideal loss function that relies purely on relevance is formulated as follows:

where the biased observation $y_{ui}$ is substituted with the pure relevance $\rho_{ui}=P(r_{ui}=1)$.

Because user interaction data are typically sparse and collected under the MNAR assumption, it is necessary to bridge the gap between the loss functions for clicks and relevance. Saito et al. (2020) proposed a loss function using inverse propensity weighting (IPW):

where $\omega_{ui}$ is the inverse propensity score that indicates the probability of observing the item $i$ by the user $u$. As proved in (Saito et al., 2020), the expectation of the unbiased loss function in Eq. (5) is equivalent to the ideal loss function in Eq. (4):

The critical issue is how to estimate the propensity score $\omega_{ui}$ from observed feedback (e.g., clicks). Previous studies (Saito et al., 2020; Saito, 2020b; Zhu et al., 2020; Qin et al., 2020; Lee et al., 2021) have developed several solutions to estimate $\omega_{ui}$. First, (Saito et al., 2020; Saito, 2020b; Lee et al., 2021) introduced a heuristic function to estimate item popularity without using an additional model. Although intuitive, it focuses only on addressing item popularity and is thus, incapable of handling exposure bias caused by recommender models. Second, additional models were used to estimate the propensity score (Zhu et al., 2020; Qin et al., 2020). However, this incurs high computational costs, as they mostly utilize three or more models.

To motivate unbiased recommender modeling, we first investigate the bias from conventional recommendation models, i.e., matrix factorization (MF) and autoencoders (AE) (Sedhain et al., 2015) on the MovieLens-100K (ML-100K) dataset. Figures 2(a) and (c) depict a positive correlation between the average predicted scores for clicked items and the number of ratings in the training set, i.e., popularity. From this case study, we confirm that popular items tend to be recommended to more users with conventional models. Even worse, the biased recommendations can exacerbate the bias of the training dataset as user feedback is continually collected.

To address this problem, it is vital to eliminate the exposure bias caused by recommender models. Existing studies (Saito, 2020b; Saito et al., 2020) have mainly focused on modeling the popularity bias of items. Because user experiences are mostly biased towards the recommended items, we aim to eliminate exposure bias caused by recommender models. We estimate the propensity score of the items and eliminate exposure bias during model training. Figures 2(b) and (d) show the effectiveness of the proposed unbiased learning method. It is clearly observed that the correlation between the actual and predicted ratings in the proposed model is weaker than that in the traditional models. This pilot study indicates that our unbiased model effectively reduces exposure bias caused by recommender models.

To account for implicit feedback, five datasets with explicit feedback (i.e., Coat¹¹1https://www.cs.cornell.edu/~schnabts/mnar/, Yahoo! R3²²2http://webscope.sandbox.yahoo.com/, MovieLens³³3http://grouplens.org/datasets/movielens/ (ML)-100K, ML-1M, and ML-10M) are converted into implicit feedback. We treat the items with four or higher scores as positive feedback and the remaining ratings are regarded as missing feedback. Unlike the other five datasets, CiteULike (Wang et al., 2015) provides implicit feedback.

We conduct the following experiments with different settings:

• •

  MNAR-MAR: Using Coat and Yahoo! R3, we train the model on the training dataset collected under the MNAR assumption and evaluate it on the test set collected under MAR. The training and test sets are provided separately. Within the training set, we set 30% of the ratings per user for validation purposes. The test set of the Coat and Yahoo! R3 datasets consists of 16 and 10 item ratings per user, respectively. For the items in the test set, we retrieve the top-$N$ items.

• •

  MNAR-MNAR: Using four traditional datasets, we train and evaluate the recommender models under the MNAR assumption. Before splitting the dataset, we removed users who rated 10 or fewer items and items rated by 5 or fewer users. For each user, we randomly held 80% and 20% as the training and test sets, respectively, and further split 30% of the training set into the validation set. Then, we evaluate the top-$N$ recommendation for all unrated items.

• •

  Matrix Factorization (MF) (Hu et al., 2008): The most popular recommender model for using linear factor embeddings in which the user preference score is predicted by a product of user- and item-embedding matrices.

• •

  User-based AutoEncoder (UAE) (Sedhain et al., 2015): This model learns non-linear item-item correlations for the users with a set of items using autoencoders.

• •

  Item-based AutoEncoder (IAE) (Sedhain et al., 2015): By transposing the rating matrix, this model learns non-linear user-user correlations, using autoencoders.

• •

  Relevance Matrix Factorization (RelMF) (Saito et al., 2020): This utilizes the MNAR scenario for model training, effectively removing the bias caused by item popularity. The IPW-based model estimates the propensity scores using a heuristic function for item popularity.

• •

  Asymmetric Tri-training (AT) (Saito, 2020a): It uses two pre-trained models to generate reliable pseudo-ratings and the other model is trained with pseudo-ratings to make final predictions. Note that RelMF (Saito et al., 2020) is used for the pre-trained models.

• •

  Combinational Joint Learning for Matrix Factorization (CJMF) (Zhu et al., 2020): As an IPW-based model, it introduces a joint training framework to estimate both unbiased relevance and unbiased propensity using multiple sub-models. For our datasets, we set the number of sub-models to eight. It is one of the most compelling competitors due to the performance advantage of unbiased recommender learning.

• •

  Popularity-bias Deconfounding (PD) (Zhang et al., 2021): This uses a causal graph to remove confounding popularity bias through causal intervention. It is implemented using BPRMF (Rendle et al., 2009), as mentioned in (Zhang et al., 2021).

• •

  Model-Agnostic Counterfactual Reasoning (MACR) (Wei et al., 2021): This utilizes a cause-effect view by considering the direct effect of item properties on rank scores to remove the popularity bias. We report MACR using LightGCN (He et al., 2020) because LightGCN is better than BPRMF (Rendle et al., 2009).

For MAR evaluation (Coat and Yahoo! R3), we adopt the Average-Over-All (AOA) evaluation, which is the conventional metric that evenly normalizes the scores for all items. For the MNAR evaluation (MovieLens and CiteULike), we use both AOA and unbiased evaluations. Yang et al. (2018) recently suggested an unbiased evaluation scheme and applies bias normalization to test scores. We follow the unbiased evaluation (Yang et al., 2018) and use $\gamma=2$ for the normalization parameter.

We employ AOA and unbiased evaluation metrics in (Yang et al., 2018):

where $\mathcal{S}_{u}$ denotes a set of clicked item to user $u$ in the test set and $\hat{Z}_{ui}$ denotes the predicted ranking of item $i$ to user $u$. Also, the function $c(\cdot)$ denotes any top-N scoring metrics (i.e., NDCG$@N$, MAP$@N$, and Recall$@N$), and the propensity $P_{i}$ is calculated by the popularity of the item $i$. Because MNAR-MNAR evaluation does not have the ground-truth for unbiased test sets, it is difficult to measure the unbiased evaluation in eliminating the exposure bias of items. As an alternative, we evaluate whether the popularity bias is effectively eliminated, as done in (Yang et al., 2018), which is a more favorable setting for existing studies (Saito et al., 2020; Saito, 2020a) that handle item popularity bias.

In this section, we report extensive experimental results for BISER in both the MNAR-MAR and MNAR-MNAR settings. We also analyze the performance of BISER by item group and the effectiveness of each component (i.e., SIPW and BU) in the MNAR-MAR setting. For the MNAR-MAR setting, we reasonably validate the effectiveness of BISER. For the MNAR-MNAR setting, we also evaluate whether BISER is still effective in general evaluation environments.

Unbiased learning has been widely proposed for causal inference (Wang et al., 2016; Joachims et al., 2017; Wang et al., 2018a; Ai et al., 2018; Hu et al., 2019; Lee et al., 2020; Wang et al., 2021a; Ovaisi et al., 2020; Schnabel et al., 2016; Wang et al., 2021b, 2019; Saito, 2020a; Saito et al., 2020; Saito, 2020b; Zhu et al., 2020; Qin et al., 2020; Bonner and Vasile, 2018; Zhang et al., 2021; Wei et al., 2021; Zheng et al., 2021; Jeon et al., 2022) and missing data analyses (Marlin et al., 2007; Steck, 2010; Hernández-Lobato et al., 2014; Marlin and Zemel, 2009; Wang et al., 2018b; Ma and Chen, 2019; Liang et al., 2016). It has since been adopted to bridge the gap between interaction and relevance data in the information retrieval (IR) community, that is, unbiased learning-to-rank (LTR). Unbiased recommender learning, inspired by unbiased LTR, has been actively studied to eliminate bias from explicit and implicit feedback.

This paper proposes a novel unbiased recommender model, namely BIlateral SElf-unbiased Recommender learning (BISER). To the best of our knowledge, this is the first paper to introduce self-inverse propensity weighting to eliminate exposure bias of items during model training. Then, we employed bilateral learning that takes advantage of user- and item-based autoencoders with heterogeneous information, capturing different hidden correlations across users/items. This process helped alleviate high variance in the estimated inverse propensity scores. Extensive experiments demonstrated that the BISER consistently outperformed existing unbiased recommender models in two evaluation protocols: MNAR-MAR and MNAR-MNAR settings.