Intent-aware Ranking Ensemble for Personalized Recommendation

Ranking ensemble, i.e., fusing multiple ranking lists for a consensus list, has been long discussed in IR scenario (Dwork et al., 2001b; Farah and Vanderpooten, 2007) and proved to be NP-hard even with small collections of basic lists (Dwork et al., 2001a).

In general, rank aggregation includes unsupervised and supervised methods. Unsupervised methods only rely on the rankings. For instance, Borda Count (Borda, 1784) computed the sum of all ranks. MRA (Fagin et al., 2003) adopted the median of rankings. Comparisons among basic ranks were also used, such as pair-wise similarity in Outrank (Farah and Vanderpooten, 2007)and distance from null ranks in RRA (Kolde et al., 2012). Recently, researchers have paid attention to supervised rank aggregation methods. For example, the Evolutionary Rank Aggregation (ERA) (Oliveira et al., 2016) was optimized with genetic programming. Differential Evolution algorithm (Bałchanowski and Boryczka, 2022a, b) and reinforcement learning (Zhang et al., 2022) were also adopted for rank aggregation optimization. However, these rank aggregation methods only utilized the rank or scores of items in basic lists without considering item contents and users in recommendation.

Another view on the fusion problem comes from ensemble learning. It is a traditional topic in machine learning (Sagi and Rokach, 2018), which has been successfully applied to various tasks (Wang et al., 2014; Verma and Mehta, 2017; Mendes-Moreira et al., 2012). A basic theory in ensemble learning is error-ambiguity (EA) decomposition analysis (Krogh and Vedelsby, 1994), which proves better performance can be achieved with aggregated results with good and diverse basic models. It was proved in classification and regression with diverse loss functions (Brown et al., 2005; Yin et al., 2014). Liu et al. (2022) generalized EA decomposition to model-level weights in ranking ensemble with list-wise loss, where different items in a list shared the same weights.

The differences between the previous studies and our method are mainly twofold: First, rather than calculate a general weight for each basic model, we extend to assign item-level weights considering item category and user behavior intents. We theoretically prove the effectiveness of this extension. Second, we aim to combine heterogeneous lists generated for different behavior objectives and simultaneously improve performance on multiple objectives.

Since we aggregate ranking lists aware of users’ multiple intents, we briefly introduce recent methods on multi-intents and multi-interests in recommender systems. Existing studies focused on capturing dynamic intents in the sequential recommendation (Chen et al., 2019; Wang et al., 2020b; Liu et al., 2021, 2020; Wang et al., 2020a). For instance, AIR (Chen et al., 2019) predicted intents and their migration in users’ historical interactions. Wang et al. (2020b) modeled users’ dynamic implicit intention at the item level to capture item relationships. MIND (Sabour et al., 2017) and ComiRec (Cen et al., 2020) adopted dynamic routing from historical interactions to capture users’ multi-intents and diversity. TimiRec (Wang et al., 2022b) distilled target user interest from predicted distribution on multi-interest of the users. With the development of contrastive learning, implicit intent representations were also applied as constraints on contrastive loss (Chen et al., 2022; Di, 2022).

Previous studies usually mixed “intent” and “interest” and paid attention to intent on item contents in single-behavior scenarios. However, we follow (Chen et al., 2019) to consider both behavior intents and item category intents. Moreover, instead of learning user preference for each intent, we utilize intents as guidance for fusing user preference with different behavior objectives.

Another brunch of related but different work is the multi-objective recommendation. It mainly contains two groups of studies. One group provides multiple recommendation lists for different objectives with shared information among objectives, such as MMOE (Ma et al., 2018) and PLE (Tang et al., 2020), where different lists are evaluated on corresponding objectives separately. The other group tried to promote the model performance on a target behavior objective with the help of other objectives, such as MB-STR (Yuan et al., 2022) predicting users’ click preferences. However, instead of generating multiple lists or specifying a target behavior, we fuse a uniform list on which multiple objectives are evaluated simultaneously.

Some studies that tried to jointly optimize ranking accuracy and other goals are also called multi-objective recommendation, such as fairness (Wang et al., 2022a), diversity (Castells et al., 2022), etc. They sought to promote other metrics while maintaining utility on some behavior. But we aim to concurrently promote performance on multiple objectives by aggregating various recommendation lists.

Let $\mathcal{F}=\{f^{1},f^{2},...,f^{K}\}$ be $K$ basic models that are trained for $K$ different objectives (such as click, buy, and favorite, etc.), $\mathcal{I}(u,c)=\{i_{1},i_{2},...,i_{N}\}$ be the union set of $K$ recommended basic item lists for user $u$ in session environment context $c$ (e.g., time and location), and $S^{k}_{n}(u,c)=f^{k}(i_{n},u,c)$ be the predicted score given by basic model $k$ on item $n$. The goal of ranking ensemble learning is to learn a weighted ensemble score $S^{ens}_{n}(u,c)$ for each item $i_{n}$ in $\mathcal{I}$,

Where $w^{k}_{n}(u,c)\in\mathbb{R}$ denotes the weight of the $k$-th basic model for item $i_{n}$. The weights are learnable with the help of side information, e.g., user intents and item categories. The items in $\mathcal{I}$ are sorted according to $S_{n}^{ens}$, and are compared with a ground truth order of ranking $\pi^{u,c}=\{\pi_{1},\pi_{2},...,\pi_{N}\}$, which is sorted to users’ interactions with a pre-defined priority of user feedback, e.g., Buy¿Click¿Examine. The priority can be defined by business realities and will not influence the model learning strategy. The definition of ranking ensemble learning is similar to previous work (Liu et al., 2022; Bałchanowski and Boryczka, 2022a; Oliveira et al., 2020), except that we conduct ensemble on heterogeneous basic models with different objectives, which makes the problem more difficult. The main notations are shown in Table 1.

When aggregating basic models optimized with different objectives, users’ intent about behaviors and item categories are both essential. Therefore, we define a user’s intent in a visit as a probability distribution of item categories and behaviors,

Where $I$ and $B$ indicate the item category intents and behavior intents, respectively. The types of categories and behaviors vary with recommendation scenarios. For instance, in online shopping, $I$ can be product class, and $B$ may include clicking and buying. In music recommender systems, $I$ may be music genre, while $B$ can contain listening and purchasing albums. In experiments, user intents $Int$ are predicted from users’ historical interactions and environment context.

Three representative losses are generally leveraged in the recommendation scenario, namely point-wise, pair-wise, and list-wise loss. We will theoretically and empirically illustrate the effectiveness of ranking ensemble with three losses in the following sections.

For a given user $u$ under session context $c$ with multi-level ground truth ranking $\pi^{u,c}=\{\pi_{1},\pi_{2},...\pi_{N}\}$ on an item set $\mathcal{I}=\{i_{1},i_{2},...,i_{N}\}$, the loss of score list $\mathbf{S}(u,c)=\{S_{1},S_{2},...,S_{N}\}$ is defined as ($u$ and $c$ are omitted):

• •

  Point-wise Loss As $\pi$ is a multi-level feedback based on a group of user feedback, the Mean Squared Error (MSE) loss is utilized as a representative point-wise loss,

  (3)

  $$l_{m}(\pi,\mathbf{S})=\frac{1}{N}\sum_{n=1}^{N}l_{m}(\pi_{n},S_{n}):=\frac{1}{N}\sum_{n=1}^{N}(S_{n}-\pi_{n})^{2}$$

• •

  Pair-wise Loss We leverage the Bayesian Personalized Ranking (BPR) loss (Rendle et al., 2012). Following the negative sampling strategy for multi-level recommendation (Luo et al., 2020), a random item from one level lower is paired with a positive item at each level,

  (4)		$\displaystyle l_{b}(\pi,\mathbf{S}):$	$\displaystyle=\frac{1}{N^{+}}\sum_{l=1}^{L}\sum_{n,m\in{I^{+}_{l},I^{-}_{l}}}l_{b}(S_{n},S_{m})$
  		$\displaystyle l_{b}(S_{n},S_{m})$	$\displaystyle=-log\sigma(S_{n}-S_{m})$

  Where $L$ is the number of interaction levels (e.g., buy, click, and exposure), $N^{+}$ is the number of positive items of all levels, $I^{+}_{l}$ and $I^{-}_{l}$ are positive and one-level-lower negative item set for level $l$, and $\sigma$ is the sigmoid function.

• •

  List-wise Loss Following (Liu et al., 2022), we adopt the Plackett-Luce (P-L) model as the likelihood function of ranking predictions,

  (5)

  $$P_{p-l}(\pi|\mathbf{S})=\frac{1}{N}\prod_{n=1}^{N}\frac{\exp(S_{\pi_{n}})}{\sum_{m=n}^{N}\exp(S_{\pi_{m}})}$$

  Where $\pi_{n}$ indicates the $n$-th item sorted by ground truth $\pi$. The corresponding list-wise loss function is

  (6)

  $$l_{p-l}(\pi,\mathbf{S}):=-\log[P_{p-l}(\pi|\mathbf{S})]$$

Since the ambiguity $A_{n}^{k}$ in Eq. 8 is positive and $w_{n}^{k}\geq 0$, Eq. 7 follows the form of EA decomposition. Ranking ensemble with item-level weights for point-wise loss is effective theoretically, since the ensemble loss is smaller than weighted sum of basic model losses with any $w_{n}^{k}$ as long as $w_{n}^{k}\geq 0$ and $\sum_{k=1}^{K}w_{n}^{k}=1$.

For brevity, we will omit statements of score lists and ensemble formulas in the following theorems.

Due to space limitation, we only show key steps in the proof:

The range limitation of $w_{n}^{k}$ leads to $\delta\leq 1$. And in pair-wise loss, the order rather than the values of scores matters. So the second term in Eq. 12 ($\delta\sum_{k=1}^{K}S_{m}^{k}<\sum_{k=1}^{K}S_{m}^{k}$) can be arbitrarily small. Meanwhile, the ambiguity $A_{nm}^{k}$ is semi-positive. Therefore, Eq.12 follows the form of EA decomposition, and our ranking ensemble method with pair-wise loss is effective theoretically.

Due to space limitation, we only show key steps in the proof:

Because in the list-wise optimization, the order rather than the values of scores matters, $S_{n}^{ens}$ can be arbitrarily small. Meanwhile, the ambiguity term $A_{n}^{k}$ is semi-positive. Therefore, Eq.17 conforms to the EA decomposition theory, and our ranking ensemble method with list-wise loss is effective.

The above theorems guarantee our proposed ranking ensemble learning method in theory for three representative loss functions. With EA decomposition theory, we prove that the loss of ensemble list is smaller than any weighted sum combination of losses of basic lists: $l_{ens}(\pi,S^{ens})\leq\sum_{k}w_{k}l_{k}(\pi,S^{k})-A+\Delta,\forall w_{k}\leq 0,\sum_{k=1}^{K}w_{k}=1$, where $A$ is a positive ambiguity term, and $\Delta$ is arbitrarily small. Therefore, the ensemble loss $l_{ens}(\pi,S^{ens})$ (i.e., differences between ensemble list and ground truth) is possible to be smaller than any basic list loss with suitable weights $\{w_{n}^{k}\}$, and larger ambiguity $A$ will lead to a smaller bound of ensemble loss. Thus, it can be effective for our ranking ensemble task.

In practice, since basic lists are fixed (so $l_{k}(\pi,S^{k})$ are constants), we aim to minimize the ensemble ranking loss $l_{ens}(\pi,S^{ens})$ and maximize the ambiguity $A$. Therefore, the loss function for ranking ensemble learning, $l_{el}$, is defined as follows,

where $l_{ens}(\pi,\mathbf{S}^{ens})$ can be any of the $l_{m}(\pi,\mathbf{S}^{ens})$, $l_{b}(\pi,\mathbf{S}^{ens})$, and $l_{p-l}(\pi,\mathbf{S}^{ens})$, and $A$ indicates the ambiguity term. For BPR and P-L loss, there exists an interpolation $\tilde{S}_{n}^{ens}=S_{n}^{ens}+\theta(S_{n}^{k}-S_{n}^{ens})$ in $A$. To simplify the calculation, we let $\theta\rightarrow 0$ without loss of generality.

After we proved the effectiveness of item-level weights $\{w_{n}^{k}\}$ for ranking ensemble with three different loss functions, we need to design a neural network for learning the weights $w_{n}^{k}$. As shown in Section 3.2, users’ intents about behaviors and item categories help aggregate the basic lists, while intents are not available in advance. Therefore, an intent predictor and an intent-aware ranking ensemble module are designed for our method.

The main framework of our Intent-aware Ensemble Learning (IntEL) method is shown in Figure 2. For a user $u$ at time $T$, user intents $Int$ are predicted with an intent predictor from her historical interactions and current environment context. Then, with $N$ candidate items generated from $K$ basic models, an intent-aware ranking ensemble module is adopted to integrate basic list scores, item categories, and the predicted user intents. The output of the ensemble module is item-level weights $\{w_{n}^{k}\}$ for each item $n$ and basic model $k$. Eventually, weighted sum of all basic list scores constructs the ensemble scores $\{S^{ens}_{k}\}$ for a final list. Since we focus on the ranking ensemble learning problem, a straight-forward sequential model is used for intent prediction in Section 5.2, and we pay more attention to the design of ensemble module in Section 5.3. IntEL is optimized with a combination of ranking loss $l_{ens}(\pi,S)$, ambiguity loss $A$, and intent prediction loss $l_{int}$. Details about the model learning strategy will be discussed in Section 5.4.

As defined in Section 3.2, user intent describes a multi-dimensional probability distribution $Int$ over different item categories and behaviors at each user visit. The goal of the intent predictor is to generate an intent probability distribution for each user visit. We predict intents with users’ historical interactions and environment context, as both historical habits and current situations will influence users’ intents.

For a user $u$ at time $T$, her historical interactions from $T-t$ to $T-1$ and environment context (such as timestamp and location) at $T$ are adopted to predict her intent at $T$, where $t$ is a pre-defined time window. Enviroment context is encoded into embedding $c(u,T)$ with a linear context encoder. Two sequential encoders are utilized to model historical interactions at user visit (i.e., session) level and item level. Session-level history helps learn users’ habits about past intents, while item-level interactions express preferences about item categories in detail. At the session level, the intents and context of each historical session are embedded with two linear encoders, respectively. Then two embeddings are concatenated and encoded with a sequential encoder to form an embedding $h_{s}(u,T)$. At the item level, “intent” of each positive historical interaction can also be represented by its behavior type and item category. Then item-level “intent”s are embedded with the same intent encoder as session-level, and fed into a sequential encoder to form item-level history $h_{i}(u,T)$. The sequential encoder can be any sequential model, such as GRU (Chung et al., 2014), transformer (Sun et al., 2019), etc. Finally, context $c(u,T)$, session-level $h_{s}(u,T)$, and item-level $h_{i}(u,T)$ are concated for a linear layer to predict intent $\hat{Int}(u,T)$ ($u$ and $T$ are omitted),

Where $\mathbf{W}^{I}$ and $b^{I}$ are linear mapping parameters.

The structure of the intent-aware ranking ensemble module is shown in Figure 3(a). Since the final weights $\{w_{n}^{k}\}$ should be learned from both behavior and item categories aware of user intents, the predicted intents, single-behavior-objective basic list scores, and categories of items in basic lists are adopted as inputs.

Firstly, lists of item scores $\{S_{n}^{k}|k\in\{1,2,...,K\},n\in\{1,2,...,N\}\}$ are fed into a self-attention layer to represent the relationship among item scores in the same basic list. Item categories $\{I_{n}|n\in\{1,2,...,N\}\}$ are also encoded with a self-attention layer to capture the intra-list category distributions. The self-attention structure consists of a linear layer to embed scores $\{S_{n}^{k}\}$ (or categories $\{I_{n}\}$) into $d_{e}$-dimensional representations $\mathbf{S}\in\mathbb{R}^{N\times d_{e}}$ (or $\mathbf{I}\in\mathbb{R}^{N\times d_{e}}$) and $T$ layers of multi-head attentions, which follow the cross-relation attention layer proposed by Wang et al. (2020b), as shown in Figure 3(b).

Secondly, user intent $Int$ is embedded into $d_{int}$ dimension with a linear projection $Int_{d}=\mathbf{W}^{i}Int\in\mathbb{R}^{N\times d_{int}}$. Then the influences of user intent on representations of scores and features are obtained with cross-attention layers,

Where the projection matrix $\mathbf{W}_{Q}\in\mathbb{R}^{d_{e}\times d_{int}}$ is shared between two intent-aware attention modules. Since behavior intents and category intents are associated when users interact with recommenders, we use the holistic user intents to guide the aggregations of both basic list scores and item categories rather than splitting the intents into two parts.

Finally, weights $\{w_{n}^{k}\}$ should be generated from all information. Intent-aware score embeddings $\mathbf{A}_{s}$, intent-aware item category embeddings $\mathbf{A}_{i}$, and intent embedding $Int_{d}$ are concatenated and projected into space of $\mathbb{R}^{K}$ to get the weight matrix $\mathbf{W}\in\mathbb{R}^{N\times K}$,

Where $\mathbf{W}^{w}\in\mathbb{R}^{K\times(2d_{e}+d_{int})}$ is a trainable projection matrix. The output matrix $\mathbf{W}=\{w_{n}^{k}\}$ is used as the weights for summing basic model scores as in Eq. 1.

Since an end-to-end framework is to train the intent predictor module and intent-aware ranking ensemble module, joint learning of two modules is utilized for model optimization.

To optimize ranking ensemble results according to theorems via EA decomposition in Section 4, ensemble learning loss $l_{el}$ consists of $l_{ens}(\pi,\mathbf{S}^{ens})$ and $A$ as in Eq.24. Meanwhile, accurate user intents will guide the ranking ensemble, so an intent prediction loss is also used for model training. Since user intents are described by multi-dimensional distributions, KL-divergence (Csiszár, 1975) loss $l_{int}$ is adopted to measure the distance between true intents $Int$ and predicted intents $\hat{Int}$. The final recommendation loss $l_{rec}$ is a weighted sum

Where $l_{ens}(\pi,\mathbf{S}^{ens})$ is the ranking ensemble loss, $A$ is the ambiguity term, and $l_{int}$ is the intent prediction loss. $\alpha$ and $\gamma$ are hyper-parameters to adjust the weights of ambiguity and intent loss, respectively.

The overall performances on Tmall and LifeData are shown in Table 4 and Table 5, respectively. We divide all models into four parts: The first part evaluates on each single-objective basic model’s scores. The second is unpersonalized baseline ensemble models, and the third contains personalized baselines: aWELv and its two variants with user intents. The last part shows our method IntEL with three loss functions. From the results, we have several observations:

First, our proposed IntEL achieves the best performance on all behavior objectives in both datasets. IntEL with three loss functions, i.e., IntEL-MSE, IntEL-BPR, and IntEL-PL, outperform the best baselines on most metrics significantly. Although the two datasets are quite different, as shown in Table 3, IntEL has stable, great ensemble results on both datasets.

Second, IntEL with different loss functions show different performances on two datasets. On Tmall, IntEL-MSE is better than IntEL-BPR and IntEL-PL. It is because there is four-level ground truth $\pi$ (three behaviors), and ranking on such diverse item lists is close to rating prediction. Therefore, IntEL-MSE, which directly optimizes the ensemble scores, performs better than IntEL-BPR and IntEL-PL, which optimize the comparison between rankings. On LifeData, IntEL-PL and IntEL-BPR perform better since LifeData has shorter sessions with fewer positive interactions (as in Table 3). So comparison-based BPR and P-L achieve better performance.

Third, comparing different baselines, we find that supervised methods ($\lambda$Rank, ERA, and aWELv) outperform unsupervised RRA and Borda greatly on Tmall. It is because heterogeneous single-behavior objective models (Single XXX) have diverse performance, making rank aggregation difficult for unsupervised methods.

Lastly, aWELv and its variants perform well on LifeData but not on Tmall since session lists are generally longer (Table 3) for Tmall, and list-level weights of aWELv miss useful intra-list information. So item-level weights that consider item category intents are necessary. Nevertheless, aWELv is better than basic models in both datasets, which is consistent with the theory. Moreover, aWELv+Int/IntEL outperform aWELv on most metrics, indicating that user intents contributes to ranking ensemble learning.

To further explore the performance of our ranking ensemble learning method, we conduct an ablation study, analysis of user intents, and hyper-parameters analysis on the best model for each dataset, i.e., IntEL-MSE for Tmall and IntEL-PL for LifeData.