Large-scale Collaborative Filtering with Product Embeddings

A commonly employed approach to collaborative filtering is to associate vectors of latent factors $\mathbf{v}_{u},\mathbf{w}_{i}\in\mathbb{R}^{d}$ with each user and item. The degree to which some query item $q\in\mathcal{A}$ is relevant for a specific user is taken to be a function of the inner product between these vectors (Koren et al., 2009),

We can then formulate the problem as recovering the observed interactions and estimate parameters by minimizing some suitable loss function,

Common choices of loss include weighted least squares (Hu et al., 2008) or the logistic loss (Johnson, 2014).

While such linear embedding approaches to collaborative filtering perform well in a variety of settings, there are a number of factors that make it difficult to apply in a large-scale recommendation setting like the one considered here.

First and foremost is the sheer number of parameters one must estimate. For example, the dataset used in this work consists of 40 million users and 10 million items. Using $d=64$ dimensional embeddings would require estimating over three billion parameters.

Furthermore, since the number of observed interactions tends to grow linearly with the number of users, the user-item interaction matrix becomes increasingly sparse. Latent factor models operating in this regime require careful regularization to ensure models generalize appropriately to unseen interactions (Mnih and Salakhutdinov, 2008).

Lastly, since latent factor models are transductive in nature, i.e. they do not generalize to new users or items, rapidly incorporating new information into the system can be problematic. For example, incorporating new item interactions into the user representation typically requires solving an optimization problem (Johnson, 2014).

The transductive nature of latent factor models can be ameliorated by replacing the linear embedding, $\mathbf{v}_{u}$, in (1) with a nonlinear embedding of user features,

A number of deep learning based recommendation systems have been proposed, that use a neural network to capture this nonlinear embedding (Dziugaite and Roy, 2015; Salakhutdinov et al., 2007). A common focus of these works is devising methods which are able to exploit various types of side information which are difficult to incorporate when using tranditional matrix factorization based techniques. In recommendation contexts, such side information often requires special handling as it may be very sparse (search queries, geographic location) (Cheng et al., 2016; Wang et al., 2017; Guo et al., 2017), or temporally structured (session or browsing history) (Quadrana et al., 2017).

Historical user behavior is commonly incorporated into deep learning based recommendation systems using a bag-of-words style encoding of past user-item interactions, i.e, summing or averaging item embeddings (Sedhain et al., 2015; Covington et al., 2016; Zheng et al., 2016b). Several recent works have explicitly explored order-independent combination operators operating over sets (Iyyer et al., 2015; Zaheer et al., 2017; Santoro et al., 2017), and surprisingly good performance can be achieved using simple sums, averages, and TF-IDF style weighted averages of word embeddings (White et al., 2015; De Boom et al., 2016) in an NLP context. The use of vector averaging is especially appealing due its simplicity, ability to deal with variable sized user behavior histories, and the ease with which new information can be incorporated.

However, as we show experimentally in Section 5, using a static combination operation may be suboptimal in some settings and significant improvements can be achieved using learned combination operators that can adapt to the context at hand.

Despite modeling users using flexible nonlinear embeddings, most deep learning based recommender systems share the common architectural feature of modeling relevance as the inner product between a separate user and item vector, as in (2). While this may be sufficient given a large enough number of factors and training dataset, simply enlarging this latent space may be suboptimal both from a generalization and a computational perspective. An alternative, which is explored in this work, is to assume that relevance is some nonlinear function of both the user and the query item.

This idea has been explored by a number of authors. Weston et al. (2013) propose learning multiple vectors for each user and select among them using a max operation. A learned weighting of individual user and item factors is proposed in He et al. (2017). In both cases, these models require learning distinct representations of users and suffer from similar problems as matrix factorization based approaches. Closer in spirit to our work is that of Vartak et al. (2017), which proposes a deep learning based recommender system motivated by a meta-learning perspective (Lake et al., 2015). Their model outputs a relevance score given a query item vector and two user vectors computed by averaging the vectors for items in the sets $\mathcal{A}_{u}$ and $\mathcal{A}\setminus\mathcal{A}_{u}$. Our approach offers greater flexibility, moving beyond simple averages and using a more flexible functional form.

Another related approach was recently explored by Grbovic and Cheng (2018). Like the work presented here, the techniques presented in Grbovic and Cheng (2018) rely on item representations learned using a word2vec (Mikolov et al., 2013) like procedure. Unlike this work, Grbovic and Cheng’s training procedure simultaneously learns a user type representation while learning item embeddings. Specific users are then mapped to user types by manually derived rules. Ultimately, these representations are used to derive relevance features (cosine distances) which are input into a decision tree based personalization model. In contrast, our approach leverages item embeddings directly to model user prefeences by learning a parametrized model operating over the items a user has previously interacted with. Decoupling these procedures and treating the embeddings as inputs sidesteps the need to define heuristic rules to map from users to types, and allows models to easily incorporate new behavioral signals without retraining.

In addition to summing or averaging item representations, we explore the use of neural attention mechanism to combine item representations. Neural attention mechanisms (Olah and Carter, 2016; Graves et al., 2014; Bahdanau et al., 2015; Kim et al., 2017) provide a method by which neural networks can focus processing on particular portions of the input data and ignore those which may be irrelevant in the current context. Although most commonly used to augment recurrent neural networks, attention mechanisms can also be used as an alternative to convolution (Kim, 2014) or recurrence when dealing with variable size structured inputs such as sequences, trees, or sets (Vaswani et al., 2017). The model we propose is related to this line of work, largely inspired by Memory-Networks and their derivatives (Weston et al., 2014; Sukhbaatar et al., 2015).

The use of attention mechanisms specifically in recommender system contexts has also been recently explored. In (Chen et al., 2017a), an attention mechanism is used to select the most important historical item interactions given a particular user. Because the attention mechanism conditions explicitly on the user rather than the query item, the attention weights applied to past item interactions, and the resulting user representation, remain constant for a particular user. More similar to our work, is the concurrent approach of (Fu et al., 2018), which applies a matching network (Vinyals et al., 2016) like averaging of past item ratings to predict ratings for unseen items. Unlike our work, (Fu et al., 2018) introduce an additional matrix factorization component to improve performance. Thus, both methods require learning an embedding for each user, something we explictly avoid for the reasons described in Section 1.

We take a probabilistic approach to ranking and aim to model the probability that user $u\in\mathcal{U}$ will interact with item $q\in\mathcal{A}$ based on observed historical user-item interactions $\mathcal{A}_{u}$. The first step in our model is transforming the embeddings of candidate item $q$ and each previously observed item $i\in\mathcal{A}_{u}$ via parametrized functions $\mathbf{h}$ and $\mathbf{f}$ respectively. The transformed representation of $q$ is then compared to each item a user has previously interacted via a dot product. The result of these comparisons is normalized using the softmax function to yield a vector of attention weights,

Next, the embeddings of the items in $\mathcal{A}_{u}$ are reduced to a single $d$-dimensional vector by transforming them using a third parameterized function $\mathbf{g}$, and taking a convex combination with weights $p_{i}$,

Multiple layers of this basic motif are stacked. Information is integrated across layers by adding the query and output of the attention mechanism and using the result as the query in the next layer (Sukhbaatar et al., 2015). A depth-$K$ version of the resulting architecture may be written as

where $k=1,\ldots,K$. The resulting joint representation $\mathbf{z}:=\mathbf{z}^{K}$ of the query item $q$ and the user’s historical item interactions $\mathcal{A}_{u}$ is reduced to a single score

where $\sigma(\cdot)$ is the logistic function, and $\phi\in\mathbb{R}^{d^{\prime}}$ is a weight vector that is shared across all user/query pairs. Figure 1 provides a visual representation of the described architecture.

The use of distinct $\mathbf{f}$, $\mathbf{g}$, and $\mathbf{h}$ is similar to the Key-Value Memory Network of Miller et al. (2016), and allows the underlying item embeddings to be adapted for the various purposes they serve within the model, without requiring us to learn multiple new representations of millions of items. Parameters are not shared between layers, so each may potentially focus on different aspects of similarity. In the experiments presented here, $\mathbf{f}$, $\mathbf{g}$, and $\mathbf{h}$ take the form of single-layer linear neural networks

To optimize the parameters of the model, we temporally partition each user’s item interactions $\mathcal{A}_{u}$ into a set of future and observed item interactions, denoted by $\mathcal{S}_{u}$ and $\mathcal{T}_{u}$ respectively. Predicting a set of future interactions from past interactions reduces the problem to multi-label classification, which could be solved by minimizing the negative log-likelihood,

Unfortunately, in practice $|\mathcal{A}|$ is quite large. This makes the sum over each item in (Eqn. 4) computationally difficult. To alleviate this issue we employ a negative sampling technique (Chen et al., 2017b; Pan et al., 2008; Mikolov et al., 2013). For each user, we sample a set of negative items, $\mathcal{N}_{u}\subset\mathcal{A}$, $\mathcal{N}_{u}\cap\mathcal{S}_{u}=\emptyset$, from a smoothed empirical distribution $P_{\gamma}$ obtained by raising the probability of each item occurring in the training data to a fractional power $0<\gamma<1$ and renormalizing,

To sample negatives while training, we employ the alias method, which allows us to sample from a categorical distribution in $O(1)$ time (Kronmal and Peterson Jr, 1979; Dahl et al., 2012). We also employ a weighting scheme in our loss which gives equal weight to the positive and negative items associated with each training instance. This keeps the scale of the loss independent of the number of positive and negative items that appear in each instance. We found this made tuning hyperparameters easier. Having done this, we arrive at the following loss

The propsed model is fully differentiable and can be trained end-to-end using backpropagation. We use the Adam (Kingma and Ba, 2015) optimizer with $\beta_{1}=0.9$, $\beta_{2}=0.999$, minibatches of size 64, and an initial learning rate of 0.002. We scale the gradient vector to have norm 10 when it exceeds this value (Pascanu et al., 2013). Model performance is assessed on a small holdout set of users every 5,000 updates, and the learning rate is reduced by a factor of 0.8 when the objective does not improve after 5 consecutive assessments (Jozefowicz et al., 2015). Training is terminated once this occurs 20 times. Parameters are initialized using the orthogonal initialization scheme of Saxe et al. (2014). We use $\gamma=0.75$ to smooth the empirical sampling distribution, $|\mathcal{S}_{u}|=10$ holdout items, and $|\mathcal{N}_{u}|=100$ negative samples for each training instance. Unless otherwise mentioned, we use a depth $K=10$ version of our model with hidden layers of size $d^{\prime}=128$. We tune hyper-parameters on the same set of users used for early stopping, although in many cases found that our initial settings gave reasonable performance and did not explore others. The model is implemented in the PyTorch (Paszke et al., 2017) framework. Training takes around 12 hours on a single GPU.

Training datasets were constructed by randomly selecting 40 million anonymized Amazon users and aggregating their product purchases and digital content streams over a 120 day period. To assess the performance of models we measured their ability to predict product interactions in the 7 days immediately following the end of the training period. We also removed any items that a user previously interacted with during the training period, choosing to focus exclusively on predicting new item interactions. Because all models use the same set of pre-trained item embeddings, we did not constrain the datasets described here such that all test items appear in the training data. The embeddings are trained on significantly larger datasets, which confers extremely high coverage to downstream algorithms.

We compare against several baselines inspired by the existing literature. Since our data does not include side-information, we do not compare against content-based or hybrid content/collaborative filtering methods such as (Cheng et al., 2016; Wang et al., 2017; Guo et al., 2017; Chen et al., 2017a). Furthermore, we do not compare to methods that require learning an embedding for each user such as (Oard and Kim, 1998; Johnson, 2014; Fu et al., 2018; He et al., 2017; Dziugaite and Roy, 2015), as doing so would be computationally difficult and distract from our primary focus, modeling user preferences with a parametric function of a user’s past item interactions.

We evaluated models with normalized discounted cumulative gain (NDCG) (Wang et al., 2013; Chen et al., 2009) and Recall@K. NDCG allows us to measure the quality of the entire ranked list, while still putting most of the emphasis on items near the top. Recall@K, which has the benefit of being highly interpretable, is the fraction of items ranked amongst the top $K$ that were actually interacted with by the user.

Because we had to rank an extremely large number of items for each user (our dataset contains several orders of magnitude more items than the widely used Netflix dataset (Bennett et al., 2007)), we followed the common practice of ranking the set of items the user interacted with during the test period amongst a set of randomly sampled negative candidate items that were not interacted with (Koren, 2008; Elkahky et al., 2015; He et al., 2017). To explore the implications of this evaluation technique, we vary both the numbers of negative candidates, and the distribution from which the negatives candidates are drawn. For the latter, three different distributions parameterized as in (Eqn. 5) are used; the empirical distribution ($\gamma=1$), the training distribution ($\gamma=0.75$), and the uniform distribution ($\gamma=0$).

In addition to being computational advantageous, the approach of sampling a pool of candidates is related to the practice of decomposing large-scale recommendation problems into the distinct sub-problems of candidate selection and candidate ranking (Covington et al., 2016; Rudin and Wang, 2018). Within this framework, the candidate selection algorithm typically focuses on surfacing generically applicable content with high precision, possibly incorporating various business criteria or eligibility constraints, and the ranking algorithm must order these items for the current user and/or context. From this perspective, all the candidate generation procedures we consider have perfect recall, but differ in terms of approximating an algorithm with high (the empirical distribution) to low (the uniform distribution) precision.

Comparisons against baselines are displayed in Figures 2 and 3. In Figure 2 we look at how NDCG varies with both the number of additional negative candidates included in the candidate pool, and the distribution from which they are sampled (recall that more negative candidates implies more candidates to rank, i.e. a strictly harder ranking problem). When the sampling distribution for negative candidates resembles that used in training ($\gamma=0.75$) or is set to the empirical distribution ($\gamma=1$), the attention based method outperforms other methods by a clear margin (statistically significant at $\alpha=0.01$), with DAN, our approximation to other state-of-the-art nonlinear recommenders, being the next best performer, and popularity being the worst performer. When negative candidates are sampled from the uniform distribution over items ($\gamma=0$), popularity performs slightly better than the attention based method. However, this performance can be explained by the fact that most items in our dataset are generally “unpopular”, so sorting by popularity discriminates extremely well between the items that were actually interacted with and a relatively small random sample of items, given the highly skewed nature of user-item interactions.

In Figure 3, we look at Recall@K where the sampling distribution is set to the empirical distribution. Similar results are found when we use $\gamma=0.75$, i.e. the training distribution. The findings mirror the results for NDCG: the attention based method achieves the best recall (again, statistically significant at $\alpha=0.01$) and the non-personalized method performs worst. The ability of attention based model to perform well across a variety of settings provides further evidence of its benefits over the alternative techniques considered here.

Figure 4 shows the effect of increasing depth in the attention based model. Increasing from depth 2 to 4 yields a large improvement in both recall and NDCG. Further increases in depth result in consistent, but diminishing improvements. These results were obtained using negative instances sampled from the empirical distribution.