Soft Retargeting Network for Click Through Rate Prediction

To measure user’s interest on retargeted item, the basic idea of HRN is to count how many times user has clicked this item in user behavior sequence and use the click count to represent user interest. Intuitively, the more user clicks an item before, the more likely user is interested in the item.

HRN first enumerates the items in user behavior sequence and checks whether target item and historical item are the same item by comparing their embedding vectors:

where $e_{t}$ is the embedding of target item $t$, $e_{b_{j}}$ is the embedding of historical item $b_{j}$, $s(i,j)\in[0,1]$ is the similarity weight between target item $i$ and historical item $j$. Note that target item and historical item share the same embedding dictionary, we can ensure two items are the same if their embedding vectors are equal.

For user behavior sequence $B=\{b_{1},b_{2},…,b_{n}\}$, we can get a sequence of similarity weights $S_{t}=\{s(t,b_{1}),s(t,b_{2}),…,s(t,b_{n})\}$. $S_{t}$ can be viewed as user’s historical click records on target item.

We perform max pooling on $S_{t}$ and get $s_{t}^{max}=max_{j}(s(t,b_{j}))$ for target item $t$. If $s_{t}^{max}=1$, target item is a retargeted item for HRN. In addition, we perform sum pooling on $S_{t}$ to get:

where $N_{S}\in[0,n]$ corresponds to the historical click number of target item. We then adopt binning method to transform $N_{S}$ into a categorical feature $FEA_{I_{S}}=binning(N_{S},1)$, where $bining(x,z)$ is an equal width interval binning method (Dougherty et al., 1995) to discrete $x$ with bin size $z$. For example, if $N_{S}=5.0$, $FEA_{I_{S}}=``6"$. HRN looks up the embedding of $FEA_{I_{S}}$ from embedding layer and feeds it into CTR model for prediction. Although simple, the performance of HRN is particularly good in e-commerce scenarios.

For simplicity, we only consider historical click item sequence during the introduction of HRN and SRN. Our approach can easily be extended to historical click brand/shop/category sequence.

In HRN, only target item the same as historical item is treated as retargeted item. The ratio of retargeted items in sample data is rather low, e.g., only 5% for item and 15% for shop in our scenario, which heavily limits the performance of HRN. To overcome the limitation, we propose a SRN model to expand the scope of retargeted items and model user retargeting interest at a broader level.

The framework of SRN is illustrated in Fig. 1. The idea of SRN is similar to that of HRN. It computes a sequence of similarity weights for user behavior sequence and aggregates similarity weights to represent user retargeting interest. We will detail the key components of SRN in the following subsections:

Graph embedding layer. To measure the similarity between target item and history items, a natural choice is just using the item embedding learned by CTR model to calculate cosine similarity. However, we find that the quality of CTR embedding is poor in practice. It is probably due to the fact that CTR embedding is learned for CTR tasks and may be incapable of capturing the similarity between items. Inspired by the success of graph embedding (Hamilton et al., 2017b; Wang et al., 2019; Hamilton et al., 2017a; Ying et al., 2018), we switch to using graph embedding to represent items in SRN. We pre-train a graph network based on user-item interaction records and generate a graph embedding dictionary $GE=[ge_{1},ge_{2},…,ge_{M}]\in R^{D\times M}$ (see Section 3.1 for more details). Graph embedding layer looks up $GE$ and generates a graph embedding $ge_{t}$ for target item $t$ and a graph embedding $ge_{b_{j}}$ for each historical item $b_{t_{j}}$.

Similarity gate layer. Based on graph embedding, similarity gate layer calculates the cosine similarity between target item $t$ and historical item $b_{j}$:

Note that cosine similarity is different from the similarity weight used in HRN. Firstly, $cosine(t,b_{j})\in[-1,1]$ but $s(t,b_{j})\in[0,1]$. Secondly, cosine similarity is a metric used to measure the similarity between embedding vectors. Similarity weight is item similarity visually perceived by user. While there is a strong correlation between them, there are also many cases where they differ.

We carefully examine items in our scenario and summarize three correlation rules for cosine similarity and similarity weight:

• •

  Rule 1: When $cosine(t,b_{j})=1.0$, item $t$ and item $b_{j}$ are the same item and $s(t,b_{j})=1.0$.

• •

  Rule 2: When $cosine(t,b_{j})\leq T$ (e.g., $T=0.90$), item $t$ and item $b_{j}$ can be two different products, e.g., iphone and microphone. In this case, $s(t,b_{j})$ drops dramatically and should approach $0.0$.

• •

  Rule 3: When $T<cosine(t,b_{j})<1.0$, item $t$ and item $b_{j}$ can be the same product sold by different shops, or two products with the same brand, e.g., iPhone 11 and iPhone SE. In this case, $s(t,b_{j})$ should be a large value, e.g., 0.95.

To satisfy these rules, we design a similarity gate function $F(x)$ to map $cosine(t,b_{j})$ to $s(t,b_{j})$:

where $w>0$ and $b>0$. Fig.  2 plots the curve of $F(x)$ for different values of $w$ and $b$. We can see $F(x)$ indeed suppresses $s(t,b_{j})$ when $cosine(t,b_{j})$ is below some threshold $T$ and the value of $T$ is determined by $b$. When $cosine(t,b_{j})\in(T,1)$, $F(x)$ outputs a high value of $s(t,b_{j})$ and the shape of $F(x)$ in $(T,1)$ is controlled by $w$. In practice, we usually set good initial values for $w$ and $b$ (e.g., $w=10.0$ and $b=9.0$) and let CTR model learn the optimal $F(x)$.

For user behavior sequence, similarity gate layer outputs a sequence of similarity weights $S_{t}=\{s(t,b_{1}),s(t,b_{2}),…,s(t,b_{n})\}$.

Weight aggregation layer. We perform max pooling on $S_{t}$ to get the maximum similarity weight $s_{t}^{max}$ just like in HRN. $s_{t}^{max}$ reflects the likelihood that target item $t$ is a retargeted item. The higher the value of $s_{t}^{max}$, the more likely $t$ is a retargeted item. We can see that the scope of retargeted item in SRN is definitely broader than that in HRN.

We can follow the operation in HRN to model user retargeting interest, i.e., sum pooling on $S_{t}$ and binning. However, the binning operation is non-differentiable and blocks the updating of similarity gate layer. Instead, we put binning operation on $cosine(t,b_{j})$ and generate a categorical feature $FEA_{j}^{ripple}=binning(cosine(t,b_{j}),0.01)$ (we name it ripple representation). We look up the embedding vector of ripple representation from embedding layer as $e_{j}^{ripple}$ and represent user retargeting interest as:

As $F(x)$ is a linear function with limited model capacity, $s(t,b_{j})$ may be inaccurate when $cosine(t,b_{j})\in(T,1)$. Ripple representation can complement the limitation of $F(x)$ by providing an embedding representation of $cosine(t,b_{j})$.

Retargeting evolution layer. We notice that user retargeting interest may evolve over time. Suppose there are two sequences of similarity weights $S_{1}=\{0.1,0.5,0.9\}$ and $S_{2}=\{0.9,0.5,0.1\}$, weight aggregation layer would produce the same results. However, $S_{1}$ suggests that user’s interest on target item grows rapidly whereas $S_{2}$ indicates that user may lose interest in target item. We use GRU network (Cho et al., 2014) to capture the evolution of user retargeting interest:

We concatenate $I_{S}$ and $G_{S}$ to get the output of SRN $I_{SRN}=[I_{S};G_{S}]$ and feed $I_{SRN}$ into MLP layer for CTR prediction.

Datasets. As SRN is mainly designed for click interest, we choose two public datasets which contain user click records and an industrial dataset for evaluation: 1) Taobao dataset¹¹1https://tianchi.aliyun.com/dataset/dataDetail?dataId=649. It is a collection of user behaviors from Taobao’s recommender system (Zhu et al., 2018) and contains 89 million records, 1 million users, and 4 million items from 9407 categories. We only use click behaviors of each user. 2) Alimama dataset²²2https://tianchi.aliyun.com/dataset/dataDetail?dataId=56. It is a dataset of click rate prediction about display ads provided by Alimama, an online advertising platform in China. It contains 26 million display/click records from 20170506 to 20170513, 1 million users, and 0.85 million items. We use the first 7 days’ data as training sample and the last day’s data as test sample. 3) Industrial dataset. It is collected from our online advertising system for a native ads scenario from 20220318 to 20220417. The dataset contains 780 million display/click records, 10.4 million users, and 0.78 million items. Each record contains a real-time click sequence from the preceding 3 days. We use the first 30 days’ data as training sample and the last day’s data as test sample.

Graph Construction. For each dataset, we pre-train a heterogeneous graph network based on the training sample of CTR task. The types of graph nodes include user, item, and its side information (brand/shop/category for Alimama dataset and industrial dataset, and category for Taobao dataset). The graph edges include: 1) user-item edge. If user $u$ clicks item $i$, there is an edge between $u$ and $i$. User-item edge is used as label in link prediction task. 2) user-side information edge. If user $u$ clicks an item with side information $v$ (e.g., shop), there is an edge between $u$ and $v$. 3) item-item edge. If item $i$ and item $j$ are adjacent in user behavior sequence and the time interval between item $i$ and item $j$ is within 60 seconds, there is an edge between $i$ and $j$. 4) item-side information edge. If item $i$ has a side info $v$, there is an edge between $i$ and $v$.

Notice that the edges are undirected and unweighted. We then traverse the graph to sample nodes and their neighborhoods according to meta paths. Meta paths include $user\rightarrow brand/shop/category\rightarrow item$, $item\rightarrow item$, $item\rightarrow brand/shop/category\rightarrow item$, $brand\rightarrow item$, $shop\rightarrow item$, $category\rightarrow item$. We aggregate the node’s neighborhoods using the HAN approach (Wang et al., 2019) and adopt a link prediction task to supervise the learning of embedding.

Competitors. We compare SRN with HRN and some widely used CTR models, including DNN, DIN (Zhou et al., 2018), DIEN (Zhou et al., 2019) and BST (Chen et al., 2019).

Parameter Configuration. Alimama and industrial dataset use historical click item/brand/shop/category sequence. Taobao dataset uses historical click item/category sequence. The maximum sequence length is 100 and 64 for public datasets and industrial dataset respectively. The dimension of CTR embedding and graph embedding are 8 and 32 respectively. The optimizer is Adagrad with learning rate 0.01. For SRN, $w$, $b$ is 10, 9 for historical item sequence and 10, 8 for historical brand/shop/category sequence, respectively.

Evaluation Metrics. We use two common used metrics, $AUC$ and $LogLoss$, to evaluate these models.

As shown in Table 1, HRN achieves solid performance gain compared to DNN base. SRN outperforms HRN with an AUC gain of 0.0015, 0.0233, 0.0076 in public datasets and industrial dataset, respectively. To explain the advantages of SRN over HRN, we assume target item with $s_{t}^{max}>0.5$ is retargeted item in SRN and define retargeting ratio for item as the ratio of the total number for retargeted items to sample number in test dataset. Retargeting ratio for brand/shop/category can be defined in the same way. We compare the retargeting ratio of SRN with that of HRN in industrial dataset. Table 2 shows the retargeting ratio of SRN is much higher than HRN for item and shop. It seems that SRN can capture more retargeted items/shops and thus boost the performance significantly.

SRN outperforms DNN, DIN, DIEN, and BST significantly with an AUC gain of at least 0.0122, 0.0208, 0.0042 in the three datasets, respectively. To demonstrate the effectiveness of SRN in modeling user retargeting interest, we compare the performance of SRN with that of baseline models on retargeted items (also selected by $s_{t}^{max}>0.5$) and the other items in industrial dataset. Table 3 shows that the AUC gain of SRN on retargeted items is much higher than that on the others. For DIEN and BST, the AUC gain on retargeted items is nearly the same as that on the others.

We conduct online A/B experiments to evaluate SRN in online system. The experiment lasts for 14 days and SRN achieves 6.88$\%$ CTR and 6.12$\%$ RPM gain compared to product model (DIN+HRN).

The result of ablation study on industrial dataset is shown in Table 5. Retargeting evolution layer (GRU) brings a notable AUC gain of 0.0009. $SRN\ w/o\ SIM\_GATE$ sets $s(t,b_{j})$ to $1.0$ and relies on ripple representation to represent the interest $I_{S}$, which causes an AUC loss of 0.0013. It demonstrates the importance of similarity weights in capturing user retargeting interest. $SRN_{binning}$ adopts the summing and binning operation used in HRN to aggregate similarity weights, which causes an AUC loss of 0.0032. This suggests that binning operation may lead to sub-optimal performance and weight aggregation layer is more flexible and powerful.

Using CTR embedding instead of graph embedding ($SRN\ w/o\ GE$) causes the performance to decrease dramatically, which indicates the quality of graph embedding is much better than CTR embedding. To evaluate the quality of graph/CTR embedding, we select top 100 categories according to PV number and sample 100 items for each category in industrial dataset. We calculate the cosine similarities between each item and the other items using graph/CTR embedding respectively. For each item, we can get the average similarity within the same category (intra-category similarity) and from the different categories (inter-category similarity). We then compute the average intra- and inter-category similairity for graph/CTR embedding respectively. Table 4 shows that intra-category similarity is much higher than inter-category similarity for graph embedding whereas the difference is negligible for CTR embedding. It demonstrates the superiority of graph embedding in identifying similar items.