Reverse Maximum Inner Product Search: How to efficiently find users who would like to buy my item?

The $k$-MIPS problem is effective for the case where a user wants to know items that s/he prefers (i.e., user-driven cases), but e-commerce companies usually face situations where they want to advertise an item, which may be new or discounted one, to users, which corresponds to item-driven cases. Trivially, an effective advertisement is to recommend such an item to users who would be interested in this item.

In the context of the $k$-MIPS problem, if this item is included in the top-k item set for a user, we should make an advertisement of the item to this user. That is, we should find a set of such users. This paper addresses this new problem, called reverse $k$-MIPS problem. To ease of presentation, this section assumes that $k=1$ (the general case is defined in Section 2). Given a query vector $\mathbf{q}$ (the vector of a target item) and two sets of $d$-dimensional vectors $\mathbf{Q}$ (set of user vectors) and $\mathbf{P}$ (set of item vectors), the reverse MIPS problem finds all user vectors $\mathbf{u}\in\mathbf{Q}$ such that $\mathbf{q}=$ arg max${}_{\mathbf{p}\in\mathbf{P}\,\cup\,\{\mathbf{q}\}}\mathbf{p}\cdot\mathbf{u}$.

The reverse $k$-MIPS problem is an effective tool not only for item-driven recommendations but also market analysis. Assume that we are given a vector of a new item, $\mathbf{q}$. It is necessary to design an effective sales strategy to gain a profit. Understanding the features of users that may prefer the item is important for the strategy. Solving the reverse $k$-MIPS of the query vector $\mathbf{q}$ supports this understanding.

The above practical situations clarify the importance of reverse MIPS. Because e-commerce services have large number of users and items, $|\mathbf{Q}|$ and $|\mathbf{P}|$ are large. In addition, a query vector is not pre-known and is specified on-demand fashion. The reverse $k$-MIPS is therefore conducted online and is computationally-intensive task. Now the question is how to efficiently obtain the reverse MIPS result for a given query.

A straightforward approach is to run a state-of-the-art exact MIPS algorithm for every vector in $\mathbf{Q}$ and check whether or not $\mathbf{q}=$ arg max${}_{\mathbf{p}\in\mathbf{P}\,\cup\,\{\mathbf{q}\}}\,\mathbf{u}\cdot\mathbf{p}$. This approach obtains the exact result, but it incurs unnecessary computation. The poor performance of this approach is derived from the following observations. First, we do not need the MIPS result of $\mathbf{u}$ when $\mathbf{q}$ does not have the maximum inner product with $\mathbf{u}$. Second, this approach certainly accesses all user vectors in $\mathbf{Q}$, although many of them do not contribute to the reverse MIPS result. However, it is not trivial to skip evaluations of some user vectors without losing correctness. Last, its theoretical cost is the same as the brute-force case, i.e., $O(nmd)$ time, where $n=|\mathbf{Q}|$ and $m=|\mathbf{P}|$, which is not appropriate for online computations. These concerns pose challenges for solving the reverse MIPS problem efficiently.

To address the above issues, we propose Simpfer, a simple, fast, and exact algorithm for reverse MIPS. The general idea of Simpfer is to efficiently solve the decision version of the MIPS problem. Because the reverse MIPS of a query $\mathbf{q}$ requires a yes/no decision for each vector $\mathbf{u}\in\mathbf{Q}$, it is sufficient to know whether or not $\mathbf{q}$ can have the maximum inner product for $\mathbf{u}$. Simpfer achieves this in $O(1)$ time in many cases by exploiting its index built in an offline phase. This index furthermore supports a constant time filtering that prunes vectors in a batch if their answers are no. We theoretically demonstrate that the time complexity of Simpfer is lower than $O(nmd)$.

The summary of our contributions is as follows:

• •

  We address the problem of reverse $k$-MIPS. To our knowledge, this is the first work to study this problem.

• •

  We propose Simpfer as an exact solution to the reverse MIPS problem. Simpfer solves the decision version of the MIPS problem at both the group-level and the vector-level efficiently. Simpfer is surprisingly simple, but our analysis demonstrates that Simpfer theoretically outperforms a solution that employs a state-of-the-art exact MIPS algorithm.

• •

  We conduct extensive experiments on four real datasets, MovieLens, Netflix, Amazon, and Yahoo!. The results show that Simpfer is at least two orders magnitude faster than baselines.

• •

  Simpfer is easy to deploy: if recommender systems have user and item vector sets that are designed in the inner product space, they are ready to use Simpfer via our open source implementation¹¹1https://github.com/amgt-d1/Simpfer. This is because Simpfer is unsupervised and has only a single parameter (the maximum value of $k$) that is easy to tune and has no effect on the running time of online processing.

This paper is an error-corrected version of (Amagata and Hara, 2021). We fixed some writing errors and minor bugs in our implementation, but our result is consistent with (Amagata and Hara, 2021).

The objective of this pre-processing phase is to build data structures that support efficient computation of a lower-bound and an upper-bound of the $k$-th highest inner product for each $\mathbf{u_{i}}\in\mathbf{Q}$, for arbitrary queries. We utilize Cauchy–Schwarz inequality for upper-bounding. Hence we need the Euclidean norm $\|\mathbf{u_{i}}\|$ for each $\mathbf{u_{i}}\in\mathbf{Q}$. To obtain a lower-bound of the $k$-th highest inner product, we need to access at least $k$ item vectors in $\mathbf{P}$. The norm computation and lower-bound computation are independent of queries (as long as $k\leq k_{max}$), so they can be pre-computed. In this phase, Simpfer builds the following array for each $\mathbf{u_{i}}\in\mathbf{Q}$.

Generally, this pre-processing is done only once. An exception is the case where a query with $k>k_{max}$ is specified. In this case, Simpfer re-builds the data structures then processes the query. This is actually much faster than the baselines, as shown in Section 5.7.

Before we present the details of Simpfer, we introduce our techniques that can quickly answer the $k$-MIPS decision problem for a given query $\mathbf{q}$. Recall that $\mathbf{Q}$ and $\mathbf{P}$ are sorted in descending order of norm. Without loss of generality, we assume that $\|\mathbf{u_{i}}\|\geq\|\mathbf{u_{i+1}}\|$ for each $i\in[1,n-1]$ and $\|\mathbf{p_{j}}\|\geq\|\mathbf{p_{j+1}}\|$ for each $j\in[1,m-1]$, for ease of presentation.

Given a query $\mathbf{q}$ and a user vector $\mathbf{u_{i}}\in\mathbf{Q}$, we have $\mathbf{u_{i}}\cdot\mathbf{q}$. Although our data structures are simple, they provide effective and “light-weight” filters. Specifically, we can quickly answer the $k$-MIPS decision problem on $\mathbf{q}$ through the following observations³³3Existing algorithms for top-k retrieval, e.g., (Ding and Suel, 2011; Fontoura et al., 2011), use similar (but different) bounding techniques. They use a bound (e.g., obtained by a block) to early stop linear scans. On the other hand, our bounding is designed to avoid linear scans and to filer multiple user vectors in a batch..

We next introduce a technique that yields “no” as the answer for all user vectors in a block $\mathbf{B}$ in $O(1)$ time.

If a user vector $\mathbf{u_{i}}$ cannot obtain a yes/no answer from Lemmas 1–3, Simpfer uses a linear scan of $\mathbf{P}$ to obtain the answer. Let $\tau$ be a threshold, i.e., an intermediate $k$-th highest inner product for $\mathbf{u}$ during the linear scan. By using the following corollaries, Simpfer can obtain the correct answer and early terminate the linear scan.

Now we are ready to present Simpfer. Algorithm 3 details it. To start with, Simpfer computes $\|\mathbf{q}\|$. Given a block $\mathbf{B}\in\mathcal{B}$, Simpfer tests Lemma 3 (line 3). If the user vectors in $\mathbf{Q}(\mathbf{B})$ may have yes as an answer, for each $\mathbf{u_{i}}\in\mathbf{Q}(\mathbf{B})$, Simpfer does the following. (Otherwise, all user vectors in $\mathbf{Q}(\mathbf{B})$ are ignored.) First, it computes $\mathbf{u_{i}}\cdot\mathbf{q}$, then tests Lemma 1 (line 3). If $\mathbf{u_{i}}$ cannot have the answer from this lemma, Simpfer tests Lemma 2. Simpfer inserts $\mathbf{u_{i}}$ into the result set $\mathbf{Q_{r}}$ if $\mathbf{u_{i}}\cdot\mathbf{q}\geq\|\mathbf{u_{i}}\|\|\mathbf{p_{k}}\|$. Otherwise, Simpfer conducts Linear-scan$(\mathbf{u_{i}})$ (Algorithm 2). If Linear-scan$(\mathbf{u_{i}})$ returns 1 (yes), $\mathbf{u_{i}}$ is inserted into $\mathbf{Q_{r}}$. The above operations are repeated for each $\mathbf{B}\in\mathcal{B}$. Finally, Simpfer returns the result set $\mathbf{Q_{r}}$.

The correctness of Simpfer is obvious, because it conducts Linear-scan$(\cdot)$ for all vectors that cannot have yes/no answers from Lemmas 1–3. Besides, Simpfer accesses blocks sequentially, so it is easy to parallelize by using multicore. Simpfer hence further accelerates the processing of reverse $k$-MIPS, see Section 5.6.

We theoretically demonstrate the efficiency of Simpfer. Specifically, we have:

Note that (Li et al., 2017; Teflioudi et al., 2015) have demonstrated that the other exact MIPS algorithms are outperformed by LEMP and FEXIPRO, so we did not use them as competitors. (Recall that this paper focuses on the exact answer, thus approximation algorithms are not appropriate for competitors, see Section 3.) In addition, LEMP and FEXIPRO also have a pre-processing (offline) phase. We did not include the offline time as the running time.

We first clarify the effectiveness of blocks employed in Simpfer. To show this, we compare Simpfer with Simpfer without blocks (which does not evaluate line 3 of Algorithm 3). We set $k=10$.

On MovieLens, Netflix, Amazon, and Yahoo!, Simpfer (Simpfer without blocks) takes 10.3 (22.0), 58.6 (100.8), 117.6 (446.2), and 1481.2 (1586.2) [msec], respectively. This result demonstrates that, although the speed-up ratio is affected by data distributions, blocks surely yield speed-up.

We investigate how $k$ affects the computational performance of each algorithm by using a single core. Figure 2 depicts the experimental results.

We first observe that, as $k$ increases, the running time of each algorithm increases, as shown in Figures 2(a)–2(d). This is reasonable, because the cost of (decision version of) $k$-MIPS increases. As a proof, Figures 2(e)–2(h) show that the number of inner product (ip) computations increases as $k$ increases. The running time of Simpfer is (sub-)linear to $k$ (the plots are log-scale). This suggests that $m^{\prime}=O(k)$.

Second, Simpfer significantly outperforms LEMP and FEXIPRO. This result is derived from our idea of quickly solving the $k$-MIPS decision problem. The techniques introduced in Section 4.2 can deal with both yes and no answer cases efficiently. Therefore, our approach functions quite well in practice.

Last, an interesting observation is the performance differences between FEXIPRO and Simpfer. Let us compare them with regard to running time. Simpfer is at least two orders of magnitude faster than FEXIPRO. On the other hand, with regard to the number of inner product computations, that of Simpfer is one order of magnitude lower than that of FEXIPRO. This result suggests that the filtering cost of Simpfer is light, whereas that of FEXIPRO is heavy. Recall that Lemmas 1–3 need only $O(1)$ time, and Corollaries 1–2 need $O(k)$ time in practice. On the other hand, for each user vector in $Q$, FEXIPRO incurs $\Omega(k)$ time, and its filtering cost is $O(d^{\prime})$, where $d^{\prime}<d$. For high-dimensional vectors, the difference between $O(1)$ and $O(d^{\prime})$ is large. From this point of view, we can see the efficiency of Simpfer.

We next study the scalability to $n=|\mathbf{Q}|$ by using a single core. To this end, we randomly sampled $s\times n$ user vectors in $\mathbf{Q}$, and this sampling rate $s$ has $s\in[0.2,1.0]$. We set $k=10$. Figure 3(h) shows the experimental result.

In a nutshell, we have a similar result to that in Figure 2. As $n$ increases, the running time of Simpfer linearly increases. This result is consistent with Theorem 3. Notice that the tendency of the running time of Simpfer follows that of the number of inner product computations. This phenomenon is also supported by Theorem 3, because the main bottleneck of Simpfer is Linear-scan$(\cdot)$.

The scalability to $m=|\mathbf{P}|$ by using a single core is also investigated. We randomly sampled $s\times m$ user vectors in $\mathbf{P}$, as with the previous section. Figure 4 shows the experimental result where $k=10$. Interestingly, we see that the result is different from that in Figure 3(h). The running time of Simpfer is almost stable for different $m$. In this experiment, $n$ and $k$ were fixed, and recall that $m^{\prime}=O(k)$. From this observation, the stable performance is theoretically obtained. This scalability of Simpfer is an advantage over the other algorithms, since their running time increases as $m$ increases.

We study the gain of multicore processing of Simpfer by setting $k=10$. We depict the speedup ratios compared with the single-core case in Table 3.

We see that Simpfer receives the benefit of multicore processing, and its running time decreases as the number of available cores increases. We here explain why Simpfer cannot obtain speedup ratio $c$, where $c$ is the number of available cores. Each core deals with different blocks, and the processing cost of a given block $\mathbf{B}$ is different from those of the others. This is because it is unknown whether $\mathbf{B}$ can be pruned by Lemma 3. Even if we magically know the cost, it is NP-hard to assign blocks so that each core has the same processing cost (Amagata and Hara, 2019; Korf, 2009). Therefore, perfect load-balancing is impossible in practice. The Yahoo! case in particular represents this phenomenon. Because many user vectors in Yahoo! have large norms, blocks often cannot be filtered by Lemma 3. This can be seen from the observation in Figure 3(h): the number of inner product computations on Yahoo! is larger than those on the other datasets. The costs of Corollaries 1–2 are data-dependent (i.e., they are not pre-known), rendering a fact that Yahoo! is a hard case for obtaining a high speedup ratio.

Last, we report the pre-processing time of Simpfer. Table 4 shows the results. As Theorem 1 demonstrates, the pre-processing time increases as $n$ increases. We see that the pre-processing time is reasonable and much faster than the online (running) time of the baselines. For example, the running time of FEXIPRO on Amazon with $k=25$ is 1206 [sec]. When $k=25$ (i.e., $k=k_{max}$), the total time of pre-processing and online processing of Simpfer is $15.10+0.16=15.26$ [sec]. Therefore, even if $k>k_{max}$ is specified, re-building blocks then processing the query by Simpfer is much faster.