Random Isn’t Always Fair: Candidate Set Imbalance and Exposure Inequality in Recommender Systems

Recommender and ranking systems organize vast troves of information to help people make decisions, like what to buy or who to hire. They also help people find relevant information, like content produced on social media or web pages. These recommender systems have historically followed the Probability Ranking Principle (Robertson, 1977): to maximize utility to a user of the system, recommended items should appear in decreasing order of their probability of relevance to the user. However, this approach completely ignores who or what is recommended despite the economic and social impacts to them. For example, a recommender system that does not return an individual’s content can preclude that content creators’ access to the $\geq$ $10 billion creator economy (Wee, 2021). Image searches that only return the images with the highest relevance can perpetuate occupational gender stereotypes in images by only, for example, returning images of men when queried for professional roles traditionally associated with men. (Kay et al., 2015). This over-indexing on the consumers of the system can lead to extreme inequality in who or what gets exposure.

These inequalities arise for reasons related to the algorithm itself and how the results of the recommendation algorithm are displayed and interacted with by users. On the non-algorithmic end, due to user interface design choices such as limited recommendation spots and human behavior such as limited attention or preference for highly ranked items regardless of relevance (Collins et al., 2018; Granka et al., 2004), a relatively small number of items are recommended and even fewer engaged with. On the algorithmic end, the Probability Ranking Principle exacerbates the disparities of outcomes between items that are predicted to be better than others since they will always be ranked in a better position even when differences in predicted relevance is negligible. In the worst case, one item can receive high levels of exposure while a similar item with only marginally worse estimated relevance can receive none. Because the estimated relevance itself can have high statistical uncertainty, it is possible that this “winners-take-all” dynamic could allocate much more exposure to an item that is, in reality, less relevant.

Stochastically ranking the items–instead of deterministically ranking them–has emerged as one standard approach to mitigate exposure inequality (Singh and Joachims, 2019; Diaz et al., 2020; Wu et al., 2022; Bower et al., 2021; Oosterhuis, 2021; Do et al., 2021; Singh and Joachims, 2018; Usunier et al., 2022; Kuhlman et al., 2021; Gorantla et al., 2022; García-Soriano and Bonchi, 2021; Singh et al., 2021; Heuss et al., 2022; Pandey et al., 2005). Stochastic rankings provide flexibility in how expected exposure is allocated to items in comparison to the rigid allocation of exposure under deterministic rankings. For example, one popular approach samples rankings from the Plackett-Luce distribution (Singh and Joachims, 2019; Diaz et al., 2020; Wu et al., 2022; Bower et al., 2021; Oosterhuis, 2021). Under this sampling approach, for each ranking request, items with higher estimated relevance are more likely to appear lower in the ranking, though there is randomness in the exact order in which items occur. In contrast to the deterministic ranking setting, under the Plackett-Luce sampling, items with similar predicted relevance will receive similar amounts of exposure in expectation. Another common approach used as a baseline for comparison of fair ranking models is to order the candidate items (Diaz et al., 2020; Wu et al., 2022) uniformly at random , i.e. every item has equal probability of appearing in each position. This approach is often described as leading to “fairer” ranking models that more equally distribute exposure across producers. However, in practice we have found that a pure randomization approach can actually result in an increase of exposure inequality.

In this paper, we explore how randomized ranking can increase inequality. In short, ranking in industry settings typically operates in a two-step process since it would be infeasible to rank every single possible item for each user (Wang and Joachims, 2022). In the first stage, a computationally inexpensive algorithm generates a relatively small candidate set, and then a more computationally expensive approach ranks the candidates. The phenomenon we identified in which randomization across the candidates increases inequality can occur when candidate sets are imbalanced, i.e. when some items are much more likely to appear in the candidate sets than others.

While there are several aspects to consider when building responsible recommender systems, like demographic biases in model performance, in this paper we specifically focus on inequality of exposure of the items being recommended. We offer two main contributions. First, we demonstrate that when striving to reduce exposure inequality in recommender systems, candidate set imbalance cannot be ignored. In particular, despite the widespread assumption that randomizing reduces inequality, we demonstrate that for recommender systems with imbalanced candidate sets, this need not be the case. Second, we propose a modification to the class of Plackett-Luce distributions that have been used as a post-processing step to reduce inequality of exposure on the individual ranking level while balancing consumer-utility (Diaz et al., 2020; Wu et al., 2022). This modification uses the system level information of how often an item appears in the candidate sets to reduce exposure inequality. Similar to Plackett-Luce post-processing, our approach is simple as it does not require re-training the candidate generation nor ranking model. Moreover, since post-processing methods like ours operate on the relevance scores regardless of how they were learned or obtained, our approach is appropriate in industry settings where the outputs of several models may be combined to produce a ranking of candidates. We illustrate the effectiveness of this approach on synthetic and real data.

Over the past few years, the fair ranking literature has rapidly grown. See (Ekstrand et al., 2022; Patro et al., 2022; Li et al., 2022; Zehlike et al., 2021; Sonboli et al., 2022) for survey papers. While there are many ethical aspects to consider when building recommender systems, in this work, we are interested in the popular problem of inequality of exposure allocation to the items being recommended. In particular, we are interested in the relationship between equity of exposure at the individual ranking level and at the system level. Exposure at the individual ranking level comes from the exposure items receive in a single ranking request from a user at a single point in time. Exposure at the system level comes from the cumulative exposure items receive over all users and all ranking requests over a time period.

There are several papers that focus on distributing exposure more equitably on the individual ranking level (Singh and Joachims, 2018, 2019; Diaz et al., 2020; Heuss et al., 2022; Wu et al., 2022) which do not take candidate set inequality into account. For example, the canonical work in (Singh and Joachims, 2018) requires that the ratio of exposure that an item receives to its relevance should be constant over all items for each ranking request. One reason to focus on exposure at the individual ranking level is that the associated optimization problems become easier to solve since the dependent nature of system level exposure can be ignored. In light of candidate set imbalance, as we show in our experiments, these type of approaches do not necessarily guarantee equity of exposure at the system level.

On the other hand, there are group-fairness notions of equality of exposure on the individual ranking level where exposure equality does generalize to the system level. In particular, since these approaches put constraints on the proportions of items in the “top $k$” belonging to each demographic group for each ranking, each group at the system level will receive the same level of exposure as they do in each individual ranking (Yang and Stoyanovich, 2017; Zehlike et al., 2017; Celis et al., 2018; Geyik et al., 2019; Celis et al., 2020; Yang et al., 2019). However, these approaches do not naturally extend to individual fairness, the main case we consider.

Another line of work focuses on distributing and measuring exposure more equitably at the system level, which implicitly takes candidate set imbalance into account (Sürer et al., 2018; Biega et al., 2018; Do et al., 2021; Do and Usunier, 2022; Lazovich et al., 2021; Zhu and Lerman, 2016). Instead of turning to stochastic rankings, the work in (Biega et al., 2018) amortizes equity of exposure on the system level over time. However, their approach as well as (Sürer et al., 2018) requires solving an integer linear program.

Recently, the works in (Do et al., 2021; Do and Usunier, 2022) overcame the previously mentioned optimization challenges that arise when reducing system level exposure directly via the Franke-Wolfe algorithm and leveraging tools in non-smooth optimization. Their approach learns a stochastic ranking model given relevance scores by optimizing welfare functions that take into account user utility and item side inequality. Since our algorithm does not require training a model, our approach can more quickly adapt to dynamic environments that effect candidate set imbalance although recently a fast online algorithm was proposed (Usunier et al., 2022) applicable to differentiable models like in (Do et al., 2021) but not (Do and Usunier, 2022). Furthermore, along the way, the authors of (Do et al., 2021) also note a similar finding as ours wherein a class of stochastic rankings theoretically behave unexpectedly in terms of producer inequality and consumer utility, albeit in a different setting.

In either case, stochastic rankings are one standard way of achieving system level or individual-level equity of exposure (Singh and Joachims, 2019; Diaz et al., 2020; Wu et al., 2022; Bower et al., 2021; Oosterhuis, 2021; Do et al., 2021; Singh and Joachims, 2018; Usunier et al., 2022; Kuhlman et al., 2021; Gorantla et al., 2022; García-Soriano and Bonchi, 2021; Singh et al., 2021; Heuss et al., 2022; Pandey et al., 2005). The work in (Diaz et al., 2020; Wu et al., 2022) proposes sampling rankings from a class of Plackett-Luce distributions as a post-processing step to achieve individual-level equity of exposure. They make the observation that as stochasticity is increased, inequality of exposure decreases. In the case of a random ranking then, expected exposure is equalized for all items in a single ranking request. However, in our work, due to candidate set inequality, we argue that this post-processing algorithm does not guarantee system level equality of exposure. We make a simple modification to this algorithm (Diaz et al., 2020; Wu et al., 2022) by leveraging system level information.

Finally, very recently, methods to change the candidate sets themselves have emerged. For example, the authors in (Wang and Joachims, 2022) propose algorithms to obtain group-fair candidate sets. Instead of modifying the candidate sets, our work leverages information about the candidate sets to achieve more equitable rankings in cases where candidate set inequality is significant but not insurmountable.

In this section, we illustrate how system level inequality of exposure can actually increase even when the final ranking a user sees is drawn uniformly at random from all permutations of their candidate set.

To illustrate how completely randomizing each ranking request can increase system level inequality, we present the following toy example. Consider a case where we have consumers {1, 2, …, 10} and producers {A, B, C, …, J}. For each consumer, the candidate generation process returns four candidates, which are ranked as shown in Figure 1. Suppose for simplicity that each consumer stops after the first item that is returned.

Then, if we use deterministic ranking, every producer receives exactly the same number of impressions–exactly one. However, if we first randomize the order among the four candidates, then producer J has a 1/4 chance of being shown to every consumer, resulting in 2.5 impressions on expectation overall. The expected number of impressions are shown in Figure 2 under both scenarios. In this case, it is clear that randomization would increase exposure inequality relative to ranking because of the imbalanced nature of the candidate sets.

While this example may seem contrived, this phenomenon, where lower ranked content tends to appear much more often in candidate sets overall, can occur in practice. One such reason is to prevent empty or small candidate sets. When the candidate sets that are returned by the first stage model are small, popular items may be used to backfill candidate sets. These popular items may not be the most relevant items to a user, resulting in low relevance scores, but could appear in many candidate sets to ensure every candidate set is of an appropriate size.

We now present a post-processing algorithm to mitigate the effect of candidate set inequality. While it is reasonable to directly intervene on the first stage of candidate generation like in (Wang and Joachims, 2022), we argue it is also valid to intervene directly at the second stage of ranking the candidates for a couple reasons. First, candidate sets are populated from many different sources, like popular accounts, accounts that are followed by accounts a user follows, etc. If we directly intervene at the candidate generation stage, we need to take into account dependencies between these multiple sources that would otherwise be independent from one another, which may increase latency to an unacceptable level and may result in small candidate sets for some users since some sources like popular account sources are used to ensure each user has a reasonable number of candidates. Second, intervening on the second step arguably has more of a direct impact on what recommendation a user ultimately sees as there may be hundreds of candidates but only a handful of candidates actually selected to be recommended.

In (Diaz et al., 2020; Wu et al., 2022), the authors propose a simple post-processing algorithm to achieve equity of exposure on the individual ranking level by taking the relevance scores output by a model, transforming the scores either by exponentiation by a constant or multiplying them by a constant, and then sampling a ranking from the Plackett-Luce distribution. Their transformation of scores allows them to smoothly interpolate between completely randomizing the candidate sets (constant is zero) and deterministically ranking the items in decreasing order of relevance score (constant is large).

We propose a modification to this post-processing algorithm by incorporating the frequency that each item appears in candidate sets in pursuit of achieving system level equity of exposure. We call our approach “Plackett-Luce With Inverse Candidate Frequency Weights.” In particular, suppose there are $n$ users $U:=\{u_{i}\}_{i=1}^{n}$ and $m$ items $V:=\{v_{i}\}_{i=1}^{m}$. For every user $u\in U$, let $\{v_{u_{i}}\}_{i=1}^{m_{u}}$ be the set of $m_{u}$ candidate items that need to be ranked for user $u$. Let $\{r_{{u}_{i}}\}_{i=1}^{m_{u}}$ be the set of relevance scores corresponding to the items. For simplicity, we assume each user requests just one ranking. Let $\alpha,\beta\in\mathbb{R}$. Let $W_{v}$ be the number of candidate sets that item $v\in V$ appears in. Then for user $u\in U$, the probability that the ranking $(v_{u_{1}},v_{u_{2}},\dots,v_{m_{u}})$ is sampled is

Considering Equation (1) for all possible rankings, it is easy to see that it is more likely to sample rankings such that items corresponding to larger, respectively smaller, $\alpha\exp(\beta r_{u_{j}})+\frac{1}{W_{v_{u_{j}}}+\alpha}$ are highly, respectively lowly, ranked. When $\beta=O(\alpha)$ and $\beta>0$, the hyperparameter $\alpha$ allows this distribution to smoothly interpolate between the deterministic ranking given by sorting the items by decreasing relevance score (when $\alpha\rightarrow\infty$) and stochastic rankings that are more likely to highly (lowly) rank items that appear less (more) frequently among all candidate sets (when $\alpha\rightarrow 0^{+}$). When each item appears in the candidate sets at equal rates and $\alpha$ is large, our algorithm behaves like the algorithm in (Diaz et al., 2020; Wu et al., 2022). For a fixed $\alpha$, $\beta$ controls the level of stochasticity. To efficiently sample rankings from the Plackett-Luce distribution, we utilize “Algorithm-A” in (Efraimidis and Spirakis, 2006), which is an $O(N\log(N))$ algorithm that solves the “weighted sampling” problem when there are $N$ items to rank.

Furthermore, we can easily extend our individual fairness type method to group fairness by replacing $W_{v_{i}}$, which is the number of times user $v_{i}$ appears in the candidate sets, with $W_{g(v_{i})}$ defined as the number of times a candidate from group $g(v_{i})$ appears in the candidate sets where $g(v_{i})$ is the demographic group user $v_{i}$ belongs to.

We consider experiments on a synthetic data set where ground truth relevance scores are known as well as the German credit data set (Dua and Graff, 2017) where relevance scores need to be learned from features of the candidates to test our approach under a more realistic scenario. The German credit data set is a real binary classification data set that is commonly repurposed for ranking tasks in the fair ranking literature (Zehlike et al., 2021).