Paired comparisons for games of chance

We thank Michael Snarski for many very helpful conversations, Oleg Maslennikov for integrating all aspects of our system into Duelyst II, and the Duelyst II community for their enthusiastic testing and thorough feedback.

The outcome of a match between two players $A$ and $B$ depends on both how well $A$ and $B$ perform in that match as well as the nature of the game that they are playing. In this section we present 2.1 for predicting match outcomes in a way that takes into consideration both of these factors.

We call $\Lambda$ a luck function. In 2.1, games are determined by the choice of $\Lambda$, and players by their associated probability measures. The luck function $\Lambda$ reflects the nature of the game being played, in particular how much luck influences the outcomes of matches, and should be chosen on a case-by-case basis. The probability measures associated to players reflect how consistently that player performs from match to match.

When two fixed players $A$ and $B$ play multiple matches against one another and multiple outcomes are observed, in many situations it is possible to explain this variation in outcome equally well as a consequence of inconsistent performances by the players (reflected in $\mu_{A}$ and $\mu_{B}$), as a consequence of an element of luck inherent in the game (reflected in $\Lambda$), or by various combinations of these two factors. Example 2.5 and Example 2.6 show how the Bradley–Terry model [27, 1] can arise from either the perspective of the game involving luck or players performing inconsistently.

2.1 makes two assumptions about the function $\Lambda$ which we use to simplify 2.11. However, there are situations in which these assumptions aren’t appropriate. We discuss the assumptions below. In both cases, it seems straightforward conceptually to omit the assumption, but then one must repeat the work done in subsequent sections without using the associated simplifications.

The first assumption is that $\Lambda(x,y)=1-\Lambda(y,x)$. This is appropriate for symmetric games, but probably isn’t for asymmetric games. For example, the game in which players flip a coin to determine who plays with white in a game of chess is symmetric, but the game of chess after having picked colours is not. At the time of writing, there have been $3,\!988,\!065,\!350$ matches of chess played on lichess.org, and in them white scored $52\%$ [13]. To model this, one might instead fix two different $\Lambda$’s, each satisfying $\Lambda(x,x)=0.5\pm 0.02$, and consider which player was playing with the white pieces to determine which one to use.

The second assumption, that $\Lambda(x,y)\in[0,1]$, builds on the first. There are situations where one is interested not in the probability of each player winning, but other notions of score. For example, in poker “cash games” players can exchange money for chips at any time, and strive to win as many chips as possible. Thus, if $A$ and $B$ win $a$ and $b$ chips respectively while playing a prescribed number of hands against each other in a poker cash game, then it is most meaningful to consider the quantities $a$ and $b$ themselves, and not whether or not $a>b$. This contrasts with games like Go, where the winner of the match is the player with the highest score, regardless of the magnitude of the scores or the difference between them. It seems more natural to model poker cash games with a choice of $\Lambda$ which takes values outside of $[0,1]$. This also allows one to consider games which are not zero-sum. In two player games, when one player wins, the other loses, but other notions of score needn’t sum to zero. In the preceding poker example, one expects that $a+b<0$, as casinos take a small portion of each pot.

2.1 posits that the player $A$ is determined by a Borel probability measure $\mu_{A}\in\mathcal{P}(\mathcal{S})$. Our rating system will not know with certainty what the true underlying measure $\mu_{A}$ is, and represents its current beliefs with a Borel probability measure $\nu_{A}\in\mathcal{P}(\mathcal{P}(\mathcal{S}))$. The only observations our rating system will consider are match outcomes, and 2.11 gives the resulting posteriors.

After updating the prior $\nu_{A}$ to the posterior $\nu_{A,A>B}$ or $\nu_{A,B>A}$, that posterior will then be used as the prior when processing the next match $A$ plays.

In some games there are match outcomes such as draws which are neither wins nor losses. When one can reasonably model these outcomes by a real number $\theta$, e.g. $\theta=\tfrac{1}{2}$ for draws, then we suggest taking the posterior, which we’ll denote $\nu_{A,\theta}$, to be any probability measure which satisfies

for all Borel-measurable sets $U\subseteq\mathcal{P}(\mathcal{S})$. Our reasoning for this is the same as the reasoning given in [8, §2]. This formula can also be viewed as encompassing the one given in 2.11 if one takes $\theta=1$ if $A>B$ and $\theta=0$ if $B>A$. It may be helpful in some cases to note that the assumptions in 2.1 imply that $L(\mu^{\prime},\mu)=1-L(\mu,\mu^{\prime})$.

One must choose what prior $\nu_{0}\in\mathcal{P}(\mathcal{P}(\mathcal{S}))$ to assign to a player which is completely unknown to the system. Below, we give one type of prior which is a convenient choice for many applications.

In practice, the strength of a player $A$ is likely change over time for a variety of reasons. For example, $A$ might read a book to learn a new chess opening, or acquire new cards in a collectible card game. Using 2.11 to process many of $A$’s matches can easily lead to situations where $\nu_{A}$ gives probability nearly $0$ to certain measures. If external factors then cause $A$’s strength to change, a purely Bayesian system might need to observe many match outcomes before it gives a non-negligible probability to the measure corresponding to $A$’s new strength. In this section, we present 2.16 for how a player’s underlying measure can change between matches.

We call the function $\kappa$ a kernel.

For Duelyst II, we take $\mathcal{S}=\mathbb{R}$. There are situations like those discussed in Example 2.10 which call for different choices of $\mathcal{S}$, but we didn’t feel that this was necessary for our application.

Our choice of $\Lambda$ is

with $\beta=0.8$. This is the $\Lambda$ from Example 2.9, and we discuss it there. When displaying ratings in game, we first apply the transformation

so that we match the parameterization of the Bradley–Terry model used by FIDE [7] and Glicko [8].

Our $\Lambda$ is a linear combination of a constant function and a sigmoid. We chose a logistic curve for the sigmoid, but we would guess that other choices, like a normal CDF as in the Thurstone–Mosteller model [23, 24, 16, 17, 18], would work just as well. In contrast, the constant term in $\Lambda$ has a very large impact on the behaviour of the system.

3.1 illustrates the effect of including a constant term in $\Lambda$. We suppose $B$’s strength is always $2000$ (i.e. $\nu_{B}=\delta_{\delta_{2000}}$; c.f. Definition 2.3), $A$’s strength is an unknown real number (i.e. $\nu_{A}$ is “Dirac-only” as in Definition 2.12), and the system’s prior $\nu_{A}$ for $A$’s strength is a normal distribution with mean $m$ and variance $\sigma^{2}$. If the match outcome $A>B$ is observed, then the system uses 2.11 to update its belief about $A$’s strength. Let $m^{\prime}$ be the mean of $A$’s updated distribution $\nu_{A,A>B}$. 3.1 plots the difference $m^{\prime}-m$ as a function of $m$ for $\Lambda$ as in (2) and reparameterized according to (3), $\beta=0.8$, $0.99$, $1$, and various $\sigma$. Unpacking the definitions, one can write the quantity being plotted explicitly:

When $\beta<1$, one interpretation is that, like in Example 2.9, there is a nonzero chance that the winner of a match is decided uniformly at random. If $B$ is vastly stronger than $A$, then, when the match outcome $A>B$ is observed, the only plausible explanation is that the outcome of the match was in fact decided by chance, and the posterior distribution $\nu_{A,A>B}$ for $A$ is very similar to their prior $\nu_{A}$. In (4), this intuition is reflected in the value of the integrals being much smaller than the constant terms, because there is nearly no overlap between the factors

which respectively come from the sigmoidal term in $\Lambda$ (defined in (2)) and the prior $\nu_{A}$.

In contrast, when $\beta=1$, the constant terms in the numerator and denominator of (4) vanish, and the difference in the means of $\nu_{A}$ and $\nu_{A,A>B}$ is the ratio of the two exponentially small integrals. Essentially, the system views the chance of $A$ having strength comparable to $B$’s as vanishingly small, but also views the chance of observing the match outcome $A>B$ as vanishingly small. Because $\nu_{A}$ has the $\exp(-x^{2})$ tails of a normal distribution, but $\Lambda$ has the much heavier tail $\exp(-x)$, almost all of the mass in the integrals comes from $x\approx m$. A straightforward calculation shows that

as $m\longrightarrow-\infty$.

The behaviour with $\beta=1$ overall seems much less reasonable to us than when $\beta<1$, since it leads to massive rating changes for $A$ based on ratios of minuscule probabilities. We view these probabilities as being substantially smaller than the chance that something bizarre has happened that the $\beta=1$ model is not equipped to consider, and believe that a model that better reflects reality should not view this match outcome as extremely strong evidence that $A$ is tremendously underrated. 3.1 shows that changing $\beta$ from $1$ to $0.99$ changes the behaviour of the model much more than changing $\beta$ from $0.99$ to $0.8$.

We note that [10, Fig. 1] reports that four widely-used systems based on the Bradley–Terry model all overestimate the performance of very highly rated competitors, that Sonas reports observing the same phenomenon in the popular article [22] with a dataset of $1.54$ million chess games from FIDE, and that FIDE truncates the tails of the $\Lambda$ it uses; see Example 2.8. This is consistent with our qualitative assessment of 3.1 above: the Bradley–Terry model, with exponential asymptotes at $0$ and $1$, overestimates the winning chances of a much higher rated player.

The two players who have played the most matches in our Duelyst II dataset of the first $1,\!126,\!592$ ranked matches played since the game’s launch, who we call $P$ and $Q$. have played $2162$ and $2142$ matches respectively, and are ranked by our system to be at the $95^{\text{th}}$ and $99.95^{\text{th}}$ percentiles among all players having played at least one ranked match. 3.2 shows the change in rating, i.e. mean of $\nu_{P}$ or $\nu_{Q}$, after each of their matches beyond the first $100$ they played. The horizontal axis shows the rating difference at the time of the match between the player and their opponent. $P$ is shown in red, and $Q$ in blue. Points above the axis are wins and points below are losses, except for the $10$ distinctly visible draws separate from the rest of the points.

The shape of the clusters in 3.2 is similar to what’s shown in 3.1, as expected. The qualitative observation that the blue points are noisier than the red points is explained by the fact that $\text{Var}(\nu_{Q})$ varies between about $52^{2}$ and $62^{2}$ for the plotted matches, whereas $\text{Var}(\nu_{P})$ is almost always between $48^{2}$ and $52^{2}$.

Duelyst II’s implementation considers only values of $\nu$ which are Dirac-only (Definition 2.12). We can then view $\nu$ as a probability distribution on $\mathbb{R}$, with the interpretation that each player’s strength could in principle be described by a single real number, and $\nu$ describes the system’s knowledge of said real number. The prior $\nu_{0}$ we assign to an unknown player, viewed as a distribution on $\mathbb{R}$, is

with

Here $\delta_{x_{k}}$ is a Dirac measure (Definition 2.3), $\varphi(\,\cdot\,|\,x,\sigma_{0}^{2})$ is the normal density with mean $x$ and variance $\sigma_{0}^{2}$, and $\propto$ indicates that the values of $\rho(x_{k})$ are rescaled so that they sum to $1$.

It would be more natural to take $\nu_{0}$ to be the normal distribution $\mathcal{N}(0,0.7^{2})$, but we choose the discrete distribution (5) with finite support because it is straightforward to implement: we can encode (5) as the tuple $\left(\rho(x_{0}),\dots,\rho(x_{1000})\right)\in\mathbb{R}^{1001}$, a list of $1001$ real numbers. The measure (5) is an approximation to $\mathcal{N}(0,0.7^{2})$ in the sense of weak convergence as $n,M\longrightarrow\infty$.

We chose the values of $n=1000$, $M=7$, and $\sigma_{0}^{2}=0.7^{2}$ by examining the system’s performance on the dataset of the first $1,\!126,\!592$ ranked matches played since Duelyst II’s launch. We were primarily concerned with producing a reliable ranking for the game’s strongest players. The resulting rankings were relatively insensitive to these choices.

We found that, for this dataset, taking $\sigma_{0}^{2}=1$ lead to a small but non-negligible chance for players who happened to do very well in their first few matches getting ranked inappropriately highly, whereas taking $\sigma_{0}^{2}=0.7^{2}$ seemingly did not. Taking smaller values of $\sigma_{0}^{2}$ would require top players to play more matches to be accurately rated.

The numbers $M$ and $n$ control how precisely the system can determine a player’s strength, but larger values make the system more computationally expensive to use in practice. We chose $M=7$ because no player had non-negligible mass outside $[-M,M]$, and $n=1000$ because doubling this value did not meaningfully change the system’s output for our dataset.

For Duelyst II, we took $\kappa$ to be

with

using the same notation as Eq. 5. Because all $\nu$’s that arise will be Dirac-only (Definition 2.12), the value of $\kappa_{\mu}$ when $\mu$ is not a Dirac measure irrelevant. As was the case in Section 3.2, a more natural choice for $\kappa$, viewed as a distribution on $\mathbb{R}$, would be the normal distribution $\mathcal{N}(x_{k},\sigma_{\kappa}^{2})$ (and in some applications it might be desirable to take $\kappa$ to be, more generally, a mixture of normal distributions with the same mean), but, for computational convenience, we want $\tilde{\nu}$ to give $0$ mass to $\mathbb{R}\backslash\{x_{0},\dots,x_{n}\}$.

In Duelyst II, each player $A$ is represented as a tuple of $1001$ real numbers:

New players are set such that

Here and later, $\propto$ indicates that the values are then normalized so that they sum to $1$.

After the match outcome $A>B$ is observed, the system updates the values of $\rho_{A}$ and $\rho_{B}$ to

We evaluate these expressions using the Fast Fourier Transform (FFT) as described in Section 4.2. The values of $\rho_{A}$ and $\rho_{B}$ are then replaced with the values of $\rho_{A,A>B}$ and $\rho_{B,A>B}$.

Immediately after completing the step above, $\rho_{A}$ is replaced with $\tilde{\rho}_{A}$, defined by

and $\rho_{B}$ is replaced with $\tilde{\rho}_{B}$ defined analogously. FFT is used to compute the values of $\tilde{\rho}_{A}$ and $\tilde{\rho}_{B}$.

Let $\rho_{A}$ be such that $\rho_{A}(x)=\nu_{A}(\{x\})$ for all $x$. If the match $A>B$ is observed, then $A$’s posterior distribution is given by

Evaluating $\rho_{A,A>B}(x)$ for a specific $x$ by summing the right hand side above directly takes time $\mathcal{O}(n)$. The function $\rho_{A}$ is supported on at most $n$ points, so overall the posterior can be computed in time $\mathcal{O}(n^{2})$. If $B>A$ is observed instead, then the same formula can be used with $\Lambda(x,x_{k})$ replaced by $\Lambda(x_{k},x)=1-\Lambda(x,x_{k})$.

Let $K:\mathbb{R}^{2}\to\mathbb{R}$ be defined by

as in (6). Then $\tilde{\rho}_{A}$ can be computed as

for $x\in S_{A}$. Like the case with match processing discussed above, computing $\tilde{\rho}_{A}$ this way takes time $\mathcal{O}(n^{2})$.