On stability of users equilibria in heterogeneous routing games

Congestion population games provide a powerful tool to model real situations where users compete for resources. An application of such games is traffic, where resources are roads, and players using roads create negative externalities because of congestion effects. The simplest traffic models always assume homogeneity of players, meaning that each player perceives the same delay over the set of the roads.

A Wardrop equilibrium is a flow distribution such that any player cannot unilaterally decrease his perceived delay by changing route [1]. Beckmann et al. showed that finding a Wardrop equilibrium in the homogeneous setting is equivalent to minimizing a convex potential with linear constraints [2]. The existence of such potential has several strong implications: firstly, the game always admits at least a Wardrop equilibrium; secondly, with the additional hypothesis that delays are strictly increasing, the Wardrop equilibrium is unique; moreover, the deterministic logit dynamics converges to a perturbation of the Wardrop equilibrium, where the magnitude of perturbations grows with noise level [3].

The homogeneity assumption simplifies the problem, but it is very restrictive. Heterogeneity, on the one hand, allows the description of many real situations, e.g., when payoffs are a combination of time delays and monetary tolls and users trade off money and time in different ways [4], [5], or when users have different information on the state of the roads [6], [7], [8]. On the other hand, heterogeneous games do not admit a potential [9]. Then, existence and uniqueness of the equilibrium in such games are not trivial, as well as convergence of deterministic logit dynamics.

Existence and uniqueness of Wardrop equilibrium in heterogeneous games have been already investigated in literature. In particular, existence has been proved to hold, even in a more general setting [10]. Conversely, unlike the homogeneous case, uniqueness does not hold in general. Uniqueness of aggregate flows (i.e. essential uniqueness) has been proved to hold just on a small subset of graphs [11].

The asymptotic behaviour of some nontrivial learning dynamics has been studied also, for instance in stochastic setting, where users, playing a repeated routing game, perceive at each stage a stochastic realization of delays on the edges [12], [13]. To the best of our knowledge, no papers in literature investigate the asymptotic behaviour of deterministic logit dynamics in heterogeneous routing games, which are not potential games. In most of the applications the authors analyze the efficiency of the equilibria, but do not investigate the convergence to such equilibria [6], [7], [8]. Given these motivations, in this work we focus on this problem, highlighting the role of the graph topology. In particular, using monotonicity arguments, we show that the dynamics converges to a unique globally asymptotically stable fixed point when the graph is simple, i.e. it has parallel routes between the origin and the destination, or is a series composition of such simple graphs. Moreover, we show that the unique fixed point of the dynamics approaches the set of Wardrop equilibria, as the noise vanishes.

The rest of the paper is organized as follows. In Section II we define the model and summarize the state of the art on existence and uniqueness of Wardrop equilibrium. In Section III we present the main result of the paper, which is that the deterministic logit dynamics on simple graphs and series composition of simple graphs converges to a unique globally asymptotically stable fixed point. Then, in Section IV, we present examples and numerical simulations. Finally, in Section V, we summarize the results and discuss future research lines.

Let $\mathcal{G}=(\mathcal{V},\mathcal{E})$ be a directed multigraph with origin $o\in\mathcal{V}$ and destination $d\in\mathcal{V}$. We consider several populations of users and denote by $\mathcal{P}$ the set of populations. We assume that every population has the same origin-destination pair $o$-$d$. For each population $p$, let $\tau^{p}\geq 0$ be the throughput which has to go from $o$ to $d$, and let $\tau=\sum_{p\in\mathcal{P}}\tau^{p}$ be the aggregate throughput. The populations differ each other for the assignment of delay functions over the edge set. Let $d_{e}^{p}(x):\mathds{R}_{+}\rightarrow\mathds{R}_{+}$ be the delay function of population $p\in\mathcal{P}$ on the edge $e\in\mathcal{E}$. In population games, it is assumed that the number of player is large, the strategy of a single player has negligible effects on payoff functions, and players interact anonymously. This means that delay functions depend only on the distribution of agents strategies, which in case of traffic are flows on the edges. Moreover, we assume separability, i.e., each delay function depends on the flow on the edge itself only. The delay functions are assumed to be continuous and non-decreasing, since edges suffer from congestion. Along the paper we will remark when additional assumption of strictly increasing delays is needed. The set of strategies correspond to the set of routes going from $o$ to $d$, and does not depend on populations. Let $\mathcal{R}$ denote the route set, and let $V$, $E$, $P$ and $R$ denote the cardinalities of $\mathcal{V}$, $\mathcal{E}$, $\mathcal{P}$ and $\mathcal{R}$, respectively.

An admissible route flow distribution for a population $p\in\mathcal{P}$ is a vector $z^{p}\in\mathds{R}_{+}^{R}$ satisfying the throughput constraint, i.e., $\sum_{r\in\mathcal{R}}z^{p}_{r}=\tau^{p}$. For a given route flow vector $z^{p}$, the (unique) edge flow vector is obtained via

where $A\in\mathds{R}^{E\times R}$ is the edge-route incidence matrix, with entries $A_{er}=1$, if the edge $e$ belongs to the route $r$, or $0$ otherwise. Let the aggregate edge flow distribution and aggregate route flow distribution be

respectively. The cost of each route is defined as the sum of delay of edges belonging to the route, i.e.:

where, given $z^{p}$ for all the populations, the aggregate edge flow vector $f$ is computed by (1) and (2). We remark that, although delay functions on the edges are specific for each population, congestion is not specific, in the sense that delay functions depend on aggregate flows only.

In heterogeneous routing games each player aims at minimizing his cost (3) according to delay functions that are specific for the population he belongs to, given the strategies of all of the other players, i.e., flows. Indeed, since the game is a population game, the strategy of the player itself does not affect the flows, and consequently the costs.

Thus, at Wardrop equilibrium, no one can unilaterally decrease his cost by changing route, since every route used by a population $p$ has the minimal cost (measured by the population $p$ itself) among all the routes.

Our main contribution in heterogeneous congestion games is on convergence of deterministic logit dynamics on a certain class of graphs. This dynamics is standard in the literature of population games (see logit dynamics in [3], or perturbed best response dynamics in [15]), and can be derived from Kurtz’s theorem [16, Ch. 11], as the mean-field approximation of the intrinsically stochastic noisy best response dynamics, as the number of player grows larger.

Logit dynamics for the heterogeneous routing game reads, for every population $p\in\mathcal{P}$, and every route $r\in\mathcal{R}$:

where $\eta$ is the inverse of noise level, and the route costs $c_{r}^{p}$ depend in general on all the components of the route flow distribution, since each edge belongs in general to several routes.

In this section we show that the logit dynamics on simple graphs and series composition of simple graphs converges to one globally asymptotically stable fixed point. Moreover, we show that such fixed point approaches the set of the Wardrop equilibria, as the noise vanishes. Let us start by defining those classes of graphs.

Simple graphs are a subset of nearly parallel graphs. Then, essential uniqueness on series of simple graphs is guaranteed if the delays are strictly increasing. In Fig. 2 two examples of those graphs are provided.

The main result of the paper will be now shown.

We are going to prove Theorem 1 through the following intermediate results. First, we establish Lemma 1 on $l_{1}$-contraction in monotone dynamical systems. Next, we show in Proposition 1 that the logit dynamics on simple graphs converges to a unique globally stable fixed point. Finally, we will show that, if the dynamics converges in two graphs, the convergence holds for the series composition of them, also.

Let us start with the plan.

Instead of studying the dynamics over all the possible simple graphs, we are going to investigate just the case of simple graphs composed of $2$ nodes and $E$ parallel edges, going from the origin to the destination. In such graphs, edge flows and route flows are equivalent, and the cost of each route only depends on the flow on the route itself.

Thus, logit dynamics on simple graphs reads:

This system is composed of $E\times P$ coupled equations describing the evolution in time of flows. The main difference between the simple and the general case is that in the simple case the evolution of each edge is affected by the other edge flows only through the normalization, because routes are parallel. This is the key point to prove the next proposition.

The proof relies on that the dynamics of the aggregated flows satisfies the hypotheses of Lemma 1. The assumption of simple graph is crucial, since it guarantees that the Jacobian J is Metzler. For the details, see Appendix B.
The next proposition states that, if the dynamics converges on two graphs, it converges on series composition of them also.

For the proof, see Appendix C. We can now conclude the proof of Theorem 1.

In this section three examples are shown. The first example shows that, on simple graphs, even if the equilibrium is not unique, the logit dynamics selects the equilibrium where each population randomizes uniformly among his optimal routes. The second example illustrates the role played by the noise. Finally, the last example shows that when the equilibrium is not essentially unique, and the noise is sufficiently small, the dynamics selects one equilibrium based on the initial conditions.

In this work the asymptotic behaviour of deterministic logit dynamics in heterogeneous routing games is analyzed, where by heterogeneous we mean that several populations of users, which differ for delay functions, are admitted. We prove that on simple graphs, and series composition of simple graphs, the dynamics converges to a globally asymptotically stable fixed point. The proof follows from monotonicity properties of the dynamics.

We are aware that series composition of simple graphs is not the largest class of graphs where uniqueness of equilibrium holds. Milchtaich proved in [11] that the largest class where essential uniqueness holds, regardless of the assignment of delay functions, is series of nearly parallel graphs. Investigating the behaviour of the dynamics on nearly parallel graphs other than simple is one of the future research lines. However, the arguments used along this paper cannot be applied for graphs where routes are not parallel.

Furthermore, Example 1 suggests that, if the equilibrium is not essentially unique and the noise is sufficiently small, the dynamics converges to one among the equilibria. Since the graphs where essential uniqueness holds are a negligible fraction of real graphs, a further interesting question is whether the dynamics always converge to one among the equilibria in case of multiple essentially different equilibria. Finally, in order to extend these results to more realistic applications, another future research line will be the characterization of the dynamics in case of multiple origin-destination pairs. Unfortunately, even for simple graphs, the dynamics is not monotone. Thus, different techniques should be used.

By definition of the $l_{1}$-norm and the linearity of the derivative, we get

where $\tilde{z}=z+h$. From

eq. (11) is equal to

Because $\text{sign}(h_{i})\in\{-1,0,1\}$, it follows

If we consider just the first part of the second term of the last equation we get

where

because for each $j\in I_{-}$ the sum $\sum_{i\in I_{+}}{\frac{\partial{f_{i}}}{\partial{z_{j}}}}$ contains for sure the element on the diagonal, but may not contain all the elements of the $j^{th}$ column out of the diagonal.
At the same time

because $h_{j}=-|h_{j}|$ for each $j\in I_{-}$ and each term $\frac{\partial{f_{i}}}{\partial{z_{j}}}$ do not belong to the diagonal and it is non-negative by hypothesis.
Similarly we can operate on the second part of (12) and we obtain

Let us consider the evolution of aggregate flows $f_{e}=\sum_{p=1}^{P}f_{e}^{p}$:

The system above has some interesting properties. Firstly, it is autonomous in the aggregate flows. Secondly, it is monotone, i.e. the non-diagonal elements of the Jacobian are non-negative: indeed, $\forall i\neq j$, $\dot{f}_{i}$ depend on $f_{j}$ only by the normalization. Moreover

From (14) it follows that the Jacobian is strictly diagonally dominant by columns, in particular $\sum_{i=1}^{E}J_{ij}=-1$ $\forall j$.
Then, (13) satisfies the hypotheses of Lemma 1 with $a=1$, and we can conclude that it admits one globally exponentially stable fixed point.
It still remains to prove that, since aggregate flows converge, also the edge flows of every population converge to a unique globally asymptotically stable fixed point. To this end, let us write (10) as:

where $f\in\mathds{R}_{+}^{E}$ is the aggregate edge flow distribution, which evolves according to (13), independently of population flows $f^{p}$, and

We now show that, since every aggregated edge flow $f_{e}$ converges, every population edge flow $f_{e}^{p}$ converges as well. Observe that $F_{e}^{p}(f)$ is continuous for every edge and population. Let $\tilde{f}=\lim_{t\rightarrow+\infty}f(t)$ denote the fixed point of aggregate flow distribution. By continuity of $F_{e}^{p}(f)$, and convergence of (13), we get that $\forall\epsilon>0$, $\exists T>0$ such that

It follows:

Since evolution of flows does not depend explicitly on time, we can, without loss of generality, translate the axis of time by $-T$, so that (18) holds $\forall t>0$. Moreover, by continuity of delay functions, in $t=0$,

It is easy to prove that:

where the right terms in (20) are solutions of

respectively, with initial condition $f_{e}^{p}(0)$. Indeed, let us assume that the first inequality in (20) does not hold, i.e., it exists a time $t_{2}>0$ such that

Then, it must exist a time $t_{1}$ such that $0\leq t_{1}<t_{2}$,

and

which is impossible by (19). A similar argument can be used to prove the second inequality in (20). Then, from (20), it follows:

and since we can choose $\epsilon$ arbitrarily small,

Hence, $f_{e}^{p}(t)$ converges to a unique fixed point for each initial condition.
Let $\tilde{f}_{e}^{p}(\eta)$ denote the unique fixed point as function of noise level. Let $\eta_{n}$ be an infinite sequence, such that $\lim_{\eta\rightarrow+\infty}\eta_{n}=+\infty$. Since $\tilde{f}_{e}^{p}(\eta_{n})$ is compact, $\eta_{n}$ admits a subsequence $\eta_{n_{k}}$ such that $\tilde{f}_{e}^{p}(\eta_{n_{k}})$ converges. Let $(\tilde{f}_{e}^{p})^{*}$ denote such limit.

From (16), it follows:

where $\mathcal{E}_{opt}^{p}(\tilde{f}^{*})$ denotes the set of edges that have minimal cost, given $\tilde{f}^{*}$, for population $p$, and we recall that the equivalence between routes and edges holds by assumption of simple graph and Remark 2. Then, since (9) admits a unique fixed point, from (15), we get:

Equations (27) and (28) imply that, for every edge $e\in\mathcal{E}$, and every population $p\in\mathcal{P}$:

that is equivalent to the definition of the Wardrop equilibrium (4) in simple graphs with $2$ nodes and $E$ parallel edges. Moreover, we also observe that, even if the game admits multiple equilibria (with same aggregated flows, since the graph is simple) $(\tilde{f}^{p}_{e})^{*}$ represents a specific equilibrium. Indeed, from (27) and (28), we observe that in $(\tilde{f}^{p}_{e})^{*}$ each population randomizes uniformly among its optimal routes. Then, every subsequence $\tilde{f}_{e}^{p}(\eta_{n_{k}})$ converges to the Wardrop equilibrium where each population randomizes uniformly among its optimal routes. Consequently, the same holds for $\tilde{f}_{e}^{p}(\eta_{n})$ and $\tilde{f}_{e}^{p}(\eta)$.

Every route in $\mathcal{G}$ is composed of a route in $\mathcal{G}_{1}$ concatenated with a route in $\mathcal{G}_{2}$. Given this motivation, let us denote every route of $\mathcal{G}$ by a double index, where the first one refers to routes in $\mathcal{G}_{1}$, and the second one to routes in $\mathcal{G}_{2}$. Then, (9) reads:

Since cost functions are additive, we have $c_{ij}(z)=c_{i}(z)+c_{j}(z)$. Then:

Obviously, we have

Then, by summing over the index $j$ (or $i$) in (31), we get the following dynamics for $z_{i}$ (or $z_{j}$):

We observe that (33) is equivalent to (9), meaning that dynamics on $\mathcal{G}_{1}$ (and $\mathcal{G}_{2}$) induced by logit dynamics on $\mathcal{G}$ is equivalent to logit dynamics on $\mathcal{G}_{1}$ (and $\mathcal{G}_{2}$) separately. Let us assume that logit dynamics on both $\mathcal{G}_{1}$ and $\mathcal{G}_{2}$ converges to a globally asymptotically stable fixed point. Following the steps of the proof of Proposition 1, (31) can be written as

where $z_{\mathcal{G}_{1}}$ and $z_{\mathcal{G}_{2}}$ denote the route flow distributions on $\mathcal{G}_{1}$ and $\mathcal{G}_{2}$, respectively. Since $z_{\mathcal{G}_{i}}$ and $z_{\mathcal{G}_{j}}$ converge for every initial condition by assumption, the convergence of (34) easily follows.
Moreover, since the route choice on $\mathcal{G}$ is decoupled in two independent route choices on $\mathcal{G}_{1}$ and $\mathcal{G}_{2}$, the Wardrop equilibria on $\mathcal{G}$ are the compositions of the equilibria on $\mathcal{G}_{1}$ and $\mathcal{G}_{2}$. Thus, since logit dynamics on $\mathcal{G}$, and logit dynamics on $\mathcal{G}_{1}$ and $\mathcal{G}_{2}$ separately, converge to same route flow distributions $z_{\mathcal{G}_{1}}$ and $z_{\mathcal{G}_{2}}$, and since the fixed point of logit dynamics on $\mathcal{G}_{1}$ and $\mathcal{G}_{2}$ approach the set of the equilibria as the noise vanishes, we conclude that the fixed point of logit dynamics on $\mathcal{G}$ approaches the set of Wardrop equilibria, as the noise vanishes.