The Four Levels of Fixed-Points
in Mean-Field Models

Let there be $N$ particles. Each particle has a state. The particle states evolves over time in a Markovian fashion. Let $X^{(N)}_{n}(t)$ denote the state of the $n$th particle at time $t$ and let $\mathcal{Z}$, a finite set, denote the state space. The macroscopic behaviour of the system at time $t$ is encoded in the empirical measure

$\mu_{N}(t)(z)$ is the fraction of particles in state $z$ at time $t$. To describe the evolution of the states of the particles, we consider a directed and connected graph $(\mathcal{Z},\mathcal{E})$. This graph encodes the set of all allowed transitions of a particle; whenever $(z,z^{\prime})\in\mathcal{E}$, a particle in state $z$ can move to state $z^{\prime}$. For each $(z,z^{\prime})\in\mathcal{E}$, we are given a function $\lambda_{z,z^{\prime}}$ on the space of probability distributions on $\mathcal{Z}$ (which we denote by $M_{1}(\mathcal{Z})$ and view as a subset of $\mathbb{R}^{|\mathcal{Z}|-1}$). A particle at state $z$ at time $t$ moves to state $z^{\prime}$ at rate $\lambda_{z,z^{\prime}}(\mu_{N}(t))$. That is, the evolution of the state of a particle depends on the states of the other particles only through the empirical measure of the states of all the particles. This description of the state evolution of the particles gives rise to two continuous-time Markov processes: (i) the Markov process $\{(X^{(N)}_{n}(t),1\leq n\leq N),t\geq 0\}$ that describes the joint evolution of the states of all the particles, and (ii) the more convenient Markov process $\{\mu_{N}(t),t\geq 0\}$ that describes the evolution of the empirical measure of the system of particles, i.e., the empirical measure process.

Often in practice, properties that are relevant for the system (e.g., throughput in the WLAN example, equilibrium distribution in the SIS model) depend only on the macroscopic behaviour of the system and can be obtained from the empirical measure $\mu_{N}(t)$. Thus it is important to study the empirical measure process $\mu_{N}$ in some detail. Towards this, we discuss a law of large numbers result for $\mu_{N}$ [8, 9].

Assume that the transition rates $\lambda_{z,z^{\prime}},\,\,(z,z^{\prime})\in\mathcal{E}$, are Lipschitz continuous on $M_{1}(\mathcal{Z})$. Assume that the initial conditions $\{\mu_{N}(0)\}_{N\geq 1}$ converge (in $M_{1}(\mathcal{Z}))$ to a probability distribution $\nu\in M_{1}(\mathcal{Z})$. Then for any fixed $T>0$, the empirical measure process $\{\mu_{N}(t),0\leq t\leq T\}$ converges in probability to the solution to the ODE

as $N\to\infty$; here, for any $\xi\in M_{1}(\mathcal{Z})$, $\Lambda_{\xi}$ denotes the $|\mathcal{Z}|\times|\mathcal{Z}|$ rate matrix (i.e., $\Lambda_{\xi}(z,z^{\prime})=\lambda_{z,z^{\prime}}(\xi)$ when $(z,z^{\prime})\in\mathcal{E}$, $\Lambda_{\xi}(z,z^{\prime})=0$ when $(z,z^{\prime})\notin\mathcal{E}$, and $\Lambda_{\xi}(z,z)=-\sum_{z^{\prime}\neq z}\lambda_{z,z^{\prime}}(\xi)$ for all $z\in\mathcal{Z}$) when the empirical measure is $\xi$, $\Lambda^{*}_{\xi}$ denotes its transpose, $\mu(t)$ is the empirical measure at time $t$ viewed as a column vector, $\dot{\mu}(t)$ is its time derivative, and the convergence is in the space of $M_{1}(\mathcal{Z})$-valued trajectories on $[0,T]$. The above ODE is called the McKean-Vlasov equation, and it evolves on the space of probability distributions on $\mathcal{Z}$. Owing to this convergence result, one can view the empirical measure process as a small random perturbation of the McKean-Vlasov equation and, for large $N$, the properties of the empirical measure process can be well approximated by the corresponding properties of the McKean-Vlasov equation.

We now discuss a decoupling approximation (also known as asymptotic independence property) for the mean-field model [10]. Again, we assume that the transition rates are Lipschitz continuous and that the initial conditions of particles are independent and identically distributed as $\nu$¹¹1As a consequence, $\mu_{N}(0)\rightarrow\nu$ in distribution as $N\to\infty$.. Consider two tagged particles, $1$ and $2$. The decoupling approximation tells us that, for each $t>0$, the random variable $(X^{(N)}_{1}(t),X^{(N)}_{2}(t))\rightarrow(Y_{1}(t),Y_{2}(t))$ in distribution as $N\to\infty$, where both the marginals $Y_{1}(t)$ and $Y_{2}(t)$ are distributed according to the solution to the McKean-Vlasov equation (1) at time $t$ with initial condition $\nu$, and $Y_{1}(t)$ and $Y_{2}(t)$ are independent. That is, the initial chaos propagates over time²²2The initial condition of independence of the particle states at time $t=0$ can be relaxed to exchangeability so long as $\mu_{N}(0)\to\nu$ in distribution as $N\to\infty$, where $\nu$ is a fixed initial condition. in the large-$N$ limit even though particles interact with each other. This property also has an obvious extension to any finite number (but not depending on $N$) of tagged particles. A similar decoupling approximation holds in the stationary regime [9]. These decoupling approximations play a crucial role in framing the fixed-point equations; see Section III.

We start with the fixed-point equations that arise at the level of macroscopic observables of the system. While an abstract description of a fixed-point equation for generic macroscopic observables is possible, for concreteness, we use the WLAN described in Section II-B1 as a running example. Recall the MAC protocol (with slotted time) and the Markovian continuous-time caricature described in Section II-B1.

Consider a tagged node in a system of $N$ nodes and let $\gamma$ denote its collision probability (at stationarity). That is, $\gamma$ is the probability that a packet transmitted by this node undergoes a collision. By symmetry of the model, this collision probability can be taken to be the same for all nodes.

We now compute the total attempt probability of the system as a response to the mean-field induced by $\gamma$ [6]. If $\beta(\gamma)$ is the attempt probability of the system, by symmetry, the attempt probability of the tagged node is $\beta(\gamma)/N$. Observe that the time instants at which the state of the tagged node becomes $0$ are renewal instants. Consider a renewal cycle between two successive visits to state $0$. Since the collision probability is $\gamma$ and the attempt probability in state $i$ is $c_{i}/N$, the mean renewal cycle duration (in number of time slots) is

Similarly, the mean number of slots used for a successful packet transmission during a renewal cycle is

Noting that the tagged node makes exactly one successful transmission in a renewal cycle, by the renewal-reward theorem, the attempt probability of the tagged node is $\frac{\beta(\gamma)}{N}=\frac{1+\gamma+\gamma^{2}+\cdots}{\frac{N}{c_{0}}+\frac{\gamma N}{c_{1}}+\cdots+\frac{\gamma^{K}N}{c_{K}}+\frac{\gamma^{K+1}}{1-\gamma}\cdot\frac{N}{c_{K}}},$ leading to

Let us now see the response of the tagged node when the mean-field is such that every other node sees a collision probability $\gamma$. Using the decoupling approximation (see Section II-D), the collision probability is

for large $N$. Defining $G(\gamma)=1-\exp\{-\beta(\gamma)\}$, we see that $G(\gamma)$ is the response collision probability seen by the tagged node starting from an assumption that the collision probability $\gamma$. Hence self consistency demands that

which is a fixed-point of $G$. It can be checked that (3) has a unique solution $\gamma^{*}\in(0,1)$ when the parameters $c_{k}$ are decreasing in $k$ [6]. For WLANs with a few tens of nodes and with the standard exponential back-off, i.e., $c_{i}=c_{i-1}/2$, $i=1,2,\ldots,K$, this approximation works well in practice and $\gamma^{*}$ is close to the collision probability observed in practice. Thus, the fixed-point analysis at the level of macroscopic observables provides a simple and elegant way to characterise the performance of the WLAN system at equilibrium.

For a generic macroscopic observable for the mean-field model in Section II-A, one can come up with a fixed-point equation involving this observable by using the decoupling approximation in a suitable manner.

We now describe a fixed-point equation in the space of (equilibrium) probability distributions on $\mathcal{Z}$.

At time $t$, suppose that the distribution of each particle’s state is $\xi$. Further, suppose that the initial conditions of the particles are exchangeable and $\mu_{N}(0)\to\nu$ in distribution as $N\to\infty$, for some deterministic $\nu$. Invoking the decoupling approximation in the transient regime, the empirical measure of the system at time $t$ concentrates near $\xi$ for large $N$. As the particles evolve over time, even though the empirical measure changes it remains close to $\xi$. We may assume that the mean-field is $\xi$. Consider a tagged particle. Let $m$ denote the equilibrium response map that maps a given mean-field to the equilibrium response of the tagged particle. That is, $m(\xi)$ denotes the equilibrium distribution of this tagged particle in response to the states of the particles distributed as $\xi$. Since the tagged particle’s transition rates are given by the rate matrix $\Lambda_{\xi}$ when the mean-field is $\xi$, $m(\xi)$ is the $m$ that solves

In (4), the left-hand side (LHS) is the total probability flux into $z$ and the right-hand side (RHS) is the total probability flux out of $z$ for the tagged particle in response to the mean-field $\xi$. At equilibrium these are in balance, which is the condition (4), and $m(\xi)$ is the $m$ that solves (4).

Self consistency demands that the appropriate equilibrium distribution is the $\xi^{*}$ that solves

a fixed-point equation on the distribution of states of the particle for the equilibrium response map $m$. Equivalently,

Existence of the fixed point would follow from Brouwer’s fixed-point theorem if one can show that mapping $m$ is continuous. If there is a unique fixed-point $\xi^{*}$ for (5), then one might expect the system to equilibrate at $\xi^{*}$. This is not always the case, as we shall explain in Section V.

We now describe a fixed-point equation on the space of $M_{1}(\mathcal{Z})$-valued trajectories for the dynamic response map that maps a given flow of the mean-field over time to the flow of the distribution over states of a particle over time under the given mean-field.

Let an $M_{1}(\mathcal{Z})$-valued trajectory $\xi(\cdot)$ denote the time-evolution of the distribution of a tagged particle, i.e., the state of the particle at time $t$ is distributed as $\xi(t)$. Suppose that each particle’s state evolution is prescribed by the same $\xi(\cdot)$. Then the empirical measure of the system of particles at time $t$ is approximately $\xi(t)$ and hence the time-trajectory of the mean-field is approximately $\xi(\cdot)$. The decoupling approximation suggests that the particles evolve, approximately, in an iid fashion. In response, the state of the tagged particle evolves according to the rate matrix $\Lambda_{\xi(t)}$ at time $t$ under the mean-field $\xi(\cdot)$. This evolution produces a response probability distribution flow over time denoted $\mathcal{M}(\xi(\cdot))$ for the tagged particle in response to the mean-field being $\xi(\cdot)$, i.e., $\mathcal{M}$ is the dynamic response map. Write $m(t)=\mathcal{M}(\xi(t))$. One can write an ODE for $m(t)$ based on the fluxes of probability distributions into and out of each state:

here, the RHS is the difference between the total probability flux into and out of state $z$ in response to the mean-field $\xi(t)$. Self-consistency then demands that

a fixed-point equation on the space of $M_{1}(\mathcal{Z})$-valued trajectories for the response dynamics map $\mathcal{M}$. That is, we must have $m(t)=\xi^{*}(t)~{}\forall t\geq 0$. Substitution of this in (6) yields

which is precisely the McKean-Vlasov equation (1).

If the transition rates are assumed to be Lipschitz continuous on $M_{1}(\mathcal{Z})$, then it is easy to check that the mapping $\xi\mapsto\Lambda_{\xi}^{*}\xi$ is also Lipschitz continuous on $M_{1}(\mathcal{Z})$. As a consequence, standard results from ODE theory imply that there exists a unique solution to the ODE (8) and hence there exists a unique fixed-point for (7).

We now describe a Level-4 fixed-point equation over the space of probability distributions $D([0,\infty),\mathcal{Z})$, which is the space of $\mathcal{Z}$-valued trajectories. Note that this is different from Level-3 in the following way. Level-3 refers to fixed-points in the space of $M_{1}(\mathcal{Z})$-valued trajectories over time whereas Level-4 refers to fixed-points in $M_{1}(D([0,\infty),\mathcal{Z}))$, the space of distributions over $D([0,\infty),\mathcal{Z})$.

Let $Q\in M_{1}(D([0,\infty),\mathcal{Z}))$ denote the probability distribution of a typical particle on the space of $\mathcal{Z}$-valued trajectories. At each time $t$, the time-marginal of $Q$ is the distribution of the state of the typical particle at time $t$. More precisely, it is the push forward $Q\circ\pi_{t}^{-1}$ of the measure $Q$ under the canonical coordinate projection map $\pi_{t}$; $Q\circ\pi_{t}^{-1}$ is a probability distribution on $\mathcal{Z}$. Assuming the decoupling approximation, the mean-field approximately evolves as $\{Q\circ\pi_{t}^{-1},t\geq 0\}$.

Now consider a tagged particle responding to the mean-field $\{Q\circ\pi_{t}^{-1},t\geq 0\}$. Let $\mathcal{T}(Q)$ denote the law of the tagged particle’s evolution over time. Note that $\mathcal{T}(Q)\in M_{1}(D([0,T),\mathcal{Z}))$, it is the response to the mean-field induced by $Q$, and it depends on $Q$ only through the flow $\{Q\circ\pi_{t}^{-1},t\geq 0\}$. Since the flow is given by $\xi(t)=\{Q\circ\pi_{t}^{-1},t\geq 0\}$, $\mathcal{T}(Q)$ is the law of the Markov process associated with the rates $\Lambda_{\xi(t)}$. Consistency demands that

In probability theory, the problem of existence and uniqueness of Markov processes given a generator (in our case, the transition rates $\Lambda_{\xi(t)}$), or an extended generator, is known as the martingale problem [11]. In our case, the generator itself is flexible, is determined by $Q$, and we are looking for a special solution that fixes the map $Q\mapsto\mathcal{T}(Q)$. This is called a nonlinear martingale problem [12]. Let $Q^{*}$ be a fixed point. The analogue of the Kolmogorov forward equation that governs the evolution of the time-marginals of $Q^{*}$ is a nonlinear ODE (1), the McKean-Vlasov equation. It turns out that there exists a unique fixed-point for (9) if the transition rates are uniformly Lipschitz continuous on $M_{1}(\mathcal{Z})$ [12].

To explain the connection between Level-1 and Level-2 fixed-points, let us consider the WLAN example. Let $\xi^{*}$ be the unique Level-2 fixed-point.

We first compute the collision probability when the mean-field is at $\xi^{*}$. Consider a tagged node. Let $\gamma$ denote the collision probability of this node when the macroscopic behaviour of the system is $\xi^{*}$; this collision probability will be the same for all the nodes. Consider a tagged node in state $z_{0}$. As explained in Section II-A, the collision probability of the tagged node is $\gamma\simeq 1-\exp\{-\langle c,\xi^{*}\rangle\}$ for large $N$.

On the other hand, the Level-1 fixed-point analysis already gives us a collision probability $\gamma^{*}$, which is a fixed-point of (3). We thus expect that the two are equal, i.e., $\gamma^{*}=1-\exp\{-\langle c,\xi^{*}\rangle\}$. This can be seen as follows. On one hand, as explained in Section III-A, a renewal argument tells us that $\beta(\gamma^{*})/N$ is the attempt probability of a tagged node. On the other hand, using the decoupling approximation, assuming that each node at state $z$ independently attempts with probability $c_{z}/N$, we see that the attempt probability of a tagged node is $\langle c,\xi^{*}\rangle/N$ when the mean-field is $\xi^{*}$. Thus, we have $\langle c,\xi^{*}\rangle=\beta(\gamma^{*})$, and hence $\gamma^{*}=1-\exp\{-\langle c,\xi^{*}\rangle\}$. This identity $\langle c,\xi^{*}\rangle=\beta(\gamma^{*})$ can indeed be checked using the expression of $\beta$ given in (2) and the fact that $\xi^{*}$ satisfies $(\Lambda_{\xi^{*}})^{*}\xi^{*}=0$. Therefore, the Level-2 fixed-point, which is a distribution over the states of a particle, explains the Level-1 fixed-point which is a macroscopic observable.

Such connections between macroscopic observables and distribution over states of the particles appear in thermodynamics. The relationship between the macroscopic variables of collision probability and attempt probability and their connection to the Level-2 fixed-point in WLAN is analogous to the relationship between macroscopic variables like temperature, pressure, and volume of an ideal gas and their connection to the Maxwell-Boltzmann distribution that describes the distribution of the velocities of the gas molecules. The passage from statistical mechanics to thermodynamics is made rigorous by large deviation theory; minimisations of rate functions in large deviation theory are mapped to the variational principles of free energy minimisation in thermodynamics [13].

We now describe the connection between the Level-2 and Level-3 fixed-points. Suppose that $\xi^{*}$ is a Level-2 fixed-point of (5), i.e., $(\Lambda_{\xi^{*}})^{*}\xi^{*}=0$. The RHS of the Level-3 fixed-point equation (8) implies that when we start the McKean-Vlasov equation at $\xi^{*}$, it stays at $\xi^{*}$ for all times. That is, $\xi(\cdot)\equiv\xi^{*}$ is a stationary Level-3 fixed-point. Conversely, any stationary Level-3 fixed-point of (8) is a Level-2 fixed-point of (5), since we must necessarily have $(\Lambda_{\xi})^{*}\xi=0$. Thus, Level-2 fixed-points are precisely those points in $M_{1}(\mathcal{Z})$ that are stationary Level-3 fixed-points.

Further, suppose that there is the unique stationary Level-3 fixed-point and that all its non-stationary Level-3 fixed-points (which are trajectories) converge to $\xi^{*}$ as time becomes large. Then there is a unique Level-2 fixed-point and it can be obtained from any Level-3 fixed-point by taking the large-time limit. In this special case, the macroscopic behaviour of the $N$-particle noisy system (which is a noisy version of the Level-2 fixed-point) in the large-$N$ limit concentrates on the stationary Level-3 fixed-point [9, Theorem 2.3].

Suppose that $Q$ is a Level-4 fixed-point of (9). (We omit the superscript * to reduce clutter). Then the time-marginal of $Q$ at time $t$, $Q(t)=Q\circ\pi_{t}^{-1}$, is the distribution of a tagged particle at time $t$ (which is also approximately the mean-field at time $t$ when we have large number of particles). This distribution over states changes over time as particles make transitions. As we did in Section III-C, the infinitesimal change in the distribution at time $t$ can be computed by the difference between the total probability flux entering and exiting various states. For state $z$, this difference is $(\Lambda_{Q(t)}^{*}Q(t))(z)$. On the other hand, this infinitesimal change is precisely $\dot{Q}(t)(z)$, the $z$-component of the time-derivative of $Q(t)$. Writing this condition as a vector condition for all $z$, we get

Thus $Q(\cdot)$ is a Level-3 fixed-point. Hence, time-marginals of Level-4 fixed-points give rise to Level-3 fixed-points.

Conversely, consider a Level-3 fixed-point $\{Q(t),t\geq 0\}$. Clearly, $Q\in\mathcal{T}(\{R:R\circ\pi_{t}^{-1}=Q(t),t\geq 0\})$, but the set $\mathcal{T}(\{R:R\circ\pi_{t}^{-1}=Q(t),t\geq 0\})$ can possibly contain other probability distributions on trajectories. However, it turns out that when the transition rates $\lambda_{z,z^{\prime}}$ are Lipschitz continuous on $M_{1}(\mathcal{Z})$, $\mathcal{T}(\{R:R\circ\pi_{t}^{-1}=Q(t),t\geq 0\})=\{Q\}$ [12]. Thus, under the assumption that the transition rates are Lipschitz continuous on $M_{1}(\mathcal{Z})$, there is a one-one correspondence between the Level-3 and Level-4 fixed-points.