Convergence Rate of LQG Mean Field Games with Common Noise

In this section, we present the formulation of the Mean Field Game in the sample space $\Omega$.

Let $T>0$ be a given time horizon. We assume that $W=\{W_{t}\}_{t\geq 0}$ is a standard Brownian motion constructed on the probability space $(\bar{\Omega},\bar{\mathcal{F}}=\bar{\mathcal{F}}_{T},\bar{\mathbb{P}},\bar{\mathbb{F}}=\{\bar{\mathcal{F}}_{t}\}_{t\geq 0})$. Similarly, the process $\tilde{W}=\{\tilde{W}_{t}\}_{t\geq 0}$ is a standard Brownian motion constructed on the probability space $(\tilde{\Omega},\tilde{\mathcal{F}}=\tilde{\mathcal{F}}_{T},\tilde{\mathbb{P}},\tilde{\mathbb{F}}=\{\tilde{\mathcal{F}}_{t}\}_{t\geq 0})$. We define the product structure as follows:

$$\Omega=\bar{\Omega}\times\tilde{\Omega},\quad\mathcal{F},\quad\mathbb{F}=\{\mathcal{F}_{t}\}_{t\geq 0},\quad\mathbb{P},$$

where $(\mathcal{F},\mathbb{P})$ is the completion of $(\bar{\mathcal{F}}\otimes\tilde{\mathcal{F}},\bar{\mathbb{P}}\otimes\tilde{\mathbb{P}})$ and $\mathbb{F}$ is the complete and right continuous augmentation of $\{\bar{\mathcal{F}}_{t}\otimes\tilde{\mathcal{F}}_{t}\}_{t\geq 0}$.

Note that, $W$ and $\tilde{W}$ are two Brownian motions from separate sample spaces $\bar{\Omega}$ and $\tilde{\Omega}$, they are independent of each other in their product space $\Omega$. In our manuscript, $W$ is called individual or idiosyncratic noise, and $\tilde{W}$ is called common noise, see their different roles in the problem formulation later defined via fixed point condition (4). To proceed, we denote by $L^{p}:=L^{p}(\Omega,\mathbb{P})$ the set of random variables $X$ on $(\Omega,\mathcal{F},\mathbb{P})$ with finite $p$-th moment with norm $\|X\|_{p}=(\mathbb{E}\left[|X|^{p}\right])^{1/p}$ and by $L_{\mathbb{F}}^{p}:=L_{\mathbb{F}}^{p}(\Omega\times[0,T])$ the space of all $\mathbb{R}$ valued $\mathbb{F}$-progressively measurable random processes $\alpha$ such that

$$\mathbb{E}\left[\int_{0}^{T}|\alpha_{t}|^{p}dt\right]<\infty.$$

Let $\mathcal{P}_{p}(\mathbb{R})$ denote the Wasserstein space of probability measures $\mu$ on $\mathbb{R}$ satisfying $\int_{\mathbb{R}}x^{p}d\mu(x)<\infty$ endowed with $p$-Wasserstein metric $\mathbb{W}_{p}(\cdot,\cdot)$ defined by

$$\mathbb{W}_{p}(\mu,\nu)=\inf_{\pi\in\Pi(\mu,\nu)}\left(\int_{\mathbb{R}\times\mathbb{R}}|x-y|^{p}d\pi(x,y)\right)^{\frac{1}{p}},$$

where $\Pi(\mu,\nu)$ is the collection of all probability measures on $\mathbb{R}\times\mathbb{R}$ with its marginals agreeing with $\mu$ and $\nu$.

Let $X_{0}\in L^{2}$ be a random variable that is independent with $W$ and $\tilde{W}$. For any control $\alpha\in L^{2}_{\mathbb{F}}$, consider the state $X=\{X_{t}\}_{t\geq 0}$ of the generic player is governed by a stochastic differential equation (SDE)

$$dX_{t}=\alpha_{t}dt+dW_{t}+d\tilde{W}_{t}$$

(1)

with the initial value $X_{0}$, where the underlying process $X:[0,T]\times\Omega\mapsto\mathbb{R}$. Given a random measure flow $m:(0,T]\times\Omega\mapsto\mathcal{P}_{2}(\mathbb{R})$, the generic player wants to minimize the expected accumulated cost on $[0,T]$:

$$\begin{array}[]{ll}J(x,\alpha)=\displaystyle\mathbb{E}\left[\left.\int_{0}^{T}\left(\frac{1}{2}\alpha_{s}^{2}+F(X_{s},m_{s})\right)\,ds\right\rvert X_{0}=x\right]\end{array}$$

(2)

with some given cost function $F:\mathbb{R}\times\mathcal{P}_{2}(\mathbb{R})\mapsto\mathbb{R}$.

The objective of the control problem for the generic player is to find its optimal control $\hat{\alpha}\in\mathcal{A}:=L^{4}_{\mathbb{F}}$ to minimize the total cost, i.e.,

$$V[m](x)=J[m](x,\hat{\alpha})\leq J[m](x,\alpha),\quad\forall\alpha\in\mathcal{A}.$$

(3)

Associated to the optimal control $\hat{\alpha}$, we denote the optimal path by $\hat{X}=\{\hat{X}_{t}\}_{t\geq 0}$.

Next, to introduce the MFG Nash equilibrium, it is useful to emphasize the dependence of the optimal path and optimal control of the generic player, as well as its associated value, on the underlying measure flow $m$. These quantities are denoted as $\hat{X}_{t}[m]$, $\hat{\alpha}_{t}[m]$, $J[m]$, and $V[m]$, respectively.

We now present the definitions of the equilibrium measure, equilibrium path, and equilibrium control. Please also refer to page 127 of [5] for a general setup with a common noise.

The flowchart of the MFG diagram is given in Figure 1. It is noted from the optimality condition (3) and the fixed point condition (4) that

$$J[\hat{m}](x,\hat{\alpha})\leq J[\hat{m}](x,\alpha),\quad\forall\alpha$$

holds for the equilibrium measure $\hat{m}$ and its associated equilibrium control $\hat{\alpha}$, while it is not

$$J[\hat{m}](x,\hat{\alpha})\leq J[m](x,\alpha),\quad\forall\alpha,m.$$

Otherwise, this problem turns into a McKean-Vlasov control problem, which is essentially different from the current Mean Field Games setup. Readers refer to [7, 6] to see the analysis of this different model as well as some discussion of the differences between these two problems.

In this subsection, we set up $N$-player game and define the Nash equilibrium of $N$-player game in the sample space $\Omega^{(N)}$. Firstly, let $W^{(N)}=(W^{(N)}_{i}:i=1,2,\dots,N)$ be an $N$-dimensional standard Brownian motion constructed on the space $(\bar{\Omega}^{(N)},\bar{\mathcal{F}}^{(N)},\bar{\mathbb{P}}^{(N)},\bar{\mathbb{F}}^{(N)}=\{\bar{\mathcal{F}}^{(N)}_{t}\}_{t\geq 0})$ and $\tilde{W}=\{\tilde{W}_{t}\}_{t\geq 0}$ be the common noise in MFG defined in Section 2.1 on $(\tilde{\Omega},\tilde{\mathcal{F}},\tilde{\mathbb{P}})$. The probability space for the $N$-player game is $\left(\Omega^{(N)},\mathcal{F}^{(N)},\mathbb{F}^{(N)},\mathbb{P}^{(N)}\right)$, which is constructed via the product structure with

$$\Omega^{(N)}=\bar{\Omega}^{(N)}\times\tilde{\Omega},\quad\mathcal{F}^{(N)},\quad\mathbb{F}^{(N)}=\left\{\mathcal{F}^{(N)}_{t}\right\}_{t\geq 0},\quad\mathbb{P}^{(N)}.$$

where $(\mathcal{F}^{(N)},\mathbb{P}^{(N)})$ is the completion of $(\bar{\mathcal{F}}^{(N)}\otimes\tilde{\mathcal{F}},\bar{\mathbb{P}}^{(N)}\otimes\tilde{\mathbb{P}})$ and $\mathbb{F}^{(N)}$ is the complete and right continuous augmentation of $\{\bar{\mathcal{F}}_{t}^{(N)}\otimes\tilde{\mathcal{F}}_{t}\}_{t\geq 0}$.

Consider a stochastic dynamic game with $N$ players, where each player $i\in\{1,2,\dots,N\}$ controls a state process $X_{i}^{(N)}=\{X_{it}^{(N)}\}_{t\geq 0}$ in $\mathbb{R}$ given by

$$dX_{it}^{(N)}=\alpha_{it}^{(N)}dt+dW_{it}^{(N)}+d\tilde{W}_{t},\quad X_{i0}^{(N)}=x^{(N)}_{i}$$

(5)

with a control $\alpha_{i}^{(N)}$ in an admissible set $\mathcal{A}^{(N)}:=L^{4}_{\mathbb{F}^{(N)}}$ and random initial state $x^{(N)}_{i}$.

Given the strategies $\alpha_{-i}^{(N)}=(\alpha_{1}^{(N)},\dots,\alpha_{i-1}^{(N)},\alpha_{i+1}^{(N)},\dots,\alpha_{N}^{(N)})$ from other players, the objective of player $i$ is to select a control $\alpha_{i}^{(N)}\in\mathcal{A}^{(N)}$ to minimize her expected total cost given by

$\displaystyle J_{i}^{N}\left(x^{(N)},\alpha_{i}^{(N)};\alpha_{-i}^{(N)}\right)$

$\displaystyle=\mathbb{E}\left[\left.\int_{0}^{T}\left(\frac{1}{2}\left(\alpha_{it}^{(N)}\right)^{2}+F\left(X_{it}^{(N)},\rho\left(X_{t}^{(N)}\right)\right)\right)dt\right\rvert X_{0}^{(N)}=x^{(N)}\right],$

(6)

where $x^{(N)}=(x_{1}^{(N)},x_{2}^{(N)},\dots,x_{N}^{(N)})$ is a $\mathbb{R}^{N}$-valued random vector in $\Omega^{(N)}$ to denote the initial state for $N$ players, and

$$\rho\left(x^{(N)}\right)=\frac{1}{N}\sum_{i=1}^{N}\delta_{x_{i}^{(N)}}$$

is the empirical measure of the vector $x^{(N)}$ with Dirac measure $\delta$. We use the notation $\alpha^{(N)}:=(\alpha_{i}^{(N)},\alpha_{-i}^{(N)})=(\alpha_{1}^{(N)},\alpha_{2}^{(N)},\ldots,\alpha_{N}^{(N)})$ to denote the control from $N$ players as a whole. Next, we give the equilibrium value function and equilibrium path in the sense of the Nash game.

We consider three convergence questions on $N$-player game defined in $\Omega^{(N)}$: The first one is the convergence of the representative path $\hat{X}_{it}^{(N)}$, the second one is the convergence of the empirical measure $\rho(\hat{X}_{t}^{(N)})$, while the last one is the $t$-uniform convergence of the empirical measure $\rho(\hat{X}_{t}^{(N)})$. To be precise, we shall assume the following throughout the paper:

Note that the equilibrium path $\hat{X}_{t}^{(N)}=(\hat{X}_{it}^{(N)}:i=1,2,\ldots,N)$ is a vector-valued stochastic process. Due to the Assumption 1, the game is invariant to index reshuffling of $N$ players and the elements in $(\hat{X}_{it}^{(N)}:i=1,2,\ldots,N)$ have identical distributions, but they are not independent of each other.

So, the first question on the representative path is indeed about $\hat{X}_{1t}^{(N)}$ in $\Omega^{(N)}$ and we are interested in how fast it converges to $\hat{X}_{t}$ in $\Omega$ in distribution:

• (Q1)

  The $\mathbb{W}_{p}$-convergence rate of the representative equilibrium path,

  $$\mathbb{W}_{p}\left(\mathcal{L}\left(\hat{X}_{1t}^{(N)}\right),\mathcal{L}\left(\hat{X}_{t}\right)\right)=O\left(N^{-?}\right).$$

The second question is about the convergence of the empirical measure $\rho(\hat{X}_{t}^{(N)})$ of the $N$-player game defined by

$$\rho\left(\hat{X}_{t}^{(N)}\right)=\frac{1}{N}\sum_{i=1}^{N}\delta_{\hat{X}_{it}^{(N)}}.$$

We are interested in how fast this converges to the MFG equilibrium measure given by

$$\hat{m}_{t}=\mathcal{L}\left(\left.\hat{X}_{t}\right|\tilde{\mathcal{F}}_{t}\right),\quad\forall t\in(0,T].$$

• (Q2’)

  The $\mathbb{W}_{p}$-convergence rate of empirical measures,

  $$\mathbb{W}_{p}\left(\rho\left(\hat{X}_{t}^{(N)}\right),\mathcal{L}\left(\left.\hat{X}_{t}\right|\tilde{\mathcal{F}}_{t}\right)\right)=O\left(N^{-?}\right).$$

Note that the left-hand side of the above equality is a random quantity and one shall be more precise about what the Big $O$ notation means in this context. Indeed, by the definition of the empirical measure, $\rho(\hat{X}_{t}^{(N)})$ is a random distribution measurable by $\sigma$-algebra generated by the random vector $\hat{X}_{t}^{(N)}$. On the other hand, $\mathcal{L}(\hat{X}_{t}\rvert\tilde{\mathcal{F}}_{t})$ is a random distribution measurable by the $\sigma$-algebra $\tilde{\mathcal{F}}_{t}$. Therefore, from the construction of the product probability space $\Omega^{(N)}$ in Section 2.2, both random distributions $\rho(\hat{X}_{t}^{(N)})$ and $\mathcal{L}(\hat{X}_{t}\rvert\tilde{\mathcal{F}}_{t})$ are measurable with respect to $\mathcal{F}^{(N)}_{t}=\bar{\mathcal{F}}_{t}^{(N)}\otimes\tilde{\mathcal{F}}_{t}$. Consequently, $\mathbb{W}_{p}(\rho(\hat{X}_{t}^{(N)}),\mathcal{L}(\hat{X}_{t}\rvert\tilde{\mathcal{F}}_{t}))$ is a random variable in the probability space $(\Omega^{(N)},\mathcal{F}^{(N)},\mathbb{P}^{(N)})$ and we will focus on a version of (Q2’) in the $L^{p}$ sense:

• (Q2)

  The $\mathbb{W}_{p}$-convergence rate of empirical measures in $L^{p}$ sense for each $t\in[0,T]$,

  $$\left(\mathbb{E}\left[\mathbb{W}_{p}^{p}\left(\rho\left(\hat{X}_{t}^{(N)}\right),\mathcal{L}\left(\left.\hat{X}_{t}\right|\tilde{\mathcal{F}}_{t}\right)\right)\right]\right)^{\frac{1}{p}}=O\left(N^{-?}\right).$$

In addition, we also study the following related question:

• (Q3)

  The $t$-uniform $\mathbb{W}_{p}$-convergence rate of empirical measures in $L^{p}$ sense,

  $$\left(\mathbb{E}\left[\sup_{0\leq t\leq T}\mathbb{W}_{p}^{p}\left(\rho\left(\hat{X}_{t}^{(N)}\right),\mathcal{L}\left(\left.\hat{X}_{t}\right|\tilde{\mathcal{F}}_{t}\right)\right)\right]\right)^{\frac{1}{p}}=O\left(N^{-?}\right).$$

In this paper, we will study the above three questions (Q1), (Q2), and (Q3) in the framework of LQG structure with Brownian motion as a common noise with the following function $F$ in the cost functional (2).

The main result of this paper is presented below. Let us recall that $q$ denotes the parameter defined in Assumption 1.

We would like to provide some additional remarks on our main result. Firstly, the cost function $F$ defined in (6) applies to the running cost for the $i$-th player in the $N$-player game, and it takes the form:

$$F\left(X_{it}^{(N)},\rho\left(X_{t}^{(N)}\right)\right)=\frac{k}{N}\sum_{j=1}^{N}\left(X_{it}^{(N)}-X_{jt}^{(N)}\right)^{2}.$$

(9)

Interestingly, if $k<0$, although $F$ does satisfy the Lasry-Lions monotonicity ([2]) as demonstrated in Appendix 6.1 of [18], there is no global solution for MFG due to the concavity in $x$. On the contrary, when $k>0$, $F$ satisfies the displacement monotonicity proposed in [11] as shown by the following derivation:

$$\mathbb{E}\left[(F_{x}(X_{1},\mathcal{L}(X_{1}))-F_{x}(X_{2},\mathcal{L}(X_{2})))(X_{1}-X_{2})\right]=2k\left(\mathbb{E}\left[(X_{1}-X_{2})^{2}\right]-\left(\mathbb{E}[X_{1}-X_{2}]\right)^{2}\right)\geq 0.$$

We first recall the convergence rate of empirical measures of i.i.d. sequence provided in Theorem 1 of [10] and Theorem 5.8 of [4].

Next, we give the definition of some notations that will be used in the following part. Denote $C_{b}(\mathbb{R}^{d})$ to be the collection of bounded and continuous functions on $\mathbb{R}^{d}$, and let $C^{1}_{b}(\mathbb{R}^{d})\subset C_{b}(\mathbb{R}^{d})$ be the space of functions on $\mathbb{R}^{d}$ whose first order derivative is also bounded and continuous.

We are going to apply lemmas from the previous subsection to study the convergence of empirical measures of a sequence with a common noise in the following sense.

By this definition, a sequence with a common noise is i.i.d. if and only if $\beta$ is a deterministic constant.

The first question is

• (Qa)

  In Example 1, where does $\rho^{N}(X)$ converge to?

For any test function $\phi\in C_{b}(\mathbb{R})$,

$$\langle\phi,\rho^{N}(X)\rangle\displaystyle=\frac{1}{N}\sum_{i=1}^{N}\phi(X_{i})=\frac{1}{N}\sum_{i=1}^{N}\phi(\gamma_{i}+\sigma\alpha_{i}+\beta).$$

Since $\beta$ is independent to $(\alpha_{i},\gamma_{i})$, by Example 4.1.5 of [9] together with the Law of Large Numbers, we have

$$\frac{1}{N}\sum_{i=1}^{N}\phi(\gamma_{i}+\sigma\alpha_{i}+c)\to\mathbb{E}[\phi(\gamma_{1}+\sigma\alpha_{1}+c)]=\mathbb{E}[\phi(\gamma_{1}+\sigma\alpha_{1}+\beta)|\beta=c],\quad\forall c\in\mathbb{R}.$$

Therefore, we conclude that

	$\displaystyle\langle\phi,\rho^{N}(X)\rangle$	$\displaystyle\to\mathbb{E}[\phi(\gamma_{1}+\sigma\alpha_{1}+\beta)|\beta],\quad\beta-a.s.$
		$\displaystyle=\langle\phi,\mathcal{L}(\gamma_{1}+\sigma\alpha_{1}+\beta|\beta)\rangle,\quad\beta-a.s.$

Hence, the answer for the (Qa) is

• •

  $\rho^{N}(X)\Rightarrow\mathcal{L}(X_{1}|\beta)$, $\beta$-a.s. More precisely, since all random variables are square-integrable, the weak convergence implies, for all $p\in[1,2]$,

  $$\mathbb{W}_{p}\left(\rho^{N}(X),\mathcal{L}\left(X_{1}|\beta\right)\right)\to 0,\quad\beta-a.s.$$

The next question is

• (Qb)

  In Example 1, what’s the convergence rate in the sense $\mathbb{E}\left[\mathbb{W}_{p}^{p}\left(\rho^{N}(X),\mathcal{L}\left(X_{1}|\beta\right)\right)\right]$?

Since $\beta$ is independent to $\gamma_{1}+\sigma\alpha_{1}$, by Example 4.1.5 of [9], we have

$$\mathbb{E}[\phi(\gamma_{1}+\sigma\alpha_{1}+\beta)|\beta=c]=\mathbb{E}[\phi(\gamma_{1}+\sigma\alpha_{1}+c)],\quad\forall\phi\in C_{b}(\mathbb{R}),c\in\mathbb{R},$$

or equivalently, if one takes $c=\beta(\omega)$,

$$\mathcal{L}(X_{1}|\beta)(\omega)=\mathcal{L}(\gamma_{1}+\sigma\alpha_{1}+\beta|\beta)(\omega)=\mathcal{L}(\gamma_{1}+\sigma\alpha_{1}+c).$$

On the other hand, with $c=\beta(\omega)$,

$$\rho^{N}(X)(\omega)=\rho^{N}(X(\omega))=\frac{1}{N}\sum_{i=1}^{N}\delta_{\gamma_{i}(\omega)+\sigma\alpha_{i}(\omega)+c}.$$

From the above two identities, with $c=\beta(\omega)$, we can write

$$\mathbb{W}_{p}\left(\rho^{N}(X)(\omega),\mathcal{L}(X_{1}|\beta=c)(\omega)\right)=\mathbb{W}_{p}\left(\frac{1}{N}\sum_{i=1}^{N}\delta_{\gamma_{i}(\omega)+\sigma\alpha_{i}(\omega)+c},\mathcal{L}(\gamma_{1}+\sigma\alpha_{1}+c)\right).$$

(15)

Now we can conclude (Qb) in the next lemma.