Graphon Mean Field Games and the GMFG Equations

Peter E. Caines Department of Electrical and Computer Engineering, McGill University, Montreal, QC, Canada peterc@cim.mcgill.ca and Minyi Huang School of Mathematics and Statistics, Carleton University, Ottawa, ON, Canada mhuang@math.carleton.ca

(Date: Aug 24, 2020; revised Apr 12, 2021, Jun 15, 2021, Dec 28, 2021.)

Abstract.

The emergence of the graphon theory of large networks and their infinite limits has enabled the formulation of a theory of the centralized control of dynamical systems distributed on asymptotically infinite networks [16, 19]. Furthermore, the study of the decentralized control of such systems was initiated in [6, 7], where Graphon Mean Field Games (GMFG) and the GMFG equations were formulated for the analysis of non-cooperative dynamic games on unbounded networks. In that work, existence and uniqueness results were introduced for the GMFG equations, together with an $\epsilon$ -Nash theory for GMFG systems which relates infinite population equilibria on infinite networks to finite population equilibria on finite networks. Those results are rigorously established in this paper.

Key words and phrases:

Mean field games, networks, graphons

2020 Mathematics Subject Classification:

49N80, 91A16, 91A43, 93E20

This work was supported by NSERC and AFOSR (P. E. Caines) and NSERC (M. Huang).

1. Introduction

One response to the problems arising in the analysis of systems of great complexity is to pass to an appropriately formulated infinite limit. This approach has a distinguished history since it is the conceptual principle underlying the celebrated Boltzmann Equation of statistical mechanics and that of the fundamental Navier-Stokes equation of fluid mechanics (see e.g. [38, 22, 14, 15]). Similarly the Fokker-Planck-Kolmogorov (FPK) equation for the macroscopic flow of probabilities [12, 27] is used to describe a vast range of phenomena which at a micro or mezzo level are modelled via the random interactions of discrete entities.

The work in this paper is formulated within two recent theories which were developed with an analogous motive to that above, namely Mean Field Game (MFG) theory for the analysis of equilibria in very large populations of non-cooperative agents (see [25, 23, 30, 31, 9, 10, 8]), and the graphon theory of the infinite limits of graphs and networks (see [33, 2, 3, 4, 32]).

A mathematically rigorous study of MFG systems with state values in finite graphs is provided in [21], and MFG systems where the agent subsystems are defined at the nodes (vertices) of finite random Erdös-Rényi graphs are treated in [11]. The system behaviour in [21] is subject to a fixed underlying network. The random graphs in [11] have unbounded growth but do not create spatial distinction of the agents due to symmetry properties of the interactions. However, graphon theory gives a rigorous formulation of the notion of limits for infinite sequences of networks of increasing size, and the first application of graphon theory in dynamics appears to be in the work of Medvedev [34, 35], and Kaliuzhnyi-Verbovetskyi and Medvedev [26]. The law of large numbers for graphon mean field systems is proven in [1] as a generalization of results for standard interacting particle systems. Furthermore, the work in [39] derives the McKean-Vlasov limit for a network of agents described by delay stochastic differential equations that are coupled by randomly generated connections.

The first applications of graphon theory in systems and control theory are those in [17, 18, 16, 19, 20] which treat the centralized and distributed control of arbitrarily large networks of linear dynamical control systems for which a direct solution would be intractable. Approximate control is achieved by solving control problems on the infinite limit graphon and then applying control laws derived from those solutions on the finite network of interest. The analogy with the strategies for finding feedback laws resulting in $\epsilon$ -Nash equilibria in the MFG framework is obvious. In this connection we note that work on static game theoretic equilibria for infinite populations on graphons was reported in [37].

A natural framework for the formulation of game theoretic problems involving large populations of agents distributed over large networks is given by Mean Field Game theory defined on graphons. The resulting basic idea and the associated fundamental equations for what we term Graphon Mean Field Game (GMFG) systems and the GMFG equations are the subject of the current paper and its predecessors [6, 7]. The GMFG equations are of significant generality since they permit the study, in the limit, of both dense and sparse, infinite networks of non-cooperative dynamical agents. Moreover the classical MFG equations are retrieved as a special case. We observe that an early analysis of linear quadratic (LQ) models in mean field games on networks with non-uniform edge weightings can be found in [24]. However, in that work there was no application of graphon theory, and in the uniform system parameter case there is one agent per node and a single mean field, whereas in the present work there is a subpopulation with its own mean field at each node.

The basic $\epsilon$ -Nash equilibrium result in MFG theory and its corresponding form in GMFG theory are vital for the application of MFG derived control laws. This is the case since the solution of the MFG and GMFG equations is necessarily simpler than the effectively intractable task of finding the solution to the game problems for the large finite population systems. Indeed, this was one of the original motives for the creation of MFG theory and it is a basic feature of graphon systems control theory [17].

The paper is organized as follows. Section 2 provides preliminary materials on graphons. Section 3 introduces the GMFG equation system and proves the existence and uniqueness of a solution. For the decentralized strategies determined by the GMFG equations, an $\epsilon$ -Nash equilibrium theorem is proven in Section 4. The GMFG equations are illustrated by an LQ example in Section 5.

Table 1. Notation

$G_{k}$	the $k$ -th graph in a sequence of graphs
$g^{k}$	weights of $G_{k}$ as a step function
$M_{k}$	the number of nodes in $G_{k}$
${\mathcal{C}}_{i}$	the cluster of agents residing at node $i$ of $G_{k}$
${\mathcal{C}}(i)$	the cluster that agent $i$ belongs to
$I_{i}^{}$ , $I^{}(i)$	the midpoint of an interval of length $1/M_{k}$
$g$	the graphon function
$\mu_{\alpha}(t)$	the local mean field generated by agents at vertex $\alpha\in[0,1]$
$\mu_{G}(t)$	an ensemble of local mean fields $(\mu_{\alpha}(t))_{0\leq\alpha\leq 1}$
${\mathcal{M}}_{[0,T]}$	a class of $\mu_{G}(\cdot)$ satisfying a Hölder continuity condition
$C_{T}$	the space of continuous functions on $[0,T]$
${\mathcal{F}}_{T}$	$\sigma$ -algebra induced by cylindrical sets in $C_{T}$
$(C_{T},{\mathcal{F}}_{T},m_{\alpha})$	probability measure space for the path space at vertex $\alpha$
${\bf M}_{T}$	the set of probability measures on $(C_{T},{\mathcal{F}}_{T})$
$D_{T}$	Wasserstein metric on ${\bf M}_{T}$
${\bf M}_{T}^{G}$	the product space $\prod_{\alpha\in[0,1]}{\bf M}_{T}$
${\bf M}_{T}^{G0}$ , ${\bf M}_{T}^{G1}$	subsets of ${\bf M}_{T}^{G}$
$m_{G}$	an ensemble of measures $(m_{\alpha})_{0\leq\alpha\leq 1}\in{\bf M}_{T}^{G}$
${\rm Proj}_{\alpha}(m_{G})$	the component $m_{\alpha}$ at vertex $\alpha$
${\rm Marg}_{t}(m_{\alpha})$	the time $t$ -marginal of $m_{\alpha}$
$x_{\alpha}$	the state of a generic agent at vertex $\alpha\in[0,1]$
$w_{\alpha}$	a generic standard Brownian motion at vertex $\alpha$
$\varphi(t,x_{\alpha}\|\mu_{G}(\cdot);g_{\alpha})$	the best response at vertex $\alpha$ with $\mu_{G}(\cdot)$ given by the GMFG system;
	abbreviated as $\varphi(t,x_{\alpha},g_{\alpha})$ or $\varphi_{\alpha}$
$\phi(t,x_{\alpha}\|\mu_{G}(\cdot);g_{\alpha})$	the best response at vertex $\alpha$ with respect to an arbitrary $\mu_{G}(\cdot)$ ;
	abbreviated as $\phi_{\alpha}(t,x_{\alpha}\|\mu_{G}(\cdot))$ or $\phi_{\alpha}$

2. The Concept of a Graphon

The basic idea of the theory of graphons is that the edge structure of each finite cardinality network is represented by a step function density on the unit square in $\mathbb{R}^{2}$ on which the so-called cut norm and cut metrics are defined. The set of finite graphs endowed with the cut metric then gives rise to a metric space, and the completion of this space is the space of graphons. Let $\mathbf{G_{0}^{sp}}$ denote the linear space of bounded symmetric Lebesgue measurable functions $W:[0,1]^{2}\rightarrow\mathbb{R}$ , which are called kernels. The space $\mathbf{G^{sp}}$ of graphons is a subset of $\mathbf{G_{0}^{sp}}$ and consists of kernels $W:[0,1]^{2}\rightarrow[0,1]$ which can be interpreted as weighted graphs on the vertex set $[0,1]$ . We note that functions $W\in\mathbf{G^{sp}}$ taking values in finite sets satisfy this definition and so, in particular, graphons are defined on finite graphs.

The cut norm of a kernel $W\in\mathbf{G_{0}^{sp}}$ then has the expression:

\|W\|_{\Box}=\sup_{M,T\subset[0,1]}\Big{|}\int_{M\times T}W(x,y)dxdy\Big{|}

with the supremum taking over all measurable subsets $M$ and $T$ of $[0,1]$ . Denote the set of measure preserving bijections $[0,1]\rightarrow[0,1]$ by $S_{[0,1]}$ . The cut metric between two graphons $V$ and $W$ is then given by $\delta_{\Box}(W,V)=\inf_{\phi\in S_{[0,1]}}\|W^{\phi}-V\|_{\Box}$ , where $W^{\phi}(x,y)\coloneqq W(\phi(x),\phi(y))$ and any pair of graphons at zero distance are identified with each other. The space $(\mathbf{G^{sp}},\delta_{\Box})$ is compact in the topology given by the cut metric [32]. Furthermore, sets in $(\mathbf{G^{sp}},\delta_{\Box})$ which are compact with respect to the $L^{2}$ metric are compact with respect to the cut metric. Since $\mathbf{G^{sp}}$ is compact in the cut metric all sequences of graphons have subsequential limits.

In this paper, we start with the modeling of the game of a finite population based on a finite graph. Specifically, the population resides on a weighted finite graph $G_{k}$ with a set of nodes (or vertices) ${\mathcal{V}}_{k}=\{1,\ldots,M_{k}\}$ and weights $g^{k}_{ij}\in[0,1]$ for $(i,j)\in{\mathcal{V}}_{k}\times{\mathcal{V}}_{k}$ , where a value $g_{ii}^{k}$ is assigned in the case $i=j$ . We call $g_{i}^{k}\coloneqq(g_{i1}^{k},\ldots,g_{iM_{k}}^{k})$ a section of $g^{k}$ at $i$ . Each node $l$ is occupied by a set of agents which is called a cluster of the population and hence the number of clusters is $M_{k}$ . We list the clusters as $\mathcal{C}_{1},\ldots,\mathcal{C}_{M_{k}}$ . Without loss of generality, we assume the $l$ th cluster occupies node $l$ . Let $\mathcal{C}(i)$ denote the cluster that agent $i$ belongs to. So $i\in\mathcal{C}(i)$ . Our further analysis in the paper is based on the convergence of $g^{k}$ to a graphon limit $g$ . We may naturally identify $(g^{k}_{ij})_{1\leq i,j\leq M_{k}}$ with a graphon $g^{k}(\alpha,\beta)$ as a step function defined on $[0,1]\times[0,1]$ (see [32]). However, convergence in the cut norm or the cut metric is inadequate for the analysis in this paper as it does not capture sufficiently strong sectional information of the difference $g^{k}-g$ . We will adopt a different convergence notion strengthening the sectional requirement as in assumption (H11) below. To indicate its arguments, we may write $g(\alpha,\beta)$ or alternatively $g_{\alpha,\beta}$ . We define the section of $g$ at $\alpha$ by $g_{\alpha}:\beta\mapsto g_{\alpha,\beta}$ , $\beta\in[0,1]$ .

Since clusters $\mathcal{C}_{i_{1}}$ and $\mathcal{C}_{i_{2}}$ reside on nodes $i_{1}$ and $i_{2}$ of $G_{k}$ , respectively, we define $g^{k}_{\mathcal{C}_{i_{1}}\mathcal{C}_{i_{2}}}=g^{k}_{i_{1}i_{2}}$ . Similarly, we define the section $g^{k}_{\mathcal{C}_{i}}=g^{k}_{i}$ .

We partition $[0,1]$ into $M_{k}$ subintervals of equal length. Here $I_{l}^{k}=[(l-1)/M_{k},l/M_{k}]$ for $1\leq l\leq M_{k}$ . When it is clear from the context, we omit the superscript $k$ and write $I_{l}$ . To relate the clusters of agents to the vertex set $[0,1]$ , we let the cluster $\mathcal{C}_{l}$ correspond to $I_{l}$ .

Throughout this paper, $C,C_{0},C_{1},\ldots$ denote generic constants, which do not depend on the graph index $k$ and population size $N$ and may vary from place to place.

3. Graphon MFG Systems and the GMFG Equations

3.1. The Standard MFG Model and Its Graphon Generalization

In the diffusion based models of large population games the state evolution of a collection of $N$ agents ${\mathcal{A}}_{i},1\leq i\leq N<\infty,$ is specified by a set of $N$ controlled stochastic differential equations (SDEs). A simplified form of the general case is given by the following set of controlled SDEs which for each agent ${\mathcal{A}}_{i}$ includes state coupling with all other agents:

(3.1)

dx_{i}(t)=\frac{1}{N}\sum_{j=1}^{N}f(x_{i}(t),u_{i}(t),x_{j}(t))dt+\sigma dw_{i}(t),\\

where $x_{i}\in\mathbb{R}^{n}$ is the state, $u_{i}\in\mathbb{R}^{n_{u}}$ the control input, and $w_{i}\in{\mathbb{R}}^{n_{w}}$ a standard Brownian motion, and where $\{w_{i},1\leq i\leq N\}$ are independent processes. For simplicity, all collections of system initial conditions are taken to be independent and have finite second moment. The cost of agent ${\mathcal{A}}_{i}$ is given by

(3.2)

J_{i}^{N}(u_{i},u_{-i})=E\int_{0}^{T}\frac{1}{N}\sum_{j=1}^{N}l(x_{i}(t),u_{i}(t),x_{j}(t))dt,

where $l(\cdot)$ is the pairwise running cost, and $u_{-i}$ denotes the controls of all other agents.

The dynamics of a generic agent ${\mathcal{A}}_{i}$ in the infinite population limit of this system is then described by the controlled McKean-Vlasov (MV) equation

(3.3)

dx_{i}=f[x_{i},u_{i},\mu_{t}]dt+\sigma dw_{i},\quad 0\leq t\leq T,

where $\mu_{t}$ is the distribution of $x_{i}(t)$ , $f[x,u,\mu_{t}]\coloneqq\int_{\mathbb{R}^{n}}f(x,u,y)\mu_{t}(dy)$ and where the initial distribution $\mu^{x}_{0}$ of $x_{i}(0)$ is specified. Setting $l[x,u,\mu_{t}]=\int_{\mathbb{R}^{n}}l(x,u,y)\mu_{t}(dy)$ , the corresponding infinite population cost for ${\mathcal{A}}_{i}$ takes the form

(3.4)

J_{i}(u_{i};\mu(\cdot))\coloneqq E\int_{0}^{T}l[x_{i}(t),u_{i}(t),\mu_{t}]dt.

For notational simplicity, we present the graphon MFG framework with scalar individual states and controls, i.e., $n=n_{u}=n_{w}=1$ . Its extension to the vector case is evident.

Now we consider a finite population distributed over the finite graph $G_{k}$ . Let $\mathbold{x}_{G_{k}}=\bigoplus_{l=1}^{M_{k}}\{x_{i}|i\in\mathcal{C}_{l}\}$ denote the states of all agents in the total set of clusters of the population. This gives a total of $N=\sum_{l=1}^{M_{k}}|\mathcal{C}_{l}|$ individual states. The key feature of the graphon MFG construction beyond the standard MFG scheme is that at any agent in a network the averaged dynamics (3.1) and cost function (3.2) decompose into averages of subpopulations distributed at that agent’s neighboring nodes plus an average term for the local cluster. In the limit, the summed subpopulation averages are given by an integral over the local mean fields of the neighbouring agents.

For ${\mathcal{A}}_{i}$ in the cluster $\mathcal{C}(i)$ , two coupling terms in the dynamics take the form

(3.5)		$\displaystyle f_{0}(x_{i},u_{i},{\mathcal{C}}(i))=\frac{1}{\|{\mathcal{C}}(i)\|}\sum_{j\in{\mathcal{C}}(i)}f_{0}(x_{i},u_{i},x_{j}),$
(3.6)		$\displaystyle f_{G_{k}}(x_{i},u_{i},g^{k}_{\mathcal{C}(i)})=\frac{1}{M_{k}}\sum_{l=1}^{M_{k}}g^{k}_{\mathcal{C}(i)\mathcal{C}_{l}}\frac{1}{\|\mathcal{C}_{l}\|}\sum_{j\in\mathcal{C}_{l}}f(x_{i},u_{i},x_{j}).$

They model intra- and inter-cluster couplings, respectively. The specification of $f_{G_{k}}$ relies on the sectional information $g^{k}_{\mathcal{C}(i)\bullet}$ . Concerning the coupling structure in (3.6) we observe that with respect to ${\mathcal{A}}_{i}$ , all individuals residing in cluster $\mathcal{C}_{l}$ are symmetric and their state average generates the overall impact of that cluster on ${\mathcal{A}}_{i}$ mediated by the graphon weighting $g^{k}_{\mathcal{C}(i)\bullet}$ . The two coupling terms are combined additively resulting in the local dynamics

\tilde{f}_{G_{k}}(x_{i},u_{i},g^{k}_{\mathcal{C}(i)})=f_{0}(x_{i},u_{i},{\mathcal{C}}(i))+f_{G_{k}}(x_{i},u_{i},g^{k}_{\mathcal{C}(i)}).

Note that ${\mathcal{A}}_{i}$ interacts with the overall population through a function of the complete system state ${\mathbold x}_{G_{k}}$ and the cluster sizes. These details shall be suppressed in this paper and we only indicate the graph $G_{k}$ and the section $g^{k}_{\mathcal{C}(i)}$ . The state process of ${\mathcal{A}}_{i}$ is then given by the stochastic differential equation

\displaystyle dx_{i}(t)=\tilde{f}_{G_{k}}(x_{i},u_{i},g^{k}_{\mathcal{C}(i)})dt+\sigma dw_{i},\quad 1\leq i\leq N,

where $\sigma>0$ and the initial states $\{x_{i}(0),1\leq i\leq N\}$ are i.i.d. with distribution $\mu^{x}_{0}\in{\mathcal{P}}_{1}(\mathbb{R})$ , the set of probability measures on $\mathbb{R}$ with finite mean.

The limit of the two dynamic coupling terms of an agent at a node $\alpha$ (called an $\alpha$ -agent), as the number of nodes of the graph $G_{k}$ and the subpopulation at each node tend to infinity, is described by the following expressions:

(3.7)		$\displaystyle{f}_{0}[x_{\alpha},u_{\alpha},{\mu}_{\alpha}]\coloneqq\int_{{\mathbb{R}}}f_{0}(x_{\alpha},u_{\alpha},z)\mu_{\alpha}(dz),$
(3.8)		$\displaystyle{f}[x_{\alpha},u_{\alpha},{\mu}_{G};g_{\alpha}]\coloneqq\int_{0}^{1}\int_{{\mathbb{R}}}f(x_{\alpha},u_{\alpha},z)g(\alpha,\beta)\mu_{\beta}(dz)d\beta,$

which give the complete local graphon dynamics via

(3.9)

\widetilde{f}[x_{\alpha},u_{\alpha},{\mu}_{G};g_{\alpha}]\coloneqq f_{0}[x_{\alpha},u_{\alpha},\mu_{\alpha}]+{f}[x_{\alpha},u_{\alpha},{\mu}_{G};g_{\alpha}].

We call $\mu_{\beta}$ the local mean field at node $\beta$ , which is interpreted as the limit of the empirical distributions of agents at node $\beta$ . And $\mu_{G}=\{\mu_{\beta},0\leq\beta\leq 1\}$ is the ensemble of local mean fields. Due to the integration with respect to $\beta$ , the dependence of $\widetilde{f}$ on the graphon limit $g$ is through the section $g_{\alpha}$ . Since $\mu_{G}$ contains $\mu_{\alpha}$ , we do not list $\mu_{\alpha}$ as an argument of $\widetilde{f}$ .

Parallel to the standard MFG case, in the graphon case the stochastic differential equation

(3.10)			$\displaystyle{\text{[MV-SDE]}}(\alpha)\quad dx_{\alpha}(t)=\widetilde{f}[x_{\alpha}(t),u_{\alpha}(t),{\mu}_{G}(t);g_{\alpha}]dt+\sigma dw_{\alpha}(t),$
(3.10)			$\displaystyle\quad 0\leq t\leq T,\quad\alpha\in[0,1],$

generalizes the standard controlled MV equation (3.3). We note that in a parallel development of graphon based stochastic dynamical populations [1] the system disturbance intensity $\sigma$ is also a function of graphon weighted state functions at other clusters. For simplicity, we consider a constant $\sigma$ and our analysis may be generalized to the case of a state and mean field dependent diffusion term. Similarly, for simplicity our dynamics and cost do not include a separate parametrization by $\alpha$ .

Analogously, in the GMFG case, we define the cost coupling terms for ${\mathcal{A}}_{i}$ to be

	$\displaystyle l_{0}(x_{i},u_{i},\mathcal{C}(i))=\frac{1}{\|\mathcal{C}(i)\|}\sum_{j\in\mathcal{C}(i)}l_{0}(x_{i},u_{i},x_{j}),$
	$\displaystyle l_{G_{k}}(x_{i},u_{i},g^{k}_{\mathcal{C}(i)})=\frac{1}{M_{k}}\sum_{l=1}^{M_{k}}g^{k}_{\mathcal{C}(i)\mathcal{C}_{l}}\frac{1}{\|\mathcal{C}_{l}\|}\sum_{j\in\mathcal{C}_{l}}l(x_{i},u_{i},x_{j}).$

Define $\tilde{l}_{G_{k}}(x_{i},u_{i},g^{k}_{\mathcal{C}(i)})=l_{0}(x_{i},u_{i},\mathcal{C}(i))+l_{G_{k}}(x_{i},u_{i},g^{k}_{\mathcal{C}(i)}).$ The cost of ${\mathcal{A}}_{i}$ in a finite population on a finite graph $G_{k}$ is given in the form

(3.11)

\displaystyle J_{i}=E\int_{0}^{T}\tilde{l}_{G_{k}}(x_{i},u_{i},g^{k}_{\mathcal{C}(i)})dt.

Denote

	$\displaystyle l_{0}[x_{\alpha},u_{\alpha},\mu_{\alpha}]=\int_{\mathbb{R}}l_{0}(x_{\alpha},u_{\alpha},z)\mu_{\alpha}(dz),$
	$\displaystyle l[x_{\alpha},u_{\alpha},\mu_{G};g_{\alpha}]=\int_{0}^{1}\int_{\mathbb{R}}l(x_{\alpha},u_{\alpha},z)g(\alpha,\beta)\mu_{\beta}(dz)d\beta,$
	$\displaystyle\widetilde{l}[x_{\alpha},u_{\alpha},\mu_{G};g_{\alpha}]=l_{0}[x_{\alpha},u_{\alpha},\mu_{\alpha}]+l[x_{\alpha},u_{\alpha},\mu_{G};g_{\alpha}].$

Then in the infinite population graphon case, the $\alpha$ -agent has the cost function given by

(3.12)

\displaystyle J_{\alpha}(u_{\alpha};\mu_{G}(\cdot))=E\int_{0}^{T}\widetilde{l}[x_{\alpha}(t),u_{\alpha}(t),\mu_{G}(t);g_{\alpha}]dt.

3.2. The Graphon MFG Model and Its Equations

In this section the standard MFG equations (see e.g. [5, 8]) will be generalized so that they subsume the standard (implicitly uniform totally connected) dense network case and cover the fully general graphon limit network case. Specifically, agent ${\mathcal{A}}_{i}$ in a population of $N$ agents will be located at the $l$ th node in an $M_{k}$ node network (identified with its graphon) and in the infinite population graphon limit that node will be taken to map to $\alpha\in[0,1]$ . It is important to note here that although the limit network is assumed dense it is not assumed to be uniformly totally connected; indeed, the connection structure of the infinite network is represented precisely by its graphon $g(\alpha,\beta)$ , $0\leq\alpha,\beta\leq 1.$

The generalized Graphon MFG scheme below on $[0,T]$ is given for each $\alpha$ by (i) the Hamilton-Jacobi-Bellman (HJB) equation generating the value function $V^{\alpha}$ when all other agents’ control laws and the ensemble $\mu_{G}$ of local mean fields are given, (ii) the FPK equation generating the local mean field $\mu_{\alpha}$ given $\mu_{G}$ , and (iii) the specification of the best response (BR) feedback law.

Suppressing the time index on the measures for simplicity of notation, we have the Graphon Mean Field Game (GMFG) equations:

	$\displaystyle{\text{[HJB]}}(\alpha)\quad-\frac{{\partial}V^{\alpha}(t,x)}{{\partial}t}=\inf_{u\in U}\bigg{\{}\widetilde{f}[x,u,{\mu}_{G};g_{\alpha}]\frac{{\partial}V^{\alpha}(t,x)}{{\partial}x}$
(3.13)		$\displaystyle\hskip 113.81102pt+\widetilde{l}[x,u,{\mu}_{G};g_{\alpha}]\bigg{\}}+\frac{\sigma^{2}}{2}\frac{{\partial}^{2}V^{\alpha}(t,x)}{{\partial}x^{2}},$
	$\displaystyle V^{\alpha}(T,x)=0,\quad(t,x)\in[0,T]\times\mathbb{R},\quad\alpha\in[0,1],$

	$\displaystyle{\text{[FPK]}}(\alpha)\quad\frac{{\partial}p_{\alpha}(t,x)}{{\partial}t}=$	$\displaystyle-\frac{{\partial}\{\widetilde{f}[x,u^{0},\mu_{G};g_{\alpha}]p_{\alpha}(t,x)\}}{{\partial}x}$
(3.14)			$\displaystyle+\frac{\sigma^{2}}{2}\frac{{\partial}^{2}p_{\alpha}(t,x)}{{\partial}x^{2}},$

\displaystyle{\text{[BR]}}(\alpha)\quad u^{0}

\displaystyle\coloneqq\varphi(t,x|{\mu}_{G};g_{\alpha}).

Here $p_{\alpha}(t,x)$ with initial condition $p_{\alpha}(0)$ is used to denote the density of the measure $\mu_{\alpha}(t)$ whenever a density is assumed to exist. The FPK equation may be replaced by the following closed-loop MV-SDE:

(3.15)

\displaystyle\text{[MV]}(\alpha)\quad dx_{\alpha}(t)=\widetilde{f}[x_{\alpha}(t),\varphi(t,x_{\alpha}(t)|{\mu}_{G};g_{\alpha}),{\mu}_{G}(t);g_{\alpha}]dt+\sigma dw_{\alpha}(t),

where $x_{\alpha}(0)$ has distribution $\mu_{0}^{x}$ . Our subsequent analysis will directly treat the pair $(V^{\alpha}(t,x),\mu_{\alpha}(t))$ , where $\mu_{\alpha}(t)$ is specified as the law of $x_{\alpha}(t)$ in (3.15).

When a solution exists for the GMFG equations, the resulting BR feedback controls depend upon the ensemble $\mu_{G}$ of local mean fields and the individual agent’s state. This is a natural generalization of the standard case. The standard MFG case is simply obtained by setting $g(\alpha,\beta)\equiv 0,0\leq\alpha,\beta\leq 1$ , which totally disconnects the network and results in $\widetilde{f}[x,u,{\mu}_{G};g_{\alpha}]={f}_{0}[x,u,\mu]$ and $\widetilde{l}[x,u,{\mu}_{G};g_{\alpha}]=l_{0}[x,u,\mu]$ [5, 8].

A collection of measures on some measurable space which are indexed by the vertex set $[0,1]$ is called a measure ensemble. Thus, for each fixed $t$ , $\mu_{G}(t)$ is a measure ensemble.

On ${\mathcal{P}}_{1}({\mathbb{R}})$ we endow the Wasserstein metric $W_{1}$ : for any $\mu,\nu\in{\mathcal{P}}_{1}({\mathbb{R}})$ , $W_{1}(\mu,\nu)=\inf_{\widehat{\gamma}}\int|x-y|\widehat{\gamma}(dx,dy)$ , where $\widehat{\gamma}$ is a probability measure on $\mathbb{R}^{2}$ with marginals $\mu,\nu$ .

Let $C([0,1],{\mathcal{P}}_{1}({\mathbb{R}}))$ be the set of measure ensembles $\nu_{G}=(\nu_{\beta})_{\beta\in[0,1]}$ satisfying $\nu_{\beta}\in{\mathcal{P}}_{1}({\mathbb{R}})$ , and $\lim_{\beta^{\prime}\to\beta}W_{1}(\nu_{\beta^{\prime}},\nu_{\beta})=0$ for any $\beta\in[0,1]$ .

In order to analyze the solvability of the GMFG equations, we need to restrict $\mu_{G}(\cdot)$ to a certain class. We say $\{\mu_{G}(t),0\leq t\leq T\}$ is from the admissible set ${\mathcal{M}}_{[0,T]}$ if:

(C1) For each fixed $t$ , $\mu_{G}(t)$ is in $C([0,1],{\mathcal{P}}_{1}({\mathbb{R}}))$ .

(C2) There exists $\eta\in(0,1]$ such that for any bounded and Lipschitz continuous function $\phi$ on $\mathbb{R}$ ,

\sup_{\beta\in[0,1]}\Big{|}\int_{\mathbb{R}}\phi(y)\mu_{\beta}(t_{1},dy)-\int_{\mathbb{R}}\phi(y)\mu_{\beta}(t_{2},dy)\Big{|}\leq C_{h}|t_{1}-t_{2}|^{\eta},

where $C_{h}$ may be selected to depend only on the Lipschitz constant $\mbox{ Lip}(\phi)$ for $\phi$ .

Condition (C1) ensures that integration with respect to $d\beta$ in (3.8) is well defined. Condition (C2) ensures that the drift term in the HJB equation (3.2) has a certain time continuity, which facilitates the subsequent existence analysis of the best response.

3.3. Existence Analysis

We introduce the following assumptions:

(H1) $U$ is a compact set.

(H2) $f_{0}(x,u,y)$ , $f(x,u,y)$ , $l_{0}(x,u,y)$ and $l(x,u,y)$ are continuous and bounded functions on $\mathbb{R}\times U\times\mathbb{R}$ and are Lipschitz continuous in $(x,y)$ , uniformly with respect to $u$ .

(H3) $f_{0}(x,u,y)$ and $f(x,u,y)$ are Lipschitz continuous in $u$ , uniformly with respect to $(x,y)$ .

(H4) For any $q\in\mathbb{R}$ , $\alpha\in[0,1]$ and probability measure ensemble ${\nu}_{G}\in C([0,1],{\mathcal{P}}_{1}({\mathbb{R}}))$ , the set

(3.16)

\displaystyle S_{\alpha}^{\nu_{G}}(x,q)

\displaystyle=\arg\min_{u\in U}\{q(\widetilde{f}[x,u,{\nu}_{G};g_{\alpha}])+\widetilde{l}[x,u,{\nu}_{G};g_{\alpha}]\}

is a singleton, and for any given compact interval ${\mathcal{I}}=[\underline{q},\bar{q}]$ , the resulting $u$ as a function of $(x,q)\in\mathbb{R}\times{\mathcal{I}}$ is Lipschitz continuous in $(x,q)$ , uniformly with respect to ${\nu}_{G}$ and $g_{\alpha}$ , $0\leq\alpha\leq 1$ .

The next two assumptions will be used to ensure that the best responses have continuous dependence on $\alpha$ . In particular, (H5) is a continuity assumption on the graphon function $g(\alpha,\beta)$ . Under (H5), $\widetilde{f}$ and $\widetilde{l}$ have continuity in $\alpha$ .

(H5) For any bounded and measurable function $h(\beta)$ , the function $\int_{0}^{1}g(\alpha,\beta)h(\beta)d\beta$ is continuous in $\alpha\in[0,1]$ .

(H6) For given $\nu_{G}\in C([0,1],{\mathcal{P}}_{1}({\mathbb{R}}))$ , $S_{\alpha}^{\nu_{G}}(x,q)$ is continuous in $(\alpha,x,q)$ .

Although the GMFG equation system only involves $\{\mu_{G}(t),0\leq t\leq T\}$ , which may be viewed as a collection of marginals at different vertices, it is necessary to develop the existence analysis in the underlying probability spaces (see related discussions in [25, p.240]).

We begin by introducing some analytic preliminaries. For the space $C_{T}=C([0,T],\mathbb{R})$ , we specify a $\sigma$ -algebra ${\mathcal{F}}_{T}$ induced by all cylindrical sets of the form $\{x(\cdot)\in C_{T}:x(t_{i})\in B_{i},1\leq i\leq j\mbox{ for some }j\}$ , where $B_{i}$ is a Borel set. Let ${\bf M}_{T}$ denote the space of all probability measures on $(C_{T},{\mathcal{F}}_{T})$ . The canonical process $X$ is defined by $X_{t}(\omega)=\omega_{t}$ for $\omega\in C_{T}$ . On $C_{T}$ , we introduce the metric $\rho(x,y)=\sup_{t}|x(t)-y(t)|\wedge 1$ . Then $(C_{T},\rho)$ is a complete metric space. Based on $\rho$ , we introduce the Wasserstein metric on ${\bf M}_{T}$ . For $m_{1},m_{2}\in{\bf M}_{T}$ , denote

\displaystyle D_{T}(m_{1},m_{2})=\inf_{\widehat{m}}\int_{C_{T}\times C_{T}}\Big{(}\sup_{s\leq T}|X_{s}(\omega_{1})-X_{s}(\omega_{2})|\wedge 1\Big{)}d\widehat{m}(\omega_{1},\omega_{2}),

where $\widehat{m}$ is called a coupling as a probability measure on $(C_{T},{\mathcal{F}}_{T})\times(C_{T},{\mathcal{F}}_{T})$ with the pair of marginals $m_{1}$ and $m_{2}$ , respectively. Then $({\bf M}_{T},D_{T})$ is a complete metric space [41].

We introduce the product of probability measure spaces $\prod_{\alpha\in[0,1]}(C_{T},{\mathcal{F}}_{T},m_{\alpha})$ , where each individual space is interpreted as the path space of the agent at vertex $\alpha$ with a corresponding probability measure $m_{\alpha}$ . Denote the product of spaces of probability measures ${\bf M}_{T}^{G}=\prod_{\alpha\in[0,1]}{\bf M}_{T}.$ An element in ${\bf M}_{T}^{G}$ is a measure ensemble. Given $m_{G}\in{\bf M}_{T}^{G}$ , the projection operator ${\rm Proj}_{\alpha}$ picks out its component $m_{\alpha}$ associated with $\alpha\in[0,1]$ . Let ${\bf M}_{T}^{G0}$ consist of all $(m_{\alpha})_{\alpha\in[0,1]}\in{\bf M}_{T}^{G}$ such that for any $\alpha\in[0,1]$ , $D_{T}(m_{\alpha^{\prime}},m_{\alpha})\to 0$ as $\alpha^{\prime}\to\alpha$ .

For two measure ensembles $m_{G}\coloneqq(m_{\alpha})_{\alpha\in[0,1]}$ and $\bar{m}_{G}\coloneqq(\bar{m}_{\alpha})_{\alpha\in[0,1]}$ in ${\bf M}_{T}^{G}$ , define $d(m_{G},\bar{m}_{G})=\sup_{\alpha\in[0,1]}D_{T}(m_{\alpha},\bar{m}_{\alpha}).$

Lemma 3.1.

$({\bf M}_{T}^{G},d)$ is a complete metric space.

Proof.

If $\{m_{G}^{k},k\geq 1\}$ is a Cauchy sequence in ${\bf M}_{T}^{G}$ , then for each given $\alpha$ , the sequence $\{{\rm Proj}_{\alpha}(m_{G}^{k}),k\geq 1\}$ (of probability measures) is a Cauchy sequence in the complete metric space ${\bf M}_{T}$ and so it contains a limit. This in turn determines a limit in ${\bf M}_{T}^{G}$ . ∎

Given the probability measure $m_{\alpha}\in{\bf M}_{T}$ , we determine the $t$ -marginal $\mu_{\alpha}(t)$ by $\mu_{\alpha}(t,B)=m_{\alpha}(\{x(\cdot)\in C_{T}:x(t)\in B\})$ for any Borel set $B\subset\mathbb{R}$ , and denote the mapping from ${\bf M}_{T}$ to ${\mathcal{P}}(\mathbb{R})$ (the set of probability measures on ${\mathbb{R}}$ ):

(3.17)

\displaystyle\mu_{\alpha}(t)={\rm Marg}_{t}(m_{\alpha}).

Consider the measure ensemble $m_{G}=(m_{\alpha})_{\alpha\in[0,1]}\in{\bf M}_{T}^{G}$ with $\mu_{\alpha}(t)$ given by (3.17). Define the time $t$ marginals by the following mapping

(3.18)

\displaystyle{\rm Marg}_{t}(m_{G})=(\mu_{\alpha}(t))_{\alpha\in[0,1]},

where the right hand side is simply written as $\mu_{G}(t)$ . For a given $t$ , $\mu_{G}(t)$ may be interpreted as a measure valued function defined on the vertex set $[0,1]$ . Further denote the mapping ${\rm Marg}(m_{G})=(\mu_{G}(t))_{t\in[0,T]}=\mu_{G}(\cdot)$ .

Take a fixed

(3.19)

\displaystyle\mu_{G}(\cdot)\in{\mathcal{M}}_{[0,T]}

with its associated Hölder parameter $\eta$ in (C2), and denote

\displaystyle\widetilde{f}_{\alpha}^{*}(t,x,u)=\widetilde{f}[x,u,\mu_{G}(t);g_{\alpha}],\quad\widetilde{l}_{\alpha}^{*}(t,x,u)=\widetilde{l}[x,u,\mu_{G}(t);g_{\alpha}].

Lemma 3.2.

Assume (H1)–(H2). For $h_{\alpha}=\widetilde{f}_{\alpha}^{*}(t,x,u)$ or $\widetilde{l}_{\alpha}^{*}(t,x,u)$ , there exist constants $C$ and $C_{\mu_{G}}$ , where the latter depends on $\mu_{G}(\cdot)$ , such that

	$\displaystyle\sup_{t,u,\alpha}\|h_{\alpha}(t,x,u)-h_{\alpha}(t,y,u)\|\leq C\|x-y\|,$
	$\displaystyle\sup_{x,u,\alpha}\|h_{\alpha}(t,x,u)-h_{\alpha}(s,x,u)\|\leq C_{\mu_{G}}\|t-s\|^{\eta},$

where the supremum is taken over $t\in[0,T]$ , $x\in\mathbb{R}$ , $u\in U$ and $\alpha\in[0,1]$ .

Proof.

The Lipschitz continuity of $\widetilde{f}^{*}_{\alpha}$ with respect to $x$ follows from (H2) and (3.7)–(3.8). For $t_{1},t_{2}\in[0,T]$ , we estimate $|\widetilde{f}[x,u,\mu_{G}(t_{1});g_{\alpha}]-\widetilde{f}[x,u,\mu_{G}(t_{2});g_{\alpha}]|$ by using the Lipschitz condition of $f_{0}$ , $f$ and condition (C2) for ${\mathcal{M}}_{[0,T]}$ . This establishes the Hölder continuity of $\widetilde{f}_{\alpha}^{*}$ in $t$ . The other cases can be similarly checked. ∎

In order to analyze the best response of the $\alpha$ -agent, we introduce the HJB equation

(3.20)

\displaystyle-V^{\alpha}_{t}(t,x)=\inf_{u\in U}\{\widetilde{f}_{\alpha}^{*}(t,x,u)V_{x}^{\alpha}(t,x)+\widetilde{l}_{\alpha}^{*}(t,x,u)\}+\frac{\sigma^{2}}{2}V_{xx}^{\alpha}(t,x),

where $V^{\alpha}(T,0)=0$ . It differs from (3.2) by allowing an arbitrary $\mu_{G}(\cdot)\in{\mathcal{M}}_{[0,T]}$ .

For studying (3.20), we introduce some standard definitions. Denote $Q_{T}=(0,T)\times\mathbb{R}$ , and $\overline{Q}_{T}=[0,T]\times\mathbb{R}$ . Let $C^{1,2}(\overline{Q}_{T})$ (resp., $C^{1,2}(Q_{T})$ ) denote the set of functions with continuous derivatives $v_{t},v_{x},v_{xx}$ on $\overline{Q}_{T}$ (resp., $Q_{T}$ ). Let $C_{b}^{1,2}(\overline{Q}_{T})$ be the set of bounded functions in $C^{1,2}(\overline{Q}_{T})$ , and let the open (or closed) set $Q_{b}$ be a bounded subset of $Q_{T}$ . $W^{1,2}_{\lambda}(Q_{b})$ , $1\leq\lambda<\infty$ , shall denote the Sobolev space consisting of functions $v$ such that each $v$ and its generalized derivatives $v_{t}$ , $v_{x}$ , $v_{xx}$ are in $L^{\lambda}(Q_{b})$ ; further we have the norm

(3.21)

\displaystyle\|v\|_{\lambda,Q_{b}}^{(2)}=\|v\|_{\lambda,Q_{b}}+\|v_{t}\|_{\lambda,Q_{b}}+\|v_{x}\|_{\lambda,Q_{b}}+\|v_{xx}\|_{\lambda,Q_{b}},

where $\|v\|_{\lambda,Q_{b}}=(\int_{Q_{b}}|v(t,x)|^{\lambda}dtdx)^{1/\lambda}$ . Set $|v|_{Q_{b}}=\sup_{(t,x)\in Q_{b}}|v(t,x)|$ . Now for $Q_{b}=(T_{1},T_{2})\times{\mathcal{I}}$ , where ${\mathcal{I}}$ is a bounded open subset of $\mathbb{R}$ , and $\beta\in(0,1)$ , define the Hölder norms

	$\displaystyle\|v\|^{\beta}_{Q_{b}}=\|v\|_{Q_{b}}+\sup_{t\in(T_{1},T_{2}),x,y\in{\mathcal{I}}}\|v(t,x)-v(t,y)\|\cdot\|x-y\|^{-\beta}$
	$\displaystyle\qquad\qquad+\sup_{s,t\in(T_{1},T_{2}),x\in{\mathcal{I}}}\|v(s,x)-v(t,x)\|\cdot\|s-t\|^{-\beta/2},$
	$\displaystyle\|v\|^{1+\beta}_{Q_{b}}=\|v\|^{\beta}_{Q_{b}}+\|v_{x}\|^{\beta}_{Q_{b}},$
	$\displaystyle\|v\|^{2+\beta}_{Q_{b}}=\|v\|^{1+\beta}_{Q_{b}}+\|v_{t}\|^{\beta}_{Q_{b}}+\|v_{xx}\|^{\beta}_{Q_{b}}.$

Lemma 3.3.

Under (H1)–(H4), the following holds:

(i) Equation (3.20) has a unique solution $V^{\alpha}$ in $C_{b}^{1,2}(\overline{Q}_{T})$ and moreover $\sup_{\overline{Q}_{T}}|V^{\alpha}_{xx}|\leq C$ .

(ii) The best response

(3.22)

\displaystyle u_{\alpha}=\phi_{\alpha}(t,x|\mu_{G}(\cdot)),\quad\alpha\in[0,1]

as the optimal control law solved from (3.20) is bounded and Borel measurable on $[0,T]\times\mathbb{R}$ , and Lipschitz continuous in $x$ , uniformly with respect to $\alpha$ for the given $\mu_{G}(\cdot)$ .

Proof.

(i) Denote

\mathbold{H}_{\alpha}(t,x,q)=\min_{u\in U}\{q\widetilde{f}_{\alpha}^{*}(t,x,u)+\widetilde{l}_{\alpha}^{*}(t,x,u)\}.

Then (3.20) may be rewritten as

(3.23)

\displaystyle-V^{\alpha}_{t}(t,x)=\mathbold{H}_{\alpha}(t,x,V^{\alpha}_{x})+\frac{\sigma^{2}}{2}V^{\alpha}_{xx},\qquad V^{\alpha}(T,x)=0.

As in the proof of [25, Theorem 5], we use Hölder and Lipschitz continuity (with respect to $t$ and $x$ , respectively) of $\widetilde{f}_{\alpha}^{*}$ and $\widetilde{l}_{\alpha}^{*}$ in Lemma 3.2, and follow the method in the proof of Theorem VI.6.2 of [13, p. 210] to show that (3.20) has a unique solution $V^{\alpha}\in C_{b}^{1,2}(\overline{Q}_{T})$ , where uniqueness follows from a verification theorem using the closed-loop state process.

Next we show that $V_{xx}^{\alpha}$ is bounded on $\overline{Q}_{T}$ . Take any $x_{0}\in\mathbb{R}$ . Denote $B_{r}(x_{0})=(x_{0}-r,x_{0}+r)$ for $r>0$ , and $Q_{T}^{x_{0},r}=(0,T)\times B_{r}(x_{0})$ . We use two steps involving local estimates. Each step gets refined information about $V^{\alpha}$ in a region based on available bound information in a larger region. It suffices to obtain a bound of $V_{xx}^{\alpha}$ on $Q_{T}^{x_{0},1}$ as long as this bound does not change with $x_{0}$ .

Step 1. First, there exists a constant $C_{1}$ such that

(3.24)

\displaystyle\sup_{t,x,\alpha}|V^{\alpha}|\leq C_{1},\quad\sup_{t,x,\alpha}|V_{x}^{\alpha}|\leq C_{1}.

The first inequality is obtained using (H1)–(H2) and the fact that $V^{\alpha}$ is the value function of the associated optimal control problem. The second inequality is proven by the difference estimate of $|V^{\alpha}(t,x)-V^{\alpha}(t,y)|$ as in [13, p. 209].

By (H1), (H2) and (3.24), we have

\sup_{\alpha}\sup_{(t,x)\in\overline{Q}_{T}}|\mathbold{H}_{\alpha}(t,x,V_{x}^{\alpha}(t,x))|\leq C_{2}.

We use a typical method for analyzing semilinear parabolic equations. Once $V^{\alpha}$ is known to be a solution of (3.23), we view $V^{\alpha}$ as the solution of a linear equation with the free term $\mathbold{H}_{\alpha}(t,x,V_{x}^{\alpha})$ . For further estimates, we need $\lambda>n+2$ when using the norm (3.21). Fix $\lambda=n+3=4$ . This yields the bound

\|V^{\alpha}\|^{(2)}_{\lambda,Q_{T}^{x_{0},2}}\leq C_{3},

where $C_{3}$ depends on $(C_{2},T,\sigma)$ and the bound of $(f,f_{0},l,l_{0})$ but not on $x_{0}$ , $\alpha$ ; see [13, p. 207] and also [29, p. 342] for local estimates of the Sobolev norm of solutions defined on unbounded domain using a cut-off function. Take $\beta=1-\frac{n+2}{\lambda}=\frac{1}{4}$ . Subsequently, since $\lambda>n+2$ , we have the Hölder estimate

(3.25)

\displaystyle|V^{\alpha}|_{Q^{x_{0},2}_{T}}^{1+\beta}\leq C_{4}\|V^{\alpha}\|^{(2)}_{{}_{\lambda,Q_{T}^{x_{0},2}}}\leq C_{3}C_{4},

where $C_{4}$ is determined by $\lambda=4$ without depending on $x_{0},\alpha$ ; see [13, p. 207], [29, p. 343].

Step 2. On $[0,T]\times\mathbb{R}\times[-C_{1},C_{1}]$ , we can show ${\mathbold H}_{\alpha}(t,x,q)$ is Hölder continuous in $t$ and Lipschitz continuous in $(x,q)$ . Denote $\beta_{1}=\min\{\eta,\beta\}$ . Next we view $\mathbold{H}_{\alpha}(t,x,V_{x}^{\alpha}(t,x))$ as a function of $(t,x)$ . Then by use of (3.25) we further obtain a bound on the Hölder norm:

(3.26)

\displaystyle\sup_{\alpha}\sup_{x_{0}}|\mathbold{H}_{\alpha}(\cdot,\cdot,V_{x}^{\alpha})|^{\beta_{1}}_{Q^{x_{0},2}_{T}}\leq C_{5}.

Subsequently, by the method in [13, p. 207-208] with its cut-off function technique and [29, p. 351-352], we use (3.26) and local Hölder estimates of (3.23) to obtain

(3.27)

\displaystyle|V^{\alpha}|^{2+\beta_{1}}_{Q^{x_{0},1}_{T}}\leq C_{6},

where $C_{6}$ depends on $C_{5}$ but not on $x_{0},\alpha$ . Since $x_{0}$ is arbitrary, it follows that

(3.28)

\displaystyle\sup_{\alpha}\sup_{\overline{Q}_{T}}|V^{\alpha}_{xx}|\leq C_{6}.

(ii) By (H4), the optimal control law (3.22) as a function of $(t,x)$ is well defined and is bounded on $[0,T]\times{\mathbb{R}}$ by compactness of $U$ . It is Borel measurable on $\overline{Q}_{T}$ ; see [13, p.168]. Since $S_{\alpha}^{\nu_{G}}(x,q)$ is Lipschitz continuous in $(x,q)\in\mathbb{R}\times[-C_{1},C_{1}]$ and $V_{x}^{\alpha}(t,x)$ is Lipschitz continuous in $x\in\mathbb{R}$ by (3.28), uniformly with respect to $\alpha$ in each case, $\phi_{\alpha}$ is uniformly Lipschitz continuous in $x$ . ∎

Denote

\Psi^{\alpha}(t,x)=(V^{\alpha}(t,x),V_{t}^{\alpha}(t,x),V^{\alpha}_{x}(t,x),V^{\alpha}_{xx}(t,x)),\quad(t,x)\in\overline{Q}_{T}.

We prove the following continuity lemma for the solution of (3.20). For $\overline{Q}_{T}$ , define the compact subsets $B_{j}=\{(t,x)|0\leq t\leq T,|x|\leq j\}$ , $j\in\mathbb{N}$ .

Lemma 3.4.

Assume (H1)–(H5) hold and let $\mu_{G}(\cdot)$ in (3.19) be fixed. Then the following holds:

(i) For all compact set $B_{j}$ , $\lim_{\alpha^{\prime}\to\alpha}|\Psi^{\alpha^{\prime}}-\Psi^{\alpha}|_{B_{j}}=0$ .

(ii) $\lim_{\alpha^{\prime}\to\alpha}V_{x}^{\alpha^{\prime}}(t,x)=V_{x}^{\alpha}(t,x)$ for all $(t,x)\in[0,T]\times\mathbb{R}$ .

Proof.

It suffices to show (i) as (ii) follows immediately from (i).

Step 1. By (3.27) and the fact that the constant $C_{6}$ can be selected without depending on $\alpha$ , there exists a constant $C$ such that $\sup_{\alpha}|V^{\alpha}|^{2+\beta_{1}}_{B_{j}}\leq C,$ which implies that $\{\Psi^{\alpha},\alpha\in[0,1]\}$ is uniformly bounded and equicontinuous on $B_{j}$ . For any sequence $\{\alpha_{k},k\geq 1\}$ converging to $\alpha$ , by Ascoli-Arzela’s lemma, for $j=1$ , there exists a subsequence denoted by $\{\bar{\alpha}_{k},k\geq 1\}$ such that $\Psi^{\bar{\alpha}_{k}}$ converges uniformly on $B_{1}$ . By a diagonal argument, we may further extract a subsequence of $\{\bar{\alpha}_{k},k\geq 1\}$ , denoted by $\{\hat{\alpha}_{k},k\geq 1\}$ , such that $\Psi^{\hat{\alpha}_{k}}$ converges uniformly on each set $B_{j}$ , $j\geq 1$ . Hence there exists a function $V^{*}$ with continuous derivatives $V^{*}_{t},V^{*}_{x},V^{*}_{xx}$ on $\overline{Q}_{T}$ such that

(3.29)

\displaystyle\lim_{k\to\infty}\Psi^{\hat{\alpha}_{k}}(t,x)=\Psi^{*}(t,x),\qquad\forall(t,x)\in\overline{Q}_{T},

where $\Psi^{*}=(V^{*},V^{*}_{t},V^{*}_{x},V^{*}_{xx})$ . Since

-V_{t}^{\hat{\alpha}_{k}}(t,x)={\mathbold H}_{\alpha_{k}}(t,x,V_{x}^{\hat{\alpha}_{k}})+\frac{\sigma^{2}}{2}V_{xx}^{\hat{\alpha}_{k}},\quad V^{\alpha_{k}}(T,x)=0,

it follows from (3.29) that

\displaystyle-V_{t}^{*}(t,x)={\mathbold H}_{\alpha}(t,x,V_{x}^{*})+\frac{\sigma^{2}}{2}V_{xx}^{*},\qquad V^{*}(T,x)=0.

We have used the fact that ${\mathbold H}_{\alpha}(t,x,q)$ is continuous in $\alpha$ due to (H5) and condition (C1) of ${\mathcal{M}}_{[0,T]}$ . It is clear that $V^{*}=V^{\alpha}$ by uniqueness of the solution of (3.23). So $\Psi^{*}=\Psi^{\alpha}$ . Now it follows that

(3.30)

\displaystyle\lim_{k\to\infty}|\Psi^{\hat{\alpha}_{k}}-\Psi^{\alpha}|_{B_{j}}=0,\quad\forall j.

Step 2. Suppose (i) does not hold so that for some $\hat{j}$ we have $|\Psi^{\alpha^{\prime}}-\Psi^{\alpha}|_{B_{\hat{j}}}$ does not converge to 0 as $\alpha^{\prime}\to\alpha$ , which implies that there exist some $\epsilon_{0}>0$ and a sequence $\{\alpha_{k}^{0}\}$ converging to $\alpha$ such that for each $k$ ,

(3.31)

\displaystyle|\Psi^{\alpha^{0}_{k}}-\Psi^{\alpha}|_{B_{\hat{j}}}\geq\epsilon_{0}.

Step 3. Recall that $\{\alpha_{k}\}$ in Step 1 is arbitrary as long as it converges to $\alpha$ . Now we just take $\{\alpha_{k}\}$ in Step 1 as $\{\alpha_{k}^{0}\}$ . By Step 1, there exists a subsequence of $\{\alpha^{0}_{k}\}$ , denoted by $\{\hat{\alpha}^{0}_{k}\}$ , such that $\lim_{k\to\infty}|\Psi^{\hat{\alpha}_{k}^{0}}-\Psi^{\alpha}|_{B_{\hat{j}}}=0$ , which contradicts (3.31). Hence (i) holds. ∎

Lemma 3.5.

Assume (H1)–(H6). For given $\mu_{G}(\cdot)\in{\mathcal{M}}_{[0,T]}$ , the best response $\phi_{\alpha}(t,x|\mu_{G}(\cdot))$ in (3.22) continuously depends on $\alpha$ . Specifically, for any $\alpha\in[0,1]$ ,

(3.32)

\displaystyle\lim_{\alpha^{\prime}\to\alpha}\phi_{\alpha^{\prime}}(t,x|\mu_{G}(\cdot))=\phi_{\alpha}(t,x|\mu_{G}(\cdot)),\quad\forall t,x.

Proof.

The best response can be written as

	$\displaystyle\phi_{\alpha}(t,x\|\mu_{G}(\cdot))=S_{\alpha}^{\mu_{G}(t)}(x,V_{x}^{\alpha}(t,x)),$
	$\displaystyle\phi_{\alpha^{\prime}}(t,x\|\mu_{G}(\cdot))=S_{\alpha^{\prime}}^{\mu_{G}(t)}(x,V_{x}^{\alpha^{\prime}}(t,x)).$

It follows that

		$\displaystyle\|S_{\alpha}^{\mu_{G}(t)}(x,V_{x}^{\alpha}(t,x))-S_{\alpha^{\prime}}^{\mu_{G}(t)}(x,V_{x}^{\alpha^{\prime}}(t,x))\|$
	$\displaystyle\leq$	$\displaystyle\|S_{\alpha}^{\mu_{G}(t)}(x,V_{x}^{\alpha}(t,x))-S_{\alpha}^{\mu_{G}(t)}(x,V_{x}^{\alpha^{\prime}}(t,x))\|$
		$\displaystyle+\|S_{\alpha}^{\mu_{G}(t)}(x,V_{x}^{\alpha^{\prime}}(t,x))-S_{\alpha^{\prime}}^{\mu_{G}(t)}(x,V_{x}^{\alpha^{\prime}}(t,x))\|.$

Given $\mu_{G}(\cdot)$ we have the prior upper bound $\sup_{\alpha,t,x}|V_{x}^{\alpha}(t,x)|\leq C$ . It suffices to show that (3.32) holds for any given $C_{0}>0$ and $t\in[0,T]$ , $|x|\leq C_{0}$ . By (H6), for the given $\mu_{G}(t)$ , $S_{\alpha}^{\mu_{G}(t)}(x,q)$ is uniformly continuous in $\alpha\in[0,1]$ , $|x|\leq C_{0}$ , $q\in[-C,C]$ . For any $\epsilon>0$ , there exists $\delta>0$ such that $|\alpha-\alpha^{\prime}|<\delta$ implies $\sup_{|x|\leq C_{0},|q|\leq C}|S_{\alpha}^{\mu_{G}(t)}(x,q)-S_{\alpha^{\prime}}^{\mu_{G}(t)}(x,q)|\leq\epsilon/2$ , and moreover,

\sup_{|x|\leq C_{0}}|S_{\alpha}^{\mu_{G}(t)}(x,V_{x}^{\alpha}(t,x))-S_{\alpha}^{\mu_{G}(t)}(x,V_{x}^{\alpha^{\prime}}(t,x))|\leq\frac{\epsilon}{2}

in view of Lemma 3.4 (i). Therefore (3.32) holds. ∎

We proceed to show the existence of a solution to the GMFG equations (3.2) and (3.15) in terms of $\{(V^{\alpha},\mu_{\alpha}(\cdot))|\alpha\in[0,1]\}$ . For $\mu_{G}\in{\mathcal{M}}_{[0,T]}$ , denote the mapping

(\phi_{\alpha})_{\alpha\in[0,1]}\coloneqq\Gamma(\mu_{G}(\cdot)),

where the left hand side is given by (3.22) as the set of best responses with respect to $\mu_{G}(\cdot)$ . Next, we combine $(\phi_{\alpha})_{\alpha\in[0,1]}$ with $\mu_{G}(\cdot)$ to determine the distribution $m_{\alpha}$ of the closed-loop state process

\displaystyle dx_{\alpha}(t)=\widetilde{f}[x_{\alpha}(t),\phi_{\alpha}(t,x_{\alpha}(t)|\mu_{G}(\cdot)),\mu_{G}(t);g_{\alpha}]dt+\sigma dw_{\alpha}(t),

where $x_{\alpha}(0)$ has distribution $\mu_{0}^{x}$ . The choice of the Brownian motion for $x_{\alpha}$ is immaterial. For $m_{\alpha}$ above, denote the mapping from ${\mathcal{M}}_{[0,T]}$ to ${\bf M}_{T}^{G}$ :

(m_{\alpha})_{\alpha\in[0,1]}=\widehat{\Gamma}(\mu_{G}(\cdot)).

Define the set

\displaystyle{\bf M}_{T}^{G1}\coloneqq\widehat{\Gamma}({\mathcal{M}}_{[0,T]})\subset{\bf M}_{T}^{G}.

Now the existence analysis may be formulated as the problem of finding a fixed point of the form

(3.33)

\displaystyle m_{G}=\widehat{\Gamma}\circ{\rm Marg}(m_{G}),

in case $m_{G}\in{\bf M}_{T}^{G1}$ . Note that ${\rm Marg}(m_{G})=\{({\rm Marg}_{t}(m_{\alpha}))_{\alpha\in[0,1]},0\leq t\leq T\}$ .

Remark 3.6.

The fixed point problem requires $m_{G}$ to be from the subset ${\bf M}_{T}^{G1}$ of ${\bf M}_{T}^{G}$ . If one simply looks for $m_{G}\in{\bf M}_{T}^{G}$ , the resulting $\mu_{G}(\cdot)={{\rm Marg}}(m_{G})$ lacks required properties such as Hölder continuity in (C2), and this will cause difficulties in establishing Lemma 3.3 for the HJB equation.

Lemma 3.7.

Under (H1)–(H6), the following assertions hold:

(i) ${\bf M}_{T}^{G1}\subset{\bf M}_{T}^{G0}$ .

(ii) For any $m_{G}\in{\bf M}_{T}^{G1}$ , $\mu_{G}(\cdot)\coloneqq{{\rm Marg}}(m_{G})\in{\mathcal{M}}_{[0,T]}$ .

(iii) The best response $\phi_{\alpha}(t,x|\mu_{G}(\cdot))$ with $\mu_{G}(\cdot)$ given in (ii) is Lipschitz continuous in $x$ , uniformly with respect to $\alpha\in[0,1]$ and $m_{G}\in{\bf M}_{T}^{G1}$ .

Proof.

(i) and (ii) For $m_{G}\in{\bf M}_{T}^{G1}$ , there exists $\mu^{\prime}_{G}\in{\mathcal{M}}_{[0,T]}$ such that $m_{G}=\widehat{\Gamma}(\mu^{\prime}_{G}(\cdot)).$ To estimate $D_{T}(m_{\alpha},m_{\bar{\alpha}})$ and $W_{1}(\mu_{\alpha}(t),\mu_{\bar{\alpha}}(t))$ , let $x_{\alpha}$ and $x_{\bar{\alpha}}$ be state processes generated by (3.10) with $\mu_{G}^{\prime}$ , the same initial state and Brownian motion under the control laws $\phi_{\alpha}(t,x|\mu_{G}^{\prime}(\cdot))$ and $\phi_{\bar{\alpha}}(t,x|\mu_{G}^{\prime}(\cdot))$ , respectively. Then $D_{T}(m_{\alpha},m_{\bar{\alpha}})\leq E\sup_{t\leq T}|x_{\alpha}(t)-x_{\bar{\alpha}}(t)|$ and $W_{1}(\mu_{\alpha}(t),\mu_{\bar{\alpha}}(t))\leq E|x_{\alpha}(t)-x_{\bar{\alpha}}(t)|$ . Fixing $\bar{\alpha}$ , we have

(3.34)		$\displaystyle\|x_{\alpha}(t)-x_{\bar{\alpha}}(t)\|\leq$	$\displaystyle\int_{0}^{t}\|\widetilde{f}[x_{\alpha}(s),\phi_{\alpha}(s,x_{\alpha}(s)\|\mu^{\prime}_{G}(\cdot)),\mu^{\prime}_{G}(s);g_{\alpha}]$
		$\displaystyle\quad-\widetilde{f}[x_{\bar{\alpha}}(s),\phi_{\bar{\alpha}}(s,x_{\bar{\alpha}}(s)\|\mu^{\prime}_{G}(\cdot)),\mu^{\prime}_{G}(s);g_{\bar{\alpha}}]\|ds.$

Denote

	$\displaystyle\delta_{1}=\|f_{0}[x_{\bar{\alpha}}(s),\phi_{\bar{\alpha}}(s,x_{\bar{\alpha}}(s)\|\mu^{\prime}_{G}(\cdot)),\mu^{\prime}_{\alpha}(s)]-f_{0}[x_{\bar{\alpha}}(s),\phi_{\bar{\alpha}}(s,x_{\bar{\alpha}}(s)\|\mu^{\prime}_{G}(\cdot)),\mu^{\prime}_{\bar{\alpha}}(s)]\|,$
	$\displaystyle\delta_{2}=\|f[x_{\bar{\alpha}}(s),\phi_{\bar{\alpha}}(s,x_{\bar{\alpha}}(s)\|\mu^{\prime}_{G}(\cdot)),\mu^{\prime}_{G}(s);g_{\alpha}]-f[x_{\bar{\alpha}}(s),\phi_{\bar{\alpha}}(s,x_{\bar{\alpha}}(s)\|\mu^{\prime}_{G}(\cdot)),\mu^{\prime}_{G}(s);g_{\bar{\alpha}}]\|.$

Then by (3.34) and the Lipschitz continuity in $x$ of $\phi_{\alpha}$ in Lemma 3.3 (ii), we obtain

(3.35)		$\displaystyle\|x_{\alpha}(t)-x_{\bar{\alpha}}(t)\|\leq C_{1}\int_{0}^{t}\|x_{\alpha}(s)-x_{\bar{\alpha}}(s)\|ds$
	$\displaystyle+C_{2}\int_{0}^{t}\{\|\phi_{\alpha}(s,x_{\bar{\alpha}}(s)\|\mu^{\prime}_{G}(\cdot))-\phi_{\bar{\alpha}}(s,x_{\bar{\alpha}}(s)\|\mu^{\prime}_{G}(\cdot))\|+\delta_{1}(s)+\delta_{2}(s)\}ds,$

where $C_{2}$ depends only on the Lipschitz constants of $f_{0},f$ ; and $C_{1}$ does not change with $\alpha$ for the fixed $\mu_{G}^{\prime}$ . Since $W_{1}(\mu^{\prime}_{\alpha}(s),\mu^{\prime}_{\bar{\alpha}}(s))\to 0$ as $\alpha\to\bar{\alpha}$ , by (H2) $E\delta_{1}(s)\to 0$ as $\alpha\to\bar{\alpha}$ . By (H5), we have $E\delta_{2}(s)\to 0$ as $\alpha\to\bar{\alpha}$ . Then using Lemma 3.5 and boundedness of the integrand below, we obtain

\lim_{\alpha\to\bar{\alpha}}E\int_{0}^{T}\{|\phi_{\alpha}(s,x_{\bar{\alpha}}(s)|\mu^{\prime}_{G}(\cdot))-\phi_{\bar{\alpha}}(s,x_{\bar{\alpha}}(s)|\mu^{\prime}_{G}(\cdot))|+\delta_{1}(s)+\delta_{2}(s)\}ds=0.

By Gronwall’s lemma and (3.35), it follows that

(3.36)

\displaystyle\lim_{\alpha\to\bar{\alpha}}E\sup_{0\leq t\leq T}|x_{\alpha}(t)-x_{\bar{\alpha}}(t)|=0.

Subsequently, as $\alpha\to\bar{\alpha}$ , we obtain $D_{T}(m_{\alpha},m_{\bar{\alpha}})\to 0$ , which implies (i); in addition, $W_{1}(\mu_{\alpha}(t),\mu_{\bar{\alpha}}(t))\to 0$ , which verifies condition (C1) of ${\mathcal{M}}_{[0,T]}$ for $\mu_{G}$ . Since each $m_{\alpha}$ is the distribution of $x_{\alpha}$ , for $\mu_{G}(\cdot)$ we take the Hölder parameter $\eta=1/2$ and a constant $C_{h}$ independent of $\mu_{G}^{\prime}$ for (C2). So (ii) holds.

(iii) Due to the choice of $\eta$ and $C_{h}$ for $\mu_{G}(\cdot)$ in (ii), we may select a fixed constant $C_{5}$ in (3.26), which does not change with $(\alpha,\mu_{G}(\cdot))$ . Subsequently the upper bound $C_{6}$ in (3.28) for $|V_{xx}^{\alpha}|$ does not change with $\alpha\in[0,1],\mu_{G}(\cdot)\in{\rm Marg}(\widehat{\Gamma}({\mathcal{M}}_{[0,T]}))$ . This ensures a uniform bound for the Lipschitz constant for $x$ in $\phi_{\alpha}$ . ∎

We introduce the sensitivity condition.

(H7) For $m_{G},\bar{m}_{G}\in{\bf M}_{T}^{G1}=\widehat{\Gamma}({\mathcal{M}}_{[0,T]})$ , there exists a constant $c_{1}$ such that

(3.37)

\displaystyle\sup_{t,x,\alpha}|\phi_{\alpha}(t,x|\mu_{G}(\cdot))-\bar{\phi}_{\alpha}(t,x|\bar{\mu}_{G}(\cdot))|\leq c_{1}d(m_{G},\bar{m}_{G}),

where the set of control laws $\{\phi_{\alpha}(t,x|\mu_{G}(\cdot)),\alpha\in[0,1]\}$ (resp., $\{\bar{\phi}_{\alpha}(t,x|\bar{\mu}_{G}(\cdot)),\alpha\in[0,1]\}$ ) is determined by use of $\mu_{G}={\rm Marg}(m_{G})$ (resp., $\bar{\mu}_{G}={\rm Marg}(\bar{m}_{G})$ ) in the optimal control problem specified by (3.10) and (3.12) with the graphon section $g_{\alpha}$ .

Assumption (H7) is a generalization from the finite type model in [25] where an illustration via a linear model is presented. Related sensitivity conditions are studied in [28].

Let $(\phi_{\alpha})_{\alpha\in[0,1]}$ in (3.22) be applied by all agents, where $\mu_{G}(\cdot)\in{\mathcal{M}}_{[0,T]}$ . We consider the following generalized McKean-Vlasov equation

(3.38)

\displaystyle dx_{\alpha}(t)=\widetilde{f}[x_{\alpha}(t),\phi_{\alpha}(t,x_{\alpha}(t)|\mu_{G}),\nu_{G}(t);g_{\alpha}]dt+\sigma dw_{\alpha}(t),

where $x_{\alpha}(0)$ is given with distribution $\mu_{0}^{x}$ . For this equation, $\nu_{G}$ is part of the solution. If $\nu_{G}$ is determined, we have a unique solution $x_{\alpha}$ on $[0,T]$ which further determines its law as the measure $m_{\alpha}$ on $(C_{T},{\mathcal{F}}_{T})$ . Note that $m_{\alpha}$ does not depend on the choice of the standard Brownian motion $w_{\alpha}$ . We look for $\nu_{G}\in{\mathcal{M}}_{[0,T]}$ to satisfy the condition:

(3.39)

\displaystyle{\rm Marg}_{t}(m_{\alpha})=\nu_{\alpha}(t),\quad\forall\alpha\in[0,1],\ t\in[0,T],

i.e., $\nu_{\alpha}(t)$ is the law of $x_{\alpha}(t)$ for all $\alpha,t$ (and we say $(x_{\alpha})_{0\leq\alpha\leq 1}$ is consistent with $\nu_{G}$ ).

Lemma 3.8.

Assume (H1)–(H6). For the best response control law $\phi_{\alpha}(t,x_{\alpha}|\mu_{G}(\cdot))$ in (3.22), where $\mu_{G}(\cdot)\in{\mathcal{M}}_{[0,T]}$ , there exists a unique $\nu_{G}(\cdot)$ for (3.38) satisfying (3.39).

Proof.

In order to solve $(x_{\alpha},\nu_{G})$ in (3.38), we specify the law of the process $x_{\alpha}$ instead of just its marginal $\nu_{\alpha}(t)$ . This extends the fixed point idea for treating standard McKean-Vlasov equations [41].

For $(m_{\alpha})_{\alpha\in[0,1]}\in{\bf M}_{T}^{G0}$ , we determine $\nu_{G}^{1}$ according to $\nu^{1}_{\alpha}(t)={\rm Marg}_{t}(m_{\alpha})$ , which is used in (3.38) by taking $\nu_{G}=\nu_{G}^{1}$ to solve $x_{\alpha}$ on $[0,T]$ . Let $m_{\alpha}^{\rm new}$ denote the law of $x_{\alpha}$ . It in general does not satisfy ${\rm Marg}_{t}(m_{\alpha}^{\rm new})=\nu_{\alpha}(t)$ for all $t$ . Denote the mapping

(m_{\alpha}^{\rm new})_{\alpha\in[0,1]}=\Phi_{{\bf M}_{T}^{G0}}((m_{\alpha})_{\alpha\in[0,1]}).

By (H5) and Lemma 3.5, $\Phi_{{\bf M}_{T}^{G0}}$ is a mapping from ${\bf M}_{T}^{G0}$ to itself. Similarly, from $(\bar{m}_{\alpha})_{\alpha\in[0,1]}\in{{\bf M}_{T}^{G0}}$ we determine $\bar{\nu}_{G}^{1}$ for (3.38) and solve $\bar{x}_{\alpha}$ with its law $\bar{m}^{\rm new}_{\alpha}$ . Denote

(\bar{m}_{\alpha}^{\rm new})_{\alpha\in[0,1]}=\Phi_{{\bf M}_{T}^{G0}}((\bar{m}_{\alpha})_{\alpha\in[0,1]}).

If $h(x,y)$ is a bounded Lipschitz continuous function with $|h(x,y)-h(\bar{x},\bar{y})|\leq C_{1}|x-\bar{x}|+C_{2}(|y-\bar{y}|\wedge 1)$ , we have

		$\displaystyle\Big{\|}\int h(x,y)g(\alpha,\beta)\nu_{\beta}^{1}(t,dy)d\beta-\int h(\bar{x},\bar{y})g(\alpha,\beta)\nu_{\beta}^{2}(t,d\bar{y})d\beta\Big{\|}$
	$\displaystyle\leq$	$\displaystyle C_{1}\|x-\bar{x}\|+\sup_{\beta}\Big{\|}\int h(\bar{x},y)\nu_{\beta}^{1}(t,dy)-\int h(\bar{x},\bar{y})\nu_{\beta}^{2}(t,d\bar{y})\Big{\|}$
	$\displaystyle=$	$\displaystyle C_{1}\|x-\bar{x}\|+\sup_{\beta}\Big{\|}\int_{C_{T}}h(\bar{x},X_{t}(\omega))dm_{\beta}(\omega)-\int_{C_{T}}h(\bar{x},X_{t}(\bar{\omega}))d\bar{m}_{\beta}(\bar{\omega})\Big{\|}$
	$\displaystyle\leq$	$\displaystyle C_{1}\|x-\bar{x}\|+C_{2}\sup_{\beta}\int_{C_{T}\times C_{T}}(\|X_{t}(\omega)-X_{t}(\bar{\omega})\|\wedge 1)d\widehat{m}_{\beta}(\omega,\bar{\omega}),$

where $X$ is the canonical process, $\omega,\bar{\omega}\in C_{T}$ , and $\widehat{m}_{\beta}$ is any coupling of $m_{\beta}$ and $\bar{m}_{\beta}$ . Hence

		$\displaystyle\|\int h(x,y)g(\alpha,\beta)\nu_{\beta}^{1}(t,dy)d\beta-\int h(\bar{x},\bar{y})g(\alpha,\beta)\nu_{\beta}^{2}(t,d\bar{y})d\beta\|$
(3.40)		$\displaystyle\leq\$	$\displaystyle C_{1}\|x-\bar{x}\|+C_{2}\sup_{\beta}D_{t}(m_{\beta},\bar{m}_{\beta}).$

By (H2), (H3), the uniform Lipschitz continuity of $\phi_{\alpha}$ in $x$ by Lemma 3.3 (ii), and (3.40), we obtain

		$\displaystyle\|\widetilde{f}[x_{\alpha},\phi_{\alpha}(t,x_{\alpha}\|\mu_{G}),\nu_{G}^{1}(t);g_{\alpha}]-\widetilde{f}[\bar{x}_{\alpha},\phi_{\alpha}(t,\bar{x}_{\alpha}\|\mu_{G}),\nu_{G}^{2}(t);g_{\alpha}]\|$
	$\displaystyle\leq$	$\displaystyle C_{1}(\|x_{\alpha}-\bar{x}_{\alpha}\|\wedge 1)+C_{2}\sup_{\beta}D_{t}(m_{\beta},\bar{m}_{\beta}).$

Hence by (3.38),

	$\displaystyle\sup_{s\leq t}\|x_{\alpha}(s)-\bar{x}_{\alpha}(s)\|$	$\displaystyle\leq C_{1}\int_{0}^{t}\|x_{\alpha}(s)-\bar{x}_{\alpha}(s)\|\wedge 1ds$
		$\displaystyle+C_{3}\int_{0}^{t}\sup_{\beta}\|D_{s}(m_{\beta},\bar{m}_{\beta})\|ds.$

Therefore, by Gronwall’s lemma,

\displaystyle\sup_{s\leq t}|x_{\alpha}(s)-\bar{x}_{\alpha}(s)|\wedge 1\leq C_{4}\int_{0}^{t}\sup_{\beta}|D_{s}(m_{\beta},\bar{m}_{\beta})|ds,

which combined with the definition of the Wasserstein metric $D_{t}(\cdot,\cdot)$ implies that

(3.41)

\displaystyle\sup_{\beta}|D_{t}(m^{\rm new}_{\beta},\bar{m}^{\rm new}_{\beta})|\leq C_{4}\int_{0}^{t}\sup_{\beta}|D_{s}(m_{\beta},\bar{m}_{\beta})|ds.

By iterating (3.41) as in [41, p. 174], we can show that for a sufficiently large $k_{0}$ , $\Phi_{{\bf M}_{T}^{G0}}^{k_{0}}$ is a contraction. We can further show that $\{\Phi_{{\bf M}_{T}^{G0}}^{k}(m_{G}),k\geq 1\}$ is a Cauchy sequence, and we obtain a unique fixed point $m^{*}_{G}$ for $\Phi_{{\bf M}_{T}^{G0}}$ . Then we obtain a solution of (3.38) by taking $\nu_{\alpha}(t)={\rm Marg}_{t}(m^{*}_{\alpha})$ . If there are two different solutions with $\nu_{G}\neq\nu_{G}^{\prime}$ , we can derive a contradiction by using uniqueness of the fixed point of $\Phi_{{\mathbf{M}}_{T}^{G0}}$ . ∎

Now we consider two sets of best response control laws $(\phi_{\alpha}(t,x_{\alpha}|\mu_{G}))_{\alpha\in[0,1]}$ and $(\bar{\phi}_{\alpha}(t,x_{\alpha}|\bar{\mu}_{G}))_{\alpha\in[0,1]}$ , where $\mu_{G}={\rm Marg}(m_{G}),\bar{\mu}_{G}={\rm Marg}(\bar{m}_{G})$ for $m_{G},\bar{m}_{G}\in{\bf M}_{T}^{G1}$ (then clearly $\mu_{G},\bar{\mu}_{G}\in{\mathcal{M}}_{[0,T]}$ ), and use Lemma 3.8 to solve $(x_{\alpha},\nu_{G})$ and $(x^{\prime}_{\alpha},\bar{\nu}_{G})$ from the generalized MV-SDEs

(3.42)		$\displaystyle dx_{\alpha}=\widetilde{f}[x_{\alpha},\phi_{\alpha}(t,x_{\alpha}\|\mu_{G}),\nu_{G}(t);g_{\alpha}]dt+\sigma dw_{\alpha}(t),$
(3.43)		$\displaystyle dx_{\alpha}^{\prime}=\widetilde{f}[x_{\alpha}^{\prime},\bar{\phi}_{\alpha}(t,x^{\prime}_{\alpha}\|\bar{\mu}_{G}),\bar{\nu}_{G}(t);g_{\alpha}]dt+\sigma dw_{\alpha}(t),$

where $x^{\prime}_{\alpha}(0)=x_{\alpha}(0)$ is given. Let $m_{\alpha}^{\rm mv}$ (resp., $\bar{m}_{\alpha}^{\rm mv}$ ) denote the law of $x_{\alpha}$ (resp., $x^{\prime}_{\alpha}$ ). The following lemma is a generalization of [25, Lemma 9] to the graphon network case.

Lemma 3.9.

For (3.42) and (3.43) there exists a constant $c_{2}$ independent of $(m_{G},\bar{m}_{G})$ such that

\displaystyle\sup_{\alpha}D_{T}(m_{\alpha}^{\rm mv},\bar{m}_{\alpha}^{\rm mv})\leq c_{2}\sup_{t,x,\alpha}|\phi_{\alpha}(t,x|\mu_{G}(\cdot))-\bar{\phi}_{\alpha}(t,x|\bar{\mu}_{G}(\cdot))|.

Proof.

For (3.42)–(3.43), denote

\displaystyle\Delta_{s}=\

\displaystyle\widetilde{f}[x_{\alpha}(s),\phi_{\alpha}(s,x_{\alpha}(s)|\mu_{G}),\nu_{G}(s);g_{\alpha}]-\widetilde{f}[x_{\alpha}^{\prime}(s),\bar{\phi}_{\alpha}(s,x^{\prime}_{\alpha}(s)|\bar{\mu}_{G}),\bar{\nu}_{G}(s);g_{\alpha}].

We have

(3.44)

\displaystyle x_{\alpha}(t)-x_{\alpha}^{\prime}(t)=\int_{0}^{t}\Delta_{s}ds.

Noting $\nu_{\alpha}(t)={\rm Marg}_{t}(m_{\alpha}^{\rm mv})$ and $\bar{\nu}_{\alpha}(t)={\rm Marg}_{t}(\bar{m}_{\alpha}^{\rm mv})$ , we have

	$\displaystyle\|\Delta_{s}\|\leq$	$\displaystyle\|\widetilde{f}[x_{\alpha}(s),\phi_{\alpha}(s,x_{\alpha}(s)\|\mu_{G}),\nu_{G}(s);g_{\alpha}]-\widetilde{f}[x_{\alpha}^{\prime}(s),\phi_{\alpha}(s,x^{\prime}_{\alpha}(s)\|\mu_{G}),\bar{\nu}_{G}(s);g_{\alpha}]\|$
		$\displaystyle\hskip-14.22636pt+\|\widetilde{f}[x_{\alpha}^{\prime}(s),\phi_{\alpha}(s,x^{\prime}_{\alpha}(s)\|\mu_{G}),\bar{\nu}_{G}(s);g_{\alpha}]-\widetilde{f}[x_{\alpha}^{\prime}(s),\bar{\phi}_{\alpha}(s,x^{\prime}_{\alpha}(s)\|\bar{\mu}_{G}),\bar{\nu}_{G}(s);g_{\alpha}]\|$
	$\displaystyle\leq$	$\displaystyle C_{1}\|x_{\alpha}(s)-x_{\alpha}^{\prime}(s)\|+C_{2}\sup_{\beta}D_{s}(m_{\beta}^{\rm mv},\bar{m}^{\rm mv}_{\beta})$
(3.45)			$\displaystyle+C_{3}\sup_{t,x}\|\phi_{\alpha}(t,x\|\mu_{G}(\cdot))-\bar{\phi}_{\alpha}(t,x\|\bar{\mu}_{G}(\cdot))\|,$

where $C_{1}$ , $C_{2}$ and $C_{3}$ do not depend on $(\alpha,m_{G},\bar{m}_{G})$ . The difference term on the first line is estimated by the method in (3.40). We have used the fact that $\phi_{\alpha}$ is uniformly Lipschitz in $x$ by Lemma 3.7 (iii). Therefore, by (3.44)–(3.45),

	$\displaystyle\|x_{\alpha}(t)-x_{\alpha}^{\prime}(t)\|\leq$	$\displaystyle\int_{0}^{t}\Big{[}C_{1}\|x_{\alpha}(s)-x_{\alpha}^{\prime}(s)\|+C_{2}\sup_{\beta}D_{s}(m^{\rm mv}_{\beta},\bar{m}^{\rm mv}_{\beta})\Big{]}ds$
		$\displaystyle+C_{3}t\sup_{t,x}\|\phi_{\alpha}(t,x\|\mu_{G}(\cdot))-\bar{\phi}_{\alpha}(t,x\|\bar{\mu}_{G}(\cdot))\|.$

By Gronwall’s lemma, we obtain

	$\displaystyle\sup_{0\leq s\leq t}\|x_{\alpha}(s)-x_{\alpha}^{\prime}(s)\|\wedge 1$	$\displaystyle\leq e^{C_{1}t}C_{2}\int_{0}^{t}\sup_{\beta}D_{s}(m^{\rm mv}_{\beta},\bar{m}^{\rm mv}_{\beta})ds$
		$\displaystyle+e^{C_{1}t}C_{3}t\sup_{t,x}\|\phi_{\alpha}(t,x\|\mu_{G}(\cdot))-\bar{\phi}_{\alpha}(t,x\|\bar{\mu}_{G}(\cdot))\|,$

which again by the definition of the metric $D_{t}(\cdot,\cdot)$ leads to

(3.46)		$\displaystyle\sup_{\alpha}D_{t}(m^{\rm mv}_{\alpha},\bar{m}^{\rm mv}_{\alpha})\leq$	$\displaystyle e^{C_{1}t}C_{2}\int_{0}^{t}\sup_{\alpha}D_{s}(m^{\rm mv}_{\alpha},\bar{m}^{\rm mv}_{\alpha})ds$
		$\displaystyle+e^{C_{1}t}C_{3}t\sup_{t,x,\alpha}\|\phi_{\alpha}(t,x\|\mu_{G}(\cdot))-\bar{\phi}_{\alpha}(t,x\|\bar{\mu}_{G}(\cdot))\|.$

The lemma follows from applying Gronwall’s lemma to (3.46). ∎

3.4. Existence Theorem

We state the main result on the existence and uniqueness of solutions to the GMFG equation system. We introduce a contraction condition:

(H8) $c_{1}c_{2}<1$ , where $c_{1}$ is the constant in the sensitivity condition (H7) and $c_{2}$ is specified in Lemma 3.9.

Remark 3.10.

By SDE estimates, one can obtain refined bound information on $c_{2}$ . When the coupling effect is weak or $T$ is small, a small value for $c_{2}$ can be obtained.

Remark 3.11.

For linear models, a verification of the contraction condition can be done under reasonable model parameters, as in [25].

Theorem 3.12.

Under (H1)–(H8), there exists a unique solution $(V^{\alpha},\mu_{\alpha}(\cdot))_{\alpha\in[0,1]}$ to the GMFG equations (3.2) and (3.15), which (i) gives the feedback control best response (BR) strategy $\varphi(t,x_{\alpha}|{\mu}_{G}(\cdot);g_{\alpha})$ depending only upon the agent’s state and the ensemble $\mu_{G}$ of local mean fields (i.e. $(x_{\alpha},{\mu}_{G})$ ), and (ii) generates a Nash equilibrium.

Proof.

Step 1 – We return to the fixed point equation (3.33), which is redisplayed below:

(3.47)

\displaystyle m_{G}=\widehat{\Gamma}\circ{\rm Marg}(m_{G}),

where $m_{G}=(m_{\alpha})_{\alpha\in[0,1]}\in{\bf M}_{T}^{G1}$ . For $m_{G}\in{\bf M}_{T}^{G1}$ , the Hölder continuity in $t$ of the regenerated $\mu_{G}(\cdot)={\rm Marg}(m_{G})$ can be checked by elementary SDE estimates by adapting the proof of [25, Lemma 7].

Step 2 – Take a general $m_{G}\in{\bf M}_{T}^{G1}$ to determine $\mu_{G}={\rm Marg}(m_{G})$ and $\phi_{\alpha}(t,x_{\alpha}|\mu_{G}(\cdot))$ . When $\bar{m}_{G}\in{\bf M}_{T}^{G1}$ is used, we determine $\bar{\mu}_{G}$ and $\bar{\phi}_{\alpha}(t,x_{\alpha}|\bar{\mu}_{G}(\cdot))$ . Once the set of strategies $(\phi_{\alpha})_{\alpha\in[0,1]}$ is applied to the generalized MV equation (3.38), by Lemma 3.8, we may solve for $(x_{\alpha},\nu_{G}(\cdot))$ such that $x_{\alpha}$ has the law $m_{\alpha}^{\rm new}$ and ${\rm Marg}_{t}(m_{\alpha}^{\rm new})=\nu_{\alpha}(t)$ . This is done in parallel for $\bar{m}_{G}$ to generate $\bar{m}_{\alpha}^{\rm new}$ . We accordingly determine $m_{G}^{\rm new}$ and $\bar{m}_{G}^{\rm new}$ .

Step 3 – By (3.37) and Lemma 3.9, we obtain

\displaystyle\sup_{\alpha}D_{T}(m_{\alpha}^{\rm new},\bar{m}_{\alpha}^{\rm new})\leq c_{1}c_{2}d(m_{G},\bar{m}_{G}),

which implies

d(m_{G}^{\rm new},\bar{m}_{G}^{\rm new})\leq c_{1}c_{2}d(m_{G},\bar{m}_{G}).

Based on the above contraction property, we construct a Cauchy sequence in the complete metric space ${\bf M}_{T}^{G}$ by iterating with $m_{G}$ and establish existence of a solution to the GMFG equation system. To show uniqueness, suppose $m_{G}$ and $\tilde{m}_{G}$ are two fixed points to (3.47). We obtain $d(m_{G},\tilde{m}_{G})\leq c_{1}c_{2}d(m_{G},\tilde{m}_{G})$ , which implies $m_{G}=\tilde{m}_{G}$ .

The Nash equilibrium property follows from the best response property of $\phi_{\alpha}$ for a given vertex $\alpha$ . ∎

3.5. An Example on Lipschitz feedback

The main analysis in Section 3 relies on (H4) to ensure Lipschitz feedback. We provide a concrete model to check this assumption.

Example 3.13.

The dynamics and cost have

	$\displaystyle f_{0}(x,u,y)=f_{0}(x,y)u,\quad f(x,u,y)=f(x,y)u,$
	$\displaystyle l_{0}(x,u,y)=l_{1}(x,y)+{l}_{2}(x,y)u^{2},\quad l(x,u,y)=l_{3}(x,y)+l_{4}(x,y)u^{2},$

where $x,y\in\mathbb{R}$ and $u\in U=[a,b]$ . The functions $f_{0}$ , $f$ , $l_{1}$ , $l_{2}$ , $l_{3}$ , $l_{4}$ satisfy (H1)–(H3), and there exists $c_{0}>0$ such that $l_{2},l_{4}\geq c_{0}$ for all $x,y$ .

Given $\nu_{G}\in C([0,1],{\mathcal{P}}_{1}({\mathbb{R}}))$ , we check the minimizer of

\displaystyle S_{\alpha}^{\nu_{G}}(x,q)=\arg\min_{u\in U}\{q(f_{0}[x,\nu_{\alpha}]+f[x,\nu_{G};g_{\alpha}])u+(l_{2}[x,\nu_{\alpha}]+l_{4}[x,\nu_{G};g_{\alpha}])u^{2}\},

where $x,q\in\mathbb{R}$ .

Proposition 3.14.

Given any compact interval ${\mathcal{I}}$ , $S_{\alpha}^{\nu_{G}}(x,q)$ in Example 3.13 is a singleton and Lipschitz continuous in $(x,q)$ , where $x\in\mathbb{R}$ and $q\in{\mathcal{I}}$ , uniformly with respect to $(\nu_{G},\alpha)$ .

Proof.

Consider the function $\Phi(u)=u^{2}-2su$ , where $u\in U$ and $s$ is a parameter. Its minimum is attained at the unique point

\hat{u}=\Theta(s)\coloneqq\begin{cases}a&\mbox{if}\quad s\leq a,\\ s&\mbox{if}\quad a<s<b,\\ b&\mbox{if}\quad s\geq b.\end{cases}

Denote the function

h_{\alpha,\nu_{G}}(x)=-\frac{f_{0}[x,\mu_{\alpha}]+f[x,\nu_{G};g_{\alpha}]}{2(l_{2}[x,\mu_{\alpha}]+l_{4}[x,\nu_{G};g_{\alpha}])}.

By elementary estimates we can show

\displaystyle|h_{\alpha,\nu_{G}}(x)-h_{\alpha,\nu_{G}}(y)|\leq C_{0}|x-y|,

where $C_{0}$ does not depend on $(\nu_{G},\alpha)$ . We have

	$\displaystyle S_{\alpha}^{\nu_{G}}(x,q)$	$\displaystyle=\arg\min_{u}(u^{2}-2qh_{\alpha,\nu_{G}}(x)u)$
		$\displaystyle=\Theta({qh_{\alpha,\nu_{G}}(x)}).$

It is clear that $S_{\alpha}^{\nu_{G}}(x,q)$ is a continuous function of $(x,q)$ . For $(x_{i},q_{i})\in\mathbb{R}\times{\mathcal{I}}$ , $i=1,2$ ,

	$\displaystyle\|S_{\alpha}^{\nu_{G}}(x_{1},q_{1})-S_{\alpha}^{\nu_{G}}(x_{2},q_{2})\|$
	$\displaystyle\leq{\rm Lip}(\Theta)\|q_{1}h_{\alpha,\nu_{G}}(x_{1})-q_{2}h_{\alpha,\nu_{G}}(x_{2})\|$
	$\displaystyle\leq{\rm Lip}(\Theta)\Big{(}\|q_{1}-q_{2}\|\sup_{x}\|h_{\alpha,\nu_{G}}(x)\|+C_{0}\|x_{1}-x_{2}\|\|q_{2}\|\Big{)}.$

In fact, the Lipschitz constant ${\rm Lip}(\Theta)=1$ . Note that there exists a fixed constant $C$ such that $|h_{\alpha,\nu_{G}}(x)|\leq C$ for all $\alpha,\nu_{G}$ . This proves the proposition. ∎

If (H1)–(H3) and (H5) hold for Example 3.13, they further imply (H4) and (H6) so that the best response is Lipschitz continuous in $x$ by Lemma 3.3 and Proposition 3.14.

4. Performance Analysis

In the MFG case it is shown [25, 8] that the joint strategy $\{u^{o}_{i}(t)=\varphi_{i}(t,x_{i}(t)|\mu_{\cdot}),1\leq i\leq N\}$ yields an $\epsilon$ -Nash equilibrium, i.e. for all $\epsilon>0$ , there exists $N(\epsilon)$ such that for all $N\geq N(\epsilon)$

(4.1)

J_{i}^{N}({u}_{i}^{\circ},{u}_{-i}^{\circ})-\epsilon\leq\inf_{u_{i}\in\mathcal{U}_{i}}J_{i}^{N}(u_{i},{u}_{-i}^{\circ})\leq J_{i}^{N}({u}_{i}^{\circ},{u}_{-i}^{\circ}).

This form of approximate Nash equilibrium is a principal result of the MFG analyses in the sequence [25, 8, 40] and in many other studies. The importance of (4.1) is that it states that the cost function of any agent in a finite population can be reduced by at most $\epsilon$ if it changes unilaterally from the infinite population MFG feedback law while all other agents remain with the infinite population based control strategies. The main result of this section is that the same property holds for GMFG systems.

Throughout this section, let $\mu_{G}(\cdot)$ be solved from the GMFG equations (3.2) and (3.15).

4.1. The $\epsilon$ -Nash Equilibrium

The analysis of GMFG systems as limits of finite objects necessarily involves the consideration of graph limits and double limits in population and graph order. A corresponding set of assumptions is given below.

(H9) $M_{k}\to\infty$ and $\min_{1\leq l\leq M_{k}}|\mathcal{C}_{l}|\to\infty$ as $k\to\infty$ .

(H10) All agents have i.i.d. initial states with distribution $\mu_{0}^{x}$ and $E|x_{i}(0)|\leq C_{0}$ .

Remark 4.1.

(H10) is a simplifying assumption to keep further notation light. It may be generalized to $\alpha$ dependent initial distributions.

(H11) The sequence $\{G_{k};1\leq k<\infty\}$ and the graphon limit satisfy

\lim_{k\to\infty}\max_{i}\sum_{j=1}^{M_{k}}\Big{|}\frac{1}{M_{k}}g_{\mathcal{C}_{i}\mathcal{C}_{j}}^{k}-\int_{\beta\in I_{j}}g_{I_{i}^{*},\beta}d\beta\Big{|}=0,

where $I^{*}_{i}$ is the midpoint of the subinterval $I_{i}\in\{I_{1},\ldots,I_{M_{k}}\}$ of length $1/{M_{k}}$ .

Remark 4.2.

Assumption (H11) specifies the nature of the approximation error between $g^{k}$ for the finite graph and the graphon function $g$ .

The next proposition shows that under (H5) and (H11), the limit $g$ is well determined.

Proposition 4.3.

For the given sequence $\{g^{k},k\geq 1\}$ under (H9), if there exists a graphon $g$ satisfying (H5) and (H11), then it is unique.

Proof.

Assume there is another graphon $\hat{g}$ satisfying (H5) and (H11). Fix any $\epsilon>0$ and any $\mathcal{S}\times\mathcal{T}\subset[0,1]\times[0,1]$ . By Lemma A.2, there exists a sufficiently large $k_{0}$ (depending on $\epsilon$ , ${\mathcal{S}}$ and ${\mathcal{T}}$ ), such that for both $g$ and $\hat{g}$ we have

\displaystyle\Big{|}\int_{\mathcal{S}\times\mathcal{T}}(g^{k^{0}}-g)dxdy\Big{|}\leq\epsilon,\qquad\Big{|}\int_{\mathcal{S}\times\mathcal{T}}(g^{k^{0}}-\hat{g})dxdy\Big{|}\leq\epsilon.

Hence

\displaystyle\Big{|}\int_{\mathcal{S}\times\mathcal{T}}(g-\hat{g})dxdy\Big{|}\leq 2\epsilon.

Since $\mathcal{S}\times\mathcal{T}$ is arbitrary, we have $\|g-\hat{g}\|_{\Box}\leq 2\epsilon.$ Since $\epsilon$ is arbitrary, we have $\|g-\hat{g}\|_{\Box}=0.$ But the cut norm is a norm, so we have $g=\hat{g}$ . ∎

For the $\epsilon$ -Nash equilibrium analysis, we consider a sequence of games each defined on a finite graph $G_{k}$ . Recall that there is a total of $N=\sum_{l=1}^{M_{k}}|\mathcal{C}_{l}|$ agents.

Suppose the cluster $\mathcal{C}(i)$ of agent ${\mathcal{A}}_{i}$ corresponds to the subinterval $I(i)\in\{I_{1},\ldots,I_{M_{k}}\}$ . The agent ${\mathcal{A}}_{i}$ takes the midpoint $I^{*}(i)$ of the subinterval $I(i)$ and uses the GMFG equations to determine its control law

(4.2)

\displaystyle\hat{u}_{i}=\varphi(t,x_{i}|\mu_{G}(\cdot);g_{I^{*}(i)}),\quad 1\leq i\leq N,

which we simply write as $\varphi(t,x_{i},g_{I^{*}(i)})$ . Denote the resulting state process by $\hat{x}_{i}$ , $1\leq i\leq N$ . Recall that

	$\displaystyle f_{0}(x_{i}^{N},u_{i}^{N},{\mathcal{C}}(i))=\frac{1}{\|\mathcal{C}(i)\|}\sum_{j\in\mathcal{C}(i)}f(x_{i}^{N},u_{i}^{N},x_{j}^{N}),$
	$\displaystyle f_{G_{k}}(x_{i}^{N},u_{i}^{N},g_{\mathcal{C}(i)}^{k})=\frac{1}{M_{k}}\sum_{l=1}^{M_{k}}g^{k}_{\mathcal{C}(i)\mathcal{C}_{l}}\frac{1}{\|\mathcal{C}_{l}\|}\sum_{j\in\mathcal{C}_{l}}f(x_{i}^{N},u_{i}^{N},x_{j}^{N}),$

where the superscript $N$ is added to indicate the population size. The closed-loop system of $N$ agents on the finite graph $G_{k}$ under the set of strategies (4.2) is given by

	$\displaystyle\mbox{\it System A:}\quad d\hat{x}_{i}^{N}=$	$\displaystyle f_{0}(\hat{x}_{i}^{N},\varphi(t,\hat{x}_{i}^{N},g_{I^{*}(i)}),{\mathcal{C}}(i))dt$
(4.3)			$\displaystyle+f_{G_{k}}(\hat{x}_{i}^{N},\varphi(t,\hat{x}_{i}^{N},g_{I^{*}(i)}),g_{\mathcal{C}(i)}^{k})dt+\sigma dw_{i},$

where $1\leq i\leq N$ and $\hat{x}_{i}^{N}(0)=x_{i}^{N}(0)$ . Note that $g_{\mathcal{C}(i)}^{k}$ appears in $f_{G_{k}}$ as determined by the finite population system dynamics. We state the following main result.

Theorem 4.4.

( $\epsilon$ -Nash equilibrium) Assume (H1)–(H11) hold. Then when the strategies (4.2) determined by the GMFG equations (3.2) and (3.15) are applied to a sequence of finite graph systems $\{G_{k};1\leq k<\infty\}$ , the $\epsilon$ -Nash equilibrium property holds where $\epsilon\to 0$ as $k\to\infty$ , and where the unilateral agent ${\mathcal{A}}_{i}$ uses a centralized Lipschitz feedback strategy $\psi(t,x_{i},x_{-i})$ , where $x_{-i}$ denotes the set of states of all other agents.

We first explain the basic idea for the demonstration of the $\epsilon$ -Nash equilibrium property. Suppose all other players, except agent ${\mathcal{A}}_{\iota}$ , employ the control strategies based on the GMFG equation system. When ${\mathcal{A}}_{\iota}$ employs a different strategy, the resulting change in its performance can be measured using a limiting stochastic control problem where both the system dynamics and the cost are subject to small perturbation due to the mean field approximation of the effects of all other agents. The proof is technical and preceded by some lemmas.

4.2. Proof of Theorem 4.4

Suppose $x_{\iota}^{N}$ is determined from a general feedback control law $u_{\iota}^{N}$ instead of the GMFG best response. With the exception of agent ${\mathcal{A}}_{\iota}$ with its unilateral strategy, all other agents ${\mathcal{A}}_{j}$ , $j\neq{\iota}$ , still have strategies determined by (4.2). We introduce the system:

(4.4)

\displaystyle\mbox{\it System B:}\quad\begin{cases}dx_{\iota}^{N}=f_{0}(x_{\iota}^{N},u_{\iota}^{N},\mathcal{C}(\iota))dt+f_{G_{k}}(x_{\iota}^{N},u_{\iota}^{N},g_{\mathcal{C}(\iota)}^{k})dt+\sigma dw_{\iota},\\ dx_{j}^{N}=f_{0}(x_{j}^{N},\varphi(t,x_{j}^{N},g_{I^{*}(j)}),\mathcal{C}(j))dt\\ \qquad\qquad+f_{G_{k}}(x_{j}^{N},\varphi(t,x_{j}^{N},g_{I^{*}(j)}),g_{\mathcal{C}(j)}^{k})dt+\sigma dw_{j},\\ \qquad\qquad j\neq{\iota},\quad 1\leq j\leq N.\end{cases}

We note that $x_{j}^{N}$ is affected by the unilateral choice of strategy by ${\mathcal{A}}_{\iota}$ due to the coupling in $f_{0}$ and $f_{G_{k}}$ . For this reason, $x_{j}^{N}$ differs from $\hat{x}_{j}^{N}$ in (4.3) although the control law of ${\mathcal{A}}_{j}$ , $j\neq\iota$ , remains the same. The central task is to estimate by how much ${\mathcal{A}}_{\iota}$ can reduce its cost.

To facilitate the performance estimate in System $B$ , we introduce two auxiliary systems below. Consider

	$\displaystyle\mbox{\it System C:}\quad dy_{i}^{N}=\$	$\displaystyle\int_{\mathbb{R}}f_{0}(y_{i}^{N},\varphi(t,y_{i}^{N},g_{I^{*}(i)}),z)m_{y_{i}^{N}}(dz)dt$
		$\displaystyle+\frac{1}{M_{k}}\sum_{l=1}^{M_{k}}g^{k}_{\mathcal{C}(i)\mathcal{C}_{l}}\frac{1}{\|\mathcal{C}_{l}\|}\sum_{j\in\mathcal{C}_{l}}\int_{\mathbb{R}}f(y_{i}^{N},\varphi(t,y_{i}^{N},g_{I^{*}(i)}),z)m_{y_{j}^{N}}(dz)dt$
		$\displaystyle+\sigma dw_{i}$
	$\displaystyle=\$	$\displaystyle\int_{\mathbb{R}}f_{0}(y_{i}^{N},\varphi(t,y_{i}^{N},g_{I^{*}(i)}),z)m_{y_{i}^{N}}(dz)dt$
		$\displaystyle+\frac{1}{M_{k}}\sum_{l=1}^{M_{k}}g^{k}_{\mathcal{C}(i)\mathcal{C}_{l}}\int f(y_{i}^{N},\varphi(t,y_{i}^{N},g_{I^{*}(i)}),z)m_{l}^{N}(t,dz)dt$
(4.5)			$\displaystyle+\sigma dw_{i},$

where $1\leq i\leq N$ and $y_{i}^{N}(0)=x_{i}^{N}(0)$ , and $m_{y_{j}^{N}(t)}$ denotes the law of $y_{j}^{N}(t)$ . Each Brownian motion $w_{i}$ is the same as in (4.3). The second equality holds since all processes in cluster $\mathcal{C}_{l}$ have the same distribution denoted by $m_{l}^{N}(t,dz)$ at time $t$ . It is clear that the processes $y_{1}^{N},\ldots,y_{N}^{N}$ are independent, and $\{y_{j}^{N},j\in\mathcal{C}_{l}\}$ are i.i.d. for any given $l$ .

Next we introduce

(4.6)

\displaystyle\mbox{\it System D:}\quad dy_{i}^{\infty}(t)

\displaystyle=\widetilde{f}[y_{i}^{\infty}(t),\varphi(t,y_{i}^{\infty}(t),g_{I^{*}(i)}),\mu_{G}(t);g_{I^{*}(i)\ }]dt+\sigma dw_{i}(t),

where $1\leq i\leq N$ and $y_{i}^{\infty}(0)=x_{i}^{N}(0)$ . Here $w_{i}$ is the same as in (4.3). The process $y_{i}^{\infty}$ is generated by the closed-loop dynamics for an agent at the node $I^{*}(i)$ associated with the cluster $\mathcal{C}(i)$ using the GMFG based control law (4.2) while situated in an infinite population represented by the ensemble $\mu_{G}(\cdot)$ of local mean fields. We view (4.6) as an instance of the generic equation (3.10) under the control law (4.2). By Theorem 3.12, $y^{\infty}_{i}(t)$ has the law $\mu_{I^{*}(i)}(t)$ . Note that if $j\in\mathcal{C}(i)$ , $y_{i}^{\infty}$ and $y_{j}^{\infty}$ are two processes of the same distribution.

We shall denote the $A$ to $C$ system deviation by $\epsilon_{1,N}$ , the $C$ to $D$ deviation by $\epsilon_{2,N}$ and the (non-unilateral agent) $B$ to $D$ deviation by $\epsilon_{3,N}$ . Specifically, we set

	$\displaystyle\epsilon_{1,N}=\sup_{i\leq N,t}E\|\hat{x}_{i}^{N}(t)-y_{i}^{N}(t)\|,\qquad\epsilon_{2,N}=\sup_{i\leq N,t}E\|y_{i}^{N}(t)-y_{i}^{\infty}(t)\|,$
	$\displaystyle\epsilon_{3,N}=\sup_{u_{\iota}^{N},t,\iota\neq j\leq N}E\|x_{j}^{N}(t)-y_{j}^{\infty}(t)\|,$

where $x_{j}^{N}$ is given by (4.4).

Lemma 4.5.

The SDE system (4.5) has a unique solution $(y_{1}^{N},\ldots,y_{N}^{N})$ .

Proof.

The proof is similar to [25, Theorem 6]. ∎

Lemma 4.6.

$\epsilon_{1,N}\to 0$ as $N\to\infty$ (due to $k\to\infty$ ).

Proof.

We write

(4.7)		$\displaystyle\hat{x}_{i}^{N}(t)-y_{i}^{N}(t)=$	$\displaystyle\int_{0}^{t}\frac{1}{\|\mathcal{C}(i)\|}\sum_{j\in\mathcal{C}(i)}\xi_{ij}^{0}(s)ds$
		$\displaystyle+\int_{0}^{t}\frac{1}{M_{k}}\sum_{l=1}^{M_{k}}g_{\mathcal{C}(i)\mathcal{C}_{l}}^{k}\frac{1}{\|\mathcal{C}_{l}\|}\sum_{j\in\mathcal{C}_{l}}\xi_{ij}(s)ds,$

where

	$\displaystyle\xi_{ij}^{0}(s)=f_{0}(\hat{x}_{i}^{N},\varphi(s,\hat{x}_{i}^{N},g_{I^{}(i)}),\hat{x}_{j}^{N})-\int_{\mathbb{R}}f_{0}(y_{i}^{N},\varphi(s,y_{i}^{N},g_{I^{}(i)}),z)m_{y_{j}^{N}(s)}(dz),$
	$\displaystyle\xi_{ij}(s)=f(\hat{x}_{i}^{N},\varphi(s,\hat{x}_{i}^{N},g_{I^{}(i)}),\hat{x}_{j}^{N})-\int_{\mathbb{R}}f(y_{i}^{N},\varphi(s,y_{i}^{N},g_{I^{}(i)}),z)m_{y_{j}^{N}(s)}(dz).$

We check the second line of (4.7) first. Write

	$\displaystyle\xi_{ij}(s)=$	$\displaystyle f(\hat{x}_{i}^{N},\varphi(s,\hat{x}_{i}^{N},g_{I^{}(i)}),\hat{x}_{j}^{N})-f(y_{i}^{N},\varphi(s,y_{i}^{N},g_{I^{}(i)}),y_{j}^{N})$
		$\displaystyle+f(y_{i}^{N},\varphi(s,y_{i}^{N},g_{I^{}(i)}),y_{j}^{N})-\int_{\mathbb{R}}f(y_{i}^{N},\varphi(s,y_{i}^{N},g_{I^{}(i)}),z)m_{y_{j}^{N}(s)}(dz).$

Denote

\zeta_{ij}=f(y_{i}^{N},\varphi(s,y_{i}^{N},g_{I^{*}(i)}),y_{j}^{N})-\int_{\mathbb{R}}f(y_{i}^{N},\varphi(s,y_{i}^{N},g_{I^{*}(i)}),z)m_{y_{j}^{N}(s)}(dz).

By the Lipschitz conditions (H2), (H3) and the best response’s uniform Lipschitz continuity in $x$ by Lemma 3.7, we obtain

		$\displaystyle\Big{\|}\frac{1}{M_{k}}\sum_{l=1}^{M_{k}}g_{\mathcal{C}(i)\mathcal{C}_{l}}^{k}\frac{1}{\|\mathcal{C}_{l}\|}\sum_{j\in\mathcal{C}_{l}}\xi_{ij}(s)\Big{\|}$
	$\displaystyle\leq$	$\displaystyle C\|\hat{x}_{i}^{N}-y_{i}^{N}\|+\frac{C}{M_{k}}\sum_{l=1}^{M_{k}}g_{\mathcal{C}(i)\mathcal{C}_{l}}^{k}\frac{1}{\|\mathcal{C}_{l}\|}\sum_{j\in\mathcal{C}_{l}}\|\hat{x}_{j}^{N}-y_{j}^{N}\|$
		$\displaystyle+\Big{\|}\frac{1}{M_{k}}\sum_{l=1}^{M_{k}}g_{\mathcal{C}(i)\mathcal{C}_{l}}^{k}\frac{1}{\|\mathcal{C}_{l}\|}\sum_{j\in\mathcal{C}_{l}}\zeta_{ij}\Big{\|}.$

Then by independence of $y_{i}^{N}$ , $1\leq i\leq N$ ,

	$\displaystyle E\Big{\|}\frac{1}{M_{k}}\sum_{l=1}^{M_{k}}g_{\mathcal{C}(i)\mathcal{C}_{l}}^{k}\frac{1}{\|\mathcal{C}_{l}\|}\sum_{j\in\mathcal{C}_{l}}\zeta_{ij}\Big{\|}^{2}$	$\displaystyle\leq C\sum_{l=1}^{M_{k}}\sum_{j\in\mathcal{C}_{l}}\frac{\|g_{\mathcal{C}(i)\mathcal{C}_{l}}^{k}\|^{2}}{M_{k}^{2}\|\mathcal{C}_{l}\|^{2}}$
		$\displaystyle\leq\frac{C}{M_{k}\min_{l}\|\mathcal{C}_{l}\|}.$

The estimate for $\frac{1}{|\mathcal{C}(i)|}\sum_{j\in\mathcal{C}(i)}\xi_{ij}^{0}(s)$ can be obtained similarly. Now it follows from (4.7) that

	$\displaystyle E\|\hat{x}_{i}^{N}(t)-y_{i}^{N}(t)\|\leq C\int_{0}^{t}E\|\hat{x}_{i}^{N}(s)-y_{i}^{N}(s)\|ds$
	$\displaystyle+\frac{C}{M_{k}}\sum_{l=1}^{M_{k}}\frac{g_{\mathcal{C}(i)\mathcal{C}_{l}}^{k}}{\|\mathcal{C}_{l}\|}\sum_{j\in\mathcal{C}_{l}}\int_{0}^{t}E\|\hat{x}_{j}^{N}(s)-y_{j}^{N}(s)\|ds$
	$\displaystyle+\frac{C}{\|\mathcal{C}(i)\|}\sum_{j\in\mathcal{C}(i)}\int_{0}^{t}E\|\hat{x}_{j}^{N}(s)-y_{j}^{N}(s)\|ds+\frac{C_{1}}{\sqrt{M_{k}\min_{l}\|\mathcal{C}_{l}\|}}+\frac{C}{\sqrt{\|\mathcal{C}(i)\|}}$
	$\displaystyle\leq C_{2}\int_{0}^{t}\Delta^{N}(s)ds+\frac{C_{3}}{\sqrt{\min_{l}\|\mathcal{C}_{l}\|}},$

where $\Delta^{N}(t)=\max_{1\leq i\leq N}E|\hat{x}_{i}^{N}(t)-y_{i}^{N}(t)|.$ The above further implies

\displaystyle\Delta^{N}(t)\leq C_{2}\int_{0}^{t}\Delta^{N}(s)ds+\frac{C_{3}}{\sqrt{\min_{l}|\mathcal{C}_{l}|}}.

The lemma follows from (H9) and Gronwall’s lemma. ∎

Lemma 4.7.

We have $\epsilon_{2,N}\to 0$ as $N\to\infty$ .

Proof.

For System $D$ and $1\leq i\leq N$ , $y_{i}^{\infty}(t)$ has the law $\mu_{I^{*}(i)}(t)$ and we write

(4.8)		$\displaystyle dy_{i}^{\infty}=\$	$\displaystyle\int_{\mathbb{R}}f_{0}(y_{i}^{\infty},\varphi(t,y_{i}^{\infty},g_{I^{}(i)}),z)\mu_{I^{}(i)}(t,dz)dt+\sigma dw_{i}$
		$\displaystyle+\int_{0}^{1}\int_{\mathbb{R}}f(y_{i}^{\infty},\varphi(t,y_{i}^{\infty},g_{I^{}(i)}),z)g(I^{}(i),\beta)\mu_{\beta}(t,dz)d\beta\ dt.$

Set

		$\displaystyle\int_{0}^{1}\int_{\mathbb{R}}f(y_{i}^{\infty},\varphi(t,y_{i}^{\infty},g_{I^{}(i)}),z)g(I^{}(i),\beta)\mu_{\beta}(t,dz)d\beta$
	$\displaystyle=\$	$\displaystyle\sum_{l=1}^{M_{k}}\int_{\beta\in I_{l}}\int_{\mathbb{R}}f(y_{i}^{\infty},\varphi(t,y_{i}^{\infty},g_{I^{}(i)}),z)g(I^{}(i),\beta)\mu_{\beta}(t,dz)d\beta$
	$\displaystyle\eqqcolon$	$\displaystyle\ \xi_{k}^{i}+\zeta_{k}^{i},$

where

	$\displaystyle\xi_{k}^{i}=\sum_{l=1}^{M_{k}}\int_{\beta\in I_{l}}g(I^{}(i),\beta)d\beta\int_{\mathbb{R}}f(y_{i}^{\infty},\varphi(t,y_{i}^{\infty},g_{I^{}(i)}),z)\mu_{I_{l}^{*}}(t,dz),$
	$\displaystyle\zeta_{k}^{i}=\sum_{l=1}^{M_{k}}\zeta^{i}_{kl},$
(4.9)		$\displaystyle\zeta_{kl}^{i}\coloneqq\int_{\beta\in I_{l}}\int_{\mathbb{R}}f(y_{i}^{\infty},\varphi(t,y_{i}^{\infty},g_{I^{}(i)}),z)g(I^{}(i),\beta)[\mu_{\beta}(t,dz)-\mu_{I_{l}^{*}}(t,dz)]d\beta.$

We rewrite

	$\displaystyle\xi_{k}^{i}=$	$\displaystyle\sum_{l=1}^{M_{k}}\frac{g^{k}_{\mathcal{C}(i)\mathcal{C}_{l}}}{M_{k}}\int_{\mathbb{R}}f(y_{i}^{\infty},\varphi(t,y_{i}^{\infty},g_{I^{}(i)}),z)\mu_{I_{l}^{}}(t,dz)$
		$\displaystyle+\sum_{l=1}^{M_{k}}\left[\int_{\beta\in I_{l}}g(I^{}(i),\beta)d\beta-\frac{g^{k}_{\mathcal{C}(i)\mathcal{C}_{l}}}{M_{k}}\right]\int_{\mathbb{R}}f(y_{i}^{\infty},\varphi(t,y_{i}^{\infty},g_{I^{}(i)}),z)\mu_{I_{l}^{*}}(t,dz)$
	$\displaystyle\eqqcolon\$	$\displaystyle\xi^{i}_{k,1}+\xi^{i}_{k,2}.$

By (H11) and boundedness of $f$ , we have $\lim_{k\to\infty}\sup_{t,\omega}\max_{1\leq i\leq N}|\xi^{i}_{k,2}|=0$ so that

(4.10)

\displaystyle\lim_{k\to\infty}\max_{i}\int_{0}^{T}E|\xi^{i}_{k,2}(t)|dt=0.

Now (4.8) may be rewritten in the form

	$\displaystyle dy_{i}^{\infty}=\$	$\displaystyle\int_{\mathbb{R}}f_{0}(y_{i}^{\infty},\varphi(t,y_{i}^{\infty},g_{I^{}(i)}),z)\mu_{I^{}(i)}(t,dz)dt+\sigma dw_{i}$
		$\displaystyle+(\xi^{i}_{k,1}+\xi^{i}_{k,2}+\zeta^{i}_{k})dt.$

In view of (4.5), we have

		$\displaystyle y_{i}^{\infty}(t)-y_{i}^{N}(t)$
	$\displaystyle=$	$\displaystyle\int_{0}^{t}\int_{\mathbb{R}}[f_{0}(y_{i}^{\infty},\varphi(s,y_{i}^{\infty},g_{I^{}(i)}),z)\mu_{I^{}(i)}(s,dz)-f_{0}(y_{i}^{N},\varphi(s,y_{i}^{N},g_{I^{*}(i)}),z)m_{y^{N}_{i}(s)}(dz)]ds$
		$\displaystyle+\frac{1}{M_{k}}\sum_{l=1}^{M_{k}}g^{k}_{\mathcal{C}(i)\mathcal{C}_{l}}\int_{0}^{t}\int_{\mathbb{R}}f(y_{i}^{\infty},\varphi(s,y_{i}^{\infty},g_{I^{}(i)}),z)\mu_{I_{l}^{}}(s,dz)ds$
		$\displaystyle-\frac{1}{M_{k}}\sum_{l=1}^{M_{k}}g^{k}_{\mathcal{C}(i)\mathcal{C}_{l}}\int_{0}^{t}\int_{\mathbb{R}}f(y_{i}^{N},\varphi(s,y_{i}^{N},g_{I^{*}(i)}),z)m_{l}^{N}(s,dz)ds$
		$\displaystyle+\int_{0}^{t}(\xi^{i}_{k,2}+\zeta^{i}_{k})ds.$

Denote

	$\displaystyle\Delta_{il}(s)=$	$\displaystyle\Big{\|}\int_{\mathbb{R}}f(y_{i}^{\infty},\varphi(s,y_{i}^{\infty},g_{I^{}(i)}),z)\mu_{I_{l}^{}}(s,dz)$
		$\displaystyle-\int_{\mathbb{R}}f(y_{i}^{N},\varphi(s,y_{i}^{N},g_{I^{*}(i)}),z)m_{l}^{N}(s,dz)\Big{\|}.$

It follows that

	$\displaystyle\Delta_{il}(s)\leq$	$\displaystyle\Big{\|}\int_{\mathbb{R}}f(y_{i}^{\infty},\varphi(s,y_{i}^{\infty},g_{I^{}(i)}),z)\mu_{I_{l}^{}}(s,dz)$
		$\displaystyle-\int f(y_{i}^{N},\varphi(s,y_{i}^{N},g_{I^{}(i)}),z)\mu_{I_{l}^{}}(s,dz)\Big{\|}$
		$\displaystyle+\Big{\|}\int_{\mathbb{R}}f(y_{i}^{N},\varphi(s,y_{i}^{N},g_{I^{}(i)}),z)\mu_{I_{l}^{}}(s,dz)$
		$\displaystyle-\int_{\mathbb{R}}f(y_{i}^{N},\varphi(s,y_{i}^{N},g_{I^{*}(i)}),z)m_{l}^{N}(s,dz)\Big{\|}$
	$\displaystyle\eqqcolon$	$\displaystyle\ \Delta_{il1}(s)+\Delta_{il2}(s).$

By the Lipschitz condition (H2), for any fixed $y\in\mathbb{R}$ , we have

		$\displaystyle\Big{\|}\int_{\mathbb{R}}f(y,\varphi(s,y,g_{I^{}(i)}),z)\mu_{I_{l}^{}}(s,dz)-\int_{\mathbb{R}}f(y,\varphi(s,y,g_{I^{*}(i)}),z)m_{l}^{N}(s,dz)\Big{\|}$
	$\displaystyle=$	$\displaystyle\|Ef(y,\varphi(s,y,g_{I^{}(i)}),y_{j}^{\infty})-Ef(y,\varphi(s,y,g_{I^{}(i)}),y_{j}^{N})\|$
	$\displaystyle\leq$	$\displaystyle CE\|y_{j}^{\infty}(s)-y_{j}^{N}(s)\|,$

where $j\in\mathcal{C}_{l}$ and we have used the fact that $y_{i}^{\infty}(t)$ in (4.8) has the law $\mu_{I^{*}(i)}(t)$ and that $y_{j}^{N}(t)$ has the law $m_{l}^{N}(t)$ . Consequently, we have for $j\in\mathcal{C}_{l}$ , with probability one,

(4.11)

\displaystyle\Delta_{il2}(s)\leq CE|y_{j}^{\infty}(s)-y_{j}^{N}(s)|.

We estimate $\Delta_{kl1}$ using the Lipschitz property of $f$ and $\varphi_{I^{*}(i)}$ . Now it follows that

\displaystyle E\Delta_{il}(s)\leq CE|y_{i}^{\infty}(s)-y_{i}^{N}(s)|+CE|y_{j}^{\infty}(s)-y_{j}^{N}(s)|,\quad j\in\mathcal{C}_{l}.

We similarly estimate the difference term involving $f_{0}$ . Therefore,

	$\displaystyle E\|y_{i}^{\infty}(t)-y_{i}^{N}(t)\|\leq\$	$\displaystyle C\int_{0}^{t}E\|y_{i}^{\infty}-y_{i}^{N}\|ds+\int_{0}^{t}E(\|\xi^{i}_{k,2}\|+\|\zeta^{i}_{k}\|)ds$
		$\displaystyle+\frac{1}{M_{k}}\sum_{l=1}^{M_{k}}g^{k}_{\mathcal{C}(i)\mathcal{C}_{l}}\int_{0}^{t}E\Delta_{il}ds$
	$\displaystyle\leq$	$\displaystyle C_{1}\int_{0}^{t}\max_{i}E\|y_{i}^{\infty}-y_{i}^{N}\|ds+\int_{0}^{t}E(\|\xi^{i}_{k,2}\|+\|\zeta^{i}_{k}\|)ds$
		$\displaystyle+\frac{C}{M_{k}}\sum_{l=1}^{M_{k}}g^{k}_{\mathcal{C}(i)\mathcal{C}_{l}}\int_{0}^{t}\max_{j}E\|y_{j}^{\infty}-y_{j}^{N}\|ds$
	$\displaystyle\leq$	$\displaystyle 2C_{2}\int_{0}^{t}\max_{i}E\|y_{i}^{\infty}-y_{i}^{N}\|ds+\int_{0}^{t}E(\|\xi^{i}_{k,2}\|+\|\zeta^{i}_{k}\|)ds.$

Consequently,

\displaystyle\max_{i}E|y_{i}^{\infty}(t)-y_{i}^{N}(t)|\leq\

\displaystyle 2C_{2}\int_{0}^{t}\max_{i}E|y_{i}^{\infty}-y_{i}^{N}|ds+\max_{i}\int_{0}^{t}E(|\xi^{i}_{k,2}|+|\zeta^{i}_{k}|)ds.

By Gronwall’s lemma,

(4.12)

\displaystyle\sup_{0\leq t\leq T}\max_{i}E|y_{i}^{\infty}(t)-y_{i}^{N}(t)|\leq C\max_{i}\int_{0}^{T}E(|\xi^{i}_{k,2}|+|\zeta^{i}_{k}|)ds.

To estimate (4.9), by (H2) we derive

	$\displaystyle\zeta^{i}_{kl,\beta}\coloneqq$	$\displaystyle\Big{\|}\int_{\mathbb{R}}f(y_{i}^{\infty},\varphi(t,y_{i}^{\infty},g_{I^{}(i)}),z)[\mu_{\beta}(t,dz)-\mu_{I_{l}^{}}(t,dz)]\Big{\|}$
	$\displaystyle=$	$\displaystyle\Big{\|}\int_{\mathbb{R}^{2}}[f(y_{i}^{\infty},\varphi(t,y_{i}^{\infty},g_{I^{}(i)}),z_{1})-f(y_{i}^{\infty},\varphi(t,y_{i}^{\infty},g_{I^{}(i)}),z_{2})]\widehat{\gamma}(dz_{1},dz_{2})\Big{\|}$
	$\displaystyle\leq$	$\displaystyle C\int_{\mathbb{R}^{2}}\|z_{1}-z_{2}\|\widehat{\gamma}(dz_{1},dz_{2}),$

where the probability measure $\widehat{\gamma}$ is any coupling of $\mu_{\beta}(t)$ and $\mu_{I_{l}^{*}}(t)$ and $C$ is the Lipschitz constant of $f$ . Since the coupling $\widehat{\gamma}$ is arbitrary, we have $\zeta^{i}_{kl,\beta}\leq CW_{1}(\mu_{\beta}(t),\mu_{I^{*}(i)}(t))$ . Denote $\delta_{k}^{\mu}=\sup_{l\leq M_{k}}\sup_{\beta\in I_{l},t\leq T}W_{1}(\mu_{\beta}(t),\mu_{I_{l}^{*}}(t)).$ Then with probability one,

\displaystyle|\zeta^{i}_{kl}(t)|\leq C\delta^{\mu}_{k}/M_{k}

in view of (4.9), and therefore $\max_{i}|\zeta^{i}_{k}(t)|\leq C\delta_{k}^{\mu}$ . Note that $\delta^{\mu}_{k}\to 0$ as $k\to\infty$ by Lemma A.1. Recalling (4.10), the right hand side of (4.12) tends to 0 as $k\to\infty$ . This completes the proof. ∎

Lemma 4.8.

$\lim_{N\to\infty}\sup_{t,i\leq N}E|\hat{x}_{i}^{N}-y_{i}^{\infty}|=0$ .

Proof.

The lemma follows from Lemmas 4.6 and 4.7. ∎

Lemma 4.9.

$\lim_{N\to\infty}\epsilon_{3,N}=0.$

Proof.

For $(\hat{x}_{1}^{N},\ldots,\hat{x}_{N}^{N})$ in System $A$ and $(x_{1}^{N},\ldots,x_{N}^{N})$ in System $B$ , we compare the SDEs of $\hat{x}_{j}^{N}$ and $x_{j}^{N}$ and apply Gronwall’s lemma to obtain

\sup_{u_{\iota}^{N},t,j\neq\iota}|x_{j}^{N}-\hat{x}_{j}^{N}|\leq\frac{C}{{\min_{l}|\mathcal{C}_{l}|}}.

Next by Lemma 4.8, we obtain the desired estimate. ∎

Consider the limiting optimal control problem with dynamics and cost

(4.13)		$\displaystyle dx_{\iota}^{\infty}=\widetilde{f}[x_{\iota}^{\infty},u_{\iota},\mu_{G};g_{I^{*}({\iota})}]dt+\sigma dw_{\iota},$
(4.14)		$\displaystyle J_{\iota}^{}=E\int_{0}^{T}\widetilde{l}[x_{\iota}^{\infty},u_{\iota},\mu_{G};g_{I^{}({\iota})}]dt,$

where $x_{\iota}^{\infty}(0)=x_{\iota}^{N}(0)$ and $\mu_{G}(\cdot)$ is given by the GMFG equation system.

To establish the $\epsilon$ -Nash equilibrium property, the cost of agent ${\mathcal{A}}_{\iota}$ within the $N$ agents can be written using the mean field limit dynamics and cost, both involving $\mu_{G}(\cdot)$ , up to a small error term that can be bounded uniformly with respect to $u_{\iota}^{N}$ , while ${\mathcal{A}}_{\iota}$ chooses its control $u_{\iota}^{N}$ . It can further have little improvement due to the best response property of $\varphi(t,x_{\iota}|\mu_{G}(\cdot);g_{I^{*}({\iota})})$ within the mean field limit. We rewrite the first equation in (4.4) of System $B$ as

(4.15)

\displaystyle dx_{\iota}^{N}=\widetilde{f}[x_{\iota}^{N},u_{\iota}^{N},\mu_{G};g_{I^{*}({\iota})}]dt+(\delta_{f_{0}}^{k}(t)+\delta_{f}^{k}(t))dt+\sigma dw_{\iota},

where $\delta_{f_{0}}^{k}=f_{0}(x_{\iota}^{N},u_{\iota}^{N},{\mathcal{C}({\iota})})-f_{0}[x_{\iota}^{N},u_{\iota}^{N},\mu_{I^{*}({\iota})}]$ and $\delta_{f}^{k}=f_{G_{k}}(x_{\iota}^{N},u_{\iota}^{N},g_{\mathcal{C}({\iota})}^{k})-f[x_{\iota}^{N},u_{\iota}^{N},\mu_{G};g_{I^{*}({\iota})}]$ . Similarly the cost of ${\mathcal{A}}_{\iota}$ in System $B$ is written as

\displaystyle J_{\iota}^{N}(u_{\iota}^{N})=E\int_{0}^{T}(\widetilde{l}[x_{\iota}^{N},u_{\iota}^{N},\mu_{G};g_{I^{*}({\iota})}]+\delta_{l_{0}}^{k}(t)+\delta_{l}^{k}(t))dt,

where we have $\delta_{l_{0}}^{k}=l_{0}(x_{\iota}^{N},u_{\iota}^{N},{\mathcal{C}({\iota})})-l_{0}[x_{\iota}^{N},u_{\iota}^{N},\mu_{I^{*}({\iota})}]$ and $\delta_{l}^{k}=l_{G_{k}}(x_{\iota}^{N},u_{\iota}^{N},g_{\mathcal{C}({\iota})}^{k})-l[x_{\iota}^{N},u_{\iota}^{N},\mu_{G};g_{I^{*}({\iota})}]$ . Note that all other agents have applied the control laws $\varphi(t,x_{j}^{N},g_{I^{*}(j)})$ , $j\neq{\iota}$ . So we only indicate $u_{\iota}^{N}$ within $J_{\iota}^{N}$ . It is clear that $\delta_{f_{0}}^{k}$ , $\delta_{f}^{k}$ , $\delta_{l_{0}}^{k}$ , and $\delta_{l}^{k}$ are all affected by the control law $u_{\iota}^{N}$ . Let ${\mathbold y}^{\infty}_{t}=(y_{1}^{\infty}(t),\ldots,y_{N}^{\infty}(t))$ for System $D$ . Our next step is to derive a uniform upper bounded for $E|\delta_{f}^{k}|$ and $E|\delta_{l}^{k}|$ with respect to $u_{\iota}^{N}$ .

Define the two random variables

	$\displaystyle\Delta_{f}^{k}(z,u,{\mathbold y}^{\infty}_{t})=\frac{1}{M_{k}}\sum_{l=1}^{M_{k}}g^{k}_{{\mathcal{C}}({\iota}){\mathcal{C}}_{l}}\frac{1}{\|{\mathcal{C}}_{l}\|}\sum_{j\in{\mathcal{C}}_{l}}f(z,u,y_{j}^{\infty}(t))-f[z,u,\mu_{G}(t);g_{I^{*}({\iota})}],$
	$\displaystyle\Delta_{l}^{k}(z,u,{\mathbold y}^{\infty}_{t})=\frac{1}{M_{k}}\sum_{l=1}^{M_{k}}g^{k}_{{\mathcal{C}}({\iota}){\mathcal{C}}_{l}}\frac{1}{\|{\mathcal{C}}_{l}\|}\sum_{j\in{\mathcal{C}}_{l}}l(z,u,y_{j}^{\infty}(t))-{l}[z,u,\mu_{G}(t);g_{I^{*}({\iota})}],$

where $z\in\mathbb{R}$ and $u\in U$ are deterministic and fixed.

Lemma 4.10.

We have

(4.16)

\displaystyle\lim_{k\to\infty}\sup_{z,u,t}E(|\Delta_{f}^{k}(z,u,{\mathbold y}^{\infty}_{t})|^{2}+|\Delta_{l}^{k}(z,u,{\mathbold y}^{\infty}_{t})|^{2})=0.

Proof.

As in the proof of Lemma 4.7, we approximate $\mu_{\beta}$ , $\beta\in[0,1]$ , by using a finite number of points of $\beta$ , and next expand the two quadratic terms in (4.16). The estimate is carried out using (H11) and Lemma A.1. ∎

Lemma 4.11.

For any given constant $C_{z}>0$ and any $\epsilon\in(0,1)$ ,

	$\displaystyle\lim_{k\to\infty}\inf_{t}P(\cap_{(z,u)\in[-C_{z},C_{z}]\times U}\{\|\Delta_{f}^{k}(z,u,{\mathbold y}^{\infty}_{t})\|\leq\epsilon\})=1,$
	$\displaystyle\lim_{k\to\infty}\inf_{t}P(\cap_{(z,u)\in[-C_{z},C_{z}]\times U}\{\|\Delta_{l}^{k}(z,u,{\mathbold y}^{\infty}_{t})\|\leq\epsilon\})=1.$

Proof.

We establish the first limit, and may deal with the second one in the same way. Note that the event

(4.17)

\displaystyle{\mathcal{E}}_{fC_{z}}^{k}\coloneqq\cap_{(z,u)\in[-C_{z},C_{z}]\times U}\{|\Delta_{f}^{k}(z,u,{\mathbold y}^{\infty}_{t})|\leq\epsilon\}

is well defined since $\Delta_{f}^{k}$ is continuous in $(z,u)$ and the intersection may be equivalently expressed using only a countable number of values of $(z,u)$ in $[-C_{z},C_{z}]\times U$ .

Take any $\epsilon\in(0,1)$ . By (H2) and (H3), we can find $\delta_{\epsilon}>0$ such that $|\Delta_{f}^{k}(z,u,{\mathbold y}^{\infty}_{t})-\Delta_{f}^{k}(z^{\prime},u^{\prime},{\mathbold y}^{\infty}_{t})|\leq\epsilon/2$ whenever $|z-z^{\prime}|+|u-u^{\prime}|\leq\delta_{\epsilon}$ . For the selected $\delta_{\epsilon}$ , we can find a fixed $p_{0}$ and $(z^{j},u^{j})\in[-C_{z},C_{z}]\times U$ , $j=1,\ldots,p_{0}$ such that for any $(z,u)\in[-C_{z},C_{z}]\times U$ , there exists some $j_{0}$ ensuring $|z-z^{j_{0}}|+|u-u^{j_{0}}|\leq\delta_{\epsilon}$ .

By Lemma 4.10 and Markov’s inequality, for any $\delta>0$ , there exists $K_{\delta,p_{0}}$ such that for all $k\geq K_{\delta,p_{0}}$ , we have

(4.18)

\displaystyle P(\{|\Delta_{f}^{k}(z^{j},u^{j},{\mathbold y}^{\infty}_{t})|\leq\epsilon/2\})\geq 1-\delta/p_{0},\quad\forall j,t.

Let ${\mathcal{E}}^{k}_{j}$ denote the event $\{|\Delta_{f}^{k}(z^{j},u^{j},{\mathbold y}^{\infty}_{t})|\leq\epsilon/2\}$ . By (4.18), $P(\cap_{j=1}^{p_{0}}{\mathcal{E}}_{j}^{k})\geq 1-\delta$ for $k\geq K_{\delta,p_{0}}$ . Now if $\omega\in{\mathcal{E}}^{k}\coloneqq\cap_{j=1}^{p_{0}}{\mathcal{E}}_{j}^{k}$ , $k\geq K_{\delta,p_{0}}$ , then for any $(z,u)\in[-C_{z},C_{z}]\times U$ , we have $|\Delta_{f}^{k}(z,u,{\mathbold y}^{\infty}_{t})|\leq\epsilon.$ Hence ${\mathcal{E}}^{k}\subset{\mathcal{E}}_{fC_{z}}^{k}.$ It follows that for all $k\geq K_{\delta,p_{0}}$ , $P({\mathcal{E}}_{fC_{z}}^{k})\geq 1-\delta.$ Since $\delta\in(0,1)$ is arbitrary and $K_{\delta,p_{0}}$ does not depend on $t$ , the first limit follows. ∎

Lemma 4.12.

We have

\displaystyle\lim_{k\to\infty}\sup_{t,u_{\iota}^{N}}E(|\Delta_{f}^{k}(x_{\iota}^{N}(t),u_{\iota}^{N}(t),{\mathbold y}^{\infty}_{t})|+|\Delta_{l}^{k}(x_{\iota}^{N}(t),u_{\iota}^{N}(t),{\mathbold y}^{\infty}_{t})|)=0.

Proof.

Fix any $\epsilon\in(0,1)$ . By (H1) and (H2) we can find a sufficiently large $C_{z}$ , independent of $(k,N)$ , such that for all $u_{\iota}^{N}(\cdot)$ ,

\displaystyle P\Big{(}\sup_{0\leq t\leq T}|x_{\iota}^{N}(t)|\leq C_{z}\Big{)}\geq 1-\epsilon.

Denote ${\mathcal{E}}_{x}=\{\sup_{0\leq t\leq T}|x_{\iota}^{N}(t)|\leq C_{z}\}$ . By Lemma 4.11, for the above $\epsilon$ and ${\mathcal{E}}^{k}_{fC_{z}}$ given by (4.17), there exists $K_{0}$ independent of $t$ such that for all $k\geq K_{0}$ ,

P({\mathcal{E}}^{k}_{fC_{z}})\geq 1-\epsilon.

Now if $\omega\in{\mathcal{E}}_{x}\cap{\mathcal{E}}^{k}_{fC_{z}}$ , then $|\Delta_{f}^{k}(x_{\iota}^{N}(t),u_{\iota}^{N}(t),{\mathbold y}^{\infty}_{t})|\leq\epsilon.$ We have $P({\mathcal{E}}_{x}\cap{\mathcal{E}}^{k}_{fC_{z}})\geq 1-2\epsilon$ , and so

P(|\Delta_{f}^{k}(x_{\iota}^{N}(t),u_{\iota}^{N}(t),{\mathbold y}^{\infty}_{t})|\leq\epsilon)\geq P({\mathcal{E}}_{x}\cap{\mathcal{E}}^{k}_{fC_{z}})\geq 1-2\epsilon.

It follows that for all $k\geq K_{0}$ ,

\displaystyle E|\Delta_{f}^{k}(x_{\iota}^{N}(t),u_{\iota}^{N}(t),{\mathbold y}^{\infty}_{t})|\leq\epsilon+2\epsilon C,

where $C$ does not depend on $(u_{\iota}^{N}(\cdot),t)$ . The bound for $\Delta_{l}^{k}$ is similarly obtained. ∎

Lemma 4.13.

We have

\displaystyle\lim_{k\to\infty}\sup_{t,u_{\iota}^{N}(\cdot)}E(|\delta_{f}^{k}|+|\delta_{l}^{k}|)=0.

Proof.

By Lipschitz continuity of $(f,l)$ , we estimate $E|\delta_{f}^{k}-\Delta_{f}^{k}(x_{\iota}^{N},u_{\iota}^{N},{\mathbold y}^{\infty}_{t})|$ and $E|\delta_{l}^{k}-\Delta_{l}^{k}(x_{\iota}^{N},u_{\iota}^{N},{\mathbold y}^{\infty}_{t})|$ , and next apply Lemma 4.9 to show that they converge to zero as $k\to\infty$ . Recalling Lemma 4.12, we complete the proof. ∎

Lemma 4.14.

We have

\displaystyle\lim_{k\to\infty}\sup_{t,u_{\iota}^{N}(\cdot)}E(|\delta_{f_{0}}^{k}|+|\delta_{l_{0}}^{k}|)=0.

Proof.

The proof is similar to that of Lemma 4.13 and the details are omitted. ∎

Denote

\epsilon_{fl}^{k}=\sup_{t,u_{\iota}^{N}(\cdot)}E(|\delta_{f_{0}}^{k}|+|\delta_{l_{0}}^{k}|+|\delta_{f}^{k}|+|\delta_{l}^{k}|).

Lemma 4.15.

For any admissible control $u_{\iota}^{N}$ in System B and $J_{\iota}^{*}$ in (4.14),

\displaystyle J_{\iota}^{N}(u_{\iota}^{N})\geq\inf_{u_{\iota}}J_{\iota}^{*}(u_{\iota})-C\epsilon_{fl}^{k},

where the constant $C$ does not depend on $u_{\iota}^{N}$ .

Proof.

Take any full state based Lipschitz feedback control $u_{\iota}^{N}$ . It together with the other agents’s control laws generates the closed-loop state processes $x_{1}^{N}(t),\ldots,x_{N}^{N}(t)$ . Let $u_{\iota}^{N}(t,\omega)$ denote the realization as a non-anticipative process. Now we take $\check{u}_{\iota}=u_{\iota}^{N}(t,\omega)$ in (4.13) and let $\check{x}_{\iota}^{\infty}$ be the resulting state process. It is clear from (4.14) that

(4.19)

\displaystyle J_{\iota}^{*}(\check{u}_{\iota})\geq\inf_{u_{\iota}}J_{\iota}^{*}(u_{\iota}).

Recalling (4.15) and applying Gronwall’s lemma to estimate the difference $\check{x}_{\iota}^{\infty}-x_{\iota}^{N}$ , we can show there exists $C$ independent of $u_{\iota}^{N}$ such that $|J_{\iota}^{N}(u_{\iota}^{N})-J_{\iota}^{*}(\check{u}_{\iota})|\leq C\epsilon_{fl}^{k},$ which combined with (4.19) completes the proof. ∎

Lemma 4.16.

Let $\varphi_{I^{*}({\iota})}=\varphi(t,x,g_{I^{*}({\iota})})$ be the GMFG based control law (4.2). We have

\displaystyle J^{N}_{\iota}(\varphi_{I^{*}({\iota})})\leq\inf_{u_{\iota}}J_{\iota}^{*}(u_{\iota})+C\epsilon_{fl}^{k}.

Proof.

Let $\varphi_{I^{*}({\iota})}$ be applied to the two systems (4.13) and (4.15). We further use Gronwall’s lemma to estimate $E|x_{\iota}^{\infty}-x_{\iota}^{N}|$ . We obtain $|J_{\iota}^{N}(\varphi_{I^{*}({\iota})})-J_{\iota}^{*}(\varphi_{I^{*}({\iota})})|\leq C\epsilon_{fl}^{k}$ . Note that $J_{\iota}^{*}(\varphi_{I^{*}({\iota})})=\inf_{u_{\iota}}J_{\iota}^{*}(u_{\iota})$ . This completes the proof. ∎

Proof of Theorem 4.4. It follows from Lemmas 4.13, 4.14, 4.15 and 4.16. ∎

5. The LQ Case

This section considers a special class of linear-quadratic-Gaussian (LQG) GMFG models. Consider the graph $G_{k}$ with vertices ${\mathcal{V}}_{k}=\{1,\ldots,M_{k}\}$ and graph adjacency matrix $g^{k}=[g^{k}_{jl}]$ . For agent ${\mathcal{A}}_{i}$ in subpopulation cluster ${\mathcal{C}}_{q}$ situated at node $q$ , let the intra- and inter-cluster coupling terms be denoted by $z_{0,i}$ and $z_{i}$ , respectively, where

z_{0,i}=\frac{1}{|\mathcal{C}_{q}|}\sum_{j\in\mathcal{C}_{q}}x_{j},\quad z_{i}=\frac{1}{|M_{k}|}\sum_{l\in{\mathcal{V}}_{k}}g^{k}_{ql}\frac{1}{|\mathcal{C}_{l}|}\sum_{j\in\mathcal{C}_{l}}x_{j},\quad x_{j},\ z_{0,i},\ z_{i}\in{\mathbb{R}}^{n}.

The dynamics of ${\mathcal{A}}_{i}$ are given by the linear system

\displaystyle dx_{i}=(Ax_{i}+D_{0}z_{0,i}+Dz_{i}+Bu_{i})dt+\Sigma dw_{i},\quad 1\leq i\leq N,

where $u_{i}\in{\mathbb{R}}^{n_{u}}$ is the control input, $w_{i}\in{\mathbb{R}}^{n_{w}}$ is a standard Brownian motion, and $A$ , $B$ , $D_{0}$ , $D$ , $\Sigma$ are conformally dimensioned matrices. Assume $Ex_{i}(0)=x_{0}$ for all $i$ .

The individual agent’s cost function takes the form

	$\displaystyle J_{i}(u_{i};\nu_{i})=$	$\displaystyle{E}\int_{0}^{T}\big{[}(x_{i}-\nu_{i})^{T}Q(x_{i}-\nu_{i})+u_{i}^{T}Ru_{i}\big{]}dt$
		$\displaystyle+E\big{[}(x_{i}(T)-\nu_{i}(T))^{T}Q_{T}(x_{i}(T)-\nu_{i}(T))\big{]},\quad 1\leq i\leq N,$

where $Q$ , $Q_{T}\geq 0$ , $R>0$ , and $\nu_{i}=\gamma_{0}z_{0,i}+\gamma z_{i}+\eta$ is the process tracked by ${\mathcal{A}}_{i}$ . Here $\eta\in\mathbb{R}^{n}$ and $\gamma_{0},\gamma\in\mathbb{R}$ .

In the infinite population and graphon limit case, denote the local mean $\int_{{\mathbb{R}}^{n}}x\mu_{\alpha}(dx)$ at $t$ for an $\alpha$ -agent situated at vertex $\alpha$ by $\bar{x}_{\alpha}$ , and the graphon weighted mean $\int_{0}^{1}g(\alpha,\beta)\bar{x}_{\beta}d\beta$ by $z_{\alpha}$ . The $\alpha$ -agent’s state equation is given by

\displaystyle dx_{\alpha}=(Ax_{\alpha}+D_{0}\bar{x}_{\alpha}+Dz_{\alpha}+Bu_{\alpha})dt+\Sigma dw_{\alpha},\quad\alpha\in[0,1].

The $\alpha$ -agent’s cost function is

	$\displaystyle J_{\alpha}(u_{\alpha};\nu_{\alpha})=$	$\displaystyle{E}\int_{0}^{T}\big{[}(x_{\alpha}-\nu_{\alpha})^{T}Q(x_{\alpha}-\nu_{\alpha})+u_{\alpha}^{T}Ru_{\alpha}\big{]}dt$
		$\displaystyle+E\big{[}(x_{\alpha}(T)-\nu_{\alpha}(T))^{T}Q_{T}(x_{\alpha}(T)-\nu_{\alpha}(T))\big{]},$

where $\nu_{\alpha}=\gamma_{0}\bar{x}_{\alpha}+\gamma z_{\alpha}+\eta$ .

Consider the Riccati equation

\displaystyle 0=\dot{\Pi}_{t}+A^{T}\Pi_{t}+\Pi_{t}A-\Pi_{t}BR^{-1}B^{T}\Pi_{t}+Q,

where $\Pi_{T}=Q_{T}$ , and

\displaystyle 0=\dot{s}_{\alpha}(t)+(A-BR^{-1}B^{T}\Pi_{t})^{T}{s}_{\alpha}(t)+{\Pi}_{t}(D_{0}\bar{x}_{\alpha}(t)+Dz_{\alpha}(t))-Q\nu_{\alpha}(t),

where $s_{\alpha}(T)=-Q_{T}\nu_{\alpha}(T)$ . The best response for the $\alpha$ -agent is given by

u_{\alpha}(t)=-R^{-1}B^{T}[\Pi_{t}x_{\alpha}(t)+s_{\alpha}(t)].

Now the mean state process of ${x}_{\alpha}$ is

\dot{\bar{x}}_{\alpha}=(A-BR^{-1}B^{T}\Pi_{t}+D_{0})\bar{x}_{\alpha}+Dz_{\alpha}-BR^{-1}B{{}^{T}}s_{\alpha},\quad\alpha\in[0,1].

The existence analysis reduces to verifying the existence and uniqueness of solutions for the equation system

(5.1)		$\displaystyle\dot{\bar{x}}_{\alpha}=(A-BR^{-1}B^{T}\Pi_{t}+D_{0})\bar{x}_{\alpha}-BR^{-1}B^{T}s_{\alpha}+D\int_{0}^{1}g(\alpha,\beta)\bar{x}_{\beta}d\beta,$
(5.2)		$\displaystyle\dot{s}_{\alpha}=-(A-BR^{-1}B^{T}\Pi_{t})^{T}s_{\alpha}+(\gamma_{0}Q-\Pi_{t}D_{0})\bar{x}_{\alpha}$
	$\displaystyle\qquad+(\gamma Q-\Pi_{t}D)\int_{0}^{1}g(\alpha,\beta)\bar{x}_{\beta}d\beta+Q\eta,$

where $\bar{x}_{\alpha}(0)=x_{0}$ and $s_{\alpha}(T)=-Q_{T}[\gamma_{0}\bar{x}_{\alpha}(T)+\gamma\int_{0}^{1}g(\alpha,\beta)\bar{x}_{\beta}(T)d\beta+\eta]$ .

To analyze (5.1)–(5.2), let $\Phi(t,s)$ and $\Psi(t,s)$ be the fundamental solution matrix of

\displaystyle\dot{x}=(A-BR^{-1}B^{T}\Pi_{t}+D_{0})x,\qquad\dot{y}=-(A-BR^{-1}B^{T}\Pi_{t})^{T}y

for $x(t),y(t)\in\mathbb{R}^{n}$ . For the special case with $D_{0}=0$ , $\Psi(t,s)=\Phi^{T}(s,t)$ holds. We convert the existence analysis into a fixed point problem. We view $\bar{x}_{\beta}(t)=\bar{x}(\beta,t)$ as a function of $(\beta,t)$ . Below we derive an equation for $\bar{x}_{\alpha}(t)$ by eliminating $s_{\alpha}(t)$ . Denote the function space $D_{\Lambda}$ consisting of continuous $\mathbb{R}^{n}$ -valued functions on $[0,1]\times[0,T]$ with norm $\|\check{x}\|=\sup_{\alpha,t}|\check{x}(\alpha,t)|$ . We use $|\cdot|$ to denote the Frobenius norm of a vector or matrix. Define the operator $\Lambda$ as follows: for $\check{x}\in D_{\Lambda}$ ,

	$\displaystyle({\Lambda}\check{x})(\alpha,t)=\$	$\displaystyle\int_{0}^{t}\Phi(t,r)BR^{-1}B^{T}\Big{\{}\int_{r}^{T}\Psi(r,\tau)\Big{[}(\gamma_{0}Q-\Pi_{\tau}D_{0})\check{x}(\alpha,\tau)$
		$\displaystyle\qquad\qquad+(\gamma Q-\Pi_{\tau}D)\int_{0}^{1}g(\alpha,\beta)\check{x}(\beta,\tau)d\beta\Big{]}d\tau$
		$\displaystyle+\Psi(r,T)Q_{T}\Big{[}\gamma_{0}\check{x}(\alpha,T)+\gamma\int_{0}^{1}g(\alpha,\beta)\check{x}(\beta,T)d\beta\Big{]}\Big{\}}dr$
		$\displaystyle+\int_{0}^{t}\Phi(t,r)D\int_{0}^{1}g(\alpha,\beta)\check{x}(\beta,r)d\beta dr.$

If (H5) holds, $\Lambda$ is from $D_{\Lambda}$ to itself.

The solution of the LQG GMFG reduces to finding a fixed point $\check{x}$ to the equation

	$\displaystyle\check{x}(\alpha,t)=$	$\displaystyle(\Lambda\check{x})(\alpha,t)+\Phi(t,0)x_{0}$
		$\displaystyle+\int_{0}^{t}\Phi(t,r)BR^{-1}B^{T}\Big{[}\!\int_{r}^{T}\Psi(r,\tau)Qd\tau+\Psi(r,T)Q_{T}\Big{]}\eta dr.$

Denote $c_{g}=\max_{\alpha}\int_{0}^{1}g(\alpha,\beta)d\beta$ . We have the bound for the operator norm:

	$\displaystyle\\|\Lambda\\|\leq c_{\Lambda}\coloneqq$	$\displaystyle\sup_{t\in[0,T]}\Big{\{}\int_{0}^{t}\int_{r}^{T}\|\Phi(t,r)BR^{-1}B^{T}\Psi(r,\tau)\|\cdot(\|\gamma_{0}Q-\Pi_{\tau}D_{0}\|$
		$\displaystyle\qquad\qquad+c_{g}\|\gamma Q-\Pi_{\tau}D\|)d\tau dr$
		$\displaystyle+\int_{0}^{t}\Big{[}\|\Phi(t,r)BR^{-1}B^{T}\Psi(r,T)Q_{T}\|\cdot(\|\gamma_{0}\|+c_{g}\|\gamma\|)+c_{g}\|\Phi(t,r)D\|\Big{]}dr\Big{\}}.$

If $c_{\Lambda}<1,$ $\Lambda$ is a contraction and (5.1)–(5.2) has a unique solution.

As an example for illustration, we assume the graphon weighted mean at vertex $\alpha$ arises from an underlying uniform attachment graphon, and consequently

z_{\alpha}=\int_{0}^{1}(1-\max(\alpha,\beta))\int_{\mathbb{R}^{n}}x\mu_{\beta}(dx)d\beta,\quad\alpha,\beta\in[0,1],

where it is readily verified that the uniform attachment graphon satisfies (H5).

Appendix

Lemma A.1.

Assume (H1)–(H8). Let $\varphi_{\alpha}$ be the GMFG based best response (4.2) and $\mu_{\alpha}(t)$ the distribution of the closed-loop process $x_{\alpha}(t)$ , $\alpha\in[0,1]$ , in (3.15) with initial distribution $\mu_{0}^{x}$ . Then we have

\lim_{r\to 0}\sup_{|t-t^{*}|+|\beta-\beta^{*}|<r}W_{1}(\mu_{\beta}(t),\mu_{\beta^{*}}(t^{*}))=0,

where $t,t^{*}\in[0,T]$ and $\beta,\beta^{*}\in[0,1]$ .

Proof.

Step 1. Take any $\beta,\beta^{*}\in[0,1]$ . For $\mu_{G}(\cdot)$ determined from the GMFG equations (3.2) and (3.15), define two processes

	$\displaystyle dy_{\beta^{}}=\widetilde{f}[y_{\beta^{}},\varphi(t,y_{\beta^{}},g_{\beta^{}}),\mu_{G};g_{\beta^{}}]dt+\sigma dw_{\beta^{}},$
	$\displaystyle dy_{\beta}=\widetilde{f}[y_{\beta},\varphi(t,y_{\beta},g_{\beta}),\mu_{G};g_{\beta}]dt+\sigma dw_{\beta^{*}},$

where $y_{\beta^{*}}(0)=y_{\beta}(0)=x_{i}^{N}(0)$ and the same Brownian motion is used. Then the distributions of $y_{\beta^{*}}(t)$ and $y_{\beta}(t)$ are $\mu_{\beta^{*}}(t)$ and $\mu_{\beta}(t)$ , respectively. We obtain

		$\displaystyle y_{\beta}(t)-y_{\beta^{*}}(t)$
	$\displaystyle=$	$\displaystyle\int_{0}^{t}\Delta_{\beta,\beta^{}}^{0}(s)ds+\int_{0}^{t}\int_{0}^{1}\int_{\mathbb{R}}\Delta_{\beta,\beta^{}}(s,z,\lambda)\mu_{\lambda}(s,dz)d\lambda ds,$

where

	$\displaystyle\Delta_{\beta,\beta^{}}^{0}(s)=\int_{\mathbb{R}}f_{0}(y_{\beta},\varphi(s,y_{\beta},g_{\beta}),z)\mu_{\beta}(s,dz)-\int_{\mathbb{R}}f_{0}(y_{\beta^{}},\varphi(s,y_{\beta^{}},g_{\beta^{}}),z)\mu_{\beta^{*}}(s,dz),$
	$\displaystyle\Delta_{\beta,\beta^{*}}(s,z,\lambda)=f(y_{\beta},\varphi(s,y_{\beta},g_{\beta}),z)g(\beta,\lambda)$
	$\displaystyle\qquad\qquad\qquad-f(y_{\beta^{}},\varphi(s,y_{\beta^{}},g_{\beta^{}}),z)g(\beta^{},\lambda).$

We will simply write $\mu_{\lambda}(s,dz)$ as $\mu_{\lambda}(dz)$ if the time argument is clear, where $\lambda$ is the vertex index. Denote $\kappa_{\beta,\beta^{*}}(s)=|\varphi(s,y_{\beta^{*}},g_{\beta})-\varphi(s,y_{\beta^{*}},g_{\beta^{*}})|$ , where the time argument $s$ in $y_{\beta}$ and $y_{\beta^{*}}$ has been suppressed. It follows that

	$\displaystyle\|\Delta_{\beta,\beta^{*}}^{0}(s)\|\leq$
	$\displaystyle\Big{\|}\int_{\mathbb{R}}f_{0}(y_{\beta},\varphi(s,y_{\beta},g_{\beta}),z)\mu_{\beta}(s,dz)-\int_{\mathbb{R}}f_{0}(y_{\beta},\varphi(s,y_{\beta},g_{\beta}),z)\mu_{\beta^{*}}(s,dz)\Big{\|}$
	$\displaystyle+\Big{\|}\int_{\mathbb{R}}f_{0}(y_{\beta},\varphi(s,y_{\beta},g_{\beta}),z)\mu_{\beta^{}}(s,dz)-\int_{\mathbb{R}}f_{0}(y_{\beta^{}},\varphi(s,y_{\beta^{}},g_{\beta^{}}),z)\mu_{\beta^{*}}(s,dz)\Big{\|}$
	$\displaystyle\leq CE\|y_{\beta}-y_{\beta^{}}\|+C\|y_{\beta}-y_{\beta^{}}\|+C\|\varphi(s,y_{\beta},g_{\beta})-\varphi(s,y_{\beta^{}},g_{\beta^{}})\|$
	$\displaystyle\leq CE\|y_{\beta}-y_{\beta^{}}\|+C_{1}\|y_{\beta}-y_{\beta^{}}\|+C\kappa_{\beta,\beta^{*}}(s),$

where the second inequality is obtained using (H2), (H3), and the method in (4.11). The last inequality has used the uniform Lipschitz continuity of $\varphi_{\beta}$ in the space variable (see Lemma 3.7). It follows that

(A.1)

\displaystyle E|\Delta_{\beta,\beta^{*}}^{0}(s)|\leq C_{2}E|y_{\beta}(s)-y_{\beta^{*}}(s)|+CE\kappa_{\beta,\beta^{*}}(s).

Next, we have

		$\displaystyle\Big{\|}\int_{0}^{1}\int_{\mathbb{R}}\Delta_{\beta,\beta^{*}}(s,z,\lambda)\mu_{\lambda}(dz)d\lambda\Big{\|}$
(A.2)		$\displaystyle\leq$	$\displaystyle\Big{\|}\int_{0}^{1}\int_{\mathbb{R}}[f(y_{\beta},\varphi(s,y_{\beta},g_{\beta}),z)-f(y_{\beta^{}},\varphi(s,y_{\beta^{}},g_{\beta^{*}}),z)]g(\beta,\lambda)\mu_{\lambda}(dz)d\lambda\Big{\|}$
		$\displaystyle+\Big{\|}\int_{0}^{1}\int_{\mathbb{R}}f(y_{\beta^{}},\varphi(s,y_{\beta^{}},g_{\beta^{}}),z)[g(\beta,\lambda)-g(\beta^{},\lambda)]\mu_{\lambda}(dz)d\lambda\Big{\|}$
	$\displaystyle=:$	$\displaystyle I_{f}(s)+I_{g}(s).$

We have

	$\displaystyle I_{f}(s)$	$\displaystyle\leq\int_{0}^{1}\int_{\mathbb{R}}C(\|y_{\beta}-y_{\beta^{}}\|+\kappa_{\beta,\beta^{}})g(\beta,\lambda)\mu_{\lambda}(dz)d\lambda$
		$\displaystyle\leq C(\|y_{\beta}-y_{\beta^{}}\|+\kappa_{\beta,\beta^{}})(s),$

where we have used the Lipschitz property of $f$ and $\varphi_{\beta}$ . Therefore,

(A.3)

\displaystyle EI_{f}(s)\leq C(E|y_{\beta}(s)-y_{\beta^{*}}(s)|+E\kappa_{\beta,\beta^{*}}(s)).

For any fixed value $y_{\beta^{*}}(s,\omega)$ , denote

\displaystyle\xi_{\beta^{*},s,\omega}(\lambda)=\int_{\mathbb{R}}f(y_{\beta^{*}},\varphi(s,y_{\beta^{*}},g_{\beta^{*}}),z)\mu_{\lambda}(dz).

We have

\displaystyle I_{g}(s)=\Big{|}\int_{0}^{1}\xi_{\beta^{*},s,\omega}(\lambda)g(\beta,\lambda)d\lambda-\int_{0}^{1}\xi_{\beta^{*},s,\omega}(\lambda)g(\beta^{*},\lambda)d\lambda\Big{|}.

Hence, by (H5), $I_{g}(s)\to 0$ $(\omega,s)$ -a.e. as $\beta\to\beta^{*}$ . It is clear $I_{g}(s)$ is bounded by a fixed constant since $f$ is a bounded function. For the fixed $\beta^{*}$ , by Lemma 3.5, the random variable $\kappa_{\beta,\beta^{*}}(s)$ is bounded and converges to zero with probability one. Denote $\delta_{g}=\int_{0}^{T}EI_{g}(s)ds$ and $\delta_{\kappa}=\int_{0}^{T}E\kappa_{\beta,\beta^{*}}(s)ds$ . By dominated convergence, we have

\lim_{\beta\to\beta^{*}}(\delta_{g}+\delta_{\kappa})=0.

By (A.1)–(A.3), it follows that

\displaystyle E|y_{\beta}(t)-y_{\beta^{*}}(t)|\leq C\int_{0}^{t}E|y_{\beta}(s)-y_{\beta^{*}}(s)|ds+C(\delta_{\kappa}+\delta_{g}).

By Gronwall’s lemma, we have

\displaystyle\sup_{0\leq t\leq T}E|y_{\beta}(t)-y_{\beta^{*}}(t)|\leq Ce^{CT}(\delta_{\kappa}+\delta_{g}).

Since $W_{1}(\mu_{\beta}(t),\mu_{\beta^{*}}(t))\leq E|y_{\beta}(t)-y_{\beta^{*}}(t)|$ , then

(A.4)

\displaystyle\sup_{t}W_{1}(\mu_{\beta}(t),\mu_{\beta^{*}}(t))\leq C_{1}(\delta_{\kappa}+\delta_{g}),

where $\delta_{\kappa}$ and $\delta_{g}$ depend on $\beta^{*}$ .

Step 2. Now we consider given $(\beta^{*},t^{*})\in[0,1]\times[0,T]$ . By use of the SDE of $y_{\beta}$ and elementary estimates, we obtain

(A.5)

\displaystyle\lim_{|t-t^{*}|\to 0}\sup_{\beta}W_{1}(\mu_{\beta}(t^{*}),\mu_{\beta}(t))=0.

We have

\displaystyle W_{1}(\mu_{\beta}(t),\mu_{\beta^{*}}(t^{*}))\leq W_{1}(\mu_{\beta}(t),\mu_{\beta}(t^{*}))+W_{1}(\mu_{\beta}(t^{*}),\mu_{\beta^{*}}(t^{*})).

Given any $\epsilon>0$ , by (A.4) and (A.5) there exists $\delta_{\epsilon,\beta^{*}}>0$ such that whenever $|t-t^{*}|+|\beta-\beta^{*}|\leq\delta_{\epsilon,\beta^{*}}$ , we have

W_{1}(\mu_{\beta}(t),\mu_{\beta}(t^{*}))\leq\frac{\epsilon}{2},\qquad W_{1}(\mu_{\beta}(t^{*}),\mu_{\beta^{*}}(t^{*}))\leq\frac{\epsilon}{2}.

Therefore, $W_{1}(\mu_{\beta}(t),\mu_{\beta^{*}}(t^{*}))\leq\epsilon$ . We conclude that $\mu_{\beta}(t)$ as a mapping from the compact space $[0,1]\times[0,T]$ to ${\mathcal{P}}_{1}({\mathbb{R}})$ with the metric $W_{1}(\cdot,\cdot)$ is continuous and hence must be uniformly continuous. The lemma follows. ∎

Lemma A.2.

Suppose the graphon $g$ satisfies (H5) and (H11). Then for any given measurable sets $\mathcal{S},\mathcal{T}\subset[0,1]$ , under (H9) we have

(A.6)

\displaystyle\lim_{k\to\infty}\Big{|}\int_{{\mathcal{S}}\times{\mathcal{T}}}(g^{k}-g)dxdy\Big{|}=0.

Proof.

Step 1. We approximate ${\mathcal{S}},{\mathcal{T}}$ by open sets. Let $\mu_{\rm L}$ denote the Lebesgue measure on $\mathbb{R}^{d}$ , where the dimension $d$ will be clear from the context. Consider the given sets ${\mathcal{S}},{\mathcal{T}}$ , and choose an arbitrary $\epsilon>0$ . Note that for any measurable set $A_{1}\subset\mathbb{R}^{d}$ and any $\delta_{0}>0$ , there exists an open set $A_{2}\supset A_{1}$ such that $\mu_{\rm L}(A_{2}\backslash A_{1})\leq\delta_{0}$ (see e.g. [36]). So there exist open sets $\mathcal{S}^{o}\subset\mathbb{R}$ and ${\mathcal{T}}^{o}\subset\mathbb{R}$ such that $\mathcal{S}\subset\mathcal{S}^{o}$ , ${\mathcal{T}}\subset{\mathcal{T}}^{o}$ and $\mu_{\rm L}(\mathcal{S}^{o}\backslash\mathcal{S})\leq\epsilon$ , $\mu_{\rm L}({\mathcal{T}}^{o}\backslash{\mathcal{T}})\leq\epsilon$ .

Define the new open sets $\mathcal{S}_{1}^{o}=\mathcal{S}^{o}\cap(0,1)$ and ${\mathcal{T}}^{o}_{1}={\mathcal{T}}^{o}\cap(0,1)$ . Each open set in $\mathbb{R}$ may be written as the union of at most countable disjoint open intervals [36]; among such a union for $\mathcal{S}_{1}^{o}$ , we may find a finite integer $s^{*}$ (depending on $({\mathcal{S}},\epsilon)$ ) and constituent disjoint open intervals $I_{i}^{\mathcal{S}}\subset[0,1]$ , $1\leq i\leq s^{*}$ , such that $U_{s^{*}}\coloneqq\cup_{i=1}^{s^{*}}I_{i}^{\mathcal{S}}\subset{\mathcal{S}}_{1}^{o}$ and $\mu_{\rm L}(\mathcal{S}_{1}^{o}\backslash U_{s^{*}})\leq\epsilon$ . Similarly, we find a finite integer $t^{*}$ and disjoint open intervals $I_{i}^{\mathcal{T}}\subset[0,1]$ such that $U_{t^{*}}\coloneqq\cup_{j=1}^{t^{*}}I_{j}^{\mathcal{T}}\subset{\mathcal{T}}_{1}^{o}$ and $\mu_{\rm L}({\mathcal{T}}_{1}^{o}\backslash U_{t^{*}})\leq\epsilon$ . Here the choice of $(s^{*},t^{*})$ depends on $(\mathcal{S},\mathcal{T},\epsilon)$ .

By the construction of $U_{s^{*}}$ and $U_{t^{*}}$ , we have the bound for the measure of the following symmetric differences:

\displaystyle\mu_{\rm L}(\mathcal{S}\Delta U_{s^{*}})\leq 2{\epsilon},\quad\mu_{\rm L}(\mathcal{T}\Delta U_{t^{*}})\leq 2{\epsilon},

which implies $\mu_{\rm L}((\mathcal{S}\times\mathcal{T})\Delta(U_{s^{*}}\times U_{t^{*}}))\leq 6\epsilon$ . Since $|g^{k}-g|\leq 1$ for any $x,y$ , we have

(A.7)

\displaystyle\Big{|}\int_{\mathcal{S}\times\mathcal{T}}(g^{k}-g)dxdy-\eta_{k}\Big{|}\leq{6\epsilon},

where

\displaystyle\eta_{k}

\displaystyle\coloneqq\Big{|}\int_{U_{s^{*}}\times U_{t^{*}}}(g^{k}-g)dxdy\Big{|}.

Step 2. Blow we estimate $\eta_{k}$ . Under (H9) we take a sufficiently large $K_{0}$ , depending on ${s^{*}}$ (and so on $({\mathcal{S}},\epsilon)$ ), such that for all $k\geq K_{0}$ ,

\frac{s^{*}}{M_{k}}\leq{\epsilon}.

Consider $k\geq K_{0}$ . We select from the subintervals $I^{k}_{1},\ldots,I_{M_{k}}^{k}$ of equal length $1/M_{k}$ in the partition of $[0,1]$ such that a subinterval is selected whenever its interior is contained in $U_{s^{*}}$ . The method here is to fill $U_{s^{*}}$ as much as possible from inside by these subintervals. This procedure determines a subcollection denoted by $I^{k}_{i_{r}}$ , $r=1,\ldots,r_{k}$ . Denote $\hat{U}_{s^{*}}=\cup_{r=1}^{r_{k}}I^{k}_{i_{r}}$ . Then the interior of $\hat{U}_{s^{*}}$ is contained in $U_{s^{*}}$ . We need to estimate the measure for the part of $U_{s^{*}}$ not covered by $\hat{U}_{s^{*}}$ . We check $I_{i}^{\mathcal{S}}$ , $1\leq i\leq s^{*}$ , to obtain two cases: (i) $I_{i}^{\mathcal{S}}\subset\hat{U}_{s^{*}}$ , (ii) $I_{i}^{\mathcal{S}}$ has a portion (allowed to be equal to its whole) of positive measure staying outside $\hat{U}_{s^{*}}$ . For case (ii), the portion of $I_{i}^{\mathcal{S}}$ that is not covered by $\hat{U}_{s^{*}}$ consists of either one interval, as part or the whole of $I_{i}^{\mathcal{S}}$ , or two intervals each having an endpoint of $I_{i}^{\mathcal{S}}$ as its boundary; hence the measure of that portion is less than $2/M_{k}$ . It follows that

(A.8)

\displaystyle\mu_{\rm L}(U_{s^{*}}\backslash\hat{U}_{s^{*}})\leq\frac{2s^{*}}{M_{k}}\leq 2{\epsilon}.

By (A.8), for all $k\geq K_{0}$ , we have

(A.9)

\displaystyle\Big{|}\int_{U_{s^{*}}\times U_{t^{*}}}(g^{k}-g)dxdy-\int_{\hat{U}_{s^{*}}\times U_{t^{*}}}(g^{k}-g)dxdy\Big{|}\leq{2\epsilon}.

Step 3. Now for $k\geq K_{0}$ we check

\displaystyle\hat{\eta}_{k}\coloneqq\Big{|}\int_{\hat{U}_{s^{*}}\times U_{t^{*}}}(g^{k}-g)dxdy\Big{|}.

By (H5), for the selected $U_{t^{*}}$ , $\int_{U_{t^{*}}}g(x,y)dy$ as a function of $x$ is uniformly continuous on $[0,1]$ . So for $\epsilon$ chosen in Step 1, there exists $\delta>0$ (depending on $g$ , $\epsilon$ and $U_{t^{*}}$ ) such that

(A.10)

\displaystyle\Big{|}\int_{U_{t^{*}}}g(x,y)dy-\int_{U_{t^{*}}}g(x^{\prime},y)dy\Big{|}\leq{\epsilon}

whenever $|x-x^{\prime}|\leq\delta$ . For the above $\delta$ , we fix $K_{1}\geq K_{0}$ such that for all $k\geq K_{1}$ , we have $1/{M_{k}}\leq 2\delta$ . Note that we use $(I_{i_{r}}^{k})^{*}$ to denote the midpoint of the interval $I_{i_{r}}^{k}$ . Now for $k\geq K_{1}$ , we have

	$\displaystyle\hat{\eta}_{k}$	$\displaystyle=\Big{\|}\sum_{r=1}^{r_{k}}\int_{I^{k}_{i_{r}}}\int_{U_{t^{*}}}[g^{k}(x,y)-g(x,y)]dydx\Big{\|}$
		$\displaystyle\leq\Big{\|}\sum_{r=1}^{r_{k}}\int_{I^{k}_{i_{r}}}\int_{U_{t^{}}}[g^{k}((I^{k}_{i_{r}})^{},y)-g((I^{k}_{i_{r}})^{*},y)]dydx\Big{\|}+{\epsilon}$
		$\displaystyle=\Big{\|}\sum_{r=1}^{r_{k}}\frac{1}{M_{k}}\int_{U_{t^{}}}[g^{k}((I^{k}_{i_{r}})^{},y)-g((I^{k}_{i_{r}})^{*},y)]dy\Big{\|}+{\epsilon}$
		$\displaystyle\leq\frac{1}{M_{k}}\sum_{r=1}^{r_{k}}\zeta_{k}+{\epsilon},$

where

\displaystyle\zeta_{k}\coloneqq\Big{|}\int_{U_{t^{*}}}[g^{k}((I^{k}_{i_{r}})^{*},y)-g((I^{k}_{i_{r}})^{*},y)]dy\Big{|}.

The first inequality follows from (A.10) and $\mu_{\rm L}(\cup_{r=1}^{r_{k}}I^{k}_{i_{r}})\leq 1$ .

Step 4. Now we estimate $\zeta_{k}$ . As in Step 2, we take a sufficiently large $K_{2}\geq K_{1}$ , depending on $({t^{*},\epsilon})$ , such that for all $k\geq K_{2}$ , ${t^{*}}/{M_{k}}\leq{\epsilon}.$ For $k\geq K_{2}$ and the subintervals $I^{k}_{1},\ldots,I_{M_{k}}^{k}$ , as in Step 2, we select a subcollection denoted by $I^{k}_{j_{\tau}}$ , $\tau=1,\ldots,\tau_{k}$ , each of which is selected whenever its interior is contained in $U_{t^{*}}$ . Then it follows that

(A.11)

\displaystyle\mu_{\rm L}(U_{t^{*}}\backslash\cup_{\tau=1}^{\tau_{k}}I^{k}_{j_{\tau}})\leq\frac{2t^{*}}{M_{k}}\leq 2\epsilon.

By (A.11), we have for all $k\geq K_{2}$ ,

	$\displaystyle\zeta_{k}\leq$	$\displaystyle\Big{\|}\int_{\cup_{\tau=1}^{\tau_{k}}I^{k}_{j_{\tau}}}[g^{k}((I^{k}_{i_{r}})^{},y)-g((I^{k}_{i_{r}})^{},y)]dy\Big{\|}+2{\epsilon}$
	$\displaystyle\leq$	$\displaystyle\sum_{\tau=1}^{\tau_{k}}\Big{\|}\frac{g^{k}_{i_{r}j_{\tau}}}{M_{k}}-\int_{\beta\in I^{k}_{j_{\tau}}}g_{(I^{k}_{i_{r}})^{*},\beta}d\beta\Big{\|}+2{\epsilon}.$

We write $g(\alpha,\beta)$ as $g_{\alpha,\beta}$ .

Step 5. Note that $r_{k},\tau_{k}\leq M_{k}$ . Subsequently, by Step 3 and Step 4, we have for $k\geq K_{2}$ ,

	$\displaystyle\hat{\eta}_{k}$	$\displaystyle\leq\frac{1}{M_{k}}\sum_{r=1}^{r_{k}}\Big{[}\sum_{\tau=1}^{\tau_{k}}\Big{\|}\frac{g^{k}_{i_{r}j_{\tau}}}{M_{k}}-\int_{\beta\in I^{k}_{j_{\tau}}}g_{(I^{k}_{i_{r}})^{*},\beta}d\beta\Big{\|}+2{\epsilon}\Big{]}+{\epsilon}$
		$\displaystyle\leq\frac{1}{M_{k}}\sum_{r=1}^{r_{k}}\sum_{\tau=1}^{\tau_{k}}\Big{\|}\frac{g^{k}_{i_{r}j_{\tau}}}{M_{k}}-\int_{\beta\in I^{k}_{j_{\tau}}}g_{(I^{k}_{i_{r}})^{*},\beta}d\beta\Big{\|}+{3\epsilon}$
(A.12)			$\displaystyle\leq\max_{i}\sum_{j=1}^{M_{k}}\Big{\|}\frac{g^{k}_{{\mathcal{C}}_{i}{\mathcal{C}}_{j}}}{M_{k}}-\int_{\beta\in I_{j}^{k}}g_{(I^{k}_{i})^{*},\beta}d\beta\Big{\|}+{3\epsilon}.$

By (A.7), (A.9) and (A.12), we obtain for all $k\geq K_{2}$ depending on $({\mathcal{S}},{\mathcal{T}},\epsilon)$ ,

\displaystyle\Big{|}\int_{\mathcal{S}\times\mathcal{T}}(g^{k}-g)dxdy\Big{|}

\displaystyle\leq\max_{i}\sum_{j=1}^{M_{k}}\Big{|}\frac{g^{k}_{{\mathcal{C}}_{i}{\mathcal{C}}_{j}}}{M_{k}}-\int_{\beta\in I_{j}^{k}}g_{(I^{k}_{j})^{*},\beta}d\beta\Big{|}+11\epsilon.

The lemma follows. ∎

References

[1] Erhan Bayraktar, Suman Chakraborty, and Ruoyu Wu, Graphon mean field systems, arXiv:2003.13180 (2020).
[2] Christian Borgs, Jennifer Chayes, László Lovász, Vera T Sós, Balázs Szegedy, and Katalin Vesztergombi, Graph limits and parameter testing, Proc. the thirty-eighth annual ACM symposium on Theory of computing, 2006, pp. 261–270.
[3] Christian Borgs, Jennifer T Chayes, László Lovász, Vera T Sós, and Katalin Vesztergombi, Convergent sequences of dense graphs I: Subgraph frequencies, metric properties and testing, Advances in Mathematics 219 (2008), no. 6, 1801–1851.
[4] by same author, Convergent sequences of dense graphs II. multiway cuts and statistical physics, Annals of Mathematics 176 (2012), no. 1, 151–219.
[5] Peter E Caines, Mean field games, Encyclopedia of Systems and Control (2015), 706–712.
[6] Peter E Caines and Minyi Huang, Graphon mean field games and the GMFG equations, Proc. 57th IEEE CDC (Miami Beach, FL, USA), 2018, pp. 4129–4134.
[7] by same author, Graphon mean field games and the GMFG equations: $\epsilon$ -Nash equilibria, Proc. the 58th IEEE CDC (Nice, France), 2019, pp. 286–292.
[8] Peter E Caines, Minyi Huang, and Roland P Malhamé, Mean field games, Handbook of Dynamic Game Theory (Tamer Başar and Georges Zaccour, eds.), Springer, Berlin, 2017, pp. 345–372.
[9] Rene Carmona and Francois Delarue, Probabilistic theory of mean field games with applications I, vol. 83, Springer International Publishing, 2018.
[10] by same author, Probabilistic theory of mean field games with applications II, vol. 84, Springer International Publishing, 2018.
[11] François Delarue, Mean field games: A toy model on an Erdös-Renyi graph, ESAIM: Proceedings and Surveys 60 (2017), 1–26.
[12] Joseph L Doob, Stochastic processes, Wiley, New York, 1953.
[13] Wendell H. Fleming and Raymond W. Rishel, Deterministic and stochastic optimal control, Springer-Verlag, New York, 1975.
[14] Isabelle Gallagher, From Newton to Navier-Stokes, or how to connect fluid mechanics equations from microscopic to macroscopic scales, Bulletin of the American Math. Society 56 (2013), no. 1, 65–85.
[15] Isabelle Gallagher, Laure Saint-Raymond, and Benjamin Texier, From Newton to Boltzmann: hard spheres and short-range potentials, European Mathematical Society, 2013.
[16] Shuang Gao and Peter E. Caines, The control of arbitrary size networks of linear systems via graphon limits: An initial investigation, Proc. 56th IEEE CDC (Melbourne, Australia), December 2017, pp. 1052–1057.
[17] by same author, Controlling complex networks of linear systems via graphon limits, Presented at the Symposium of Controlling Complex Networks of NetSci17, Indianapolis, IN, USA (2017).
[18] by same author, Minimum energy control of arbitrary size networks of linear systems via graphon limits, Presented at the SIAM Workshop on Network Science, Pittsburgh, PA, USA (2017).
[19] by same author, Graphon linear quadratic regulation of large-scale networks of linear systems, Proc. 57th IEEE Conference on Decision and Control (Miami Beach, FL, USA), December 2018, pp. 5892–5897.
[20] by same author, Graphon control of large-scale networks of linear systems, IEEE Transactions on Automatic Control 65 (2020), no. 10, 4090–4105.
[21] Olivier Guéant, Existence and uniqueness result for mean field games with congestion effect on graphs, Applied Mathematics & Optimization 72 (2015), no. 2, 291–303.
[22] Isom H Herron and Michael R Foster, Partial differential equations in fluid dynamics, Cambridge University Press, 2008.
[23] Minyi Huang, Peter E Caines, and Roland P Malhamé, Large-population cost-coupled LQG problems with nonuniform agents: individual-mass behavior and decentralized $\varepsilon$ -Nash equilibria, IEEE Transactions on Automatic Control 52 (2007), no. 9, 1560–1571.
[24] by same author, The NCE (mean field) principle with locality dependent cost interactions, IEEE Transactions on Automatic Control 55 (2010), no. 12, 2799–2805.
[25] Minyi Huang, Roland P Malhamé, and Peter E Caines, Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle, Communications in Information & Systems 6 (2006), no. 3, 221–252.
[26] Dmitry Kaliuzhnyi-Verbovetskyi and Georgi S Medvedev, The semilinear heat equation on sparse random graphs, SIAM Journal on Mathematical Analysis 49 (2017), no. 2, 1333–1355.
[27] Ioannis Karatzas and Steven Shreve, Brownian motion and stochastic calculus, vol. 113, Springer Science & Business Media, 2012.
[28] Vassili Kolokoltsov and Wei Yang, Sensitivity analysis for HJB equations with an application to coupled backward-forward systems, arXiv preprint arXiv:1303.6234v2 (2015).
[29] Olga Aleksandrovna Ladyzhenskaya, NN Ural’ceva, and VA Solonnikov, Linear and quasi-linear equations of parabolic type, American Mathematical Society, 1968.
[30] Jean-Michel Lasry and Pierre-Louis Lions, Jeux á champ moyen. I - le cas stationnaire, Comptes Rendus Mathématique 343 (2006), no. 9, 619–625.
[31] by same author, Jeux á champ moyen. II horizon fini et controle optimal, Comptes Rendus Mathématique 343 (2006), no. 10, 679–684.
[32] László Lovász, Large networks and graph limits, vol. 60, American Mathematical Soc., 2012.
[33] László Lovász and Balázs Szegedy, Limits of dense graph sequences, Journal of Combinatorial Theory, Series B 96 (2006), no. 6, 933–957.
[34] Georgi S Medvedev, The nonlinear heat equation on dense graphs and graph limits, SIAM J. Math. Anal. 46 (2014), no. 4, 2743–2766.
[35] by same author, The nonlinear heat equation on w-random graphs, Archive for Rational Mechanics and Analysis 212 (2014), no. 3, 781–803.
[36] I. P. Natanson, Theory of functions of a real variable, vol. I, F. Ungar Publishing Co., 1983, 5th printing.
[37] Francesca Parise and Asuman Ozdaglar, Graphon games, arXiv preprint arXiv:1802.00080 (2018).
[38] Wolfgang Pauli and Charles P Enz, Thermodynamics and the kinetic theory of gases, vol. 3, Courier Corporation, 2000.
[39] Cristobal Quininao and Jonathan Touboul, Limits and dynamics of randomly connected neuronal networks, Acta Appl Math 136 (2015), 167–192.
[40] Nevroz Sen and Peter E Caines, Mean field game theory with a partially observed major agent, SIAM Journal on Control and Optimization 54 (2016), no. 6, 3174–3224.
[41] Alain-Sol Sznitman, Topics in propagation of chaos, Ecole d’été de probabilités de Saint-Flour XIX—1989, Springer, 1991, pp. 165–251.

	$\displaystyle\|v\|^{\beta}_{Q_{b}}=\|v\|_{Q_{b}}+\sup_{t\in(T_{1},T_{2}),x,y\in{\mathcal{I}}}\|v(t,x)-v(t,y)\|\cdot\|x-y\|^{-\beta}$
	$\displaystyle\qquad\qquad+\sup_{s,t\in(T_{1},T_{2}),x\in{\mathcal{I}}}\|v(s,x)-v(t,x)\|\cdot\|s-t\|^{-\beta/2},$
	$\displaystyle\|v\|^{1+\beta}_{Q_{b}}=\|v\|^{\beta}_{Q_{b}}+\|v_{x}\|^{\beta}_{Q_{b}},$
	$\displaystyle\|v\|^{2+\beta}_{Q_{b}}=\|v\|^{1+\beta}_{Q_{b}}+\|v_{t}\|^{\beta}_{Q_{b}}+\|v_{xx}\|^{\beta}_{Q_{b}}.$

		$\displaystyle\|S_{\alpha}^{\mu_{G}(t)}(x,V_{x}^{\alpha}(t,x))-S_{\alpha^{\prime}}^{\mu_{G}(t)}(x,V_{x}^{\alpha^{\prime}}(t,x))\|$
	$\displaystyle\leq$	$\displaystyle\|S_{\alpha}^{\mu_{G}(t)}(x,V_{x}^{\alpha}(t,x))-S_{\alpha}^{\mu_{G}(t)}(x,V_{x}^{\alpha^{\prime}}(t,x))\|$
		$\displaystyle+\|S_{\alpha}^{\mu_{G}(t)}(x,V_{x}^{\alpha^{\prime}}(t,x))-S_{\alpha^{\prime}}^{\mu_{G}(t)}(x,V_{x}^{\alpha^{\prime}}(t,x))\|.$

(3.34)		$\displaystyle\|x_{\alpha}(t)-x_{\bar{\alpha}}(t)\|\leq$	$\displaystyle\int_{0}^{t}\|\widetilde{f}[x_{\alpha}(s),\phi_{\alpha}(s,x_{\alpha}(s)\|\mu^{\prime}_{G}(\cdot)),\mu^{\prime}_{G}(s);g_{\alpha}]$
		$\displaystyle\quad-\widetilde{f}[x_{\bar{\alpha}}(s),\phi_{\bar{\alpha}}(s,x_{\bar{\alpha}}(s)\|\mu^{\prime}_{G}(\cdot)),\mu^{\prime}_{G}(s);g_{\bar{\alpha}}]\|ds.$

		$\displaystyle\Big{\|}\int h(x,y)g(\alpha,\beta)\nu_{\beta}^{1}(t,dy)d\beta-\int h(\bar{x},\bar{y})g(\alpha,\beta)\nu_{\beta}^{2}(t,d\bar{y})d\beta\Big{\|}$
	$\displaystyle\leq$	$\displaystyle C_{1}\|x-\bar{x}\|+\sup_{\beta}\Big{\|}\int h(\bar{x},y)\nu_{\beta}^{1}(t,dy)-\int h(\bar{x},\bar{y})\nu_{\beta}^{2}(t,d\bar{y})\Big{\|}$
	$\displaystyle=$	$\displaystyle C_{1}\|x-\bar{x}\|+\sup_{\beta}\Big{\|}\int_{C_{T}}h(\bar{x},X_{t}(\omega))dm_{\beta}(\omega)-\int_{C_{T}}h(\bar{x},X_{t}(\bar{\omega}))d\bar{m}_{\beta}(\bar{\omega})\Big{\|}$
	$\displaystyle\leq$	$\displaystyle C_{1}\|x-\bar{x}\|+C_{2}\sup_{\beta}\int_{C_{T}\times C_{T}}(\|X_{t}(\omega)-X_{t}(\bar{\omega})\|\wedge 1)d\widehat{m}_{\beta}(\omega,\bar{\omega}),$

	$\displaystyle\|\Delta_{s}\|\leq$	$\displaystyle\|\widetilde{f}[x_{\alpha}(s),\phi_{\alpha}(s,x_{\alpha}(s)\|\mu_{G}),\nu_{G}(s);g_{\alpha}]-\widetilde{f}[x_{\alpha}^{\prime}(s),\phi_{\alpha}(s,x^{\prime}_{\alpha}(s)\|\mu_{G}),\bar{\nu}_{G}(s);g_{\alpha}]\|$
		$\displaystyle\hskip-14.22636pt+\|\widetilde{f}[x_{\alpha}^{\prime}(s),\phi_{\alpha}(s,x^{\prime}_{\alpha}(s)\|\mu_{G}),\bar{\nu}_{G}(s);g_{\alpha}]-\widetilde{f}[x_{\alpha}^{\prime}(s),\bar{\phi}_{\alpha}(s,x^{\prime}_{\alpha}(s)\|\bar{\mu}_{G}),\bar{\nu}_{G}(s);g_{\alpha}]\|$
	$\displaystyle\leq$	$\displaystyle C_{1}\|x_{\alpha}(s)-x_{\alpha}^{\prime}(s)\|+C_{2}\sup_{\beta}D_{s}(m_{\beta}^{\rm mv},\bar{m}^{\rm mv}_{\beta})$
(3.45)			$\displaystyle+C_{3}\sup_{t,x}\|\phi_{\alpha}(t,x\|\mu_{G}(\cdot))-\bar{\phi}_{\alpha}(t,x\|\bar{\mu}_{G}(\cdot))\|,$

Graphon Mean Field Games and the GMFG Equations

Abstract.

Key words and phrases:

2020 Mathematics Subject Classification:

1. Introduction

2. The Concept of a Graphon

3. Graphon MFG Systems and the GMFG Equations

3.1. The Standard MFG Model and Its Graphon Generalization

3.2. The Graphon MFG Model and Its Equations

3.3. Existence Analysis

Lemma 3.1.

Proof.

Lemma 3.2.

Proof.

Lemma 3.3.

Proof.

Lemma 3.4.

Proof.

Lemma 3.5.

Proof.

Remark 3.6.

Lemma 3.7.

Proof.

Lemma 3.8.

Proof.

Lemma 3.9.

Proof.

3.4. Existence Theorem

Remark 3.10.

Remark 3.11.

Theorem 3.12.

Proof.

3.5. An Example on Lipschitz feedback

Example 3.13.

Proposition 3.14.

Proof.

4. Performance Analysis

4.1. The ϵ\epsilon-Nash Equilibrium

Remark 4.1.

Remark 4.2.

Proposition 4.3.

Proof.

Theorem 4.4.

4.2. Proof of Theorem 4.4

Lemma 4.5.

Proof.

Lemma 4.6.

Proof.

Lemma 4.7.

Proof.

Lemma 4.8.

Proof.

Lemma 4.9.

Proof.

Lemma 4.10.

Proof.

Lemma 4.11.

Proof.

Lemma 4.12.

Proof.

Lemma 4.13.

Proof.

Lemma 4.14.

Proof.

Lemma 4.15.

Proof.

Lemma 4.16.

Proof.

5. The LQ Case

Appendix

Lemma A.1.

Proof.

Lemma A.2.

Proof.

References

4.1. The $\epsilon$ -Nash Equilibrium