Mean Field Games of Major-Minor Agents with Recursive Functionals

Mean field game (MFG) theory was independently introduced by [27] and [24] from different perspectives, serving as an effective methodology for analyzing controlled large population (LP) systems. Typically, a LP system comprises a large number of agents with interactions through their empirical distribution or averages. These interactions are in a weak sense, as the degrees of couplings among agents diminish rapidly when the number of agents tends to infinity. A core element of the MFG theory is the construction of a limiting auxiliary problem, under which all agents can be largely disentangled, allowing for characterizations of some decentralized approximate equilibrium through a consistency matching. As such, MFG analysis can significantly reduce the dimension of controlled LP systems that each agent needs to analyze, greatly simplifying related numerical analysis. A substantial body of research has been dedicated to the MFG theory, yielding fruitful outcomes. A partial list of literature relevant to this current work includes [5, 6, 8, 11, 22, 23, 32, 34]. This paper focuses on a new class of MFG problems with the following weakly-coupled LP system including a major $\mathcal{A}_{0}$ and multiple minors $\{\mathcal{A}_{i}\}_{i=1}^{N}$, whose states $X^{0}$ and $\{X^{i}\}_{i=1}^{N}$ satisfy the following forward stochastic differential equations (SDEs)

with initial conditions $X_{0}^{0}=x_{0}^{0}\in\mathbb{R}^{n},X_{0}^{i}=x_{0}\in\mathbb{R}^{n}.$ Here, $X_{\cdot}^{(N)}=\frac{1}{N}\sum_{i=1}^{N}X_{\cdot}^{i},\ u_{\cdot}^{(N)}=\frac{1}{N}\sum_{i=1}^{N}u_{\cdot}^{i}$ stand for the (forward) state-average and control-average of all minors, respectively, and $W=(W^{0},W^{1},$ $\cdots,W^{N})^{\top}$ is a $(N+1)$-dimensional standard Brownian motion on a complete probability space $(\Omega,\mathcal{F},{\mathbb{P}})$, where $W^{i}$ is the idiosyncratic noise for $\mathcal{A}_{i}$ while $W^{0}$ is the common noise. $\{\mathcal{A}_{k}\}_{k=0}^{N}$ aim to maximize recursive-type functionals $\{J_{k}\}_{k=0}^{N}$ given by

where $u^{-i}=\{\cdots,u^{i-1},u^{i+1},\cdots\}$ is the control profile except that of $\mathcal{A}_{i}$; $\Theta_{\cdot}^{(N)}=(X_{\cdot}^{(N)},Y_{\cdot}^{(N)},Z_{\cdot}^{(N)})$ is state-average triple with $Y^{(N)}_{\cdot}=\frac{1}{N}\sum_{i=1}^{N}Y_{\cdot}^{i}$ as the recursive state average and $Z^{(N)}_{\cdot}=\frac{1}{N}\sum_{i=1}^{N}Z_{\cdot}^{i}$ the intensity state average (see [12]); the state-triple $\{\Theta_{\cdot}^{k}=(X_{\cdot}^{k},Y_{\cdot}^{k},{Z}_{\cdot}^{k})\}_{k=0}^{N}$ satisfy (1.1) and the backward SDE (BSDE) motivated by recursive utilities in economics [15, 28, 29]:

where for $i=1,\cdots,N$, $Z^{i}=(Z^{i,0},Z^{i,i},{\{Z^{i,j}\}_{j\neq i,j=1}^{N}})$ and $(Z^{i,0},Z^{i,i})$ is the principal intensity component, and $\{Z^{i,j}\}_{j\neq i}$ the marginal components. Since remainder terms $\sum_{j=1}^{N}Z_{t}^{0,j}dW_{t}^{j}$ and $\sum_{j=1,j\neq i}^{N}Z_{t}^{i,j}dW_{t}^{j}$ vanish as $N\rightarrow\infty$ (see Remark 3.1), we focus on the principal terms and set $Z^{(N)}_{\cdot}=\frac{1}{N}\sum_{i=1}^{N}(Z^{i,0}_{\cdot},Z^{i,i}_{\cdot})$ as averages on principal intensities.

We refer (1.1)-(1.3) as the recursive major-minor (RMM) problem. We defer its detailed assumptions later, and highlight its modeling features first as follows.

Modeling features. (i) The RMM model delves into the interaction between a major $\mathcal{A}_{0}$ and a large number of minors $\{\mathcal{A}_{i}\}_{i=1}^{N}$. Traditional MFG studies assume that agents are all “minor” or “negligible”, meaning that an individual agent’s action cannot significantly impact the behaviors of the population at a macro scale. Associated MFGs are thus referred to as “symmetric” because it suffices to examine a representative agent, provided agents are homogenous hence statistically exchangeable. In contrast, our RMM explores asymmetric interactions where agents having varying decisional capacities. A major agent may significantly influence the population’s behavior through her own decisions, whereas numerous minor agents can only affect the population through collective actions. This model is more realistic than the homogeneous minor setting, as it captures a range of diversified interaction mechanisms, see [5, 11, 23, 32, 34].

(ii) The RMM model further posits that objectives of all agents are represented recursively through nonlinear BSDEs, such as (1.3), with non-additive drivers $f^{0}$ or $f$. The inclusion of recursive functionals in MFG studies is motivated by their advantageous decision-theoretic properties, especially in the current LP context featuring complex decision couplings. Indeed, recursive functionals are well-suited for decision theory due to their capability to explain various observed non-standard decision behaviors, such as the separation of inter-temporal substitution and risk aversion. Consistently, recursive functionals extend classical expected functionals (see [4, 7, 16]), which are relevant to a special class of BSDEs with additive drivers.

(iii) The RMM problem restricts its weak coupling to empirical averages, refraining from discussing a more extensive empirical measures or distributions. Despite this limitation, the weak coupling of RMM remains quite general, as it is integrated into both the dynamics and payoff functionals of the major $\mathcal{A}_{0}$ and all minors $\{\mathcal{A}_{i}\}_{i=1}^{N}$, encompassing elements from both the state and the control. Moreover, due to recursive functionals, the state averages is enriched by including not only $X^{(N)}$ on the objective (forward) states; but also $(Y^{(N)},Z^{(N)})$ on the (recursive and intensity) states reflecting the subjective averaged-out beliefs. Specifically, the intensity coupling $Z^{(N)}$ characterizes some average on risk (ambiguity) aversion across all agents.

Literature comparison. [10, 23] and [32] introduced the major-minor MFG within a linear-quadratic-Gaussian (LQG) framework on finite and infinite horizon, respectively. They employed augmented Riccati equations to characterize consistency conditions. [34] extended these major-minor studies to a nonlinear setting using the stochastic Hamilton-Jacobi-Bellman (HJB) approach, where the weak coupling is restrictive; for instance, the major’s state cannot enter the dynamics of the minors. Besides, [5] investigated a class of major-minor MFG problems also through the stochastic HJB approach. [8] studied major-minor MFG by master equations where the agents take closed loop control. Recently, [6] investigated a type of MFG problems with asymmetric information between major and minor agents. [11] explored nonlinear major-minor MFG with general weak couplings, allowing the major’s state to enter the dynamics of the minor agents. Additionally, the limiting control problem of the major agent incorporates an endogenous mean-field term, based on an approximation through a two-agent non-zero-sum game. A forward type of maximum principle was utilized in this context.

Our paper distinguishes itself from aforementioned works by its focus on a nonlinear major-minor interaction with recursive functionals, and associated methodology of a backward-forward type of stochastic maximum principle. Our RMM modeling is particularly noteworthy for its introduction and detailed analysis of the weak couplings of the backward (recursive and intensity) state-average $(Y^{(N)},Z^{(N)})$ originating from the recursive functional. As previously mentioned, these couplings hold significant decision-making impacts and, to the best of our knowledge, have not been systematically addressed in the MFG literature. Consequently, the maximum principle we adopt and the consistency condition we derive take unique forms that differ from those in [11]. Additionally, we apply our general nonlinear outcomes to specific RMM problems in LQG settings. Our LQG-RMM studies not only recap and extend existing results on the forward MFG studies, but also provide new insights into its backward counterpart.

Another relevant work is [7], which also explored the major-minor interaction and recursive functionals. However, it is framed on a weak formulation, substantially different from our strong formulation. Specifically, the approach of [7] originates from a variant of the Girsanov transformation and optimization of a Hamiltonian function, whereas our analysis is rooted in a refined backward-forward stochastic maximum principle. More significantly, all minors in [7] are cooperative, and the associated MFG thus encompasses a two-layer mixed structure: all (cooperative) minor agents form a mean-field team (rather than a game) in an inner layer, while in an outer layer, the interacting major agent and a representative minor agent induce a two-person, non-mean-field game. Although also termed a MFG, the model in [7] is essentially a hybrid of a mean-field team and a non-mean-field-type two-person game, which contrasts with the decision structure we investigate. Consequently, the consistency matching, a central step in MFG analysis, is not applicable in [7] at all. In contrast, our work is distinguished by a novel scheme to auxiliary control construction and consistency matching, as detailed below. Furthermore, the limiting equilibria in [7] are characterized as saddle points, which are remarkably different from our non-zero-sum setup. Additionally, the admissible controls in [7] take feedback forms and are compact, unlike our open-loop and unbounded convex admissibility.

A unified structural scheme. Last but not least, as the core element in MFG, the auxiliary problem of the RMM is formulated through a novel structural scheme, which incorporates a bilateral perturbation and a hierarchical recomposition. This scheme facilitates a more incisive auxiliary construction through a sequential network, enabling a clear-cut realization of a complex mixture involving two minor agents: one is exogenous and the other is endogenous, alongside the endogenous major agent. This scheme is new in the MFG literature and distinguishes our work from previous studies, where auxiliary constructions are based on heuristic arguments. More importantly, such scheme offers a unified methodology to tackle more complex LP interactions for which heuristic arguments are no longer tractable. For instance, when the LP system consists of heterogenous agents with varying beliefs on model uncertainties.

Contributions. (i) We introduce a new class of RMM problems featuring major-minor asymmetric interactions and recursive objectives. (ii) We present a novel unified structural scheme to construct its pivotal auxiliary problem. (iii) We derive a new class of mean-field type of forward-backward SDEs (FBSDEs) to characterize the consistency condition of the RMM problem. (iv) We examine LQG studies in the RMM context in detail to gain deeper insights.

The remainder of this paper is organized as follows: Section 2 introduces basic assumptions of the RMM problem. Section 3 presents a unified structural scheme for the RMM problem, including a bilateral perturbation and a mixed triple-agent two-layer analysis. Section 4 studies the auxiliary control and associated consistency condition (CC) of the RMM problem, and verifies its approximate Nash equilibrium. Section 5 devotes to some LQG-RMM problems. Section 6 concludes, and some technical proofs and heavy notations are given in Appendix.

For $i=0,1,\cdots,N$, let $\mathbb{F}^{i}=\{{\mathcal{F}}_{t}^{i}\}$ be the complete filtration generated by the Brownian motion $W^{i}$; namely, ${\mathcal{F}}_{t}^{i}=\sigma(W_{i}(s),0\leq s\leq t)\vee\mathcal{N}_{\mathbb{P}}$ with $\mathcal{N}_{\mathbb{P}}$ the set of all $\mathbb{P}$-null sets in $\mathcal{F}$. Then, $\mathbb{F}=\{{\mathcal{F}}_{t}\}_{0\leq t\leq T}=\{\sigma(\bigcup_{i=0}^{N}{\mathcal{F}}_{t}^{i})\}_{0\leq t\leq T}$ denotes the centralized information generated by the Brownian motion $W$, and by $\mathbb{F}^{i,0}=\{{\mathcal{F}}^{i,0}_{t}\}_{0\leq t\leq T}=\{\sigma({\mathcal{F}}_{t}^{i}\bigcup{\mathcal{F}}_{t}^{0})\}_{0\leq t\leq T}$ the decentralized information for a generic minor agent $\mathcal{A}_{i}$, $i=1,\cdots,N$. Let $U_{0}\subset\mathbb{R}^{k_{0}}$ and $U\subset\mathbb{R}^{k}$ be two convex sets.

The exact Nash equilibrium corresponds to the case when $\varepsilon=0$. Now we impose some assumptions on the following coefficients of (1.1)-(1.3) of the RMM problem:

Assumption (A1) (i) $(b^{0},\sigma^{0})$ and $(b,\sigma)$ are continuously differentiable in $(x^{0},u^{0};x^{(N)},u^{(N)})$ and $(x,u;x^{0},u^{0};$ $x^{(N)},u^{(N)})$, respectively. All the derivatives of $(b^{0},\sigma^{0},b,\sigma)$ are bounded.
(ii) $(f^{0},g^{0})$ and $(f,g)$ are continuously differentiable in $(\theta^{0},\theta^{(N)})$ and $(\theta,\theta^{0},\theta^{(N)})$, respectively, where $\theta^{0}=(x^{0},y^{0},z^{0},u^{0}),\ \theta^{(N)}=(x^{(N)},y^{(N)},z^{(N)},u^{(N)}),\ \theta=(x,y,z,u).$ $\Phi^{0}$, $\Phi$ and $(\Gamma^{0},\Gamma)$ are continuously differentiable in $(x^{0},x^{(N)})$, $(x,x^{0},x^{(N)})$ and $y$, respectively.
(iii) All derivatives of $(f^{0},f,\Phi^{0},\Phi)$ are bounded. The derivatives of $g^{0},g$ and $(\Gamma^{0},\Gamma)$ are bounded by $C(1+|\theta^{0}|+|\theta^{(N)}|)$, $C(1+|\theta|+|\theta^{0}|+|\theta^{(N)}|)$ and $C(1+|y|)$, respectively, for some $C>0$.
(iv) $\xi^{0}$ and $\xi^{i}$, $1\leq i\leq N$, are square-integrable, $\mathcal{F}_{T}^{0}$- and $\mathcal{F}_{T}^{i,0}$-measurable, respectively. Moreover, $\xi^{1},\cdots,\xi^{N}$ are independent and identically distributed conditionally on $\mathcal{F}_{T}^{0}$.
(v) $(b^{0},\sigma^{0},b,\sigma;f^{0},f,g^{0},g)$ are uniformly continuous in $t$.

For each admissible $(N+1)$-tuple $\{u^{j}\}_{j=0}^{N}$, the coupled SDE system (1.1) (resp. BSDE (1.3)) admits a unique solution tuple $\{X^{j}\}_{j=0}^{N}$ (resp. $\{Y^{j},Z^{j}\}_{j=0}^{N}$) under (A1), and the recursive-type payoffs $\{J_{i}\}_{i=0}^{N}$ are well-defined.

For a fixed $N$, if each agent has access to centralized information about all agents, including their instantaneous states realized and controls adopted, the RMM becomes a classical but high-dimensional $(N+1)$-agent game. Existence or uniqueness of its exact Nash equilibrium(s) can be ensured under certain mild but high-dimensional conditions, including semi-continuity, coercivity and concavities on $\{J_{i}\}_{0=1}^{N}$, or compactness on control admissibility. The open-loop (exact) equilibrium, denoted as $\{u^{i,*}\}_{i=0}^{N}\in{\mathcal{U}_{0}^{c}}\times\prod_{i=1}^{N}\mathcal{U}_{i}^{c}$, can be further characterized through a system of $N+1$ stationary conditions. However, this procedure to exact equilibriums is only feasible in theory and becomes impractical due to the curse of dimensionality when $N$ is large.

MFG theory offers one resolution to constructing near-optimal decentralized strategies, as an alternative approximation to the exact Nash equilibria. A key challenge in MFG is the construction of an auxiliary problem for dimension reduction. Previous MFG studies have constructed auxiliary problems intuitively based on heuristic arguments, effective only when the underlying coupling structure is not overly complex. For instance, when all agents are symmetric as in [14, 22, 24, 27], or even an asymmetric dominant major agent is included as in [5, 6, 8, 10, 11, 23, 32, 34]. However, heuristic analysis becomes infeasible to analyze LP systems with more intricate couplings. One reason is that it fails to effectively configure a complex logic network in which various representative agents shall be mutually connected by an “exogenous—endogenous” relation. Alternatively, we propose a structural scheme that can not only well amount for the extreme generality of the weak coupling in RMM problems, but also lay down an unified foundation to analyze more general and complex LP couplings. In current RMM context, this scheme yields a bilateral perturbation and a triple-agent two-layer game, as discussed below.