\authormark

QI et al.

\corres

Xinbei Lv, Qingdao Innovation and Development Center, Harbin Engineering University, Qingdao, China 266000.

Decentralized Control for Discrete-time Mean-Field Systems with Multiple Controllers of Delayed Information

Qingyuan Qi Zhiqiang Liu Qianqian Zhang Xinbei Lv \orgdivQingdao Innovation and Development Center, \orgnameHarbin Engineering University, \orgaddress\stateQingdao, \countryChina \orgdivInstitute of Complexity Science, School of Automation, \orgnameQingdao University, \orgaddress\stateQingdao, \countryChina lvxinbei@hrbeu.edu.cn

Abstract

[Abstract]In this paper, the finite horizon asymmetric information linear quadratic (LQ) control problem is investigated for a discrete-time mean field system. Different from previous works, multiple controllers with different information sets are involved in the mean field system dynamics. The coupling of different controllers makes it quite difficult in finding the optimal control strategy. Fortunately, by applying the Pontryagin’s maximum principle, the corresponding decentralized control problem of the finite horizon is investigated. The contributions of this paper can be concluded as: For the first time, based on the solution of a group of mean-field forward and backward stochastic difference equations (MF-FBSDEs), the necessary and sufficient solvability conditions are derived for the asymmetric information LQ control for the mean field system with multiple controllers. Furthermore, by the use of an innovative orthogonal decomposition approach, the optimal decentralized control strategy is derived, which is based on the solution to a non-symmetric Riccati-type equation.

keywords:

Pontryagin’s maximum principle, asymmetric information control, mean-field system, orthogonal decomposition approach

^†^†articletype: Research Article

1 Introduction

Different from the controlled linear stochastic differential/difference equations (SDEs) studied in classical stochastic control problem, the mathematical expectation terms appear in the mean-field SDEs. Due to the applications in large population stochastic dynamics games, and various physical and sociological dynamical systems, the study of mean-field SDEs can be traced back to 1950s, and abundant research results have been obtained, see ¹-¹³. Particularly, the continuous time mean-field LQ control problem was firstly studied in ³, and a necessary and sufficient solvability condition was proposed. Furthermore, both the finite horizon LQ control problem and the infinite horizon stabilization control problem for discrete time mean-field systems were solved in ⁷, ⁸. Moreover, the indefinite LQ control for mean-field systems is investigated in ⁹, ¹⁰.

It is important to note that the previously mentioned studies on mean-field control problem are mainly symmetric information control problem, only a centralized control strategy needs to be designed. As is well known, the traditional centralized control problem with one single controller were well studied since the last century. In addition, for the system dynamics with multiple controllers of the same information structure, the corresponding decentralized control problems can be converted into a centralized control problem by using the system augmentation approach. Decentralized control systems have multiple controllers that are collaboratively trying to control a system by taking actions based on their individual observations. The observations of one controller may not be available to the other controllers. Whereas, in contrast to centralized control systems, the decentralized control problem with controllers of asymmetric information structure remains less investigated.

We should emphasis that real world systems usually contain multiple controllers, and each controller accesses its individual information. For this situation, finding optimal control strategy is usually difficult in view of the coupling of different controllers. The pioneering study of asymmetric information control is the well-celebrated Witsenhausen’s counterexample raised in 1968 (see ¹⁴), which shows that the linear control strategy is no longer optimal for a decentralized control of linear dynamical system, and the associated optimal control problem remains unsolved. Since then, in view of the wide applications in many fields, the research of decentralized control with asymmetric information controllers has attracted much interest from researchers in recent years. For example, the stochastic game problem with asymmetric information was investigated and solved in ²¹, ²². The optimal local and remote decentralized control problem was investigated via the common information approach, see ¹⁶, ¹⁷. Besides, the decentralized control for linear stochastic system with multiple controllers of different information structures was studied in ¹⁵-¹⁸.

Different from the previous works mentioned above, we will investigate a special kind of decentralized control problem for discrete-time mean-field systems. Specifically, multiple controllers with delayed information patterns as well as the mean-field terms (i.e., the expectations of the controllers and the state) are contained within the system dynamics. Meanwhile, each controller can access its individual information, which is different with each other. Our goal is to design the optimal decentralized control strategy to minimize a given quadratic cost function. It should be pointed out that the studied problem has not been solved so far. The existence of mean-field terms and the delayed asymmetric information structure make the studied problem challenging in the following aspects: 1) In view of the asymmetric information structure, the controllers are coupled with each other, hence finding the optimal decentralized control is difficult, see ¹⁶, ¹⁷; 2) Due to the existence of the mean-filed terms, the original optimal decentralized control problem cannot be considered as a standard LQ control problem, and the explicit solvability conditions have not been derived, see ³.

In this paper, by applying the maximum principle, the corresponding decentralized control problem for mean-field systems with controllers of delayed information structures is well studied. In the first place, by the use of the variational method, the necessary and sufficient solvability conditions are given in accordance with a group of MF-FBSDEs. Subsequently, in order to decouple the associated MF-FBSDEs, an innovative orthogonal decomposition approach is proposed. Hence, in view of the delayed information structure, it is shown that the MF-FBSDEs are decoupled, and the relationship between the costate and the state is established. Finally, the optimal decentralized control strategy is derived by introducing asymmetric Riccati equations.

Very recently, the optimal LQ control problem with multiple controllers of non-symmetric information structure was investigated in ¹⁸. It is worth noting that the adopted methodology exhibits innovation in the following aspects: Firstly, to deal with the mean field terms in mean-field system, a group of mean-field forward and backward stochastic difference equations (MF-FBSDEs) is firstly introduced. Obviously, the MF-FBSDEs were not mentioned in reference ¹⁸, and we show that the optimal decentralized control can be uniquely solved if and only if the MF-FBSDEs can be uniquely decoupled. Secondly, as a result, in order to obtain the optimal decentralized control for mean-field systems, we introduce a novel technique for decoupling the MF-FBSDEs. Specifically, we derive the control mathematical expectation first, followed by obtaining the optimal decentralized control. It is worth noting that this approach has not been utilized in reference ¹⁸. Furthermore, this paper demonstrates that the optimal decentralized control can be computed by solving a set of new asymmetric Riccati equations.

The contributions of the paper are: 1) The necessary and sufficient conditions for the discrete-time mean field decentralized control problem with multiple controllers of delayed information patterns are derived. 2) Utilizing a novel orthogonal decomposition approach, we have successfully obtained the solution to the group of MF-FBSDEs, which, to the best of our knowledge, represents a new and original contribution. 3) With regards to the optimal predictor, we have successfully overcome the challenges associated with the coupling between multiple controllers, enabling us to derive the optimal decentralized control for the first time.

The necessary and sufficient conditions for the discrete-time mean field decentralized control problem with multiple controllers of delayed information patterns are derived. 2) According to a novel orthogonal decomposition approach, the solution to the group of MF-FBSDEs is obtained, which is new as far as our knowledge. 3) In terms of the optimal predictor, the barrier of the coupling between the multiple controllers are overcome, the optimal decentralized control are thus derived for the first time.

The structure of the paper is as follows. In Section 2, the problem under consideration is formulated, while the solvability conditions of the problem is shown in Section 3. The optimal decentralized control strategy is developed in Section 4. A numerical example is given in Section 5, and we conclude the paper in Section 6. Finally, there are some relevant detailed proofs in Appendix.

Notations: $I_{n}$ means the unit matrix with rank $n$ ; Superscript $\mathcal{A}^{T}$ denotes the transpose of a matrix. Real symmetric matrix $\mathcal{A}>0\ (\text{or}\geq 0)$ implies that $\mathcal{A}$ is strictly positive definite (or positive semi-definite). $\mathbb{R}^{n}$ signifies the $n$ -dimensional Euclidean space. $\mathcal{B}^{-1}$ represents the inverse of real matrix $\mathcal{B}$ . Denote the natural filtration $\mathcal{F}_{i}(\tau)$ as an the $\sigma$ -algebra generated by $\{x(0),\omega(0),\cdots,\omega(\tau-i-1)\}$ and augmented by all the $\mathcal{P}$ -null sets. $\mathbb{E}[\cdot|\mathcal{F}_{i}(\tau)]$ means the conditional expectation with respect to $\mathcal{F}_{i}(\tau)$ . For the convenience of presentation, the following concise notations are introduced.

\mathbf{R}=\begin{bmatrix}\mathcal{R}_{0}&0&\cdots&0\\ 0&\mathcal{R}_{1}&\cdots&0\\ \vdots&\vdots&\ddots&\vdots\\ 0&0&\cdots&\mathcal{R}_{h}\end{bmatrix},\mathbf{\bar{R}}=\begin{bmatrix}\bar{\mathcal{R}}_{0}&0&\cdots&0\\ 0&\bar{\mathcal{R}}_{1}&\cdots&0\\ \vdots&\vdots&\ddots&\vdots\\ 0&0&\cdots&\bar{\mathcal{R}}_{h}\end{bmatrix},\\

\mathbf{B}=\begin{bmatrix}\mathcal{B}{{}_{0}}^{T}\\ \mathcal{B}{{}_{1}}^{T}\\ \vdots\\ \mathcal{B}{{}_{h}}^{T}\end{bmatrix}^{T},\mathbf{\bar{B}}=\begin{bmatrix}\bar{\mathcal{B}}{{}_{0}}^{T}\\ \bar{\mathcal{B}}{{}_{1}}^{T}\\ \vdots\\ \bar{\mathcal{B}}{{}_{h}}^{T}\end{bmatrix}^{T},\mathbf{D}=\begin{bmatrix}D{{}_{0}}^{T}\\ D{{}_{1}}^{T}\\ \vdots\\ D{{}_{h}}^{T}\end{bmatrix}^{T},\mathbf{\bar{D}}=\begin{bmatrix}\bar{D}{{}_{0}}^{T}\\ \bar{D}{{}_{1}}^{T}\\ \vdots\\ \bar{D}{{}_{h}}^{T}\end{bmatrix}^{T},\\

\mathbf{V}(\tau)=\begin{bmatrix}v{{}_{0}}(\tau)\\ v{{}_{1}}(\tau)\\ \vdots\\ v{{}_{h}}(\tau)\end{bmatrix},\Delta\mathbf{V}(\tau)=\begin{bmatrix}\Delta v{{}_{0}}(\tau)\\ \Delta v{{}_{1}}(\tau)\\ \vdots\\ \Delta v{{}_{h}}(\tau)\end{bmatrix},\mathbb{E}\mathbf{V}(\tau)=\begin{bmatrix}\mathbb{E}v{{}_{0}}(\tau)\\ \mathbb{E}v{{}_{1}}(\tau)\\ \vdots\\ \mathbb{E}v{{}_{h}}(\tau)\end{bmatrix},\mathbb{E}\Delta\mathbf{V}(\tau)=\begin{bmatrix}\mathbb{E}\Delta v{{}_{0}}(\tau)\\ \mathbb{E}\Delta v{{}_{1}}(\tau)\\ \vdots\\ \mathbb{E}\Delta v{{}_{h}}(\tau)\end{bmatrix},\\

\mathcal{A}(\tau)=\mathcal{A}+\omega(\tau)C,\bar{\mathcal{A}}(\tau)=\bar{\mathcal{A}}+\omega(\tau)\bar{C},\\

\mathcal{B}_{i}(\tau)=\mathcal{B}_{i}+\omega(\tau)D_{i},\bar{\mathcal{B}}_{i}(\tau)=\bar{\mathcal{B}}_{i}+\omega(\tau)\bar{D}_{i},\\

\mathbf{B}(\tau)=\mathbf{B}+\omega(\tau)\mathbf{D},\bar{\mathbf{B}}(\tau)=\bar{\mathbf{B}}+\omega(\tau)\bar{\mathbf{D}},\\

\mathbf{R}_{i}=\begin{bmatrix}\mathcal{R}_{0}&0&\cdots&0\\ 0&\mathcal{R}_{1}&\cdots&0\\ \vdots&\vdots&\ddots&\vdots\\ 0&0&\cdots&\mathcal{R}_{i}\end{bmatrix},\mathbf{\bar{R}}_{i}=\begin{bmatrix}\bar{\mathcal{R}}_{0}&0&\cdots&0\\ 0&\bar{\mathcal{R}}_{0}&\cdots&0\\ \vdots&\vdots&\ddots&\vdots\\ 0&0&\cdots&\bar{\mathcal{R}}_{i}\end{bmatrix},\\

\mathbf{V}_{i}(\tau)=\begin{bmatrix}\hat{v}_{0,i}(\tau)\\ \vdots\\ \hat{v}_{i-1,i}(\tau)\\ v_{i}(\tau)\end{bmatrix},\mathbb{E}\mathbf{V}_{i}(\tau)=\begin{bmatrix}\mathbb{E}\hat{v}_{0,i}(\tau)\\ \vdots\\ \mathbb{E}\hat{v}_{i-1,i}(\tau)\\ \mathbb{E}v_{i}(\tau)\end{bmatrix},

(1)

\mathbf{B}_{i}=\begin{bmatrix}\mathcal{B}_{0}^{T}\\ \mathcal{B}_{1}^{T}\\ \vdots\\ \mathcal{B}_{i}^{T}\end{bmatrix},\mathbf{\bar{B}}_{i}=\begin{bmatrix}\bar{\mathcal{B}}_{0}^{T}\\ \bar{\mathcal{B}}_{1}^{T}\\ \vdots\\ \bar{\mathcal{B}}_{i}^{T}\end{bmatrix},\mathbf{D}_{i}=\begin{bmatrix}D_{0}^{T}\\ D_{1}^{T}\\ \vdots\\ D_{i}^{T}\end{bmatrix},\mathbf{\bar{D}}_{i}=\begin{bmatrix}\bar{D}_{0}^{T}\\ \bar{D}_{1}^{T}\\ \vdots\\ \bar{D}_{i}^{T}\end{bmatrix},\\

\mathbf{B}_{i}(\tau)=\mathbf{B}_{i}+\omega(\tau)\mathbf{D}_{i},\mathbf{\bar{B}}_{i}(\tau)=\mathbf{\bar{B}}_{i}+\omega(\tau)\mathbf{\bar{D}}_{i},\text{for}\ i=0,\cdots,h.

2 Problem Formulation

We consider the following discrete-time mean-field stochastic system with multiple controllers

\displaystyle x(\tau+1)=\Big{[}\mathcal{A}x(\tau)+\bar{\mathcal{A}}\mathbb{E}x(\tau)+\sum_{i=0}^{h}[\mathcal{B}_{i}v_{i}(\tau)+\bar{\mathcal{B}}_{i}\mathbb{E}v_{i}(\tau)]\Big{]}+\Big{[}Cx(\tau)+\bar{C}\mathbb{E}x(\tau)+\sum_{i=0}^{h}[D_{i}v_{i}(\tau)+\bar{D}_{i}\mathbb{E}v_{i}(\tau)]\Big{]}\omega(\tau),

(2)

where $\tau$ is the time instant, $x(\tau)\in{\mathbb{R}^{n}}$ is the system state, $v_{i}(\tau)\in\mathbb{R}^{m_{i}}$ is the $i$ -th control input. $\mathcal{A},\bar{\mathcal{A}},C,\bar{C}\in\mathbb{R}^{n\times n}$ , and $\mathcal{B}_{i},\bar{\mathcal{B}}_{i},D_{i},\bar{D}_{i}\in\mathbb{R}^{n\times m},0\leq i\leq h$ are matrices of appropriate dimensions, $\omega(\tau)$ is a scalar-valued Gaussian white noise with $\omega(\tau)\sim\mathcal{N}(0,\sigma^{2})$ . The initial state $x(0)=\beta$ is given, and $\mathbb{E}$ is the expectation taken over the noise $\omega(\tau)$ and initial state $\beta$ .

To guarantee the solvability of system (2), the initial control strategies $v_{i}(\tau)(0\leq i\leq h,0\leq\tau\leq i-1)$ are given arbitrarily.

It is noted that the expectations $\mathbb{E}x(\tau),\mathbb{E}v_{i}(\tau)$ are involved in the system dynamics (2), which will cause essential difficulties in deriving the optimal decentralized control strategy. For the sake of discussion, it can be derived from (2) that

\displaystyle\mathbb{E}x(\tau+1)=(\mathcal{A}+\bar{\mathcal{A}})\mathbb{E}x(\tau)+\sum_{i=0}^{h}(\mathcal{B}_{i}+\bar{\mathcal{B}}_{i})\mathbb{E}v_{i}(\tau).

(3)

Throughout this paper, for (2)-(3), the following assumption on the information structure is made: {assumption} The $i$ -th controller $v_{i}(\tau)$ is $\mathcal{F}_{i}(\tau)$ -measurable, in which $\mathcal{F}_{i}(\tau)$ is subject to $\mathcal{F}_{i}(\tau)=\sigma\{\beta,\omega(0),\cdots,\omega(\tau-i-1)\}.$

Clearly, for $0\leq i\leq h$ and $i\leq\tau\leq\Gamma$ , we can infer $\mathcal{F}_{0}(\tau)\supset\mathcal{F}_{1}(\tau)\supset\cdots\supset\mathcal{F}_{h}(\tau),h\leq\tau,\text{and}\ \mathcal{F}_{i}(\Gamma)\supset\mathcal{F}_{i}(\Gamma-1)\supset\cdots\supset\mathcal{F}_{i}(\tau),i\leq\tau.$

Remark 2.1.

For mean-field system (2), $h+1$ controllers are involved, and each controller can access its individual information, which is different with each other. Such kind of system is called asymmetric information stochastic system, and the corresponding optimal control problem turns into a decentralized control problem, and finding the optimal decentralized control strategy is usually difficult, see ¹⁶, ¹⁷, ¹⁸, ¹⁹, ²⁰, ²¹.

Corresponding with system (2)-(3), the cost functional is given by

	$\displaystyle J_{\Gamma}(v)=$	$\displaystyle\sum_{\tau=0}^{\Gamma}\mathbb{E}\Big{[}x^{T}(\tau)Qx(\tau)+[\mathbb{E}x(\tau)]^{T}\bar{Q}\mathbb{E}x(\tau)+\sum_{i=0}^{h}[v_{i}^{T}(\tau)\mathcal{R}_{i}v_{i}(\tau)+[\mathbb{E}v_{i}(\tau)]^{T}\bar{\mathcal{R}}_{i}\mathbb{E}v_{i}(\tau)]\Big{]}$
		$\displaystyle+\mathbb{E}[x^{T}(\Gamma+1)\Phi(\Gamma+1)x(\Gamma+1)]+[\mathbb{E}x(\Gamma+1)]^{T}\bar{\Phi}(\Gamma+1)\mathbb{E}x(\Gamma+1),$		(4)

where $Q,\bar{Q},\mathcal{R}_{i},\bar{\mathcal{R}}_{i},\Phi(\Gamma+1),\bar{\Phi}(\Gamma+1)$ are deterministic symmetric weighting matrices with appropriate dimensions.

For the weighting matrices in (2), we might as well assume: {assumption} $Q\geq 0,Q+\bar{Q}\geq 0,\mathcal{R}_{i}>0,\mathcal{R}_{i}+\bar{\mathcal{R}}_{i}>0,\Phi(\Gamma+1)\geq 0,\Phi(\Gamma+1)+\bar{\Phi}(\Gamma+1)\geq 0.$

In what follows, we will present an introduction to the decentralized control problem of mean-field system with multiple controllers of delayed information.

Problem 2.2.

Find $\mathcal{F}_{i}(\tau)$ -measurable controllers $v_{i}(\tau)\in\mathbb{R}^{m_{i}},i=0,\cdots,h$ to minimize (2).

Remark 2.3.

The solution to Problem 2.2 is hard to obtain, and the optimal decentralized control strategy has not been derived before. The reasons are twofold: 1) The information set available to each controller is different, which results in the coupling of $h+1$ controllers, and makes the finding of optimal decentralized control strategy difficult. 2) The mathematical expectation terms $\mathbb{E}x(\tau),\mathbb{E}v_{i}(\tau)$ appear in (2)-(2), which destroys the adaptability of the control inputs, and consequently, Problem 2.2 cannot be solved by applying traditional methods such as system augmentation.

3 Existence of Optimal Decentralized Control Strategy

We are going to consider the existence of the solution to Problem 2.2 via the variational method. To begin with, the following lemma shall be given, which serves as the preliminary.

Lemma 3.1.

For (2) and (2), set $\varepsilon\in\mathbb{R}$ , and denote $v_{i}^{\varepsilon}(\tau)=v_{i}(\tau)+\varepsilon\Delta v_{i}(\tau)$ , $0\leq i\leq h,i\leq\tau\leq\Gamma$ , in which $\Delta v_{i}(\tau)$ is $\mathcal{F}_{i}(\tau)$ -adapted with $\sum_{\tau=0}^{\Gamma}\Delta{v_{i}}^{T}(\tau)\Delta v_{i}(\tau)<+\infty$ . We have

	$\displaystyle J_{\Gamma}(v^{\varepsilon})$	$\displaystyle-J_{\Gamma}(v)=\varepsilon^{2}\Delta J_{\Gamma}(\Delta v)$
		$\displaystyle+2\varepsilon\sum_{t=0}^{\Gamma}\mathbb{E}\Big{[}\big{[}\mathbf{B}^{T}(\tau)\theta(\tau)+\mathbb{E}[\mathbf{\bar{B}}^{T}(\tau)\theta(\tau)]+\mathbf{R}\mathbf{V}(\tau)+\mathbf{\bar{R}}\mathbb{E}\mathbf{V}(\tau)\big{]}^{T}\Delta\mathbf{V}(\tau)\Big{]}.$		(5)

In the above, $x^{\varepsilon}(\tau)$ and $J_{\Gamma}(v^{\varepsilon})$ are the state variable and the cost functional corresponding with $v_{i}^{\varepsilon}(\tau)$ , respectively. Moreover, $\Delta J_{\Gamma}(\Delta v)$ can be calculated as

	$\displaystyle\Delta J_{\Gamma}(\Delta v)$	$\displaystyle=\sum_{\tau=0}^{\Gamma}\mathbb{E}\big{[}\eta^{T}(\tau)Q\eta(\tau)+[\mathbb{E}\eta(\tau)]^{T}\bar{Q}\mathbb{E}\eta(\tau)+\Delta\mathbf{V}^{T}(\tau)\mathbf{R}\Delta\mathbf{V}(\tau)+[\mathbb{E}\Delta\mathbf{V}(\tau)]^{T}\bar{R}\mathbb{E}\Delta\mathbf{V}(\tau)\big{]}$
		$\displaystyle+\mathbb{E}[\eta^{T}(\Gamma+1)\Phi(\Gamma+1)\eta(\Gamma+1)]+[\mathbb{E}\eta(\Gamma+1)]^{T}\bar{\Phi}(\Gamma+1)\mathbb{E}\eta(\Gamma+1),$		(6)

in which $\eta(\tau)=\frac{x^{\varepsilon}(\tau)-x(\tau)}{\varepsilon}$ , and the costate $\theta(\tau)$ satisfies the following backward iteration

\displaystyle\theta(\tau-1)=

\displaystyle Qx(\tau)+\bar{Q}\mathbb{E}x(\tau)+\mathbb{E}[\mathcal{A}^{T}(\tau)\theta(\tau)|\mathcal{F}_{0}(\tau)]+\mathbb{E}[\bar{\mathcal{A}}^{T}(\tau)\theta(\tau)],

(7)

with terminal condition $\theta(\Gamma)=\Phi(\Gamma+1)x(\Gamma+1)+\bar{\Phi}(\Gamma+1)\mathbb{E}x(\Gamma+1),$ $\eta(\tau+1)$ satisfies the following iteration

\displaystyle\eta(\tau+1)=

\displaystyle\mathcal{A}(\tau)\eta(\tau)+\bar{\mathcal{A}}(\tau)\mathbb{E}\eta(\tau)+\mathbf{B}(\tau)\Delta\mathbf{V}(\tau)+\bar{\mathbf{B}}(\tau)\mathbb{E}\Delta\mathbf{V}(\tau),\eta(0)=0.

(8)

Proof 3.2.

In view of space limitations, the detailed proof is omitted here, which can be deduced from Lemma 1 of ¹⁸.

Based on the results of Lemma 3.1, we can present the following lemma on the solvability conditions of Problem 2.2.

Lemma 3.3.

Under Assumptions 2-2, Problem 2.2 can be uniquely solved if and only if the following equilibrium condition can be uniquely solvable for $0\leq i\leq h,i\leq\tau\leq\Gamma$ ,

\displaystyle 0=\mathcal{R}_{i}v_{i}(\tau)+\bar{\mathcal{R}}_{i}\mathbb{E}v_{i}(\tau)+\mathbb{E}[\mathcal{B}_{i}^{T}(\tau)\theta(\tau)|\mathcal{F}_{i}(\tau)]+\mathbb{E}[\bar{\mathcal{B}}_{i}^{T}(\tau)\theta(\tau)],

(9)

in which the costate $\theta(\tau)$ satisfies (7).

Proof 3.4.

‘Necessity’: If Problem 2.2 can be uniquely solved and Assumptions 2-2 hold, we will show (9) should be uniquely solved.

In fact, for any $\Delta v_{i}(\tau)$ and $\varepsilon\in\mathbb{R}$ , if we denote $v_{i}(\tau)$ as the optimal control strategy for $0\leq i\leq h,i\leq\tau\leq\Gamma$ , then it can be implied from (3.1) that,

\displaystyle J_{\Gamma}(v^{\varepsilon})-J_{\Gamma}(v)=\varepsilon^{2}\Delta J_{\Gamma}(\Delta v)+2\varepsilon\sum_{\tau=0}^{\Gamma}\sum_{i=0}^{h}\mathbb{E}\Big{[}\big{[}\mathcal{B}_{i}^{T}(\tau)\theta(\Gamma)+\mathbb{E}[\bar{\mathcal{B}}_{i}^{T}(\tau)\theta(\Gamma)]+\mathcal{R}_{i}v_{i}(\tau)+\bar{\mathcal{R}}_{i}\mathbb{E}v_{i}(\tau)\big{]}^{T}\Delta v_{i}(\tau)\Big{]}\geq 0,

(10)

Note that $\Delta J_{\Gamma}(\Delta v)\geq 0$ can be shown from Assumption 2, then we will show (9) holds by contraction. In other words, it is assumed that

\displaystyle R_{i}

\displaystyle v_{i}(\tau)+\bar{R}_{i}\mathbb{E}v_{i}(\tau)+\mathbb{E}[\mathcal{B}_{i}^{T}(\tau)\theta(\tau)|\mathcal{F}_{i}(\tau)]+\mathbb{E}[\bar{\mathcal{B}}_{i}^{T}(\tau)\theta(\tau)]=\Theta_{i}(\tau)\neq 0.

(11)

By letting $\Delta v_{i}(\tau)=\Theta_{i}(\tau)$ , we have

J_{\Gamma}(v^{\varepsilon})-J_{\Gamma}(v)=\varepsilon^{2}\Delta J_{\Gamma}(\Delta v)+2\varepsilon\sum_{\tau=0}^{\Gamma}\sum_{i=0}^{h}\Theta_{i}^{T}(\tau)\Theta_{i}(\tau)\\ .

Obviously, we can always find some $\varepsilon<0$ such that $J_{\Gamma}(v^{\varepsilon})-J_{\Gamma}(v)<0,$ which contradicts with (9). The proof of Necessity is straightforward.

‘Sufficiency’: Suppose (9) is uniquely solvable, we shall prove the uniquely solvable of Problem 2.2. Actually, under Assumption 2, from (3.1) we know that for any $\varepsilon\in\mathbb{R}$ and $\Delta v_{i}(t),$ it follows that

J_{\Gamma}(v^{\varepsilon})-J_{\Gamma}(v)=\varepsilon^{2}\Delta J_{\Gamma}(\Delta v)\geq 0,

it is evident to see that Problem 2.2 is uniquely solvable.

Remark 3.5.

Combining Lemma 3.1 and Lemma 3.3, the necessary and sufficient unique solvability conditions of Problem 2.2 are derived. Subsequently, in order to obtain the optimal decentralized control strategy, we must pay our attention on solving the following MF-FBSDEs,

\displaystyle\left\{\begin{aligned} x(\tau+1)=&\Big{[}\mathcal{A}x(\tau)+\bar{\mathcal{A}}\mathbb{E}x(\tau)+\sum_{i=0}^{h}[\mathcal{B}_{i}v_{i}(\tau)+\bar{\mathcal{B}}_{i}\mathbb{E}v_{i}(\tau)]\Big{]}\\ &+\Big{[}Cx(\tau)+\bar{C}\mathbb{E}x(\tau)+\sum_{i=0}^{h}[D_{i}v_{i}(\tau)+\bar{D}_{i}\mathbb{E}v_{i}(\tau)]\Big{]}\omega(\tau),\\ \theta(\tau-1)=&Qx(\tau)+\bar{Q}\mathbb{E}x(\tau)+\mathbb{E}[\mathcal{A}^{T}(\tau)\theta(\tau)|\mathcal{F}_{0}(\tau)]+\mathbb{E}[\bar{\mathcal{A}}^{T}(\tau)\theta(\tau)],\\ 0=\mathcal{R}_{i}&v_{i}(\tau)+\bar{\mathcal{R}}_{i}\mathbb{E}v_{i}(\tau)+\mathbb{E}[\mathcal{B}_{i}^{T}(\tau)\theta(\tau)|\mathcal{F}_{i}(\tau)]+\mathbb{E}[\bar{\mathcal{B}}_{i}(\tau)^{T}\theta(\tau)],0\leq i\leq h,i\leq\tau\leq\Gamma,\\ \theta(\Gamma)=&\Phi(\Gamma+1)x(\Gamma+1)+\bar{\Phi}(\Gamma+1)\mathbb{E}x(\Gamma+1),\end{aligned}\right.

(12)

which consists of system (2), costate equation (7) and equilibrium condition (9), where (2) is forward and (7) is backward.

4 Optimal Decentralized Control

In this section, we will explore the optimal decentralized control strategy by solving the MF-FBSDEs (12). For this reason, an innovative orthogonal decomposition approach will be adopted.

4.1 Orthogonal Decomposition Approach

By applying the orthogonal decomposition approach, the following lemma can be presented.

Lemma 4.1.

System (2) and cost functional (2) can be rewritten as, respectively

	$\displaystyle x(\tau$	$\displaystyle+1)=\mathcal{A}(\tau)x(\tau)+\bar{\mathcal{A}}(\tau)\mathbb{E}x(\tau)+\mathbf{B}_{i}(\tau)\mathbf{V}_{i}(\tau)+\mathbf{\bar{B}}_{i}(\tau)\mathbb{E}\mathbf{V}_{i}(\tau)$
		$\displaystyle+\sum_{j=0}^{i-1}[\mathcal{B}_{j}(\tau)\tilde{v}_{j,i}(\tau)+\bar{\mathcal{B}}_{j}(\tau)\mathbb{E}\tilde{v}_{j,i}(\tau)]+\sum_{j=i+1}^{h}[\mathcal{B}_{j}(\tau)v_{j}(\tau)+\bar{\mathcal{B}}_{j}(\tau)\mathbb{E}v_{j}(\tau)],$		(13)

and

$\displaystyle J_{\Gamma}(v)$	$\displaystyle=\sum_{\tau=0}^{\Gamma}\mathbb{E}\Big{[}x^{T}(\tau)Qx(\tau)+[\mathbb{E}x(\tau)]^{T}\bar{Q}\mathbb{E}x(\tau)+\mathbf{V}_{i}^{T}(\tau)\mathbf{R}_{i}\mathbf{V}_{i}(\tau)+[\mathbb{E}\mathbf{V}_{i}(\tau)]^{T}\mathbf{\bar{R}}_{i}\mathbb{E}\mathbf{V}_{i}(\tau)$
	$\displaystyle+\sum_{j=0}^{i-1}\big{[}\tilde{v}_{j,i}^{T}(\tau)\mathcal{R}_{j}\tilde{v}_{j,i}(\tau)+[\mathbb{E}\tilde{v}_{j,i}(\tau)]^{T}\bar{\mathcal{R}}_{j}\mathbb{E}\tilde{v}_{j,i}(\tau)\big{]}+\sum_{j=i+1}^{h}\big{[}v_{j}^{T}(\tau)\mathcal{R}_{j}v_{j}(\tau)+[\mathbb{E}v_{j}(\tau)]^{T}\bar{\mathcal{R}}_{j}\mathbb{E}v_{j}(\tau)\big{]}\Big{]}$
	$\displaystyle+\mathbb{E}[x^{T}(\Gamma+1)\Phi(\Gamma+1)x(\Gamma+1)]+[\mathbb{E}x(\Gamma+1)]^{T}\bar{\Phi}(\Gamma+1)\mathbb{E}x(\Gamma+1),$	(14)

where for $0\leq j\leq i-1,$ there holds

	$\displaystyle\hat{v}_{j,i}(\tau)=$	$\displaystyle\mathbb{E}[v_{j}(\tau)\mid\mathcal{F}_{i}(\tau)],$
	$\displaystyle\tilde{v}_{j,i}(\tau)=$	$\displaystyle v_{j}(\tau)-\hat{v}_{j,i}(\tau).$		(15)

Moreover, (7) and (9) can be rewritten as follows, respectively

\displaystyle 0=

\displaystyle\mathbf{R}_{i}\mathbf{V}_{i}(\tau)+\mathbf{\bar{R}}_{i}\mathbb{E}\mathbf{V}_{i}(\tau)+\mathbb{E}[\mathbf{B}^{T}(\tau)\theta(\tau)\mid\mathcal{F}_{i}(\tau)]+\mathbb{E}[\mathbf{\bar{B}}^{T}(\tau)\theta(\tau)],

(16)

\displaystyle 0=

\displaystyle\mathcal{R}_{j}\tilde{v}_{j,i}(\tau)+\bar{\mathcal{R}}_{j}\mathbb{E}\tilde{v}_{j,i}(\tau)-\mathbb{E}[\mathcal{B}_{j}^{T}(\tau)\theta(\tau)\mid\mathcal{F}_{i}(\tau)]+\mathbb{E}[\mathcal{B}_{j}^{T}(\tau)\theta(\tau)|\mathcal{F}_{j}(\tau)],0\leq j\leq i-1,

(17)

\displaystyle 0=

\displaystyle\mathcal{R}_{j}v_{j}(\tau)+\bar{\mathcal{R}}_{j}\mathbb{E}v_{j}(\tau)+\mathbb{E}[\mathcal{B}_{j}^{T}(\tau)\theta(\tau)|\mathcal{F}_{j}(\tau)]+\mathbb{E}[\bar{\mathcal{B}}_{j}^{T}(\tau)\theta(\tau)],i\leq j\leq h,

(18)

where $0\leq i\leq h,$ and $i\leq\tau\leq\Gamma.$

Proof 4.2.

By plugging (9) into (1) and (4.1), we get:

	$\displaystyle 0=$	$\displaystyle\mathcal{R}_{j}\mathbb{E}[v_{j}(\tau)\mid\mathcal{F}_{i}(\tau)]+\bar{\mathcal{R}}_{j}\mathbb{E}[\mathbb{E}v_{j}(\tau)\mid\mathcal{F}_{i}(\tau)]$
		$\displaystyle+\mathbb{E}\Big{[}\mathbb{E}[\mathcal{B}_{j}^{T}(\tau)\theta(\tau)\mid\mathcal{F}_{j}(\tau)]\mid\mathcal{F}_{i}(\tau)\Big{]}+\mathbb{E}\Big{[}\mathbb{E}[\bar{\mathcal{B}}_{j}^{T}(\tau)\theta(\tau)]\mid\mathcal{F}_{i}(\tau)\Big{]}.$

Then, for $0\leq j\leq i-1,$

\displaystyle 0=

\displaystyle\mathcal{R}_{j}\hat{v}_{j,i}(\tau)+\bar{\mathcal{R}}_{j}\mathbb{E}\hat{v}_{j,i}(\tau)+\mathbb{E}[\mathcal{B}_{j}^{T}(\tau)\theta(\tau)\mid\mathcal{F}_{i}(\tau)]+\mathbb{E}[\bar{\mathcal{B}}_{j}^{T}(\tau)\theta(\tau)],

for simplicity of calculation, we have

\displaystyle\mathcal{R}_{j}\hat{v}_{j,i}(\tau)

\displaystyle+\bar{\mathcal{R}}_{j}\mathbb{E}\hat{v}_{j,i}(\tau)=-\mathbb{E}[\mathcal{B}_{j}^{T}(\tau)\theta(\tau)\mid\mathcal{F}_{i}(\tau)]-\mathbb{E}[\bar{\mathcal{B}}_{j}^{T}(\tau)\theta(\tau)].

(19)

What’s more, for $i\leq j\leq h,$ obviously (17) is established.

We know that (17) can be obtained by combining (9) and (19). Next, using (1) and (19), (16) can be verified. Before proving (4.1) and (4.1), it is worth noting that the following two relationships are established:

1) For $0\leq j\leq i-1,$ we infer that

	$\displaystyle\mathbb{E}[v_{i}^{T}(\tau)T_{i,j}\tilde{v}_{j,i}(\tau)]=$	$\displaystyle\mathbb{E}\Big{[}\mathbb{E}[v_{i}^{T}(\tau)T_{i,j}\tilde{v}_{j,i}(\tau)\mid\mathcal{F}_{i}(\tau)]\Big{]}=\mathbb{E}\Big{[}v_{i}^{T}(\tau)T_{i,j}\mathbb{E}[v_{j}(\tau)-\hat{v}_{j,i}(\tau)\mid\mathcal{F}_{i}(\tau)]\Big{]}$
	$\displaystyle=$	$\displaystyle\mathbb{E}\bigg{[}v_{i}^{T}(\tau)T_{i,j}\mathbb{E}\Big{[}v_{j}(\tau)-\mathbb{E}[v_{j}(\tau)\mid\mathcal{F}_{i}(\tau)]\mid\mathcal{F}_{i}(\tau)\Big{]}\bigg{]}=0.$

2) For $0\leq j,n\leq i-1,$ there holds that

	$\displaystyle\mathbb{E}$	$\displaystyle[\hat{v}_{n,i}^{T}(\tau)T_{n,j}\tilde{v}_{j,i}(\tau)]=\mathbb{E}\Big{[}\mathbb{E}[\hat{v}_{n,i}^{T}(\tau)T_{n,j}\tilde{v}_{j,i}(\tau)\mid\mathcal{F}_{i}(\tau)]\Big{]}$
	$\displaystyle=$	$\displaystyle\mathbb{E}\Big{[}\hat{v}_{n,i}^{T}(\tau)T_{n,j}\mathbb{E}[\tilde{v}_{j,i}(\tau)\mid\mathcal{F}_{i}(\tau)]\Big{]}=0.$

In summary, we have completed the proof of the orthogonality of $\mathbf{V}_{i}(\tau)$ and $\tilde{v}_{0,i}(\tau),\cdots,\tilde{v}_{i-1,i}(\tau)$ for $0\leq i\leq h,$ and then we can immediately verify that (4.1) and (4.1) hold.

Remark 4.3.

The decoupling of the MF-FBSDEs (12) is difficult in view of the non-classic information structure of Problem 2.2. Therefore, the orthogonal decomposition approach is introduced in Lemma 4.1, which will play a critical role in decoupling (12). In what follows, we will show the method to derive the optimal decentralized control strategy by the use of the orthogonal decomposition approach to decouple MF-FBSDEs (12).

In view of the delayed information pattern for the $h+1$ controllers, it is necessary to derive the associated mathematical conditional expectation, i.e., the optimal predictor.

Lemma 4.4.

The optimal predictor $\hat{x}_{\tau/\tau-i}=\mathbb{E}[x(\tau)\mid\mathcal{F}_{i}(\tau)]=\mathbb{E}[x(\tau)\mid\mathcal{F}_{0}(\tau-i)]$ can be given as follows:

\displaystyle\left\{\begin{aligned} \hat{x}_{\tau/\tau-i}&=\mathcal{A}^{\tau}x(0)+[(\mathcal{A}+\bar{\mathcal{A}})^{\tau}-\mathcal{A}^{\tau}]\mathbb{E}x(0)+\sum_{j=1}^{\tau}\begin{bmatrix}\mathcal{A}^{j-1}&(\mathcal{A}+\bar{\mathcal{A}})^{j-1}-\mathcal{A}^{j-1}\end{bmatrix}\big{\{}\begin{bmatrix}\mathbf{B}_{i-j}&\mathbf{\bar{B}}_{i-j}\\ 0&\mathbf{B}_{i-j}+\mathbf{\bar{B}}_{i-j}\end{bmatrix}\\ &\times\begin{bmatrix}\mathbf{V}_{i-j}(\tau-j)\\ \mathbb{E}\mathbf{V}_{i-j}(\tau-j)\end{bmatrix}+\sum_{m=i-j+1}^{h}\begin{bmatrix}\mathcal{B}_{m}&\bar{\mathcal{B}}_{m}\\ 0&\mathcal{B}_{m}+\bar{\mathcal{B}}_{m}\end{bmatrix}\begin{bmatrix}v_{m}(\tau-j)\\ \mathbb{E}v_{m}(\tau-j)\end{bmatrix}\big{\}},i=1,\cdots,h,\\ \hat{x}_{\tau+1/\tau-i}&=\mathcal{A}\hat{x}_{\tau/\tau-i}+\bar{\mathcal{A}}\mathbb{E}x(\tau)+\mathbf{B}_{i}\mathbf{V}_{i}(\tau)+\mathbf{\bar{B}}_{i}\mathbb{E}\mathbf{V}_{i}(\tau)+\sum_{m=i+1}^{h}[\mathcal{B}_{m}v_{m}(\tau)+\bar{\mathcal{B}}_{m}\mathbb{E}v_{m}(\tau)],i=0,\cdots,h-1,\\ \hat{x}_{\tau+1/\tau-h}&=\mathcal{A}\hat{x}_{\tau/\tau-h}+\bar{\mathcal{A}}\mathbb{E}x(\tau)+\mathbf{B}_{h}\mathbf{V}_{h}(\tau)+\mathbf{\bar{B}}_{h}\mathbb{E}\mathbf{V}_{h}(\tau).\end{aligned}\right.

(20)

Proof 4.5.

For (2), it can be easily induced that by taking the mathematical expectation of system (2):

\displaystyle\mathbb{E}x(\tau+1)=

\displaystyle(\mathcal{A}+\bar{\mathcal{A}})\mathbb{E}x(\tau)+\sum_{i=0}^{h}(\mathcal{B}_{i}+\bar{\mathcal{B}}_{i})\mathbb{E}v_{i}(\tau).

(21)

Subsequently,

$\displaystyle x(\tau)=$	$\displaystyle\begin{bmatrix}\mathcal{A}(\tau-1)&\bar{\mathcal{A}}(\tau-1)\end{bmatrix}\begin{bmatrix}x(\tau-1)\\ \mathbb{E}x(\tau-1)\end{bmatrix}+\sum_{m=0}^{h}\begin{bmatrix}\mathcal{B}_{m}(\tau-1)&\bar{\mathcal{B}}_{m}(\tau-1)\end{bmatrix}\begin{bmatrix}v_{m}(\tau-1)\\ \mathbb{E}v_{m}(\tau-1)\end{bmatrix}$
$\displaystyle=$	$\displaystyle\begin{bmatrix}\mathcal{A}(\tau-1)&\bar{\mathcal{A}}(\tau-1)\end{bmatrix}\begin{bmatrix}\mathcal{A}(\tau-2)&\bar{\mathcal{A}}(\tau-2)\\ 0&\mathcal{A}+\bar{\mathcal{A}}\end{bmatrix}\times\cdots\times\begin{bmatrix}\mathcal{A}(0)&\bar{\mathcal{A}}(0)\\ 0&\mathcal{A}+\bar{\mathcal{A}}\end{bmatrix}\begin{bmatrix}x(0)\\ \mathbb{E}x(0)\end{bmatrix}$
	$\displaystyle+\sum_{j=1}^{\tau}\sum_{m=0}^{h}\begin{bmatrix}\mathcal{A}(\tau-1)&\bar{\mathcal{A}}(\tau-1)\end{bmatrix}\begin{bmatrix}\mathcal{A}(\tau-2)&\bar{\mathcal{A}}(\tau-2)\\ 0&\mathcal{A}+\bar{\mathcal{A}}\end{bmatrix}\times\cdots\times\begin{bmatrix}\mathcal{A}(\tau-j+1)&\bar{\mathcal{A}}(\tau-j+1)\\ 0&\mathcal{A}+\bar{\mathcal{A}}\end{bmatrix}$
	$\displaystyle\times\begin{bmatrix}\mathcal{B}_{m}(\tau-j)&\bar{\mathcal{B}}_{m}(\tau-j)\\ 0&\mathcal{B}_{m}+\bar{\mathcal{B}}_{m}\end{bmatrix}\begin{bmatrix}v_{m}(\tau-j)\\ \mathbb{E}v_{m}(\tau-j)\end{bmatrix}$
$\displaystyle=$	$\displaystyle\begin{bmatrix}\mathcal{A}(\tau-1)&\bar{\mathcal{A}}(\tau-1)\end{bmatrix}\begin{bmatrix}\mathcal{A}(\tau-2)&\bar{\mathcal{A}}(\tau-2)\\ 0&\mathcal{A}+\bar{\mathcal{A}}\end{bmatrix}\times\cdots\times\begin{bmatrix}\mathcal{A}(0)&\bar{\mathcal{A}}(0)\\ 0&\mathcal{A}+\bar{\mathcal{A}}\end{bmatrix}\begin{bmatrix}x(0)\\ \mathbb{E}x(0)\end{bmatrix}$
	$\displaystyle+\sum_{j=1}^{\tau}\sum_{m=0}^{i-j}\begin{bmatrix}\mathcal{A}(\tau-1)&\bar{\mathcal{A}}(\tau-1)\end{bmatrix}\begin{bmatrix}\mathcal{A}(\tau-2)&\bar{\mathcal{A}}(\tau-2)\\ 0&\mathcal{A}+\bar{\mathcal{A}}\end{bmatrix}\times\cdots\times\begin{bmatrix}\mathcal{A}(\tau-j+1)&\bar{\mathcal{A}}(\tau-j+1)\\ 0&\mathcal{A}+\bar{\mathcal{A}}\end{bmatrix}$
	$\displaystyle\times\begin{bmatrix}\mathcal{B}_{m}(\tau-j)&\bar{\mathcal{B}}_{m}(\tau-j)\\ 0&\mathcal{B}_{m}+\bar{\mathcal{B}}_{m}\end{bmatrix}\begin{bmatrix}v_{m}(\tau-j)\\ \mathbb{E}v_{m}(\tau-j)\end{bmatrix}+\sum_{j=1}^{\tau}\sum_{m=i-j+1}^{h}\begin{bmatrix}\mathcal{A}(\tau-1)&\bar{\mathcal{A}}(\tau-1)\end{bmatrix}$
	$\displaystyle\times\begin{bmatrix}\mathcal{A}(\tau-2)&\bar{\mathcal{A}}(\tau-2)\\ 0&\mathcal{A}+\bar{\mathcal{A}}\end{bmatrix}\times\cdots\times\begin{bmatrix}\mathcal{A}(\tau-j+1)&\bar{\mathcal{A}}(\tau-j+1)\\ 0&\mathcal{A}+\bar{\mathcal{A}}\end{bmatrix}\begin{bmatrix}\mathcal{B}_{m}(\tau-j)&\bar{\mathcal{B}}_{m}(\tau-j)\\ 0&\mathcal{B}_{m}+\bar{\mathcal{B}}_{m}\end{bmatrix}\begin{bmatrix}v_{m}(\tau-j)\\ \mathbb{E}v_{m}(\tau-j)\end{bmatrix}.$	(22)

By taking the conditional mathematical expectation of $x(\tau)$ , the optimal predictor $\mathbb{E}[x(\tau)\mid\mathcal{F}_{i}(\tau)]$ can be derived as

	$\displaystyle\hat{x}_{\tau/\tau-i}=$	$\displaystyle\mathbb{E}[x(\tau)\mid\mathcal{F}_{i}(\tau)]$
	$\displaystyle=$	$\displaystyle\mathcal{A}^{\tau}x(0)+[(\mathcal{A}+\bar{\mathcal{A}})^{\tau}-\mathcal{A}^{\tau}]\mathbb{E}x(0)+\sum_{j=1}^{\tau}\begin{bmatrix}\mathcal{A}^{j-1}&(\mathcal{A}+\bar{\mathcal{A}})^{j-1}-\mathcal{A}^{j-1}\end{bmatrix}\big{\{}\begin{bmatrix}\mathbf{B}_{i-j}&\mathbf{\bar{B}}_{i-j}\\ 0&\mathbf{B}_{i-j}+\mathbf{\bar{B}}_{i-j}\end{bmatrix}$
		$\displaystyle\times\begin{bmatrix}\mathbf{V}_{i-j}(\tau-j)\\ \mathbb{E}\mathbf{V}_{i-j}(\tau-j)\end{bmatrix}+\sum_{m=i-j+1}^{h}\begin{bmatrix}\mathcal{B}_{m}&\bar{\mathcal{B}}_{m}\\ 0&\mathcal{B}_{m}+\bar{\mathcal{B}}_{m}\end{bmatrix}\begin{bmatrix}v_{m}(\tau-j)\\ \mathbb{E}v_{m}(\tau-j)\end{bmatrix}\big{\}},i=1,\cdots,h.$

Note that the following relationship can be holds,

\displaystyle\mathbb{E}[x(\tau)\mid\mathcal{F}_{i}(\tau)]=

\displaystyle\mathbb{E}[x(\tau)\mid\mathcal{F}_{0}(\tau-i)]=\mathbb{E}[x(\tau)\mid\mathcal{F}_{i-j}(\tau-j)].

Moreover, it is noted that $v_{m}(\tau-j)$ is $\mathcal{F}_{i-j}(\tau-j)$ -measurable for $i-j+1\leq m\leq h$ . Otherwise, $v_{m}(\tau-j)$ is not $\mathcal{F}_{i-j}(\tau-j)$ -measurable for $m=0,\cdots,i-j$ , subsequently, by using (4.1), we know that $\mathbb{E}[v_{m}(\tau-j)\mid\mathcal{F}_{i-j}(\tau-j)]$ can be represented by $\hat{v}_{m,i-j}(\tau-j)$ for $m=0,\cdots,i-j.$

Combining the properties of Gaussian white noise and $v_{i}(\tau)$ is $\mathcal{F}_{i}(\tau)$ -measurable, and by using (21) and (4.5) we can immediately verify that the relationship in (20) holds.

4.2 Decoupling MF-FBSDEs

In this section, we will adopt the induction approach to decouple the MF-FBSDEs (12) and derive a solution to Problem 2.2.

For simplicity, let’s define the following asymmetric Riccati equations,

\displaystyle\left\{\begin{aligned} \Phi(\tau)=&Q+\mathcal{A}^{T}\Phi(\tau+1)\mathcal{A}+\sigma^{2}C^{T}\Phi(\tau+1)C\\ &-[\bar{L}_{0}^{T}(\tau)+\mathcal{A}^{T}\varphi_{1}(\tau+1)\mathcal{B}_{0}]\bar{\mathbf{\Upsilon}}_{0}^{-1}(\tau)\bar{Y}_{0,0}(\tau)+\mathcal{A}^{T}\varphi_{1}(\tau+1)\mathcal{A},\\ \bar{\Phi}(\tau)=&\bar{Q}+(\mathcal{A}+\bar{\mathcal{A}})^{T}[\Phi(\tau+1)+\bar{\Phi}(\tau+1)+\sum_{j=1}^{h}\varphi_{j}(\tau+1)](\mathcal{A}+\bar{\mathcal{A}})\\ &-\mathcal{A}^{T}[\Phi(\tau+1)+\sum_{j=1}^{h}\varphi_{j}(\tau+1)]\mathcal{A}+\sigma^{2}(C+\bar{C})^{T}\Phi(\tau+1)(C+\bar{C})\\ &-\sigma^{2}C^{T}\Phi(\tau+1)C-\sum_{i=0}^{h}\Big{[}[L_{i}^{T}(\tau)\mathbf{I}_{i}+(\mathcal{A}+\bar{\mathcal{A}})^{T}\varphi_{i+1}(\tau+1)\\ &\times(\mathbf{B}_{i}+\mathbf{\bar{B}}_{i})]\sum_{j=i}^{h}\mathbf{\Upsilon}_{i}^{-1}(\tau)Y_{i,j}(\tau)-[\bar{L}_{i}^{T}(\tau)\mathbf{I}_{i}+\mathcal{A}^{T}\varphi_{i+1}(\tau+1)\mathbf{B}_{i}]\sum_{j=i}^{h}\mathbf{\bar{\Upsilon}}_{i}^{-1}(\tau)\bar{Y}_{i,j}(\tau)\Big{]},\\ \varphi_{j}(\tau)=&-\sum_{i=0}^{j}[\bar{L}_{i}^{T}(\tau)\mathbf{I}_{i}+\mathcal{A}^{T}\varphi_{i+1}(\tau+1)\mathbf{B}_{i}]\mathbf{\bar{\Upsilon}}_{i}^{-1}(\tau)\sum_{j=i}^{h}\bar{Y}_{i,j}(\tau)+\mathcal{A}^{T}\varphi_{j+1}(\tau+1)\mathcal{A},1\leq j\leq h,\end{aligned}\right.

(23)

with terminal conditions $\Phi(\Gamma+1),\bar{\Phi}(\Gamma+1)$ and $\varphi_{j}(\Gamma+1)=0,\varphi_{h+1}(\tau)=0$ for $1\leq j\leq h$ and $0\leq\tau\leq\Gamma.$ In the above, for $1\leq i\leq h,i+1\leq j\leq h,$

$\displaystyle\mathbf{\bar{\Upsilon}}_{i}(\tau)=$	$\displaystyle\mathbf{R}_{i}+\mathbf{B}_{i}^{T}[\Phi(\tau+1)+\sum_{j=1}^{h}\varphi_{j}(\tau+1)]\mathbf{B}_{i}+\sigma^{2}\mathbf{D}_{i}\Phi(\tau+1)\mathbf{D}_{i},$
$\displaystyle\mathbf{\Upsilon}_{i}(\tau)=$	$\displaystyle\mathbf{R}_{i}+\mathbf{\bar{R}}_{i}+(\mathbf{B}_{i}+\mathbf{\bar{B}}_{i})^{T}[\Phi(\tau+1)+\bar{\Phi}(\tau+1)+\sum_{j=1}^{h}\varphi_{j}(\tau+1)]$
	$\displaystyle\times(\mathbf{B}_{i}+\mathbf{\bar{B}}_{i})+\sigma^{2}(\mathbf{D}_{i}+\mathbf{\bar{D}}_{i})\Phi(\tau+1)(\mathbf{D}_{i}+\mathbf{\bar{D}}_{i}),$
$\displaystyle\bar{Y}_{i,i}(\tau)=$	$\displaystyle\mathbf{B}_{i}^{T}[\Phi(\tau+1)+\sum_{j=1}^{h}\varphi_{j}(\tau+1)]\mathcal{A}+\sigma^{2}\mathbf{D}_{i}^{T}\Phi(\tau+1)C,$
$\displaystyle Y_{i,i}(\tau)=$	$\displaystyle(\mathbf{B}_{i}+\mathbf{\bar{B}}_{i})^{T}[\Phi(\tau+1)+\bar{\Phi}(\tau+1)+\sum_{j=1}^{h}\varphi_{j}(\tau+1)](\mathcal{A}+\bar{\mathcal{A}})$
	$\displaystyle+\sigma^{2}(\mathbf{D}_{i}+\mathbf{\bar{D}}_{i})^{T}\Phi(\tau+1)(C+\bar{C}),$
$\displaystyle\bar{Y}_{i,j}(\tau)=$	$\displaystyle-\big{\{}\mathbf{B}_{i}^{T}[\Phi(\tau+1)+\sum_{m=1}^{h}\varphi_{m}(\tau+1)]\mathcal{B}_{j}+\sigma^{2}\mathbf{D}_{i}^{T}\Phi(\tau+1)D_{j}\big{\}}$
	$\displaystyle\times\mathbf{I}_{j}\mathbf{\bar{\Upsilon}}_{j}^{-1}(\tau)\sum_{m=j}^{h}\bar{Y}_{j,m}(\tau),$
$\displaystyle Y_{i,j}(\tau)=$	$\displaystyle-\big{\{}(\mathbf{B}_{i}+\mathbf{\bar{B}}_{i})^{T}[\Phi(\tau+1)+\bar{\Phi}(\tau+1)+\sum_{m=1}^{h}\varphi_{m}(\tau+1)](\mathcal{B}_{j}+\bar{\mathcal{B}}_{j})$
	$\displaystyle+\sigma^{2}(\mathbf{D}_{i}+\mathbf{\bar{D}}_{i})^{T}\Phi(\tau+1)(D_{j}+\bar{D}_{j})\big{\}}\mathbf{I}_{j}\mathbf{\Upsilon}_{j}^{-1}(\tau)\sum_{m=j}^{h}Y_{j,m}(\tau),i+1\leq j\leq h,$	(24)

where

\displaystyle\mathbf{I}_{i}=

\displaystyle[0,\cdots,0,I_{m_{i}}],\mathbf{I}_{0}=I_{m_{0}},

	$\displaystyle L_{i}(\tau)=$	$\displaystyle(\mathcal{B}_{i}+\bar{\mathcal{B}}_{i})^{T}[\Phi(\tau+1)+\bar{\Phi}(\tau+1)+\sum_{j=1}^{h}\varphi_{j}(\tau+1)](\mathcal{A}+\bar{\mathcal{A}})$
		$\displaystyle+\sigma^{2}(D_{i}+\bar{D}_{i})^{T}\Phi(\tau+1)(C+\bar{C}),1\leq i\leq h,$

\displaystyle\bar{L}_{i}(\tau)=

\displaystyle\mathcal{B}_{i}^{T}[\Phi(\tau+1)+\sum_{j=1}^{h}\varphi_{j}(\tau+1)]\mathcal{A}+\sigma^{2}D_{i}^{T}\Phi(\tau+1)C,

\displaystyle L_{0}(\tau)=

\displaystyle(\mathcal{B}_{0}+\bar{\mathcal{B}}_{0})^{T}[\Phi(\tau+1)+\bar{\Phi}(\tau+1)](\mathcal{A}+\bar{\mathcal{A}})+\sigma^{2}(D_{0}+\bar{D}_{0})^{T}\Phi(\tau+1)(C+\bar{C}),

\displaystyle\bar{L}_{0}(\tau)=

\displaystyle\mathcal{B}_{0}^{T}\Phi(\tau+1)\mathcal{A}+\sigma^{2}D_{0}^{T}\Phi(\tau+1)C.

Now it is the position to state the solution to Problem 2.2.

Theorem 4.6.

Suppose Assumptions 2-2 hold, for (2) and (2), Problem 2.2 is uniquely solvable if and only if $\mathbf{\Upsilon}_{i}(\tau)$ and $\mathbf{\bar{\Upsilon}}_{i}(\tau)$ are invertible, for $0\leq i\leq h,i\leq\tau\leq\Gamma,$ where $\mathbf{\Upsilon}_{i}(\tau)$ and $\mathbf{\bar{\Upsilon}}_{i}(\tau)$ are given in (4.2).

In this case, the unique optimal decentralized control strategy $v_{i}(\tau)$ can be calculated as

	$\displaystyle v_{i}(\tau)=$	$\displaystyle\mathbf{I}_{i}\mathbf{V}_{i}(\tau)$
	$\displaystyle=$	$\displaystyle-\mathbf{I}_{i}\mathbf{\bar{\Upsilon}}_{i}^{-1}(\tau)\sum_{j=i}^{h}\bar{Y}_{i,j}(\tau)\hat{x}_{\tau/\tau-j}-\sum_{j=i}^{h}\mathbf{I}_{i}[\mathbf{\Upsilon}_{i}^{-1}(\tau)Y_{i,j}(\tau)-\mathbf{\bar{\Upsilon}}_{i}^{-1}(\tau)\bar{Y}_{i,j(\tau)}]\mathbb{E}x(\tau),i\leq\tau\leq\Gamma,$		(25)

where $\hat{x}_{\tau/\tau-j}$ can be calculated from Lemma 4.4, and $Y_{i,j}(\tau),\bar{Y}_{i,j}(\tau)$ are given in (4.2). Furthermore, the corresponding optimal cost functional is given by

\displaystyle J_{\Gamma}^{*}(v)=

\displaystyle x^{T}(0)[\Phi(0)+\bar{\Phi}(0)+\sum_{j=1}^{h}\varphi_{j}(0)]x(0)+\sum_{i=0}^{h}\sum_{\tau=0}^{i-1}v_{i}^{T}(\tau)[\mathcal{R}_{i}+\bar{\mathcal{R}}_{i}]v_{i}(\tau).

(26)

The relationship between the state $x(\tau)$ and the costate $\theta(\tau)$ (i.e., the solution to MF-FBSDEs (4.1) and (16)-(18)) can be given as

\displaystyle\theta(\tau)=

\displaystyle\Phi(\tau+1)x(\tau+1)+\bar{\Phi}(\tau+1)\mathbb{E}x(\tau+1)+\sum_{j=1}^{h}\varphi_{j}(\tau+1)\hat{x}_{\tau+1/\tau+1-j},-1\leq\tau\leq\Gamma.

(27)

Proof 4.7.

Please refer to the Appendix for detailed proof.

Remark 4.8.

In Theorem 4.6, it is shown that the gain matrices of the optimal decentralized control strategy can be calculated via some asymmetric Riccati equations, which can be calculated offline. Moreover, we prove that the optimal control strategy is a linear feedback of the optimal predictor $\hat{x}_{\tau/\tau-j}$ and the state mean $\mathbb{E}x(\tau)$ , which is feasible in calculation. The obtained results are derived for the first time.

Remark 4.9.

From the main results of Theorem 4.6, it is not hard to verify that the obtained results include the results of ⁸, ¹⁸ as special cases. On one hand, with $h=0$ , Problem 2.2 can be reduced as the control problem for mean-field system without delay in ⁸, and the optimal control with $h=0$ in Theorem 4.6 is exactly the optimal control shown in ⁸. On the other hand, with $\bar{\mathcal{A}}=0,\bar{\mathcal{B}}_{i}=0,\bar{C}=0,\bar{D}_{i}=0$ , Problem 2.2 turns into the case investigated in ¹⁸, and the optimal decentralized control strategy in Theorem 4.6 can be presented the same as that in ¹⁸.

5 Numerical Example

To illustrate the obtained results in Theorem 4.6, the following numerical example shall be given as below.

Without loss of generality, we consider (2) and (2) with $n=2,h=2,m_{0}=m_{1}=m_{2}=2,\Gamma=5,$

	$\displaystyle\mathcal{A}=$	$\displaystyle\begin{bmatrix}0.6&0.3\\ 0.4&0.2\end{bmatrix},\bar{\mathcal{A}}=\begin{bmatrix}0.2&0\\ 0&-0.6\end{bmatrix},C=\begin{bmatrix}0.6&0.3\\ 0.4&0.2\end{bmatrix},\bar{C}=\begin{bmatrix}0.2&0\\ 0&-0.6\end{bmatrix},$
	$\displaystyle\mathcal{B}_{0}=$	$\displaystyle\begin{bmatrix}0.3&0.2\\ 0.4&-0.1\end{bmatrix},\mathcal{B}_{1}=\begin{bmatrix}0.1&0.2\\ 0&0.1\end{bmatrix},\mathcal{B}_{2}=\begin{bmatrix}0.3&0.1\\ 0.5&0.8\end{bmatrix},$
	$\displaystyle D_{0}=$	$\displaystyle\begin{bmatrix}0.3&0.2\\ 0.4&-0.1\end{bmatrix},D_{1}=\begin{bmatrix}0.1&0.2\\ 0&0.1\end{bmatrix},D_{2}=\begin{bmatrix}0.3&0.1\\ 0.5&0.8\end{bmatrix},$
	$\displaystyle\bar{\mathcal{B}}_{0}=$	$\displaystyle\begin{bmatrix}-0.4&0.8\\ 0.2&0.9\end{bmatrix},\bar{\mathcal{B}}_{1}=\begin{bmatrix}0&0.1\\ 0&-1\end{bmatrix},\bar{\mathcal{B}}_{2}=\begin{bmatrix}-0.8&0\\ 0&0.2\end{bmatrix},$
	$\displaystyle\bar{D}_{0}=$	$\displaystyle\begin{bmatrix}-0.4&0.8\\ 0.2&0.9\end{bmatrix},\bar{D}_{1}=\begin{bmatrix}0&0.1\\ 0&-1\end{bmatrix},\bar{D}_{2}=\begin{bmatrix}-0.8&0\\ 0&0.2\end{bmatrix},$
	$\displaystyle Q=$	$\displaystyle I_{2},\mathcal{R}_{0}=I_{2},\mathcal{R}_{1}=I_{2},\mathcal{R}_{2}=I_{2},\bar{Q}=I_{2},\bar{\mathcal{R}}_{0}=I_{2},\bar{\mathcal{R}}_{1}=I_{2},\bar{\mathcal{R}}_{2}=I_{2},$
	$\displaystyle\Phi(\Gamma$	$\displaystyle+1)=I_{2},\bar{\Phi}(\Gamma+1)=I_{2},\varphi_{1}(\Gamma+1)=0,\varphi_{2}(\Gamma+1)=0.$

By using Theorem 4.6, $\mathbf{\Upsilon}_{i}(\tau),\mathbf{\bar{\Upsilon}}_{i}(\tau),Y_{i,j}(\tau),\bar{Y}_{i,j}(\tau),i=0,1,2,$ $0\leq\tau\leq 4$ can be calculated, and it can be found $\mathbf{\Upsilon}_{i}(\tau)$ and $\mathbf{\bar{\Upsilon}}_{i}(\tau)$ are invertible. On the other hand, since $Q,\bar{Q},\mathcal{R}_{i},\bar{\mathcal{R}}_{i},\Phi(\Gamma+1),\bar{\Phi}(\Gamma+1)$ given above are positive definite, we can verify $\mathbf{\Upsilon}_{i}(\tau),\mathbf{\bar{\Upsilon}}_{i}(\tau)\geq 0$ in (4.2).

Thus, from Theorem 4.6, we know Problem 2.2 can be uniquely solved, and the optimal decentralized control is given as:

	$\displaystyle v_{0}(0)=$	$\displaystyle\begin{bmatrix}-0.2932&-0.1466\\ -0.1631&-0.0815\end{bmatrix}x(0)+\begin{bmatrix}0.2889&0.2390\\ -0.2938&-0.0064\end{bmatrix}\mathbb{E}x(0),$
	$\displaystyle v_{0}(1)=$	$\displaystyle\begin{bmatrix}0.2487&0.1243\\ -0.0205&-0.0102\end{bmatrix}\hat{x}_{1/0}+\begin{bmatrix}-0.5338&-0.2669\\ -0.1607&-0.0804\end{bmatrix}x(1)$
		$\displaystyle+\begin{bmatrix}0.2800&0.2347\\ -0.2946&0.0033\end{bmatrix}\mathbb{E}x(1),$
	$\displaystyle v_{0}(2)=$	$\displaystyle\begin{bmatrix}0.1567&0.0783\\ -0.0369&-0.0185\end{bmatrix}\hat{x}_{2/0}+\begin{bmatrix}0.0230&0.0115\\ 0.0097&0.0049\end{bmatrix}\hat{x}_{2/1}$
		$\displaystyle+\begin{bmatrix}-0.4693&-0.2346\\ -0.1364&-0.0682\end{bmatrix}x(2)+\begin{bmatrix}0.2857&0.2374\\ -0.2911&-0.0035\end{bmatrix}\mathbb{E}x(2),$
	$\displaystyle v_{0}(3)=$	$\displaystyle\begin{bmatrix}0.2185&0.1093\\ -0.0297&-0.0149\end{bmatrix}\hat{x}_{3/1}+\begin{bmatrix}0.0264&0.0132\\ 0.0111&0.0055\end{bmatrix}\hat{x}_{3/2}$
		$\displaystyle+\begin{bmatrix}-0.5207&-0.2603\\ -0.1558&-0.0779\end{bmatrix}x(3)+\begin{bmatrix}0.2709&0.2303\\ -0.2910&0.0017\end{bmatrix}\mathbb{E}x(3),$
	$\displaystyle v_{0}(4)=$	$\displaystyle\begin{bmatrix}0.1907&0.0953\\ -0.0235&-0.0117\end{bmatrix}\hat{x}_{4/2}+\begin{bmatrix}0.0175&0.0087\\ 0.0075&0.0037\end{bmatrix}\hat{x}_{4/3}$
		$\displaystyle+\begin{bmatrix}-0.4499&-0.2249\\ -0.1291&-0.0645\end{bmatrix}x(4)+\begin{bmatrix}0.2351&0.2105\\ -0.2957&0.0034\end{bmatrix}\mathbb{E}x(4),$
	$\displaystyle v_{1}(1)=$	$\displaystyle\begin{bmatrix}-0.0901&-0.0450\\ -0.1823&-0.0911\end{bmatrix}\hat{x}_{1/0}+\begin{bmatrix}0.0487&0.0265\\ 0.1279&-0.0750\end{bmatrix}\mathbb{E}x(1),$
	$\displaystyle v_{1}(2)=$	$\displaystyle\begin{bmatrix}-0.0836&-0.0418\\ -0.2131&-0.1065\end{bmatrix}\hat{x}_{2/1}+\begin{bmatrix}0.0004&-0.0002\\ 0.0373&0.0187\end{bmatrix}\hat{x}_{2/0}$
		$\displaystyle+\begin{bmatrix}0.0444&0.0236\\ 0.1222&-0.0786\end{bmatrix}\mathbb{E}x(2),$
	$\displaystyle v_{1}(3)=$	$\displaystyle\begin{bmatrix}-0.0936&-0.0468\\ -0.2365&-0.1182\end{bmatrix}\hat{x}_{3/2}+\begin{bmatrix}0.0067&0.0034\\ 0.0606&0.0303\end{bmatrix}\hat{x}_{3/1}$
		$\displaystyle+\begin{bmatrix}0.0463&0.0250\\ 0.1224&-0.0783\end{bmatrix}\mathbb{E}x(3),$
	$\displaystyle v_{1}(4)=$	$\displaystyle\begin{bmatrix}-0.0804&-0.0402\\ -0.2054&-0.1027\end{bmatrix}\hat{x}_{4/3}+\begin{bmatrix}-0.0070&0.0035\\ 0.0546&0.0273\end{bmatrix}\hat{x}_{4/2}$
		$\displaystyle+\begin{bmatrix}0.0352&0.0200\\ 0.1036&-0.0840\end{bmatrix}\mathbb{E}x(4),$
	$\displaystyle v_{2}(2)=$	$\displaystyle\begin{bmatrix}-0.2931&-0.1465\\ -0.1580&-0.0790\end{bmatrix}\hat{x}_{2/0}+\begin{bmatrix}0.4550&0.3004\\ 0.0456&0.1841\end{bmatrix}\mathbb{E}x(2),$
	$\displaystyle v_{2}(3)=$	$\displaystyle\begin{bmatrix}-0.3054&-0.1527\\ -0.1628&-0.0814\end{bmatrix}\hat{x}_{3/1}+\begin{bmatrix}0.4702&0.3065\\ 0.0468&0.1863\end{bmatrix}\mathbb{E}x(3),$
	$\displaystyle v_{2}(4)=$	$\displaystyle\begin{bmatrix}-0.2646&-0.1323\\ -0.1468&-0.0734\end{bmatrix}\hat{x}_{4/2}+\begin{bmatrix}0.4178&0.2763\\ 0.0341&0.1784\end{bmatrix}\mathbb{E}x(4).$

In the following, we choose

	$\displaystyle x(0)=$	$\displaystyle\begin{bmatrix}x^{1}(0)\\ x^{2}(0)\end{bmatrix}=\begin{bmatrix}2\\ 1\end{bmatrix},\mathbb{E}x(0)=\begin{bmatrix}\mathbb{E}x^{1}(0)\\ \mathbb{E}x^{2}(0)\end{bmatrix}=\begin{bmatrix}2\\ 1\end{bmatrix},$
	$\displaystyle\omega(\tau)$	$\displaystyle\sim\mathcal{N}(0,1),\Gamma=100.$

By using the results of Theorem 4.6, the state trajectory $x(\tau)=\begin{bmatrix}x^{1}(\tau)\\ x^{2}(\tau)\end{bmatrix}$ for $0\leq\tau\leq 100$ can be obtained, which is depicted as in FIGURE 1. From FIGURE 1, it is clear that the system state converges to 0.

Refer to caption — Figure 1: State trajectory $x(\tau)$ with the optimal decentralized control strategy.

6 Conclusion

In this paper, we have investigated the decentralized control problem for the discrete-time mean field systems with multiple controllers of delayed information. Firstly, by the use of Pontryagin’s maximum principle, the necessary and sufficient solvable conditions for the decentralized control problem have been derived. Next, in order to handle with the asymmetric information pattern, the orthogonal decomposition approach has been proposed to decouple the associated MF-FBSDEs. Finally, the optimal decentralized control strategy has been derived, which is based on an asymmetric Riccati equation. For future research, we will extend the results of this paper to study the infinite time horizon case.

Acknowledgments

This work was supported by National Natural Science Foundation of China under grant 61903210, Natural Science Foundation of Shandong Province under grant ZR2019BF002, China Postdoctoral Science Foundation under grants 2019M652324, 2021T140354, Qingdao Postdoctoral Application Research Project, Major Basic Research of Natural Science Foundation of Shandong Province under grant ZR2021ZD14.

Appendix: Proof of Theorem 1

Proof 6.1.

According to Lemma 3.1-Lemma 3.3, we know that the necessary and sufficient conditions for the unique solution of Problem 2.2 is that equilibrium condition (9) is uniquely solvable. Consequently, based on the induction method, we will solve the MF-FBSDEs (12) to find the solution of Problem 2.2.

In the first place, for $\tau=\Gamma,$ we have $\theta(\Gamma)=\Phi(\Gamma+1)x(\Gamma+1)+\bar{\Phi}(\Gamma+1)\mathbb{E}x(\Gamma+1),$ then (16) with $i=h$ implies

$\displaystyle 0=$	$\displaystyle\mathbf{R}_{h}\mathbf{V}_{h}(\Gamma)+\mathbf{\bar{R}}_{h}\mathbb{E}\mathbf{V}_{h}(\Gamma)+\mathbb{E}[\mathbf{B}_{h}^{T}(\Gamma)\theta(\Gamma)\mid\mathcal{F}_{h}(\Gamma)]+\mathbb{E}[\mathbf{\bar{B}}_{h}^{T}(\Gamma)\theta(\Gamma)]$
$\displaystyle=$	$\displaystyle[\mathbf{B}_{h}^{T}\Phi(\Gamma+1)\mathcal{A}+\sigma^{2}\mathbf{D}_{h}^{T}\Phi(\Gamma+1)C]\hat{x}_{\Gamma/\Gamma-h}+[\mathbf{R}_{h}+\mathbf{B}_{h}^{T}\Phi(\Gamma+1)\mathbf{B}_{h}+\sigma^{2}\mathbf{D}_{h}^{T}\Phi(\Gamma+1)\mathbf{D}_{h}]\mathbf{V}_{h}(\Gamma)$
	$\displaystyle+\Big{[}(\mathbf{B}_{h}+\mathbf{\bar{B}}_{h})^{T}[\Phi(\Gamma+1)+\bar{\Phi}(\Gamma+1)](\mathcal{A}+\bar{\mathcal{A}})-\mathbf{B}_{h}^{T}\Phi(\Gamma+1)\mathcal{A}$
	$\displaystyle+\sigma^{2}(\mathbf{D}_{h}+\mathbf{\bar{D}}_{h})^{T}\Phi(\Gamma+1)(C+\bar{C})-\sigma^{2}\mathbf{D}_{h}^{T}\Phi(\Gamma+1)C\Big{]}\mathbb{E}x(\Gamma)$
	$\displaystyle+\Big{[}\mathbf{\bar{R}}_{h}+(\mathbf{B}_{h}+\mathbf{\bar{B}}_{h})^{T}[\Phi(\Gamma+1)+\bar{\Phi}(\Gamma+1)](\mathbf{B}_{h}+\mathbf{\bar{B}}_{h})-\mathbf{B}_{h}^{T}\Phi(\Gamma+1)\mathbf{B}_{h}$
	$\displaystyle+\sigma^{2}(\mathbf{D}_{h}+\mathbf{\bar{D}}_{h})^{T}\Phi(\Gamma+1)(\mathbf{D}_{h}+\mathbf{\bar{D}}_{h})-\sigma^{2}\mathbf{D}_{h}^{T}\Phi(\Gamma+1)\mathbf{D}_{h}\Big{]}\mathbb{E}\mathbf{V}_{h}(\Gamma)$
$\displaystyle=$	$\displaystyle\mathbf{\bar{\Upsilon}}_{h}(\Gamma)\mathbf{V}_{h}(\Gamma)+[\mathbf{\Upsilon}_{h}(\Gamma)-\mathbf{\bar{\Upsilon}}_{h}(\Gamma)]\mathbb{E}\mathbf{V}_{h}(\Gamma)+\bar{Y}_{h,h}(\Gamma)\hat{x}_{\Gamma/\Gamma-h}+[Y_{h,h}(\Gamma)-\bar{Y}_{h,h}(\Gamma)]\mathbb{E}x(\Gamma).$	(28)

By taking the mathematical expectation of the (6.1), we have

$\displaystyle 0=$	$\displaystyle\Big{[}\mathbf{R}_{h}+\mathbf{\bar{R}}_{h}+(\mathbf{B}_{h}+\mathbf{\bar{B}}_{h})^{T}[\Phi(\Gamma+1)+\bar{\Phi}(\Gamma+1)](\mathbf{B}_{h}+\mathbf{\bar{B}}_{h})+\sigma^{2}(\mathbf{D}_{h}+\mathbf{\bar{D}}_{h})^{T}\Phi(\Gamma+1)(\mathbf{D}_{h}+\mathbf{\bar{D}}_{h})\Big{]}\mathbb{E}\mathbf{V}_{h}(\Gamma)$
	$\displaystyle+\Big{[}(\mathbf{B}_{h}+\mathbf{\bar{B}}_{h})^{T}[\Phi(\Gamma+1)+\bar{\Phi}(\Gamma+1)](\mathcal{A}+\bar{\mathcal{A}})+\sigma^{2}(\mathbf{D}_{h}+\mathbf{\bar{D}}_{h})^{T}\Phi(\Gamma+1)(C+\bar{C})\big{]}\mathbb{E}x(\Gamma)$
$\displaystyle=$	$\displaystyle\mathbf{\Upsilon}_{h}(\Gamma)\mathbb{E}\mathbf{V}_{h}(\Gamma)+Y_{h,h}(\Gamma)\mathbb{E}x(\Gamma).$	(29)

Further, $\mathbb{E}\mathbf{V}_{h}(\Gamma)$ can be calculated as

\displaystyle\mathbb{E}\mathbf{V}_{h}(\Gamma)=-\mathbf{\Upsilon}_{h}^{-1}(\Gamma)Y_{h,h}(\Gamma)\mathbb{E}x(\Gamma).

(30)

Substitute (30) into (6.1), we have

$\displaystyle 0=$	$\displaystyle\mathbf{\bar{\Upsilon}}_{h}(\Gamma)\mathbf{V}_{h}(\Gamma)+[\mathbf{\Upsilon}_{h}(\Gamma)-\mathbf{\bar{\Upsilon}}_{h}(\Gamma)]\mathbb{E}\mathbf{V}_{h}(\Gamma)$
	$\displaystyle+\bar{Y}_{h,h}(\Gamma)\hat{x}_{\Gamma/\Gamma-h}+[Y_{h,h}(\Gamma)-\bar{Y}_{h,h}(\Gamma)]\mathbb{E}x(\Gamma)$
$\displaystyle=$	$\displaystyle\mathbf{\bar{\Upsilon}}_{h}(\Gamma)\mathbf{V}_{h}(\Gamma)-[\mathbf{\Upsilon}_{h}(\Gamma)-\mathbf{\bar{\Upsilon}}_{h}(\Gamma)]\mathbf{\Upsilon}_{h}^{-1}(\Gamma)Y_{h,h}(\Gamma)\mathbb{E}x(\Gamma)$
	$\displaystyle+\bar{Y}_{h,h}(\Gamma)\hat{x}_{\Gamma/\Gamma-h}+[Y_{h,h}(\Gamma)-\bar{Y}_{h,h}(\Gamma)]\mathbb{E}x(\Gamma)$
$\displaystyle=$	$\displaystyle\mathbf{\bar{\Upsilon}}_{h}(\Gamma)\mathbf{V}_{h}(\Gamma)+\bar{Y}_{h,h}(\Gamma)\hat{x}_{\Gamma/\Gamma-h}+[\mathbf{\bar{\Upsilon}}_{h}(\Gamma)\mathbf{\Upsilon}_{h}^{-1}(\Gamma)Y_{h,h}(\Gamma)-\bar{Y}_{h,h}(\Gamma)]\mathbb{E}x(\Gamma).$	(31)

According to Lemma 3.1, it is noted that the equilibrium condition (9) for $\tau=\Gamma$ is uniquely solvable if and only if $\mathbf{\Upsilon}_{h}(\Gamma)$ and $\mathbf{\bar{\Upsilon}}_{h}(\Gamma)$ are invertible. The unique optimal control $v_{h}(\Gamma)$ can be obtained as

	$\displaystyle v_{h}(\Gamma)=$	$\displaystyle\mathbf{I}_{h}\mathbf{V}_{h}(\Gamma)=\mathbf{I}_{h}\begin{bmatrix}\hat{v}_{0,h}(\Gamma)\\ \vdots\\ \hat{v}_{h-1,h}(\Gamma)\\ v_{h}(\Gamma)\end{bmatrix}$
	$\displaystyle=$	$\displaystyle-\mathbf{I}_{h}\mathbf{\bar{\Upsilon}}_{h}^{-1}(\Gamma)\bar{Y}_{h,h}(\Gamma)\hat{x}_{\Gamma/\Gamma-h}-\mathbf{I}_{h}[\mathbf{\Upsilon}_{h}^{-1}(\Gamma)Y_{h,h}(\Gamma)-\mathbf{\bar{\Upsilon}}_{h}^{-1}(\Gamma)\bar{Y}_{h,h}(\Gamma)]\mathbb{E}x(\Gamma).$		(32)

Next, we will calculate $V_{h-1}(\Gamma).$ In fact, $x(\Gamma+1)$ can be rewritten as

$\displaystyle x(\Gamma+1)=$	$\displaystyle[\mathcal{A}+\omega(\Gamma)C]x(\Gamma)+[\bar{\mathcal{A}}+\omega(\Gamma)\bar{C}]\mathbb{E}x(\Gamma)$
	$\displaystyle+[\mathbf{B}_{h-1}+\omega(\Gamma)\mathbf{D}_{h-1}]\mathbf{V}_{h-1}(\Gamma)+[\mathbf{\bar{B}}_{h-1}+\omega(\Gamma)\mathbf{\bar{D}}_{h-1}]\mathbb{E}\mathbf{V}_{h-1}(\Gamma)$
	$\displaystyle+\sum_{j=0}^{h-2}\Big{[}[\mathcal{B}_{j}+\omega(\Gamma)D_{j}]\tilde{v}_{j,i}(\tau)+[\bar{\mathcal{B}}_{j}+\omega(\Gamma)\bar{D}]\mathbb{E}\tilde{v}_{j,i}(\tau)\Big{]}$
	$\displaystyle+[\mathcal{B}_{h}+\omega(\Gamma)D_{h}]v_{h}(\Gamma)+[\bar{\mathcal{B}}_{h}+\omega(\Gamma)\bar{D}_{h}]\mathbb{E}v_{h}(\Gamma).$	(33)

Then, applying (6.1) to (16) for $i=h-1$ , we can obtain

$\displaystyle 0=$	$\displaystyle\mathbf{R}_{h-1}\mathbf{V}_{h-1}(\Gamma)+\mathbf{\bar{R}}_{h-1}\mathbb{E}\mathbf{V}_{h-1}(\Gamma)+\mathbb{E}[\mathbf{B}_{h-1}^{T}(\Gamma)\theta(\Gamma)\mid\mathcal{F}_{h-1}(\Gamma)]+\mathbb{E}[\mathbf{\bar{B}}_{h-1}^{T}(\Gamma)\theta(\Gamma)]$
$\displaystyle=$	$\displaystyle\mathbf{\bar{\Upsilon}}_{h-1}(\Gamma)\mathbf{V}_{h-1}(\Gamma)+[\mathbf{\Upsilon}_{h-1}(\Gamma)-\mathbf{\bar{\Upsilon}}_{h-1}(\Gamma)]\mathbb{E}\mathbf{V}_{h-1}(\Gamma)$
	$\displaystyle+\bar{Y}_{h-1,h-1}(\Gamma)\hat{x}_{\Gamma/\Gamma-h+1}+[Y_{h-1,h-1}(\Gamma)-\bar{Y}_{h-1,h-1}(\Gamma)]\mathbb{E}x(\Gamma)$
	$\displaystyle+[\mathbf{B}_{h-1}^{T}\Phi(\Gamma+1)\mathcal{B}_{h}+\sigma^{2}\mathbf{D}_{h-1}^{T}\Phi(\Gamma+1)D_{h}]v_{h}(\Gamma)$
	$\displaystyle+\Big{[}(\mathbf{B}_{h-1}+\mathbf{\bar{B}}_{h-1})^{T}[\Phi(\Gamma+1)+\bar{\Phi}(\Gamma+1)](\mathcal{B}_{h}+\bar{\mathcal{B}}_{h})-\mathbf{B}_{h-1}^{T}\Phi(\Gamma+1)\mathcal{B}_{h}$
	$\displaystyle+\sigma^{2}(\mathbf{D}_{h-1}+\mathbf{\bar{D}}_{h-1})^{T}\Phi(\Gamma+1)(D_{h}+\bar{D}_{h})-\sigma^{2}\mathbf{D}_{h-1}^{T}\Phi(\Gamma+1)D_{h}\Big{]}\mathbb{E}v_{h}(\Gamma)$
$\displaystyle=$	$\displaystyle\mathbf{\bar{\Upsilon}}_{h-1}(\Gamma)\mathbf{V}_{h-1}(\Gamma)+[\mathbf{\Upsilon}_{h-1}(\Gamma)-\mathbf{\bar{\Upsilon}}_{h-1}(\Gamma)]\mathbb{E}\mathbf{V}_{h-1}(\Gamma)$
	$\displaystyle+\bar{Y}_{h-1,h-1}(\Gamma)\hat{x}_{\Gamma/\Gamma-h+1}+\bar{Y}_{h-1,h}(\Gamma)\hat{x}_{\Gamma/\Gamma-h}$
	$\displaystyle+\Big{[}[Y_{h-1,h-1}(\Gamma)-\bar{Y}_{h-1,h-1}(\Gamma)]+[Y_{h-1,h}(\Gamma)-\bar{Y}_{h-1,h}(\Gamma)]\Big{]}\mathbb{E}x(\Gamma).$	(34)

Similarly, by taking the mathematical expectation of the (6.1), there holds

\displaystyle 0=

\displaystyle\mathbf{\Upsilon}_{h-1}(\Gamma)\mathbb{E}\mathbf{V}_{h-1}(\Gamma)+[Y_{h-1,h-1}(\Gamma)+Y_{h-1,h}(\Gamma)]\mathbb{E}x(\Gamma),

(35)

it yields that

\displaystyle\mathbb{E}\mathbf{V}_{h-1}(\Gamma)=

\displaystyle-\mathbf{\Upsilon}_{h-1}^{-1}(\Gamma)[Y_{h-1,h-1}(\Gamma)+Y_{h-1,h}(\Gamma)]\mathbb{E}x(\Gamma).

(36)

Substitute (36) into (6.1), we have

$\displaystyle 0=$	$\displaystyle\mathbf{\bar{\Upsilon}}_{h-1}(\Gamma)\mathbf{V}_{h-1}(\Gamma)+[\mathbf{\Upsilon}_{h-1}(\Gamma)-\mathbf{\bar{\Upsilon}}_{h-1}(\Gamma)]\mathbb{E}\mathbf{V}_{h-1}(\Gamma)$
	$\displaystyle+\bar{Y}_{h-1,h-1}(\Gamma)\hat{x}_{\Gamma/\Gamma-h+1}+\bar{Y}_{h-1,h}(\Gamma)\hat{x}_{\Gamma/\Gamma-h}$
	$\displaystyle+\Big{[}[Y_{h-1,h-1}(\Gamma)-\bar{Y}_{h-1,h-1}(\Gamma)]+[Y_{h-1,h}(\Gamma)-\bar{Y}_{h-1,h}(\Gamma)]\Big{]}\mathbb{E}x(\Gamma)$
$\displaystyle=$	$\displaystyle\mathbf{\bar{\Upsilon}}_{h-1}(\Gamma)\mathbf{V}_{h-1}(\Gamma)-[\mathbf{\Upsilon}_{h-1}(\Gamma)-\mathbf{\bar{\Upsilon}}_{h-1}(\Gamma)]\mathbf{\Upsilon}_{h-1}^{-1}(\Gamma)[Y_{h-1,h-1}(\Gamma)$
	$\displaystyle+Y_{h-1,h}(\Gamma)]\mathbb{E}x(\Gamma)+\bar{Y}_{h-1,h-1}(\Gamma)\hat{x}_{\Gamma/\Gamma-h+1}+\bar{Y}_{h-1,h}(\Gamma)\hat{x}_{\Gamma/\Gamma-h}$
	$\displaystyle+\Big{[}[Y_{h-1,h-1}(\Gamma)-\bar{Y}_{h-1,h-1}(\Gamma)]+[Y_{h-1,h}(\Gamma)-\bar{Y}_{h-1,h}(\Gamma)]\Big{]}\mathbb{E}x(\Gamma)$
$\displaystyle=$	$\displaystyle\mathbf{\bar{\Upsilon}}_{h-1}(\Gamma)\mathbf{V}_{h-1}(\Gamma)+\bar{Y}_{h-1,h-1}(\Gamma)\hat{x}_{\Gamma/\Gamma-h+1}+\bar{Y}_{h-1,h}(\Gamma)\hat{x}_{\Gamma/\Gamma-h}$
	$\displaystyle+\Big{[}\mathbf{\bar{\Upsilon}}_{h-1}(\Gamma)\mathbf{\Upsilon}_{h-1}^{-1}(\Gamma)[Y_{h-1,h-1}(\Gamma)+Y_{h-1,h}(\Gamma)]-[\bar{Y}_{h-1,h-1}(\Gamma)+\bar{Y}_{h-1,h}(\Gamma)]\Big{]}\mathbb{E}x(\Gamma).$	(37)

Similar to the analysis above (6.1), (9) is uniquely solvable if and only if $\mathbf{\Upsilon}_{h-1}(\Gamma)$ and $\mathbf{\bar{\Upsilon}}_{h-1}(\Gamma)$ are invertible. Apparently, $v_{h-1}(\Gamma)$ can be derived.

The remainder of the argument is analogous to the above analysis, so by repeating the above procedures step by step, it can be verified that $\mathbf{V}_{i}(\Gamma)$ can be given by

\displaystyle\mathbf{V}_{i}(\Gamma)=

\displaystyle-\mathbf{\bar{\Upsilon}}_{i}^{-1}(\Gamma)\sum_{j=i}^{h}\bar{Y}_{i,j}(\Gamma)\hat{x}_{\Gamma/\Gamma-j}-\sum_{j=i}^{h}[\mathbf{\Upsilon}_{i}^{-1}(\Gamma)Y_{i,j}(\Gamma)-\mathbf{\bar{\Upsilon}}_{i}^{-1}(\Gamma)\bar{Y}_{i,j}(\Gamma)]\mathbb{E}x(\Gamma),

(38)

and $v_{i}(\Gamma)$ is given by

\displaystyle v_{i}(\Gamma)=

\displaystyle-\mathbf{I}_{i}\mathbf{\bar{\Upsilon}}_{i}^{-1}(\Gamma)\sum_{j=i}^{h}\bar{Y}_{i,j}(\Gamma)\hat{x}_{\Gamma/\Gamma-j}-\sum_{j=i}^{h}\mathbf{I}_{i}[\mathbf{\Upsilon}_{i}^{-1}(\Gamma)Y_{i,j}(\Gamma)-\mathbf{\bar{\Upsilon}}_{i}^{-1}(\Gamma)\bar{Y}_{i,j}(\Gamma)]\mathbb{E}x(\Gamma),

(39)

where $Y_{i,i}(\Gamma)$ , $\bar{Y}_{i,i}(\Gamma)$ , $Y_{i,j}(\Gamma)$ , $\bar{Y}_{i,j}(\Gamma)$ satisfy the following relationships

\displaystyle\left\{\begin{aligned} \bar{Y}_{i,i}(\Gamma)=&\mathbf{B}_{i}^{T}\Phi(\Gamma+1)\mathcal{A}+\sigma^{2}\mathbf{D}_{i}^{T}\Phi(\Gamma+1)C,\\ Y_{i,i}(\Gamma)=&(\mathbf{B}_{i}+\mathbf{\bar{B}}_{i})^{T}[\Phi(\Gamma+1)+\bar{\Phi}(\Gamma+1)](\mathcal{A}+\bar{\mathcal{A}})+\sigma^{2}(\mathbf{D}_{i}+\mathbf{\bar{D}}_{i})^{T}\Phi(\Gamma+1)(C+\bar{C}),\\ \bar{Y}_{i,j}(\Gamma)=&-[\mathbf{B}_{i}^{T}\Phi(\Gamma+1)\mathcal{B}_{j}+\sigma^{2}\mathbf{D}_{i}^{T}\Phi(\Gamma+1)D_{j}]\mathbf{I}_{j}\mathbf{\bar{\Upsilon}}_{j}^{-1}\sum_{m=j}^{h}\bar{Y}_{j,m}(\Gamma),\\ Y_{i,j}(\Gamma)=&-[(\mathbf{B}_{i}+\mathbf{\bar{B}}_{i})^{T}[\Phi(\Gamma+1)+\bar{\Phi}(\Gamma+1)](\mathcal{B}_{j}+\bar{\mathcal{B}}_{j})\\ &+\sigma^{2}(\mathbf{D}_{i}+\mathbf{\bar{D}}_{i})^{T}\Phi(\Gamma+1)(D_{j}+\bar{D}_{j})]\mathbf{I}_{j}\mathbf{\Upsilon}_{j}^{-1}\sum_{m=j}^{h}Y_{j,m}(\Gamma),i+1\leq j\leq h.\\ \end{aligned}\right.

(40)

In other words, $v_{i}(\Gamma)$ in (4.6) has been verified for $i=0,\cdots,h$ .

Afterwards, we will concentrate on calculating $\theta(\Gamma-1)$ . From (7), we can obtain that

$\displaystyle\theta(\Gamma-1)=$	$\displaystyle Qx(\Gamma)+\bar{Q}\mathbb{E}x(\Gamma)+\mathbb{E}[\mathcal{A}^{T}(\Gamma)\theta(\Gamma)\mid\mathcal{F}_{0}(\Gamma)]+\mathbb{E}[\bar{\mathcal{A}}^{T}(\Gamma)\theta(\Gamma)]$
$\displaystyle=$	$\displaystyle\big{[}Q+\mathcal{A}^{T}\Phi(\Gamma+1)\mathcal{A}+\sigma^{2}C^{T}\Phi(\Gamma+1)C\big{]}x(\Gamma)$
	$\displaystyle+\Big{[}\bar{Q}+(\mathcal{A}+\bar{\mathcal{A}})^{T}[\Phi(\Gamma+1)+\bar{\Phi}(\Gamma+1)](\mathcal{A}+\bar{\mathcal{A}})-\mathcal{A}^{T}\Phi(\Gamma+1)\mathcal{A}+\sigma^{2}(C+\bar{C})^{T}\Phi(\Gamma+1)(C+\bar{C})$
	$\displaystyle-\sigma^{2}C^{T}\Phi(\Gamma+1)C+\sum_{i=0}^{h}[\bar{L}_{i}^{T}(\Gamma)\mathbf{I}_{i}\mathbf{\bar{\Upsilon}}_{i}^{-1}(\Gamma)\sum_{j=i}^{h}\bar{Y}_{i,j}(\Gamma)-L_{i}^{T}(\Gamma)\mathbf{I}_{i}\mathbf{\Upsilon}_{i}^{-1}(\Gamma)\sum_{j=i}^{h}Y_{i,j}(\Gamma)]\Big{]}\mathbb{E}x(\Gamma)$
	$\displaystyle-\sum_{i=0}^{h}\bar{L}_{i}^{T}(\Gamma)\mathbf{I}_{i}\mathbf{\bar{\Upsilon}}_{i}^{-1}(\Gamma)\sum_{j=i}^{h}\bar{Y}_{i,j}(\Gamma)\hat{x}_{\Gamma/\Gamma-j}$
$\displaystyle=$	$\displaystyle\Phi(\Gamma)x(\Gamma)+\bar{\Phi}(\Gamma)\mathbb{E}x(\Gamma)+\sum_{j=1}^{h}\varphi_{j}(\Gamma)\hat{x}_{\Gamma/\Gamma-j},$	(41)

where we have used the relationships

	$\displaystyle\sum_{i=0}^{h}[\bar{L}_{i}^{T}(\Gamma)$	$\displaystyle\mathbf{I}_{i}\mathbf{\bar{\Upsilon}}_{i}^{-1}(\Gamma)\sum_{j=i}^{h}\bar{Y}_{i,j}(\Gamma)-L_{i}^{T}(\Gamma)\mathbf{I}_{i}\mathbf{\Upsilon}_{i}^{-1}(\Gamma)\sum_{j=i}^{h}Y_{i,j}(\Gamma)]$
	$\displaystyle=\sum_{j=0}^{h}\Big{[}\sum_{i=0}^{j}[$	$\displaystyle\bar{L}_{i}^{T}(\Gamma)\mathbf{I}_{i}\mathbf{\bar{\Upsilon}}_{i}^{-1}(\Gamma)\bar{Y}_{i,j}(\Gamma)-L_{i}^{T}(\Gamma)\mathbf{I}_{i}\mathbf{\Upsilon}_{i}^{-1}(\Gamma)Y_{i,j}(\Gamma)]\Big{]},$		(42)

and

\displaystyle\sum_{i=0}^{h}\bar{L}_{i}^{T}(\Gamma)\mathbf{I}_{i}\mathbf{\bar{\Upsilon}}_{i}^{-1}

\displaystyle(\Gamma)\sum_{j=i}^{h}\bar{Y}_{i,j}(\Gamma)=\sum_{j=0}^{h}[\sum_{i=0}^{j}\bar{L}_{i}^{T}(\Gamma)\mathbf{I}_{i}\mathbf{\bar{\Upsilon}}_{i}^{-1}(\Gamma)\bar{Y}_{i,j}(\Gamma)],

(43)

which mean that (27) has been proved for $\tau=\Gamma-1$ .

In order to adopt the induction method, we assume that the assertions in Theorem 4.6 also hold when $\xi+1\leq\tau\leq\Gamma$ . In other words, for $\xi+1\leq\tau\leq\Gamma$ , it is assumed that the following assertions hold:

1)

(9) is uniquely solvable if and only if $\mathbf{\Upsilon}_{i}(\tau)$ and $\mathbf{\bar{\Upsilon}}_{i}(\tau)$ are invertible, and the control strategy $v_{i}(\tau)$ satisfies (4.6);
2)

The Riccati equations (23) hold;
3)

$\theta(\tau-1)$ is given by (27).

In the following, we will verify (4.6) for $\tau=\xi$ . In fact, from (16), we have

$\displaystyle 0=$	$\displaystyle\mathbf{R}_{h}\mathbf{V}_{h}(\xi)+\mathbf{\bar{R}}_{h}\mathbb{E}\mathbf{V}_{h}(\xi)+\mathbb{E}[\mathbf{B}_{h}^{T}(\xi)\theta(\xi)\mid\mathcal{F}_{h}(\xi)]+\mathbb{E}[\mathbf{\bar{B}}_{h}^{T}(\xi)\theta(\xi)]$
$\displaystyle=$	$\displaystyle\mathbf{R}_{h}\mathbf{V}_{h}(\xi)+\mathbf{\bar{R}}_{h}\mathbb{E}\mathbf{V}_{h}(\xi)+\mathbb{E}\Big{[}[\mathbf{B}_{h}+\omega(\xi)\mathbf{D}_{h}]^{T}[\Phi(\xi+1)x(\xi+1)$
	$\displaystyle+\bar{\Phi}(\xi+1)\mathbb{E}x(\xi+1)+\sum_{j=1}^{h}\varphi_{j}(\xi+1)\hat{x}_{\xi+1/\xi+1-j}]\mid\mathcal{F}_{h}(\xi)\Big{]}+\mathbb{E}\Big{[}[\mathbf{\bar{B}}_{h}+\omega(\xi)\mathbf{\bar{D}}_{h}]^{T}$
	$\displaystyle\times[\Phi(\xi+1)x(\xi+1)+\bar{\Phi}(\xi+1)\mathbb{E}x(\xi+1)+\sum_{j=1}^{h}\varphi_{j}(\xi+1)\hat{x}_{\xi+1/\xi+1-j}]\Big{]}$
$\displaystyle=$	$\displaystyle\mathbf{\bar{\Upsilon}}_{h}(\xi)\mathbf{V}_{h}(\xi)+[\mathbf{\Upsilon}_{h}(\xi)-\mathbf{\bar{\Upsilon}}_{h}(\xi)]\mathbb{E}\mathbf{V}_{h}(\xi)+\bar{Y}_{h,h}(\xi)\hat{x}_{\xi/\xi-h}+[Y_{h,h}(\xi)-\bar{Y}_{h,h}(\xi)]\mathbb{E}x(\xi).$	(44)

Hence, (9) is uniquely solvable if and only if $\mathbf{\Upsilon}_{h}(\xi)$ and $\mathbf{\bar{\Upsilon}}_{h}(\xi)$ are invertible. In this case, $\mathbf{V}_{h}(\xi)$ and $v_{h}(\xi)$ in (4.6) can be calculated from (6.1), respectively.

In the following, we will verify $v_{h-1}(\xi)$ . Similar to the arguments in (6.1), it follows that

$\displaystyle 0=$	$\displaystyle\mathbf{R}_{h-1}\mathbf{V}_{h-1}(\xi)+\mathbf{\bar{R}}_{h-1}\mathbb{E}\mathbf{V}_{h-1}(\xi)+\mathbb{E}[\mathbf{B}_{h-1}^{T}(\xi)\theta(\xi)\mid\mathcal{F}_{h-1}(\xi)]+\mathbb{E}[\mathbf{\bar{B}}^{T}_{h-1}(\xi)\theta(\xi)]$
$\displaystyle=$	$\displaystyle\mathbf{R}_{h-1}\mathbf{V}_{h-1}(\xi)+\mathbf{\bar{R}}_{h-1}\mathbb{E}\mathbf{V}_{h-1}(\xi)+\mathbb{E}\Big{[}[\mathbf{B}_{h-1}+\omega(\xi)\mathbf{D}_{h-1}]^{T}[\Phi(\xi+1)x(\xi+1)$
	$\displaystyle+\bar{\Phi}(\xi+1)\mathbb{E}x(\xi+1)+\sum_{j=1}^{h}\varphi_{j}(\xi+1)\hat{x}_{\xi+1/\xi+1-j}]\mid\mathcal{F}_{h-1}(\xi)\Big{]}$
	$\displaystyle+\mathbb{E}\Big{[}[\mathbf{\bar{B}}_{h-1}+\omega(\xi)\mathbf{\bar{D}}_{h-1}]^{T}[\Phi(\xi+1)x(\xi+1)+\bar{\Phi}(\xi+1)\mathbb{E}x(\xi+1)+\sum_{j=1}^{h}\varphi_{j}(\xi+1)\hat{x}_{\xi+1/\xi+1-j}]\Big{]}$
$\displaystyle=$	$\displaystyle\mathbf{\bar{\Upsilon}}_{h-1}(\xi)\mathbf{V}_{h-1}(\xi)+[\mathbf{\Upsilon}_{h-1}(\xi)-\mathbf{\bar{\Upsilon}}_{h-1}(\xi)]\mathbb{E}\mathbf{V}_{h-1}+\bar{Y}_{h-1,h-1}(\xi)\hat{x}_{\xi/\xi-h+1}+\bar{Y}_{h-1,h}(\xi)\hat{x}_{\xi/\xi-h}$
	$\displaystyle+\Big{[}[Y_{h-1,h-1}(\xi)-\bar{Y}_{h-1,h-1}(\xi)]+[Y_{h-1,h}(\xi)-\bar{Y}_{h-1,h}(\xi)]\Big{]}\mathbb{E}x(\xi),$	(45)

i.e., $v_{h-1}(\xi)$ can be proved from (6.1).

Following the derivations of (6.1)-(40) and (6.1), it can be verified that $\mathbf{V}_{i}(\xi)$ satisfies (4.6), and $v_{i}(\xi)$ can be expressed as (4.6).

Subsequently, to end the induction method, $\theta(\xi-1)$ will be calculated.

$\displaystyle\theta(\xi$	$\displaystyle-1)=Qx(\xi)+\bar{Q}\mathbb{E}x(\xi)+\mathbb{E}[\mathcal{A}^{T}(\xi)\theta(\xi)\mid\mathcal{F}_{0}(\xi)]+\mathbb{E}[\bar{\mathcal{A}}^{T}(\xi)\theta(\xi)]$
$\displaystyle=$	$\displaystyle Qx(\xi)+\bar{Q}\mathbb{E}x(\xi)+\mathbb{E}\Big{[}[\mathcal{A}+\omega(\xi)C]^{T}\big{[}\Phi(\xi+1)x(\xi+1)+\bar{\Phi}(\xi+1)\mathbb{E}x(\xi+1)+\sum_{j=1}^{h}\varphi_{j}(\xi+1)\hat{x}_{\xi+1/\xi+1-j}\big{]}\mid\mathcal{F}_{0}(\xi)\Big{]}$
	$\displaystyle+\mathbb{E}\Big{[}[\bar{\mathcal{A}}+\omega(\xi)\bar{C}]^{T}\big{[}\Phi(\xi+1)x(\xi+1)+\bar{\Phi}(\xi+1)\mathbb{E}x(\xi+1)+\sum_{j=1}^{h}\varphi_{j}(\xi+1)\hat{x}_{\xi+1/\xi+1-j}\big{]}\Big{]}$
$\displaystyle=$	$\displaystyle\Big{[}Q+\mathcal{A}^{T}\Phi(\xi+1)\mathcal{A}+\sigma^{2}C^{T}\Phi(\xi+1)C+\mathcal{A}^{T}\varphi_{1}(\xi+1)\mathcal{A}-[\bar{L}_{0}^{T}(\xi)+\mathcal{A}^{T}\varphi_{1}(\xi+1)\mathbf{B}_{0}]\mathbf{\bar{\Upsilon}}_{0}^{-1}(\xi)\bar{Y}_{0,0}(\xi)\Big{]}x(\xi)$
	$\displaystyle+\bigg{[}\bar{Q}+(\mathcal{A}+\bar{\mathcal{A}})^{T}[\Phi(\xi+1)+\bar{\Phi}(\xi+1)+\sum_{j=1}^{h}\varphi_{j}(\xi+1)](\mathcal{A}+\bar{\mathcal{A}})-\mathcal{A}^{T}[\Phi(\xi+1)+\sum_{j=1}^{h}\varphi_{j}(\xi+1)]\mathcal{A}$
	$\displaystyle+\sigma^{2}(C+\bar{C})^{T}\Phi(\xi+1)(C+\bar{C})-\sigma^{2}C^{T}\Phi(\xi+1)C-\sum_{i=0}^{h}\Big{[}[L_{i}^{T}(\xi)\mathbf{I}_{i}+(\mathcal{A}+\bar{\mathcal{A}})^{T}\varphi_{i+1}(\xi+1)(\mathbf{B}_{i}+\mathbf{\bar{B}}_{i})]$
	$\displaystyle\times\sum_{j=i}^{h}\mathbf{\Upsilon}_{i}^{-1}(\xi)Y_{i,j}(\xi)-[\bar{L}_{i}^{T}(\xi)\mathbf{I}_{i}+\mathcal{A}^{T}\varphi_{i+1}(\xi+1)\mathbf{B}_{i}]\sum_{j=i}^{h}\mathbf{\bar{\Upsilon}}_{i}^{-1}(\xi)\bar{Y}_{i,j}(\xi)\Big{]}\bigg{]}\mathbb{E}x(\xi)$
	$\displaystyle-\sum_{j=1}^{h}\Big{[}\sum_{i=0}^{j}[\bar{L}_{i}^{T}(\xi)\mathbf{I}_{i}+\mathcal{A}^{T}\varphi_{i+1}(\xi+1)\mathbf{B}_{i}]\mathbf{\bar{\Upsilon}}_{i}^{-1}(\xi)\bar{Y}_{i,j}(\xi)-\mathcal{A}^{T}\varphi_{j+1}(\xi+1)\mathcal{A}\Big{]}\hat{x}_{\xi/\xi-j}$
$\displaystyle=$	$\displaystyle\Phi(\xi)x(\xi)+\bar{\Phi}(\xi)\mathbb{E}x(\xi)+\sum_{j=1}^{h}\varphi_{j}(\xi)\hat{x}_{\xi/\xi-j},$	(46)

which leads to (27). This ends the induction method.

As a consequence, we have shown that the unique solvability of (9) is equivalent to the invertibility of $\mathbf{\Upsilon}_{i}(\tau)$ and $\mathbf{\bar{\Upsilon}}_{i}(\tau)$ . In other words, Problem 2.2 is uniquely solvable if and only if $\mathbf{\Upsilon}_{i}(\tau)$ and $\mathbf{\bar{\Upsilon}}_{i}(\tau)$ are invertible for $0\leq i\leq h,i\leq\tau\leq\Gamma.$ Furthermore, the unique optimal control strategy can be presented as (4.6).

Finally, we shall calculate the optimal cost functional with the optimal control $v_{i}(\tau)$ in (4.6). Actually, it follows that:

$\displaystyle\mathbb{E}[x^{T}$	$\displaystyle(\tau)\theta(\tau-1)-x^{T}(\tau+1)\theta(\tau)]$
$\displaystyle=$	$\displaystyle E\bigg{[}x^{T}(\tau)\Big{[}Qx(\tau)+\bar{Q}\mathbb{E}x(\tau)+\mathbb{E}[\mathcal{A}^{T}(\tau)\theta(\tau)\mid\mathcal{F}_{0}(\tau)]+\mathbb{E}[\bar{\mathcal{A}}^{T}(\tau)\theta(\tau)]\Big{]}$
	$\displaystyle-\big{\{}\mathcal{A}(\tau)x(\tau)+\bar{\mathcal{A}}(\tau)\mathbb{E}x(\tau)+\sum_{i=0}^{h}[\mathcal{B}_{i}(\tau)v_{i}(\tau)+\bar{\mathcal{B}}_{i}(\tau)\mathbb{E}v_{i}(\tau)]\big{\}}^{T}\theta(\tau)\bigg{]}$
$\displaystyle=$	$\displaystyle\mathbb{E}\bigg{[}x^{T}(\tau)Qx(\tau)+[\mathbb{E}x(\tau)]^{T}\bar{Q}\mathbb{E}x(\tau)-\sum_{i=0}^{h}v_{i}^{T}(\tau)\Big{[}\mathbb{E}[\mathcal{B}_{i}^{T}(\tau)\theta(\tau)\mid\mathcal{F}_{i}(\tau)]+\mathbb{E}[\bar{\mathcal{B}}_{i}^{T}(\tau)\theta(\tau)]\Big{]}\bigg{]}$
$\displaystyle=$	$\displaystyle\mathbb{E}\bigg{[}x^{T}(\tau)Qx(\tau)+[\mathbb{E}x(\tau)]^{T}\bar{Q}\mathbb{E}x(\tau)+\sum_{i=0}^{h}\Big{[}v_{i}^{T}(\tau)\mathcal{R}_{i}v_{i}(\tau)+[\mathbb{E}v_{i}(\tau)]^{T}\bar{\mathcal{R}}_{i}\mathbb{E}v_{i}(\tau)\Big{]}\bigg{]}.$	(47)

Then adding from $0$ to $\Gamma$ on both sides of (6.1), there holds that

$\displaystyle\mathbb{E}[x^{T}$	$\displaystyle(0)\theta(-1)-x^{T}(\Gamma+1)\theta(\Gamma)]$
$\displaystyle=$	$\displaystyle E\Big{[}x^{T}(0)\theta(-1)-x^{T}(\Gamma+1)[\Phi(\Gamma+1)x(\Gamma+1)+\bar{\Phi}(\Gamma+1)\mathbb{E}x(\Gamma+1)]\Big{]}$
$\displaystyle=$	$\displaystyle\sum_{\tau=0}^{\Gamma}\mathbb{E}\Big{[}x^{T}(\tau)Qx(\tau)+[\mathbb{E}x(\tau)]^{T}\bar{Q}\mathbb{E}x(\tau)\Big{]}+\sum_{i=0}^{h}\sum_{\tau=i}^{\Gamma}\Big{[}v_{i}^{T}(\tau)\mathcal{R}_{i}v_{i}(\tau)+[\mathbb{E}v_{i}(\tau)]^{T}\bar{\mathcal{R}}_{i}\mathbb{E}v_{i}(\tau)\Big{]}.$	(48)

Moreover, (2) can be rewritten as

	$\displaystyle J_{\Gamma}(v)=$	$\displaystyle\sum_{\tau=0}^{i-1}\mathbb{E}\Big{[}x^{T}(\tau)Qx(\tau)+[\mathbb{E}x(\tau)]^{T}\bar{Q}\mathbb{E}x(\tau)+\sum_{i=0}^{h}\big{[}v_{i}^{T}(\tau)\mathcal{R}_{i}v_{i}(\tau)$
		$\displaystyle+[\mathbb{E}v_{i}(\tau)]^{T}\bar{\mathcal{R}}_{i}\mathbb{E}v_{i}(\tau)\big{]}\Big{]}+\sum_{\tau=i}^{\Gamma}\mathbb{E}\Big{[}x^{T}(\tau)Qx(\tau)+[\mathbb{E}x(\tau)]^{T}\bar{Q}\mathbb{E}x(\tau)$
		$\displaystyle+\sum_{i=0}^{h}\big{[}v_{i}^{T}(\tau)\mathcal{R}_{i}v_{i}(\tau)+[\mathbb{E}v_{i}(\tau)]^{T}\bar{\mathcal{R}}_{i}\mathbb{E}v_{i}(\tau)\big{]}\Big{]}$
		$\displaystyle+\mathbb{E}[x^{T}(\Gamma+1)\Phi(\Gamma+1)x(\Gamma+1)]+[\mathbb{E}x(\Gamma+1)]^{T}\bar{\Phi}(\Gamma+1)\mathbb{E}x(\Gamma+1).$

Subsequently, by using (6.1), we have

\displaystyle J_{\Gamma}(v)=

\displaystyle\mathbb{E}[x^{T}(0)\theta(-1)]+\sum_{i=0}^{h}\sum_{\tau=0}^{i-1}\mathbb{E}\Big{[}v_{i}^{T}(\tau)\mathcal{R}_{i}v_{i}(\tau)+[\mathbb{E}v_{i}(\tau)]^{T}\bar{\mathcal{R}}_{i}\mathbb{E}v_{i}(\tau)\Big{]}.

Finally, the optimal cost functional is given, with the optimal control $v_{i}(\tau)$ in (4.6), we can obtain that

	$\displaystyle J_{\Gamma}^{*}(v)=$	$\displaystyle\mathbb{E}[x^{T}(0)\theta(-1)]+\sum_{i=0}^{h}\sum_{\tau=0}^{i-1}\mathbb{E}\Big{[}v_{i}^{T}(\tau)\mathcal{R}_{i}v_{i}(\tau)+[\mathbb{E}v_{i}(\tau)]^{T}\bar{\mathcal{R}}_{i}\mathbb{E}v_{i}(\tau)\Big{]}$
	$\displaystyle=$	$\displaystyle\mathbb{E}\Big{[}x^{T}(0)\Phi(0)x(0)+[\mathbb{E}x(0)]^{T}\bar{\Phi}(0)\mathbb{E}x(0)+x^{T}(0)\sum_{j=1}^{h}\varphi_{j}(0)\hat{x}_{0/-j}\Big{]}$
		$\displaystyle+\sum_{i=0}^{h}\sum_{\tau=0}^{i-1}\mathbb{E}\Big{[}v_{i}^{T}(\tau)\mathcal{R}_{i}v_{i}(\tau)+[\mathbb{E}v_{i}(\tau)]^{T}\bar{\mathcal{R}}_{i}\mathbb{E}v_{i}(\tau)\Big{]}$
	$\displaystyle=$	$\displaystyle x^{T}(0)[\Phi(0)+\bar{\Phi}(0)+\sum_{j=1}^{h}\varphi_{j}(0)]x(0)+\sum_{i=0}^{h}\sum_{\tau=0}^{i-1}v_{i}^{T}(\tau)[\mathcal{R}_{i}+\bar{\mathcal{R}}_{i}]v_{i}(\tau),$

which is (26). The proof is complete.

References

1 M. Kac, “Foundations of kinetic theory,” In Proceedings of The third Berkeley symposium on mathematical statistics and probability, 3(1956), pp. 171-197.
2 H. P. McKean, “A class of Markov processes associated with nonlinear parabolic equations,” In Proceedings of the National Academy of Sciences of the United States of America, 56(1966), no. 6, pp. 1907-1911.
3 J. Yong, “Linear-quadratic optimal control problems for mean-field stochastic differential equations,” SIAM Journal on Control and Optimization, 51(2013), no. 4, pp. 2809-2838.
4 R. Elliott, X. Li, and Y. Ni, “Discrete time mean-field stochastic linear-quadratic optimal control problems,” Automatica, 49(2013), no. 11, pp. 3222-3233.
5 Y. Ni, R. Elliott and X. Li, “Discrete-time mean-field stochastic linear-quadratic optimal control problems, II: Infinite horizon case,” Automatica, 57(2015), pp. 65-77.
6 Q. Qi, H. Zhang and Z. Wu, “Stabilization control for linear continuous time mean-field systems,” IEEE Transactions on Automatic Control, 64(2019), no. 8, pp. 3461-3468.
7 H. Zhang and Q. Qi, “Optimal control for mean-field system: discrete time case,” IEEE 55th Conference on Decision and Control, (2016), pp. 4474-4480.
8 H. Zhang, Q. Qi and M. Fu, “Optimal stabilization control for discrete time mean-field stochastic systems,” IEEE Transactions on Automatic Control, 64(2019), no. 3, pp. 1125-1136.
9 B. C. Wang and H. Zhang, “Indefinite linear quadratic mean field social control problems with multiplicative noise,” IEEE Transactions on Automatic Control, 66(2021), no. 11, pp. 5221-5236.
10 N. Li, X. Li and Z. Yu, “Indefinite mean-field type linear-quadratic stochastic optimal control problems,” Automatica, 122(2020), pp. 109267:1-109267:10.
11 W. Yu, F. Wang, Y. Huang and H. Liu, “Social optimal mean field control problem for population growth model,” Asian Journal of Control, 22(2019), no. 6, pp. 2444-2451.
12 Z. Li, D. Marelli, M. Fu and Z. Ren, “Decentralized social-optimal solution of finite number of average field linear quadratic control,” Asian Journal of Control, 24(2022), no. 4, pp. 1916-1927.
13 Q. Qi, L. Xie and H. Zhang, “Linear quadratic optimal control for discrete-time mean-field systems with input delay,” IEEE Transactions on Automatic Control, 67(2022), no. 8, pp. 3806-3821.
14 H. S. Witsenhausen, “A Counterexample in stochastic optimum control,” SIAM Journal on Control, 6(1968), no. 1, pp. 131-147.
15 L. Li and H. Zhang, “Stabilization of discrete-time systems with multiplicative noise and multiple delays in the control variable,” SIAM Journal on Control and Optimization, 54(2016), no. 2, pp. 894-917.
16 X. Liang and J. Xu, “Control for network control systems with remote and local controllers over unreliable communication channel,” Automatica, 98(2018), pp. 86-94.
17 S. M. Asghari, Y. Ouyang and A. Nayyar, “Optimal local and remote controllers with unreliable uplink channels,” IEEE Transactions on Automatic Control, 64(2019), no. 5, pp. 1816-1831.
18 Q. Qi, L. Xie and H. Zhang, “Optimal control for stochastic systems with multiple controllers of different information structures,” IEEE Transactions on Automatic Control, 66(2021), no. 9, pp. 4160-4175.
19 Q. Wang, B. Fu, C. Lin and P. Li, “Exponential synchronization of chaotic Lur’e systems with time-triggered intermittent control,” Communications in Nonlinear Science and Numerical Simulation, 109(2022), pp. 106298.
20 Q. Wang and Y. He, “Time-triggered intermittent control of continuous systems,” International Journal of Robust and Nonlinear Control, 31(2021), pp. 6867-68789.
21 A. Nayyar, A. Gupta, C. Langbort and T. Başar, “Common information based Markov perfect equilibria for stochastic games with asymmetric information: finite games,” IEEE Transactions on Automatic Control, 59(2014), no. 3, pp. 555-570.
22 A. Gupta, A. Nayyar, C. Langbort and T. Başar, “Common information based Markov perfect equilibria for linear-Gaussian games with asymmetric information,” SIAM Journal on Control and Optimization, 52(2014), no. 5, pp. 3228-3260.