Stochastic Maximum Principle for a generalized Volterra Control System

To better describe the real-world, the stochastic integral differential equations have been studied in many areas, such as in biological science, applied mathematics, physics, and other disciplines, etc ^{1, 2, 3, 4}. Mao and Riedle ⁵ study the stability of some types of stochastic volterra integral differential equations. Nesterenko⁶ substantiate the application of a modified projection-iterative method to the solution of boundary value problems for weakly nonlinear integrodifferential equations with parameters. Dzhumabaev⁷ establish the necessary and sufficient conditions for the well-posedness of linear boundary value problems for Fredholm integro-differential equations . Zhang et al.⁸ investigate numerical analysis of the following generalized stochastic volterra integral differential equations

They prove the existence and uniqueness of the solution when $\|k_{i}\|$ and $\|\sigma_{i}\|$ are bounded.

Control problems for integral differential equations have also been studied. Kim⁹ discuss a reachability problem for a second-order integro-differential equation based upon a new kind of unique continuation property. Mashayekhi et al.¹⁰ give a new numerical method for solving the optimal control of a class of systems described by integro-differential equations with quadratic performance index. Assanova et al.¹¹ present the existence of optimal controls of systems governed by impulsive integro-differential equations of mixed type. Wang¹² investigate the optimal control problems in terms of maximum principles and linear quadratic control problems of optimal control for forward stochastic Volterra integro-differential equations.

In this paper we focus on the following Volterra control system

to minimize the cost function

We call the corresponding stochastic differential equation (SDE in short) as a generalized stochastic Volterra integral differential equation (SVIDE in short), which is a specific type of integral differential equations. This type of equation is influenced by the past information of both state process and control process from beginning to present.

It should be pointed out that most SVIDEs can not be written as stochastic Volterra integral equations with the following form

For example, consider the following SDE,

where the drift term is nonlinear on the integral of state process.

We study the uniqueness of the solution of the SVIDE. Different from classical condition, to deal with the term contain the past information, we use Gronwall inequality to $\sup_{0\leq r\leq t}E|\tilde{X}_{r}-X_{r}|^{2}$, which is a upper bound for $CE\left[\left|\frac{1}{t}\int_{0}^{t}\tilde{X}_{s}ds-\frac{1}{t}\int_{0}^{t}X_{s}ds\right|\right]^{2}$. Then a new stochastic maximum principle for control system (3) is established. We define the Hamiltonian function and the adjoint equation to obtain the optimal system. To study the properties of the adjoint equation, we prove the existence and uniqueness of the solution of the following equation

which is singular at point 0. Compared with classical type investigated by El Karoui and Peng¹³, we construct a contraction mapping under a new $\beta$-norm

Furthermore, we get the necessary condition that the optimal control process should satisfy. Consider the linear quadratic case, which can be applied to a Volterra linear quadratic state regulator, we obtain the unique optimal control process for linear quadratic Volterra control system.

The rest of this paper is organized as follows. In section 2, we introduce a type of generalized SVIDE and prove the existence and the uniqueness of the solution of this type of equation. In section 3, we prove the stochastic maximum principle by introducing a kind of anticipated backward stochastic differential equations, and the existence and uniquenes of this kind of equations is proved. In the section 4, the linear quadratic case is investigated to illustrate the main result.

Let $(\Omega,\mathcal{F},\mathbb{P})$ be a probability space. $\mathcal{F}_{0}\subset\mathcal{F}$ be a sub $\sigma$-algebra, and $\mathbb{F}=(\mathcal{F}_{t})_{0\leq t\leq T}$ be the filtration generated by $\mathcal{F}_{0}$ and a $m$-dimensional standard Brownian motion $\textbf{W}=(W_{t})_{0\leq t\leq T}$. We consider the following stochastic differential equation.

where $E|X_{0}|^{2}<\infty$, $b$ and $\sigma$ be measurable functions on $[0,T]\times\mathbf{R}^{d}\times\mathbf{R}^{d}$ with values in $\mathbf{R}^{d}$ and $\mathbf{R}^{d\times m}$, respectively. Here

where $k(t,s)$ satisfies $\sup_{0\leq t\leq T}\int_{0}^{t}|k(t,s)|\leq M$ for some constants $M>0$. It is obviously that $Y_{t}$ is continuous. This class of equations provides a description of the effect of past situations on the current situation. Assume that

and

for some constant $L>0$ (where $|\sigma|^{2}=\sum|\sigma_{ij}|^{2}$).

Now we show the existence and uniqueness of the solution of equation (6).  

Lemma 2.1  If condition (10) and (2) holds, there exist a unique solution to equation (6).  

Proof: Uniqueness. Let $X_{t}$ and $\tilde{X}_{t}$ be two solutions of the equation (6), $Y_{t}$ and $\tilde{Y}_{t}$ are corresponding moving average processes, and $X_{0}=Y_{0}=\tilde{X}_{0}=\tilde{Y}_{0}=x$. Thus, we have

The last inequality holding is because

For every $\varepsilon>0$, there exits $\xi_{t}\in[0,t]$, such that

so that

Through the Gronwall’s inequality and the arbitrariness of $\varepsilon$, we get $\sup_{0\leq t\leq T}E|\tilde{X}_{t}-X_{t}|^{2}=0$. Thus, the solution $X_{t}$ is unique.  

Existence. Let

and

Similar to the proof of classical case, we get

for some constants $A>0$. Let $\lambda$ be Lebesgue measure on $[0,T]$, $0\leq n<m$ and $m,n\to\infty$. Then we have  

Therefore, $\{X_{t}^{(n)}\}_{n\geq 0}$ is a Cauchy sequence in ${L^{2}(\lambda\times P)}$. Define

Then $X_{t}$ and $Y_{t}$ are $\mathcal{F}_{t}$-measurable for all $t$. Since this holds for each $X_{t}^{(n)}$ and $Y_{t}^{(n)}$, thus $X_{t}$ is the solution of (6).

Consider the following control problem. The state equation is

with the cost function

where $b(t,x,y,u,v)$ and $\sigma(t,x,y,u,v)$ are measurable functions on $\mathbf{R}\times\mathbf{R}^{d}\times\mathbf{R}^{d}\times\mathbf{R}^{k}\times\mathbf{R}^{k}$ with values in $\mathbf{R}^{d}$ and $\mathbf{R}^{d\times m}$, respectively. $f(t,x,y,u,v)$ and $g(x)$ be measurable functions on $\mathbf{R}\times\mathbf{R}^{d}\times\mathbf{R}^{d}\times\mathbf{R}^{k}\times\mathbf{R}^{k}$ and $\mathbf{R}^{d}$, respectively, with values in $\mathbf{R}$. We denote by $\mathbb{U}$ the set of progressively measurable process u$=(u_{t})_{0\leq t\leq T}$ taking values in a given closed-convex set $\textbf{U}\subset\mathbb{R}^{k}$ and satisfying $E\int_{0}^{T}|u_{t}|^{2}dt<\infty$.

To simplify the notation without losing the generality, we just consider the case $d=m=k=1$. We assume $u_{t}^{*}$ is the optimal control process, i.e.,

For all $0<\varepsilon<1$, let

where $\alpha_{t}$ is any other admissible control.

We define the Hamiltonian function $H$ by

Denote

for $\phi=b,\sigma,f,b_{x},\sigma_{x},f_{x},b_{y},\sigma_{y},f_{y},b_{u},\sigma_{u},f_{u},b_{v},\sigma_{v},f_{v}$.  

Theorem 3.1   If $(u_{t}^{*})_{0\leq t\leq T}$ is the optimal control process, $(X_{t}^{*})_{0\leq t\leq T}$ and $(p_{t},q_{t})$ is the process satisfying

Then we have

for any control process $\alpha_{t}$, where

Remark 3.2  To investigate the adjoint equation (21), we consider a more general type of anticipated backward stochastic differential equations:

Without losing the generality, we assume $\sup_{0\leq t\leq T}\int_{0}^{t}|k(t,s)|ds=\sup_{0\leq t\leq T}\int_{0}^{t}|l(t,s)|ds=1$. This type of anticipated backward stochastic differential equations may be singular even we assume $h(t,y,z,\tilde{y},\tilde{z})$ is Lipscitz continous, such as $k(t,s)=\frac{1}{t}$. We would deal with it in the following Lemma.

Anticipated backward stochastic differential equations was first studied by Peng and Yang ¹⁴, they study the following type of equation:

the unique solutions, a comparison theorem, and a duality between them and stochastic differential delay equations are introduced. More properties of generalized anticipated backward stochastic differential equations refer to Yang and Elliott ¹⁵.

Lemma 3.3 The anticipated backward stochastic differential equation (26) has the unique solution pair if the following conditions hold:

and

for some constants $M_{1},M_{2}>0$ satisfy $M_{1}M_{2}<8^{-1}T^{-\frac{1}{2}}$.

Proof: Denote $\mathbb{H}_{T}^{2}\left(\mathbb{R}^{d}\right)$ is the space of all predictable processes $\phi:\Omega\times[0,T]\mapsto\mathbb{R}^{d}$ such that $\|\varphi\|^{2}=$ $\mathbb{E}\int_{0}^{T}\left|\varphi_{t}\right|^{2}dt<+\infty.$ We define $\beta$-norm: $\|(Y,Z)\|_{\beta}=\sup_{0\leq s\leq T}Ee^{\beta s}|Y_{s}|^{2}+E\int_{0}^{T}e^{\beta s}Z_{s}^{2}ds$ on $\mathbb{H}_{T}^{2}\left(\mathbb{R}^{d}\right)\times\mathbb{H}_{T}^{2}\left(\mathbb{R}^{d\times m}\right)$. For any $\mathcal{F}_{t}$-adapted continuous process pair $(y_{t}^{1},z_{t}^{1}),(y_{t}^{2},z_{t}^{2})$ with bounded $\beta$-norm, let

for $i=1,2$.

We denote

for $\phi=Y,Z,y,z$, and

We can directly get

By it$\hat{\rm o}$’s formula,

Taking the expectation and integral, we have

Notice that

and

so we have

In the same way

Substitute (3) and (30) into (3), choose $\beta>(c_{1}^{-1}+c_{2}^{-1})M_{1}$ and let $c_{1},c_{2}\to\infty$, we get

Then it is not difficult to get

which shows