Lifting of Volterra processes:
optimal control in UMD Banach spaces

Giulia di Nunno^†,∗ and Michele Giordano^‡
June 22nd 2023

Abstract.

We study a stochastic control problem for a Volterra-type controlled forward equation with past dependence obtained via convolution with a deterministic kernel. To be able to apply dynamic programming to solve the problem, we lift it to infinite dimensions and we formulate a UMD Banach-valued Markovian problem, which is shown to be equivalent to the original finite-dimensional non-Markovian one. We characterize the optimal control for the infinite dimensional problem and show that this also characterizes the optimal control for the finite dimensional problem.

^†^†^†Department of Mathematics, University of Oslo, P:O: Box 1053 Blindern, N-0316 Oslo, Email: giulian@math.uio.no.^†^†^∗Department of Business and Management Science, NHH Norwegian School of Economics, Helleveien 30, N-5045 Bergen.^†^†^‡Department of Mathematics, University of Oslo, P:O: Box 1053 Blindern, N-0316 Oslo, Email: michelgi@math.uio.no

Keywords: Backward stochastic integral equation; Dynamic programming principle;
Hamilton Jacobi Bellman; Optimal control; UMD Banach space; Markovian Lift;
MSC 2020: 60H10; 60H20; 93E20; 35R15; 49L20; 91B70;

1. Introduction

We intend to minimize a performance functional of the form

J(t,x,u)=\mathbb{E}\left[\int_{t}^{T}F(\tau,X^{u}_{\tau},u_{\tau})d\tau+G(X^{u}_{T})\right],

(1.1)

where $t\in[0,T]$ , and $x$ is given in the controlled Volterra-type dynamics of the process $X_{\tau}^{u}$ :

	$\displaystyle X^{u}(\tau)$	$\displaystyle=x(\tau)+\int_{t}^{\tau}K(\tau-s)\Big{[}\beta(s,X^{u}(s))+\sigma(s,X^{u}(s))R(s,X^{u}(s),u(s))\Big{]}ds$
		$\displaystyle\quad+\int_{t}^{\tau}K(\tau-s)\sigma(s,X^{u}(s))dW(s).$		(1.2)

Here $x:[0,T]\longrightarrow\mathbb{R}$ , $\beta:[0,T]\times\mathbb{R}\longrightarrow\mathbb{R}$ , $\sigma:[0,T]\times\mathbb{R}\longrightarrow\mathbb{R}$ , $R:[0,T]\times\mathbb{R}\times\mathcal{U}\longrightarrow\mathbb{R}$ , and the convolution kernel $K:[0,T]\longrightarrow\mathbb{R}^{+}$ are all measurable mappings on which additional hypothesis will be stated later on, and $u:[0,T]\times\Omega\longrightarrow\mathcal{U}\subset\mathbb{R}$ is an admissible control. Also, $W$ is a real-valued Brownian motion on a complete filtered probability space $(\Omega,\mathcal{F},\mathbb{P})$ .

Stochastic control problems as (1.1)-(1.2) appear, e.g. when studying optimal advertising strategies (see e.g. [15] for the case of Volterra dynamics in (1.2) and [17, 16, 20] for the case of delay). Other cases of applications are found in electrodynamics [24] and in epidemiology [25]. When dealing with such problems, one cannot directly apply a dynamic programming principle (DPP) in view of the non Markovianity of the framework. While in some particular cases is still possible to derive the DPP also for Volterra forward dynamics (see [1, 18, 5]), most authors approached the general problem by means of a maximum principle (see, e.g., [12, 29, 30, 3, 2] and references therein). Even though the maximum principle approach might seem practical, one usually has to impose regularity conditions on both the drift and volatility which are not always easy to satisfy. In this paper, thanks to the developments on the lift theory for Volterra processes (see [1, 9, 8, 7, 15, 5]) we aim to move the stochastic control problem (1.1)-(1.2) to an infinite dimensional UMD-Banach setting and solve the newly formulated problem by means of DPP.

The main purpose of this lifting approach is to recover the Markov property for the forward process (1.2) which, in turns, allows us to derive a DPP in terms of the Hamilton-Jacobi-Bellman (HJB) equations. In fact, one can show that solving the lifted problem is equivalent to solving the original one, with the fundamental difference that, by moving to an infinite dimensional setting, we work in a Markovian framework. Focusing on Markovian lifts, we assume that the kernel $K$ can be represented as $K(t)=\langle g,\mathcal{S}_{t}^{*}\nu\rangle_{Y\times Y^{*}}$ , for $\mathcal{S}_{t}^{*}$ a uniformly continuous semigroup acting on a Banach space $Y^{*}$ , $\nu\in Y^{*}$ , $g\in Y$ with $Y$ the pre dual of $Y^{*}$ and pairing $\langle\cdot,\cdot\rangle_{Y\times Y^{*}}$ . Examples of kernels that satisfy this condition can be found both in [9, 8, 15] and in the last section of this paper.

Our goal is to find $\hat{u}$ such that, for all $t\in[0,T]$ ,

J(t,x,\hat{u})=\inf_{u\in\mathds{A}}J(t,x,u),

(1.3)

with $J(t,x,u)$ as in (1.1) and for $u$ belonging to some admissible control set $\mathds{A}$ defined as

\mathds{A}=\left\{u:[0,T]\times\Omega\longrightarrow\mathcal{U},\ \text{s.t. $u$ is predictable}\right\},

where $\mathcal{U}$ is a closed convex subset of $\mathbb{R}$ and the information flow is associated to the Brownian motion in (1.2). Our approach consists in formulating a new infinite-dimensional Banach-valued optimization problem that can be shown to be equivalent to (1.1)-(1.2) (in the sense that the optimal control $\hat{u}$ and optimal value $J(t,x,\hat{u})$ , $t\in[0,T]$ are the same of the original one) and then solve such infinite dimensional optimization problem, which is Markovian. The solution is achieved exploiting Malliavin calculus for unconditional martingale differences (UMD) Banach spaces.

The Markovian lift to the infinite dimensional setting that we present here was originally introduced in [9], and then developed in [8] for the multi-dimensional case, and in [7] for a Lévy drivers. Our work can be seen as a generalization of the case presented in [1], in which a kernel $K$ that can be expressed as the Laplace transform of a measure is considered, and where the performance functional (1.1) is of linear-quadratic type. Our work differs from [1] as we are able to consider a broader class of kernels and performance functionals thanks to the nature of the lift we apply.

The present work introduces an element of novelty also with respect to infinite dimensional stochastic control. Indeed, we consider a setting which is different from both the ones presented in [14] and [22]. In [14] the authors consider a Hilbert valued forward controlled process, whereas in [22] the forward process has values in a general Banach space, but with a volatility term $\sigma$ not depending on $X$ . Here we are able to take general volatility dynamics for the forward process and this will require to work in Banach spaces of the UMD type so to be able to apply Malliavin calculus techniques.

In the context of optimal control for lifted process, we also mention [5]. There, the authors follow an approach close to the one presented here. However, we remark that, in our framework, we are able to consider a wider class of kernel $K$ thanks to the nature of our lift, which allows us to work in UMD Banach spaces instead of Hilbert spaces. On the other side, in [5] the authors consider a forward equation driven by a Lévy process instead of a Brownian motion (like (1.2)). While the lift theory for Lévy-driven forward processes is available (see [7]), the optimal control of an infinite-dimensional Lévy-driven forward equation in the present setting is a topic for future research.

This paper is structured as follows: in Section 2 we present some preliminary results both on the Gâteaux differentiability in general Banach spaces and on the lift for Volterra processes. We recall the essentials on UMD Banach spaces and some results of Malliavin calculus in this framework. In Section 3 we give an existence and continuity result for the forward equation, and in Section 4 we introduce the backward equation and the Hamiltonian function associated with the lifted optimization problem. Here we present a solution method via HJB equations. To conclude, in Section 5 we present a problem of optimal consumption where we obtain a characterization of the optimal control $u$ via DPP.

2. Some preliminary results

We recall some useful notions and results that are used throughout the paper. Then we show how the Markovian lift is performed in the present context of stochastic control (1.1)-(1.2). Lastly, we introduce UMD Banach spaces and state some crucial results for Malliavin calculus in this setting. We refer to [14] for the results on Gâteaux derivatives and Banach spaces, to [9, 8, 7, 15] for the ones concerning Markovian lifts, and to [19, 23, 26, 21] for the results concerning UMD Banach spaces.

2.1. The class of Gâteaux differentiable functions

For a mapping $F:U\longrightarrow V$ , with $U,V$ two Banach spaces, we say that the directional derivative at $u\in U$ in the direction $h\in U$ is defined as

\nabla F(u;h):=\lim_{s\rightarrow 0}\frac{F(u+sh)-F(u)}{s},

whenever the limit exists in the topology of $V$ . The mapping $F$ is said to be Gâteaux differentiable at the point $u$ if it has directional derivative at $u$ in every direction and there exists an element $\nabla F(u)$ in $L(U,V)$ such that $\nabla F(u;h)=\nabla F(u)h$ for every $h\in U$ . We call $\nabla F(u)$ the Gâteaux derivative at $u$ .

Definition 2.1.

A mapping $F:U\longrightarrow V$ belongs to $\mathcal{G}^{1}(U;V)$ if it is continuous, Gâteaux differentiable for all $u\in\mathcal{U}$ and $\nabla F:U\longrightarrow L(U;V)$ is strongly continuous, i.e. the map $\nabla F(\cdot)h:U\longrightarrow V$ is continuous for every $h\in U$ .

Remark 2.2.

Let $U,V,Z$ be three Banach spaces and $F\in\mathcal{G}^{1}(U,V)$ . If $G\in\mathcal{G}^{1}(V,Z)$ , then $G(F)\in\mathcal{G}^{1}(U,Z)$ and $\nabla(G(F))(u)=\nabla G(F(u))\nabla F(u)$ .

We also introduce the partial directional derivative for a mapping $F:U\times V\longrightarrow Z$ , with $U$ , $V$ , $Z$ Banach spaces as

\nabla_{u}F(u,v;h):=\lim_{s\rightarrow 0}\frac{F(u+sh,v)-F(u,v)}{h},

with $u,h\in U$ , $v\in V$ and the limit taken in the topology of $Z$ . We say that $F$ is partially Gâteaux differentiable with respect to $u$ at $(u,v)\in U\times V$ if there exists $\nabla_{u}F:U\times V\longrightarrow L(U,Z)$ such that $\nabla_{u}F(u,v;h)=\nabla_{u}F(u,v)h$ for all $h\in U$ .

Definition 2.3.

We say that $F:U\times V\longrightarrow Z$ belongs to the class $\mathcal{G}^{1,0}(U\times V;Z)$ if it is continuous, Gâteaux differentiable with respect to $u$ , for all $(u,v)\in U\times V$ and $\nabla_{u}F:U\times V\longrightarrow L(U,Z)$ is strongly continuous.

For $F$ depending on additional arguments, the definition above can be easily generalized.

Lemma 2.4.

Given $U,V,Z$ three Banach spaces, a continuous map $F:U\times V\longrightarrow Z$ belongs to $\mathcal{G}^{1,0}(U\times V,Z)$ provided the following conditions hold:

(1)

The partial directional derivatives $\nabla_{u}F(u,v;h)$ exist at every point $(u,v)\in U\times V$ and in every directoin $h\in U$ .
(2)

For every $(u,v)$ the mapping $h\longmapsto\nabla_{u}F(u,v;h)$ is continuous from $U$ to $Z$ .
(3)

For every $h$ , the mapping $\nabla_{u}F(\cdot,\cdot;h):U\times V\longrightarrow Z$ is continuous.

We are going to use the following parameter depending contraction principle to study the regular dependence of the solution of stochastic differential equations on their initial data.

Proposition 2.5.

Let $U,V,Z$ be Banach spaces and let $F:U\times V\times Z\longrightarrow U$ a continuous mapping satisfying

|F(u_{1},v,z)-F(u_{2},v,z)|\leq\alpha|u_{1}-u_{2}|,

for some $\alpha\in[0,1)$ and every $u_{1},u_{2}\in U$ , $v\in V$ , $z\in Z$ . Let $\phi(v,z)$ denote the unique fixed point of the mapping $F(\cdot,v,z):U\longrightarrow U$ . Then $\phi:V\times Z\longrightarrow U$ is continuous. If, in addition $F\in\mathcal{G}^{1,1,0}(U\times V\times Z,U)$ , then $\phi\in\mathcal{G}^{1,0}(V\times Z,U)$ and

\nabla_{v}\phi(v,z)=\nabla_{u}F\big{(}\phi(v,z),v,z\big{)}\nabla_{v}\phi(v,z)+\nabla_{v}F(\phi(v,z),v,z).

2.2. Lift approach to optimal control

In the sequel, we exploit an infinite dimensional lift to reformulate the optimization problem (1.1)-(1.2) in an infinite dimensional setting. Our first step is to rewrite $X^{u}$ in (1.2) in terms of a process $\mathcal{Z}^{u}$ with values in a Banach space, using the lift procedure presented in [9]. Notice that we do not actually work in the affine framework of [9], but the approach presented here is actually a particular case of the one introduced in [7].

Definition 2.6.

Let $Y$ be a Banach space with dual $Y^{*}$ and denote with $\langle\cdot,\cdot\rangle_{Y\times Y^{*}}$ the pairing between $Y$ and $Y^{*}$ . We say that a kernel $K\in L^{2}_{loc}(\mathbb{R}_{+},\mathbb{R})$ is liftable if there exist $g\in Y$ , $\nu\in Y^{*}$ and a uniformly continuous semigroup $\mathcal{S}_{t}^{*}$ , $t\in[0,T]$ with generator $\mathcal{A}^{*}$ , acting on $Y^{*}$ , such that

•

$K(t)=\langle g,\mathcal{S}_{t}^{*}\nu\rangle_{Y\times Y^{*}}$
•

$\mathcal{S}_{t}^{*}\nu\in Y^{*}$ for all $t>0$
•

$\int_{0}^{t}\|\mathcal{S}_{s}^{*}\nu\|^{2}_{Y^{*}}ds<\infty$ for all $t>0$ .

For notational simplicity we write $\langle\cdot,\cdot\rangle$ for $\langle\cdot,\cdot\rangle_{Y\times Y^{*}}$ when no confusion arises.

From now on, we make the following assumption:

Hypothesis 2.7.

The kernel $K$ in (1.2) is liftable.

We rewrite $X^{u}$ as

	$\displaystyle X^{u}(\tau)$	$\displaystyle=x(\tau)+\int_{t}^{\tau}K(\tau-s)\Big{[}\beta(s,X^{u}(s))+\sigma(s,X^{u}(s))R(s,X^{u}(s),u(s))\Big{]}ds$
		$\displaystyle\quad+\int_{t}^{\tau}K(\tau-s)\sigma(s,X^{u}(s))dW(s)$
		$\displaystyle:=x(\tau)+\int_{t}^{\tau}K(\tau-s)dV^{u}(s),$

where

dV^{u}(s):=\Big{[}\beta(s,X^{u}(s))+\sigma(s,X^{u}(s))R(s,X^{u}(s),u(s))\Big{]}ds+\sigma(s,X^{u}(s))dW(s).

(2.1)

Defining $\zeta$ as an element in $Y^{*}$ such that $x(\tau)=:\langle g,\mathcal{S}_{\tau}^{*}\zeta\rangle$ we can now rewrite (1.2) as follows:

$\displaystyle X^{u}(\tau)$	$\displaystyle=x(\tau)+\int_{t}^{\tau}K(\tau-s)dV^{u}(s)$
	$\displaystyle=\langle g,\mathcal{S}_{\tau}^{}\zeta\rangle+\int_{t}^{\tau}\langle g,\mathcal{S}_{\tau-s}^{}\nu\rangle dV^{u}(s)$
	$\displaystyle=\Bigg{\langle}g,\mathcal{S}_{\tau}^{}\zeta+\int_{t}^{\tau}\mathcal{S}_{\tau-s}^{}\nu dV^{u}(s)\Bigg{\rangle}$
	$\displaystyle=:\langle g,\mathcal{Z}_{\tau}^{u}\rangle,$	(2.2)

where $\mathcal{Z}_{\tau}^{u}:=\mathcal{S}_{\tau}^{*}\zeta+\int_{t}^{\tau}\mathcal{S}_{\tau-s}^{*}\nu dV^{u}(s)$ . One can then check that $\mathcal{Z}_{\tau}^{u}$ follows the dynamics:

\mathcal{Z}_{\tau}^{u}=\mathcal{S}_{t}^{*}\zeta+\int_{t}^{\tau}\mathcal{A}^{*}\mathcal{Z}_{s}^{u}ds+\int_{t}^{\tau}\nu dV^{u}(s),

(2.3)

In fact, we have that

	$\displaystyle\int_{t}^{\tau}\mathcal{A}^{*}\mathcal{Z}^{u}_{s}ds$	$\displaystyle=\int_{t}^{\tau}\mathcal{A}^{}\left[\mathcal{S}_{s}^{}\zeta+\int_{t}^{s}\mathcal{S}^{*}_{s-v}\nu dV^{u}(v)\right]ds$
		$\displaystyle=e^{\mathcal{A}^{}\tau}\zeta-e^{\mathcal{A}^{}t}\zeta+\int_{t}^{\tau}\!\!\!\int_{v}^{\tau}\mathcal{A}^{}e^{\mathcal{A}^{}(s-v)}\nu dsdV^{u}(v)$
		$\displaystyle=e^{\mathcal{A}^{}\tau}\zeta-e^{\mathcal{A}^{}t}\zeta+\int_{t}^{\tau}e^{\mathcal{A}^{*}(\tau-v)}\nu dV^{u}(v)-\int_{t}^{\tau}\nu dV^{u}(v)$
		$\displaystyle=\mathcal{Z}_{\tau}^{u}-e^{\mathcal{A}^{*}t}\zeta-\int_{t}^{\tau}\nu dV^{u}(v),$

and, rearranging the terms we obtain (2.3).

Remark 2.8.

By defining $B(t,X^{u}(t),u(t)):=\beta(s,X^{u}(s))\sigma(s,X^{u}(s))R(s,X^{u}(s),u(s))$ , and exploiting (2.2), we actually get that the function $x(\tau)=\langle g,\mathcal{S}_{t}^{*}\zeta\rangle$ is given by the expression

	$\displaystyle x(\tau)$	$\displaystyle=\mathbb{E}\left[X^{u}(\tau)-\int_{t}^{\tau}K(\tau-s)B(s,X^{u}(s),u(s))ds\right]$
		$\displaystyle=\left\langle g,\mathbb{E}\left[\mathcal{Z}^{u}_{\tau}-\int_{t}^{\tau}\mathcal{S}_{\tau-s}^{*}\nu B(s,X^{u}(s),u(s))ds\right]\right\rangle$

Set $\mathcal{Z}_{\tau}^{u,g}:=\langle g,\mathcal{Z}_{\tau}^{u}\rangle=X^{u}(\tau)$ , and plug (2.1) into (2.3), then we can rewrite (2.3) in differential notation as

d\mathcal{Z}_{\tau}^{u}=\mathcal{A}^{*}\mathcal{Z}_{\tau}^{u}d{\tau}+\nu\Big{(}\beta({\tau},\mathcal{Z}_{\tau}^{u,g})d\tau+\sigma({\tau},\mathcal{Z}_{\tau}^{u,g})\big{[}R({\tau},\mathcal{Z}_{\tau}^{u,g},u_{\tau})d{\tau}+dW_{\tau}\big{]}\Big{)},

(2.4)

with $\mathcal{Z}^{u}_{t}=e^{\mathcal{A}^{*}t}\zeta:=\zeta_{t}$ . We also rewrite (2.3) as

d\mathcal{Z}_{\tau}^{u}=\mathcal{A}^{*}\mathcal{Z}_{\tau}^{u}d{\tau}+\nu\beta^{g}({\tau},\mathcal{Z}_{\tau}^{u})d\tau+\nu\sigma^{g}({\tau},\mathcal{Z}_{\tau}^{u})[R^{g}({\tau},\mathcal{Z}_{\tau}^{u},u_{\tau})d\tau+dW_{\tau}],

(2.5)

where

	$\displaystyle\beta^{g}(s,\mathcal{Z}_{s}^{u})$	$\displaystyle:=\beta(s,\mathcal{Z}_{s}^{u,g})=\beta(s,\langle g,Z_{s}^{u}\rangle)=\beta(s,X(s)),$
	$\displaystyle\sigma^{g}(s,\mathcal{Z}_{s}^{u})$	$\displaystyle:=\sigma(s,\mathcal{Z}_{s}^{u,g})=\sigma(s,\langle g,Z_{s}^{u}\rangle)=\sigma(s,X(s)),$
	$\displaystyle R^{g}(s,\mathcal{Z}_{s}^{u},u_{s})$	$\displaystyle:=R(s,\mathcal{Z}_{s}^{u,g},u_{s})=R(s,\langle g,Z_{s}^{u}\rangle,u_{s})=R(s,X(s),u_{s}).$

We are going to discuss existence and uniqueness results for equation (2.4) in Section 3.1.

Remark 2.9.

We point out that the term

\int_{0}^{\tau}\nu\sigma^{g}(s,\mathcal{Z}_{s})dW_{s},

(2.6)

in (2.5) can be regarded in two different ways. On the one hand, it can be seen as the element of $Y^{*}$ :

\nu\left(\int_{0}^{\tau}\sigma^{g}(s,\mathcal{Z}_{s})dW_{s}\right),

where the integration of $\int_{0}^{\tau}\sigma^{g}(s,\mathcal{Z}_{s})dW_{s}=\int_{0}^{\tau}\sigma(s,X(s))dW_{s}$ is done on $\mathbb{R}$ and then lifted to $Y^{*}$ by multiplying it by $\nu$ . On the other hand, by writing (2.6) as

\int_{0}^{\tau}\sigma^{g}(s,\mathcal{Z}_{s})d(\nu W_{s}),

we have that $\nu W_{s}$ can be considered as a cylindrical Wiener process on $\mathbb{H}^{\nu}:=\left\{\nu x,\ x\in\mathbb{R}\right\}$ , which is a Hilbert space with the scalar product $\langle\cdot,\cdot\rangle_{\mathbb{H}^{\nu}}:=\|\nu\|_{Y^{*}}\langle\cdot,\cdot\rangle_{\mathbb{R}}$ . In this case we also see that $\mathbb{H}^{\nu}\subsetneq Y^{*}$ .

In section 3.1 we are going to provide sufficient conditions that guarantee the existence of a solution of (2.3)-(2.4). Due to the nature of the lift and identification (2.2), this will, in turn, provide sufficient conditions also for the existence of a solution to (1.2).

Remark 2.10.

From [9, 8] we see that we could perform the lift also under weaker hypothesis, by taking a subspace $Z\subset Y$ with their relative duals $Y^{*}\subset Z^{*}$ such that:

•

$Z$ and $Y$ are Banach spaces $Z\subset Y$ and $Z$ embeds continuously into $Y$ .
•

The semigroup $\mathcal{S}^{*}$ with generator $\mathcal{A}^{*}$ acts in a strongly continuous way on $Y^{*}$ and $Z^{*}$ with respect to the respective norm topologies.
•

The map $\mathcal{Z}\longmapsto\mathcal{S}^{*}_{t}\mathcal{Z}$ is weak-* continuous on $Y^{*}$ and on $Z^{*}$ for every $t\geq 0$ .
•

The pre-adjoint operator of $\mathcal{A}^{*}$ , generates a strongly continuous semigroup on $Z$ with respect to the respective norm topology (but not necessarily on $Y$ ).

In this case every kernel of the form $K(t)=\langle g,\mathcal{S}_{t}^{*}\nu\rangle$ with $\nu\in Z^{*}$ and $S_{t}^{*}\nu\in Y^{*}$ is liftable. While this setting would allow to work with a wider class of kernels, we would not be able to formulate the HJB equations. This is due to the fact that, when considering a kernel $K(t)=\langle g,\mathcal{S}_{t}^{*}\nu\rangle$ with $\nu\in Z^{*}$ , some of the inner products in the definition of the Hamilton-Jacobi-Bellman equation (3.16), would not be well defined. Being the goal of this work a control problem, we restrict ourselves to the case $\nu\in Y^{*}$ , as originally stated.

In a similar fashion to what we did for (1.2), recalling that $X^{u}(\tau):=\mathcal{Z}_{\tau}^{u,g}$ , we rewrite the performance functional (1.1) so to make its dependence from the lifted process $\mathcal{Z}^{u}_{\tau}$ explicit:

	$\displaystyle J(t,x,u)$	$\displaystyle=\mathbb{E}\left[\int_{t}^{T}F(\tau,\mathcal{Z}^{u,g}_{\tau},u_{\tau})d\tau+G(\mathcal{Z}^{u,g}_{T})\right]$
		$\displaystyle:=\mathbb{E}\left[\int_{t}^{T}F^{g}(\tau,\mathcal{Z}_{\tau}^{u},u_{\tau})d\tau+G^{g}(\mathcal{Z}_{T}^{u})\right]:=J^{g}(t,\zeta,u),$		(2.7)

where the functions $F:[0,T]\times\mathbb{R}\times\mathcal{U}\longrightarrow\mathbb{R}$ , $G:\mathbb{R}\longrightarrow\mathbb{R}$ are lifted to the functions

	$\displaystyle F^{g}$	$\displaystyle:[0,T]\times Y^{*}\times\mathcal{U}\longrightarrow\mathbb{R},$
	$\displaystyle G^{g}$	$\displaystyle:Y^{*}\longrightarrow\mathbb{R}$

where $Y^{*}$ is the Banach space associated to the liftable kernel, see Definition 2.6. The lifted maps $F^{g}$ and $G^{g}$ are

	$\displaystyle F^{g}(\cdot,\mathcal{Z}^{u}_{\tau},\cdot)$	$\displaystyle:=F(\cdot,\mathcal{Z}^{u,g}_{\tau},\cdot)=F(\cdot,\langle g,\mathcal{Z}^{u}_{\tau}\rangle,\cdot)=F(\cdot,X^{u}(\tau),\cdot),$
	$\displaystyle G^{g}(\mathcal{Z}^{u}_{\tau})$	$\displaystyle:=G(\mathcal{Z}^{u,g}_{\tau})=G(\langle g,\mathcal{Z}^{u}_{\tau}\rangle)=G(X^{u}(\tau)).$

The stochastic optimal control problem (1.1)-(1.3) is then lifted to

J^{g}(t,\zeta,\hat{u})=\inf_{u\in\mathds{A}}J^{g}(t,\zeta,u)=\inf_{u\in\mathds{A}}\mathbb{E}\left[\int_{t}^{T}F^{g}(\tau,\mathcal{Z}_{\tau}^{u},u)d\tau+G^{g}(\mathcal{Z}_{T}^{u})\right],

(2.8)

where the process $\mathcal{Z}^{u}$ takes values in the Banach space $Y^{*}$ , and where the dynamics for the controlled process are given by (2.3)-(2.4). Notice that, while the performance functional has not changed, we write $J^{g}$ instead of $J$ in order to highlight the dependence on $\mathcal{Z}^{u}_{t}$ instead of $X^{u}(t)$ , as underneath there is a passage from finite to infinite dimensions. Indeed, this change of notation embodies a crucial change of framework from a finite to an infinite dimensional setting, allowing us to move from functions $\beta$ , $\sigma$ , $R$ , $F$ and $G$ taking values from $\mathbb{R}$ to new functions $\beta^{g}$ , $\sigma^{g}$ , $R^{g}$ , $F^{g}$ and $G^{g}$ that now take values from $Y^{*}$ . This lift allows us to consider a new optimization problem, written on a space which is not the original one. Nonetheless, we have that $J(t,x,u)=J^{g}(t,\zeta,u)$ for $t\in[0,T]$ , $u\in\mathcal{U}$ . Also, being $g$ fixed and only depending on the kernel representation, finding the pair $(\hat{u},\mathcal{Z}^{\hat{u}})$ that minimizes (2.5)-(2.8) is equivalent to finding the pair $(\hat{u},X^{\hat{u}})$ that solves (1.1)-(1.2).

2.3. UMD Banach spaces

In the sequel we use techniques of Malliavin calculus on the space $Y^{*}$ . For this, we assume:

Hypothesis 2.11.

The space $Y^{*}$ is a unconditional martingale differences (UMD) Banach space.

For convenience we report here below the essentials on UMD Banach spaces.

Definition 2.12.

Let $(M_{n})_{n=1}^{N}$ be a Banach-space valued martingale, the sequence $d_{n}=M_{n+1}-M_{n}$ is called the martingale difference sequence associated with $(M_{n})_{n=1}^{N}$ . A Banach space $E$ is said to be a $UMD_{p}$ $(1<p<\infty)$ , space if there exists a constant $\beta$ such that for all $E$ -valued $L^{p}$ -martingale difference sequences $(d_{n})_{n=1}^{N}$ we have

\mathbb{E}\left\|\sum_{n=1}^{N}\epsilon_{n}d_{n}\right\|^{p}\leq\beta^{p}\mathbb{E}\left\|\sum_{n=1}^{N}d_{n}\right\|^{p},

where $\epsilon_{n}\in\mathbb{R}$ for all $n$ and $|\epsilon_{n}|=1.$ Thanks to [26] we also know that, if a Banach space $E$ is $UMD_{p}$ for some $1<p<\infty$ , then $E$ is a $UMD_{p}$ Banach space for all $p\in(1,\infty)$ , and we simply call it a UMD Banach space.

In the context of stochastic analysis in Banach spaces, martingale difference sequences provide a substitute for orthogonal sequences. In the following parts, we will see that this hypothesis is not very restrictive, as the UMD Banach spaces include all Hilbert spaces, $L^{q}$ spaces for $q\in(1,\infty)$ , reflective Sobolev spaces and many others, thus allowing us to consider a wide class of liftable kernels. In our framework, the process $\mathcal{Z}^{u}$ takes values in a UMD Banach space whenever we consider, for example, a shift operator or a quasi-exponential kernel or a kernel that can be expressed as the Laplace transform of a measure with density in $L^{q}([0,\infty))$ , $q\in(1,\infty)$ .

Assuming that $Y^{*}$ is UMD, allows us to define the Malliavin derivative operator $D$ on $L^{p}(\Omega,Y^{*})$ . From [23, Proposition 2.5], we know that $D$ is a closed operator and we denote with $\mathbb{D}^{1,p}(Y^{*})$ the closure of the domain.

For the results on UMD Banach spaces exploited in the following parts, we refer to [28] for the BDG inequality, [27] for the Fubini Theorem and to [23] for general Malliavin calculus results. In this framework we will also use a Clark-Okone formula for UMD Banach spaces (see [21]) and the following chain rule linking the Malliavin derivative and the Gâteaux derivative (see [23])

Proposition 2.13.

Let $E$ be a UMD Banach space and let $p\in(1,\infty)$ . Suppose that $\varphi\in\mathcal{G}^{1}(E,E)$ . If $F\in\mathbb{D}^{1,p}(E)$ , then $\varphi(F)\in\mathbb{D}^{1,p}(E)$ with

D(\varphi(F))=\nabla\varphi(F)DF.

3. The optimal control problem

We are now interested in solving the lifted optimal control problem (2.8), where the process $\mathcal{Z}$ follows the controlled dynamics given by

\begin{cases}d\mathcal{Z}^{u}_{\tau}=&\mathcal{A}^{*}\mathcal{Z}_{\tau}^{u}d\tau+\nu\beta^{g}(\tau,\mathcal{Z}^{u}_{\tau})d\tau+\nu\sigma^{g}(\tau,\mathcal{Z}^{u}_{\tau})R^{g}(\tau,\mathcal{Z}^{u}_{\tau},u_{\tau})d\tau\\ &+\nu\sigma^{g}(\tau,\mathcal{Z}^{u}_{\tau})dW_{\tau},\\ \mathcal{Z}^{u}_{t}=&\zeta_{t}.\end{cases}

(3.1)

For our results to hold, we add some Hypothesis on $R$ , which directly translates into hypothesis on $R^{g}$ .

Hypothesis 3.1.

$R:[0,T]\times\mathbb{R}\times\mathcal{U}\longrightarrow\mathbb{R}$ is measurable and $\|R(\tau,x,u)\|_{\mathbb{R}}\leq K_{R}$ for a suitable positive constant $K_{R}>0$ and every $\tau\in[0,T]$ , $x\in\mathbb{R}$ , $u\in\mathcal{U}$ .

In order to find the optimal value $J(\hat{u})$ , we associate the following partially coupled system of forward-backward equations

\begin{cases}d\mathcal{Z}_{\tau}&=\mathcal{A}^{*}\mathcal{Z}_{\tau}d\tau+\nu\beta^{g}(\tau,\mathcal{Z}_{\tau})d\tau+\nu\sigma^{g}(\tau,\mathcal{Z}_{\tau})dW_{\tau},\ \ \tau\in[t,T],\\ \mathcal{Z}_{t}&=\zeta_{t},\\ dp_{\tau}&=-\mathcal{H}(\tau,\mathcal{Z}_{\tau},q_{\tau})d\tau+q_{\tau}\nu dW_{\tau},\quad\tau\in[t,T],\\ p_{T}&=G(\mathcal{Z}_{T}),\end{cases}

(3.2)

to (3.1). Here above $\mathcal{H}:[0,T]\times Y^{*}\times Y^{**}\longrightarrow\mathbb{R}$ is the Hamiltonian function defined as

\mathcal{H}(t,z,\xi)=\inf_{u\in\mathcal{U}}\left[F^{g}(t,z,u)+\xi\nu R^{g}(t,z,u)\right].

(3.3)

Notice that the control $u$ only appears in the Hamiltonian functional. The solution of the backward equation is denoted by $(p,q)$ . We often write $(p_{\tau},q_{\tau})=(p(\tau,t,\zeta_{t}),q(\tau,t,\zeta_{t}))$ , $\tau\in[t,T]$ when we want to emphasize the dependence of $p$ and $q$ on the parameter $\zeta_{t}$ at time $t$ . Analogously, when we want to emphasize the dependence of $\mathcal{Z}$ on the initial value $\zeta_{t}$ at time $t$ , we write $\mathcal{Z}_{\tau}=\mathcal{Z}(\tau,t,\zeta_{t})$ .

Define now

v(t,z):=p(t,t,z),

(3.4)

with $p$ the solution to the backward SDE in (3.2). In the sequel we show that $J^{g}(t,\zeta_{t},\hat{u})$ in (2.8) is such that

J^{g}(t,\zeta_{t},\hat{u})=v(t,\zeta_{t})

(3.5)

and that the optimal control $\hat{u}$ can be retrieved explicitly via a verification theorem once $v(t,\zeta_{t})$ is known. In order to achieve (3.5) we proceed as follow. First we study the forward equation $\mathcal{Z}(\tau,t,\zeta_{t})$ in Section 3.1, then we study the backward equation $(p(\tau,t,\zeta_{t}),q(\tau,t,\zeta_{t}))$ in Section 3.2 and there we prove the crucial identification:

q_{t}=\nabla_{z}v(t,\mathcal{Z}_{t})\nu\sigma^{g}(t,\mathcal{Z}_{t}),

(3.6)

(see Proposition 3.11). In Section 3.3 we provide an approach to find $v(t,z)$ through HJB equations and at last, in Section 3.4, we prove (3.5) and we provide a characterization of the optimal control $\hat{u}$ .

Notice that, for (3.6) to hold, the backward process $p(\cdot,t,z)$ has to be differentiable with respect to $z$ . This can be obtained by showing that $\mathcal{Z}_{t}^{u}$ is differentiable with respect to the initial condition $\zeta_{t}$ , and by assuming the following:

Hypothesis 3.2.

Let us assume that

1)

There exists $L_{1}>0$ such that

$|\mathcal{H}(t,z,\xi_{1})-\mathcal{H}(t,z,\xi_{2})|\leq L_{1}\|\xi_{1}-\xi_{2}\|_{Y^{**}}$

for every $t\in[0,T]$ , $z\in Y^{*}$ and $\xi_{1},\xi_{2}\in Y^{**}$ .
2)

For all $t\in[0,T]$ , $\mathbb{E}\left[\int_{t}^{T}|\mathcal{H}(s,0,0)|^{2}ds\right]<\infty$ .
3)

For every $t\in[0,T]$ we have $\mathcal{H}(t,\cdot,\cdot)\in\mathcal{G}^{1,1}(Y^{*}\times Y^{**})$ .
4)

There exist $L_{2}>0$ and $m\geq 0$ such that

$|\nabla_{z}\mathcal{H}(t,z,\xi)h|\leq L_{2}\|h\|_{Y^{*}}(1+\|z\|_{Y^{*}})^{m}(1+\|\xi\|_{Y^{**}})$

for every $t\in[0,T]$ , $z,h\in Y^{*}$ and $\xi\in Y^{**}$ .
5)

$G^{g}\in\mathcal{G}^{1}(Y^{*})$ and there exists $L_{3}>0$ such that, for every $z_{1},z_{2}\in Y^{*}$

$|G^{g}(z_{1})-G^{g}(z_{2})|\leq L_{3}\|z_{1}-z_{2}\|_{Y^{*}}.$

Further details on the continuous dependence on $\zeta_{t}$ of the forward equation can be found in Section 3.1, while we refer to Section 3.2 for details on the differentiability of $p(\cdot,t,z)$ with respect to $z$ .

Remark 3.3.

The identification (3.6), in the case where $\sigma^{g}$ does not depend on $\mathcal{Z}$ in (3.2), can be proven following [22] and dropping the UMD hypothesis on $Y^{*}$ . In our case though, being $\sigma^{g}$ dependent on $\mathcal{Z}_{t}$ , we need to exploit some Malliavin calculus techniques on Banach spaces, and thus assume that $Y^{*}$ is UMD.

3.1. On the lifted forward equation

In this section we study the lifted forward equation $\mathcal{Z}_{t}$ in (3.2). In particular, we prove that it admits a unique Markovian solution and we study its continuous dependence from the initial parameter $\zeta_{t}$ . We thus take

\begin{cases}d\mathcal{Z}_{\tau}&=\mathcal{A}^{*}\mathcal{Z}_{\tau}d\tau+\nu\beta^{g}(\tau,\mathcal{Z}_{\tau})d\tau+\nu\sigma^{g}(\tau,\mathcal{Z}_{\tau})dW_{\tau},\quad\tau\in[t,T],\\ \mathcal{Z}_{t}&=\zeta_{t},\end{cases}

(3.7)

we recall that $\mathcal{A}^{*}$ is the generator of a uniformly continuous semigroup on the Banach space $Y^{*}.$ We assume the following:

Hypothesis 3.4.

Suppose that

i)

$\beta^{g}:[0,T]\times Y^{*}\longrightarrow\mathbb{R}$ is continuous and, for all $t\in[0,T]$ $z_{1},z_{2}\in Y^{*}$ , there exists a constant $L_{1}>0$ such that

$|\nu\beta^{g}(t,z_{1})-\nu\beta^{g}(t,z_{2})|\leq L_{1}\|z_{1}-z_{2}\|_{Y^{*}},$

the map $\nu\beta^{g}:[0,T]\times Y^{*}\longrightarrow Y^{*}$ is measurable. Moreover, for all $t\in[0,T]$ and $z\in Y^{*}$ ,

$|\nu\beta^{g}(t,z)|^{2}\leq L_{2}(1+\|z\|_{Y^{*}}^{2}),$

for some constant $L_{2}>0$ .
ii)

$\sigma^{g}:[0,T]\times Y^{*}\longrightarrow\mathbb{R}$ is such that, for every $v\in Y^{**}$ the map $\nu\sigma^{g}v:[0,T]\times Y^{*}\longrightarrow\mathbb{R}$ is measurable, $e^{s\mathcal{A}^{*}}\nu\sigma^{g}(t,z)\in L^{2}(Y^{*})$ for every $s>0$ , $t\in[0,T]$ and $z\in Y^{*}$ , and

$\|\nu\sigma^{g}(t,z)\|^{2}_{L(Y^{*})}\leq L_{3}(1+\|z\|_{Y^{*}}^{2}),$

for some constant $L_{3}>0$ .

Moreover, for $s>0$ , $t\in[0,T]$ , $z_{1},z_{2}\in Y^{*}$ there exists a constant $L_{4}>0$ such that

$\|\nu\sigma^{g}(t,\zeta_{1})-\nu\sigma^{g}(t,z_{2})\|_{L^{2}(Y^{*})}\leq L_{4}\|z_{1}-z_{2}\|_{Y^{*}},$
iii)

For every $s>0$ , $t\in[0,T]$ , $\nu\beta^{g}(t,\cdot)\in\mathcal{G}^{1}(Y^{*},Y^{*})$ .

Our first result is the following:

Proposition 3.5.

Assume Hypothesis 3.4 holds. For every $p\in[2,\infty)$ , we have that:

i)

The map $(t,z)\longmapsto\mathcal{Z}(\cdot,t,z)$ is in $\mathcal{G}^{0,1}([0,T]\times Y^{*};L^{p}(\Omega;C([0,T];Y^{*})))$ .

ii)

For every $h\in Y^{*}$ the partial directional derivative process $\nabla_{z}\mathcal{Z}(\tau,t,z)h$ , $\tau\in[0,T]$ solves $\mathbb{P}$ -a.s. the equation

	$\displaystyle\nabla_{z}\mathcal{Z}(\tau,t,z)h$	$\displaystyle=e^{(\tau-t)\mathcal{A}^{}}h+\int_{t}^{\tau}e^{(\tau-s)\mathcal{A}^{}}\nabla_{z}\nu\beta^{g}(s,\mathcal{Z}(s,t,z))\nabla_{z}\mathcal{Z}(s,t,z)h\ ds$
		$\displaystyle+\int_{t}^{\tau}\nabla_{z}\left(e^{(\tau-s)\mathcal{A}^{*}}\nu\sigma^{g}(s,\mathcal{Z}(s,t,z))\right)\nabla_{z}\mathcal{Z}(s,t,z)h\ dW_{s},\quad\tau\in[t,T],$
	$\displaystyle\nabla_{z}\mathcal{Z}(\tau,t,z,)h$	$\displaystyle=h,\quad\tau\in[0,t).$

iii)

$\|\nabla_{z}\mathcal{Z}(\tau,t,z)h\|_{L^{p}(\Omega;C([0,T];Y^{*}))}\leq c\|h\|_{Y^{*}}$ for some constant $c$ .

We also find that

iv)

(3.7) admits a unique adapted solution $\mathcal{Z}\in L^{p}(\Omega,C([t,T]);Y^{*})$ .

Moreover, we have the following estimate

\|\mathcal{Z}\|_{p}^{p}:=\mathbb{E}\left[\sup_{\tau\in[t,T]}\|\mathcal{Z}_{\tau}\|_{Y^{*}}^{p}\right]\leq C(1+\|\zeta_{t}\|_{Y^{*}}^{p}),

(3.8)

where $C$ is a constant depending only on $p,T,L$ , where $L:=\max\{L_{1},L_{2},L_{3},L_{4}\}$ and $M:=\sup_{\tau\in[t,T]}\|e^{\tau\mathcal{A}^{*}}\|_{L^{2}(Y^{*})}$ .

Proof.

The proof is inspired by [10, Theorem 7.4] and [14, Proposition 3.2] The main difference with our work are the spaces at play. Consider the map

\Phi(\mathcal{Z},t,z)_{\tau}:L^{p}(\Omega;C([0,T];Y^{*}))\times[0,T]\times Y^{*}\longrightarrow L^{p}(\Omega;C([0,T];Y^{*}))

defined as

	$\displaystyle\Phi(\mathcal{Z},t,z)_{\tau}$	$\displaystyle:=e^{\mathcal{A}^{}(\tau-t)}z+\int_{0}^{\tau}\mathds{1}_{[t,T]}(s)e^{\mathcal{A}^{}(\tau-s)}\nu\beta^{g}(s,\mathcal{Z}_{s})ds$
		$\displaystyle+\int_{0}^{\tau}\mathds{1}_{[t,T]}(s)e^{\mathcal{A}^{*}(\tau-s)}\nu\sigma^{g}(s,\mathcal{Z}_{s})dW_{s}$
		$\displaystyle:=S_{0}(\mathcal{Z},t,z)_{\tau}+S_{1}(\mathcal{Z},t,z)_{\tau}+S_{2}(\mathcal{Z},t,z)_{\tau}\quad\tau\in[0,T].$

We want to show that $\Phi$ is a contraction with respect to the first variable. We notice that

	$\displaystyle\\|S_{1}(\mathcal{Z},t,z)\\|^{p}$	$\displaystyle\leq M^{p}\mathbb{E}\left[\left(\int_{0}^{T}\\|\nu\beta^{g}(s,\mathcal{Z}_{s})\\|_{Y^{*}}ds\right)^{p}\right]$
		$\displaystyle\leq T^{p-1}M^{p}\mathbb{E}\int_{0}^{T}\mathbb{E}\left[\int_{0}^{T}\\|\nu\beta^{g}(s,\mathcal{Z}_{s})\\|_{Y^{*}}^{p}ds\right]$
		$\displaystyle\leq 2^{p/2-1}T^{p-1}M^{p}L^{p}\mathbb{E}\left[\int_{0}^{T}(1+\\|\mathcal{Z}_{s}\\|_{Y^{*}}^{p})ds\right]$
		$\displaystyle\leq 2^{p/2-1}(TLM)^{p}(1+\\|\mathcal{Z}\\|_{p}),$

and

	$\displaystyle\\|S_{2}(\mathcal{Z},t,z)\\|^{p}$	$\displaystyle\leq\sup_{\tau\in[0,T]}\mathbb{E}\left[\left\\|\int_{0}^{\tau}e^{(\tau-s)\mathcal{A}^{}}\nu\sigma^{g}(s,\mathcal{Z}_{s})dW(s)\right\\|_{Y^{}}^{p}\right]$
		$\displaystyle\leq M^{p}C_{p/2}LT^{p/2-1}2^{p/2-1}\mathbb{E}\left[\int_{0}^{T}(1+\\|\mathcal{Z}_{s}\\|^{p}_{Y^{*}})ds\right]$
		$\displaystyle\leq M^{p}C_{p/2}L(2T)^{p/2-1}(T+\\|\mathcal{Z}\\|_{p}),$

where we used the linear growth conditions on $\beta^{g}$ and $\sigma^{g}$ and the Burkholder-Davis-Gundy inequality for $UMD$ Banach spaces (see [28]). We thus have showed that $\Phi(\cdot,t,z)$ is a well defined mapping. Now, taking $\mathcal{Z}_{1}$ and $\mathcal{Z}_{2}$ arbitrary processes in $Y^{*}$ , then

	$\displaystyle\\|\Phi(\mathcal{Z}_{1},t,z)-\Phi(\mathcal{Z}_{2},t,z)\\|_{p}$	$\displaystyle\leq\\|S_{1}(\mathcal{Z}_{1},t,z)-S_{1}(\mathcal{Z}_{2},t,z)\\|_{p}+\\|S_{2}(\mathcal{Z}_{1},t,z)-S_{2}(\mathcal{Z}_{2},t,z)\\|_{p}$
		$\displaystyle:=I_{1}+I_{2}.$

With computations similar to the ones above, exploiting the Lipschitz condition on $\beta^{g}$ and $\sigma^{g}$ (see Hypothesis 3.4, i) and ii)), one finds that

I_{1}^{p}\leq(TML)^{p}\|\mathcal{Z}_{1}-\mathcal{Z}_{2}\|_{p}^{p},

and

I_{2}^{p}\leq C_{p/2}(ML)^{p}T^{p/2}\|\mathcal{Z}_{1}-\mathcal{Z}_{2}\|_{p}^{p}.

Summing up, we have that

\|\Phi(\mathcal{Z}_{1})-\Phi(\mathcal{Z}_{2})\|_{p}\leq LM(T^{p}+C_{p/2}T^{p/2})^{1/p}\|\mathcal{Z}_{1}-\mathcal{Z}_{2}\|_{p}.

This means that $\Phi(\mathcal{Z},t,z)$ is a contraction only for $t\in[0,T]$ when $T$ satisfies

LM(T^{p}+C_{p/2}T^{p/2})^{1/p}<1.

(3.9)

Condition (3.9) on $T$ can be easily removed by considering the equation on intervals $[0,\tilde{T}]$ , $[\tilde{T},2\tilde{T}]$ ,…, where $\tilde{T}$ satisfies (3.9). Thanks to the fixed point theorem we find that (2.5) admits a unique solution. We conclude that (3.8) holds by applying Gronwall’s Lemma with arguments in line with [10, Theorem 7.4 (iii)]. Notice now that, being $\Phi(\cdot,t,z)$ a contraction uniformly with respect to $t\in[0,T]$ , $z\in Y^{*}$ , by Proposition 2.5 we obtain $ii)$ if

\Phi\in\mathcal{G}^{1,0,1}\left(L^{p}(\Omega;C([0,T];Y^{*}))\times[0,T]\times Y^{*},L^{p}(\Omega;C([0,T];Y^{*}))\right).

This is verified by a (slight modification) of Lemma 2.4. Indeed we notice that $\Phi$ is differentiable in $z$ . For more details we refer to [10, 14]. ∎

Remark 3.6.

We notice that $\mathcal{Z}$ is Markovian (see e.g. [13, Theorem 1.157]).

Corollary 3.7.

Assume Hypothesis 3.1 and 3.4 hold. Then (3.1) admits a unique solution.

Proof.

Thanks to the boundedness of $R$ , one can apply the Girsanov Theorem, see e.g. [10, Theorem 10.14], and proceed like in the proof of Proposition (3.5). ∎

3.2. On the backward equation

In this section we study the backward equation

p_{\tau}=G^{g}(\mathcal{Z}_{T})+\int_{\tau}^{T}\mathcal{H}(s,\mathcal{Z}_{s},q_{s})ds-\int_{\tau}^{T}q_{s}\nu dW_{s},

(3.10)

introduced in (3.2). We study existence and uniqueness of a solution as well as its continuous dependence from the parameter $\zeta_{t}$ . Later on we will exploit (3.10) to prove (3.6), as well as show that the optimal value $J^{g}(t,z,\hat{u})$ for the optimization problem (2.5) - (2.8) is achieved for $J^{g}(t,\zeta_{t},\hat{u})=v(t,\zeta_{t})=p(t,t,\zeta_{t})$ .

We observe that the following a priori estimate for the pair process $(p,q)$ holds (see [22] and [14, Proposition 4.3] ):

\mathbb{E}\Bigg{[}\sup_{\tau\in[t,T]}|p_{\tau}|^{2}\Bigg{]}+\mathbb{E}\Bigg{[}\int_{t}^{T}\|q_{\tau}\|^{2}_{Y^{**}}d\tau\Bigg{]}\leq c\mathbb{E}\Bigg{[}\int_{t}^{T}|\mathcal{H}(\tau,0,0)|^{2}d\tau\Bigg{]}+c\mathbb{E}\Bigg{[}|G^{g}(\mathcal{Z}_{T})|^{2}\Bigg{]},

where $c$ is a constant depending on $T$ and $L:=\max\{L_{1},L_{2},L_{3}\}$ , where $L_{i}$ , $i=1,..,3$ are the coefficients in Hypothesis 3.2.

Proposition 3.8.

Assume that Hypotheses 3.2 and 3.4 hold true. Then (3.10) admits a unique solution $(p,q)\in L^{2}(\Omega,C[0,T];Y^{*})\times L^{2}(\Omega,L^{2}[0,T];L^{2}(Y^{*}))$ such that the map

z\longmapsto(p(\cdot,\cdot,z),q(\cdot,\cdot,z))\text{ \emph{ belongs to } }\mathcal{G}^{1}(L^{\eta}(\Omega;C([0,T];Y^{*})),\mathcal{K}_{cont}([0,T]))

for $\eta=\ell(m+1)(m+2)$ , where $\mathcal{K}_{cont}([0,T])$ is the space of adapted processes $(p,q)$ taking values in $\mathbb{R}\times Y^{**}$ such that $p$ has continuous paths and

\mathbb{E}\left[\sup_{\tau\in[0,T]}|p_{\tau}|^{2}\right]+\mathbb{E}\left[\int_{\tau}^{T}\|q_{s}\|_{Y^{**}}^{2}ds\right]<\infty.

Moreover, for every $\ell\geq 2$ .

\left(\mathbb{E}\left[\sup_{t\in[0,T]}|\nabla_{z}p(t,z)h|^{\ell}\right]\right)^{1/\ell}\leq C\|h\|_{Y^{*}}\left(1+\|z\|_{Y^{*}}^{(m+1)^{2}}\right)

Proof.

See [22] Proposition 4.2. ∎

Still aiming to prove (3.6), we provide yet another crucial result that links the directional derivative of $\mathcal{Z}$ to its Malliavin derivative.

Proposition 3.9.

Assume that Hypothesis 3.4 holds. Then for almost all $s,\tau$ such that $t\leq s\leq\tau<T$ we have that

D_{s}\mathcal{Z}(\tau,t,z)=\nabla_{z}\mathcal{Z}(\tau,s,\mathcal{Z}(s,t,z))\nu\sigma^{g}(s,\mathcal{Z}(s,t,z)),\ \mathbb{P}-a.s.

(3.11)

moreover

D_{s}\mathcal{Z}(T,t,z)=\nabla_{z}\mathcal{Z}(T,s,\mathcal{Z}(s,t,z))\nu\sigma^{g}(s,\mathcal{Z}(s,t,z)),\ \mathbb{P}-a.s.\text{ for almost all }s.

(3.12)

Proof.

Thanks to Proposition 3.5, for every $s\in[0,T]$ and every direction $h\in Y^{*}$ , the directional derivative process $\nabla_{z}\mathcal{Z}(\tau,s,z)h$ , $\tau\in[s,T)$ solves $\mathbb{P}$ -a.s. the equation

	$\displaystyle\nabla_{z}\mathcal{Z}(\tau,t,z)h$	$\displaystyle=e^{(\tau-t)\mathcal{A}^{}}h+\int_{t}^{\tau}e^{(\tau-s)\mathcal{A}^{}}\nabla_{z}\nu\beta^{g}(s,\mathcal{Z}(s,t,z))\nabla_{z}\mathcal{Z}(s,t,z)hds$
		$\displaystyle+\int_{t}^{\tau}\nabla_{z}\left(e^{(\tau-s)\mathcal{A}^{*}}\nu\sigma^{g}(s,\mathcal{Z}(s,t,z))\right)\nabla_{z}\mathcal{Z}(s,t,z)hdW_{s},\quad\tau\in[t,T],$
	$\displaystyle\nabla_{z}\mathcal{Z}(\tau,t,z)h$	$\displaystyle=h,\quad\tau\in[0,t),$

Given $v\in Y^{*}$ and $t\in[0,s]$ , we can replace $z$ by $\mathcal{Z}(s,t,z)$ and $h$ by $\nu\sigma^{g}(s,\mathcal{Z}(s,t,z))v$ in the previous equation, since $\mathcal{Z}(s,t,z)$ is $\mathcal{F}_{s}$ measurable. Note now that

\mathcal{Z}(\eta,s,\mathcal{Z}(s,t,z))=\mathcal{Z}(\eta,t,z)\quad\mathbb{P}-a.s.,

for $\eta\in[s,T)$ , as a consequence of the uniqueness of the solution of (3.7). This yields

	$\displaystyle\nabla_{z}$	$\displaystyle\mathcal{Z}(\tau,\mathcal{Z}(s,t,z))\nu\sigma^{g}(s,\mathcal{Z}(s,t,z))v=e^{(\tau-s)\mathcal{A}^{*}}\nu\sigma^{g}(s,\mathcal{Z}(s,t,z))v$
		$\displaystyle+\int_{s}^{\tau}e^{(\tau-\eta)\mathcal{A}^{*}}\nabla_{z}\nu\beta^{g}(\eta,\mathcal{Z}(\eta,t,z))\nabla_{z}\mathcal{Z}(\eta,s,\mathcal{Z}(s,t,z))\nu\sigma^{g}(s,\mathcal{Z}(s,t,z))vd\eta$
		$\displaystyle+\int_{s}^{\tau}\nabla_{z}(e^{(\tau-\eta)\mathcal{A}^{*}}\nu\sigma^{g}(\eta,\mathcal{Z}(\eta,t,z)))\nabla_{z}\mathcal{Z}(\eta,s,\mathcal{Z}(s,t,z))\nu\sigma^{g}(s,\mathcal{Z}(s,t,z))vdW_{\eta},$

for $\tau\in[s,T)$ , $\mathbb{P}$ -a.s. This shows that the process

\left\{\nabla_{z}\mathcal{Z}(\tau,t,\mathcal{Z}(s,t,z,))\nu\sigma^{g}(s,\mathcal{Z}(s,t,z))v\right\}_{t\leq s\leq\tau<T},

is a solution of the equation

	$\displaystyle Q_{s,\tau}$	$\displaystyle=e^{(\tau-s)\mathcal{A}^{}}\nu\sigma^{g}(s,\mathcal{Z}_{s})v+\int_{s}^{\tau}e^{(\tau-\eta)\mathcal{A}^{}}\nabla_{z}\nu\beta^{g}(\eta,\mathcal{Z}_{\eta})Q_{s,\eta}d\eta$
		$\displaystyle+\int_{s}^{\tau}\nabla_{z}(e^{(\tau-\eta)\mathcal{A}^{*}}\nu\sigma^{g}(\eta,\mathcal{Z}_{\eta}))Q_{s,\eta}dW_{\eta},$

where $Q_{s,\tau}:=D_{s}\mathcal{Z}_{\tau}v$ . The thesis now follows from the uniqueness property, as proved e.g. in [14] Proposition 3.5. To complete the proof of (3.12), we take a sequence $\tau_{n}\uparrow T$ such that (3.11) holds for every $\tau_{n}$ , and we let $n\rightarrow\infty$ (see [14]). The result follows from the regularity properties of $D\mathcal{Z}$ and $\nabla_{z}\mathcal{Z}$ , as well as the closedness of the operator $D$ on UMD Banach spaces. ∎

In this framework, using the results presented in [21] and [23] we find that:

Proposition 3.10.

Assume Hypotheses 3.2 - 3.4 Then for a.a. $s,\tau$ such that $t\leq s\leq\tau\leq T$ we have that

	$\displaystyle D_{s}p(\tau,t,z)$	$\displaystyle=\nabla_{z}p(\tau,s,\mathcal{Z}(s,t,z))\nu\sigma^{g}(s,\mathcal{Z}(s,t,z))\quad\mathbb{P}-a.s.,$		(3.13)
	$\displaystyle D_{s}q(\tau,t,z)$	$\displaystyle=\nabla_{z}q(\tau,s,\mathcal{Z}(s,t,z))\nu\sigma^{g}(s,\mathcal{Z}(s,t,z))\quad\mathbb{P}-a.s..$		(3.14)

Moreover, for a.a. $s\in[t,T]$ ,

q(s,t,z)=\nabla_{z}p(s,s,\mathcal{Z}(s,t,z))\nu\sigma^{g}(s,\mathcal{Z}(s,t,z))\ \mathbb{P}-a.s.

(3.15)

Proof.

The proof follows the same arguments as [14, Proposition 5.6] though the spaces at play are different. Indeed, the main tools are provided in Proposition 3.9. So, thanks to the extension of Malliavin calculus to UMD Banach spaces, and the chain rule linking Malliavin derivative and Gateaux derivative (see Proposition 2.13), the result is secured. ∎

Finally, the next result provides the proof of (3.6).

Proposition 3.11.

Assume that Hypothesis 3.2 and 3.4 hold true. Then the function $v(t,z):=p(t,t,z)$ in (3.4) is continuous and for every $t\in[0,T]$ , $v(t,\cdot)$ belongs to $\mathcal{G}^{1}(Y^{*},\mathbb{R})$ and there exists $C>0$ such that

|\nabla_{z}v(t,z)h|\leq C\|h\|_{Y^{*}}(1+\|z\|_{Y^{*}}^{(m+1)^{2}}).

Moreover we have that

q(s,t,z)=\nabla_{z}v(t,\mathcal{Z}(s,t,z))\nu\sigma^{g}(s,\mathcal{Z}(s,t,z)).

Proof.

The first part is a corollary of Proposition 3.8. The second is derived from (3.15). ∎

3.3. The HJB equation

Formally define

\displaystyle\mathcal{L}_{t}[f](x)

\displaystyle:=\frac{1}{2}\text{Trace}(G^{g}(t,x)G^{g}(t,x)^{*}\ \nabla^{2}f(x))+\langle\mathcal{A}^{*}x+\nu\beta^{g}(t,x),\nabla f(x)\rangle_{Y^{*}\times Y^{**}}.

We can consider the Hamilton-Jacobi-Bellman equation associated with the control problem (2.5) - (2.8), which is given by

\begin{cases}\frac{\partial w}{\partial t}(t,z)&=-\mathcal{L}_{t}w(t,z)-\mathcal{H}(t,z,\nabla w(t,z)\nu\sigma^{g}(t,z)),\\ w(T,z)&=G^{g}(z).\end{cases}

(3.16)

A solution of this equation provides a way to compute $v(t,z)$ in (3.4) by PDE methods (see e.g. [6]). The connection between (3.16) and (3.4) is actually detailed in the forthcoming Theorem 3.13 by means of the forward backward system (3.2). Later on, in Theorem 3.18, we shall see how $w(t,z)$ is connected with the optimal performance, see (3.5). Thus, we are interested in finding mild solutions to the previous equation, which we are going to defined soon. This problem was tackled in [14] (for a Hilbert space) in the case of a general $\nu\sigma^{g}$ , and in [22] (for a Banach space) in the case of a constant $\nu\sigma^{g}$ . Our result is then extending the on of [22].

Let $\mathcal{Z}(\tau,t,\zeta_{t})$ be a solution to (3.7), with $\mathcal{A}^{*}$ , $\beta^{g}$ and $\sigma^{g}$ satisfying Hypotheses 3.2 - 3.4. We recall that this solution is a $Y^{*}$ -valued Markov process (see Remark 3.6). We can then define the transition semigroup on continuous and bounded functions $\varphi:Y^{*}\longrightarrow\mathbb{R}$ as

P_{t,\tau}[\varphi](z)=\mathbb{E}\left[\varphi(\mathcal{Z}(\tau,t,z))\right].

Moreover, we have that this semigroup is also well defined on continuous functions $\varphi:Y^{*}\longrightarrow\mathbb{R}$ with polynomial growth with respect to $z$ .

Definition 3.12.

A function $w:[0,T]\times Y^{*}\longrightarrow\mathbb{R}$ is a mild solution of the Hamilton-Jacobi-Bellman equation (3.16) if:

•

For every $t\in[0,T]$ $w(t,\cdot)\in\mathcal{G}^{1}(Y^{*})$ , $w$ is continuous and $(t,z)\longmapsto w(t,z)$ is measurable from $[0,T]\times Y^{*}$ with values in $Y^{**}$
•

For every $t\in[0,T]$ , there exists $C>0$ such that $|w(t,z)|\leq C(1+\|z\|^{j}_{Y^{*}})$ and $|\nabla_{z}w(t,z)h|\leq C\|h\|_{Y^{*}}(1+\|z\|^{k}_{Y^{*}})$ , with $z,h\in Y^{*}$ and $j$ and $k$ positive integers.

•

The following equality holds.

w(t,z)=P_{t,T}[G^{g}](z)+\int_{t}^{T}P_{t,\tau}[\mathcal{H}(\tau,\cdot,\nabla w(\tau,\cdot)\nu\sigma^{g}(t,\cdot))](z)d\tau,\quad t\in[0,T],z\in Y^{*}.

In order to prove that there exists a unique solution of (3.16) we need once again the forward-backward system (3.2):

\begin{cases}d\mathcal{Z}_{\tau}&=\mathcal{A}^{*}\mathcal{Z}_{\tau}d\tau+\nu\beta^{g}(\tau,\mathcal{Z}_{\tau})+\nu\sigma^{g}(\tau,\mathcal{Z}_{\tau})dW_{\tau},\qquad\tau\in[t,T]\\ \mathcal{Z}_{t}&=\zeta_{t}\\ dp_{t}&=-\mathcal{H}(\tau,\mathcal{Z}_{\tau},q_{\tau})d\tau+q_{\tau}\nu dW_{\tau},\qquad\tau\in[t,T]\\ p_{T}&=G^{g}(\mathcal{Z}_{T})\end{cases}

Theorem 3.13.

Assume that $G^{g}$ and $\mathcal{H}$ satisfy Hypothesis 3.2 and that Hypothesis 3.4 hold true. Then there exists a unique mild solution of the Hamilton-Jacobi-Bellman equation (3.16) given by

w(t,\zeta_{t})=v(t,\zeta_{t})

(3.17)

where $(\mathcal{Z},p,q)$ is the solution of (3.2) and $v(t,z)=p(t,t,z)$ , see (3.4).

Proof.

The proof is based on arguments similar to [22, Theorem 6.2] and [14, Theorem 6.2], though adapted to the current framework. Note that the main difference with [14] is the nature of the spaces considered. ∎

3.4. Solving the optimal control problem

As we have proven the identification (3.6) to be true (see Proposition 3.11), we can finally move to the study of the optimal control problem (2.5) - (2.8). As anticipated, we want to show that the optimal value

\inf_{u\in\mathds{A}}J^{g}(t,\zeta_{t},u)=J^{g}(t,\zeta_{t},\hat{u})=v(t,\zeta_{t}),

where we have defined $v(t,\zeta_{t})=p(t,t,\zeta_{t})$ in (3.4), $(p,q)$ solve the backward stochastic differential equation (3.2) and a solution of $v(t,\zeta_{t})$ can be obtained through the HJB equation (3.16) (see Theorem 3.13). We define the, possibly empty, set

\Gamma(\tau,z,\xi)=\left\{u\in\mathcal{U}:F^{g}(\tau,z,u)+\xi\nu R^{g}(\tau,z,u)=\mathcal{H}(\tau,z,\xi)\right\},

(3.18)

where $\tau\in[t,T]$ , $z\in Y^{*}$ , $\xi\in Y^{**}$ .

Hypothesis 3.14.

We notice that, intuitively, $\Gamma(t,z,\xi)$ represents the set of cotrols that allow us to obtain the minimum in the Hamiltonian (3.3). We will thus assume that for all $\tau\in[t,T]$ , $z\in Y^{*}$ , $\xi\in Y^{**}$ , $\Gamma(\tau,z,\xi)\neq\emptyset$

Remark 3.15.

Thanks to the Filippov theorem (see [4]), being $\Gamma(\tau,z,\xi)$ non empty for all $\tau\in[t,T]$ , $z\in Y$ , $\xi\in Y^{**}$ , there exists a Borel measurable map $\Gamma_{0}:[0,T]\times Y^{*}\times Y^{**}\longrightarrow\mathcal{U}$ such that, for $t\in[0,T]$ , $z\in Y^{*}$ and $\xi\in Y^{**}$ , $\Gamma_{0}(t,z,\xi)\in\Gamma(t,z,\xi)$ .

Proposition 3.16.

Assume that Hypothesis 3.1 - 3.2 and 3.4 and hold true, and let $v$ be defined in (3.4) and $u$ in $\mathds{A}$ . Then for all $t\in[0,T]$ and $z\in Y^{*}$ , we have that

J^{g}(t,z,u)\geq v(t,z).

Proof.

Let $u$ in $\mathds{A}$ and take $\mathcal{Z}_{\tau}^{u}$ to be a solution of (3.1) corresponding to the control $u$ (for the existence of such a solution see Corollary 3.7). Define

W_{\tau}^{u}=W_{\tau}+\int_{t\vee\tau}^{\tau}R^{g}(s,\mathcal{Z}_{s}^{u},u_{s})ds,\quad\tau\in[0,T].

We notice that $\mathcal{Z}_{\tau}^{u}$ solves the equation

\begin{cases}d\mathcal{Z}^{u}_{\tau}=&\mathcal{A}^{*}\mathcal{Z}_{\tau}^{u}d\tau+\nu\beta^{g}(\tau,\mathcal{Z}^{u}_{\tau})d\tau+\nu\sigma^{g}(\tau,\mathcal{Z}^{u}_{\tau})dW^{u}_{\tau},\\ \mathcal{Z}_{t}=&\zeta_{t},\end{cases}

and, being $R$ and thus $R^{g}$ bounded, we can find a probability $\mathbb{P}^{u}$ equivalent to $\mathbb{P}$ such that $W^{u}$ is a Wiener process on $\mathbb{R}$ and thus $\nu W^{u}$ is a cylindrical Wiener process with values in $Y^{*}$ (see [10, Theorem 7.2 (iii)]). We consider the backward equation w.r.t. $\mathbb{P}^{u}$ for the unknowns $(p_{\tau}^{u},q_{\tau}^{u})$ , $\tau\in[t,T]$ given by

p_{\tau}^{u}+\int_{\tau}^{T}q_{s}^{u}\nu dW_{s}^{u}=G^{g}(\mathcal{Z}_{T}^{u})+\int_{\tau}^{T}\mathcal{H}(s,\mathcal{Z}_{s}^{u},q_{s}^{u})ds.

(3.19)

By taking $(p^{u},q^{u})$ at $\tau=t$ in (3.19), we get that $p(t,t,\zeta_{t})$ depends only on $t,\zeta_{t},\beta^{g},\sigma^{g},G^{g}$ and $\mathcal{H}$ . With the same approach as in [22, Proposition 5.5], one obtains immediately that $\int_{\tau}^{T}q_{s}^{u}\nu dW^{u}_{s}$ is actually a $\mathbb{P}^{u}$ -martingale.

By recalling that $v(t,z):=p(t,t,z)$ (see (3.4)) and taking expectation with respect to the original probability $\mathbb{P}$ , we obtain that

v(t,\zeta_{t})=\mathbb{E}[G^{g}(\mathcal{Z}_{T}^{u})]+\mathbb{E}\left[\int_{t}^{T}\mathcal{H}(s,\mathcal{Z}_{s}^{u},q_{s}^{u})-q_{s}^{u}\nu R(s,\mathcal{Z}_{s}^{u},u_{s})ds\right].

Adding and subtracting $\mathbb{E}\left[\int_{t}^{T}F^{g}(s,\mathcal{Z}_{s}^{u},u_{s})ds\right]$ , we arrive at

v(t,\zeta_{t})=J^{g}(t,\zeta_{t},u)+\mathbb{E}\left[\int_{t}^{T}\mathcal{H}(s,\mathcal{Z}_{s}^{u},q_{s}^{u})-q_{s}^{u}\nu R^{g}(s,\mathcal{Z}_{s}^{u},u_{s})-F^{g}(s,\mathcal{Z}_{s}^{u},u_{s})ds\right].

(3.20)

Noticing that

\mathcal{H}(s,\mathcal{Z}_{s}^{u},q_{s}^{u})-q_{s}^{u}\nu R^{g}(s,\mathcal{Z}_{s}^{u},u_{s})-F^{g}(s,\mathcal{Z}_{s}^{u},u_{s})\leq 0,

by definition of $\mathcal{H}$ in (3.3), we conclude. ∎

Corollary 3.17.

Let $t\in[0,T]$ and $\zeta_{t}\in Y^{*}$ . If $J^{g}(t,\zeta_{t},u^{*})=v(t,\zeta_{t})$ then $u^{*}$ is optimal for the control problem starting from $\zeta_{t}$ at time $t$ . Assume Hypothesis 3.14 holds and take $\Gamma_{0}(\tau,z,\xi)$ be the Borel measurable map defined in Remark 3.15. Then, an admissible control satisfying

\hat{u}_{\tau}=\Gamma_{0}(\tau,\mathcal{Z}_{\tau}^{\hat{u}},q_{\tau}^{\hat{u}}),\quad\mathbb{P}-a.s.\text{ for a.e. }\tau\in[t,T]

is optimal and $J^{g}(t,z,\hat{u})=v(t,z)$ .

Proof.

The proof is in line with the one in [22, Corollary 5.6]. ∎

Theorem 3.18.

Assume that Hypothesis 3.1, 3.2, 3.4 and 3.14 hold true. For all admissible controls $u$ in $\mathds{A}$ , we have that

{J^{g}}(t,\zeta_{t},u)\geq v(t,\zeta_{t}),

and the equality holds true if and only if

\hat{u}_{\tau}\in\Gamma(\tau,\mathcal{Z}^{u}_{\tau},\nabla v(\tau,\mathcal{Z}_{\tau}^{u})\nu\sigma^{g}(\tau,\mathcal{Z}^{u}_{\tau}))\quad\mathbb{P}-a.s.\text{ for a.a. }\tau\in[t,T].

(3.21)

Moreover, let us denote by $\Gamma_{0}(\tau,z,\xi)$ be the measurable selection of $\Gamma(t,z,\xi)$ defined in Remark 3.15. A control satisfying the feedback law, defined as:

u_{\tau}=\Gamma_{0}(\tau,\mathcal{Z}^{u}_{\tau},\nabla v(\tau,\mathcal{Z}^{u}_{\tau})\nu\sigma^{g}(\tau,\mathcal{Z}_{\tau}^{u}))\quad\mathbb{P}-a.s.\text{ for a.a. }\ \tau\in[t,T]

(3.22)

is optimal. Define the closed loop equation:

\begin{cases}\widetilde{\mathcal{Z}}_{\tau}=&\left[\mathcal{A}^{*}\widetilde{\mathcal{Z}}_{\tau}+\nu\beta^{g}(\tau,\widetilde{\mathcal{Z}}_{\tau})+\nu\sigma^{g}(\tau,\widetilde{\mathcal{Z}}_{\tau})R^{g}\Bigg{(}\tau,\widetilde{\mathcal{Z}}_{\tau},\Gamma_{0}\Big{(}\nabla v(\tau,\widetilde{\mathcal{Z}}_{\tau})\nu\sigma^{g}(\tau,\widetilde{\mathcal{Z}}_{\tau})\Big{)}\Bigg{)}\right]d\tau\\ &+\nu\sigma^{g}(\tau,\widetilde{\mathcal{Z}}_{\tau})dW_{\tau},\ \ \tau\in[t,T]\\ \widetilde{\mathcal{Z}}_{t}=&\zeta_{t}.\end{cases}

(3.23)

Then (3.23) admits a weak solution which is unique in law, and the corresponding pair $(u,\widetilde{\mathcal{Z}}^{u})$ is optimal.

For more details about the definition of feedback law and closed loop equation in the case of optimal control for Hilbert spaces we refer to [13, Section 2.5].

Proof.

Using (3.20) and Proposition 3.10 we can rewrite $v(t,\zeta_{t})$ as

	$\displaystyle v(t,\zeta_{t})=$	$\displaystyle J^{g}(t,\zeta_{t},u)+\int_{t}^{T}\left[\mathcal{H}(\tau,\mathcal{Z}^{u}_{\tau},\nabla v(\tau,\mathcal{Z}^{u}_{\tau})\nu\sigma^{g}(\tau,\mathcal{Z}^{u}_{\tau}))\right.$
		$\displaystyle\left.-\nabla v(\tau,\mathcal{Z}^{u}_{\tau})\nu\sigma^{g}(\tau,\mathcal{Z}^{u}_{\tau}))R^{g}(\tau,\mathcal{Z}_{t}^{u},u_{\tau})-F^{g}(\tau,\mathcal{Z}^{u}_{\tau},u_{\tau})\right]d\tau.$

The proof of the first statement now follows from Corollary 3.17. The closed loop equation can be solved in the weak sense via a Girsanov change of measure. Recall that $(\Omega,\mathcal{F},\mathbb{P})$ is the probability space on which the Wiener process $(W_{\tau})_{\tau\geq 0}$ in (3.2) is defined. Define $(\hat{W}_{\tau})_{\tau\geq 0}$ as

\hat{W}_{t}:=W_{t}-\int_{0}^{t}R^{g}\Bigg{(}\tau,\tilde{\mathcal{Z}}_{\tau}^{u},\Gamma_{0}\Big{(}\tau,\tilde{\mathcal{Z}}_{\tau},\nabla v(s,\tilde{\mathcal{Z}}_{\tau})\nu\sigma^{g}(\tau,\tilde{\mathcal{Z}}_{\tau})\Big{)}\Bigg{)}d\tau.

Due to the Girsanov theorem there exists a probability $\hat{\mathbb{P}}$ on $\Omega$ such taht $\hat{W}_{\tau}$ is a Wiener process. We then notice that $\nu W$ and $\nu\hat{W}$ are cylindrical Wiener processes with values in $Y^{*}$ , and that the closed loop equation (3.23) can be rewritten under $\hat{\mathbb{P}}$ as

\begin{cases}d\tilde{\mathcal{Z}}^{u}_{\tau}=&\mathcal{A}^{*}\tilde{\mathcal{Z}}_{\tau}^{u}d\tau+\nu\beta^{g}(\tau,\tilde{\mathcal{Z}}^{u}_{\tau})d\tau+\nu\sigma^{g}(\tau,\tilde{\mathcal{Z}}^{u}_{\tau})d\hat{W}_{\tau},\\ \mathcal{Z}_{t}=&\zeta_{t}.\end{cases}

Then, thanks to Proposition 3.5, we have a unique solution to this new process related to the probability $\hat{\mathbb{P}}$ and the Wiener process $\hat{W}_{\tau}$ , which implies that also the closed loop equation (3.23) always admits a solution in the weak sense. Thanks to Hypothesis 3.14, we know that $\Gamma(\tau,\mathcal{Z}_{\tau}^{u},\nabla v(\tau,\mathcal{Z}_{\tau}^{u})\nu\sigma^{g}(\tau,\mathcal{Z}_{\tau}^{u}))$ is non empty and thus, by the Filippov theorem, a measurable selection $\Gamma_{0}(\tau,\mathcal{Z}_{\tau}^{u},\nabla v(\tau,\mathcal{Z}_{\tau}^{u})\nu\sigma^{g}(\tau,\mathcal{Z}_{\tau}^{u}))$ of $\Gamma$ exists and the optimal control can be obtained. This proof is in line with [22, Theorem 5.7] and [14, Theorem 7.2]. ∎

Remark 3.19.

Notice that, having solved the lifted optimization problem, thanks to (2.2)-(2.7), we have also solved the original problem (1.1)-(1.2). Indeed, we have that a control $\hat{u}$ which is optimal for (2.8) where the dynamics for the forward process are given by $\mathcal{Z}^{u}_{\tau}$ in (3.1) is also optimal for the original problem (1.1), as $J^{g}(t,\zeta_{t},u)=J(t,x,u)$ by definition and $X^{u}(t)=\langle g,\mathcal{Z}_{t}^{u}\rangle$ .

4. A problem of optimal consumption

A cash flow admits consumption with rate $c$ according to the forward dynamics

X^{c}_{t}=x(t)+\int_{0}^{t}K(t-\tau)\ \mu(\tau,X^{c}_{\tau})d\tau+\int_{0}^{t}K(t-\tau)\ \sigma(\tau,X_{\tau}^{c})(-R(c_{\tau})d\tau+dW_{\tau}),

(4.1)

where $x:[0,T]\longrightarrow\mathbb{R}$ , $K(t-\tau)=\sqrt{t-\tau}$ , $\mu$ and $\sigma$ satisfy Hypothesis 3.4, and $R$ satisfy Hypothesis 3.1. In this case we lift $K(t):=\sqrt{t}$ on $L^{q}(\mathbb{R})\times L^{p}(\mathbb{R})$ for $q\in(1,2)$ and $p$ such that $\frac{1}{p}+\frac{1}{q}=1$ by considering

e^{\mathcal{A}^{*}t}:L^{p}(\mathbb{R})\longrightarrow L^{p}(\mathbb{R})

the left shift semigroup defined as $(e^{\mathcal{A}^{*}t}f)(s)=f(s-t)$ , for all $f\in L^{p}(\mathbb{R})$ , $g(x)=\frac{1}{2\sqrt{x}}\mathds{1}_{[0,1]}(x)$ , $\nu(x)=\mathds{1}_{[-1,0]}(x)$ . Then we have that

K(t)=\langle g,e^{\mathcal{A}^{*}t}\nu\rangle_{L^{p}(\mathbb{R})\times L^{q}(\mathbb{R})}=\int_{\mathbb{R}}\frac{1}{2\sqrt{x}}\mathds{1}_{[0,1]}(x)\mathds{1}_{[-1,0]}(x-t)dx=\int_{0}^{t}\frac{dx}{2\sqrt{x}}=\sqrt{t}.

We consider a classical optimal control problem given by the maximization of the performance functional

J(t,x,c)=\mathbb{E}\left[\int_{t}^{T}F(\tau,X_{\tau}^{c},c_{\tau})d\tau+G(X_{T}^{c})\right],

(4.2)

for some functions $F(\tau,X_{\tau}^{c},c_{\tau})=F(c_{\tau}):=-a_{1}c_{\tau}^{2}$ and $G(X_{t}^{c}):=a_{2}X_{T}^{c}$ ( $a_{1},a_{2}\in\mathbb{R}_{>0}$ ) satisfying Hypothesis 3.2. Linear-quadratic performance functionals such as (4.2) appear, for example, when considering optimal advertising problems (see e.g. [16, 17] and [15]). In this case we have that the stochastic control problem can be reformulated in $Y^{*}$ with forward dynamics given by

\mathcal{Z}_{\tau}^{c}=\zeta_{0}+\int_{0}^{\tau}\mathcal{A}^{*}\mathcal{Z}_{s}^{c}+\nu\mu^{g}(s,\mathcal{Z}_{s}^{c})ds+\nu\sigma^{g}(s,\mathcal{Z}_{s}^{c})(-R(c_{s})ds+dW_{s}),

(4.3)

with $\zeta_{t}=e^{\mathcal{A}^{*}t}\zeta$ such that $x(t)=\langle g,\zeta_{t}\rangle$ for all $t\in[0,T]$ . The goal is to minimize

J^{g}(0,\zeta_{0},c)=\mathbb{E}\left[\int_{0}^{T}-a_{1}c_{\tau}^{2}d\tau+a_{2}\langle g,\mathcal{Z}^{c}_{T}\rangle\right].

(4.4)

In this case the Hamiltonian functional (3.3) is given by

\mathcal{H}(t,\mathcal{Z},\xi)=\inf_{c\in\mathcal{U}}[-a_{1}c^{2}-\xi\nu R(c)]

(4.5)

and the forward-backward system is

\begin{cases}d\mathcal{Z}_{\tau}&=\mathcal{A}^{*}\mathcal{Z}_{\tau}d\tau+\nu\mu^{g}(s,\mathcal{Z}_{\tau})d\tau+\nu\sigma^{g}(s,\mathcal{Z}_{s})dW_{\tau},\quad\tau\in[0,T],\\ \mathcal{Z}_{0}&=\zeta_{0},\\ dp_{\tau}&=-\mathcal{H}(\tau,\mathcal{Z}_{\tau}^{c},q_{\tau})d\tau+q_{\tau}\nu dW_{\tau},\quad\tau\in[t,T],\\ p_{T}&=a_{2}\langle g,\mathcal{Z}_{T}\rangle.\end{cases}

(4.6)

In particular, using (4.5), we have that

p_{t}=a_{2}\langle g,\mathcal{Z}_{T}\rangle-\int_{t}^{T}\inf_{c\in\mathcal{U}}\left(-a_{1}c^{2}-q_{s}\nu R(c)\right)ds+q_{s}\nu dW_{s}\ s\in[t,T].

We thus get that the set $\Gamma$ defined in (3.18) is

\Gamma(t,\mathcal{Z},\xi)=\Big{\{}c\in\mathcal{U}:-a_{1}c^{2}-\xi\nu R(c)=\mathcal{H}(t,\mathcal{Z},c)\Big{\}},

(4.7)

and thus the optimal $u_{\tau}$ can be characterized by Theorem 3.18 as

c_{\tau}=\Gamma_{0}(\tau,\mathcal{Z}^{c}_{\tau},\nabla v(\tau,\mathcal{Z}_{\tau}^{c})\nu\sigma^{g}(\tau,\mathcal{Z}_{\tau}^{c}))\quad\ \mathbb{P}-a.s.\text{ for a.a. }\tau\in[0,T],

(4.8)

for a certain function $\Gamma_{0}$ such that $\Gamma_{0}(t,\mathcal{Z},\xi)\in\Gamma(t,\mathcal{Z},\xi)$ . In this case the HJB equations (3.16) become

\begin{cases}\frac{\partial v}{\partial t}(t,\zeta_{0})&=-\mathcal{L}_{t}v(t,\mathcal{Z}(t,0,\zeta_{0}))\\ &\quad-\inf_{c\in\mathcal{U}}[F^{g}(t,\mathcal{Z}(t,0,\zeta_{0}),c_{t})-\nabla v(t,\mathcal{Z}(t,0,\zeta_{0}))\nu\sigma(t,\mathcal{Z}(t,0,\zeta_{0}))c_{t}],\\ v(T,\zeta_{0})&=a_{2}X(0).\end{cases}

(4.9)

Where, we remind, $\mathcal{Z}(\tau,0,\zeta_{0})=\mathcal{Z}_{\tau}$ , $\tau\in[t,T]$ , $\mathcal{Z}(0)=\zeta_{0}$ and

	$\displaystyle\mathcal{L}_{t}[v](t,t,z)$	$\displaystyle:=\frac{1}{2}\text{Trace}(G^{g}(v(t,z)G^{g}(v(t,z))^{*}\ \nabla^{2}v(t,t,z))$
		$\displaystyle\quad+\langle\mathcal{A}^{*}v(t,t,z)+\nu\beta^{g}(t,v(t,t,z)),\nabla v(t,t,z)\rangle_{L^{q}([0,\infty))\times L^{p}([0,\infty))}.$

For more details about solving the HJB equation (4.9) we refer to [15]. Now, thanks to Theorem 3.13 we have that

Theorem 4.1.

Equation (4.9) has a unique mild solution $v$ . If the cost is given by (4.4), then for all admissible couples $(c,z)$ we have that $J(t,z,c)\geq v(t,z)$ , and the equality holds if and only if $\hat{c}\in\Gamma(t,\mathcal{Z},\xi)$ in (4.7), characterized as (4.8). Vice versa, if (4.8) holds, then

	$\displaystyle\tilde{\mathcal{Z}}_{\tau}$	$\displaystyle=\mathcal{A}^{*}\tilde{\mathcal{Z}}_{\tau}+\nu\mu^{g}(\tau,\mathcal{Z}_{\tau}^{c})d\tau$
		$\displaystyle\quad+\nu\sigma^{g}(\tau,\mathcal{Z}_{\tau}^{c})\Big{(}-R(\Gamma_{0}(\tau,\tilde{\mathcal{Z}}^{c}_{\tau},\nabla v(\tau,\tilde{\mathcal{Z}}_{\tau}^{c})\nu\sigma^{g}(\tau,\tilde{\mathcal{Z}}_{\tau}^{c})))d\tau+dW_{\tau}\Big{)},\quad\tau\in[0,T],$

with initial condition $\tilde{\mathcal{Z}}_{0}=\zeta_{0}$ , admits a weak solution, which is unique in law, and the corresponding pair $(c,\tilde{\mathcal{Z}}^{c})$ is optimal.

Remark 4.2.

This gives us a characterization of the optimal control for the lifted problem (4.4) and thus also for (4.2): in fact in our case we have that $X_{t}=\langle 1,\mathcal{Z}_{t}\rangle$ . Thanks to this we are able to find the optimal process $X^{\hat{u}}$ . Moreover, we note that, the optimal control for the lifted problem and the optimal control for the original problem coincide., This allows us to retrieve the optimal pair $(\hat{u},X^{\hat{u}})$ for the optimal control problem (4.2). Lastly we notice that the HJB equations (4.9), gives us the optimal value $J^{g}(0,\zeta_{0},\hat{u})$ , which in turn gives us the optimal value of $J(t,x,\hat{u})$ , thanks to (2.7).

Remark 4.3.

Inspired by [11], we could also have considered the kernel $K(t)=\frac{1}{t+\varepsilon}$ , $\varepsilon>0$ in (4.1). In this case we can take the space of $L^{p}([0,\infty))$ of $L^{p}$ functions on $[0,\infty)$ and its dual $L^{q}([0,\infty))$ of measures with density in $L^{q}$ , where $\frac{1}{p}+\frac{1}{q}=1$ and $p>1$ . We have that

K(t)=\frac{1}{t+\varepsilon}=\langle g,\mathcal{S}_{t}^{*}\nu\rangle_{L^{p}([0,\infty))\times L^{q}([0,\infty))},

where $g=\nu=e^{-x\varepsilon/2}$ , and $\mathcal{S}_{t}^{*}=e^{-tx}$ i.e. $K(t)$ is the Laplace trasnform of $e^{-x\varepsilon}$ . We notice that kernel is liftable (see Definition 2.6) and that we are in a UMD Banach space. It is clear that $e^{-\varepsilon x/2}$ is actually in $L^{p}([0,\infty))$ for all $p\geq 1$ . In particular we can also take $p=q=2$ and work on the Hilbert space $L^{2}([0,\infty))$ .

Acknowledgment

We would like to thank Anton Yurchenko-Tytarenko and Dennis Schroers for the nice input on kernel decompositions. The research leading to these results is within the project STORM: Stochastics for Time-Space Risk Models, receiving founding from the Research Council of Norway (RCN). Project number: 274410.

References

[1] E. Abi Jaber, E. Miller, and H. Pham. Linear-Quadratic control for a class of stochastic Volterra equations: solvability and approximation. The annals of probability, 31:2244–2274, 2021.
[2] N. Agram and B. Øksendal. Malliavin Calculus and Optimal Control of Stochastic Volterra Equations. Journal of Optimization Theory and Applications, 167:1070–1094, 2015.
[3] N. Agram, B. Øksendal, and S. Yakhlef. Optimal Control of Forward-Backward Stochastic Volterra Equations. Non-linear Partial Differential Equations, Mathematical Physics, and Stochastic Analysis: The Helge Holden Anniversary Volume, pages 3–36, 2009.
[4] J.P. Aubin and H. Frankowska. Set-valued analysis. Modern Birkhäuser Classics, 1990.
[5] S. Bonaccorsi and F. Confortola. Optimal control for stochastic Volterra equations with multiplicative Lévy noise. Nonlinear differential equations and applications, pages 1–26, 2020.
[6] P. Cannarsa and G. Da Prato. Second-order Hamilton–Jacobi equations in infinite dimensions. SIAM Journal on Control and Optimization, 29(2):474–492, 1991.
[7] C. Cuchiero and G. Di Nunno. Notes - Markovian lifts. to appear, 2019.
[8] C. Cuchiero and J. Teichman. Markovian lifts of positive semidefinite affine Volterra-type processes. Decisions in Economics and Finance, 42:407–448, 2019.
[9] C. Cuchiero and J. Teichman. Generalized Feller processes and Markovian lifts of stochastic Volterra processes: the affine case. Journal of Evolution Equations, pages 1–48, 2020.
[10] G. Da Prato and J. Zabczyk. Stochastic equations in infinite dimensions. Encyclopedia of Mathematics and its Applications. Cambridge University Press, 2 edition, 2014.
[11] G. Di Nunno, A. Fiacco, and E. Hove Karlsen. On the approximation of Lévy driven Volterra processes and their integrals. Journal of Mathematical Analysis and Applications, 476:120–148, 2019.
[12] G. Di Nunno and M. Giordano. Stochastic Volterra equations with time-changed Lévy noise and maximum principles. Annals of Operations Research, 2023.
[13] G. Fabbri, F. Gozzi, and A. Swiech. Stochastic Optimal Control in Infinite Dimension Dynamic Programming and HJB Equations. Springer, 2017.
[14] M. Fuhrman and G. Tessitore. Nonlinear Kolmogorov equations in infinite dimensional spaces: the backward stochastic differential equations approach and applications to optimal control. The Annals of Probability, 30:1397–1465, 2002.
[15] M. Giordano and A. Yurchenko-Tytarenko. Optimal control in linear stochastic advertising models with memory. ArXiv, 2021.
[16] F. Gozzi and C. Marinelli. Stochastic optimal control of delay equations arising in advertising models. In Stochastic Partial Differential Equations and Applications, Lecture Notes in Pure and Applied Mathematics, page 133–148. Chapman and Hall/CRC, 2005.
[17] F. Gozzi, C. Marinelli, and S. Savin. On controlled linear diffusions with delay in a model of optimal advertising under uncertainty with memory effects. Journal of optimization theory and applications, 142(2):291–321, 2009.
[18] C. Hernandez and D. Possamaï. A unified approach to well-posedness of type-I backward stochastic Volterra integral equations. ArXiv, 2020.
[19] T. Hytönen, J. Van Neerven, M. Veraar, and L. Weis. Analysis in Banach Spaces, volume I: Martingales and Littlewood-Paley Theory. Springer, Cham, 2016.
[20] C. Li and W. Zhen. Stochastic optimal control problem in advertising model with delay. Journal of Systems Science and Complexity, 33:968–987, 2020.
[21] J. Maas and J. Van Neerven. A Clark-Ocone formula in UMD Banach spaces. Electronic communications in probability, 2008.
[22] F. Masiero. Stochastic optimal control problems and parabolic equations in Banach spaces. SIAM Journal on Control and Optimization, 47:251–300, 2008.
[23] M. Pronk and M. Veraar. Tools for Malliavin calculus in UMD Banach spaces. Potential Analysis, 40:307–344, 2014.
[24] J. Prüss. Evolutionary integral equations and applications, volume 87. Birkhäuser, 2013.
[25] M. Saeedian, M. Khalighi, N. Azimi-Tafreshi, G.R. Jafari, and M. Ausloos. Memory effects on epidemic evolution: The susceptible-infected-recovered epidemic model. Physical Review E, 95(2):022409, 2017.
[26] J. Van Neerven. Stochastic evolution equations. Lecture notes, 2007.
[27] J. Van Neerven and M. Veraar. On the stochastic Fubini theorem in infinite dimensions. In In Stochastic partial differential equations and applications—VII, volume 245 of Lect, 2005.
[28] J. Van Neerven, M. Veraar, and L. Weis. Stochastic integration in UMD Banach spaces. The Annals of Probability, 35, 2007.
[29] J. Yong. Backward Stochastic Volterra Integral Equations and some Related Problems. Stochastic Processes and their Applications, 116:770–795, 2006.
[30] J. Yong. Well-Posedness and Regularity of Backward Stochastic Volterra Integral Equations. Probability Theory and Related Fields, 142:21–77, 2007.

	$\displaystyle\int_{t}^{\tau}\mathcal{A}^{*}\mathcal{Z}^{u}_{s}ds$	$\displaystyle=\int_{t}^{\tau}\mathcal{A}^{}\left[\mathcal{S}_{s}^{}\zeta+\int_{t}^{s}\mathcal{S}^{*}_{s-v}\nu dV^{u}(v)\right]ds$
		$\displaystyle=e^{\mathcal{A}^{}\tau}\zeta-e^{\mathcal{A}^{}t}\zeta+\int_{t}^{\tau}\!\!\!\int_{v}^{\tau}\mathcal{A}^{}e^{\mathcal{A}^{}(s-v)}\nu dsdV^{u}(v)$
		$\displaystyle=e^{\mathcal{A}^{}\tau}\zeta-e^{\mathcal{A}^{}t}\zeta+\int_{t}^{\tau}e^{\mathcal{A}^{*}(\tau-v)}\nu dV^{u}(v)-\int_{t}^{\tau}\nu dV^{u}(v)$
		$\displaystyle=\mathcal{Z}_{\tau}^{u}-e^{\mathcal{A}^{*}t}\zeta-\int_{t}^{\tau}\nu dV^{u}(v),$

	$\displaystyle\\|S_{1}(\mathcal{Z},t,z)\\|^{p}$	$\displaystyle\leq M^{p}\mathbb{E}\left[\left(\int_{0}^{T}\\|\nu\beta^{g}(s,\mathcal{Z}_{s})\\|_{Y^{*}}ds\right)^{p}\right]$
		$\displaystyle\leq T^{p-1}M^{p}\mathbb{E}\int_{0}^{T}\mathbb{E}\left[\int_{0}^{T}\\|\nu\beta^{g}(s,\mathcal{Z}_{s})\\|_{Y^{*}}^{p}ds\right]$
		$\displaystyle\leq 2^{p/2-1}T^{p-1}M^{p}L^{p}\mathbb{E}\left[\int_{0}^{T}(1+\\|\mathcal{Z}_{s}\\|_{Y^{*}}^{p})ds\right]$
		$\displaystyle\leq 2^{p/2-1}(TLM)^{p}(1+\\|\mathcal{Z}\\|_{p}),$

	$\displaystyle\\|S_{2}(\mathcal{Z},t,z)\\|^{p}$	$\displaystyle\leq\sup_{\tau\in[0,T]}\mathbb{E}\left[\left\\|\int_{0}^{\tau}e^{(\tau-s)\mathcal{A}^{}}\nu\sigma^{g}(s,\mathcal{Z}_{s})dW(s)\right\\|_{Y^{}}^{p}\right]$
		$\displaystyle\leq M^{p}C_{p/2}LT^{p/2-1}2^{p/2-1}\mathbb{E}\left[\int_{0}^{T}(1+\\|\mathcal{Z}_{s}\\|^{p}_{Y^{*}})ds\right]$
		$\displaystyle\leq M^{p}C_{p/2}L(2T)^{p/2-1}(T+\\|\mathcal{Z}\\|_{p}),$

Lifting of Volterra processes: optimal control in UMD Banach spaces

Abstract.

1. Introduction

2. Some preliminary results

2.1. The class of Gâteaux differentiable functions

Definition 2.1.

Remark 2.2.

Definition 2.3.

Lemma 2.4.

Proposition 2.5.

2.2. Lift approach to optimal control

Definition 2.6.

Hypothesis 2.7.

Remark 2.8.

Remark 2.9.

Remark 2.10.

2.3. UMD Banach spaces

Hypothesis 2.11.

Definition 2.12.

Proposition 2.13.

3. The optimal control problem

Hypothesis 3.1.

Hypothesis 3.2.

Remark 3.3.

3.1. On the lifted forward equation

Hypothesis 3.4.

Proposition 3.5.

Proof.

Remark 3.6.

Corollary 3.7.

Proof.

3.2. On the backward equation

Proposition 3.8.

Proof.

Proposition 3.9.

Proof.

Proposition 3.10.

Proof.

Proposition 3.11.

Proof.

3.3. The HJB equation

Definition 3.12.

Theorem 3.13.

Proof.

3.4. Solving the optimal control problem

Hypothesis 3.14.

Remark 3.15.

Proposition 3.16.

Proof.

Corollary 3.17.

Proof.

Theorem 3.18.

Proof.

Remark 3.19.

4. A problem of optimal consumption

Theorem 4.1.

Remark 4.2.

Remark 4.3.

Acknowledgment

References

Lifting of Volterra processes:
optimal control in UMD Banach spaces