On the convergences and applications of the inertial-like proximal point methods for null point problems

Let $A:H\rightarrow 2^{H}$ be a set-valued operator in real Hilbert space $H$.

(1)  The graph of $A$ is given by

$$gphA=\{(x,y)\in H\times H:y\in Ax\}.$$

(2)  The operator $A$ is said to be monotone if

$$\langle x-y,u-v\rangle\geq 0,u\in Ax,v\in Ay,\,\,\,\forall x,y\in H.$$

(3)  The operator $A$ is said to be maximal monotone if $gphA$ is not properly contained in the graph of any other monotone operator. Let $A,B$ be two maximal monotone operators in $H$. We are concern with the well-studied null point problem that is formulated as follows:

$\displaystyle 0\in(A+B)x^{*}$

(1.1)

with solution set denoted by $\Omega$. A special interesting case of (1.1) is a minimization of sum of two proper, lower semi-continuous and convex functions $f,g:H\rightarrow\mathbb{R}$, that is,

$\displaystyle\min_{x\in H}\{f(x)+g(x)\}.$

(1.2)

So this is equivalent to (1.1) with $A=\partial f$ and $B=\partial g$ being the subdifferentials of $f$ and $g$, respectively. We recall the resolvent operator $J_{r}^{A}=(I+rA)^{-1},r>0,$ which is called the backward operator and plays an significant role in the approximation theory for zero points of maximal monotone operators. Due to the work of Aoyama et al. [3], we have the following properties:

$\displaystyle\langle J_{r}^{A}x-y,x-J_{r}^{A}x\rangle\geq 0,y\in A^{-1}(0),$

(1.3)

where $A^{-1}(0)=\{z\in H:0\in Az\}$. Moreover, the following key facts that will be needed in the sequel.

Fact 1: The resolvent is not only always single-valued, but also firmly monotone:

$\displaystyle\|x-y\|^{2}-\|(I-J_{r}^{A})x-(I-J_{r}^{A})y\|^{2}\geq\|J_{r}^{A}x-J_{r}^{A}y\|^{2}.$

(1.4)

Fact 2: Using the resolvent operator, we can write down inclusion problem (1.1) as a fixed point problem. It is known that

$\displaystyle x^{*}=J_{\lambda}^{A}(I-\lambda{B})x^{*},\lambda>0.$

Douglas-Rachford‘s splitting method (DRSM) (or forward-backward splitting method) was first introduced in [10] as an operator splitting technique to solve partial differential equations arising in heat conduction and soon later has been extended to find solutions for the sum of two maximal monotone operators by Lions and Mercier [13]. Douglas-Rachford‘s splitting method(DRSM) is formulated as

$$x_{n+1}=J^{A}_{\lambda}(I-\lambda B)x_{n},$$

(1.5)

where $\lambda>0$, and $(I-\lambda B)$ is called the forward operator. Basing on the method (1.5), many researchers improved and modified the algorithms for the inclusion problem (1.1) and obtained nice results, see e.g., Boikanyo [7], Dadashi and Postolache[8], Kazmi and Rizvi [12], Moudafi[20][19], Sitthithakerngkiet et al. [27]. On the other hand, one classical way of looking at the null point problem $0\in Ax$ is to consider the forward discretization for $\frac{dx}{dt}\approx\frac{x_{n+1}-x_{n}}{h_{n}},\forall h_{n}>0$ in the evolution system:

$\displaystyle\begin{cases}\frac{dx}{dt}+Ax(t)\ni 0,\\ x(0)=x_{0},\end{cases}$

(1.6)

and then the evolution system is discretized as

$$\frac{x_{n+1}-x_{n}}{h_{n}}+Ax_{n+1}\ni 0\,\,\Longleftrightarrow\,\,(I+h_{n}F)x_{n+1}=x_{n},$$

which inspired Alvarez and Attouch [2] to introduce the inertial method for the following nonsmooth case of a nonlinear oscillator with damping

$\displaystyle\frac{d^{2}x}{dt^{2}}+\gamma\frac{dx}{dt}+A(x(t))=0,a.e.t\geq 0.$

(1.7)

By discretizing, Alvarez and Attouch [2] obtained the implicit iterative sequence

$\displaystyle x_{n+1}-x_{n}-\alpha_{n}(x_{n}-x_{n-1})+\lambda_{n}A(x_{n+1})\ni 0,$

(1.8)

where $\alpha_{n}=1-\gamma h_{n}$ and $\lambda_{n}=h_{n}^{2}$, which yielded the Inertial-Prox algorithm

$\displaystyle x_{n+1}=J_{\lambda_{n}}^{A}(x_{n}+\alpha_{n}(x_{n}-x_{n-1})),$

(1.9)

the extrapolation term $\alpha_{n}(x_{n}-x_{n-1})$ is called inertial term. Note that when $\alpha_{n}\equiv 0$, the recursion (1.8) corresponds to the standard proximal iteration

$\displaystyle x_{n+1}-x_{n}+\lambda_{n}A(x_{n+1})\ni 0,$

which has been well studied by Martinet [15]and Moreau[17] and other researchers, and the weak convergence of $x_{n}$ to a solution of $0\in Ax$ has being well known since the classical work of Rockafellar[25]. For ensuring the convergence of the Inertial-Prox sequence, Alvarez and Attouch[2] pointed the following key assumption: there exists $\alpha\in(0,1)$ such that $\alpha_{n}\in[0,\alpha]$ and

$\displaystyle\sum_{n=1}^{\infty}\alpha_{n}\|x_{n}-x_{n-1}\|^{2}<\infty.$

(1.10)

Inertial method is shown to have nice convergence properties in the field of continuous optimization and is studied intensively in split inverse problem by many authors soon later (because which could be utilized in some situations to accelerate the convergence of the sequences). For some recent works applied to various fields, see Alvarez [1], Attouch et al. [6, 5, 4], Ochs et al. [23, 22]. Although Alvarez and Attouch [2] pointed that one can choose appropriate rule to enable the the assumption (1.10) applicable, the parameter $\alpha_{n}$ involving the iterates $x_{n}$ and $x_{n-1}$ should be computed in advance. Namely, to make the condition (1.10) hold, the researchers have to set constraints on the inertia coefficient $\alpha_{n}$ and estimate the value of the $\|x_{n}-x_{n-1}\|$ before choosing $\alpha_{n}$. In recent works, Gibali et al. [11] improved the inertial control condition by constraining inertial coefficient $\alpha_{n}$ such that $0<\alpha_{n}<\bar{\alpha}_{n}$, where $\alpha\in(0,1),$ $\epsilon_{n}\in[0,\infty)$ and $\sum_{n=0}^{\infty}\epsilon_{n}<\infty$,

$\displaystyle\bar{\alpha}_{n}=\begin{cases}\min\{\alpha,\frac{\epsilon_{n}}{\|x_{n}-x_{n-1}\|^{2}}\},&\hbox{if}\,\,x_{n}\neq x_{n-1},\\ \alpha,&\text{otherwise}.\end{cases}$

More works on inertial methods, one can refer Dang et al. [9], Moudafi et al.[18], Suantai et al. [26], Tang [28], and therein. Theoretically, the condition (1.10) on the parameter $\alpha_{n}$ is too strict for the convergence of the inertial algorithm. Practically, estimating the value of the $\|x_{n}-x_{n-1}\|$ before choosing the inertial parameter $\alpha_{n}$ may need large amount of computation. The two drawbacks may make the Inertial-Prox method inconvenient in the practical test in the sense. So it is natural to think about the following question: Question 1.1 Can we delete the condition (1.10) in inertial method? Namely, can we construct a new inertial algorithm for solving (1.1) without any constraint on the the inertial parameter or the computation of norm of the difference between $x_{n}$ and $x_{n-1}$ before choosing the inertial parameter? The purpose of this paper is to present an affirmative answer to the above question. In this paper, we study the convergence problem of a new inertial-like technique for the solution of the null point problem (1.1) without the assumption (1.10) and the prior computation of the $\|x_{n}-x_{n-1}\|$ before choosing the inertial parameter $\theta_{n}$. The outline of the paper is as follows. In section 2, we collect definitions and results which are needed for our further analysis. In section 3, our novel approach for the null point problem is introduced and analyzed, the convergence theorems of the presented algorithms are obtained. Moreover, convex optimization and variational inequality problem are studied as the applications of the null point problem in section 4. Finally, in section 5,some numerical experiments, using the inertial-like method, are carried out in order to support our approach.

Let $\langle\cdot,\cdot\rangle$ and $\|\cdot\|$ be the inner product and the induced norm in a Hilbert space $H$, respectively. For a sequence $\{x_{n}\}$ in $H$, denote $x_{n}\rightarrow x$ and $x_{n}\rightharpoonup x$ by the strong and weak convergence to $x$ of $\{x_{n}\}$, respectively. Moreover, the symbol $\omega_{w}(x_{n})$ represents the $\omega$-weak limit set of $\{x_{n}\}$, that is,

$\displaystyle\omega_{w}(x_{n}):=\{x\in H:x_{n_{j}}\rightharpoonup x\hskip 5.69046pt\mathrm{for}\hskip 5.69046pt\mathrm{some}\hskip 5.69046pt\mathrm{subsequence}\hskip 5.69046pt\{x_{n_{j}}\}\hskip 5.69046pt\mathrm{of}\hskip 5.69046pt\{x_{n}\}\}.$

The identity below is useful:

	$\displaystyle\|\alpha x+\beta y+\gamma z\|^{2}$	$\displaystyle=\alpha\|x\|^{2}+\beta\|y\|^{2}+\gamma\|z\|^{2}$
		$\displaystyle\quad-\alpha\beta\|x-y\|^{2}-\beta\gamma\|y-z\|^{2}-\gamma\alpha\|x-z\|^{2}$		(2.1)

for all $x,y,z\in R$ and $\alpha+\beta+\gamma=1$.

The following significant characterization of the projection $P_{C}$ should be recalled : Given $x\in H$ and $y\in C$,

$\displaystyle P_{C}x=z\quad\Longleftrightarrow\quad\langle x-z,y-z\rangle\leq 0,\quad y\in C.$

(2.2)