Optimization Methods Rooting in Optimal Control

The research history of optimization problems is long and rich, spanning several centuries and evolving with the development of mathematics, engineering, operations research, and computer science. This originates from that optimization problems are widespread in the modeling of real-world systems for a very broad range of applications. Such applications include economies of scale, fixed charges, finance, allocation and location problems, structural optimization, engineering design, network and transportation problems, chip design and database problems, mechanical design, chemical engineering design and control, molecular biology, and several other combinatorial optimization problems such as integer programming and related graph problems. With the development and implementation of practical optimization algorithms, more and more scientists in diverse disciplines have been using optimization techniques to solve problems [1, 2, 3, 4].

Predominant optimization methods, such as the gradient descent [5] and Newton’s methods [6], are proposed based on the function approximation. Despite being powerful, favorable, and widely used optimization algorithms, they also have limitations and potential drawbacks. The learning rate is a crucial hyperparameter in gradient descent [7]. Gradient descent’s performance is susceptible to the learning rate (If the learning rate is too small, the algorithm may converge slowly, while if it is too large, the algorithm may overshoot the minimum and fail to converge [8]) while the hyperparameters tuning is challenging [9]. Newton’s method may diverge if the Hessian matrix is singular or the algorithm encounters numerical instability (when the Hessian matrix is ill-conditioned). Both gradient descent and Newton’s methods are sensitive to the choice of the initial value. Suppose the initial guess is far from the actual root or minimum. In this case, Newton’s method might converge to a different solution or fail to converge altogether and gradient descent may get stuck in local minima or saddle points. To mitigate some of the aforementioned issues, a body of variations and enhancements, such as stochastic gradient descent [10], momentum [11], adaptive learning rate methods [12, 13, 14], quasi-Newton method [15], inexact Newton method [16], and modified Newton method [17] have been developed. However, it still seems somewhat blind to tune parameters in diverse methods.

The paper is the first to propose solving OPs and understanding the existing optimization methods from the optimal control view. Every parameter or variable in OCP has a clear physical meaning whereby it is easy to tune the parameters to alter the interested variables [18]. Accordingly, the convergence rate of algorithms based on OCP can be altered by adjusting those parameters inherited from OCP. It should be emphasized that in this paper, we do not apply the results of optimal control mechanically, but we propose a new optimization algorithm based on the optimal control and make a convergence analysis. The algorithm features converging more rapidly than gradient descent, meanwhile, it is superior to Newton’s method because it can be applied in the case of a singular Hessian matrix. We also point out that the convergence rate of the proposed algorithm can be accelerated by decreasing the magnitude of the control weight matrix or increasing the control terminal time inherited from OCP.

The remainder is arranged as follows. In Section II, we propose an update relation, an optimization method, for OP by the solution to OCP. We explore the relationship between the existing (gradient descent and Newton’s method) and proposed methods in Section III. Section IV is devoted to the convergence analysis of explicitly implementing the proposed method. Some conclusions are achieved in Section V.

Assume that $f(x)$ is twice continuously differentiable. In most practical settings, one would almost surely need to resort to numerical techniques to address the optimization problem

The traditional numerical method, for instance, gradient descent or Newton’s method, directly finds zeros of $f^{\prime}(x)$ by an update recursion

with $\Phi(\cdot)$ being a designed function.

Instead, we will propose an updated relation with the aid of the following OCP:

where $x_{k}$ and $u_{k}$ are the state and control of system (4), respectively, integer $N>0$ is the control terminal time, positive definite matrix $R$ is the control weight matrix, and (3) is the cost functional of OCP, closely related to OP (1).

Take (5) as an update direction. Then it can be obtained from (4) an update relation

Although the relation (6) is implicit, some explicit approximation implementations are possible. We will give an explicit approximation recursion later.

It should be stressed that the proposal of a new update scheme is not our ultimate goal. We are also interested in revealing the relationship between existing methods and our methods.

To explore the relationship between the existing method and our method, we reformulate the update relation (6) now. Make the first-order Taylor expansion of $f^{\prime}(x_{i})$ at point $x_{i-1}$ in (6). It can be derived that

where $f^{\prime}_{k}\dot{=}f^{\prime}(x_{k})$ and $f^{(2)}_{k}\dot{=}f^{(2)}(x_{k})$.

To facilitate calculation, we replace all $x_{i},i\geq k+1$, with $x_{k}$ in the right-hand side of (8) and obtain the following recursion

According to (9)-(10), the following algorithm is proposed.

Algorithm 1:
1). Fix integer $N>0$ and prescribe any $x_{0}$;
2). Carry out the update relation (9)-(10) and obtain $x_{N+1}$;

In the sequel, we will make some convergence analysis pertinent to Algorithm 1. To compare with Newton’s method, we assume that $f(x)$ is cubic differentiable.

By direct manipulation, $\phi^{(2)}(x_{*})$ in (13) can be given as

On account of $|[(2N+3)f^{(2)}(x_{*})+R]R^{N+1}-(R+f^{(2)}(x_{*}))^{N+2}|<|(R+f^{(2)}(x_{*}))^{N+2}|$, we have $|\phi^{(2)}(x_{*})|<|\frac{f^{(3)}(x_{*})}{f^{(2)}(x_{*})}|$.

Remark 5-7 are devoted to the convergence of (11)-(12). It can be regarded that (11)-(12) enjoys a unified step size $\bar{g}_{0}(k)$ in each iteration because of the unified convergence rate. As in the gradient descent method [19, 20], the unified step size may lead to a slow convergence. For this, Algorithm 1 with varying step size $\bar{g}_{k}(x_{k})$ is a better alternative to the recursion (11)-(12).

To analyze the convergence rate of Algorithm 1, for $i=1,\cdots,N$, let

Then the following theorem holds.

In the paper, a novel algorithm has been developed based on OCP. The algorithm converges more rapidly than gradient descent, meanwhile, unlike Newton’s method, it still works even if the singular Hessian matrix appears in iterate. We also pointed out that Newton’s method is recovered when the control weight matrix disappears in the developed algorithm. The convergence rate of the algorithm depends on two adjustable parameters. Because of their clear physical meanings inherited from OCP, it is very convenient to tune them to speed up the convergence rate.