Secant Penalized BFGS: A Noise Robust Quasi-Newton Method Via Penalizing The Secant Condition

Noise is inevitably introduced into machine learning problems due to the approximations required to handle large datasets, and numerical simulations due to the effects of finite precision arithmetic, and parts of the simulator containing inherently stochastic components. In this paper, we return to the fundamental theory underlying the design of quasi-Newton methods, which allows us to design a new variant of the BFGS method that explicitly handles the corrupting effects of noise. We do this as follows:

1. 1.

   In Section 2, we review the setup and derivation of the original BFGS method.

2. 2.

   In Section 3, motivated by regularized least squares estimation, we treat the secant condition of BFGS with a penalty method. This creates a new BFGS update formula that we refer to as secant penalized BFGS (SP-BFGS), which we show reduces to the original BFGS update formula in a limiting case, as expected.

3. 3.

   In Section 4, we present an algorithmic framework for practically implementing SP-BFGS updating. We also discuss implementation details, including how to perform a line search and choose the penalty parameter in the presence of noise.

4. 4.

   In Section 5, we discuss the theoretical properties of SP-BFGS, including how the penalty parameter influences the eigenvalues of the approximate inverse Hessian. This allows us to show that under appropriate conditions SP-BFGS iterations are guaranteed to converge linearly to a neighborhood of the global minimizer when minimizing strongly convex functions in the presence of uniformly bounded noise.

5. 5.

   In Section 6, we study the empirical performance of SP-BFGS updating compared to BFGS updating by performing extensive numerical experiments with both convex and nonconvex objective functions corrupted by function and gradient noise. Results from a diverse set of over 30 problems from the CUTEst test problem set demonstrate that intelligently implemented SP-BFGS updating frequently outperforms BFGS updating in the presence of noise.

6. 6.

   Finally, Section 7 concludes the paper and outlines directions for further work.

The BFGS method was originally designed to solve the following unconstrained optimization problem

with $x\in\mathbb{R}^{n}$, $\phi:\mathbb{R}^{n}\mapsto\mathbb{R}$, and $\phi$ being a smooth twice continuously differentiable and nonnoisy function. Below, we use the notational conventions of Nocedal2006 , including $\phi_{k}=\phi(x_{k})$. We begin by using the Taylor expansion of $\phi$ to build a local quadratic model $m_{k}$ of the objective function $\phi$ at the $k^{th}$ iterate $x_{k}$ of the optimization procedure

where $B_{k}$ is an $n\times n$ symmetric positive definite matrix that approximates the Hessian matrix (i.e. $B_{k}\approx\nabla^{2}\phi_{k}$). By setting the gradient of $m_{k}$ to zero, we see that the unique minimizer $p_{k}$ of this local quadratic model is

and thus it is natural to update the next iterate $x_{k+1}$ as

where $\alpha_{k}$ is the step size along the direction $p_{k}$, which is often chosen using a line search.

To avoid computing $B_{k}$ from scratch at each iteration $k$, we use the curvature information from recent gradient evaluations to update $B_{k}$, and thus relatively economically form $B_{k+1}$. A Taylor expansion of $\nabla\phi$ reveals

and so it is reasonable to require that the new approximate Hessian $B_{k+1}$ satisfies

which rearranges to

Now, define the two new quantities $s_{k}$ and $y_{k}$ as

Thus, we arrive at (9), which is known as the secant condition

In words, the secant condition dictates that the new approximate Hessian $B_{k+1}$ must map the measured displacement $s_{k}$ into the measured difference of gradients $y_{k}$. If we denote the approximate inverse Hessian $H_{k}=B_{k}^{-1}\approx\nabla^{2}\phi_{k}^{-1}$, then the secant condition can be equivalently expressed as (10)

As $H_{k+1}$ is not yet uniquely determined, to obtain the BFGS update formula, we impose a minimum norm restriction. Specifically, we choose $H_{k+1}$ to be the solution of the following quadratic program over matrices

where $||\cdot||_{F}$ denotes the Frobenius norm, and $W^{1/2}$ the principal square root (see MatrixAnalysisHornJohnson or a similar reference) of a symmetric positive definite weight matrix $W$ satisfying

As we will see, choosing the weight matrix $W$ to satisfy (12) ensures that the resulting optimization method is scale invariant. The weight matrix $W$ can be chosen to be any symmetric positive definite matrix satisfying (12), and the specific choice of $W$ is not of great importance, as $W$ will not appear directly in the main results of this paper. However, as a concrete example from Nocedal2006 , one could assume $W=\bar{G}_{k}$, where $\bar{G}_{k}$ is the average Hessian defined by

To solve the quadratic program given by (11), we setup a Lagrangian $\mathcal{L}(H,q,\Gamma)$ involving the constraints. Recalling that

this gives the Lagrangian defined by (15) below

where $q$ is a vector of Lagrange multipliers associated with the secant condition, and $\Gamma$ is a matrix of Lagrange multipliers associated with the symmetry condition. Taking the derivative of the Lagrangian $\mathcal{L}(H,q,\Gamma)$ with respect to the matrix $H$ yields

and so we have the Karush-Kuhn-Tucker (KKT) system defined by the three equations (17a), (17b), and (17c) below

For brevity, we omit the details of the solution of the KKT system defined above because it is a limiting case of the system solved in Theorem 3.1. For an alternative geometric solution technique, we refer the interested reader to Section 2 of doi:10.1080/10556780802367205 . The minimizer $H^{*}=H_{k+1}$ is given by the well known BFGS update formula

which, if we define the curvature parameter $\rho_{k}=\frac{1}{s_{k}^{T}y_{k}}$, can be equivalently written as

Applying the Sherman-Morrison-Woodbury formula (see 10.2307/2030425 ) to the BFGS update formula immediately above, one can also write the BFGS update in terms of the approximate Hessian $B_{k}=H_{k}^{-1}$ instead of the approximate inverse Hessian. Again, for brevity, the details are omitted because they are a special case of Theorem 3.2 shown later. The result is

To ensure the updated approximate Hessian $B_{k+1}$ is positive definite, we must enforce that

Substituting $B_{k+1}s_{k}=y_{k}$ from the secant condition, the condition (21) becomes

which is known as the curvature condition, as it is equivalent to

By applying a penalty method (see Chapter 17 of Nocedal2006 ) to the secant condition instead of directly enforcing the secant condition as a constraint, we obtain the problem

where $\beta_{k}\in[0,+\infty]$ is a penalty parameter that determines how strongly to penalize violations of the secant condition. As we will see, one recovers the solution to the constrained problem (11) in the limit $\beta_{k}=+\infty$, so $\beta_{k}$ can be intuitively thought of as the cost of violating the secant condition. By treating the symmetry constraint with a matrix $\Gamma$ of Lagrange multipliers again, we obtain the following Lagrangian

Defining the residual associated with the secant condition as $r_{k}(H)\coloneqq Hy_{k}-s_{k}$ and $u\coloneqq\beta_{k}Wr_{k}$, the first order optimality conditions of (25) can be written as the system

Note that, as expected, in the limit $\beta_{k}=+\infty$, the system given by (26a), (26b), and (26c) reduces to the KKT system given by (17a), (17b), and (17c).

We now find an explicit closed form solution to the problem given by (24), which is given in Theorem 3.1.

At this point, a few comments are in order regarding the SP-BFGS update given by (27). First, observe that as $\beta_{k}\rightarrow+\infty$, we have that $\omega_{k}\rightarrow\rho_{k}$ and $\gamma_{k}\rightarrow\rho_{k}$. As a result, when $\beta_{k}=+\infty$, one recovers the original BFGS update, as expected. Second, also observe that as $\beta_{k}\rightarrow 0$, we have that $\omega_{k}\rightarrow 0$ and $\gamma_{k}\rightarrow 0$. As a result, we see that in the case $\beta_{k}=0$ the SP-BFGS update reduces to $H_{k+1}=H_{k}$. This is again expected because as $\beta_{k}\rightarrow 0$, the cost of violating the secant condition goes to zero, and the minimum norm symmetric update is simply $H_{k+1}=H_{k}$.

We now examine what the analog of the curvature condition (22) is for SP-BFGS. Lemma 1 demonstrates that (29) is the SP-BFGS analog of the BFGS curvature condition (22).

The result in Lemma 1 warrants some discussion. First, the limiting behaviour with respect to $\beta_{k}$ is consistent with Theorem 3.1. As $\beta_{k}\rightarrow+\infty$, condition (29) reduces to the BFGS curvature condition (22). As $\beta_{k}\rightarrow 0$, condition (29) reduces to no condition at all, as $s_{k}^{T}y_{k}>-\infty$ is always true. This is consistent with the observation that when $\beta_{k}=0$, the minimum norm symmetric update is $H_{k+1}=H_{k}$, and in this case $H_{k+1}$ is guaranteed to be positive definite if $H_{k}$ is positive definite, regardless of $s_{k}^{T}y_{k}$.

From the proof of Lemma 1 (see (93)), it is now clear that

and so $y_{k}^{T}H_{k+1}y_{k}$ is a convex combination of $y_{k}^{T}s_{k}$ and $y_{k}^{T}H_{k}y_{k}$. Thus, $H_{k+1}$ interpolates between the current inverse Hessian approximation $H_{k}$ and the original BFGS update, and as $\beta_{k}$ decreases, the interpolation is increasingly biased towards the current approximation $H_{k}$. From a regularized least squares estimation perspective, $\beta_{k}$ plays the role of a regularization parameter that controls the amount of bias in the estimate of $H_{k+1}$. Note that this behaviour is somewhat similar to the behaviour of Powell damping Powell1978 , although Powell damping was introduced to handle approximating a potentially indefinite Hessian of the Lagrangian in constrained optimization problems, and not noise.

We finish introducing the SP-BFGS update by applying the Sherman-Morrison-Woodbury formula to (27), which allows us to write the update in terms of the approximate Hessian $B_{k}$ instead of the approximate inverse Hessian $H_{k}$. The result is given in Theorem 3.2.

Note that the limiting behaviour of Theorem 3.2 with respect to $\beta_{k}$ is again consistent. When $\beta_{k}=+\infty$, we obtain the original BFGS inverse update (20), and when $\beta_{k}=0$, we obtain $B_{k+1}=B_{k}$. One complication with respect to the SP-BFGS inverse update (98) is that $B_{k+1}$ cannot in general be expressed solely in terms of $B_{k}$ due to the presence of $y_{k}^{T}B_{k}^{-1}y_{k}$ (i.e. $y_{k}^{T}H_{k}y_{k}$) in the denominator.

Algorithm 1 outlines a general procedure for minimizing a noisy function with noisy function and gradient values $f$ and $g$ that can be decomposed as shown in (31) and (32). The inputs to the procedure in Algorithm 1 are a means of evaluating the noisy objective function $f(x)$ and gradient $g(x)$, the starting point $x^{0}$, and an initial inverse Hessian approximation $H^{0}$. As the best convergence/stopping test is problem dependent, we note that standard gradient and function value based tests can be employed in conjuction with smoothing and noise estimation techniques (e.g. see Section 3.3.4 of DFONoisyFunctionsQuasiNewton ). In the next several subsections, we discuss how to choose the penalty parameter $\beta_{k}$ and step size $\alpha_{k}$, and appropriate courses of action for when the SP-BFGS curvature condition (29) fails.

As the choice of $\beta_{k}$ determines how strongly to bias the estimate of $H_{k+1}$ towards $H_{k}$, the choice of $\beta_{k}$ is fundamentally connected to the amount of noise present in the measured gradients $g_{k+1}$ and $g_{k}$. In brief, if the amount of noise present in the measured gradients is large, $\beta_{k}$ should be small to avoid overfitting the noise, and if the amount of noise present in the measured gradients is small, $\beta_{k}$ should be large to avoid underfitting curvature information. To make this point more rigorous, we introduce the following assumption.

As $\nabla\phi(x)$ is continuous, for each $k\geq 0$ we have

However, due to noise we cannot in general guarantee

Using the continuity of $\nabla\phi(x)$, Assumption 1, and the triangle inequality, one can conclude that

As a result, it is now clear that in the presence of uniformly bounded gradient noise, sending the step size $\alpha_{k}$ to zero, and thus $s_{k}$ to zero, only bounds the difference of measured gradients within a ball with radius dependent on the gradient noise bound $\bar{\epsilon}_{g}$.

As $g_{k+1}$ and $g_{k}$ can be decomposed into smooth and noise components, so can $s_{k}^{T}y_{k}$, giving

In conjunction with the Cauchy-Schwarz inequality, Assumption 1 implies that

and so we have the lower and upper bounds

From (39), it is clear that the bound on the effect of the noise grows linearly with $\left\|s_{k}\right\|_{2}$. However, by using the average Hessian $\bar{G}_{k}$ from (13) and applying Taylor’s theorem to $\nabla\phi$, it is also clear that

and so

where the $O\big{(}\left\|s_{k}\right\|_{2}^{2}\big{)}$ term is due to the true curvature of the smooth function $\phi$, and the $O\big{(}\left\|s_{k}\right\|_{2}\big{)}$ term is due to noise. Thus, we have now illustrated an important general behaviour given Assumption 1. As $\left\|s_{k}\right\|_{2}$ dominates $\left\|s_{k}\right\|_{2}^{2}$ as $\left\|s_{k}\right\|_{2}\rightarrow 0$, the effects of noise can dominate the true curvature for small $s_{k}$. Conversely, as $\left\|s_{k}\right\|_{2}^{2}$ dominates $\left\|s_{k}\right\|_{2}$ as $\left\|s_{k}\right\|_{2}\rightarrow+\infty$, the true curvature can dominate the effects of noise for large $s_{k}$.

Given the above analysis, a simple strategy for choosing $\beta_{k}$ is to make $\beta_{k}$ grow linearly with $\left\|s_{k}\right\|_{2}$, such as

where $N_{s}>0$ is a slope parameter. As $\left\|s_{k}\right\|_{2}\rightarrow 0$, $H_{k+1}\rightarrow H_{k}$, which is desirable because the effects of noise likely dominate as $\left\|s_{k}\right\|_{2}\rightarrow 0$. Increasingly biasing the estimate of $H_{k+1}$ towards $H_{k}$ reduces how much $H_{k+1}$ can be corrupted by noise, and relaxes the SP-BFGS curvature condition (29), reducing the likelihood of needing to trigger a recovery procedure described in Section 4.4. Also, as shown earlier, because $\nabla\phi$ is continuous, the true difference of gradients is guaranteed to go to zero as $s_{k}$ approaches zero. As a result, without noise present, it is natural that $H_{k+1}\rightarrow H_{k}$ as $s_{k}\rightarrow 0$. In the presence of noise, we wish for this behaviour to be preserved. Informally, one can intuitively think of wanting $H_{k}$ to behave as an approximate average inverse Hessian, and the averaging should remove the corrupting effects of noise, leaving $H_{k}$ to behave as if no noise were present. Similarly, as $\left\|s_{k}\right\|_{2}\rightarrow+\infty$, $\beta_{k}\rightarrow+\infty$, and one recovers the BFGS update in the limit, which is desirable because the effects of noise are likely dominated by the true curvature as $\left\|s_{k}\right\|_{2}\rightarrow+\infty$. The slope parameter $N_{s}$ dictates how sensitive $\beta_{k}$ is to $\left\|s_{k}\right\|_{2}$, and should be set proportional to the gradient noise level (i.e. $\bar{\epsilon}_{g}$). Intuitively, if the gradient noise level is low, $\beta_{k}$ should grow quickly with $\left\|s_{k}\right\|_{2}$, as the effect of noise diminishes quickly, and vice versa.

It may also be desirable to modify (42) to

where $N_{o}>0$ is an intercept parameter. The inclusion of $N_{o}$ allows one to stop updating $H_{k}$ if $\left\|s_{k}\right\|_{2}$ is sufficiently small. For example, it may be desirable to stop updating $H_{k}$ when one is very close to a stationary point, as gradient measurements are likely heavily dominated by noise.

Classically, during BFGS updating $\alpha_{k}$ is chosen to satisfy the Armijo-Wolfe conditions. As function and gradient evaluations are not corrupted by noise in the classical BFGS setting, we can write the Armijo condition, also known as the sufficient decrease condition, as

and the Wolfe condition, also known as the curvature condition, as

where $0<c_{1}<c_{2}<1$, with well known choices being $c_{1}=10^{-4}$ and $c_{2}=0.9$. Observe that by adding $\nabla\phi_{k}^{T}p_{k}$ to both sides of (45) and multiplying by $\alpha_{k}$, (45) becomes

If $p_{k}$ is a descent direction then $\nabla\phi_{k}^{T}p_{k}<0$, and combined with $(c_{2}-1)<0$ and $\alpha_{k}>0$, one sees that (46) implies

so (45) effectively enforces (22) when no gradient noise is present.

In the presence of noisy gradients, we argue that in general it no longer makes sense to enforce the Wolfe condition (45). In the presence of gradient noise, (45) becomes

which can behave erratically once the noise vectors $e_{k+1}$ and $e_{k}$ start to dominate the gradient of $\phi$. For example, the noise vectors $e_{k+1}$ and $e_{k}$ can cause both sides of (48) to erratically change sign, in which case whether or not the Wolfe condition is satisfied can be governed by randomness more than anything else.

We argue that because the SP-BFGS update allows one to relax the curvature condition based on the value of $\beta_{k}$ as shown in the SP-BFGS curvature condition (29), it is appropriate to drop the Wolfe condition entirely in the presence of gradient noise and instead employ only a version of the sufficient decrease condition when choosing $\alpha_{k}$. In the situation where gradient noise is present but function noise is not (i.e. $f(x)=\phi(x)$ in (31)), one can use a backtracking line search based on the sufficient decrease condition, which can guarantee convergence to a neighborhood of a stationary point of $\phi$. The situation where noise is present in both function and gradient evaluations is trickier. Similar to the approach presented in Section 4.2 of DFONoisyFunctionsQuasiNewton , one option is to use a backtracking line search with a relaxed sufficient decrease condition of the form

where $\epsilon_{A}\geq 0$ is a noise tolerance parameter and $p_{k}=-H_{k}g_{k}$. In Theorem 4.2 of DFONoisyFunctionsQuasiNewton , the authors show that under Assumptions 1 and 2, using the iteration (4) and a backtracking line search governed by the relaxed Armijo condition (49) with $p_{k}=-g_{k}$ guarantees linear convergence to a neighborhood of the global minimizer for strongly convex functions.

We agree with the authors of DFONoisyFunctionsQuasiNewton that it is possible to prove an extension of Theorem 4.2 of DFONoisyFunctionsQuasiNewton to a quasi-Newton iteration with positive definite $H_{k}$, and briefly outline why in Section 5.2. A quasi-Newton extension of Theorem 4.2 of DFONoisyFunctionsQuasiNewton is relevant to SP-BFGS updating because, as we will formally see in Section 5.1, control of $\beta_{k}$ makes it possible to uniformly bound the minimum and maximum eigenvalues of $H_{k+1}$.

In the classical BFGS scenario where no gradient noise is present, the curvature condition (22) may fail if $\alpha_{k}$ is not chosen based on the Armijo-Wolfe conditions and $\phi$ is not strongly convex. One of the most common strategies to handle this scenario is to skip the BFGS update (i.e. set $H_{k+1}=H_{k}$) when this occurs, which corresponds to an SP-BFGS update with $\beta_{k}=0$. However, this simple strategy has the downside of potentially producing poor inverse Hessian approximations if updates are skipped too frequently.

Conditionally skipping BFGS updates is an option in the presence of noisy gradients as well. In addition to skipping BFGS updates when (22) fails, as described above, another course of action sometimes recommended in the presence of noise is to replace (22) with

where $\varepsilon>0$ is a small positive constant, and skip the BFGS update if (51) is not satisfied. This strategy may be somewhat effective if $\left\|s_{k}\right\|_{2}$ is large, but reduces back to the initial update skipping approach as $\left\|s_{k}\right\|_{2}\rightarrow 0$. A similar strategy (e.g. see Section 3.3.3 of DFONoisyFunctionsQuasiNewton ) is to replace (22) with

and skip the BFGS update if (52) is not satisfied for some $\zeta\in(0,1)$. Notice that none of the aforementioned update skipping strategies allow for curvature information to be incorporated if the measured curvature $s_{k}^{T}y_{k}$ is negative.

Unlike in the classical BFGS scenario, with SP-BFGS updating, curvature information can be incorporated even if the measured curvature $s_{k}^{T}y_{k}$ is negative by decreasing $\beta_{k}$ towards $0$. In addition to having the option of conditionally skipping updates (i.e. setting $\beta_{k}=0$), one can also alternatively relax the SP-BFGS curvature condition by decreasing $\beta_{k}$ towards $0$ if (29) fails. Since $\beta_{k}$ is chosen after $s_{k}$ and $y_{k}$ are fixed in Algorithm 1, one can solve for $\beta_{k}$ values satisfying (29) when $s_{k}^{T}y_{k}<0$, yielding

for all $c_{3}>1$, assuming that $s_{k}\neq 0$ and $y_{k}\neq 0$. Note that if $s_{k}\neq 0$ and $y_{k}\neq 0$ and (29) fails, then the measured curvature $s_{k}^{T}y_{k}$ must be negative. The choice of $c_{3}$ determines how much to shrink $\beta_{k}$ compared to the largest value of $\beta_{k}$ that still satisfies (29) and thus guarantees the positive definiteness of $H_{k+1}$. Hence, if the value of $\beta_{k}$ produced by (42) or (43) is too large and (29) fails, one can choose an acceptable value of $\beta_{k}$ by using (53) and selecting a $c_{3}>1$. Thus, instead of skipping the update (i.e. setting $\beta_{k}=0$) if (29) fails, one can reduce $\beta_{k}$ towards $0$, which has the effect of reducing the magnitude of the update by increasing how much $H_{k+1}$ is biased towards $H_{k}$. An approach based on reducing $\beta_{k}$ towards $0$ never entirely skips incorporating measured curvature information, even if the measured curvature information is negative, but instead weights how heavily the measured curvature information affects $H_{k+1}$.

We first examine how $\beta_{k}$ determines how much the maximum and minimum eigenvalues $\lambda_{max}(H_{k+1})$ and $\lambda_{min}(H_{k+1})$ can change. In what follows, $\lambda(H)$ denotes the set of eigenvalues $\lambda_{1},\dots,\lambda_{n}$ of the matrix $H\in\mathbb{R}^{n\times n}$. We provide upper bounds on $\lambda_{max}(B_{k+1})$ and $\lambda_{max}(H_{k+1})$ via Theorem 5.1. As $H_{k}=B_{k}^{-1}$, $1/\lambda_{min}(H_{k+1})=\lambda_{max}(B_{k+1})$, and putting an upper bound on $\lambda_{max}(B_{k+1})$ is equivalent to putting a lower bound on $\lambda_{min}(H_{k+1})$.

With Theorem 5.1 in hand, we now formally see that when $s_{k}^{T}y_{k}>0$, as $\beta_{k}$ increases from $0$ to $+\infty$, an upper bound on $\lambda_{max}(H_{k+1})$ interpolates between $\operatorname{Tr}(H_{k})$ and $+\infty$, and an upper bound on $\lambda_{max}(B_{k+1})$ interpolates between $\operatorname{Tr}(B_{k})$ and $+\infty$. Similarly, when $s_{k}^{T}y_{k}<0$, as $\beta_{k}$ increases from $0$ towards $-\frac{1}{(s_{k}^{T}y_{k})}$, an upper bound on $\lambda_{max}(H_{k+1})$ interpolates between between $\operatorname{Tr}(H_{k})$ and $+\infty$, and an upper bound on $\lambda_{max}(B_{k+1})$ interpolates between $\operatorname{Tr}(B_{k})$ and $+\infty$. Standard BFGS updating corresponds to setting $\beta_{k}=+\infty$ for all $k$, and as this is the largest possible value of $\beta_{k}$, one can no longer formally guarantee that $\lambda_{max}(B_{k+1})$ and $\lambda_{max}(H_{k+1})$ are bounded from above at each iteration because the measured curvature $s_{k}^{T}y_{k}$ may become arbitrarily close to zero due to the effects of noise. The key takeaway is that upper bounds on $\lambda_{max}(H_{k+1})$ and $\lambda_{max}(B_{k+1})$ can be tightened arbitrarily close to $\operatorname{Tr}(H_{k})$ and $\operatorname{Tr}(B_{k})$ by shrinking $\beta_{k}$ towards zero, as $s_{k}$, $y_{k}$, and $H_{k}$ are fixed before the value of $\beta_{k}$ is chosen in Algorithm 1.

Thus, if one must enforce a bound of the form $\lambda_{max}(H_{k+1})\leq C_{H}$ or a bound of the form $\lambda_{max}(B_{k+1})\leq C_{B}$ for all $k\geq 0$, where $C_{H}>\operatorname{Tr}(H_{0})>0$ and $C_{B}>\operatorname{Tr}(B_{0})>0$ are positive constants, there exist nontrivial sequences of sufficiently small $\beta_{k}$ with $\lim_{k\rightarrow\infty}\beta_{k}=0$ that ensure the bounds hold for all $k$. To see this, observe that the interval $[\lambda_{max}(H_{0}),C_{H}]$ can be partitioned into subintervals corresponding to each iteration, and the sum of the subintervals cannot exceed $C_{H}-\lambda_{max}(H_{0})$, which can be guaranteed by assigning a small enough value of $\beta_{k}$ to each subinterval, as this guarantees the maximum eigenvalue does not grow too much at each iteration $k$. Furthermore, although there clearly exist sequences of $\beta_{k}$ that ensure the bounds hold for all $k$ that satisfy $\beta_{k}=0$ for all $k\geq K$, where $K$ is a positive integer, there also exist sequences of $\beta_{k}$ that ensure the bounds hold for all $k$ where $\beta_{k}$ instead only approaches zero in the limit $k\rightarrow\infty$.

Having established that SP-BFGS iterations can maintain bounds on the maximum and minimum eigenvalues of the approximate inverse Hessians via sufficiently small choices of $\beta_{k}$, we now consider minimizing strongly convex functions in the presence of bounded noise. We introduce Assumption 3, and the notation $x^{\star}$ to denote the argument of the unique minimum of $\phi$, and $\phi^{\star}=\phi(x^{\star})$ to denote the minimum.