Analyzing the Effect of Persistent Asset Switches on a Class of Hybrid-Inspired Optimization Algorithms

Convex optimization challenges are pervasive across many current science and technology fields. Such optimization problems are often solved using iterative algorithms such as first-order gradient-based methods that can be naturally represented and analyzed as dynamical systems. However, most studies of these algorithms do not account for applications in which the objective function to be optimized can instantaneously change at discrete moments in time during the algorithm’s execution. Switching objectives arise in increasingly many real-world applications, such as multi-agent systems in which the agents must be replaced according to a real-time mission constraint, as well as resource allocation problems in which the allocated assets can experience persistent changes in the reward they generate. Other examples can be found in branches of human science, such as sociology, psychology, and organization science, which study the group interactions and performance of teams with multiple individuals engaged in a common task [1], [2]. In [3], for instance, an algorithm is proposed and studied for optimizing the sum of the team members’ performance measures, and each member’s performance measure is determined by variables such as the member’s specific skill level. Thus, the objective will switch whenever a team member is replaced during execution of the algorithm. Persistent switches of assets could also play an important role in the realm of intelligent control of unmanned aerial vehicles (UAVs), also referred to as drones. In recent years, there has been a surge in the use of UAVs for surveillance and security, parcel shipment, traffic monitoring, disaster recovery, and military reconnaissance [4],[5]. Cooperative and collaborative optimization of UAV performance is essential in such applications. In [6], applications and engineering constraints for UAV-based mobile ad hoc networking are surveyed. In all these settings, there are few studies of how optimization dynamics are impacted by objectives that persistently switch due to failures or replacements of UAVs in a network/relay. In this paper, we analyze how the presence of a persistently switching objective impacts the asymptotic stability properties of optimization dynamics. Our analysis aims to be applicable to systems such as the aforementioned UAV ad hoc networks, where the UAVs that suffer from low battery, potential damages, or other disabling aspects may be replaced during execution of optimization dynamics. Our analysis targets applications where a system must instantaneously replace assets (UAVs, team members, etc.) at discrete moments in time, while it optimizes performance continuously in time. To prevent instability and be able to characterize the set to which the optimization dynamics converge, such switches should satisfy an average dwell-time constraint. In [7], stability is studied for systems involving switches between multiple differential equations with distinct equilibria, with switches satisfying an average dwell-time condition. We extend the results from [7] and apply the provided asymptotic characterization method to consider switching between differential inclusions with distinct equilibria. The differential inclusions each take the form of the Hybrid-inspired Heavy Ball System from [8], which takes advantage of the differential inclusion to achieve efficiency comparable to accelerated gradient methods while also retaining certain robustness properties. We characterize the set to which the resulting switched system converges, in terms of the Omega-limit set of an associated ideal hybrid system. This ideal hybrid system involves an automaton with solutions that are in one-to-one correspondence with time domains satisfying the average dwell-time constraint and with the rate parameter set to zero. Finally, we show that the system switching between differential inclusions with a small disturbance is a perturbed version of the mentioned ideal hybrid system with a globally asymptotically stable Omega-limit set. Thus, we can establish semi-global, practical asymptotic stability for the perturbed system. The robust stability of switched systems with multiple equilibria is a well studied subject (see [9],[10]). For example, [11] studies the steady-state optimization of switched systems with time-varying cost functions. We demonstrate our stability results on an application involving a data relay formed by UAVs, where the objective models various performance measures such as the communication quality or battery consumption.

In this work, $\mathbb{R}^{n}$ is used to demonstrate the $n$-dimensional Euclidean space. The $\mathbb{R}_{\geq 0}$ is used to show the nonnegative real numbers. We use $|x|$ to denote the Euclidean norm of the verctor $x\in\mathbb{R}^{n}$. Given $r>0$, we use $r\mathbb{B}$ for the set $\{x\in\mathbb{R}^{n}:|x|\leq r\}$. For a function $\alpha$, we say $\alpha\in\mathcal{K}^{+}$ if $\alpha:\mathbb{R}_{\geq 0}\rightarrow\mathbb{R}_{\geq 0}$ is continuous and strictly increasing.

The hybrid systems framework that we use is described in [12]. Formally, hybrid systems are modeled as

where $x\in\mathbb{R}^{n}$ is the state, $C\subset\mathbb{R}^{n}$ is the flow set, $D\subset\mathbb{R}^{n}$ is the jump set, $F:\mathbb{R}^{n}\rightrightarrows\mathbb{R}^{n}$ is the flow map, and $G:\mathbb{R}^{n}\rightrightarrows\mathbb{R}^{n}$ is the jump map. The data $(C,F,D,G)$ is said to satisfy the hybrid basic conditions if $C$ and $D$ are closed, the graphs of $F$ and $G$ are closed, $F$ and $G$ are locally bounded, the values of $F$ are nonempty and convex on $C$ and the values of $G$ are nonempty on $D$. We use $\mathcal{S}(K)$ to denote the set of solutions to (1) that start in $K$.

In the first part of this section, we introduce some notions later used in this work and state some stability concepts for hybrid systems. In the second part of this section, we review the concept of average dwell-time constraint [13]. Next, we consider a perturbed hybrid system, in which the perturbation is parameterized by a dwell-time parameter satisfying the average dwell-time condition from [14]. Finally, we discuss the advantages of optimization methods employing differential inclusions for applications with persistent asset switches, introducing an efficient differential inclusion-based optimization algorithm inspired by the hybrid algorithms studied in [8]. This optimization algorithm has been shown in [8] to have desirable robustness properties and will be shown in Section VI to be especially suitable for online optimization problems with persistent asset switches.

In this section, we discuss an optimization problem that models a team of drones collectively executing a task. The drones are the assets that persistently switch in this scenario, due to each drone’s limited battery life, potential physical damages, or other disabling aspects. Consider a team of $n$ drones with the collective task of forming a relay to transmit data from a source to a destination. The drones form a straight path of length $d\in\mathbb{R}_{>0}$ from the first to $n$-th drone. The network-wide state vector is denoted by $q\in\mathbb{R}^{n}$, where $q_{i}$ is the relative distance from drone $i$ to the node that precedes it, while the data source is considered to precede the first drone. We assume that the drones $1$ to $n$ never cross over their neighboring drones, and their ordering remains the same during their movement. In practice, enforcement of a box constraint on each $q_{i}$ is needed, which can be done using the penalty-based approach in [27], but the details are beyond the scope of this work. The problem can be modeled as a variant of the problem (6), in which the objective switches according to a switching signal $\sigma:\mathbb{R}_{\geq 0}\to\Sigma$:

In a similar setting to (2), the switching signal satisfies the average dwell-time constraint parametrized by a small $\delta$.

The proof of Theorem 2 is given in Section VII.

In this section, we give numerical examples for the application and the optimization method discussed in Sections V and IV-D respectively. In Figure 1, we consider a team of $2$ drones in a relay of distance $d=100$, and we consider $N_{0}=1$ in order to demonstrate the $\Omega$-limit set. The blue line represents the position of the first drone (initial condition: distance of $35.5071$ units from the data source), and the green line represents the position of a second drone relative to the first drone in the relay (initial condition: distance of $33.7398$ units from the first drone). These two lines depict the $\Omega$-limit set of the ideal hybrid system given in (5). The red lines are created by allowing for the small dwell-time parameter $\delta=0.0338$. The switches satisfy the average dwell-time condition. Considering the switching behavior to be a small disturbance as in Theorem 2, we observe that the solutions converge to a small neighborhood of the $\Omega$-limit set of the ideal hybrid system. These observations agree with the results from Theorem 2. Figure 2(a) shows the decrease in the value of the objective function under the dynamics of the HiHBM algorithm applied to the problem (17) for $20$ drones in a relay of length $d=100$, with the network-wide objective being $\phi_{\sigma}(q)=\frac{1}{2}q^{T}P_{\sigma}q+b_{\sigma}^{T}q$ for $\sigma\in\{1,2\}$. Each $P_{\sigma}$ is diagonal with eigenvalues i.i.d. uniform on $[10,20]$, and each $b_{\sigma}$ has its entries i.i.d. uniform on $[-10,10]$. The objective at each drone is $\phi_{i,\sigma}(q_{i})=\frac{1}{2}P_{\sigma,ii}q_{i}^{2}+b_{\sigma,i}q_{i}$, where $q_{i}$ is the distance from drone $i$ to drone $i-1$, as described in Sec. V. We allow persistent switches satisfying average dwell time with $\delta=0.06$. Figures 2(a) shows that HiHBM converges efficiently even as the objective switches persistently. Figure 2(b) shows the same example with fewer switches. In Figure 3, the gray line displays the gradient descent optimization. The red line demonstrates the convergence of the HBM optimization algorithm with $K=5$ and the green line demonstrates the same optimization algorithm with $K=1$. The black line in Figure 3 shows the HiHBM optimization algorithm with a faster, more efficient convergence. The lower and upper-bound for HiHBM are $0.01$ and $35.5$ respectively. In Figure 3, we see the efficiency of the HiHBM algorithm that generates smaller errors in comparison with the simple gradient descent and the HBM method.

We discussed the importance of enabling persistent switches of objective functions during an online execution of an optimization algorithm. The reason for these persistent switches is motivated by many engineering applications of convex optimization. We further extend the existing results of [7] on differential equations with multiple equilibria to differential inclusions with distinct multiple equilibria. We present the asset switches occurring in an optimization problem consisting of drones in a relay aiming for maximizing the signal strength. The asset switches are analyzed through the use of average dwell-time parameter which determines the rate of objective’s switching during the online HiHBM optimization. Hence, we establish semi-global practical asymptotic stability of a certain set with respect to this parameter. We characterize this set via Omega-limit sets.