Group Steering: Approaches Based on Power Moments

With a vast group of agents, i.e., $N$ is large, a conventional approach is to approximate $x(k)$ and $u(k)$ as random variables for which the density functions are denoted as $q_{k}$ and $p_{k}$. It is a mean-field approximation [3] of the state evolution of the agents. The problem of steering the group of agents is then turned to steering an initial density function to a terminal one. The density control problem is formulated as follows.

The dynamics of the system is

Given an initial probability density function $q_{0}(x)$ of the random variable $x(0)$, determine the control sequence, i.e., $\left(u(0),\cdots,u(K-1)\right)$ such that the terminal density function is $\tau(x)$ for $x(K)$.

However it is not always feasible for us to obtain a closed form of solution to this problem. If the distributions are not assumed to fall within specific classes, the problem is intrinsically infinite-dimensional. We note that the density function of $x(k+1)$ can be written as

where $x\perp\!\!\!\perp u$ denotes that $x(k)$ and $u(k)$ are independent. Addressing the density steering problem requires obtaining an analytical solution for $q_{k+1}(t)$ in (3). The scenario would be more complicated with $x(k),u(k)$ being correlated. However, except for specific function classes like Gaussian distributions or trigonometric functions, obtaining such an analytic solution is impractical for the majority of function classes. This limitation is a key factor, as evident in prior results with a problem setting akin to the density steering problem, where the examples predominantly feature Gaussian or trigonometric densities. Consequently, the applicability of these results in real-world scenarios is severely constrained.

There is a similar problem in non-Gaussian Bayesian filtering. In our previous results [29], we proposed a method of using the power moments to treat this intractable problem, mainly for characterizing the macroscopic property of the distributions. However, even it is theoretically feasible to characterize the distribution of the agents by the full power moment sequence, the problem is infinite dimensional. A common treatment is to truncate the first $2n$ moment terms [7, 14], which turns the problem we treat to a truncated moment problem.

By the system equation (2), the power moments of the states up to order $2n$ are written as

for $l\in\mathbb{N}_{0}$ ($\mathbb{N}_{0}$ denotes the set of all nonnegative integers), $l\leq 2n$. It is difficult to treat the term $\mathbf{E}\left[x^{j}(k)u^{l-j}(k)\right]$. However, we note that if $x(k)$ and $u(k)$ are independent from each other, i.e., $\mathbf{E}\left[x^{j}(k)u^{l-j}(k)\right]=\mathbf{E}\left[x^{j}(k)\right]\mathbf{E}\left[u^{i-j}(k)\right]$, the dynamics of the moments can be written in a linear matrix equation

where the new state vector is composed of the power moment terms up to order $2n$,

The new input vector is written as

where we have denoted

and

The matrix $\mathscr{A}(\mathscr{U}(k))$ of the new system can then be written as (9).

The sets $\mathcal{S}$ and $\mathcal{C}$ are the support of $q_{k}(x)$ and $p_{k}(u)$ correspondingly. For control problems without state constraints, $\mathcal{S}$ can be chosen as the real line $\mathbb{R}$. Or it can be chosen as a compact subset of $\mathbb{R}$ according to its bounds. Similarly, $\mathcal{C}$ can be chosen as $\mathbb{R}$ if the control input is unconstrained, or a compact subset of $\mathbb{R}$ if constrained.

By employing truncated power moments to describe the dynamics of the system (1), where $x(k)$ and $u(k)$ represent probability densities, we aim to reformulate the control problem as the manipulation of the power moments of these probability densities. The system (4) is denoted as the moment system associated with (1).

By the proposed moment system, the original density steering problem can be reduced to distribution steering by power moments, which is formulated as follows. The dynamics of the moment system is

where $\mathscr{X}(k),\mathscr{U}(k)$ are obtained by (7),(8). Given an arbitrary initial density $q_{0}(x)$, determine the control sequence $\left(u(0),\cdots,u(K-1)\right)$ such that the first $2n$ order power moments of the terminal density are identical to those of an arbitrarily specified one, i.e.,

for $l=1,\cdots,2n$, where $\tau$ is the specified terminal density function.

However, when using the moment system for control, there is still a need to design control laws that satisfy

With a limited number of agents, it might be possible to propose feedback control laws for each agent, i.e., the control input is a function of the states of all the agents. However with a large number of agents, it is no longer tractable. In our density steering problem, $N$ is infinity (for the problem of steering occupation measures, $N$ is still a large number). A feasible control law needs to satisfy that the control vector is independent of the current state vector. In the next part of section, we propose an algorithm for steering power moments to desired ones.

In the previous section, controlling the group of agents has been reduced to controlling the moment system corresponding to it. Then the task is now to figure out an algorithm to determine a sequence of $\left(\mathscr{U}(0),\cdots,\mathscr{U}(K-1)\right)$. Nonetheless, there exist two primary distinctions from conventional control problems. Firstly, in this problem, the system matrix of the moment system becomes a function of the control vector. Secondly, the sequence of elements in the control vector, denoted as $\mathscr{U}(k)$, must adhere to the condition that the corresponding Hankel matrix

is positive definite. Here $\bm{\mathrm{H}}_{\mathscr{U}}$ denotes the Hankel matrix of the vector $\mathscr{U}$. We define such a subspace of $\mathbb{R}^{2n}$ as $\mathbb{V}^{2n}_{++}:=\{\mathscr{U}\in\mathbb{R}^{2n}\mid\bm{\mathrm{H}}_{\mathscr{U}(k)}\succ 0\}$.

In prior studies, the distribution steering problems have consistently employed optimal control strategies. However, such an approach is not entirely feasible in this specific problem. The challenge lies in the necessity to ensure that both $\mathscr{X}(k)$ and $\mathscr{U}(k)$ belong to $\mathbb{V}^{2n}_{++}$. To the best of our knowledge, there has yet to be a viable result capable of addressing the optimal control problem while imposing the constraint that states and control inputs must reside within $\mathbb{V}^{2n}_{++}$ - in other words, ensuring that the corresponding Hankel matrices are positive definite.

Now we formulate the problem we are to treat in this part of section. Let $\mathcal{U}$ be the feasible set of control sequences $\bm{\mathscr{U}}:=\left(\mathscr{U}(0),\cdots,\mathscr{U}(K-1)\right)$, which satisfies

and effects the terminal system state $x(K)$ to be distributed satisfying (10). Then the family $\mathcal{U}$ represents admissible control inputs which achieve the desired moment transfer. Denote the error of the moments from the specified ones as

where the elements of $\mathscr{X}_{T}$ are the power moments corresponding to the specified terminal density function $\tau(x)$.

Simultaneously confining both $\mathscr{U}(k)$ and $\mathscr{X}(k)$ to belong to the set $\mathbb{V}^{2n}_{++}$ is not always feasible. However, it’s worth noting that a sub-optimal solution to the control problem can be attained by first determining the trajectory of the state and subsequently obtaining the control inputs corresponding to this trajectory. A similar approach was also utilized in [25] to address the general distribution steering problem. We first determine the trajectory of the states.

Now it remains to prove that there exists a time step $k_{0}$ at which $\mathscr{X}_{T}-\mathscr{X}(k_{0})\in\mathbb{V}^{2n}_{++}$.

By Proposition 2.2, it is possible to choose a time step $k_{0}$ which satisfies $\mathscr{X}_{T}-\mathscr{X}(k_{0})\in\mathbb{V}^{2n}_{++}$. We assume that the system is uncontrolled before $k_{0}$, i.e. $u(k)=0,k\leq k_{0}$. From step $k_{0}$, we impose controls on the system. Lemma 2.1 has proved the positiveness of $\mathscr{X}(k),k=k_{0},\cdots,K$. Therefore it remains to determine the parameters $\omega_{k},k=k_{0},\cdots,K-1$ and the corresponding control inputs $\mathscr{U}(k)$.

It is a non-trivial problem. We give an empirical scheme to treat it. To obtain a relatively smooth transition of states, it is desired that the $\omega_{i}$’s are close to each other. It is usually feasible for us to choose

We note that to determine the trajectory of the system state a priori has also been adopted in the literature. In [25], prespecified system states, which was called to form a reference trajectory, were used to determine the control inputs. After that the parameters $\omega_{i}^{\prime}s$ are determined, the control inputs of the moment system $\mathscr{U}(i)$ for $i=k_{0},\cdots,K-1$ can then be calculated by solving the equation (4), provided with $\mathscr{X}(k),k=k_{0}+1,\cdots,K$ calculated by (13).

However sometimes the control inputs $\mathscr{U}(k)\notin\mathbb{V}^{2n}_{++}$ by choosing the $\omega_{i}$’s to be all equal. It usually happens when the specified initial/terminal density has several modes (peaks). If so, we can choose a larger $\omega_{0}/\omega_{K-1}$.

In conclusion, we have proposed to use the moment system to control the vast group of agents characterized as a probability density function. And an empirical control law has been proposed which ensures the existence of $u(k)$. However, it is only feasible for us to obtain the power moments of the control inputs $u(k)$. We need to obtain the $u(k)$ given its power moments, which we call realization of the control inputs.

In this part of the section, we are to realize the probability density of $u(k)$ given the power moments of the designed controls $\mathscr{U}(k)$ for the moment system.

For the sake of simplicity, we omit $k$ if there is no ambiguity in the following part of this section. The problem now comes to proposing an estimation algorithm which is feasible of estimating the probability density of which the power moments are as specified.

We typically examine two scenarios for controller design. The first scenario involves unconstrained control, i.e., $u$ is supported across the entire real line. The second scenario is constrained control, where we consider $u$ to be supported on a compact interval within $\mathbb{R}$. These situations correspond to two distinct types of moment problems we aim to address: the Hamburger moment problem for the first scenario and the Hausdorff moment problem for the second one.

Let $\mathcal{D}$ denote the space of probability density functions on the real line with support there. Let $\mathcal{D}_{2n}$ be the subset of all $p\in\mathcal{D}$ that have at least $2n$ finite moments (in addition to $\mathbf{E}[u^{0}(k)]$, which is 1 by definition). The Kullback-Leibler distance is then defined as

where $r$ is an arbitrary reference probability density in $\mathcal{D}$. According to the type of the control problem, $\mathcal{C}$ is either $\mathbb{R}$ (unconstrained control), or a compact interval on $\mathbb{R}$ (constrained control). We define

and the linear integral operator

where $p(u)$ is defined on the space $\mathcal{D}_{2n}$. Moreover, let

where $\mathbf{E}[u^{i}],i=1,\cdots,2n$, are the entries of the designed control $\mathscr{U}$. Moreover, since $\mathcal{D}_{2n}$ is convex, then so is $\operatorname{range}(\Theta)=\Theta\mathcal{D}_{2n}$.

Let

For any $r\in\mathcal{D}$ and any positive definite $\Sigma$, there exists a unique $\hat{p}\in\mathcal{D}_{2n}$ that minimizes (16) subject to $\Theta(\hat{p})=\Sigma$. Specifically,

where $\hat{\Lambda}$ is the unique solution to the minimization problem

over all $\Lambda\in\mathcal{L}_{+}$ [29]. Here, $\operatorname{tr}(\cdot)$ denotes the trace of the matrix.

Then the density estimation is formulated as a convex optimization problem. Unlike other methods of moments, the power moments of our proposed density estimate are exactly identical to those specified, which makes it a satisfactory approach for realization of the control inputs. Since the prior density $r(u)$ and the density estimate $\hat{p}(u)$ are both supported on $\mathbb{R}$, $r(u)$ can be chosen as a Gaussian distribution (or a Cauchy distribution if $\hat{p}(u)$ is assumed to be heavy-tailed).

We note that for the Hausdorff moment problem, where the control $u$ is supported on a compact subset of $\mathbb{R}$, the proofs and results for the Hamburger moment problem are also valid by substituting the domain $\mathbb{R}$ by a compact interval on the real line. By doing this, we will obtain a density estimate $\hat{p}$ in the form of (17). The prior $r(u)$ can then be chosen as a truncated Gaussian or a truncated Laplacian.

We conclude the algorithm for density steering of the vast group of agents in this section as in Algorithm 1.

Through the proposed density steering scheme, the first $2n$-order power moments of the terminal density function match those of the desired one. As $n$ approaches infinity, the full moment sequence of the terminal density converges to that of the desired density. By Theorem 4.5.5 in [12], we can straightforwardly demonstrate that the terminal distribution obtained from our distribution steering algorithm converges almost everywhere to the true desired distribution as $n$ tends to infinity. Assuming both the initial and desired terminal distributions are continuous, we can further conclude that the terminal density generated by the proposed algorithm converges to the desired distribution. This convergence to the desired density distinguishes our proposed algorithm from other existing methods.

Since we used the truncated power moments of the initial and terminal density functions for steering, there may exist an error between the terminal density and the desired one. We will propose an upper bound of error of the terminal density, as to characterize the maximal difference between the terminal density by our proposed algorithm and the specified one.

In [29], we proposed an error upper bound for the Hamburger moment problem in the sense of total variation distance, which is a measure widely used in the moment problem [27, 26].

The total variation distance between the terminal density $q_{K}(x)$ and the desired terminal density $\tau(x)$ is defined as follows:

where $F_{q_{K}}$ and $F_{\tau}$ are the two distribution functions of $q_{K}$ and $\tau$.

Shannon-entropy is used to calculate the upper bound of the total variation distance in [27]. The Shannon-entropy [23] is defined as

We first introduce the Shannon-entropy maximizing distribution $F_{\breve{q}_{K}}$, of which the moments are the power moments of $q_{K}$. It has the following density function [15],

where $\lambda_{0},\cdots,\lambda_{2n}$ are determined by the following constraints,

for $k=0,1,\cdots,2n$. By referring to [27], the KL distance between the true density and the Shannon-entropy maximizing density can be written as

Similarly, we can obtain $KL\left(\tau\|q_{K}\right)=H\left[q_{K}\right]-H\left[\tau\right]$.

By [16, 27], we obtain

and

Then the error upper bound of the terminal density can be written as