Distributed Control-Estimation Synthesis for Stochastic Multi-Agent Systems via Virtual Interaction between Non-neighboring Agents

Given the equivalent static form of the stochastic linear MAS dynamics over the time horizon $T$ (4), we seek to address the optimal distributed control problem.

Due to the structural constraints imposed on the control input space $\mathbb{F}$, Problem 1 is a highly non-convex problem, which is indeed NP-hard and a formidable computational burden (Gupta et al., 2005). To circumvent such a difficulty, we propose a concept of $virtual\ network\ topology$ that allows for the interactions between non-neighboring agents, i.e., $virtual\ interaction$ as depicted in Figure. 1.

Since the state information of the non-neighboring agent is not available, an appropriate estimator is required for each agent to obtain the estimates of non-neighboring agents’ states. Using the Bayesian approach, Kalman-like filter is adopted for estimation as we consider a linear MAS along with the Gaussian uncertainties.

The nominal recursive structure of Kalman-like filter is represented by:

$$\begin{split}{}^{i}\hat{x}(t)&={}^{i}\hat{x}^{-}(t)+L_{i}(t)H_{i}\left(Z_{i}(t)-{}^{i}\hat{x}^{-}(t)\right)\end{split}$$

(7)

where ${}^{i}\hat{x}^{-}(t):=\mathbb{E}\left[x(t)|Z_{i,(0:t-1)}\right]$ denotes the predicted state estimate from the $i^{th}$ agent’s perspective. $H_{i}\in\mathrm{R}^{n|\Omega_{i}|\times nN}$ only encodes the neighbor of the $i^{th}$ agent, that is, $H_{i}=[h_{1}\ h_{2}\ \cdots h_{|\Omega_{i}|}]^{\mathrm{T}}\otimes I_{n}$ where $h_{m}\in\mathbb{R}^{N}$, and $m={1,2,...,|\Omega_{i}|}$ are the nonzero column vectors of the matrix $diag(c_{i1},c_{i2}....,c_{iN})$. And, $L_{i}(t)\in\mathbb{R}^{nN\times n|\Omega_{i}|}$ represents the estimator gain at time step $t$ for estimating the states of the MAS from the $i^{th}$ agent’s perspective. Once the entire MAS state estimate information becomes available for each agent, one can replace (5) with the estimation-based feedback control law. Accordingly, Problem 1, distributed control law subject to structural constraint, can be reformulated into the problem that simultaneously designs both distributed control and distributed estimator.

Albeit Problem 2 can successfully relax the structural constraint on the control law $\mathcal{F}$, it is not straightforward to solve as the control law and the state estimation errors mutually affect each other (Kwon and Hwang, 2018). To resolve such complexity, we propose an iterative optimization procedure in a distributed fashion such that: i) divide the primal problem (Problem 2) into the set of sub-problems, each is viewed from an individual agent’s perspective; ii) sequentially design the distributed estimation and control laws for each sub-problem; iii) mix the results from individual sub-problems to approximate the optimal solution to the primal problem. The overall schematic of the proposed distributed control-estimation synthesis is delineated in Figure. 2.

To begin with, the distributed estimation algorithm is optimized by means of estimator gains $\Upsilon_{i}^{(l)},\forall i\in\mathcal{V}$. As offline design phase, individual estimators can be designed based on the entire MAS model information along with the control law computed from the previous iteration, ($A$, $B$, and $\mathcal{F}^{(l-1)}$). For brevity, we use $\mathcal{F}$ to designate $\mathcal{F}^{(l-1)}$ in this subsection. Recalling (7), the distributed estimator embedded in the $i^{th}$ agent calculates ${}^{i}\hat{x}(t),\forall t\in\{0,\cdots,T\}$, whose performance can be measured by the estimation error.

Based on the prior knowledge, the estimation based control input of the $i^{th}$ agent at time step $t$ can be written by:

$$u_{i}(t)=M_{i}\sum_{k=0}^{t}\mathcal{F}_{kt}\ {}^{i}\hat{x}(k)$$

(9)

where $\mathcal{F}_{kt}\in\mathbb{R}^{pN\times nN}$ is the block matrix which spans from $(knN)^{th}$ to $(knN+nN-1)^{th}$ columns and from $(kpN)^{th}$ to $(kpN+pN-1)^{th}$ rows of the control law matrix $\mathcal{F}$. With (9), the entire MAS dynamics (2) can be expressed by:

$$\begin{split}x(t+1)&=\tilde{A}x(t)+\tilde{B}\mathcal{F}_{tt}x(t)+\sum_{k=0}^{t-1}\tilde{B}\mathcal{F}_{kt}x(k)-\sum_{k=0}^{t}\bar{B}\tilde{M}\tilde{\mathcal{F}}_{kt}e(k)+\tilde{w}(t)\end{split}$$

(10)

where $\bar{B}=1_{N}^{\mathrm{T}}\otimes\tilde{B}$, $\tilde{M}=blkdg(M_{1}^{\mathrm{T}}M_{1},\dots,M_{N}^{\mathrm{T}}M_{N})\in\mathbb{R}^{NpN\times NpN}$, and $\tilde{\mathcal{F}}_{kt}=I_{N}\otimes\mathcal{F}_{kt}$. $blkdg(\bullet)$ denotes a block-diagonal matrix with block matrices $\bullet$, and the vector $1_{N}\in\mathbb{R}^{N}$ indicates every element equals to $1$. Then the predicted state estimate of the entire MAS from the $i^{th}$ agent’s perspective is given by:

$$\begin{split}{}^{i}\hat{x}^{-}(t+1)=\tilde{A}\ {}^{i}\hat{x}(t)+\tilde{B}\mathcal{F}_{tt}{}^{i}\hat{x}(t)+\sum_{k=0}^{t-1}\tilde{B}\mathcal{F}_{kt}{}^{i}\hat{x}(k)\end{split}$$

(11)

Subtracting (11) from (10), and concatenating the results for all agents in $\mathcal{V}$ gives:

$$\begin{split}e^{-}(t+1)&=\Lambda_{t}e(t)+\sum_{k=0}^{t-1}\Psi_{kt}e(k)+1_{N}\otimes\tilde{w}(t)\\ \mathrm{where}\ \Lambda_{t}&=I_{N}\otimes(\tilde{A}+\tilde{B}\mathcal{F}_{tt})-1_{N}\otimes\bar{B}\tilde{M}\tilde{\mathcal{F}}_{tt},\ \ \Psi_{kt}=I_{N}\otimes\tilde{B}\mathcal{F}_{kt}-1_{N}\otimes\bar{B}\tilde{M}\tilde{\mathcal{F}}_{kt}\end{split}$$

(12)

Correspondingly, $\Sigma^{-}(t+1)$ is represented by:

$$\begin{split}\Sigma^{-}(t+1)&=\Lambda_{t}\Sigma(t)\Lambda_{t}^{\mathrm{T}}+\Sigma_{\tilde{w}}(t)+\sum_{q=0}^{t-1}\Lambda_{t}\mathbb{E}[e(t)e(q)^{\mathrm{T}}]\Psi_{qt}^{\mathrm{T}}+\sum_{p=0}^{t-1}\Psi_{pt}\mathbb{E}[e(p)e(t)^{\mathrm{T}}]\Lambda_{t}^{\mathrm{T}}+\sum_{p=0}^{t-1}\sum_{q=0}^{t-1}\Psi_{pt}\mathbb{E}[e(p)e(q)^{\mathrm{T}}]\Psi_{qt}^{\mathrm{T}}\end{split}$$

(13)

where $\Sigma_{\tilde{w}}(t)=(1_{N}1_{N}^{\mathrm{T}})\otimes blkdg(\Theta_{1}(t),\dots,\Theta_{N}(t))$. Note that the summation terms in the RHS of (13) (e.g., $\mathbb{E}[e(p)e(q)^{\mathrm{T}}],q\neq p$) imply the correlations of state estimates over time induced by the control law (9). Now, the predicted error, (7) can be rewritten by:

$$\begin{split}{}^{i}\hat{x}(t+1)=&{}^{i}\hat{x}^{-}(t+1)+L_{i}(t+1)H_{i}({}^{i}e^{-}(t+1)+v_{i}(t+1))\end{split}$$

(14)

Like the Kalman gain, $L_{i}(t+1)$ can be computed in a way minimizing the mean-squared error of the state estimate, i.e., $\mathbb{E}\left[\left\|{}^{i}e(t+1)\right\|^{2}\right]$. This is in fact equivalent to minimizing the trace of the posterior covariance matrices, i.e., $\mathrm{Tr}\left({}^{i}\Sigma(t+1)\right),\forall i\in\mathcal{V}$. By the definition of ${}^{i}\Sigma(t+1)$, we have:

$$\begin{split}{}^{i}\Sigma&(t+1):=\mathbb{E}[{}^{i}e(t+1){}^{i}e(t+1)^{\mathrm{T}}|Z_{i,(0:t+1)}]\\ =&\left(I_{nN}-L_{i}(t+1)H_{i}\right){}^{i}\Sigma^{-}(t+1)(I_{nN}-L_{i}(t+1)H_{i})^{\mathrm{T}}+L_{i}(t+1)H_{i}{}^{i}\Xi(t+1)(L_{i}(t+1)H_{i})^{\mathrm{T}}\end{split}$$

(15)

where

$$\begin{split}L_{i}(k+1)&={}^{i}\Sigma^{-}(t+1)H_{i}^{\mathrm{T}}(S_{i}(t+1))^{-1}\\ S_{i}(t+1)&=H_{i}({}^{i}\Sigma^{-}(t+1)+{}^{i}\mathrm{\Xi}(x+1))H_{i}^{\mathrm{T}}\\ {}^{i}\mathrm{\Xi}(t+1)&=blkdg(\mathrm{\Xi}_{i1}(t+1),\ \mathrm{\Xi}_{i2}(t+1),\dots\mathrm{\Xi}_{iN}(t+1))\end{split}$$

(16)

Correspondingly, $\Sigma(t+1)$ can be updated by:

$$\begin{split}\Sigma(t+1)&=(I-\tilde{L}(t+1))\Sigma^{-}(t+1)(I-\tilde{L}(t+1))^{\mathrm{T}}+\tilde{L}(t+1)\Sigma_{\Xi}(t+1)\tilde{L}(t+1)^{\mathrm{T}}\end{split}$$

(17)

where $\Sigma_{\Xi}(t+1)=blkdg({}^{1}\mathrm{\Xi}(t+1),\dots,{}^{N}\mathrm{\Xi}(t+1))$, and $\tilde{L}(t+1)=blkdg(L_{1}(t+1)H_{1},\dots,L_{N}(t+1)H_{N})$. Note that, the covariance between the state estimation errors at current and past steps, i.e., $\mathbb{E}[e(t+1)e(s)^{\mathrm{T}}]$ and $\mathbb{E}[e(s)e(t+1)^{\mathrm{T}}],\forall s<t$ need to be updated using the computed $\tilde{L}(t+1)$. The cross-covariance between the $i^{th}$ and the $j^{th}$ agents’ state estimates ${}^{ij}\Sigma(t+1):=\mathbb{E}[{}^{i}e(t+1){}^{j}e(t+1)^{\mathrm{T}}]\in\mathbb{R}^{Nn\times Nn}$ is at the off-diagonal block entry, while ${}^{i}\Sigma(t+1)\in\mathbb{R}^{Nn\times Nn}$ is at the diagonal block entry of $\Sigma(t+1)\in\mathbb{R}^{NnN\times NnN}$. The detailed expansions of $\Sigma(t)$ is as follows:

$$\begin{split}\Sigma(t)&=\left[\begin{matrix}{}^{1}\Sigma(t)&{}^{12}\Sigma(t)&\cdots&{}^{1N}\Sigma(t)\\ {}^{21}\Sigma(t)&{}^{2}\Sigma(t)&\cdots&{}^{2N}\Sigma(t)\\ \vdots&\vdots&\ddots&\vdots\\ {}^{N1}\Sigma(t)&{}^{N2}\Sigma(t)&\cdots&{}^{N}\Sigma(t)\end{matrix}\right]\end{split}$$

Based on the computed $\Upsilon_{i}$ from (16), each agent can update ${}^{i}\hat{x}(t)$, ${}^{i}\Sigma(t)$ and $\Sigma(t)$, respectively using (14), (15), and (17). This completes the implementation of the distributed estimation algorithm.

Apparently, the objective cost (19) has high-dimensional, highly coupled optimization variables, i.e., $\mathcal{F}$, which is our main interest, and $\Sigma_{ij},\forall i,j\in\mathcal{V}$, which are the implicit functions of both $\mathcal{F}$ and $\Upsilon_{i},\forall i\in\mathcal{V}$. The proposed iterative optimization procedure alleviates these coupling complexities in two aspects. First, akin to the alternating direction method of multipliers (ADMM) technique (Lin et al., 2013), we set $\Sigma_{ij},\forall i,j\in\mathcal{V}$ constant while optimizing $\mathcal{F}$ at the $l^{th}$ iteration, thereby treating $J$ as the function of $\mathcal{F}$ only. Note that $\Upsilon_{i},\forall i\in\mathcal{V}$ is designed over the constant $\mathcal{F}$ in the distributed estimator design phase of the next iteration. Second, we interpret the global objective cost from the individual agent’s viewpoint, and translate the primal problem (Problem 2) into the agent-wise objective cost. The objective cost, locally seen by the $i^{th}$ agent’s viewpoint at the $l^{th}$ iteration, is denoted by ${}^{i}J^{(l)}$ which can be constructed using the estimated MAS input ${}^{i}u^{(l)}={}^{i}\mathcal{F}^{(l)}C(x-{}^{i}e)$ instead of (18). ${}^{i}\mathcal{F}^{(l)}$ is constructed from the $i^{th}$ agent’s perspective by optimizing the agent-wise objective cost, ${}^{i}J^{(l)}$. Then, the resulting agent-wise optimization problem is represented as follows: