Anytime Control under Practical Communication Model

We consider wireless fading channels for the S-C and C-A transmissions [15, 16, 17], where wireless channel status varies with time due to multi-path propagation and shadowing caused by obstacles affecting the radio-frequency (RF) wave propagation. The time-varying channel conditions can be modeled as Markov processes [14]. Furthermore, in practice, the two channel conditions can be correlated, named as spatial correlation, which is caused by the same environment obstacles [18].

For wireless packet transmissions, there exists a fundamental tradeoff between reliability, i.e., the packet drop probability, and data rate, which determines the amount of information bits that a packet can carry [19]. For a fixed channel condition, increasing the data rate of a packet can lead to a higher packet drop probability. For a good channel condition, one can reduce the packet drop probability while maintain the fixed data rate. Alternatively, one can increase the data rate while maintaining the packet drop probability [20]. In this work, we will keep the data rate fixed for the S-C transmissions, since there is nothing gained from aggregating past sensor measurements in the state feedback case. However, it is well known that sending control sequences can be beneficial to compensate for packet dropouts. To accommodate this in a fading channel environment, we allow C-A transmissions to contain packets of varying data rate. The rates depend on the channel condition and provide a guaranteed packet-dropout probability. Thus, a longer control sequence can be transmitted to the actuator under a better channel condition with the same reliability.

C-A channel. At time slot $t$, the controller can at most transmit $B(t)\in\mathcal{B}\triangleq\{0,1,\cdots,\bar{B}\}$ commands to the actuator with a guaranteed packet drop probability $\bar{\gamma}$. In other words, $B(t)$ denotes the C-A channel quality. In this sense, $B(t)$ can be treated as the capacity of the channel under the packet drop probability requirement $\bar{\gamma}$. Let $\gamma(t)=1$ and $\gamma(t)=0$ denote the successful and failed transmissions in time slot $t$.

S-C channel. At time slot $t$, the S-C channel power gain takes values from $\{h_{1},h_{2},\cdots,h_{\bar{B}^{\prime}}\}$. Let $B^{\prime}(t)\in\mathcal{B}^{\prime}\triangleq\{1,2,\cdots,\bar{B}^{\prime}\}$ denote the index of the channel power gain. Thus, $B^{\prime}(t)$ denotes the S-C channel quality. Let $\gamma^{\prime}(t)=1$ and $\gamma^{\prime}(t)=0$ denote the successful and failed transmissions in time slot $t$. The packet drop probability at time $t$ is

where $g(\cdot)$ is the packet drop probability function in terms of the channel power gain.

Then, we assume that the joint C-A and S-C channel condition $\{(B,B^{\prime})\}_{\mathbb{N}}$ is a time-homogeneous Markov process, and the state transition probability is given as

When considering perfect transmission between the controller and the actuator, as in [10, 11, 9], the system only needs one buffer at the controller to store the computed control commands. If imperfect transmissions are taken into account, it is convenient to include a command buffer at the actuator to provide robustness against packet dropouts, see e.g. [22] for a general packetized predictive control method³³3Markovian communication and computational resources were not considered in [22].. Clearly, the dual-buffer system introduces a more complex state updating process.

Let $\Lambda_{c}$ and $\Lambda_{a}$ denote the length of the controller’s and the actuator’s buffer, respectively. Then, the buffer state at the controller after its transmission phase is denoted as

where $\mathbf{b}_{c,i}(t)\in\mathbb{R}^{l_{u}},\forall i\in\{1,2,\cdots,\Lambda_{c}\}$. The buffer state at the actuator right after the C-A transmission but before the implementation of a control command, i.e., the pre-control buffer, is denoted as

where $\mathbf{b}_{a,i}(t)\in\mathbb{R}^{l_{u}},\forall i\in\{1,2,\cdots,\Lambda_{a}\}$. In general, the buffers $\vec{\mathbf{b}}_{a}(t)$ and $\vec{\mathbf{b}}_{c}(t)$ keep the calculated sequences of control command, and the buffer updating rules will be given in the following part.

The control input is the first element in the actuator’s buffer, i.e.,

which can be treated as the previously predicted control command for the current time slot. The buffer state at the actuator right after the control implementation, i.e., the post-control state, is

where the buffer shift matrices are defined as

Let $N(t)$ denote the number of calculated tentative future control commands at time $t$. The process $\{N\}_{\mathbb{N}}$ is a time-homogeneous Markov process with the transition probability

where $\mathcal{N}\triangleq\{0,1,\cdots,\bar{N}\}$. It is assumed that $\bar{N}\leq\Lambda_{c}$.

The controller’s operations are described as

1. 1.

   If $\gamma^{\prime}(t)N(t)>0$, the controller has a new update from the sensor and is available for computation. In this case, it discards all the existing commands in its buffer and generates a sequence of $N(t)$ control commands to control the plant in time slots $t$ to $(t+N(t)-1)$. The sequence of tentative controls is

   $$\vec{\mathbf{u}}(t)=[\mathbf{u}_{1}(t)^{\top},\mathbf{u}_{2}(t)^{\top},\cdots,\mathbf{u}_{N(t)}(t)^{\top}]^{\top}.$$

   (10)

   The buffer state before transmission is written as

   $$[\vec{\mathbf{u}}(t)^{\top},\underbrace{\mathbf{0}_{l_{u}\times 1}^{\top},\cdots,\mathbf{0}_{l_{u}\times 1}^{\top}}_{\Lambda_{c}-N(t)}]^{\top}.$$

   Specifically, the controller calculates the control sequence based on the anytime control algorithm proposed in [9], which is rewritten as

   		$\displaystyle\mathbf{u}_{i}(t)=\kappa(\mathbf{x}_{i}^{\prime}(t)),i=1,\cdots,N(t)$		(11)
   		$\displaystyle\mathbf{x}^{\prime}_{i}(t)=\begin{cases}\mathbf{x}(t),&i=1\\ f(\mathbf{x}^{\prime}_{i-1}(t),\mathbf{u}_{i-1}(t)),&i=2,\cdots,N(t)\end{cases}$

   where $\kappa(\cdot)$ is defined in Assumption 1.

   Considering the C-A channel capacity and the actuator’s buffer length, the controller transmits $\min\{B(t),N(t),\Lambda_{a}\}$ commands to the actuator. If the transmission is successful, the buffer shifts by $\min\{B(t),N(t),\Lambda_{a}\}$ steps. Otherwise, the controller erases its buffer. This is because the first computed control command $\mathbf{u}_{1}(t)$ cannot be implemented in the current time slot $t$, and the rest of computed control commands, which are calculated based on the successful implementation of the first control command, become useless. It is clear that at time instances where the actuator has run out of buffer contents, we have $\vec{\mathbf{b}}_{a}(t)=\mathbf{0}$ and $\mathbf{u}(t)=\mathbf{0}\neq\mathbf{u}_{1}(t)$. For the case that $\vec{\mathbf{b}}_{a}(t)\neq\mathbf{0}$, the predicted control input $\mathbf{u}(t)$ is equal to currently calculated control command $\mathbf{u}_{1}(t)$ only in the perfect process-noise-free scenario, and $\mathbf{u}(t)\neq\mathbf{u}_{1}(t)$ in general.

2. 2.

   If $\gamma^{\prime}(t)=0$, the controller does not have a new update from the sensor. In this case, it does not generate any new control command. If $N(t)=0$, the controller does not have the computational resource to generate any new control command.
   In these two cases, if the plant is out of control in the previous time slot, i.e., $\vec{\mathbf{b}}_{a}(t-1)=\mathbf{0}$, the controller erases its buffer due to the same reason in case 1); otherwise, the controller sends the buffered commands to the actuator as much as it can, subject to the constraints of the C-A channel capacity and the actuator’s buffer length.

The actuator’s operations are described as

1. 1.

   If $\gamma(t)=0$, the actuator’s buffer is shifted by one step, i.e., $\vec{\mathbf{b}}_{a}(t)=\mathbf{S}_{a}\vec{\mathbf{b}}_{a}(t-1)$, since the first command in the buffer of the previous time slot was used for control.

2. 2.

   If $\gamma(t)=1$ and $\gamma^{\prime}(t)N(t)>0$, the actuator erases the previous commands and stores the received ones. This is because the newly calculated control commands are expected to perform better than the previous calculated commands, especially when the process noise has a large variance. ⁴⁴4 Note that when the number of tentative future commands in the newly received packet is less than the previous ones, we still need to erase all the previous commands, not just part of them. The reason is that each buffered control command at the actuator was calculated assuming the successful implementation of the previously calculated control command. Thus, if the first few control commands at the actuator’s buffer are removed (i.e., cannot be applied for control), then the rest of the buffer is of little use.

3. 3.

   If $\vec{\mathbf{b}}_{a}(t-1)=\mathbf{0}$ and $\gamma^{\prime}(t)N(t)=0$, no operation on the actuator’s buffer is required as there is no new commands transmitted.

4. 4.

   If $\vec{\mathbf{b}}_{a}(t-1)\neq\mathbf{0}$, $\gamma^{\prime}(t)=1$ and $\gamma^{\prime}(t)N(t)=0$, the actuator shifts its buffer by one step due to the same reason in 1) and stores the received commands in the buffer right after the existing commands.

Let $\lambda_{c}(t)$ and $\lambda_{a}(t)$ denote the effective buffer lengths at the controller and the actuator, respectively. Intuitively, $\lambda_{c}(t)$ and $\lambda_{a}(t)$ jointly determine the closed-loop performance of plant, as a larger $\lambda_{c}(t)$ and a larger $\lambda_{a}(t)$ indicate that the plant will be delivered tentative control values for a longer time. Based on the controller’s and the actuator’s operations, the actuator is not necessary to have a larger buffer than the controller. Thus, we assume that $\Lambda_{a}\leq\Lambda_{c}$.

Let $L(t)$ denote the number of tentative commands to be transmitted. Based on the controller’s operations, $L(t)$ can be written as

Then, the buffer-updating rules based on the controller’s and the actuator’s operations are

and

From (13) and (14), the updating rules of the effective buffer lengths $\lambda_{c}(t)$ and $\lambda_{a}(t)$ are

and

For ease of analysis, we need the following assumption about the aggregated process $\{(B,B^{\prime},N)\}_{\mathbb{N}_{0}}$.

Note that wireless fading channel conditions are commonly modeled as IA Markov processes [14, 15, 23, 24], thus it is reasonable to consider an IA Markov process $\{(B,B^{\prime})\}_{\mathbb{N}_{0}}\in\mathcal{B}\times\mathcal{B}^{\prime}$. Also the computation availability $\{N\}_{\mathbb{N}_{0}}\in\mathcal{N}$ is commonly modeled as an IA Markov process [10]. Since an aggregation of IA Markov processes is still an IA Markov process [25], it is reasonable to consider the IA Markov process $\{(B,B^{\prime},N)\}_{\mathbb{N}_{0}}$.

For the initial state of $\{Z\}_{\mathbb{N}_{0}}$, it is practical to assume that both buffers are empty:

From Lemma 1, $\{Z\}_{\mathbb{N}_{0}}$ has a unique stationary distribution. We shall denote the state transition probability of $\{Z\}_{\mathbb{N}_{0}}$ as per

Note that $v_{i,j}$ can be numerically calculated based on (3), (9), (15) and (16), though it does not have a closed-from expression due to the complexity introduced by the dual-buffer updating process (15) and (16).

Let $\mathbf{V}\in\mathbb{R}^{S\times S}$ denote the state transition probability matrix, i.e., $[\mathbf{V}]_{i,j}=v_{i,j}$, and

where $\mathbf{V}_{0,0}\in\mathbb{R}^{S_{0}\times S_{0}}$, $\mathbf{V}_{0,1}\in\mathbb{R}^{S_{0}\times(S-S_{0})}$, $\mathbf{V}_{1,0}\in\mathbb{R}^{(S-S_{0})\times S_{0}}$ and $\mathbf{V}_{1,1}\in\mathbb{R}^{(S-S_{0})\times(S-S_{0})}$.

Similar to [10, 11, 9], the stability of the WNCS depends on the statistics of $\{\Delta_{n}\}_{n\in\mathbb{N}}$ in (18), which denotes the time duration between consecutive open-loop events. Different to [10, 9], in the considered case, the process $\{\Delta_{n}\}_{n\in\mathbb{N}}$ is not i.i.d. For WNCS with a single channel, in [11] an event-triggered setup was considered, leading to $\{\Delta_{n}\}_{n\in\mathbb{N}}$ which is not i.i.d. However, our current setup is different from [11]. In particular, $\{\Delta_{n}\}_{n\in\mathbb{N}}$ is formed by the first return time of a set of states in $\mathcal{S}_{0}$ rather than a single state (as in [11]) . Therefore, the approach in [11] cannot be adopted directly. In the following, we propose a novel cycle-cost-based approach to obtain sufficient stability conditions.