Risk-Aware Stability of Discrete-Time Systems

Stability theory of stochastic systems has a long-standing history with contributions from the late 1950s and early 1960s [1, 2, 3] and is an especially active research area currently [4, 5, 6, 7, 8, 9]. The applications of stochastic stability theory are numerous, and we list just a few here: trajectory tracking despite modeling errors [10, p. 426], ensuring that state estimation error stays bounded [11], target tracking for partially observable systems [12], and establishing the convergence of sampling-based optimization algorithms [13, 14].

Stochastic stability theory typically assesses system behavior by evaluating the magnitude $|\cdot|$, e.g., Euclidean norm, of the state $x_{t}$ over time in probability or in expectation. Evaluating the probability of a harmful event, e.g., $|x_{t}|$ exceeds a desired threshold, provides one approach to analyze stability. To analyze stability from a different perspective, one may evaluate the moments of $|x_{t}|$. For example, the uniform exponential $p$-stability property (informally) means that the expectation of $|x_{t}|^{p}$ decays exponentially at a rate independent of the initial condition. However, instead of merely analyzing $E(|x_{t}|^{p})$, we may wish to analyze additional characteristics of the distribution of the state energy $|x_{t}|^{p}$ over time. For instance, the state energy may represent parameter estimation error in stochastic gradient descent, state estimation error in a Kalman filter, or trajectory tracking error of a robotic system. Rarer larger realizations of the state energy may arise in practice but may be concealed at the analysis stage by restricting one’s attention to the expectation of $|x_{t}|^{p}$. There may be incomplete knowledge about the system model, suggesting the importance of analyzing the state energy under distributional ambiguity. Also, the classical $p$-stability property ignores the variability of the state energy with respect to its mean, whereas it may be desirable to stabilize such variability in practice.

Ultimately, we wish to design control systems whose energy dynamics enjoy specific desirable distributional characteristics. For instance, we may wish the state energy to decay in expectation conditioned on a given fraction of the worst cases. In other settings, we may wish the variance of the state energy to decay, or we may wish the expected state energy to decay in a distributionally robust sense. To design such control systems in the future, we require techniques to analyze state energy distributions from more diverse perspectives.

In the literature, the term stability refers to different properties, for example, uniform boundedness of the state trajectory in probability [15, (D4), p. 31], driving the state to the origin within a given time period (i.e., prescribed-time stabilization) [8], and input-to-state and noise-to-state stability concepts. Since Sontag’s pioneering work in the setting of deterministic systems in the late 1980s [16], stochastic input-to-state and noise-to-state stability notions have been developed, e.g., see [17, 18, 19, 20, 21, 22, 6, 7]. In particular, deterministic input-to-state stability was extended to stochastic noise-to-state stability by Krstić, Deng, and colleagues in the late 1990s and early 2000s [17, 19]. In the current work, we emphasize uniform stability notions of the form

where $\rho$ is a map from random variables to the extended real line, $\psi$ is a state energy function, $\mathbf{x}\in\mathbb{R}^{n}$ is an initial condition, and $\lambda\in[0,1)$, $a\geq 0$, and $b\in\mathbb{R}$ are parameters. If $b$ depends on a disturbance process, e.g., $b=\sup_{t\in\{0,1,\dots\}}E(|w_{t}|^{2})<\infty$, then the above stability notion describes a noise-to-state stability property.

In the current work, we develop a uniform stability framework for the analysis of diverse characteristics of the state energy’s distribution by permitting a fairly general map $\rho$ in (1) called a risk functional. A risk functional (i.e., risk measure) is a map from a space of random variables representing costs to the extended real line [23, p. 261]. Exponential utility is the classical risk functional in the control systems literature pioneered by Whittle in the 1980s and 1990s [24, 25, 26]. Other seminal works about exponential utility include those by Howard and Matheson [27] and by Jacobson [28] in the early 1970s. The exponential utility $\rho_{\text{e},\theta}(Z)\coloneqq\textstyle\frac{-2}{\theta}\log E(\exp(\frac{-\theta}{2}Z))$ of a nonnegative random variable $Z$ with $\theta\neq 0$ is interpreted as a mean-variance approximation, i.e., if the magnitude of $\theta\textbf{var}(Z)$ is small enough, then $\rho_{\text{e},\theta}(Z)\approx E(Z)-\textstyle\frac{\theta}{4}\textbf{var}(Z)$ [24, p. 765]. From a viewpoint based on expected utility theory [29], $\rho_{\text{e},\theta}$ assumes that an exponential function represents the user’s preferences. Such assumptions need not be appropriate for every application, motivating investigations of additional risk-aware criteria.

Building on the above contributions, additional types of risk functionals have been developed [30, 31, 23, 32]. A real-valued coherent risk functional satisfies four useful properties: convexity, monotonicity, translation equivariance $\rho(Z+a)=\rho(Z)+a$ if $a\in\mathbb{R}$, and positive homogeneity $\rho(\lambda Z)=\lambda\rho(Z)$ if $\lambda>0$ [30], [23, Ch. 6.3]. Such a functional also enjoys a distributionally robust representation [23, Th. 6.6]. A common example is the conditional value-at-risk of an integrable random variable $Z$, which represents the expectation of $Z$ in a given fraction of the worst cases [23, Th. 6.2]. A mean-dispersion risk functional takes the form $\rho(Z)=E(Z)+\lambda D(Z)$, where $\lambda\geq 0$ and $D$ is a measure of dispersion relative to $E(Z)$. Examples of $D$ include variance, standard deviation, and upper-semideviation, i.e., $E(\max\{Z-E(Z),0\})$. A recursive risk functional of $Z=Z_{0}+Z_{1}+Z_{2}$ takes the form $\rho(Z)=\rho_{0}(Z_{0}+\rho_{1}(Z_{1}+\rho_{2}(Z_{2})))$, where $\rho_{i}$ is a map between spaces of random variables.¹¹1To introduce a recursive risk functional, we have used a time horizon of length $N=2$ for clarity. More generally, one can use a finite time horizon of length $N\in\mathbb{N}$, an arbitrary natural number, or one can use an infinite time horizon [32, 14]. While the structure of a recursive risk functional $\rho$ facilitates the development of risk-aware algorithms, it can be difficult to interpret $\rho$, except in special cases, such as when $\rho$ is the expectation. We will further describe specific risk functionals in Section 2.

To meet application-specific needs and preferences about managing uncertainty, it is desirable to have at hand a broad collection of maps that assess different distributional characteristics. This is one motivation for the theory of risk functionals and the pursuit of additional research in the intersection of risk functionals and control systems. Research in this intersection has been gaining much momentum in recent years. There are new contributions, for example, in risk-aware mean-field games [33, 34], model predictive control [35, 36], temporal logic [37, 38], barrier functions [39], and optimal-control-based safety analysis [40, 41]. We refer the reader to our recent survey article about risk-aware optimal control theory [42] for additional literature. We will focus the remainder of the literature review on existing work related to risk-aware stability theory, which has been less studied.

Consider a fully observable stochastic discrete-time system

where $f:\mathbb{R}^{n}\times\mathbb{R}^{d}\rightarrow\mathbb{R}^{n}$ is a Borel-measurable function, $n$ and $d$ are natural numbers, $(x_{0},x_{1},\dots)$ is an $\mathbb{R}^{n}$-valued stochastic process, and $(w_{0},w_{1},\dots)$ is an $\mathbb{R}^{d}$-valued independent stochastic process. The processes are defined on an arbitrary probability space $(\Omega,\mathcal{F},P)$. We assume that the initial state is fixed at an arbitrary vector $\mathbf{x}\in\mathbb{R}^{n}$.²²2More precisely, we assume that the distribution of $x_{0}$ is the Dirac measure concentrated at $\mathbf{x}$, where $\mathbf{x}$ can be any vector in $\mathbb{R}^{n}$.

There are many types of energy-like functions in control theory, e.g., class-$\mathcal{K}$ functions, locally positive definite functions, and decrescent functions [53, Sec. 5.3.1]. We will find the following energy-like function useful in this work.

It is natural to choose $\psi$ so that $\psi(x)$ increases as $|x|$ increases. However, we do not need this condition in the current section. In this section, we will study the relationships between different types of risk-aware stability notions for any $\psi$ satisfying Definition 1. In contrast, in Section 3, we will specialize to $\psi(x)=x^{\top}Rx$ with $R\in\mathcal{S}_{n}^{+}$.

The next definition is a non-risk-aware (i.e., risk-neutral) uniform stability property. Variations of the definition to follow are available from [51, Def. 1], [50, Def. 1], and [10, Def. 7.3.8], for example.

Definition 2 describes a uniform property because $(\lambda,a,b)$ does not depend on $\mathbf{x}$ or $t$. While one can also consider a definition in which $(\lambda,a,b)$ can depend on $\mathbf{x}$ or $t$, we focus on the uniform case in this work. Definition 2 describes a local property if $S$ is a strictly proper subset of $\mathbb{R}^{n}$ and a global property otherwise. If $\psi(x)=|x|^{2}$ and $b=0$, then Definition 2 is uniform exponential mean-square stability. If $b=c\sup_{t\in\mathbb{N}_{0}}E(w_{t}^{\top}Mw_{t})<\infty$ for some $c\in(0,\infty)$ and $M\in\mathcal{S}_{n}^{+}$, then Definition 2 describes a risk-neutral noise-to-state stability property. (Additional risk-neutral noise-to-state stability definitions have been introduced in the literature; see [17, Def. 4.1] and [22, Def. 2] for two examples.)

As described in the Introduction, we are interested in analyzing the state energy over time, which is a random process $(\psi(x_{1}),\psi(x_{2}),\dots)$. If we only consider the average state energy, as in Definition 2, then we ignore other characteristics of the distribution of the state energy, including the dispersion with respect to the mean (e.g., variance) and the shape of the upper tail (e.g., the mean in a fraction of the worst cases). Hence, we will develop a risk-aware stability concept to facilitate the analysis of a broader range of distributional characteristics of the state energy.

To generalize Definition 2, the key observation is that the expectation is a map from a family of random variables to $\bar{\mathbb{R}}$. Hence, we can consider maps that characterize additional features of the distribution of a random variable. Let $\mathcal{Z}$ be a family of random variables defined on $(\Omega,\mathcal{F},P)$, and let $\rho$ be a map from $\mathcal{Z}$ to $\bar{\mathbb{R}}$.³³3We prefer smaller realizations of any $Z\in\mathcal{Z}$. That is, $Z$ represents a random cost rather than a random reward. We assume that $\rho(Z)\in\mathbb{R}\cup\{\infty\}$ for every $Z\in\mathcal{Z}$, and we assume that there exists a $Z\in\mathcal{Z}$ such that $\rho(Z)$ is finite, following convention [23, p. 261]. The term risk functional (i.e., risk measure) invokes these assumptions implicitly. $\rho$ is called a risk functional (i.e., risk measure), and several examples will follow. The precise definition of $\mathcal{Z}$ will depend on the risk functional of interest. While Examples 1–3 can be found in [23], they are necessary to present to keep the current work self-contained.

The value-at-risk and the conditional value-at-risk assess risk in terms of quantiles. The following functionals assess risk in terms of dispersions relative to the mean.

Conditional value-at-risk and the mean-dispersion functionals of Example 3 for particular choices of $q$ and $\beta$ belong to the class of real-valued coherent risk functionals [23, Ch. 6.3]. As described previously, such a functional satisfies four desirable properties and also admits a dual representation as a distributionally robust expectation (to be presented). Let $\varrho$ be a real-valued coherent risk functional defined on $\mathcal{L}^{q}$ with $q\in[1,\infty)$. Then, there is a bounded family $\mathcal{A}\subset\mathcal{L}^{q*}$ of densities such that

by one direction of [23, Th. 6.6].⁴⁴4Moreover, the existence of a dual representation (9) implies that $\varrho$ is a real-valued coherent risk functional; for technical details and information about additional properties of $\mathcal{A}$, please see [23, Th. 6.6]. In particular, every $\xi\in\mathcal{A}$ satisfies $\xi\in\mathcal{L}^{q*}$, $E(\xi)=1$, and $\xi\geq 0$ a.e. [23, Eq. (6.38)]. Using (9), one can show that if $Z=0$ a.e., then $\varrho(Z)=0$. We will find the exact forms of $\mathcal{A}$ for Examples 2 and 3 useful for Theorem 1 to come.

The next risk functional $\rho_{\nu}$ to be described, which takes inspiration from [44], is related to Example 3 because it also assesses risk in terms of a dispersion relative to the mean. While $\rho_{\nu}$ is noncoherent in general, it enjoys a special meaning in dynamical systems applications, as it depends on a sub $\sigma$-algebra, which can encode the history of a stochastic process.

Any risk functional, e.g., see Examples 1–4, can be incorporated into the following definition.

In Section 3, we will see how $b$ can depend on the disturbance process and the chosen measure of risk, leading to a risk-aware noise-to-state stability property.⁵⁵5In this work, we do not need to write Definition 3 in terms of the existence of class-$\mathcal{K}$ and class-$\mathcal{K}\mathcal{L}$ functions. In future work, using these function classes will likely be useful for extending the results of Section 3 (Theorem 3.3 and Theorem 3.7) to nonlinear systems. In Definition 3, the condition $\psi(x_{t})\in\mathcal{Z}$ for every $t\in\mathbb{N}$ is required because the domain of $\rho$ need not be the entire family of random variables on $(\Omega,\mathcal{F},P)$, e.g., see Examples 2–4. In the special case of $\rho$ being the expectation and $\mathcal{Z}$ being the entire family of random variables on $(\Omega,\mathcal{F},P)$, Definition 3 reduces to Definition 2. While Definition 3 is inspired by [35, Def. V.1], [36, Def. 4], and [43, Def. 3.1], Definition 3 invokes a generalized viewpoint because $\rho$ can be any risk functional. In contrast, a recursive risk functional was considered in [35, Def. V.1] and [36, Def. 4], and a distributionally robust CVaR functional was considered in [43, Def. 3.1]. The next theorem will demonstrate connections between several instances of Definition 3.

The proof of Theorem 1 has five parts, one for each item. Each part is based on the properties enjoyed by the risk functional at hand.

The previous theorem provides connections between some different instances of Definition 3. In the next section, we will focus on linear systems and derive sufficient conditions for stability with respect to any real-valued coherent risk functional on $\mathcal{L}^{q}$ (Theorem 3.3). Then, we will provide a stability result in the context of the mean-conditional-variance functional on $\mathcal{L}^{1}$ (Theorem 3.7). The techniques will involve a combination of the properties of the risk functionals of interest and Lyapunov stability theory for discrete-time linear systems.

Now, we consider a stochastic linear time-invariant system, which is a special case of (3), of the form

where $A\in\mathbb{R}^{n\times n}$ is deterministic, $(x_{0},x_{1},\dots)$ is an $\mathbb{R}^{n}$-valued stochastic process, and $(w_{0},w_{1},\dots)$ is an $\mathbb{R}^{n}$-valued independent stochastic process. The processes are defined on $(\Omega,\mathcal{F},P)$. The initial state is fixed at an arbitrary vector $\mathbf{x}\in\mathbb{R}^{n}$. For every $t\in\mathbb{N}_{0}$, we define the random object $h_{t}\coloneqq(x_{0},x_{1},\dots,x_{t})$ and the $\sigma$-algebra $\mathcal{F}_{t}\coloneqq\sigma(h_{t})$ induced by $h_{t}$. Thus, $x_{t}:(\Omega,\mathcal{F}_{t})\rightarrow(\mathbb{R}^{n},\mathcal{B}_{\mathbb{R}^{n}})$. As well as $(w_{0},w_{1},\dots)$ being independent, we assume that $w_{t}$ and $h_{t}$ are independent and $E(w_{t}w_{t}^{\top})\in\mathbb{R}^{n\times n}$ for every $t\in\mathbb{N}_{0}$. The above conditions are standard and ensure that $E(|x_{t}|^{2})$ is finite for every $t\in\mathbb{N}_{0}$. Observe that $(w_{0},w_{1},\dots)$ need not be zero-mean, and the disturbance process need not be identically distributed. When we consider the linear system (19), we implicitly assume the conditions stated in this paragraph.

The first result of the section will develop sufficient conditions for stability of the linear system (19) with respect to any real-valued coherent risk functional on $\mathcal{L}^{q}$. Then, we will examine how the conditions are related to those that guarantee risk-neutral stability of (19).