Zames-Falb Multipliers: don’t panic

This article is entirely different from the review paper [6] which provides more of an overview of the state-of-the-art, and really concentrates on searches and properties of Zames-Falb multipliers. Instead, this article attempts to give a complete but understandable proof of the main results of Zames-Falb, introduced in their paper [44]. The article also tries to give some background to the work of Zames-Falb and towards the end touches on the problems associated with actually finding or computing Zames-Falb multipliers - this is biased towards my own research.

This article is not a review of absolute stability in general, nor does it cover Zames’ other significant contribution on conic sector stability analysis [42, 43] (see also [27, 3, 34]).

I have tried to keep the article as readable as possible but inevitably there is a certain amount of notation and some non-trivial concepts which need introducing in order to establish the results. Most of the notation follows that in [46]. The article tries to use simple terminology and language where possible and the treatment of some concepts (such as causality and well-posedness) are deliberately light-touch. There is a certain amount of whimsicality in the writing which is there to, hopefully, ensure the article is fairly light. The quotes from Reference [1] are hopefully appropriate and in keeping with the nature of the article.

In this article we are concerned with input-output stability and more specifically with so-called $\mathcal{L}_{2}~$stability. Central to this definition is the so-called $\mathcal{L}_{2}~$space and its extension

	$\displaystyle\mathcal{L}_{2}$	$\displaystyle:=\left\{x(t)~:~\int_{0}^{\infty}\|x(t)\|^{2}~dt<\infty\right\}$		(1)
	$\displaystyle\mathcal{L}_{2e}$	$\displaystyle:=\left\{x(t)~:~\int_{0}^{T}\|x(t)\|^{2}~dt<\infty\quad\forall\quad 0\leq T<\infty\right\}$		(2)

The space $\mathcal{L}_{2}~$represents all square integrable signals and roughly speaking represents most well behaved signals. The space $\mathcal{L}_{2e}$ represents most signals which we may have to deal with and it is important since, from an operator perspective we represent systems as mappings between $\mathcal{L}_{2e}$ spaces. For example an LTI system $\mathbf{G}$ can be considered as a mapping between these spaces, that is

$$\mathbf{G}:\mathcal{L}_{2e}\rightarrow\mathcal{L}_{2e}$$

From an operator perspective, the nonlinearity $\phi$ also maps $\mathcal{L}_{2e}$ to $\mathcal{L}_{2e}$.

With reference to Figure 1 stability is then defined with respect to the signal space $\mathcal{L}_{2}$. Specifically, given a LTI system $\mathbf{G}$ and a static nonlinearity $\phi(\cdot)$:

In other words:

“Well behaved inputs lead to well behaved outputs”

Good references on $\mathcal{L}_{2}~$stability are [10, 13] and [16]. Here it is sufficient to understand stability as defined above.

A standing assumption throughout the paper is that the LTI element in Figure 1 is stable and causal: its transfer function contains only open-left-half-plane poles. Such a system is (again following the robust control literature) labelled $\mathcal{RH}_{\infty}$. Specifically, the assumption

$$\mathbf{G}\in\mathcal{RH}_{\infty}$$

is made throughout the paper. The assumption made on the Zames-Falb multiplier, introduced shortly, will not be as strict. In this case, the only requirement is that the multiplier is bounded on the imaginary axis¹¹1Here, the multiplier is assumed to be rational, although this is not actually required.. This is often denoted as

$$\mathbf{M}\in\mathcal{RL}_{\infty}$$

and can be interpreted as $\mathbf{M}$ consisting of both causal and anti-causal elements:

$$\mathbf{M}=\mathbf{M}_{c}+\mathbf{M}_{a}$$

where $\mathbf{M}_{c}\in\mathcal{RH}_{\infty}$ and $\mathbf{M}_{a}\in\mathcal{RH}_{\infty}^{\perp}$ i.e. $\mathbf{M}_{a}$ contains poles only in the open right half complex plane. For much of this article, one may essentially ignore this detail, but it is stressed that for searches there is often a useful advantage in allowing the multiplier to have an anti-causal component [5]. On a first read this subtlety can probably be ignored.

The classical Circle and Popov criteria both consider so-called sector bounded nonlinearities in which the nonlinearity $\phi(.)$ satisfies the sector bounds

$\displaystyle\alpha_{1}\leq\frac{\phi(\sigma)}{\sigma}\leq\alpha_{2}\quad\forall\sigma\in\mathbb{R}$

(3)

where $\alpha_{1}<\alpha_{2}$ are two real, not necessarily positive constants. The short hand

$$\phi\in{\rm Sector}[\alpha_{1},\alpha_{2}]$$

is used to indicate a nonlinearity satisfying inequality (3). Many nonlinearities satisfy sector bounds: the saturation and deadzone functions being two common ones. The concept of a sector is illustrated in Figure 2; note often the lower sector bound is taken as zero (something which can be achieved with a loopshift - see [17], Chapter 10).

The main problem with the sector bounds is that many nonlinearities inhabit the same sector and thus results obtained using sector information tend to be quite conservative because they need to hold for all nonlinearities in the same sector. To overcome this, various researchers ([22, 44, 24]) have used the concept of slope restrictions to lessen the conservatism of the results. A slope restricted nonlinearity is one where the generalised derivative of the nonlinearity is assumed to lie between certain bounds. Denoting $\partial\phi(.)$ as the generalised derivative of $\phi(.)$, the slope restriction is equivalent to requiring

$$\partial\phi\in[0,\alpha]$$

which is often written as

$\displaystyle 0\leq\frac{\phi(x)-\phi(y)}{x-y}\leq\alpha\quad\forall x,y\in\mathbb{R}$

(4)

where the lower bound on the slope has been taken to be zero (again possibly by a loopshift). This therefore implies (setting $y=0$ and under the assumption that $\phi(0)=0$) that $\phi\in{\rm Sector}[0,\alpha]$, but the converse is not generally true.

Two final classes of nonlinearities are needed in order to arrive at the Zames-Falb results: the classes of monotone and odd monotone nonlinearites. These classes of nonlinearities are essentially 2nd-4th quadrant (odd) nonlinearities with a linear bound on their “gain”. They are of independent interest, but are also the stepping stone to perhaps the most useful Zames-Falb results: those devoted to slope-restricted nonlinearities. Formally, these classes of nonlinearities are defined below.

Although, strictly speaking, the above definition defines monotone non-decreasing nonlinearities, with some abuse of terminology, they are often described as monotone increasing nonlinearities: in this article no distinction will be made.

A few further concepts need emphasizing or introducing. In addition to the plant $\mathbf{G}$, it is often necessary to introduce other linear operators which will be denoted in bold font. A linear operator $\mathbf{P}$ will have an associated Laplace transform $P(s)$, Fourier transform $P(j\omega)$ and impulse response $p(t)$. It is recalled that the output (in the time-domain) of such a system subject to an input $u(t)$ may be calculated using the convolution integral

$$y(t)=\int_{-\infty}^{\infty}g(\tau)u(t-\tau)d\tau$$

When $\mathbf{G}$ is causal and the output at time $t=0$ is zero, this can be replaced by

$$y(t)=\int_{0}^{t}g(\tau)u(t-\tau)d\tau$$

The Fourier transform of a signal $u(t)$, denoted $\hat{u}(j\omega)$, is needed for the IQC approach introduced shortly.

Stability analysis based on input-output notions of passivity is intrinsically linked to the concept of an inner product and this is also used frequently in the IQC approach. The inner product of two signals $g(t)$ and $h(t)$ is defined as

$$\langle g(t),h(t)\rangle=\int_{-\infty}^{\infty}g(t)h(t)~dt$$

By Parseval’s Identity (e.g. Appendix B in [10]), this is equivalent to the frequency domain inner product

$$\langle\hat{g}(j\omega),\hat{h}(j\omega)\rangle=\frac{1}{2\pi}\int_{-\infty}^{\infty}\hat{g}^{*}(j\omega)\hat{h}(j\omega)~d\omega$$

where $(\cdot)^{*}$ represents complex conjugate transpose. Obviously if the time-domain inner product is non-negative, then so is the frequency domain inequality.

The main technical idea of the Zames-Falb approach is to ensure a certain integral is non-negative. It transpires that this is the case if the so-called $\mathcal{L}_{1}$ norm of a certain “system” (linear operator) is less than unity. Given a linear operator $\mathbf{P}$, the $\mathcal{L}_{1}$ norm is defined as

$$\|\mathbf{P}\|_{1}=\int_{-\infty}^{\infty}|p(t)|dt$$

In the subsequent sections it will be seen that this norm arises naturally in the analysis.

There are various approaches one might take to prove stability of the system in Figure 1. In the 1960s and 1970s the favoured approach was to use multipliers, which is where the term Zames-Falb multiplier originates. The idea with multipliers effectively comes from the desire to overcome limitations imposed by passivity. Referring again to Figure 1, it is well known that the (negative) feedback interconnection is stable if $\mathbf{G}$ and $\phi(\cdot)$ are both passive: actually it is more subtle, $\mathbf{G}$ must be strictly passive, but a discussion of this is out of scope. The key point to note is that passivity places a very strong requirement on $\mathbf{G}$ which is often not, and in practice rarely (never?), satisfied. To overcome this strict requirement, one can introduce a so-called multiplier $\mathbf{M}$, whose inverse $\mathbf{M}^{-1}$ is well-defined, and re-draw Figure 1 as in Figure 3. Using the same passivity arguments it follows that the interconnection is stable if the LTI system $\mathbf{\tilde{G}}=\mathbf{MG}$ is strictly passive and the new nonlinearity $\tilde{\phi}=\phi\circ\mathbf{M}^{-1}$ is passive. The advantage of this new arrangement is that by introducing the multiplier $\mathbf{M}$ the passivity requirements are on $\mathbf{GM}$ and are hopefully easier to enforce.

Remark: informality. This section has been written in an informal manner. These concepts are used extensively in classical texts describing Zames-Falb multipliers [41, 10] as well as in the original papers on the subject. However, the approach taken in this article differs somewhat to that described above. $\square$

Instead of using multipliers directly, the proofs given here are based on the IQC approach to stability analysis [21]. The IQC approach is powerful because it a) captures existing results in robust control; and b) enables one to combine different stability results into a common analysis framework: The Zames-Falb multiplier results are a good fit to this. A further advantage of the IQC results is that the approach is arguably more direct than the multiplier approach and dispenses with some of the loopshifting involved.

The key to applying IQC results is to establish that a nonlinearity satisfies an IQC. In the frequency domain the nonlinearity $\phi(\cdot)$ is said to satisfy the IQC defined by $\Pi(j\omega)$, where $\Pi(j\omega)=\Pi^{*}(j\omega)$ and ${\Pi}\in\mathcal{RL}_{\infty}$, if the following inequality holds for all $\hat{v}(j\omega)$:

$\displaystyle\int_{-\infty}^{\infty}\begin{bmatrix}\hat{v}(j\omega)\\ \widehat{\phi(v(j\omega))}\end{bmatrix}^{*}\Pi(j\omega)\begin{bmatrix}\hat{v}(j\omega)\\ \widehat{\phi(v(j\omega))}\end{bmatrix}~d\omega\geq 0$

(5)

Such an IQC is often established, and will be in this article, by first deriving a time-domain integral quadratic constraint and then converting this into the frequency domain form (5) via Parseval’s Identity.

The beauty of establishing an IQC of the form (5) is that the IQC theory established in [21] can then be directly applied in order to obtain a frequency domain stability condition, as indicated in the theorem below.

This is the main stability result which will be used in this article: it is a considerable simplification of the original result in [21] and is allowed due to the form of the nonlinearites $\phi(\cdot)$ and the IQC’s which will be developed. In general, more attention needs to be paid to the well-posedness of the feedback system and the form of the IQC - see [21, 16]. Much more could be written about IQC’s but in an effort to keep the article reasonably brief and readable, further discussion is omitted. An excellent reference on the subject is [16] and interesting connections between time-domain and frequency domain IQC’s are made in [30] (see also [25]). A recent review of IQC’s is given in [39].

Firstly, note that if $\phi\in\mathcal{N}_{M}$, then the integral

$$F(\sigma)=\int_{0}^{\sigma}\phi(x)dx$$

is a convex function, since its derivative exists and is monotone non-decreasing by assumption, viz

$$\frac{dF(\sigma)}{d\sigma}=\phi(\sigma)$$

Convexity of $F(\sigma)$ implies that, for all $\sigma_{1},\sigma_{2}\in\mathbb{R}$,

		$\displaystyle(\sigma_{1}-\sigma_{2})F^{\prime}(\sigma_{1})\geq F(\sigma_{1})-F(\sigma_{2})$		(7)
	$\displaystyle\Leftrightarrow$	$\displaystyle(\sigma_{1}-\sigma_{2})\phi(\sigma_{1})\geq F(\sigma_{1})-F(\sigma_{2})$		(8)

where $F^{\prime}(\sigma_{1})$ represents the derivative of $F(\sigma)$ at $\sigma_{1}$. Since this inequality holds for any $\sigma_{1},\sigma_{2}\in\mathbb{R}$, then it holds for $\sigma_{1}=x(t)$ and $\sigma_{2}=x(t-\tau)$, where $t$ and $\tau$ are arbitrary, which gives

$\displaystyle\big{(}x(t)-x(t-\tau)\big{)}\phi(x(t))\geq F(x(t))-F(x(t-\tau))$

(9)

Integrating both sides w.r.t. $t$ then gives ^††margin: “If, he thought to himself, an infinite improbability machine is a virtual impossibility, then it must logically be a finite improbability.”

		$\displaystyle\int_{-\infty}^{\infty}\big{(}x(t)-x(t-\tau)\big{)}\phi(x(t))dt\geq\underbrace{\int_{-\infty}^{\infty}\Big{(}F(x(t))-F(x(t-\tau))\Big{)}dt}_{=0}$		(10)
	$\displaystyle\Rightarrow$	$\displaystyle\int_{-\infty}^{\infty}\big{(}x(t)-x(t-\tau)\big{)}\phi(x(t))dt\geq 0$		(11)

which can then be re-arranged to give the remarkable, and perhaps unexpected, result that

$\displaystyle\int_{-\infty}^{\infty}x(t)\phi(x(t))dt\geq\int_{-\infty}^{\infty}x(t-\tau)\phi(x(t))dt$

(12)

for all $\tau$. Inequality (12) is sometimes called the area inequality.

In fact, it will also be useful to develop another version of the area inequality as well which applies in the special case that $\phi(\cdot)$ is an odd function. First note that if $\phi(.)$ is odd then its integral, $F(\cdot)$ is even. Hence replacing $\sigma_{2}$ with $-\sigma_{2}$ in inequality (8) thus gives:

	$\displaystyle(\sigma_{1}-(-\sigma_{2}))\phi(\sigma_{1})$	$\displaystyle\geq F(\sigma_{1})-F(-\sigma_{2})$		(13)
	$\displaystyle\Leftrightarrow(\sigma_{1}+\sigma_{2})\phi(\sigma_{1})$	$\displaystyle\geq F(\sigma_{1})-F(\sigma_{2})$		(14)

Again, noting that the above inequality holds for any $\sigma_{1},\sigma_{2}\in\mathbb{R}$, then it holds for $\sigma_{1}=x(t)$ and $\sigma_{2}=x(t-\tau)$, which gives

$\displaystyle\big{(}x(t)+x(t-\tau)\big{)}\phi(x(t))\geq F(x(t))-F(x(t-\tau))$

(15)

As before, integrating both sides w.r.t. $t$ then gives

		$\displaystyle\int_{-\infty}^{\infty}\big{(}x(t)+x(t-\tau)\big{)}\phi(x(t))dt\geq\int_{-\infty}^{\infty}\Big{(}F(x(t))-F(x(t-\tau))\Big{)}dt$		(16)
	$\displaystyle\Rightarrow$	$\displaystyle\int_{-\infty}^{\infty}\big{(}x(t)+x(t-\tau))\phi(x(t))\big{)}dt\geq 0$		(17)

which can then be re-arranged to give a mirrored result to inequality (12):

$\displaystyle\int_{-\infty}^{\infty}x(t)\phi(x(t))dt\geq-\int_{-\infty}^{\infty}x(t-\tau)\phi(x(t))dt$

(18)

Combining inequalities (12) and (20) we have

$\displaystyle-\int_{-\infty}^{\infty}x(t)\phi(x(t))dt\leq\int_{-\infty}^{\infty}x(t-\tau)\phi(x(t))dt\leq\int_{-\infty}^{\infty}x(t)\phi(x(t))dt$

(19)

which of course implies that

$\displaystyle\left|\int_{-\infty}^{\infty}x(t-\tau)\phi(x(t))dt\right|\leq\int_{-\infty}^{\infty}x(t)\phi(x(t))dt$

(20)

which is an area inequality for odd nonlinearities. The two area inequalities can be packaged into the following Lemma.

This result is covered in [44, 10, 41] and as Willems notes, has its roots in re-arrangement inequalities [14].

This section establishes an IQC involving a dynamic multiplier M, with transfer function $M(s)=1-H(s)$ and impulse response $m(t)=\delta(t)-h(t)$, for the nonlinearity $\phi\in\mathcal{N}_{M}$. Consider the integral

	$\displaystyle I_{1}=$	$\displaystyle\langle\mathbf{M}x(t),\phi(x(t))\rangle$		(21)
	$\displaystyle=$	$\displaystyle\langle(I-\mathbf{H})x(t),\phi(x(t))\rangle$		(22)
	$\displaystyle=$	$\displaystyle\langle x(t),\phi(t)\rangle-\langle\mathbf{H}x(t),\phi(x(t))\rangle$		(23)
	$\displaystyle=$	$\displaystyle\langle x(t),\phi(t)\rangle-\int_{-\infty}^{\infty}\left(\int_{-\infty}^{\infty}h(\tau)x(t-\tau)d\tau\right)\phi(x(t))dt$		(24)

Changing the order of integration then gives

$\displaystyle I_{1}=$

$\displaystyle\langle x(t),\phi(t)\rangle-\int_{-\infty}^{\infty}h(\tau)\left(\int_{-\infty}^{\infty}x(t-\tau)\phi(x(t))dt\right)d\tau$

(25)

Next, assuming $h(t)\geq 0\quad\forall t$, the area inequality (12) means that

$\displaystyle h(\tau)\int_{-\infty}^{\infty}x(t)\phi(x(t))dt\geq h(\tau)\int_{-\infty}^{\infty}x(t-\tau)\phi(x(t))dt$

(26)

for all $\tau$, which then implies

	$\displaystyle I_{1}$	$\displaystyle\geq\langle x(t),\phi(t)\rangle-\int_{-\infty}^{\infty}h(\tau)\left(\int_{-\infty}^{\infty}x(t)\phi(x(t))dt\right)d\tau$		(27)
		$\displaystyle=\langle x(t),\phi(t)\rangle-\int_{-\infty}^{\infty}h(\tau)d\tau\int_{-\infty}^{\infty}x(t)\phi(x(t))dt$		(28)
		$\displaystyle=\langle x(t),\phi(t)\rangle-\|\mathbf{H}\|_{1}\langle x(t),\phi(x(t))\rangle$		(29)
		$\displaystyle=(1-\|\mathbf{H}\|_{1})\langle x(t),\phi(t)\rangle$		(30)

Now since $\langle x(t),\phi(t)\rangle\geq 0$, we see that $I_{1}\geq 0$ if $\|\mathbf{H}\|_{1}\leq 1$. The results can be summarised in the following theorem.

It is more usual to express this in the frequency domain and, indeed using Parseval’s Identity, it follows that above “time-domain” IQC can be written in the more familiar frequency domain form as

$\displaystyle\int_{-\infty}^{\infty}\begin{bmatrix}\hat{x}(j\omega)\\ \widehat{\phi(x)}(j\omega)\end{bmatrix}^{*}\begin{bmatrix}0&M^{*}(j\omega)\\ M(j\omega)&0\end{bmatrix}\begin{bmatrix}\hat{x}(j\omega)\\ \widehat{\phi(x)}(j\omega)\end{bmatrix}d\omega\geq 0$

(31)

The strange thing about Theorem 2 is that there is a requirement for the impulse response of the dynamic element of the multiplier (that is, $\mathbf{H}$) to be non-negative: $h(t)\geq 0$ for all time. If more is known about the nonlinearity, namely that it is odd and hence $\phi\in\mathcal{N}_{MO}$, the requirement that $h(t)\geq 0$ can be omitted. Consider again the integral:

	$\displaystyle I_{1}=$	$\displaystyle\langle\mathbf{M}x(t),\phi(x(t))\rangle$		(32)
		$\displaystyle=\langle x(t),\phi(t)\rangle-\int_{-\infty}^{\infty}h(\tau)\left(\int_{-\infty}^{\infty}x(t-\tau)\phi(x(t))dt\right)d\tau$		(33)

where the same steps have been applied as in the previous section. Previously, at this point, positivity of $h(t)$ was invoked, but suppose this is not assumed; instead use will be made of the other version of the area inequality - inequality (20). From equation (33) it follows that

	$\displaystyle I_{1}$	$\displaystyle\geq\langle x(t),\phi(t)\rangle-\left|\int_{-\infty}^{\infty}h(\tau)\left(\int_{-\infty}^{\infty}x(t-\tau)\phi(x(t))dt\right)d\tau\right|$		(34)
		$\displaystyle\geq\langle x(t),\phi(t)\rangle-\int_{-\infty}^{\infty}|h(\tau)|\left|\int_{-\infty}^{\infty}x(t-\tau)\phi(x(t))dt\right|d\tau$		(35)

Using the second version of the area inequality (20) yields

	$\displaystyle I_{1}$	$\displaystyle\geq\langle x(t),\phi(t)\rangle-\int_{-\infty}^{\infty}|h(\tau)|\left(\int_{-\infty}^{\infty}x(t)\phi(x(t))dt\right)d\tau$		(36)
		$\displaystyle=\langle x(t),\phi(t)\rangle-\|\mathbf{H}\|_{1}\langle x(t),\phi(t)\rangle$		(37)
		$\displaystyle=(I-\|\mathbf{H}\|_{1})\langle x(t),\phi(t)\rangle$		(38)

Again, note that if $\|\mathbf{H}\|\leq 1$ it is clear that $I_{1}\geq 0$. This result can be stated as the following theorem:

Again, this time-domain integral can be expressed as a more familiar frequency domain IQC (31).

Consider the modified nonlinearity shown in Figure 4. This nonlinearity, $\tilde{\phi}$ can be written as

$\displaystyle\tilde{\phi}(\tilde{v})=\left\{\begin{array}[]{ccl}\phi(\tilde{v})&=&\phi(v)\\ v&=&\tilde{v}+\frac{1}{\alpha}\phi(v)\end{array}\right.$

(41)

Assuming solutions exist to this implicit equation, one can write:

$\displaystyle\tilde{\phi}(\tilde{v})=\phi\circ(1-\frac{1}{\alpha}\phi(.))^{-1}(\tilde{v})$

(42)

Note that because $\alpha\sigma-\phi(\sigma)$ is a monotone nonlinearity (by virtue of the slope restriction $\partial\phi\in[0,\alpha]$), then

$$1-\frac{1}{\alpha}\phi(.)$$

also is and so is its inverse. Furthermore, again by the slope restriction, $\phi(.)$ is monotonically increasing function, and hence so is $\tilde{\phi}$ defined above. From Section 3.2, on monotone nonlinearities, it follows that $\tilde{\phi}$ therefore satisfies the IQC (31) where $\mathbf{H}$ should have a positive impulse response. Note that this IQC essentially defines a relationship between $\tilde{\phi}$ and $\tilde{v}$: one between $\phi$ and $v$ is actually what is sought. However, from equation (41), the following relationship holds:

$\displaystyle\begin{bmatrix}v\\ \phi(v)\end{bmatrix}=\underbrace{\begin{bmatrix}1&\frac{1}{\alpha}\\ 0&1\end{bmatrix}}_{X}\begin{bmatrix}\tilde{v}\\ \tilde{\phi}(\tilde{v})\end{bmatrix}$

(43)

Therefore if $\tilde{\phi}(.)$ satisfies the IQC defined by $\Pi(j\omega)$ given in equation (31), then the following inequality holds

$\displaystyle\int_{-\infty}^{\infty}\begin{bmatrix}\tilde{v}(j\omega)\\ \tilde{\phi}(j\omega)\end{bmatrix}^{*}\begin{bmatrix}0&M^{*}(j\omega)\\ M(j\omega)&0\end{bmatrix}\begin{bmatrix}\tilde{v}(j\omega)\\ \tilde{\phi}(j\omega)\end{bmatrix}~d\omega\geq 0$

(44)

The relationship in equation (43) can be reversed to get

$\displaystyle\begin{bmatrix}\tilde{v}\\ \tilde{\phi}(\tilde{v})\end{bmatrix}=\underbrace{\begin{bmatrix}1&-\frac{1}{\alpha}\\ 0&1\end{bmatrix}}_{X^{-1}}\begin{bmatrix}v\\ \phi(v)\end{bmatrix}$

(45)

and then used in inequality (44) to get

$\displaystyle\int_{-\infty}^{\infty}\begin{bmatrix}{v}(j\omega)\\ {\phi}(j\omega)\end{bmatrix}^{*}\begin{bmatrix}1&0\\ -\frac{1}{\alpha}&1\end{bmatrix}\begin{bmatrix}0&M^{*}(j\omega)\\ M(j\omega)&0\end{bmatrix}\begin{bmatrix}1&-\frac{1}{\alpha}\\ 0&1\end{bmatrix}\begin{bmatrix}{v}(j\omega)\\ {\phi}(j\omega)\end{bmatrix}~d\omega\geq 0$

(46)

Therefore $\phi$ satisfies the relationship defined by the matrix

	$\displaystyle{\Pi}_{S}(j\omega)$	$\displaystyle=(X^{-1})^{\prime}\Pi(j\omega)X^{-1}$		(47)
		$\displaystyle=\begin{bmatrix}1&0\\ -\frac{1}{\alpha}&1\end{bmatrix}\begin{bmatrix}0&M^{*}(j\omega)\\ M(j\omega)&0\end{bmatrix}\begin{bmatrix}1&-\frac{1}{\alpha}\\ 0&1\end{bmatrix}$		(48)
		$\displaystyle=\begin{bmatrix}0&M^{*}(j\omega)\\ M(j\omega)&-\frac{1}{\alpha}(M^{*}(j\omega)+M(j\omega))\end{bmatrix}$		(49)

or, multiplying through by $\alpha>0$, the more familiar form emerges:

$\displaystyle{\Pi}_{S}(j\omega)=\begin{bmatrix}0&\alpha M^{*}(j\omega)\\ \alpha M(j\omega)&-M^{*}(j\omega)-M(j\omega)\end{bmatrix}$

(50)

As before, this result can be packaged into the theorem below.

The previous subsection effectively explained how a slope-restricted nonlinearity could be transformed into a monotone nonlinearity and how the results of Section 3.2, or more precisely Theorem 2, could be used to derive an IQC for the original slope restricted nonlinearity. The same is done in this section, but the additional assumption that $\phi(\cdot)$ is odd will be imposed: this then allows the results of Section 3.3 to be used, and in particular Theorem 3, to be harnessed to provide a similar result.

Firstly, again consider the modified nonlinearity depicted in Figure 4. This can again be described by the implicit equation (41) or, more compactly by equation (42). Note that if it is assumed that $\phi(.)$ is odd, then this implies that $\tilde{\phi}(\cdot)$ is also odd because the composition and inverse of odd functions is also odd. Therefore, it follows that $\tilde{\phi}(\cdot)$ is both odd and monotone (from the discussion in the previous section). With this in mind, from Theorem 3, $\tilde{\phi}(.)$ satisfies the IQC defined by (31). In exactly the same way as in the previous subsection, using the relationship (43), it thus follows that $\phi(\cdot)$ satisfies the IQC defined by (50) except without the stipulation that the impulse response of the dynamic part of the multiplier, $\mathbf{H}$, is positive. The conclusion of this section can be summarised in the following theorem.

The relationship between the results is perhaps best summarised by the Venn diagram in Figure 5, assuming that the lower slope restriction is zero. Note that Theorem 3 is a special case of Theorem 2 with the added restriction that $\phi(\cdot)$ should be odd, but with the corresponding relaxation that the impulse response of $\mathbf{H}$ need not be positive. Likewise Theorem 4 is a special case of Theorem 3 but with the added restriction that $\phi(\cdot)$ should be odd, but again with the relaxation that the impulse response of $\mathbf{H}$ need not be positive.

Using Zames-Falb multipliers for stability analysis is significantly more difficult than using the Circle or Popov criteria because one has to somehow find or choose the dynamics of the Zames-Falb multiplier, $\mathbf{M}$. This is made more difficult by the observation that they may be drawn from a very wide range of linear operators: it was noted in the original work of Zames and Falb that these operators do not have to even be rational or causal ²²2Irrationality has been exploited in the work of [12, 7, 4].. The natural question is then: how does one pick “the best” Zames-Falb multiplier for stability analysis? Or more precisely: what is the Zames-Falb multiplier associated with the largest slope restriction or the smallest $\mathcal{L}_{2}~$gain (of an appropriate mapping)?

In the frequency domain, for slope-restricted nonlinearities, the IQC results of Theorem 1 mean that the following inequality must be verified for some $\epsilon>0$:

	$\displaystyle\begin{bmatrix}G(j\omega)\\ I\end{bmatrix}^{*}\begin{bmatrix}0&\alpha M(j\omega)^{*}\\ \alpha M(j\omega)&-M(j\omega)-M^{*}(j\omega)\end{bmatrix}\begin{bmatrix}G(j\omega)\\ I\end{bmatrix}\leq-\epsilon$		(51)
	$\displaystyle M(j\omega)^{*}(\alpha G(j\omega)^{*}-I)+(\alpha G(j\omega)-I)M(j\omega)\leq-\epsilon$		(52)

To assist in the search, the above inequality is often transformed into a search over symmetric matrices using the KYP Lemma (see [26]). This yields a matrix inequality of the form

$\displaystyle\begin{bmatrix}A^{\prime}\mathbf{P}+\mathbf{P}A&\mathbf{P}B+C^{\prime}\\ B^{\prime}\mathbf{P}+C&D+D^{\prime}\end{bmatrix}<0\quad\mathbf{P}=\mathbf{P}^{\prime}$

(53)

Unfortunately, matrix inequality is bilinear because the state-space matrices $(A,B,C,D)$ are affine functions of the multiplier parameters. Therefore, one still needs to find some way in to search, efficiently, for an appropriate multiplier. It is important to emphasize that:

This observation is, effectively, what makes analysis using Zames-Falb multipliers challenging in practice.

There are three prevailing approaches to Zames-Falb multiplier searches in the literature. Perhaps the most common was proposed by [8] which, in essence, requires one to choose multipliers of a particular form:

$\displaystyle M(s)=1-H(s)=1-\prod_{i=1}^{n}\frac{k_{i}}{s+a_{i}}$

(54)

Note that in the case that $\mathbf{M}\in\mathcal{RH}_{\infty}$ $a_{i}>0$ and therefore choosing $k_{i}>0$ one immediately has that $h(t)\geq 0$ for all $t$. The $\mathcal{L}_{1}$ condition then simply becomes a case of ensuring $\|\mathbf{H}\|_{1}=\prod_{i=1}^{n}k_{i}/a_{i}\leq 1$. Equipped with this form, one can then simply insert this multiplier in equation (50) and use this to evaluate the frequency domain inequality in Theorem 1. Note that this frequency domain inequality is equivalent to a matrix inequality by the KYP Lemma [26]. The main issue with this approach is that one needs to choose the poles of the multiplier $a_{i}$ to ensure that the arising matrix inequalities are linear. The key advantage is that one has an exact expression for the $\mathcal{L}_{1}$ norm of $\mathbf{H}$. References [40, 39] discuss this approach further.

An alternative approach to the above, which was originally suggested in [36] and then used in [5, 4, 32, 33] and elsewhere, is to assume a particular order of multiplier (at least plant-order) and then to use a change of variables similar to that used in [29] to obtain convex conditions for stability. The approach is algebraically involved and either uses a bound on the $\mathcal{L}_{1}$ norm [29], which is known to be extremely conservative in some cases; or one can use symmetric multipliers [32] which remove the conservatism in the $\mathcal{L}_{1}$ norm, but then one must accept the phase restrictions imposed by the symmetry of the multiplier. My own - and somewhat biased view - is that this general approach is somewhat more intellectually satisfying than the previous one where effectively one makes, in practice, some arbitrary choices for poles and order - but improvements need to be made to overcome the current conservatism which plagues the approach in some cases.

The final approach that is worth mentioning is that in Zames and Falb’s original results, the multiplier $\mathbf{M}$ may actually be chosen irrational and this has been exploited by several authors [12] and [7] in order to obtain stability results. Recently [4] Carrasco, Heath and co-authors have proposed discrete-time search for multipliers, based on FIR functions, which when interpreted in continuous time provides irrational multipliers which appear to be competitive with the state of the art. The potential disadvantage with this technique is that it may not be easy to combine this with synthesis techniques.

Table 1 gives a rough comparison between the three different approaches: a detailed comparison is out of scope and the interested reader should consult [6] or [11] for more detailed discussion. It is interesting that opinions on the superiority of a particular form of multiplier vary and that the same property may be perceived to be an advantage by some researchers, but a disadvantage by others. At this point it is fair to say “the jury is out” on which approach is best.

This article, and the original results of Zames and Falb, has been dedicated to the case when $\phi(\cdot)$ is a scalar nonlinearity. More generally, $\phi(\cdot):\mathbb{R}^{m}\mapsto\mathbb{R}^{m}$ and hence multivariable generalisations of Zames-Falb multipliers must be used. This extension was only carried out relatively recently by Safonov and co-authors in a number of papers [28, 19, 20]. The generalisations are not trivial and require further assumptions on the form of the nonlinearity to be made, but they do have IQC interpretations in much the same way as their scalar-valued counterparts [9]. The search for multipliers is somewhat more involved however: the generalisation of [36] to the multivariable case [38] is effectively too conservative and too computationally demanding³³3The word “tractable” in the title is, in hindsight, misleading; “intractable” might be more appropriate.. The approaches given in [11, 31] both have merit but require choices to be made which are not entirely systematic. Therefore, while it seems that much of the spirit of the Zames-Falb multipliers carries over to the multivariable case, the search for suitable multipliers seems inherently more difficult ⁴⁴4This section needs improvement and more detail. Similar sentiments probably apply to the article as a whole..

It has been known for some time that Zames-Falb multipliers are potentially much less conservative than either the Circle or Popov criteria – in the sense that they may conclude stability when the Circle/Popov Criteria fail to do so. It is not yet certain whether a Zames-Falb Multiplier is a necessary as well as sufficient condition for absolute stability. Results in both continuous ([18]) and discrete time ([45]) seem to suggest that the existence of a Zames-Falb multiplier is fairly close to necessary⁵⁵5It is believed by some that W.P. Heath’s dog has worked out the necessary conditions. for absolute stability, and thus using Zames-Falb multipliers should give the control engineer some confidence in the results obtained.

An often overlooked contributor to the development of Zames-Falb multipliers is the person at their origin: R O’Shea ⁶⁶6It was J. Carrasco and W.P. Heath who impressed upon me the importance of R. O’Shea in the development of Zames-Falb multipliers. His contribution is acknowledged in the paper [6].. In a few papers [22, 23], O’Shea introduces the idea of the sort of multipliers which have now become known as “Zames-Falb” multipliers. Although O’Shea did not quite work out the precise mathematics of the multipliers, he should at least be credited with their inception.