Fixed points of compositions of nonexpansive mappings:
finitely many linear reflectors¹¹1 Dedicated to Terry Rockafellar on the occasion of his 85th birthday

Throughout, we assume that

and induced norm $\|\cdot\|\colon X\to\mathbb{R}\colon x\mapsto\sqrt{\left\langle{x},{x}\right\rangle}$. A mapping $R\colon X\to X$ is nonexpansive if $(\forall x\in X)(\forall y\in X)$ $\|Rx-Ry\|\leq\|x-y\|$. Nonexpansive operators play a central role in modern optimization because the set of fixed points $\operatorname{Fix}R:=\big{\{}{x\in X}~{}\big{|}~{}{x=Rx}\big{\}}$ often represents solutions to optimization or inclusion problems (see, e.g., [3]). A central question is the following

Given nonexpansive maps $R_{1},\ldots,R_{m}$ on $X$ with $\bigcap_{i=1}^{m}\operatorname{Fix}R_{i}\neq\varnothing$, what can we say about $\operatorname{Fix}(R_{m}R_{m-1}\cdots R_{1})$?

Clearly,

Note that one cannot expect equality to hold in 2:

However, equality in 2 does hold for “nice” nonexpansive maps such as averaged mappings (see, e.g., [3, Corollary 4.51]) or even strongly nonexpansive mappings (see [5, Lemma 2.1]).

In this note, we aim to study $\operatorname{Fix}(R_{m}R_{m-1}\cdots R_{1})$ for certain mappings that are not nice but that do have some additional structure. To describe this, let us denote the projector (or nearest point mapping) associated with a nonempty closed convex subset $C$ of $X$ by ${\operatorname{P}}_{C}$. The corresponding reflector

is known to be nonexpansive (see, e.g., [3, Corollary 4.18]). Note that

and that ${\operatorname{R}}_{\{0\}}=-\operatorname{Id}$, so the class of reflectors is “bad” (see Example 1.1). We also have

When $C$ is a hyperplane containing the origin, then we shall refer to ${\operatorname{R}}_{C}$ as a classical reflector. Classical reflectors are basic building blocks: indeed, the Cartan-Dieudonné Theorem (see, e.g., [6, Theorem 8.1] or [10, Section 2.4]) states that every linear isometry on $\mathbb{R}^{n}$ is the composition of at most $n$ classical reflectors.

A very satisfying result is available for two general linear reflectors:

1.2 was used in [2] to analyze the Douglas–Rachford operator $T:=\tfrac{1}{2}(\operatorname{Id}+{\operatorname{R}}_{U_{2}}{\operatorname{R}}_{U_{1}})$. Note that $\operatorname{Fix}T=\operatorname{Fix}({\operatorname{R}}_{U_{2}}{\operatorname{R}}_{U_{1}})$! It was shown that ${\operatorname{P}}_{U_{1}}T^{n}\to{\operatorname{P}}_{U_{1}\cap U_{2}}$ pointwise. Iterating $T$ is actually a special case of employing Rockafellar’s proximal point algorithm [9].

We also note that 1.2 provides an alternative explanation of Example 1.1: indeed, set $U_{1}=U_{2}=\{0\}$ in 1.2. Then ${\operatorname{R}}_{U_{1}}={\operatorname{R}}_{U_{2}}=-\operatorname{Id}$, $U_{1}^{\perp}=U_{2}^{\perp}=X$, and $\operatorname{Fix}({\operatorname{R}}_{U_{2}}{\operatorname{R}}_{U_{2}})=(U_{1}\cap U_{2})\oplus(U_{1}^{\perp}\cap U_{2}^{\perp})=X$.

1.2 nurtures the hope that there might exist a nice formula for $\operatorname{Fix}({\operatorname{R}}_{U_{3}}{\operatorname{R}}_{U_{2}}{\operatorname{R}}_{U_{1}})$ and that there might be a way to recover $U_{1}\cap U_{2}\cap U_{3}$ by projecting $\operatorname{Fix}({\operatorname{R}}_{U_{3}}{\operatorname{R}}_{U_{2}}{\operatorname{R}}_{U_{1}})$ suitably. Unfortunately, this hope was crushed with the following example:

We are now in a position to describe precisely our aim.

The goal of this note is to study the fixed point set of the composition of finitely many reflectors associated with closed linear subspaces.

In Section 2, we obtain several positive results (see Lemma 2.3 and Theorem 2.6), an upper bound (see Theorem 2.7) as well as limiting examples. Section 3 focusses mainly on classical reflectors in the Euclidean plane for which precise information is available. The notation employed is standard and follows largely [3].

We start with a simple observation.

The following result, which is a consequence of Lemma 2.1, provides a case when we have precise knowledge of the fixed point set of the composition of three reflectors:

We now turn to $m$ operators and obtain a very general result which clearly shows the effect of cyclically shifting a composition:

Lemma 2.3 allows us to derive a result complementary to Proposition 2.2:

While Remark 2.5 points out the importance of the order of the operators, there does exist a nice permutation of the reflectors yielding the same fixed point set. To describe this result, observe first $({\operatorname{R}}_{U_{m}}\cdots{\operatorname{R}}_{U_{1}})^{*}={\operatorname{R}}_{U_{1}}^{*}\cdots{\operatorname{R}}_{U_{m}}^{*}={\operatorname{R}}_{U_{1}}\cdots{\operatorname{R}}_{U_{m}}$ because linear projectors and (hence) reflectors are self-adjoint. Combining this with an old result by Riesz and Sz.-Nagy which states that $\operatorname{Fix}T=\operatorname{Fix}T^{*}$ for any nonexpansive linear operator $T\colon X\to X$ (see [8, page 408 in Section X.144] or [7]), we obtain the following positive result:

We now turn to three linear subspaces. The next result narrows down the location of fixed points.

The approach utilized in the proof of Theorem 2.7 to derive the description of the fixed point set also works for any odd number of reflectors; however, the resulting algebraic expressions don’t seem to provide further insights. The superset obtained; however, will easily generalize to an odd number of reflectors:

The next example shows that the upper bound provided in Theorem 2.8 is sometimes sharp:

Now assume that $m$ is even. Set $U_{m+1}:=\{0\}$. Then ${\operatorname{R}}_{U_{m+1}}=-\operatorname{Id}$ and $m+1$ is odd. Therefore, we obtain 21 from the odd case we just proved. $\hfill\quad\blacksquare$

In contrast to Example 2.10, we conclude this section with another example which will illustrate that the upper bound in Theorem 2.8 is not always attained:

Let us now specialize the general result of the last section to the Euclidean plane and classical reflectors. We start with some is well known results whose statements can be found, e.g., in [11].

Set

It is clear that ${\operatorname{Refl}}$ is periodic, with minimal period $\pi$. The importance of ${\operatorname{Refl}}$ stems from the fact that it describes all classical reflectors on $\mathbb{R}^{2}$; indeed,

for every $\alpha\in\mathbb{R}$. It is convenient to also define

Note that for every $\alpha\in\mathbb{R}$, ${\operatorname{Rot}}({\alpha})$ describes the counterclockwise rotation by $\alpha$; the operator ${\operatorname{Rot}}$ is periodic with minimal period $2\pi$.

The following result provides “calculus rules” for the composition of reflectors and rotators. It can be verified using matrix multiplication and addition theorems for sine and cosine.

We are now in a position to classify the fixed point sets of compositions of classical reflectors on $\mathbb{R}^{2}$:

Base case: Case 1: Assume that $m=1$. Then $\beta_{1}=\alpha_{1}$ and $S_{1}={\operatorname{Refl}}({\alpha_{1}})={\operatorname{Refl}}({\beta_{1}})$ so $\operatorname{Fix}S_{1}=\operatorname{Fix}{\operatorname{Refl}}({\alpha_{1}})=\mathbb{R}(\cos(\alpha_{1}),\sin(\alpha_{1}))=\mathbb{R}(\cos(\beta_{1}),\sin(\beta_{1}))$ by 28 as announced. Case 2: Now assume that $m=2$. Then $\beta_{2}=\alpha_{2}-\alpha_{1}$. Using 3.1(ii), we obtain $S_{2}={\operatorname{Refl}}({\alpha_{2}}){\operatorname{Refl}}({\alpha_{1}})={\operatorname{Rot}}({2(\alpha_{2}-\alpha_{1})})={\operatorname{Rot}}({2\beta_{2}})$ and the claim follows.

Inductive step: We assume that the result is true for some integer $m\geq 2$. Then

and

Case 1: $m+1$ is odd; equivalently, $m$ is even. Using the inductive hypothesis, 3.1(iv), and 32, we obtain

and the result follows.

Case 2: $m+1$ is even; equivalently, $m$ is odd. Using the inductive hypothesis, 3.1(ii), and 32, we obtain

and the result follows. $\hfill\quad\blacksquare$

The next example, which was used in an algorithmic context in [4, Example 2.30], illustrates Proposition 2.2, Proposition 2.4, and Theorem 3.2.

We conclude with a comment on higher-dimensional Euclidean space.