On the representation of stack operators by mathematical morphology

Stack filters, proposed by [13] in the 80’s, is a classical family of filters for signal and image processing. Frameworks for learning stack filters from data have been extensively studied [3, 4, 6] and many algorithms to design stack filters have been proposed [5, 14]. In the context of mathematical morphology, properties of stack filters have been deduced in two important works by Maragos and Schafer [7, 8].

It is well-known that an operator $\psi$ acting on functions $f:E\mapsto\{0,\dots,m\}$ (i.e., grey-scale images), for a set $E$ and $m\in\mathbb{Z}_{+}$, is a stack filter if, and only if, there exists an increasing set operator $\tilde{\psi}$, that acts on functions $X:E\mapsto\{0,1\}$ (i.e., binary images), such that

for all $f\in\{0,\dots,m\}^{E}$, in which $T_{t}$ is the cross-section of $f$ at point $t$. We refer to [7, 8] for a proof of this fact. In this paper, we study the class of stack operators that are the operators $\psi$ that satisfy (1) for a not necessarily increasing set operator $\tilde{\psi}$.

We define the stack operators as those that (a) map binary functions into binary functions and (b) such that $\psi(f)$ equals the mean of $\psi(mT_{t}[f])$ over $t=1,\dots,m$, and then show that these operators can be written as (1). In particular, we show that stack operators are an extension of set operators (i.e, binary image operators) to function operators (i.e., grey-scale image operators). Our main result is that the stack operator $\psi$ inherits many properties of the respective set operator $\tilde{\psi}$, such as being an erosion, a dilation, an anti-erosion, an anti-dilation, increasing, decreasing, extensive, anti-extensive, sup-generating and inf-generating. For simplicity, we focus on the case of translation-invariant and locally defined stack operators and show the main result by deducing the characteristic function, kernel and basis representation of stack operators.

The results of this paper have implications on the design of image operators, since allow to solve grey-scale image processing tasks by designing an operator for performing the desired transformation on binary images and then considering its extension given by a stack operator. We also discuss how the results can be leveraged to develop learning methods for grey-scale image processing based on deep neural networks and discrete morphological neural networks [9].

In Section 2, we define the concepts necessary to introduce the stack operators in Section 3, where we show that the space of stack operators is isomorphic to the space of set operators, and that stack operators are 1-Lipschitz extensions of set operators. In Section 4, we show that the stack filters are the increasing stack operators. In Section 5, we establish the characteristic function, kernel and basis representation of stack W-operators. In Section 6, we state and prove the main result of this paper establishing that stack W-operators inherit lattice properties of the associated set operator, and we discuss the implications of our results to the design of image operators. In Section 7, we give our final remarks. Proofs for the main results are presented in the Appendix 0.A.

Let $E$ be a countable set, $(E,+)$ be an Abelian group, with zero element $o\in E$, and $M=\{0,\dots,m\}$ for a positive integer $m$ fixed. A function $f:E\mapsto M$ is a grey-scale image, in which $f(x)$ represents the intensity of the pixel $x\in E$. In particular, a grey-scale binary image is such that $f(x)\in\{0,m\}$ for all $x\in E$. The set $M^{E}$ of all such images is equipped with a partial order satisfying $f,g\in M^{E},f\leq g\iff f(x)\leq g(x),\forall x\in E$. The image threshold at level $t$ is a mapping $T_{t}:M^{E}\mapsto\{0,1\}^{E}$ given by, for any $f\in M^{E},t\in M$ and $x\in E$, $T_{t}[f](x)=\mathds{1}\{f(x)\geq t\}$. The binary function $T_{t}[f]$ is called the cross-section of $f$ at level $t$. The collection $\{T_{t}:t\in M\}$ has some simple, yet important, properties, that we state without proof as a proposition.

An image operator is a mapping $\psi:M^{E}\mapsto M^{E}$ and the set of all such operators, denoted by $\Psi_{M}$, is equipped with a partial order satisfying $\psi_{1},\psi_{2}\in\Psi_{M},\psi_{1}\leq\psi_{2}\iff\psi_{1}(f)\leq\psi_{2}(f),\forall f\in M^{E}$. The poset $(\Psi_{M},\leq)$ is a complete lattice.

A binary image can be represented as a function $X:E\mapsto\{0,1\}$ in which $X(x)=1$ if, and only if, $x\in E$ is a pixel in the image foreground. The set $\{0,1\}^{E}$ of all binary images equipped with the partial order satisfying $X,Y\in\{0,1\}^{E},X\leq Y\iff X(x)\leq Y(x),\forall x\in E$, is a complete lattice. Mappings from $\{0,1\}^{E}$ into itself are called set operators, a nomenclature inspired by the fact that $\{0,1\}^{E}$ is isomorphic to the power-set of $E$. We denote the space of all set operators by $\Psi$ and equip it with the point-wise partial order $\tilde{\psi}_{1},\tilde{\psi}_{2}\in\Psi,\tilde{\psi}_{1}\leq\tilde{\psi}_{2}\iff\tilde{\psi}_{1}(X)\leq\tilde{\psi}_{2}(X),\forall X\in\{0,1\}^{E}$. An image operator is an extension of a set operator if they coincide on binary images.

For each $h\in E$, denote by $\tau_{h}:E\mapsto E$ the translation by $h$ function that maps $x\mapsto x+h$. An operator $\psi\in\Psi_{M}$, and a set operator $\tilde{\psi}\in\Psi$, are translation-invariant if they commute with translation, that is, $\psi(f\circ\tau_{h})=\psi(f)\circ\tau_{h}$ and $\tilde{\psi}(X\circ\tau_{h})=\tilde{\psi}(X)\circ\tau_{h}$ for all $f\in M^{E}$, $X\in\{0,1\}^{E}$ and $h\in E$.

For $W\in\{0,1\}^{E}$, denote by $X|_{W}$ and $f|_{W}$ the restriction of $X\in\{0,1\}^{E}$ and $f\in M^{E}$ to the set $\chi(W)\coloneqq\{x\in E:W(x)=1\}$. An operator $\psi\in\Psi_{M}$, and a set operator $\tilde{\psi}\in\Psi$, are locally defined within $W$ if $\psi(f)(x)=\psi(f|_{W\circ\tau_{-x}})(x)$ and $\tilde{\psi}(X)(x)=\tilde{\psi}(X|_{W\circ\tau_{-x}})(x)$ for all $f\in M^{E}$, $X\in\{0,1\}^{E}$ and $x\in E$. The locally defined condition implies, for example, that the value of the transformed image $\psi(f)$ at point $x$ depends only on the value of $f$ in the neighbourhood $\chi(W\circ\tau_{-x})=\{h\in E:W(h-x)=1\}$ of $x$. Operators that are translation-invariant and locally defined are called W-operators. The collections of image and set W-operators are denoted by $\Psi_{M}^{W}\subseteq\Psi_{M}$ and $\Psi^{W}\subseteq\Psi$, respectively.

In this paper, we study the collection of stack image operators, defined as follows, and denoted by $\Sigma_{S}\subseteq\Psi_{M}$.

Condition (a) of Definition 2 implies that stack operators take grey-scale binary images $mX\in\{0,m\}^{E}$ into grey-scale binary images. It follows from this condition that $m^{-1}\psi(mT_{t}[f])\in\{0,1\}^{E}$ and therefore the right-hand side of (2) is an element of $M^{E}$. Condition (b) then implies that $\psi(f)$ is the average of the transformation by $\psi$ of the cross-sections of $f$ represented as grey-scale binary images. Stack operators have an associated set operator $\tilde{\psi}:\{0,1\}^{E}\mapsto\{0,1\}^{E}$ and the collection $\Psi$ of set operators is lattice isomorphic to the collection of stack image operators $\Sigma_{S}$. In particular, $(\Sigma_{S},\leq)$ is a complete lattice. We state without proof this result that is a direct consequence of the definitions of the isomorphisms $\mathcal{F}$ and $\mathcal{F}^{-1}$.

We call $\mathcal{F}^{-1}(\psi)$ the characteristic set operator of $\psi\in\Sigma_{S}$. It follows from Proposition 2 that $m^{-1}\psi(mX)=\mathcal{F}^{-1}(\psi)(X)$ and therefore $\psi$ is an extension of its characteristic set operator (ref. Definition 1). Moreover, this extension is 1-Lipschitz considering the $L^{1}$-norm in the domain of $\psi$ and the $L^{\infty}$-norm in its image, that is:

In particular, stack operators are $(L^{1},L^{\infty})$-continuous. A proof for the next corollary is in Appendix 0.A.

An important class of stack operators is that of the stack W-operators, denoted by $\Sigma_{S}^{W}=\Psi_{M}^{W}\cap\Sigma_{S}$. The isomorphism $\mathcal{F}$ preserves the W-operator properties, hence maps set W-operators to stack W-operators. We state without proof this result that is a direct consequence of the definitions of $\mathcal{F}$ and $\mathcal{F}^{-1}$.

In Figure 1 we present two simple examples of stack operators: for boundary recognition and noise filtering. In these cases we can clearly see that the stack operators extend to grey-scale images the transformation performed on set operators. In the example of boundary recognition, the operator is not increasing, therefore is not a stack filter.

The stack filters are operators $\psi$ that commute with threshold in the following sense:

for all $t\geq 1$ and $f\in M^{E}$. Equivalently, it follows from the definition of $\mathcal{F}^{-1}$ that a stack operator $\psi$ with characteristic set operator $\tilde{\psi}$ is a stack filter if, and only if, $T_{t}[\psi(f)]=\tilde{\psi}(T_{t}[f])$ for all $t\geq 1$ and $f\in M^{E}$. A stack filter is a stack operator since it follows from (4) that $\psi(mX)(x)=mT_{1}[\psi(X)](x)\in\{0,m\}$ for all $x\in E$ and $X\in\{0,1\}^{E}$ and that $\psi(f)=\sum_{t=1}^{m}T_{t}[\psi(f)]=\frac{1}{m}\sum_{t=1}^{m}\psi(mT_{t}[f])$ in which the first equality follows from Proposition 1 (i) and the second from (4). The singularity of stack filters is that they satisfy (4) for all $t\geq 1$ while general stack operators satisfy (4) summing over $t$ (cf. (2)). Actually, the stack filters are the stack operators with increasing characteristic set operator. It follows from Proposition 4 that the definition of stack filters by equation (4) is equivalent to the classical definition of stack filters [7]. The proof of this proposition is analogous to that of [7, Theorem 3].

The representation of set and lattice operators by mathematical morphology is a classical research topic, and results about the representation of increasing set operators [7], translation-invariant set operators [1], and general lattice operators [2] have been established. In this section we deduce the representation of stack W-operators by a characteristic function, a kernel and a basis, and establish isomorphisms with the respective representation of the associated set W-operator, that are illustrated in Figure 2.

It follows from Proposition 2 that restricted classes of stack operators can be obtained by constraining $\Psi$ and applying $\mathcal{F}$. Actually, $\mathcal{F}$ preserves many lattice properties of $\tilde{\psi}$, hence restricting $\Psi$ based on these properties will generate analogous restrictions in $\Sigma_{S}$. The next theorem, for which a proof is presented in Appendix 0.A, establishes these properties for the case of stack W-operators.

The isomorphisms depicted in Figure 2 together with Theorem 6.1 imply that a stack W-operator can be designed by designing a set operator that performs the desired transformation on sets, what actually reduces to designing a binary function in $\mathcal{P}(W)$. Prior information about properties that the optimal operator satisfy can be used to define a restricted class of set operators that, due to Theorem 6.1, will generate a restricted class of stack operators with the same properties, from which an operator can be learned from data. In particular, it is possible to adapt the loss function considered to train discrete morphological neural networks (DMNN) [9], in particular unrestricted DMNN [10], to learn a characteristic set W-operator by actually minimising the error associated with the respective stack operator. Also, a deep neural network might also be considered to represent the characteristic function of the set W-operator and be trained by minimising the error associated with the respective stack operator. We will investigate these learning methods in future studies.

This paper defined the stack operators, that are those that commute in average with cross-sectioning, and showed that they can be represented by a characteristic set operator, being equivalent to applying the set operator to the cross-sections of the image and summing. In particular, stack operators are 1-Lipschitz extensions of set operators and also the generalisation of stack filters, for which the characteristic set operator is increasing. Isomorphisms between the characteristic function, kernel and basis representation of stack W-operators and set W-operators were deduced and we showed that many properties of the set W-operators are preserved in the respective stack W-operators. The perspectives of learning stack operators by learning the characteristic set operator via DMNN and deep neural networks was briefly discussed.

There are many topics for future studies based on our representation results about stack operators. First, it is necessary to generalise the results for stack-operators that are not W-operators, what should be straightforward. Furthermore, it would be interesting to study the representation of stack operators with other kinds of invariance, such as group equivariant operators [11] and scale invariant operators [12]. Moreover, as discussed in Section 6, it is necessary to develop learning methods for stack operators.

Given the simplicity of the representation of stack W-operators, that are equivalent to Boolean functions in $\mathcal{P}(W)$, they could be the preferred choice when a solving a practical image transformation problem. However, it is necessary to characterise what problems they can solve, that is, what image operators they can approximate. Therefore, the main topic of future research is to characterise the stack operators within the family of 1-Lipschitz extensions of set operators by investigating if they have universal approximating properties, i.e., by characterising spaces in which the stack operators are dense. Stack operators will be suitable to solve a problem when the optimal operator is in such a space.

This work was partially funded by grant #22/06211-2 São Paulo Research Foundation (FAPESP).