The Weighted Euler Characteristic Transform for Image Shape Classification

Computations of the WECT requires a representation of a weighted simplicial complexes. In this paper, we compute the WECT using a simplicial complex that represents an image.²²2For images, a cubical complex provides a computationally preferable representation [19]; however, we choose to use simplicial complexes in order to stay closer to the theory developed in [13]. A geometric $k$-simplex $\sigma$ in $\mathbb{R}^{d}$ is the convex hull of $k+1$ affinely independent points in $\mathbb{R}^{d}$. For example, a zero-simplex is a point, a one-simplex is a segment, and a two-simplex is a triangle. If we want to list the points of a simplex, we write $\sigma=[v_{0},v_{1},\ldots,v_{k}]$ and, for each $i\in\{0,1,\ldots,k\}$, we write $v_{i}\in\sigma$. A simplex $\tau$ is a face of $\sigma$, denoted $\tau<\sigma$, if it is the convex hull of a subset of those $k+1$ points. A simplicial complex is a collection of simplexes that are combined in such a way that (i) for all simplexes $\sigma\in K$, if $\tau$ is a face of $\sigma$ then $\tau\in K$, and (ii) if simplexes $\sigma,\tau\in K$, then either $\sigma\cap\tau\in K$ or $\sigma\cap\tau=\emptyset$. $K_{i}$ is used to denote the set of $i$-simplices of $K$. The simplicial complex $K$ is topologized using the Alexandroff topology [1]; with a slight abuse of notation, $K$ denotes both the complex (as a set of simplices) and the underlying topological space.

For a topological space $\mathbb{X}$, the EC of $\mathbb{X}$, denoted $\chi(\mathbb{X})$, is the alternating sum of Betti numbers, $\chi(\mathbb{X}):=\sum_{i\in\mathbb{N}}(-1)^{i}\beta_{i}$. If $\mathbb{X}$ is a CW complex (in particular, if it is a simplicial complex), $K$, then we have two alternate (but equivalent) definitions displayed below,

The EC definition was generalized to include a weight function [4, 13]. For a weighted simplicial complex $(K,\omega)$, the weighted EC may be defined as

We call a function $f\colon K\to\mathbb{R}$ a filter function if for all $t\in\mathbb{R}$, the sublevel set $f^{-1}(-\infty,t]$ is either empty or a simplicial complex. Given a filter function $f\colon K\to\mathbb{R}$, we consider filtering $K$ by $f$; that is, we have a nested family of sublevel sets $f^{-1}(-\infty,t]$ for $t\in\mathbb{R}$. See Fig. 3(3(a)) for a filtration of the shape in Fig. 2(2(d)) in the direction $(1,0)$.

The EC of $f^{-1}(-\infty,t]$ with respect to $t$ is the Euler characteristic function (ECF), denoted $\chi_{f}\colon\mathbb{R}\to\mathbb{Z}$ and is defined by

If $f$ is a lower-star filter in direction $s$, then we may use $s$ in the subscript instead of $f$. Given a filter function $f\colon K\to\mathbb{R}$ and an arbitrary function $\Omega\colon K\to\mathbb{R}$, the weighted Euler characteristic function (WECF) for filter function $f$ with respect to the weight function $\Omega$ is defined as the function $\chi_{f,\Omega}\colon\mathbb{R}\to\mathbb{R}$, where

We follow the setting of [13], where $f$ is a lower-star filter (i.e., a discretized version of a height filter) in a given direction in $\mathbb{S}^{d-1}$; see Equation (4). Again, if $f$ is a lower-star filter in direction $s$, then we may use $s$ in the subscript instead of $f$. See Fig. 3 for an example ECF and WECF for a filtered shape.

Using an ensemble of directional filter functions, one for each direction in $s\in\mathbb{S}^{1}$, a parameterized collection of ECFs, the ECT, is defined as the function $\text{ECT}_{K}\colon\mathbb{S}^{1}\times\mathbb{R}\to\mathbb{Z}$, where³³3The ECT and the WECT can be generalized to higher dimensions, but our setting is focused on 2D images so here we consider directions in $\mathbb{S}^{1}$ only.

For each direction $s\in\mathbb{S}^{1}$, we can think of this as a function $\mathbb{R}\to\mathbb{Z}$, namely $\chi_{s}$ described above. No two distinct shapes have the same ECT [18]. Since its introduction, the ECT has developed both in theory and in practice [12, 13, 3, 15, 6].

As noted previously, the WECT is a generalization of the ECT for weighted simplicial complexes, and was introduced in [13]. Let $K$ be a simplicial complex in $\mathbb{R}^{2}$ and let $\Omega\colon K\to\mathbb{R}$ be its weight function (note that $\Omega$ need not be a filter function). The WECT for $K$ assigns to each $s\in\mathbb{S}^{1}$ the function $\text{WECT}_{K,\Omega}(s,t)\colon\mathbb{R}\to\mathbb{Z}$ defined by:

where we recall that $H_{s}\colon K\to\mathbb{R}$ is the lower-star filter in direction $s$. When subscripts are difficult to read, we use $\text{WECT}(s,t;K,\Omega):=\text{WECT}_{K,\Omega}(s,t)$.

In Algorithm 1, we provide the algorithm used to compute the WECT of a given weighted shape image. We recall that the WECT is the family of WECFs, one for each direction in $\mathbb{S}^{1}$. Computationally, knowing all WECFs is infeasible so often much fewer are used. We first consider how to compute the WECF in one direction, $s$, as given in Lines 2-17 for $(K,\Omega)$.

In Section 2.1, we discussed several ways to extend a weight function on the vertices to a weighted simplicial complex. Here, we investigate how the choice of extension affects the weighted EC. In what follows, let $K$ be a two-complex, and let $\omega\colon K_{0}\to\mathbb{R}$ be the assignment of intensities to the vertices, drawn independently from a distribution centered at $0.5$.

We generated image datasets using the following parameters: shape, pixel intensity distribution, and function extension (see Equations (1)-(3)). The domain of an image is a $65\times 65$ grid centered at $(0,0)$, with vertices on an integer lattice, triangulated using the Freudenthal triangulation [11]; see Fig. 2(2(c)).

The seven shapes, four distributions, and two function extensions define 56 image classes. The WECT is evaluated on the task of binary classification using support vector machines (SVM) [5] and $K$-nearest-neighbors [10]. For each classification model, 250 images (half from each class) are independently generated, with 80 percent of the images (200 images) used for training and the remaining 20 percent (50 images) used for testing. To approximate the WECT, we sample equally-spaced directions from $\mathbb{S}^{1}$. We discuss results below for choosing different numbers of directions from $\mathbb{S}^{1}$. To vectorize each WECF, $91$ equally-spaced threshold values from $-45$ to $45$ are considered. This ensures that there is at least one discretization value for each height threshold in any direction of filtration.

To better understand the WECT, we considered several experiments that provide insight into how different factors affect the WECT and the information it encodes. In particular, we analyze changes in the number of directions, pixel intensity distributions, function extensions, and use of the WECT vs. the vectorized WECT. However, first, as Fig. 7 demonstrates, we verify that for all four pixel intensity distributions, the computation of the WECT matches the theoretical expectation given in Equation (37).

Next, we analyze how changing the number of directions of the filtration impacts the ability of the WECT to capture information about the underlying image. In order to do so, the following classification task was executed using the SVM classifier. For each shape, the $U(0,1)$ image was compared against the $N(0.5,25)$ in a binary classification task for a varying number of directions. The average function extension was used when computing the WECTs. For a given shape, if the WECT successfully encoded the structure of the two different images, the classification accuracy is closer to one. The classifier accuracy was tested on all the shapes for $2,3,5,8,15$ and $30$ directions. As Fig. 8 shows, as the number of directions increases, accuracy of the classifier improved only for some shapes. In order to balance an increase of encoded information in the WECT with computational cost, the remainder of the experiments are executed using 15 directions.

In order to gain a deeper understanding of the WECT properties, we conducted experiments to observe how changing pixel intensity distributions in various filtration directions affect its behavior. Again, we examined the WECT’s capacity to extract image information through binary classification tasks using an SVM classifier. For each shape, we consider the task of binary classification of WECTs in $15$ directions from each of the four distributions against the WECT from the $U(0,1)$ distribution. All pairwise classification models were considered, but when image shape differed the classifier performed nearly perfect regardless of the pixel intensity distribution so those results are not summarized further. Results where the two classes have the same shape can be found in Fig. 9 for both function extensions. As anticipated, the accuracy of the classifier for $U(0,1)$ against $U(0,1)$ is roughly random, $U(0,1)$ against $N(0.5,0.17)$ is nearly perfect, $U(0,1)$ against $N(0.5,0.25)$ is slightly worse, and $U(0,1)$ against $N(0.5,0.5)$ is barely better than random. Ultimately, the decrease in accuracy as the normal distribution becomes more similar to the uniform distribution (i.e., as $\sigma$ increases) demonstrates that the WECT encodes information about the pixel intensity distribution. In other words, the WECT captures and encodes structural information from the images that is not encoded by the ECT.

Next, we compare the results using the maximum function extension (Fig. 9(a)) and the average function extension (Fig. 9(b)). Overall, across nearly all the distributions, the classifier performs better on WECTs computed using the maximum function extension as opposed to the average function extension. As the distributions differ more at their extremes in the current setting, it is likely that the maximum function extension better emphasizes these differences, which in turn increases the accuracy of the classifier.

To better understand the classification performance in the above experiments, we consider visualizations of the WECT for the setting that considers fifteen directions. Fig. 10 displays the average WECT in one direction of 250 generated images under the two different function extensions. The shaded region indicates the values within one standard deviation of the mean. For both function extensions, the average WECT generally follows the same pattern, but the values of the WECT depend on the distribution of the pixel intensities. The WECT using the $U(0,1)$ distribution generally has the lowest values, and the WECT using the $N(0.50,0.50)$ and distribution tends to have slightly greater values, then $N(0.50,0.25)$, and finally the $N(0.50,0.17)$ distribution tends to have the highest values. These results hold also for the rest of the shapes, as indicated in Fig. 11 and Fig. 12. This, along with the classification results, indicate that incorporating the weights into the ECT is important for distinguishing images generated under different intensity distributions. Note also that the average WECT for different shape classes have varying levels and patterns of smoothness, which reveals why the classification rate between different shapes is nearly perfect.

A final result is the use of the true (non-vectorized) WECT in the classification experiments. As explained in Section 2.3, one can compute a distance between two WECTs by integrating the difference between the piecewise WECFs in each direction (and taking the maximum difference across all directions). Thus, a natural extension use of this distance is in a nonparametric classification method such as $K$-nearest-neighbors (KNN). Fig. 13 summarizes results for the KNN classifier based on the WECT distance with $K=5$ for eight directions with the maximum extension. In general, it appears that performance of SVM was better in these binary classification tasks. Extensive experiments with KNN were less feasible due to the high computational cost of nonparametric methods. However, the focus is not on using the WECT as a classifier, but instead using the classification tasks to understand the information that is encoded in the WECT. For both methods, we observe a decrease in accuracy as the pixel intensity distribution becomes more similar, which indicates that both the vectorized WECT and the WECT distance successfully encode underlying information about the image.