Causal Identification of Sufficient, Contrastive and Complete Feature Sets
in Image Classification

Recent progress in artificial intelligence and the ever increasing deployment of AI systems has highlighted the need to understand better why some decisions are made by such systems. For example, one may need to know why a classifier decides that an MRI scan showed evidence of a tumor (Blake et al. 2023). Answering such questions is the province of causality. A causal explanation for an image classification is a special case of explanations in actual causality (Halpern 2019) and identifies a minimal set of pixels which, by themselves, are sufficient to re-create the original decision (Chockler and Halpern 2024).

Logic-based approaches provide formal guarantees for explanations, but their framework assumes that the model is given explicitly as a function. Their formal abductive explanations, or prime implicants (PI), are defined as sets of features such that, if they take the given values, always lead to the same decision (Shih, Choi, and Darwiche 2018). Logic-based methods can also compute contrastive explanations, that is, those features which, if altered, change the original decision. These abductive and contrastive explanations require a model to be monotonic or linear to be effectively computable (Marques-Silva et al. 2021), and therefore are not suitable for image classifiers.

In this paper, we show that causal explanations enjoy all the formal properties of logic-based explanations, while not putting any restrictions on the model and being efficiently computable for black-box image classifiers. We prove equivalence between abductive explanations and causal explanations and introduce a variation of causal explanations equivalent to contrastive explanations. Furthermore, we augment the actual causality framework with the model’s confidence in the classification, introducing $\delta$-confident explanations, and use these to produce more fine-grained results. We also introduce a causal version of the completeness property for explanations, following (Srinivas and Fleuret 2019). The differences between explanations of the classification of the same image reveal important information about the decision process of the model. In particular, we call the the difference between a complete explanation and a sufficient one adjustment pixels.

We examine the relationship between contrastive and adjustment pixel sets by examining the semantic distance between the original classification and the classifications of these pixel sets, as illustrated in Figure 1. Our approach allows us to formally subdivide an image into its sufficient pixels, contrastive pixels, and adjustment pixels. We prove complexity results for our definitions, giving a justification to efficient approximation algorithms.

Our algorithms are based on rex (Chockler et al. 2024). In Section 4 we introduce black-box approximation algorithms to compute contrastive, complete, and $\delta$-confident causal explanations for image classifiers. Our algorithms do not require any knowledge of the model, no access to the model internals, nor do they require any specific properties of the model. We implemented our algorithms and present experimental results on three state-of-the-art models and three standard benchmark datasets.

We use the word ‘explanation’ for the remainder of the paper as a convenience. We do not intend, nor seek to imply, that causal explanations are more or less interpretable than other form of explanation. Instead, we are interested in the formal partitioning of pixels in an image into different functional sets. These functional sets reveal important information about the inner workings of the model.

Due to the lack of space, some theoretical and evaluation results are deferred to the supplementary material.

There are many different definitions of explanation in the literature, some are saliency-based (Selvaraju et al. 2017), some are gradient-based (Srinivas and Fleuret 2019), Shapley-based (Lundberg and Lee 2017b) or train locally interpretable models (Ribeiro, Singh, and Guestrin 2016). Logic-based explanations are rather different in having a mathematically precise definition of explanation. Causal explanations, as defined in (Chockler and Halpern 2024) (and their approximations computed by rex (Chockler et al. 2024)), have much more in common with the rigorous definitions of logic-based explanations.

In this section we prove equivalences between definitions and properties satisfied by causal explanations. We start with formalization of logic-based explanations in the actual causality framework. For a given classification problem as defined in Section 2.3, we define a depth $2$ causal model $M$ as follows. The set of endogenous input variables is the set of features $\mathcal{F}$, with each variable $i\in\mathcal{F}$ having a domain $D_{i}$. The output of the classifier is the output variable $O$ of the model, with the domain $K$. An instance $(v,c)$ corresponds to a context $\vec{u}$ for $M$ that assigns to $\mathcal{F}$ the values defined by $v$, and the output is $c$. The set $\mathcal{K}$ of contexts is defined as the feature space $\mathbb{F}=D_{1}\times D_{2}\dots\times D_{m}$. As the classifier is not constant, there exist two inputs $v$ and $v^{\prime}$ such that $(M,v)\models O=c$, $(M,v^{\prime})\models O=c^{\prime}$, and $c\neq c^{\prime}$. It is easy to see that $M$ is a depth-$2$ causal model with all input variables being causally independent.

We use rex (Chockler et al. 2024) as a basis for computing the definitions provided in this paper. rex uses an approximation of causal responsibility to rank features. Responsibility is a quantitative measure of causality and, roughly speaking, measures the amount of causal influence on the classification (Chockler and Halpern 2004). Algorithm 1 details the greedy algorithm for approximation of $\delta$-sufficient-contrastive explanations, that is, explanations which are both sufficient and contrastive. Unfortunately, the precise computation of contrastive explanations is intractable, as follows from the intractability of actual causes in depth-$2$ causal models with independent inputs (Chockler et al. 2024).

We use a responsibility map created by rex (Algorithm 1, 4). A full description of rex can be found in Chockler et al. (2024).  rex approximates the causal responsibility of pixels in an image $I$ by performing an initial partition of $I$. Intuitively, a part of $I$ has the highest causal responsibility if it, by itself, is sufficient for the classification. A part of $I$ is lower in causal responsibility if it is not sufficient by itself, but it is sufficient together with several other parts. A part is completely irrelevant if it is neither. rex starts from large parts and then refines the important parts iteratively.

We use the resulting responsibility map to rank pixels by their responsibility for the desired classification. We use $2$ different sets of contexts, $\mathcal{K}^{+}$ and $\mathcal{K}^{-}$ (Algorithm 1, 5) to discover explanations. $\mathcal{K}^{+}$ is created by inserting pixels into an image created from the baseline occlusion value, as dictated by their approximate causal responsibility. In the case of a typical ImageNet model, which accepts an input tensor $224\times 224$, the total size of $\mathcal{K}^{+}$ is $50176$. In order to calculate the contrast of this, we also consider the opposite: the context $\mathcal{K}^{-}$, which replaces pixels from the original image with the baseline value, in accordance with their ranking by responsibility. The procedure is similar to the one used for insertion/deletion curves (Petsiuk, Das, and Saenko 2018). $\mathcal{K}^{+}$ ($\mathcal{K}^{-}$) is the set of contexts of all images produced by inserting (occluding) pixels, based on their pixel ranks, over a baseline value. In our experiments, we use the same baseline value for both $\mathcal{K}^{+}$ and $\mathcal{K}^{-}$. Our algorithm computes explanations which are both sufficient and contrastive.

Algorithm 2 details the procedure for extracting the adjustment pixels, given an image and a $\delta$-sufficient-contrastive explanation. In practice, the equality of 8 is difficult to achieve. Our implementation allows for a user-provided degree of precision. In our experiments, we set this to $4$ decimal places.

In this section, we present an analysis of various models and datasets viewed through the lens of contrast and completeness. To the best of our knowledge, there are no other tools that compute sufficient, contrastive and complete explanations for image classifiers. As shown in Section 3.2, these causal explanations have the same formal properties as logic-based explanations, but are also efficiently computable. Our algorithms have been included in the publicly available XAI tool rex ¹¹1https://github.com/ReX-XAI/ReX. rex was used with default parameters for all experiments, in particular, the random seed is $0$, baseline value $0$, and the number of iterations was $20$.

To the best of our knowledge, no-one has previously investigated the relationship of original classifications to contrastive explanations. Due to the hierarchical nature of the ImageNet dataset—it forms a tree—we can calculate the shortest path between the original classification and its contrastive class. We do the same with the completeness requirements, isolating the adjustment pixels, classifying them and calculating the shortest path to the original classification. For reasons of space, we include only a representative selection of results here. Complete results may be found in the supplementary material.

We examine $3$ models, all from TorchVision: a ResNet50, a MobileNet and Swin_t. All models were used with their default preprocessing procedures as provided by TorchVision. We also used $3$ different datasets, ImageNet-1K validation (approximately $4000$ images) (Russakovsky et al. 2015), PascalVoc20212 (Everingham et al. 2012) and a dataset of ‘complex’ images, ECSSD (Shi et al. 2016). We chose these datasets due to their being publicly available and well-studied. On a NVidia A100 GPU running Ubuntu LTS 22.04 we found that runtime varied greatly depending on the model under test. The ResNet50 and MobileNet were both very quick, taking approximately $6$ seconds per image. The Swin_t model was slower, taking approximately $16$ seconds per image. Our implementation makes extensive use of batched inference for efficiency, so models which do not accept batches will likely be slower.

Figure 4 shows the relative sizes of sufficient, contrastive and adjustment pixel sets for the $3$ different models on ImageNet. These are $\delta$-explanations where $\delta=1$. See supplementary material for other models and settings of $\delta$ . In general, ResNet50 requires the fewest pixels for both sufficiency and contrast, and also has very low adjustment pixel size. MobileNet and Swin_t appear to be much more similar in their behavior, though Swin_t has slightly greater contrastive requirements in general.

Figure 6 shows the shortest path between the original classification and its contrast class, according to the ImageNet hierarchy. In general, across all models, the distance between the two classes is not great, given that the maximum distance is $24$. This is not always the case, however: Figure 3(a) shows an example of a (mis-)classification where the contrast classification is ‘moped’. The adjustment pixels are classified as ‘picket fence’. It is worth noting that the initial confidence was already low on this image. At the other extreme Figure 6 reveals a few cases where the distance between the original classification and its contrast was very small. Manual inspection reveals that these cases represent refinements inside a larger class. Figure 5, for example, reveals the ResNet50 model required the highlighted pixels to refine the classification to colobus monkey. Without them, the classification is still monkey, but a different subclass – guenon.

We have demonstrated the concordances between logic-based and causal explanations. We have also introduced definitions for causal contrastive explanations and causal complete explanations. We have created algorithms for approximating these definitions and incorporated them into the tool rex. We have computed contrastive and complete explanations on $3$ different datasets with $3$ different models. We find that different models have different sufficiency, contrastive and adjustment requirements. Finally, we have examined the relationship between the original, contrastive and adjustment pixel predictions.