An ASP-Based Approach to Counterfactual Explanations for Classification

Providing explanations to results obtained from machine-learning models has been recognized as critical in many applications, and has become an active research direction in the broader area of explainable AI, and explainable machine learning, in particular [23]. This becomes particularly relevant when decisions are automatically made by those models, possibly with serious consequences for stake holders. Since most of those models are algorithms learned from training data, providing explanations may not be easy or possible. These models are or can be seen as black-box models.

In AI, explanations have been investigated in several areas, and in particular, under actual causality [16], where counterfactual interventions on a causal model are central [24]. They are hypothetical updates on the model’s variables, to explore if and how the outcome of the model changes or not. In this way, explanations for an original output are defined and computed. Counterfactual interventions have been used with ML models, in particular with classification models [21, 27, 26, 17, 10, 20, 6].

In this work we introduce the notion of causal explanation as a set of feature value for the entity under classification that is most responsible for the outcome. The responsibility score is adopted and adapted from the general notion of responsibility used in actual causality [9]. Experimental results with the responsibility score, and comparisons with other scores are reported in [6]. We also introduce answer-set programs (ASPs) that specify counterfactual interventions and causal explanations, and allow to specify and compute the responsibility score. The programs can be applied with black-box models, and with rule-based classification models.

As we show in this work, our declarative approach to counterfactual interventions is particularly appropriate for bringing into the game additional declarative semantic knowledge, which is much more complicated to do with purely procedural approaches. In this way, we can combine logic-based specifications, and use the generic and optimized solvers behind ASP implementations.

This paper is structured as follows. Section 2 introduces the background, and the notions of counterfactual intervention and causal explanation; and the explanatory responsibility score, x-resp, on their basis. Section 3 introduces ASPs that specify causal explanations, the counterfactual ASPs. Section 4 argues for the need to include semantic domain knowledge in the specification of causal explanations. Section 5 discusses several issues raised by this work and possible extensions.

We consider classification models, $\mathcal{C}$, that are represented by an input/output relation. Inputs are the so-called entities, $\mathbf{e}$, which are represented each by a record (or vector), $\mathbf{e}=\langle x_{1},\ldots,x_{n}\rangle$, where $x_{i}$ is the value $F_{i}(\mathbf{e})\in{\it Dom}(F_{i})$ taken on $\mathbf{e}$ by a feature $F_{i}\in\mathcal{F}=\{F_{1},\ldots,F_{n}\}$, a set of functions. The output is represented by a label function $L$ that maps entities $\mathbf{e}$ to $0$ or $1$, the binary result of the classification. That is, to simplify the presentation, we concentrate here on binary classifiers, but this is not essential.  We also concentrate on features whose domains ${\it Dom}(F_{i})$ take a finite number of categorical values. C.f. Section 4 for the transformation of numerical domains into categorical ones.

Building a classifier, $\mathcal{C}$, from a set of training data, i.e. a set of pairs $T=\{\langle\mathbf{e}_{1},c(\mathbf{e}_{1})\rangle,$ $\ldots,\langle\mathbf{e}_{M},c(\mathbf{e}_{M})\rangle\}$, with $c(\mathbf{e}_{i})\in\{0,1\}$, is one of the most common tasks in machine learning [13]. It is about learning the label function $L$ for the entire domain of values, beyond $T$. We say that $L$ “represents” the classifier $\mathcal{C}$.

Classifiers may take many different internal forms. They could be decision trees, random forests, rule-based classifiers, logistic regression models, neural network-based (or deep) classifiers, etc. [13]. Some of them are more “opaque” than others, i.e. with a more complex and less interpretable internal structure and results [25]. Hence the need for explanations to their classification outcomes. In this work, we are not assuming that we have an explicit classification model, and we do not need it. All we need is to be able to invoke and use it. It could be a “black-box” model.

The problem is the following: Given an entity $\mathbf{e}$ that has received the label $L(\mathbf{e})$, provide an “explanation” for this outcome. In order to simplify the presentation, and without loss of generality, we assume that label $1$ is the one that has to be explained. It is the “negative” outcome one has to justify, such as the rejection of a loan application.

Causal explanations are defined in terms of counterfactual interventions that simultaneously change feature values in $\mathbf{e}$ in such a way that the updated record gets a new label. A causal explanation for the classification of $\mathbf{e}$ is then a set of its original feature values that are affected by a minimal counterfactual interventions. These explanations are assumed to be more informative than others. Minimality can be defined in different ways, and we adopt an abstract approach, assuming a partial order relation $\preceq$ on counterfactual interventions.

Several minimality criteria can be expressed in terms of partial orders, such as:  (a)  $\iota_{1}\leq^{s}\iota_{2}$ iff $\pi_{1}(\iota_{1})\subseteq\pi_{1}(\iota_{2})$, with $\pi_{1}(\iota)$ the projection of $\iota$ on the first position.  (b)  $\iota_{1}\leq^{c}\iota_{2}$ iff $|\iota_{1}|\leq|\iota_{2}|$. That is, minimality under set inclusion and cardinality, resp.  In the following, we will consider only these; and mostly the second.

Notice that, by taking a projection, the partial order $\preceq^{s}$ does not care about the values that replace the original feature values, as long as the latter are changed.  Furthermore, given $\mathbf{e}$, it would be good enough to indicate the features whose values are relevant, e.g. $\epsilon_{7}=\{F_{2}\}$ in the previous example. However, the introduced notation emphasizes the fact that the original values are those we concentrate on when providing explanations.

Clearly, every $\preceq^{c}$-minimal explanation is also $\preceq^{s}$-minimal. However, it is easy to produce an example showing that a $\preceq^{s}$-minimal explanation may not be $\preceq^{c}$-minimal.

This definition characterizes explanations as sets of (interventions on) features. However, it is common that one wants to quantify the “causal strength” of a single feature value in a record representing an entity [20, 6], or a single tuple in a database (as a cause for a query answer) [22], or a single attribute value in a database tuple [3, 4], etc.  Different scores have been proposed in this direction, e.g. SHAP in [20] and Resp in [6]. The latter has it origin in actual causality [16], as the responsibility of an actual cause [9], which we adapt to our setting.

Notice that (b) can be stated as  $\mbox{\sf x-resp}_{\mathbf{e},F}(v):=\frac{1}{|\epsilon^{\star}|}$, with  $\epsilon^{\star}=\mbox{\sf argmin}\{|\epsilon|~{}:~{}\epsilon$ $\mbox{is s-explanation with }\langle F,v\rangle\in\epsilon\}$.

Adopting the usual terminology in actual causality [16], a counterfactual value-explanation for $\mathbf{e}$’s classification is a value-explanation $v$ with $\mbox{\sf x-resp}_{\mathbf{e}}(v)=1$, that is, it suffices, without company of other feature values in $\mathbf{e}$, to justify the classification.  Similarly, an actual value-explanation for $\mathbf{e}$’s classification is a value-explanation $v$ with $\mbox{\sf x-resp}_{\mathbf{e}}(v)>0$. That is, $v$ appears in an s-explanation $\epsilon$, say as $\langle F,v\rangle$, but possibly in company of other feature values. In this case, $\epsilon\smallsetminus\{\langle F,v\rangle\}$ is called a contingency set for $v$  [22].  It turns out that maximum-responsibility value-explanations appear in c-explanations.

Entities will be represented by a predicate with $n+2$ arguments $E(\cdot;\cdots;\cdot)$. The first one holds a record (or entity) id (which may not be needed when dealing with single entities). The next $n$ arguments hold the feature values.¹¹1For performance-related reasons, it might be more convenient to use $n$ 3-are predicates to represent an entity with an identifier, but the presentation here would be more complicated.The last argument holds an annotation constant from the set $\{\mbox{\bf\sf o},\mbox{\bf\sf do},\mathbf{\star},\mbox{\bf\sf s}\}$. Their semantics will be specified below, by the generic program that uses them.

Initially, a record $\mathbf{e}=\langle x_{1},\ldots,x_{n}\rangle$ has not been subject to interventions, and the corresponding entry in predicate $E$ is of the form $E(\mathbf{e};\bar{x};\mbox{\bf\sf o})$, with $\bar{x}$ an abbreviation for $x_{1},\ldots,x_{n}$, and constant o standing for “original entity”.

When the classifier gives label $1$ to $\mathbf{e}$, the idea is to start changing feature values, one at a time. The intervened entity becomes then annotated with constant do in the last argument. When the resulting intervened entities are classified, we may not have the classifier specified within the program. For this reason, the program uses a special predicate $\mathcal{C}[\cdot;\cdot]$, whose first argument takes (a representation of) an entity under classification, and whose second argument returns the binary label. We will assume this predicate can be invoked by an ASP as an external procedure, much in the spirit of HEX-programs [11, 12].  Since the original instance may have to go through several interventions until reaching one that switches the label to $0$, the intermediate entities get the “transition” annotation $\mathbf{\star}$.  This is achieved by a generic program.

Counterfactual interventions in the presence of semantic conditions requires consideration. As the following example shows, not every intervention, or combination of them, may be admissible [5]. It is in this kind of situations that declarative approaches to counterfactual interventions, like the one presented here, become particularly useful.

Another situation where not all interventions are admissible occurs when features take continuous values, and their domains have to be discretized. The common way of doing this, namely the combination of bucketization and one-hot-encoding, leads to the natural and necessary imposition of additional constraints on interventions, as we will show.  Through bucketization, a feature range is discretized by splitting it into finitely many, say $N$, usually non-overlapping intervals. This makes the feature basically categorical (each interval becoming a categorical value). Next, through one-hot-encoding, the original feature is represented as a vector of length $N$ of indicator functions, one for each categorical value (intervals here) [6]. In this way, the original feature gives rise to $N$ binary features. For example, if we have a continuous feature “External Risk Estimate” (ERE), its buckets could be: $[0,64),[64,71),[71,76),[76,81),$ $[81,\infty)$. Accordingly, if for an entity $\mathbf{e}$, $\mbox{\sf ERE}(\mathbf{e})=65$, then, after one-hot-encoding, this value is represented as the vector $[0,1,0,0,0,0]$, because $65$ falls into the second bucket.

In a case like this, it is clear that counterfactual interventions are constrained by the assumptions behind bucketization and one-hot-encoding. For example, the vector cannot be updated into, say $[0,1,0,1,0,0]$, meaning that the feature value for the entity falls both in intervals $[64,71)$ and $[76,81)$.  Bucketization and one-hot-encoding can make good use of program constraints, such as  $\longleftarrow\mbox{\sf ERE}(\mathbf{e};x,1,y,1,z,w;\mathbf{\star})$, etc.  Of course, admissible interventions on predicate ERE could be easily handled with a disjunctive rule like that in P3., but without the “transition” annotation $\mathbf{\star}$. However, the ERE record is commonly a component of a larger record containing all the feature values for an entity [6]. Hence the need for a more general and uniform form of specification.

This work is about interacting with possibly external classifiers and reasoning with their results and potential inputs. That is, the classifier is supposed to have been learned by means of some other methodology. In particular, this is not about learning ASPs, which goes in a different direction [18].

We have treated classifiers as black-boxes that are represented by external predicates in the ASP. However, in some cases it could be the case that the classifier is given by a set of rules, which, if compatible with ASPs, could be appended to the program, to define the classification predicate $\mathcal{C}$.  The domains used by the programs can be given explicitly. However, they can be specified and extracted from other sources. For example, for the experiments in [6], the domains were built from the training data, a process that can be specified and implemented in ASP.

The ASPs we have used are inspired by repair programs that specify and compute the repairs of a database that fails to satisfy the intended integrity constraints [8]. Actually, the connection between database repairs and actual query answer causality was established and exploited in [3]. ASPs that compute attribute-level causes for query answering were introduced in [4]. They are much simpler that those presented here, because, in that scenario, changing attribute values by nulls is good enough to invalidate the query answer (the “equivalent” in that scenario to switching the classification label here). Once a null is introduced, there is no need to take it into account anymore, and a single “step” of interventions is good enough.

Here we have considered only s- and c-explanations, specially the latter. Both embody specific and different, but related, minimization conditions. However, counterfactual explanations can be cast in terms of different optimization criteria [17, 26]. One could investigate in this setting other forms on preferences, the generic $\preceq$ in Definition 1, by using ASPs as those introduced in [14]. These programs could also de useful to compute (a subclass of) s-explanations, when c-explanations are, for some reason, not useful or interesting enough. The ASPs, as introduced in this work, are meant to compute c-explanations, but extending them is natural and useful.

This article reports on preliminary work that is part of longer term and ongoing research. In particular, we are addressing the following:  (a) multi-task classification.   (b) inclusion of rule-based classifiers. (c) scores associated to more than one intervention at a time [6], in particular, to full causal explanations.  (d) experiments with this approach and comparisons with other forms of explanations.  However, the most important direction to explore, and that is a matter of ongoing work, is described next.