Rationally Inattentive Utility Maximization for Interpretable Deep Image Classification

The question we address is: Can the decisions of deep CNNs in image classification be explained by a rationally inattentive Bayesian utility maximizer?

This paper uses a data-driven Bayesian revealed preference approach to interpretable deep CNN image classification. Bayesian revealed preference performs a post-hoc analysis of agent decisions. It constructs a generative³³3A generative model is image-independent, and hence provides a global explanation for deep image classification. In contrast, local approximation models for deep image classification are image-specific; they approximate model decisions via tractable functionals in a $\delta$-neighborhood of every input. explanatory model for the agent decisions, parameterized by a utility function and an information acquisition cost. Our approach draws important parallels between human decision making and deep neural networks; namely that deep neural networks satisfy economics based rationality.

By very nature, Bayesian revealed preference reconstructs a set of feasible utility functions and information acquisition costs. Every element in the feasible set explains the deep CNN decisions equally well. The computed utility function induces a preference ordering on the set of image classes. That is, how much a deep CNN prioritizes accurate classification over an inaccurate classification. The information acquisition cost abstracts the penalty incurred by the deep CNN to ‘learn’ an accurate latent feature representation. The key results in this paper are:

1. 1.

   We show that the image classification decisions of deep CNNs satisfy the necessary and sufficient conditions for rationally inattentive utility maximization by a large margin. The margin by which the decisions satisfy these conditions are displayed in Table I. Hence, we establish that rationally inattentive utility maximization is a robust fit to deep image classification. This result is schematically shown in Fig. 1.

2. 2.

   To aid visualization of our interpretable model, we provide a sparsity-enhanced decision test that computes the sparsest utility function and information acquisition cost which rationalizes deep CNN decisions. The sparsest solution yields a parsimonious representation of hundreds of thousands of layer weights of the deep CNNs in terms of a few hundred parameters. The utility function of the sparsest interpretable model also induces a useful preference ordering amongst the set of hypotheses (image labels) considered by the CNN; for example, how much additional priority is allocated to the classification of a cat as a cat compared to a cat as a dog. In classical deep learning, this preference ordering is not explicitly generated. The sparsity results for various deep CNN architectures are displayed in Table II and Fig. 2.

3. 3.

   Our final result demonstrates the usefulness of our interpretable model. We show that the interpretable model computed from CNN decisions can predict the classification accuracy of a CNN trained with arbitrary parameters with accuracy exceeding 94%. This by-passes the need to re-train the deep CNN when its accuracy is observed for a finite number of training parameters. The prediction results are displayed in Table III.

The above results are backed by experiments performed on several deep CNN architectures using the benchmark image dataset, namely, CIFAR-10 [8]. The first two results use deep CNN decisions aggregated over varying training epochs. The third (prediction) result uses deep CNN decisions trained on noisy image datasets parameterized by the noise variance.

Since we study interpretable deep learning using behavioral and micro- economics, we briefly discuss related works in these areas.

Bayesian revealed preference and Rational inattention. Estimating utility functions given a finite sequence of decisions and budget constraints is the central theme of revealed preference in micro-economics. The seminal work of [9, 10] (see also [11]) give necessary and sufficient conditions for the existence of a utility function that rationalizes a finite time series of consumption bundles of a decision-maker. Rationally inattentive models for Bayesian decision making have been studied extensively in [3, 4, 5]. In the last decade, the area of Bayesian revealed preference [6, 7] develops necessary and sufficient conditions to test for rationally inattentive Bayesian utility maximization.

Interpretable ML. Providing transparent models for de-obfuscating ‘black-box’ ML algorithms under the area of interpretable machine learning is a subject of extensive research [12, 13, 14]. Interpretable machine learning is defined in [15] as “the use of machine-learning models for the extraction of relevant knowledge about domain relationships contained in data”.

Since the literature is enormous, we only discuss a subset of works pertaining to interpretability of deep neural networks for image classification [16, 17, 18]. One prominent approach, namely, saliency maps, reconstructs the most preferred or typical image pertaining to each image class the deep neural network has learned [19, 20]. Related work includes creating hierarchical models for determining the importance of image features that determine its label [21]. In this paper, this feature importance is encoded into the utility function that parameterizes our interpretable model. Another approach seeks to provide local approximations to the trained model, local w.r.t the input image [22, 23]. In contrast, our generative interpretable model provides a global black-box approximation for deep image classification. A third approach approximates the decisions of the deep neural networks by a linear function of simplified individual image features [24, 25, 26, 23]. In contrast, our interpretable model fits a stochastic non-linear map that relates the true and predicted image labels. The parameters of the map are obtained by solving a convex feasibility problem parameterized by the deep CNN decisions. Finally, deep neural networks have also been modeled by Bayesian inference frameworks using probabilistic graphical methods [27].

To the best of our knowledge, an economics based approach for the post-hoc analysis of deep neural networks has not been explored in literature. However, we note that behavioral economics based interpretable models have been applied to domains outside interpretable machine learning, for example, in online finance platforms for efficient advertising [28, 29], training neural networks [30] and more recently in YouTube to rationalize user commenting behavior [31]. Finally, due to our recent equivalence result [32], our behavioral economics approach to interpretable deep image classification can be related to classical revealed preference methods [9, 10] in microeconomics.

Bayesian revealed preference aims to determine if the decisions of a Bayesian agent are consistent with expected utility maximization subject to a rational inattention sensing cost. We start by describing the utility maximization model with rational inattention (henceforth called UMRI) for a collection of Bayesian decision makers/agents.

Abstractly, the UMRI model is parameterized by the tuple

With respect to the abstract parametrization of the UMRI model for a collection of Bayesian agents, the following elements constitute the tuple $\Theta$ defined in (1).
Agents: $\mathcal{K}=\{1,2,\ldots,K\}~(K\geq 2)$ indexes the finite set of Bayesian agents.
State: $\mathcal{X}$ is the finite set of ground truths with prior probability distribution $\pi_{0}$. With respect to our image classification context, $\mathcal{X}=\{1,2,\ldots 10\}$ is the set of image classes in the CIFAR-10 dataset and $\pi_{0}$ is the empirical probability distribution of the image classes in the test dataset of CIFAR-10.
Observation and attention strategy: Agent $k\in\mathcal{K}$ chooses attention strategy $\alpha_{k}:\mathcal{X}\rightarrow\Delta(\mathcal{Y})$, a stochastic mapping from $\mathcal{X}$ to a finite set of observations $\mathcal{Y}$. Given state $x$ and attention strategy $\alpha_{k}$, the agent samples observation $y$ with probability $\alpha_{k}(y|x)$. The agent then computes the posterior probability distribution $p(x|y)$ via Bayes formula as

The observation and attention strategy are latent variables that abstractly represent the learned feature representations in the deep image classification context. Bayesian revealed preference theory tests their existence via the convex feasibility test in Theorem 1 below.
Action: Agent $k\in\mathcal{K}$ chooses action $a$ from a finite set of actions $\mathcal{A}$ after computing the posterior probability distribution $p(x|y)$. In the image classification context, $a$ is the image class predicted by the neural network, hence $\mathcal{A}=\mathcal{X}$.
Utility function: Agent $k\in\mathcal{K}$ has a utility function $u_{k}(x,a)\in\mathbb{R}^{+}$, $x\in\mathcal{X},a\in\mathcal{A}$ and aims to maximize its expected value, with the expectation taken wrt the random state $x$ and random observation $y$. A key feature in our approach is to show that the utility function rationalizes the decisions of the deep CNNs (made precise in Definition 1).
Information Acquisition Cost: The information acquisition cost $C(\alpha,\pi_{0})\in\mathbb{R}^{+}$ depends on attention strategy $\alpha$ and prior pmf $\pi_{0}$. It is the sensing cost the agent incurs in order to estimate the underlying state (2). In the context of machine learning, $C(\cdot)$ abstractly captures the ‘learning’ cost incurred during training of the deep neural networks. In rational inattention theory from behavioral economics, a higher information acquisition cost is incurred for more accurate attention strategies (equivalently, more accurate state estimates (2) given observation $y$). We refer the reader to the influential work of [1, 2].

Each Bayesian agent $k\in\mathcal{K}$, aims to maximize its expected utility while minimizing its cost of information acquisition. Hence, the action $a$ given observation $y$, and attention strategy $\alpha_{k}$ are chosen as follows:

Eq. 3,4 in Definition 1 constitute a nested optimization problem. The lower-level optimization task is to choose the the ‘best’ action for any observation $y$ based on the computed posterior belief of the state. The upper-level optimization task is to sample the observations optimally by choosing the ‘best’ attention strategy.

Remark. The multiple Bayesian agents in $\Theta$ have the same state space $\mathcal{X}$, observation space $\mathcal{Y}$, action space $\mathcal{A}$, prior $\pi_{0}$ and cost of information acquisition $C$, but only differ in their utility functions. Bayesian revealed preference theory relies on this crucial constraint on the optimization variables in (3), (4) for detecting optimal behavior in a finite number of agents.

Having described the UMRI  model (collection of rationally inattentive utility maximizers), we are now ready to state our key result. Theorem 1 below says that the decisions of a collection of Bayesian agents is rationalized by a UMRI tuple $\Theta$ if and only if a set of convex inequalities have a feasible solution. These inequalities comprise our Bayesian Revealed Preference (henceforth called BRP) test for rationally inattentive utility maximization.

For notational convenience, the decisions of the Bayesian agents in the UMRI model are compacted into the dataset $\mathbb{D}$ defined as:

In (5), $\pi_{0}\in\Delta^{|\mathcal{X}|-1}$ denotes the prior pmf over the set of states $\mathcal{X}$ in $\Theta$ (1). The variable $p_{k}(a|x)$ is the conditional probability that agent $k\in\mathcal{K}=\{1,2,\ldots,K\}$ takes action $a$ given state $x$. $\mathbb{D}$ characterizes the input-output behavior of the collection of Bayesian agents and serves as the input for BRP feasibility test described below.

Computational Aspects of BRP Test. Suppose the dataset $\mathbb{D}$ is obtained from $K$ Bayesian agents. Then, BRP$(\mathbb{D})$ comprises a feasibility test with $K~(|\mathcal{X}||\mathcal{A}|+1)$ free variables and $K^{2}+K~(|\mathcal{A}|^{2}-|\mathcal{A}|-1)$ convex inequalities. Thus, the number of free variables and inequalities in the BRP feasibility test scale linearly and quadratically, respectively, with the number of observed Bayesian agents.

We now discuss how the above BRP test relates to interpretable image classification using deep CNNs. The BRP convex feasibility test in Theorem 1 comprises two sets of inequalities, namely, the NIAS (No-Improving-Action-Switches) (8) and NIAC (No-Improving-Action-Cycles) (9) inequalities (Algorithm 1). NIAS ensures that the agent takes the best action given a posterior pmf. NIAC ensures that every agent chooses the best attention strategy. BRP test checks if there exist $K$ utility functions and $K$ positive reals that, together with $\mathbb{D}$, satisfy the NIAS and NIAC inequalities.

Image Dataset. In our experiments, we trained and validated the deep CNNs using the CIFAR-10 benchmark image dataset [8]. This public dataset consists of $60000$ $32$x$32$ colour images in $10$ distinct classes (for example, airplane, automobile, ship, cat, dog etc.), with $6000$ images per class. There are $50000$ training images and $10000$ test images. We will use the terms image classes and image labels interchangeably.⁵⁵5Our experiments are confined to the CIFAR-10 dataset for clarity of exposition. Our approach to interpretable deep learning can be easily extended to richer benchmark image datasets like ImageNet and CIFAR-100 (that comprise over a 100 image labels).

Network Architecture and Training Parameters. In this paper, we use $5$ well-known deep CNN architectures for our experiments. 1. LeNet [33], 2. AlexNet [34] 3. VGG16 [35] 4. ResNet-50 [18] 5. Network-in-Network (NiN) [36] The deep CNNs are trained and validated on the CIFAR-10 image dataset, using $3$ learning rate schedules, namely, L.R. 1, L.R. 2 and L.R. 3. All $3$ schedules use the RMSprop optimizer [37] with the decay parameter and maximum training epochs (full passes of the training dataset) set to $10^{-6}$ and $200$, respectively, and initial step size set to $0.01$. The step size is halved every $20,30,40$ epochs, respectively, for L.R. 1, 2 and 3.

Relation to Bayesian revealed preference. We now relate the deep CNN setup to the Bayesian revealed preference framework in Sec. 2. For each CNN architecture, we use the decisions of $K=20$ CNNs, i.e. , $20$ Bayesian agents in the terminology of Sec. 2, for our BRP and S-BRP decision tests. The CNN decisions from $K$ CNNs on the test image dataset of CIFAR-10 are aggregated into dataset $\mathbb{D}$ (5). The results of the decision tests are discussed below. In the deep image classification context, the parameter $p_{k}(a|x)$ in (5) is the probability that the $k^{\text{th}}$ deep CNN classifies an image from category $x$ into category $a$ in the CIFAR-10 test image dataset. The prior $\pi_{0}$ in $\mathbb{D}$ (5) is the empirical pmf over the set of image categories in the CIFAR-10 test dataset. Constructing $\mathbb{D}$ from raw CNN decisions is discussed in Appendix A.3.

Training datasets are often noisy; for example, [38] considers noisy datasets for hand-written character recognition. We now exploit the proposed interpretable model to predict how the deep CNN will perform with a noisy training dataset without actually implementing the deep CNN.

Our predictive procedure is as follows. We first train the CNNs on noisy datasets that are generated by adding simulated Gaussian noise with noise variances chosen from a finite set. ⁸⁸8Injecting artificial noise in training datasets is also used in variational autoencoders for robust feature learning [39, 40]. Then given the CNN decisions, we compute our interpretable model over this finite set of noise variances. Finally, to predict how the CNN will perform for a noise variance not in the set, we interpolate the utility function of the interpretable model at this noise variance. Then given the interpolated utility function and information acquisition cost from our interpretable model, the predicted classification performance is computed by solving convex optimization problem (4). The above procedure is formalized in Algorithm 2. Hence, our interpetable model serves as a computationally efficient method for predicting the performance of a CNN without implementing the CNN. The interpretable model can be viewed as a low dimension projection of the high-dimension CNN with predictive accuracy exceeding 94%.

Remark. An alternative procedure is to directly interpolate the performance over the space of CNN weights (several hundreds of thousands). Due to the high dimensionality, this is an intractable interpolation. In comparison, interpolation over the utility functions in our interpretable model is over a few hundred variables.

Proof of necessity of NIAS and NIAC:

1. 1.

   NIAS (8): For agent $k\in\mathcal{K}$, define the subset $\mathcal{Y}_{a}\subseteq\mathcal{Y}$ so that for any observation $y\in\mathcal{Y}_{a}$, given posterior pmf $p_{k}(x|y)$, the optimal choice of action is $a$ (3). We define the revealed posterior pmf given action $a$ as $p_{k}(x|a)$. The revealed posterior pmf is a stochastically garbled version of the actual posterior pmf $p_{k}(x|y)$, that is,

   $$p_{k}(x|a)=\sum_{y\in\mathcal{Y}}\frac{p_{k}(x,y,a)}{p_{k}(a)}=\sum_{y\in\mathcal{Y}}p_{k}(y|a)p_{k}(x|y)$$

   (16)

   Since the optimal action is $a$ for all $y\in\mathcal{Y}_{a}$, (3) implies:

   	$\displaystyle\hskip-11.38092pt\quad\quad~~\sum_{x\in\mathcal{X}}p_{k}(x|y)(u_{k}(x,b)-u_{k}(x,a))\leq 0$
   	$\displaystyle\hskip-11.38092pt\implies\sum_{y\in\mathcal{Y}_{a}}p_{k}(y|a)\sum_{x\in\mathcal{X}}p_{k}(x|y)(u_{k}(x,b)-u_{k}(x,a))\leq 0$
   	$\displaystyle\hskip-11.38092pt\implies\sum_{y\in\mathcal{Y}}p_{k}(y|a)\sum_{x\in\mathcal{X}}p_{k}(x|y)(u_{k}(x,b)-u_{k}(x,a))\leq 0$
   	$\displaystyle\hskip-11.38092pt\quad\quad~~(\text{since }p_{k}(y|a)=0,~\forall y\in\mathcal{Y}\backslash\mathcal{Y}_{a})$
   	$\displaystyle\hskip-11.38092pt\implies\sum_{x\in\mathcal{X}}\sum_{y\in\mathcal{Y}}p_{k}(y|a)p_{k}(x|y)(u_{k}(x,b)-u_{k}(x,a))\leq 0$
   	$\displaystyle\hskip-11.38092pt\implies\sum_{x\in\mathcal{X}}p_{k}(x|a)(u_{k}(x,b)-u_{k}(x,a))\leq 0~(\text{from }\eqref{eqn:revpos})$

   This is precisely the NIAS inequality (8).

2. 2.

   NIAC (9): Let $c_{k}=C(\alpha_{k})>0$, where $C(\cdot)$ denotes the information acquisition cost of the collection of agents $\mathcal{K}$. Also, let $J(\alpha_{k},u_{k})$ denote the expected utility of the $k^{\text{th}}$ agent given attention strategy $\alpha_{k}$ (first term in RHS of (4)). Here, the expectation is taken wrt both the state $x$ and observation $y$. It can be verified that $J(\cdot,u_{k})$ is convex in the first argument. Finally, for the $k^{\text{th}}$ agent, we define the revealed attention strategy $\alpha_{k}^{\prime}$ over the set of actions $\mathcal{A}$ as

   $$\alpha_{k}^{\prime}(a|x)=p_{k}(a|x),~\forall a\in\mathcal{A},$$

   where the variable $p_{k}(a|x)$ is obtained from the dataset $\mathbb{D}$. Clearly, the revealed attention strategy is a stochastically garbled version of the true attention strategy since

   $$\alpha_{k}^{\prime}(a|x)=p_{k}(a|x)=\sum_{y\in\mathcal{Y}}p_{k}(a|y)\alpha_{k}(y|x)$$

   (17)

   From Blackwell dominance [42] and the convexity of the expected utility functional $J(\cdot,u_{k})$, it follows that:

   $$J(\alpha_{k}^{\prime},u_{j})\leq J(\alpha_{k},u_{j}),$$

   (18)

   when $\alpha_{k}$ Blackwell dominates $\alpha_{k}^{\prime}$. The above relationship holds with equality if $k=j$ (this is due to NIAS (8)). We now turn to condition (4) for optimality of attention strategy. The following inequalities hold for any pair of agents $j\neq k$:

   		$\displaystyle J(\alpha_{k}^{\prime},u_{k})-c_{k}\overset{\eqref{eqn:ineq_BLK}}{=}J(\alpha_{k},u_{k})-c_{k}$
   	$\displaystyle\overset{\eqref{eqn:attentionmaximization}}{\geq}~$	$\displaystyle J(\alpha_{j},u_{k})-c_{j}\overset{\eqref{eqn:ineq_BLK}}{\geq}~J(\alpha_{j}^{\prime},u_{k})-c_{j}.$		(19)

   This is precisely the NIAC inequality (9).

Consider the following UMRI model tuple:

In (20), $C(\cdot)$ is a convex cost since it is a point-wise maximum of monotone convex functions. Further, since NIAC is satisfied, (20) implies $C(\alpha_{k})=c_{k}$. It only remains to show that inequalities (3) and (4) in Definition 1 are satisfied for all agents in $\mathcal{K}$.

1. 1.

   NIAS implies (3) holds. This is straightforward to show since the observation and action sets are identical.

2. 2.

   Information Acquisition Cost (20) implies (4) holds. Fix agent $j\in\mathcal{K}$. Then, for any attention strategy $p(a|x)$, the following inequalities hold.

   	$\displaystyle C(p(a|x))=\max_{k\in\mathcal{K}}~c_{k}+J(p(a|x),u_{k})-J(p_{k}(a|x),u_{k})$
   	$\displaystyle\hskip-11.38092pt\implies J(p_{j}(a|x))-c_{j}\geq J(p(a|x))-C(p(a|x)),~\forall~p(a|x)$
   	$\displaystyle\hskip-11.38092pt\implies p_{k}(a|x)\in\operatorname*{argmax}_{p(a|x)}J(p(a|x),u_{k})-C(p(a|x))=\eqref{eqn:attentionmaximization}.$