Improving Knowledge Distillation with
Teacher’s Explanation

The idea of KD first appeared in [2], where the authors showed that it is possible to distill the knowledge of an ensemble of machine learning models to a single model by matching logits. Subsequently, [1] popularized the technique to design low-complexity models. These models are often easy to deploy and provide faster inference.

There have been several extensions to the original KD framework. The authors of [3] unified Hinton’s KD with the privileged teacher framework of [4] from a generalization error perspective. The authors of [5] further proposed gradual KD through a teaching assistant. In [6], the authors argued that KD is equivalent to label smoothing regularization and proposed a teacher-free distillation framework where the student is self-taught or it learns from a designed regularization distribution. A data-free distillation method was introduced in [7] based on contrastive model inversion, where a generative model was used to obtain synthetic data for distillation. The authors of [8] presented decoupled KD, enabling efficient distillation for target class and non-target classes. This work also demonstrated the efficacy of logit-based distillation. In [9] and [10], the authors investigated online KD, where the teacher and the student learn together mutually from each other.

None of these works consider teachers that explain the predictions that are taught to the students. In contrast, our work shows that substantial learning improvement can be achieved by a teacher that explains.

The hidden-representation distillation framework has been developed in parallel to KD. In this framework, the student attempts to match the teacher’s intermediate layer outputs, instead of the teacher’s logits or final inference outcomes as in KD. In [12], the student is a thin model and its latent representation is aligned with the teacher’s intermediate layer hints using a projector. In [13] and [14], the dependency in [12] upon the projector is removed by attention transfer and similarity preservation over training samples. The authors of [15] proposed paraphrasing a complex teacher by extracting so-called factors from the hidden layers. The authors of [16] and [17] proposed distillation via contrastive representation and softmax regression, respectively. In [18], the authors explored the distillation of relational knowledge between different samples as characterized by the teacher. An information-theoretic knowledge transfer scheme was introduced in [19] that maximizes mutual information between the teacher and the student. The authors of [20] investigated the distillation of activation boundaries produced by the hidden neurons where a student matches the teacher’s boundaries instead of the output of the layers. A structured knowledge distillation framework was proposed in [21] for semantic segmentation. In [22], the channel-wise probability maps are used for distillation so that the student learns the most salient parts of each channel for dense prediction. Cross-layer matching of hidden representations was studied in [23] and [24], which transfers knowledge via connection paths between different stages of the teacher and the student network.

However, learning directly from the teacher’s hidden layers may not be always effective. Since the student model usually has far lower complexity than the teacher, trying to match the teacher’s hidden layers may distract from the student’s final classification task. In contrast, KED constructs a teacher that offers explicit explanations on groups of features, and it allows a hierarchy of different numbers of superfeatures. Furthermore, a distinguishing feature of KED is that the explanations can be considered as Shapley values [30] of a cooperative game. More importantly, as shown later, KED can be augmented with existing hidden-representation distillation methods in scenarios when they are effective.

To design an explaining teacher, we must circumvent the general lack of interpretability in modern machine learning techniques. Interpretable AI deals with this problem by providing explanations for the model’s predictions. This is often achieved through feature attribution. The authors of [25] proposed local interpretable model-agnostic explanation (LIME), which provides insights about the classifier’s perception for a given sample. The authors of [26] unified existing feature attribution methods in the Shapley value-based framework and introduced low complexity algorithms for computing approximate Shapley values. The cooperative game defined in [27] was generalized in [28, 29] using the set of superfeatures as players and thus obtaining superfeature-based explanation. However, as detailed in Section IV-A, the explanations generated by the above methods are local to specific samples and hence not conducive to knowledge transfer in distillation. A new approach is needed to integrate interpretable AI and knowledge distillation in our KED framework.

It is nontrivial to add explanation to the teacher in KD. For a sample $\mathbf{x}\in\mathcal{X}$ and a pretrained teacher $f$, the existing feature attribution-based explanation methods [27, 26] quantify the contribution of each feature to the final prediction $f(\mathbf{x})$. However, this explanation is local and specific to the sample $\mathbf{x}$, in the sense that the same feature value can be mapped to different explanations for different samples and thus the underlying relation between input features and explanation becomes one-to-many. Thus the target function between the features and the teacher’s explanation obtained using these methods is simply undefined. Therefore, a student cannot estimate the target function from such an explanation in a distillation framework. This argument holds also for the superfeature-based explanation methods [25, 28, 29].

The limitation of existing explanation methods motivates us to develop more interpretable teachers instead of directly applying current methods to explain black-box KD teachers. To facilitate effective learning by students, interpretable teachers should be learning models hardwired to provide consistent explanations over different samples. In the next section, we will see that the function between the features and their explanation is built into the architecture of the proposed superfeature-explaining teachers. Therefore, this class of teachers is perfectly suitable for guiding the students in a distillation setting.

In one extreme, the black-box teacher in the original KD does not provide any explanation to the student. In the other extreme, however, a teacher that provides an explanation for each feature may be too restrictive. Therefore, a more general approach is to obtain an explanation for groups of features. In this work, we consider superfeature-based explanation for constructing an interpretable teacher in a distillation framework. We define superfeatures as disjoint groups of features. Let $\mathbf{x}\in\mathcal{X}$ be the feature vector of a sample. Then, we can group the features in $\mathbf{x}$ as a set of superfeatures $\{\mathbf{x}_{1},...,\mathbf{x}_{M}\}$ where $\mathbf{x}_{m},\;1\leq m\leq M,$ is the $m$-th superfeature. We will defer the discussion on how to group features to Section IV-D.

Given an input $\mathbf{x}$, any classifier’s job is to estimate the distribution $p(y|\mathbf{x})$ for each particular class label $y\in\mathcal{Y}$. To motivate our design, let’s first suppose we have $M$ independent superfeatures. Then, by Bayes’ rule, we have

where $p(y)$ is the prior of class $y$. Thus $p(y|\mathbf{x}_{m})$ may be viewed as the contributions of the superfeature $\mathbf{x}_{m}$ to the prediction $p(y|\mathbf{x})$. We will use $p(y|\mathbf{x}_{m}),\forall m$, as the teacher’s explanation for its prediction $p(y|\mathbf{x})$.

Figure 1 illustrates the architecture of a superfeature-explaining teacher with $M$ superfeatures. This teacher estimates the contribution of each superfeature using a subnet, which is a neural network with a softmax layer at the output. We compute the logarithm of the softmax outputs from all the subnets and sum them up to produce the total logit adjusted by the prior. An ultimate softmax function is applied to this total logit in order to obtain the final prediction $p(y|\mathbf{x})$. Both the teacher’s final prediction $p(y|\mathbf{x})$ and its superfeature explanation $p(y|\mathbf{x}_{m}),\forall m$, will be used as inputs to train the student model.

We remark that there is a hierarchy of superfeature-explaining teachers according to the number of superfeatures. A black-box teacher may be viewed as having only one superfeature, i.e., the set of all features. On the other hand, the generalized additive model, e.g., NAM [31] may be viewed as having $d$ superfeatures, i.e., each feature is a superfeature. The hierarchy of superfeature-explaining teachers is shown in Figure 2. At the top of this hierarchy sits type-1 teachers or the black-box teachers that are the most unrestricted. Type-1 teachers do not assume independence between their input features and therefore theoretically can estimate any distribution that can be estimated by the types below it. At the bottom of this hierarchy is type-$d$ teachers that are the most restricted as they assume independence of all the features. As we go down the hierarchy, we see more interpretability of the teacher model at the expense of a drop in its prediction accuracy. The proposed KED in this work can be applied to any type. In particular, for type-1 teachers, our method reduces to Hinton’s KD.

Interestedly, an alternative interpretation of superfeature-explaining teachers can be obtained using cooperative games. A cooperative game $\mathcal{G}=\langle\mathcal{N},v\rangle$ is defined by a set of players, $\mathcal{N}$, and a value function, $v:2^{\mathcal{N}}\rightarrow\mathbb{R}$. The Shapley value [30] of each player measures the average marginal contribution of the player to the game’s outcome. Let $\mathcal{N}$ be the set of superfeatures in KED, and we define the value function for any coalition $\mathcal{S}$ as

Note that the value of the grand coalition in this game is the total logit, i.e.,

For any sample $\mathbf{x}$, the Shapley value of the $m$-th superfeature can be obtained as

where $v(\mathcal{S}\cup\{\mathbf{x}_{m}\})$ indicates the value of the game when the $m$-th superfeature is included in the coalition $\mathcal{S}$. Substituting (5) in (7), the Shapley value of the $m$-th superfeature in this game is given by

Hence, we establish that the outputs of the superfeature-explaining teacher are Shapley values.

In the KED framework, a type-$M$ student $g$ is trained with the hard labels from $\mathcal{D}$, and the soft labels and explanations provided by a type-$M$ teacher $f$. As in the original KD, the soft labels are the teacher’s predictions $f(x)$ passed through function $\sigma_{T}(\cdot)$ as defined in (2). Let $f_{m}(\mathbf{x}_{m}),\;1\leq m\leq M,$ be the teacher’s explanation for the $m$-th superfeature. The student outputs $g_{m}(\mathbf{x}_{m})$, which is matched against $f_{m}(\mathbf{x}_{m})$ to compute the loss. We denote the teacher’s as well as the student’s prior by $f_{0}$. We further apply another temperature parameter $\tau$ to the teacher’s explanation passing through function $\sigma_{\tau}(\cdot)$, which is as defined in (2) with $T$ replaced by $\tau$.

The student’s total loss is measured as a weighted combination of individual loss against the hard labels, the soft labels, and the explanations. Therefore, the student’s loss function consists of three terms. The first term is the categorical crossentropy loss between the student’s predictions $g(\mathbf{x})$ and the hard labels $y$. The second term is the KL divergence loss between the student’s soft label predictions $\sigma_{T}(g(\mathbf{x}))$ and the teacher’s soft labels $\sigma_{T}(f(\mathbf{x}))$ at temperature $T$. The third term is the average KL divergence loss over $M$ superfeatures between the student’s outputs $\sigma_{\tau}(g_{m}(\mathbf{x}_{m}))$ and the teacher’s explanation $\sigma_{\tau}(f_{m}(\mathbf{x}_{m})),\;1\leq m\leq M,$ at temperature $\tau$. Thus we write the student’s total loss as

The loss weights $\lambda$ and $\mu$ are two hyperparameters. Figure 3 illustrates the student in the KED framework.

One key aspect of superfeature-explaining teachers is the superfeatures themselves. If there is sufficient domain knowledge, the data can be collected in a way such that the groups of features naturally form independent superfeatures. However, this is not always possible in general machine learning tasks. An alternative strategy is to group strongly dependent features together into a superfeature. Here we use a Hessian-based measure of feature dependency. Referring to Figure 1, we define the total logit $z$ for a type-$M$ superfeature-explaining teacher as follows:

We see that $z$ is additively separable over the superfeatures $\mathbf{x}_{1},...,\mathbf{x}_{M}$. Therefore, if two features $x_{i}$ and $x_{j}$ belong to two different superfeatures, the cross-partial derivatives of $z$ with respect to those two features should be zero, i.e.,

Thus, we measure the dependency between each pair of features by the entries of $d\times d$ Hessian matrix of $z$. Note that only the magnitude of Hessian entries is significant for determining dependency.

We note that the total logit $z\approx\log p(y|\mathbf{x})$. We first estimate $p(y|\mathbf{x})$ using a type-$1$ black-box teacher. Then, we approximate the expectation of the Hessian matrix, $\mathbf{\bar{H}}\approx\sum_{y\in\mathcal{Y}}\mathbb{E}[\nabla\otimes\nabla\log p(y|\mathbf{x})]$ over a random subset of training samples. Let $\mathbf{\bar{H}}_{\text{abs}}$ denote the matrix of absolute values of the entries in $\mathbf{\bar{H}}$. We define the $d\times d$ pairwise dependency matrix as $\mathbf{W}=\mathbf{\bar{H}}_{\text{abs}}+\mathbf{\bar{H}}_{\text{abs}}^{T}$. Thus the $(i,j)$ entry of this matrix is

Now we are ready to build a weighted undirected graph $G=(V,E)$ where $V$ is the set of vertices, i.e., $d$ features, $E$ is the edges between each pair of those features, and each edge is assigned a weight given by the matrix $\mathbf{W}$. Then, the grouping of features into approximately independent superfeatures corresponds to finding approximately independent communities in graph $G$. We may use any community detection method, e.g., Louvain [32], to find such superfeatures. Even though many practical problems require a large number of features and hence a large graph $G$, Louvain is a simple and fast algorithm that can detect communities efficiently in $O(|V|\log|V|)$ run time.

Algorithm 1 summarizes the above procedure for constructing the superfeatures. The detected communities constitute our set of superfeatures. Typically, we find a higher or lower number of superfeatures by tuning the resolution parameter of the Louvain method.

Since the pixels of the input images are usually unstructured, a CNN has to extract its own features. As mentioned in Section IV-D, if we can design the features such that they form independent groups, then we do not need any posterior process of constructing superfeatures. Thus, for building a type-$M$ CNN-based superfeature-explaining teacher, it is enough to split the architecture into $M$ subnets at some intermediate layer so that during training, the backpropagation algorithm can construct the superfeatures automatically. As the explanations from $M$ subnets are combined at the output end using (4), reducing the training loss is equivalent to promoting independence among the superfeatures.

An efficient approach for designing low-complexity CNN student models is to split the architecture along the filter dimension (see Figure 4). Let $F$ be the number of output filters of the shared convolutional layers. If we split it into $M$ superfeatures, the input to each subnet will have only $F/M$ filters. We remark that the modern CNN architectures, particularly Resnets, are narrow and thus splitting into $M$ subnets often creates a bottleneck at each subnet for signal propagation. In such cases, we use a projector to increase the filter dimension before splitting into $M$ superfeatures. A projector $r:\mathbb{R}^{H\times W\times F}\rightarrow\mathbb{R}^{H\times W\times F^{\prime}}$ is a function with $F^{\prime}>F$, where $H\times W$ is the spatial dimension of the shared convolutional layer output. We use a 1$\times$1 convolutional layer as projector, which contains very few parameters. Such a linear projection also has whitening effect which decorrelates the filters and helps in promoting independence among the superfeatures. In Section VI, we will see that with this approach, the KED type-$M$ teacher can achieve nearly identical classification accuracy as the original black-box CNN, leading to superior student performance.

KED can be straightforwardly augmented with existing hidden-representation distillation methods. Let $h^{g}_{m}(\mathbf{x}_{m})$, $1\leq m\leq M$, be the student’s representations that we want to align with the teacher’s intermediate layer outputs $h^{f}_{m}(\mathbf{x}_{m})$, $1\leq m\leq M$. Adding the loss between the teacher’s and the student’s hidden representations, we rewrite (9) as

where $\rho$ is an additional hyperparameter. The loss term $\mathcal{L}_{\text{h}}$ is generic and can be set according to most existing hidden-representation distillation methods, e.g., FitNet [12], attention transfer [13], and similarity-preserving distillation [14].

In scenarios where only a small training dataset is available, it is important to regularize the student. We propose to use the chimeric set as a natural technique for performance enhancement. For a type-$M$ teacher-student pair, let us rewrite the training dataset $\mathcal{X}$ as $\mathcal{X}=\{\mathcal{X}_{1},...,\mathcal{X}_{M}\}$ where $\mathcal{X}_{m}$ denotes the set of m-th superfeatures among all training samples. Note that $|\mathcal{X}_{m}|=|\mathcal{X}|=K$. Then, we define the chimeric set as the $M$-fold Cartesian product $\mathcal{X}^{M}=\mathcal{X}_{1}\times...\times\mathcal{X}_{M}$. Thus, $|\mathcal{X}^{M}|=K^{M}$. This set contains only $K$ in-distribution samples and the rest are out-of-distribution (OOD) samples.

Given a type-$M$ teacher’s explanations for all the samples in the training dataset $\mathcal{X}$, we know the contribution of superfeatures in $\mathcal{X}_{m},\forall m$. Therefore, we can calculate the teacher’s explanations and predictions for all the samples in the chimeric set. Note that the chimeric samples do not have any true labels. However, to apply the framework of KED, we still treat the teacher’s predictions converted to hard labels as if they were true labels. Then we can train a type-$M$ student over the chimeric set as described in Section IV-C.

We remark that, for KD with a black-box teacher, training on the chimeric set may result in “catastrophic forgetting” [33] for black-box students, i.e., seeing OOD samples, the student may forget the information learned from the in-distribution samples. In contrast, for KED students, the chimeric set serves as a special data augmentation technique. This is different from other data augmentation method such as CutMix [34], particularly since here we take advantage of the architecture of type-$M$ students. When being trained on the chimeric set, a type-$M$ student always sees in-distribution superfeatures even in OOD samples. In other words, it learns from the same explanations but different predictions.

We run experiments on classification over various datasets, including the Unicauca network traffic dataset [35], MNIST [36], FashionMNIST [37], CIFAR10 [38], CIFAR100 [38], and Tiny Imagenet [39]. We implement multi-layer perceptrons (MLPs), Resnets [40], wide Resnets [41], and VGGs for the teacher and student models.

For the Unicauca dataset, we group the network traffic application types into three delay classes and categorize flows according to their delay sensitivity. In KD, the teacher has size [200, 200, 200, 200], i.e., four hidden layers with 200 neurons each. The architecture of the student is [50, 50, 50, 50]. For MNIST and FashionMNIST, the KD teacher has size [500, 500]. The black-box student in KD has size [20, 20] for MNIST and [60, 60] for FashionMNIST. For KED, we construct the superfeature-explaining teacher and student by choosing the number of neurons per hidden layer such that they have a similar number of total model parameters as the benchmark KD with a black-box teacher. In particular, we note that a type-$M$ model with $L$ hidden layers has $M(L-1)n_{h}^{2}+(ML+MC+d)n_{h}+MC$ parameters where $n_{h}$ is the number of neurons per hidden layer. Thus, we set this expression equal to the number of model parameters in black-box KD and solve for $n_{h}$.

For CIFAR10, CIFAR100 and Tiny Imagenet, we use standard Resnets, wide Resnets and VGGs. We use the labels “Resnet$n$x$k$,” “WRN-$n$-$k$,” and “VGG$n$” where the variables $n$ and $k$ denote the number of layers and the widening factor, respectively. For KED, type-$M$ models of similar complexity are constructed by splitting the last stack of convolutional and fully connected layers in the aforementioned models.

For all experiments, the default setting for the hyperparameters is $T=10$, $\tau=10$, $\lambda=0.7$, $\mu=0.7$, and $\rho=0.7$. We keep the number of superfeatures fixed at $M=4$. For the Unicauca, MNIST, and FashionMNIST datasets, the teachers and the students, respectively, are trained for 100 epochs with a batch size of 500 and 100. We use RELU activation and the Adam optimizer with a learning rate of $0.001$. The prior for type-$M$ models $p(y)$ is estimated by taking a sample average of the black-box teacher’s predictions over the training set. For picking superfeatures, we apply Algorithm 1 computing the average Hessian of the black-box teacher over a random subset of 1000 training samples. We use the Louvain community detection method in scikit-network version 0.27.1 [42]. We tune the resolution with a stepsize of 0.01. For CIFARs and Tiny Imagenet, we train the teachers and the students for 150 and 75 epochs, respectively, with a batch size of 100. For data augmentation, we pad 4 pixels on each side of a training image, and then apply random crop and random horizontal flip to it. We use RELU activation and the SGD optimizer with Nesterov momentum 0.9. The initial learning rate is set to 0.01 for the first 10 epochs as warm-up period. Then, for CIFARs, we use a learning rate of 0.05 and divide it by 10 after 120 and 140 epochs. The L2 regularization coefficient is set to $5\times 10^{-4}$. For Tiny Imagenet, we use a learning rate of 0.1 and divide it by 10 after 60 and 70 epochs. The L2 regularization coefficient is set to $1\times 10^{-4}$. As described in Section V-A, for CNNs, the superfeatures are created from the output of a projector, which is a 1x1 convolutional layer. We assume uniform prior over the classes and thus eliminate $p(y)$ from (4).

For the experiments on the chimeric set, we randomly generate one million samples and continue training the distilled students for 5 epochs with a batch size of 100. To avoid numerical instability, we have added a small bias of $10^{-15}$ to the argument of the $\log$ function. In all experiments, we obtain a 95% confidence intervals for inference by bootstrapping the test set.

Table 1 presents the test accuracy of KED for the Unicauca dataset varying the student’s training dataset size. We observe that KED substantially outperforms KD and no distillation. For example, the student with 10000 samples in the training dataset achieves 66.78% and 67.51% accuracy without and with KD, respectively. The KED student achieves an accuracy of 68.99%. On the full dataset, the black-box student achieves 74.12% and 72.29% accuracy without and with KD, respectively, whereas the KED student reaches an accuracy of 75.84%. In this case, KD fails to improve the performance of the student. In contrast, KED still benefits the student significantly.

We show further experimental results for the MNIST and FashionMNIST datasets in Tables 2 and 3, respectively. For MNIST, with 10000 samples in the training dataset, the black-box student achieves an accuracy of 93.32%, which increases to 95.15% under KD. In comparison, the KED student achieves an improved accuracy of 96.87%. When trained on the full dataset, the black-box student provides 95.41% and 96.69% accuracy without and with KD, respectively. The KED student reaches an accuracy of 97.52%.

We see a similar trend for the FashionMNIST dataset as well. Training the black-box student using 10000 samples results in a test accuracy of 84.86% and 85.31% under no distillation and KD, respectively. In comparison, the KED student attains 87.50% accuracy. Furthermore, when trained on the full dataset, the black-box student provides an accuracy of 87.96% and 88.11% without and with KD, respectively. KED enables the student to achieve 89.38% accuracy.

In all of these experiments, we observe that the accuracy gap between the KED teacher and the KED student is much narrower compared with their black-box KD counterparts. This clearly demonstrates that a superfeature-explaining teacher can transfer more knowledge than a black-box teacher of similar complexity.

In Tables 4, 5 and 6, we present the test accuracy of KED with CNNs over full datasets of CIFAR10, CIFAR100 and Tiny Imagenet, respectively. Here the labels FitNet, AT, and SP indicate the hidden-representation methods of [12], [13], and [14], respectively.

Let us consider Resnet56x2 over CIFAR10 as shown in Table 4. When distilled to Resnet8 using KD, we achieve 88.53% accuracy. However, the KED student outperforms the KD student, reaching 89.43% accuracy. We notice a similar trend for all other architectures.

Furthermore, take Resnet56x2 over CIFAR100 for another example, which is shown in Table 5. When distilled to Resnet20 using KD, we achieve 69.19% accuracy. However, the KED student outperforms KD reaching 70.94% accuracy. Augmented with FitNet, the KED student further improves to 71.79%, but the same augmentation of KD reaches only 69.42%. We observe a similar trend for all other combinations of teacher and student architectures.

As shown in Table 6, for Tiny Imagenet, when the teacher is a Resnet56x2 model, a black-box Resnet20 student achieves an accuracy of 52.72% with KD. In KED, the student attains 54.96% test accuracy. Augmenting KED with attention transfer, the student’s performance further improves to 55.35%, while the same augmentation of KD achieves only 53.71%. More importantly, the gap between the KED teacher and the KED student is always significantly narrower than in KD.

We show the performance of knowledge distillation using the chimeric set in Tables 7, 8, and 9. We run these experiments with fewer than 10000 samples in the student’s training dataset to illustrate the impact of the chimeric set in a limited-data scenario. For each of these experiments, the student is first trained via the regular KD or KED and then we continue training it on the chimeric set. We observe that for many cases in MNIST and FashionMNIST, the chimeric set improves the performance of both KD and KED. However, for Unicauca, only KED benefits significantly from the chimeric set.

We conduct further experiments to study the impact of the hyperparameters $M$, $T$, $\tau$, $\lambda$ and $\mu$ on KED and to evaluate the performance of Algorithm 1. We perform these experiments on Unicauca and MNIST with 10000 samples in the student’s training dataset. We also provide ablation studies on CIFAR10, CIFAR100, and Tiny Imagenet.

In addition to the above experiments to test the performance of the overall KED framework, here we present a special experiment to directly evaluate the superfeature construction method in Algorithm 1. We combine images and their labels from MNIST and FashionMNIST datasets to create a new dataset MNIST+FashionMNIST with 1568 features and 100 classes. For simplicity of illustration, let’s say MNIST has 10 classes, namely ‘0’,…, ‘9’, and FashionMNIST has 10 classes, namely ‘A’,…, ‘J’. Then the generated MNIST+FashionMNIST dataset has 100 classes, namely ‘0A’,…, ‘9J’. In this combined dataset, we know the natural superfeature composition: there are two superfeatures, one containing MNIST features, and the other containing FashionMNIST features. In Figure 5, we show an example of the combined image and the superfeatures constructed by Algorithm 1. Note that this is different from image segmentation since our algorithm does not utilize pixel arrangement information and thus will discover the superfeatures even if the pixels of the combined image are randomly shuffled. Our experimental setup for this dataset is the same as that for MNIST in Section VI-A.

Table 16 shows the distillation performance using the actual superfeatures, the superfeatures constructed by Algorithm 1, and two sets of randomly generated superfeatures. We observe that the superfeatures constructed by Algorithm 1 achieve similar performance as the actual superfeatures. Furthermore, the superfeatures constructed by Algorithm 1 provides significant performance gain over random groupings.