Detecting Out-of-distribution Examples via Class-conditional Impressions Reappearing

The goal of out-of-distribution (OOD) detection is to distinguish anomalous inputs ($\mathcal{D}_{\text{out}}$) from original training data ($\mathcal{D}_{\text{in}}$) based on a well-trained classifier $f_{\theta}$. This problem can be considered as a binary classification task with a score function $\mathcal{S}(\cdot)$. Formally, given an input sample $x$, the level-set estimation is:

$$g(x)=\begin{cases}\text{out},&\text{if}\quad\mathcal{S}(x)>\gamma\\ \text{in},&\text{if}\quad\mathcal{S}(x)\leq\gamma\\ \end{cases}$$

(1)

In our work, lower scores indicate that the sample $x$ is more likely to be classified as in-distribution (ID) and $\gamma$ represents a threshold for separating the ID and OOD.

Under a Gaussian assumption about feature statistics in the network, feature representations at one layer are denoted as $\textbf{Z}\in\mathbb{R}^{d\times d}$. For training distribution $\mathcal{X}$, each $z\in\textbf{Z}$ follows the gaussian distribution $\mathcal{N}(\mu_{\text{in}},\sigma^{2}_{\text{in}})$. Normally the statistics of $\mathcal{X}$ are calculated from training samples. For the purposes of estimation efficiency [9, 7] and data privacy [8], several works propose that running average from the batch normalization (BN) layer can be an alternative to original in-distribution statistics. During training time, BN expectation $\mathbb{E}_{\text{bn}}(z)$ and variance $\text{Var}_{\text{bn}}(z)$ are updated after a batch of samples $\mathcal{T}$:

$$\begin{matrix}\mathbb{E}_{\text{bn}}(z)\leftarrow\lambda\mathbb{E}_{\text{bn}}(z)+(1-\lambda)\frac{1}{|\mathcal{T}|}\sum_{x_{b}\in\mathcal{T}}\mu(x_{b})\\ \text{Var}_{\text{bn}}(z)\leftarrow\lambda\text{Var}_{\text{bn}}(z)+(1-\lambda)\frac{1}{|\mathcal{T}|}\sum_{x_{b}\in\mathcal{T}}\sigma^{2}(x_{b})\end{matrix}$$

(2)

Therefore the in-distribution statistics are approximately estimated: $\mathbb{E}_{\text{in}}(z)\approx\mathbb{E}_{\text{bn}}(z),\text{Var}_{\text{in}}(z)\approx\text{Var}_{\text{bn}}(z)$.

Feature statistics have been applied in some advanced post-hoc methods and reach striking performances [2, 7, 10]. Their paradigm involves utilizing statistical data from the feature map as a measure of difference and calculating the final score by weighting and summing the scores from each layer. Prior studies obtain the weights via training in in-distribution (ID) images [7, 10] or even additionally estimating on some OOD datasets [2, 11]. Since training (ID) samples are unreachable in our setting, BN statistics are readily available resources from pretrained model $f_{\theta}$ for data-free OOD detection.

Previous works in the data-free knowledge distillation[8, 12, 13] prove that teacher-supervised synthesized images are as effective as the natural data in knowledge transferring. A reasonable assumption is that these pseudo samples constrained by the model’s BN statistics recall the impression of training data. Therefore, unlike the direct use of BatchNorm mean [7], we consider an inverting and recollecting method can be more appropriate in the data-free scenario.

The framework of our method detects an anomalous input $x$ by computing layer-wise deviations $\Delta_{l}(\cdot)$. With the scaling values $\alpha^{c}_{l}$ for class $c$, score function $S(\cdot)$ is formulated as,

$$S(x)=\sum\limits_{l=1}^{L}\alpha^{c}_{l}\Delta_{l}(x)$$

(3)

where class $c$ is determined by maximum softmax probability (MSP) [1]: $c=\mathop{\text{argmax}}_{i\in[1,C]}f_{\theta}(x)_{i}$. Suppose unknown inputs $\{x\}$ from distribution $\mathcal{Q}$ and original training data $\{x_{\text{in}}\}$ from $\mathcal{X}$. The deviation between $\mathcal{X}$ and $\mathcal{Q}$ can be defined following Integral Probability Metrics[14]: $\text{sup}_{\psi}(\mathbb{E}_{x\in\mathcal{Q}}[\psi(x)]-\mathbb{E}_{x_{\text{in}}\in\mathcal{X}}[\psi(x_{\text{in}})])$, where $\psi(\cdot)$ denotes a witness function. Consider the real condition that only a single input $x$ from $\mathcal{Q}$ is measured. Specially, we further estimate the class-conditional deviation by,

$$\text{sup}_{\psi}(\psi(x)-\mathbb{E}_{x_{\text{in}}\in\mathcal{X}}[\psi(x_{\text{in}})\mid y=c])$$

(4)

As the in-distribution statistics are invisible to our data-free detection method, we use the expectation of virtual images $\{\hat{x}\}$ to be the replacement. The details of layer-wise deviations and scaling values are discussed in the following sections.

Model inversion is initially designed for artistic effects on clean images [15] and has been developed for synthesizing virtual training data in data-free knowledge distillation [8]. In this paper, we utilize this technique to excavate prior class-conditional statistics from the model’s BatchNorm layers.

Data Recovering. Given a random noise $\hat{x}$ as input and a label $c$ as target, recovering process aims at optimizing $\hat{x}$ into a pseudo image with discernible visual information by,

	$\displaystyle\min\limits_{\hat{x}}\underbrace{\mathcal{L}_{\text{CE}}(f_{\theta}(\hat{x},c))}_{\text{(i)}}$	$\displaystyle+\sum\nolimits_{l}\|\mu_{l}(\hat{x})-\mathbb{E}^{l}_{bn}(z)\|_{2}$		(5)
		$\displaystyle+\underbrace{\sum\nolimits_{l}\|\sigma^{2}_{l}(\hat{x})-\mathbb{\text{Var}}^{l}_{bn}(z)\|_{2}}_{\text{(ii)}}$

where (i) represents class prior loss (cross entropy loss) and (ii) represents feature statistic (mean and variance) divergences between $\hat{x}$ and BN running average at each $l$ layer.

Class-conditional Activation Mean. The activation units’ discrepancy is introduced to differentiate examples from ID and OOD. For each class $c$, we control the class prior loss and generate the one-class dataset $\mathcal{D}^{c}_{\text{syn}}$. Denotes the $l$-layer activation map for the input $x$ as $A^{l}(x)\in\mathbb{R}^{h\times d_{l}\times d_{l}}$, where $h$ is the channel size and $d_{l}$ is the dimension. Given input $x$ and its MSP label $c$, our layer-wise deviations $\Delta_{l}(\cdot)$ transforms the equation4 with the channel-wise linear weighted average C-Avg($\cdot$):

	$\displaystyle\Delta_{l}(x)$	$\displaystyle=\text{sup}_{\psi}\mid\psi(x)-\frac{1}{|\mathcal{D}^{c}_{\text{syn}}|}\sum\limits_{\hat{x}\in\mathcal{D}^{c}_{\text{syn}}}\psi(\hat{x})\mid$		(6)
		$\displaystyle=\mid\text{C-Avg}(A^{l}(x))-\underbrace{\frac{1}{|\mathcal{D}^{c}_{\text{syn}}|}\sum\limits_{\hat{x}\in\mathcal{D}^{c}_{\text{syn}}}\text{C-Avg}(A^{l}(\hat{x}))}_{\text{(i)}}\mid$
		$\displaystyle=\mid\sum\limits_{k=1}^{h}\beta^{c}_{l,k}\cdot\frac{1}{d_{l}^{2}}\sum\limits_{i=1}^{d_{l}}\sum\limits_{j=1}^{d_{l}}A^{l}(x)_{k,i,j}$
		$\displaystyle\quad\quad-\frac{1}{|\mathcal{D}^{c}_{\text{syn}}|}\sum\limits_{\hat{x}\in\mathcal{D}^{c}_{\text{syn}}}\sum\limits_{k=1}^{h}\beta^{c}_{l,k}\cdot\frac{1}{d_{l}^{2}}\sum\limits_{i=1}^{d_{l}}\sum\limits_{j=1}^{d_{l}}A^{l}(\hat{x})_{k,i,j}\mid$

where $\beta^{c}_{l,k}$ denotes the k-th channel weight for class $c$ and (i) represents the empirical $\overline{\text{C-Avg}}_{c}$ from $\mathcal{D}_{\text{syn}}^{c}$.

The scaling values $\alpha^{c}_{l},\beta^{c}_{l,k}$ greatly influence the performance of OOD detection. In our data-free method, the process of image evolution naturally provides an observation for the weights of neuron importance.

Channel-wise Gradient Average. Let $y^{c}$ be the model output score for class $c$. We introduce the gradient-based attribution method [16] to compute the gradient average $\overline{w}^{c}_{l,k}$ as the weight of $k$-th channel activation map at $l$-th layer.

$\displaystyle\overline{w}^{c}_{l,k}=\frac{1}{|\mathcal{D}^{c}_{\text{syn}}|}\sum\limits_{\hat{x}\in\mathcal{D}^{c}_{\text{syn}}}\frac{1}{d_{l}^{2}}\sum\limits_{i=1}^{d_{l}}\sum\limits_{j=1}^{d_{l}}\frac{\partial y^{c}}{\partial A^{l}(\hat{x})_{k,i,j}}$

(7)

Layer-wise Activation Mean Sensitivity. Denotes the image sequence during the optimizing procedure with $T$ iterations as $\{\hat{x}_{0},\hat{x}_{1},...\hat{x}_{T}\}$. Compared with $\hat{x}_{T}$ containing useful in-distribution features that induce high-confidence prediction [17] for target class $c$, random noise $\hat{x}_{0}$ is considered to be a baseline without any prior information. Based on the variations of their feature statistics, we define the $l$-layer activations mean sensitivity for $t$-iteration image $\hat{x}_{t}$ as,

	$\displaystyle\delta_{l}(\hat{x}_{t})$	$\displaystyle=\frac{1}{\Delta y_{t}^{c}}\cdot\frac{\partial y_{t}^{c}}{\partial\text{C-Avg}(A^{l}(\hat{x}_{t}))}$		(8)
		$\displaystyle=\frac{1}{y^{c}_{t}-y^{c}_{t-1}}\cdot\sum\limits_{k=1}^{h}\beta^{c}_{l,k}\frac{1}{d_{l}^{2}}\sum\limits_{i=1}^{d_{l}}\sum\limits_{j=1}^{d_{l}}\frac{\partial y^{c}_{t}}{\partial A^{l}(\hat{x}_{t})_{k,i,j}}$

This is, a larger variation of feature statistics during the evolution of $c$-class output score $y^{c}_{t}$ indicates that this layer is more sensitive to the label shifts on inputs. In other words, the layer’s magnitudes can better differentiate abnormal inputs. Therefore, we use the relative sensitivity $\delta_{l}(\hat{x}_{t})$ to measure the layer-contribution for OOD detection. The total $\delta_{l}(\hat{x})$ and empirical $\overline{\delta}^{c}_{l}$ for $\mathcal{D}^{c}_{\text{syn}}$ are computed by,

$\displaystyle\delta_{l}(\hat{x})=\frac{1}{T}\sum\limits_{t=1}^{T}\delta_{l}(\hat{x}_{t}),\overline{\delta}^{c}_{l}=\frac{1}{|\mathcal{D}^{c}_{syn}|}\sum\limits_{\hat{x}\in\mathcal{D}^{c}_{\text{syn}}}\delta_{l}(\hat{x})$

(9)

Then the $\beta^{c}_{l,k}$ and $\alpha_{l}^{c}$ are obtained by,

$\displaystyle\beta^{c}_{l,k}=\frac{\text{exp}(\overline{w}^{c}_{l,k})}{\sum_{i=1}^{h}\text{exp}(\overline{w}^{c}_{l,i})},\alpha^{c}_{l}=\frac{\text{exp}(\overline{\delta}^{c}_{l})}{\sum_{i=1}^{L}\text{exp}(\overline{\delta^{c}_{i}})}$

(10)

For the sake of computational efficiency, $\alpha^{c}_{l}$, $\beta^{c}_{l,k}$ and $\overline{\text{C-Avg}}_{c}$ are calculated in advance during the data recovering for each one-class dataset $\mathcal{D}^{c}_{\text{syn}}$.

We use CIFAR10 as the in-distribution (ID) dataset and Resnet34 as the base model. By default, the model is first trained on natural data and fixed during test time to detect anomalous inputs from a mixture of 10000 ID images (CIFAR10 dataset) and 10000 OOD images (other natural datasets). Especially for our data-free approach, we inverted 2500 virtual samples as the labeled data at each class.

We apply some post-hoc methods as baselines: Softmax Score [1], ODIN [3], and Energy Based Method (EBM) [4]. Besides, some approaches that only depend on training data (1-D [18], G-ODIN [19], Gram [10]) and data-available reference ma-dis. [2] are also considered. Following OOD literature [1, 2, 3, 4], we adopt the threshold-free metric [20] to evaluate the performance: the TNR at 95% TPR, AUROC, Detection accuracy, and AUPRin.

As shown in table 1, we evaluate the above metrics on 7 OOD datasets. Among these baselines, Softmax Score [1] and EBM [4] (without fine-tuning) are hyperparameter-free and the hyperparameters of ODIN [3] are from training data. For all OOD datasets, our C2IR outperforms other post-hoc methods without any support from natural data and, in particular, achieves the average improvement of +14.94% TNR when 95% CIFAR10 images are correctly classified. Moreover, our method significantly enhances the performance of the far-OOD dataset SVHN compared with near-OOD datasets (e.g., CIFAR100).

Figure 2a and 2b show the visualization of natural bird images in CIFAR10 and impressions from the pretrained network (ResNet34). The differences of intermediate activations means are shown in figure 2c. As in- and out-of-distribution statistics are distinguishable at each layer, the virtual ones are closer to the natural data compared with BN running average. It demonstrates that these virtual statistics can contribute more to detecting OOD examples.

As shown in table 2, we compare C2IR with several advanced in-distribution training methods (1-D [18], G-ODIN [19], GRAM [10]) with the only access to virtual data. Besides, data-available (both ID and OOD) method Ma-dis. [2] is also introduced as reference. The distribution shifts in the synthesized images cause poor performances in other baselines. Our method can utilize these virtual data to reach an ideal performance of 98.9% average AUROC, even comparable to the data-available methods.

To verify the effectiveness of our MGI method and the virtual activations mean, we conduct the ablation study as figure 2(d). MGI is compared with three baselines: 1) only use the penultimate layer (penu.) [6]; 2) average all layers’ statistics without weights (mean); 3) random weights. We additionally evaluate our metrics only with BN running averages for OOD detection. The results show that our virtual class-conditional activations mean outperforms BN statistics on all these strategies. Furthermore, compared with other baselines, activations mean equipped with MGI achieve the best performance.