AdCorDA: Classifier Refinement via Adversarial Correction and Domain Adaptation

We validated our approach through experiments on the CIFAR-10 and -100 datasets, each containing 50K images, which are randomly split into 45K training data and 5K validation data. Each dataset has a separate test set of 10K images. We first initialize ResNets [10] of different sizes (i.e., ResNet-18, ResNet-34, ResNet-50) and EfficientNetV2-M [27] with parameters pre-trained on the ImageNet dataset [7] from PyTorch [18] and then fine-tune [29] on the CIFAR training sets to obtain the corresponding baseline models. Input images are resized to $224\times 224$ and use the same data transform (i.e., apply normalization using the mean and standard deviation of ImageNet data) as the pre-trained models. During the fine-tuning, we use a stochastic gradient descent (SGD) optimizer [4] with a momentum of 0.9, a weight decay of 1e-4, a batch size of 128 for ResNets on and of 64 (due to limitations in computing resources) for EfficientNetV2-M, a fixed learning rate of 1e-4, and we train for a total of 100 epochs on both CIFAR datasets. We define the fine-tuned models with the best validation accuracy as our baseline models.

We use PyTorch and Nvidia V100L and P100L GPUs for all implementations. We split the training and validation datasets using three random seeds: 1, 2, and 5. We find that using random seeds 3 and 4 does not allow some attacks (e.g., BI and BIH, described in the next subsection) to successfully attack any incorrect images in the majority of instances. Therefore, we present the averaged results of the main experiments based on the three chosen random seeds. To determine the optimal hyper-parameters for our model, we perform a basic parameter grid search for the batch size, base learning rate, and weight decay of the SGD optimizer.

To apply adversarial attacks on misclassified images of train domains, we use a selection of methods, including three major types of gradient-based attacks: basic iterative method [14] and its variants, iterative least likely class [14], decoupled direction and norm [21], as well as a non-gradient-based salt and pepper noise attack, briefly described below.

• •

  Untargeted Basic Iterative (BI) [14]: This extends the “fast” method [9], which generates adversarial images through iterative processes using a small step size ($\alpha$) and clip pixel values of intermediate results at each step to ensure that they remain within an $\epsilon$-neighbourhood of the source image [14]:

  	$\displaystyle\boldsymbol{X}_{N+1}^{BI}$	$\displaystyle=Clip_{X,\epsilon}\{\boldsymbol{X}_{N}^{BI}+\alpha\text{sign}(\nabla_{X}J(\boldsymbol{X}_{N}^{BI},y_{true}))\},$		(1)
  	$\displaystyle\boldsymbol{X}_{0}^{BI}$	$\displaystyle=\boldsymbol{X},$		(2)

  where $\boldsymbol{X}$ represents an image, $y_{true}$ denotes the true class for the image $\boldsymbol{X}$, $J(\boldsymbol{X},y)$ is the cross-entropy cost function of the neural network, $Clip_{X,\epsilon}\{\boldsymbol{X^{\prime}}\}$ is the per-pixel clipping function applied to the image $\boldsymbol{X^{\prime}}$ to ensure it falls within an $L_{\infty}$ $\epsilon$-neighbourhood of the original image $\boldsymbol{X}$.

• •

  Basic Iterative method with Highest probability class (BIH): When attacking a correct image, BI uses the true class gradient, where the highest probability class aligns with the true class. However, when targeting an incorrect output, this changes – the highest probability class no longer represents the truth. Therefore, we adapt BI to use the gradient of the highest probability class to weaken the accuracy of the incorrect output, illustrated below:

  	$\displaystyle\boldsymbol{X}_{N+1}^{BIH}$	$\displaystyle=Clip_{X,\epsilon}\{\boldsymbol{X}_{N}^{BIH}+\alpha\text{sign}(\nabla_{X}J(\boldsymbol{X}_{N}^{BIH},y_{H}))\}$		(3)
  	$\displaystyle y_{H}$	$\displaystyle=\underset{y}{\mathrm{argmax}}\{p(y|\boldsymbol{X})\}.$		(4)

• •

  Targeted Variant of Basic Iterative (VBI): In addition to the standard untargeted BI method, we created a targeted variant called VBI. Unlike BI (Eq. (2)), which moves away from the true label, VBI (Eq. (5)) operates in the opposite direction, moving towards the true label by negating the sign of the gradient sign function.

  $$\boldsymbol{X}_{N+1}^{VBI}=Clip_{X,\epsilon}\{\boldsymbol{X}_{N}^{VBI}-\alpha\text{sign}(\nabla_{X}J(\boldsymbol{X}_{N}^{VBI},y_{true}))\}$$

  (5)

  $VBI_{iter1}$ is a fast version of VBI that just does one step towards the target.

• •

  Iterative Least-Likely class (LL) [14]: This method generates an attack targeting the least-likely class, as predicted by the trained model on the source image:

  	$\displaystyle\boldsymbol{X}_{N+1}^{LL}$	$\displaystyle=Clip_{X,\epsilon}\{\boldsymbol{X}_{N}^{LL}-\alpha\text{sign}(\nabla_{X}J(\boldsymbol{X}_{N}^{LL},y_{LL}))\}$		(6)
  	$\displaystyle y_{LL}$	$\displaystyle=\underset{y}{\mathrm{argmin}}\{p(y|\boldsymbol{X})\}$		(7)

  The LL method moves the input in the direction of the gradient toward the least probable class. While this may lower the probability of the true class, in some cases it will also lower the probability of the maximum probability (incorrect) class by a larger amount, potentially resulting in a correction of the output label.

• •

  Decoupled Direction and Norm (DDN) [21]: This attack is an iterative approach that refines the noise added to the input image in each iteration to make it adversarial. At iteration $i$, the adversarial input image, $x_{i}$, is generated as $x_{i}=x+\eta_{i}$, where $\eta_{i}$ is the noise with a norm of $\sigma_{i}$. If $x_{i}$ is adversarial, the norm of the next iteration noise is decreased ($\sigma_{i+1}=\sigma_{i}(1-\epsilon)$). Otherwise, the norm of the next noise is increased ($\sigma_{i+1}=\sigma_{i}(1+\epsilon)$). This process repeats until the minimum required perturbation is found [21]. The DDN method is a targeted attack that moves the network output towards the true label.

• •

  Salt and Pepper noise (SP): A non-gradient-based attack that repeatedly adds Salt & Pepper noise to the input to fool the model.

For the DDN and SP attacks, we use the default hyper-parameters provided by the Foolbox framework [19, 20]. Note that the input images are subject to the ImageNet transformation with a lower and upper bound of 0 and 1, respectively. The BI and LL attacks are applied according to the experimental setting outlined in [14]. In Tab. 1, we have applied VBI with one iteration (referred to as VBI${}_{\text{iter1}}$) and with five iterations (referred to as VBI). The maximum iteration limit for VBI is set to 5, which is sufficient for effectively correcting the vast majority of erroneous samples on CIFAR datasets.

We desire adversarial attack methods which are fast. The adversarial correction process may be quite computationally expensive if many iterations are needed per element of $T_{w}$. For example, a small network trained on CIFAR-100, which has 50,000 samples in the training set, might have a training accuracy of 95%. In this case, there would be 2,500 incorrect samples needing to be adversarially corrected. This could take a lot of time if the method takes many expensive iterations. Thus, we would prefer adversarial attack methods which are quick, using only a few iterations, possibly at the cost of not being able to correct all samples. Note that we do not necessarily require that the corrections be imperceptible, and so we can use relatively large perturbations of the input.

To investigate the effect of our proposed method on the adversarial robustness of the corrected models, we evaluated the models against AutoAttack [5] on CIFAR-10 and CIFAR-100 test sets. Composed of four different attacks from those used in our experiments for the correction, AutoAttack is a well-known, powerful, and diverse ensemble of parameter-free attacks. We applied the standard version of AutoAttack: $\text{APGD}_{\text{CE}}$, targeted $\text{APGD}_{\text{DLR}}$ [5], targeted FAB [6], and Square Attack [2] with $\ell_{\infty}$-norm. The attacks were applied sequentially. We set $\epsilon$ to 5e-4 for all of the AutoAttack experiments. Other AutoAttack parameters such as iterations and number of restarts are identical to the parameters used in the standard version. The batch size used for ResNet-18, ResNet-34, and EfficientNetV2-M experiments is 512, 512, and 100, respectively. We reported the average accuracy obtained across three different random seeds: 1, 2, and 5.

In the domain adaptation stage, we utilize Deep CORAL [26], which aligns the second-order covariance matrices between a source domain and a target domain through CORAL loss. This alignment helps to bridge the distribution gap between the domains and improve the model’s performance on the target domain. Aligning the implementation with the original paper, CORAL loss is only applied to the last classification layer in the neural networks. The total loss is the sum of the classification loss and the CORAL loss, defined as

where $\lambda$ is a weight between classification and CORAL loss. Its value is 1/750 for CIFAR-10 and 1/25 for CIFAR-100, intending to align the classification loss and the CORAL loss nearly the same at the end of the training process. The CORAL loss term is given by the following equation [26]:

where $C_{S}$ and $C_{T}$ are the covariance matrices of features induced by samples from the source domain and target domain, respectively, and the norm is the squared-matrix Frobenius norm. In our application, the source domain is the adversarially corrected training dataset ($T^{\prime}$) and the target domain is the original training dataset ($T$).

Our experimental setup closely adheres to the guidelines in [26]. However, we deviate by using batch sizes of 16 for ResNets on CIFAR-10 and CIFAR-100 and 16/32 for EfficientNet-M on CIFAR-10/100, differing from the original paper’s settings. Note that we shuffle the dataset $T^{\prime}$ before conducting domain adaptation training. This ensures a mixture of training samples, including those from the original dataset for which the trained network gets the correct answers and the successfully perturbed incorrect samples. We then proceed with 20 epochs on the CIFAR datasets. We also initialize the model with pre-trained weights from the baseline models rather than using the pre-trained model on ImageNet from PyTorch. These adjustments ensure a fair comparison with baseline models. When applying domain adaptation to quantized models, we facilitate the back-propagation process by approximating the gradients in these models. We achieve this approximation by utilizing the gradients derived from their corresponding full-precision models. This approach enables us to effectively conduct back-propagation on the quantized models. We define the best adapted model as the one that achieves the highest validation accuracy on the target domain, the original dataset $T$.

We also test the effectiveness of the AdCorDA method on network quantization, which reduces the precision of computations and weight storage by using lower bit-widths instead of floating-point precision. In our experiments, we choose post-training static quantization (PTSQ) [11], which is one of the most common and fastest quantization techniques in practice. This technique determines the scales and zero points prior to inference. Specifically, we quantize the full-precision 32-bit (FP32) weights (e.g., $w\in[\alpha,\beta]$) and activations of the trained baseline models to 8-bit integer (Int8) values (e.g., $w_{q}\in[\alpha_{q},\beta_{q}]$). The quantization process is defined as

where $s$ is the scale, and $d$ is the zero-point, defined as

To obtain quantized models, we compress the baseline models using post-training static quantization (PTSQ) [11]. We use the built-in quantization modules provided by PyTorch. These modules facilitate the fusion of different model components, calibration of the model using training data to determine suitable scale factors, and the actual quantization of weights and activations in the model. Note that we perform the adversarial correction on the training samples that the quantized network gets wrong, not the ones that the full precision network gets wrong.

The training, validation, and test accuracies of various networks obtained by applying AdCorDA for different attack methods on CIFAR-10 and CIFAR-100 are shown in Tab. 1. The “None” attack case corresponds to the situation where we do not apply any adversarial correction, effectively relying only on the curriculum modification of the training set. Our approach enhances the model performance by as much as 2.68% and 5.23% on CIFAR-10 and CIFAR-100, respectively, when utilizing ResNets of various sizes. As for the effect of our method when applied to EfficientNet, we observe an improvement of about 0.7-1.1% on CIFAR datasets. More specifically, the ResNet-34 baseline model, operating at full precision, achieved a test accuracy of 78.41% on CIFAR-100. Our adversarial correction method, using a DDN adversarial attack, improves the test accuracy to 83.64%, representing a notable increase of 5.23%. Upon incorporating adversarial correction using the LL adversarial attack on the training set, we observed a decrease in the initial training loss from 0.254 (on the original training set $T$) to 0.173 (on the corrected training set $T^{\prime}$) on CIFAR-100. This shows that the adversarial correction does indeed reduce the training loss. In Fig. 2(b) and Fig. 2(d) we can see that both targeted (VBI) and untargeted (LL) adversarial attacks can successfully reduce the logit level of the initially maximum probability incorrect label as compared with the logit level of the true label, resulting in correction.

Table 2 shows that our method also improves the baseline performance of quantized networks. For example, the full precision baseline ResNet-34 achieves a test accuracy of 78.41% on CIFAR-100. The Int8 quantized baseline ResNet34 has a test accuracy of 77.13% on CIFAR-100. When applying our method using the BIH adversarial attack on the full precision baseline, we achieve a test accuracy of 82.99%, representing an improvement of +4.58%. After Int8 PTSQ quantization, the modified FP network achieves a test accuracy of 82.18% - an improvement of +5.05% over the original quantized network (and an improvement of +3.77% over the original full precision network!).

It is also worth noting in Tab. 2, that the decrease in accuracy on ResNet-34 after quantization using our method is only 0.81% (i.e., from 82.99% to 82.18%), whereas the drop in performance in the original network after quantization is 1.28% (i.e., from 78.41% to 77.13%). This comparison shows that our adversarial correction method makes the full-precision network less affected by quantization.

The quantized ResNet-34 network after using our adversarial correction technique achieves a higher accuracy (82.18%) than even that of a normally trained full-precision ResNet-152 baseline model (81.52%), while significantly reducing the model size (20.76MB vs 223.49MB).

In the Feng and Tu theory, all that is needed in the first step of the IST is to perturb the input so as to reduce the loss. It is not necessary to actually change the input so as to have the network give the correct answer; all that is required is that the loss be reduced.

In the experiments shown in Tab. 1, we defined $T_{a}$ as the set of successfully corrected samples in step 3 of our adversarial correction approach. If we now consider $T_{a}$ to include all perturbed samples, whether the outputs are corrected or not, $T^{\prime}$ will have the same size as the original training dataset. We refer to the network adapted using this variation as BL-IST-A. In our original approach, the accuracy of the original network on $T^{\prime}$ reaches 100% when we consider only the successfully perturbed samples and the original correctly detected samples. Inspired by [25], we can think of $T^{\prime}$ as an easy dataset, given its 100% accuracy, while considering $T$ as a hard dataset. In Table 3 we observe a drop in performance improvement in BL-IST-A as compared to our first approach. This could be attributed to the adversarial perturbations increasing the loss rather than decreasing it, as compared with the baseline, for the uncorrected inputs. We conclude that we should only retain the corrected input samples.

To help visualize the impact of the adversarial correction technique on misclassified images, we employ Gradient-weighted Class Activation Mapping (Grad-CAM) [23] to provide visual explanations. Grad-CAM utilizes gradient-based localization to identify important regions in an image that contribute to the model’s concept prediction. In our study, Fig. 3(a) is an example initially misclassified as an ‘automobile’ by ResNet-34. However, applying the DDN attack, the image can be correctly identified as a ‘horse’. To better understand the differences between the Grad-CAM of the original (Fig. 3(c)) and its corrected image (Fig. 3(d)), we present a visualization in Fig. 3(b). This visualization clearly illustrates that the incorrect detection was primarily influenced by the surrounding contextual information rather than the object itself. This demonstrates that by modifying the surrounding contextual information of the image using the adversarial attack, correct classification becomes possible.

Our adversarial correction technique has many similarities to adversarial training methods for enhancing robustness to adversarial attacks. Such methods generate adversarial examples, for which networks give the wrong answer, and add these as augmentations of the original dataset. Fine-tuning on the augmented dataset leads to enhanced robustness against adversarial attacks [16]. Our approach is similar in that we create new images resulting from adversarial attacks, and use these in concert with images from the original dataset in further training. There are significant differences, however, between our method and standard adversarial training. First, we do not augment the original dataset, but instead replace some of the samples in the original dataset with the adversarial examples. Second, the adversarial attacks are only applied to samples that the network gets wrong, rather than samples that the network gets right, and we only keep the adversarial examples which are corrective - that the network now gets right. Finally, rather than doing fine-tuning using standard training on the augmented training set, we do domain adaptation from the adversarially corrected training set to the original training set. We tested the robustness of ResNets to the AutoAttack suite of attacks [5]. As seen in Tab. 4, our method provides significant robustness to adversarial attacks. For CIFAR-10 with ResNet-18 we see an improvement from 15.92% on the baseline model to 50.97% on the adversarially corrected model with the SP correction method. On CIFAR-100 with ResNet-18 we see an improvement from 7.56% to 21.8%. Note that using only curriculum domain adaptation (the “None” case) also gives significant robustness. While current state-of-the-art robust network techniques get higher accuracies under attack than ours (e.g., 27.67% on CIFAR-100 by [1] and 55.54% on CIFAR-10 by [22], both with ResNet-18), our focus is on attaining higher clean (before attacks) accuracies, and the enhanced robustness is a welcome byproduct. Jointly optimizing both clean accuracy and adversarial robustness is an interesting avenue for future work.