CAT: Collaborative Adversarial Training

Since Szegedy [34] discovers that DNNs are vulnerable to adversarial examples, a large number of works have been proposed to craft adversarial examples. Based on the accessibility to the knowledge of the target model, it can be divided into white-box attacks and black-box attacks. White-box attacks craft adversarial examples based on the knowledge of the target model, while black-box attacks are agnostic to the knowledge of the target model.

White-box Attack: Goodfellow [9] proposes FGSM to efficiently craft adversarial examples, which can be generated in just one step. Madry proposes PGD to generate adversarial examples, which is the most efficient way of using the first-order information of the network. MI-FGSM [7] combines momentum into the iterative process to help the model escape from local optimal points. And the adversarial examples generated by this method are also more transferable. Boundary-based attacks such as deepfool [22] and CW [2] also make the model more challenging. Recently, the ensemble approach of diverse attack methods (Auto-Attack), consisting of APGD-CE [5], APGD-DLR [5], FAB [4] and Square Attack [1], become a benchmark for testing model robustness.

Black-box Attack: Block-box attacks can be categorized into transfer-based and query-based attacks. Transfer-based methods attack the target model by using the transferability of adversarial examples, i.e., the adversarial examples generated on the surrogate model can be transferred to fool the target model. There are many ways to explore the transferability of adversarial examples for black-box attacks. Dong [7] combines momentum with an iterative approach to obtain better transferability. Scale-invariance [19] boosts the transferability of adversarial examples by transforming the inputs on multiple scales. Square Attack [1] approximates model’s decision boundary based on a randomized search scheme to be the most efficient query-based attack method.

Adversarial attacks present a significant threat to DNNs. For this reason, many methods have been proposed to defend against adversarial examples, including denoising [35], adversarial training [21], data aumentation [29], and input purification [23]. ANP [20] finds the vulnerability of latent features and uses pruning to improve robustness. Madry uses PGD to generate adversarial examples for adversarial training, which is also the most effective way to defend against adversarial examples. A large body of work uses new regularization or objective functions to improve the effectiveness of standard adversarial training. Adversarial logit pairing [15] improves robustness by encouraging the logits of normal and adversarial examples to be closer together. TRADES [37] uses KL divergence to regularize the output of adversarial and pure examples.

Knowledge distillation (KD) is commonly used for model compression and was first used by Hinton [11] to distill knowledge from a well-trained teacher network to a student network. KD can significantly improve the accuracy of student models. There have been many later works to improve the effectiveness of KD [32]. In recent years, KD has been extended to other areas. Goldblum [8] analyzes the application of knowledge distillation to adversarial robustness and proposes ARD to transfer knowledge from a large teacher model with better robustness to a small student model. ARD can produce a student network with better robustness than training from scratch. In this paper, we propose a more effective collaborative training framework to improve the robustness of the network.

We investigated the characteristics of the robust models obtained by different adversarial training methods on sample instances. We found that different models perform differently on sample instances: for some samples, the model trained by correctly classifies AT [21], while the model trained by TRADES [37] misclassifies. Confusion exists in different methods. A straightforward conclusion can be drawn that the networks trained by different methods master different knowledge, although their accuracy values are about the same. So can we use the knowledge learned from these two networks to improve the robustness of neural networks? A simple idea is to let two networks that master different knowledge learn collaboratively. For this purpose, we propose collaborative adversarial training. CAT improves the robustness of neural networks by making the knowledge of both networks interact during the training procedure. And the framework is illustrated in Fig. 2.

We take the methods of AT and TRADES as an example to introduce collaborative adversarial training. We first briefly introduce the training objective functions of AT and TRADES and then introduce CAT in detail.

Adversarial training is defined as a min-maximization problem. PGD is used to generate adversarial examples for the internal maximization process, while external minimization uses the internal PGD-generated adversarial examples and the ground-truth label $y$ to optimize the model parameters. AT is formulated as:

$$\underset{\theta}{\min}\ \mathbb{E}_{(x,y)\in D_{data}}(\underset{\delta}{\arg\max}\ L(f^{AT}_{\theta}(x^{adv}_{AT}),y)),$$

(1)

$$x^{adv}_{AT}=x+\delta.$$

(2)

where $D_{data}$ is the training data distribution, $x$ and $y$ are training data and corresponding label samples from $D_{data}$. $f_{\theta}$ is a neural network parameterized by $\theta$. $L$ is the standard cross-entropy loss used in image classification tasks. $\delta$ is the adversarial perturbations generated by PGD. Following previous study, $\delta$ is bounded by $l_{\infty}$.

Neural Networks trained by AT can obtain a certain level of robustness, with compromises on the accuracy of natural samples. For this purpose, TRADES uses a new training objective function for adversarial training. Formulated as:

$$\begin{split}\underset{\theta^{\prime}}{\min}\ \mathbb{E}_{(x,y)\in D_{data}}L(g^{TRADES}_{\theta^{\prime}}(x),y)\quad\quad\\ +\lambda D_{KL}(g^{TRADES}_{\theta^{\prime}}(x),g^{TRADES}_{\theta^{\prime}}(x^{adv}_{TRADES})),\end{split}$$

(3)

where $x^{adv}$ is the adversarial data corresponding to natural data $x$ and y is the true label. $L$ is the cross-entropy loss in classification task. $D_{KL}$ is the KL divergence to push natural logits and adversarial logits together. The losses are balanced by a trade-off parameter $\lambda$.

CAT expects to improve robustness by letting neural networks trained by different methods interact with knowledge information, i.e., collaborative adversarial learning. As illustrated in Fig. 2. we use the logit of a peer network to guide the learning of this network. Specifically, we input the adversarial data crafted by the network trained by AT into the network trained by TRADES to get the corresponding logit. Then use the logit obtained by the network trained by TRADES to guide the training of the network trained by AT. The formulation goes to:

$$L_{1}=D_{KL}(f^{AT}(x^{adv}_{AT}),\hat{g}^{TRADES}(x^{adv}_{AT})),$$

(4)

$f^{AT}$ is the network trained with AT and $g^{TRADES}$ is the network trained with TRADES. $\hat{g}^{TRADES}(x^{adv}_{AT})$ represents that we take the logit obtained by the network trained by TRADES as constant. $x^{adv}_{AT}$ is the adversarial data generated by $f^{AT}$ with PGD.

Similarly, to make the two networks learn collaboratively. We need to feed the adversarial samples generated by the TRADES network to the AT network to obtain the corresponding logit. And then the logit obtained by the peer network is used to guide the training of the network trained by TRADES. The loss is formulated as:

$$L_{2}=D_{KL}(g^{TRADES}(x^{adv}_{TRADES}),\hat{f}^{AT}(x^{adv}_{TRADES})).$$

(5)

$x^{adv}_{TRADES}$ is the adversarial example crafted by the network trained by TRADES using KL divergence function. $\hat{f}^{AT}(x^{adv}_{TRADES})$ represents that we take the logit obtained by the network trained by AT as constant.

Empirically, models trained purely on collaborative loss will collapse. An intuitive explanation is that the purpose of collaborative loss is to interact knowledge, but there is no knowledge when the model are lack supervision. It is not enough to let the two networks learn from each other in this way. Real class labels are needed to guide them. For this purpose, we induce supervision by combining the respective training objective functions of the two networks and the collaborative learning objective function to guide the learning of the networks together. Therefore, the training objective function for collaborative adversarial training based on AT and TRADES is:

$$L_{total}=\alpha L_{TRADES}+(1-\alpha)L_{2}+\alpha L_{AT}+(1-\alpha)L_{1},$$

(6)

where $\alpha$ is the trade-off parameter to balance the guidance of the peer network knowledge and the original objective function. $L_{TRADES}$ is the training objective of TRADES defined in Eq. 3. And $L_{AT}$ is the training objective of AT defined in Eq. 1. The first two items in Eq. 6 are used to train model $g$ and the last two items are used to train model $f$, due to we take the peer logit as constant.

The decision boundaries learned by different adversarial training methods are different. Under the guidance of peer network knowledge, i.e., Eq. 4 and Eq. 5, the two networks trained by different methods continuously optimize the classification decision boundaries in the process of collaborative learning. Finally, both networks learn better decision boundaries than learning alone to obtain better adversarial robustness.

Our collaborative adversarial learning is a generalized adversarial training method that can be used with any two adversarial methods. Generally, CAT can use any number of different adversarial training methods for collaborative learning. Results of CAT with three adversarial training methods are delayed to Sec. 5.3.

Difference with ensemble methods: The main difference between collaborative adversarial learning and ensemble methods is that collaborative learning involves multiple models that learn from each other, while ensemble methods involve multiple models that are combined to produce a single output. During testing, each model trained by collaborative learning predicts singly, since they have interacted with their knowledge during the training time. Refer to Appendix B for details.

We use WideResNet-34-10 [36] networks for collaborative adversarial training to compare with previous sota methods. Tab. 4 shows the accuracy of the different methods for natural examples and the robustness against Auto-Attack. From the table, we can conclude that the robustness of both networks trained with CAT outperforms the previous methods, demonstrating the state-of-the-art performance obtained by our CAT. AWP further boosts the robustness of CAT with 2.41% improvement.

In general, the robustness of large models is higher than that of small models under the same training settings. For example, WideResNet-34-10 [36] trained by TRADES can achieve 53.08% robustness against AA, while the accuracy of ResNet-18 is only 49.21%. Researchers use knowledge distillation to distill the robustness of large models to small models and obtained good results. We call these methods KD-AT. Considering that CAT also involves the collaborative training of two models, we compare CAT with the KD-AT method. To give a fair comparison, we use two different-size networks for CAT training, the same as the teacher and student network used in KD-AT. Note that, unlike the KD method where the teacher is trained in advance, our CAT is trained with both the large model and the small model simultaneously, so there is no concept of teacher and student. In another word, we extend previous offline distillation (2 stages) to an online way (1 stage) and achieve better performance with lower computation resources. Illustration comparison is shown in Appendix B.

Tab. 5 shows the results of Knowledge Distillation-AT and CAT, where ARD [8], IAD [39], and RSLAD [40] are trained by KD-AT using TRADES trained WideResNet-34-10 network as teacher. CAT is collaboratively trained using two networks of different sizes. It can be seen that our method obtains high adversarial robustness and also obtains high clean accuracy. More importantly, The robustness of CAT is higher than RSLAD equipped with AWP.

Overfitting in adversarial training is first proposed by [31], which shows the test robustness decreases after peak robustness. And overfitting is one of the most concerning problems in adversarial training. Here, we investigated the overfitting problem in CAT with VGG-16. Results are illustrated in Fig. 4. Our CAT can alleviate the overfitting problem that widely occurs in previous adversarial methods. Moreover, the performance for CAT has not saturated, and high performance is expected with longer epoch training.

We analyze the correlation between the discrepancy of different adversarial training methods and their adversarial robustness after CAT. First, we compute the prediction intersection between different methods, formulated as:

$$intersection=\frac{1}{N}\sum_{x_{i}\in D}\mathbb{I}(f^{AT}(x_{i}),g^{TRADES}(x_{i})),$$

(7)

where D is the datasets, and $\mathbb{I}$ is indicator function, which is 1 when $f^{AT}(x_{i})=g^{TRADES}(x_{i})$ and 0 otherwise. Prediction discrepancy equals 1 minus intersection. The larger this value is, the greater the discrepancy. Then, we report the adversarial robustness of CAT trained in different settings. Results are reported in Tab. 6. A conclusion can be drawn that the greater the discrepancy between different methods is, the higher the adversarial robustness after CAT.

Our CAT is a generalized method, which can use any number of different adversarial training methods for collaborative learning. We conduct an experiment on CAT by collaborating three adversarial training methods. The results are reported in Tab. A4. The robustness improvement is more significant than CAT trained with two adversarial-trained methods, which shows the generalizability of our CAT. Collaborating three methods can bring 0.7% improvement against Auto-Attack.