Boosting Transferability of Targeted Adversarial Examples via Hierarchical Generative Networks

We use $\bm{x}_{s}$ to denote an input image belonging to an unlabeled training set $\mathcal{X}_{s}\subset\mathbb{R}^{d}$, and use $c\in\mathcal{C}$ to denote a specific target class. Let $\mathcal{F}_{\phi}:\mathcal{X}_{s}\rightarrow\mathbb{R}^{K}$ denote a classification network that outputs a class probability vector with $K$ classes. To craft a targeted adversarial example $\bm{x}_{s}^{*}$ from a real example $\bm{x}_{s}$, the targeted attack aims to fool the classifier $\mathcal{F}_{\phi}$ by outputting a specific label $c$ as $\operatorname*{arg\,max}_{i\in\mathcal{C}}{\mathcal{F}_{\phi}(\bm{x}_{s}^{*})}_{i}=c$, meanwhile the $\ell_{\infty}$ norm of the adversarial perturbation is required to be no more than a threshold $\epsilon$ as $\|\bm{x}_{s}^{*}-\bm{x}_{s}\|_{\infty}\leq\epsilon$.

Although some generative methods [39, 36] can learn targeted adversarial perturbation, they do not take into account the effectiveness of multi-target generation, thus leading to inconvenience. To make the generative model learn how to specify multiple targets, we propose a conditional generative network $\mathcal{G}_{\theta}$ that effectively crafts multi-target adversarial perturbations by modeling class-conditional distribution. Different from previous single-target methods [36, 39], the target label $c$ is regarded as a discrete variable rather than a constant. As illustrated in Fig. 1(b), our model contains a conditional generator $\mathcal{G}_{\theta}$ and a classification network $\mathcal{F}_{\phi}$ parameterized by $\theta$ and $\phi$, respectively. The conditional generative model $\mathcal{G}_{\theta}:(\mathcal{X}_{s},\mathcal{C})\rightarrow\mathcal{P}$ learns a perturbation $\bm{\delta}=\mathcal{G}_{\theta}(\bm{x}_{s},c)\in\mathcal{P}\subset\mathbb{R}^{d}$ on the training data. The output $\bm{\delta}$ of $\mathcal{G}_{\theta}$ is projected within the fixed $\ell_{\infty}$ norm, thus generating the perturbed image $\bm{x}_{s}^{*}=\bm{x}_{s}+\bm{\delta}$.

Given a pretrained network $\mathcal{F}_{\phi}$ parameterized by $\phi$, we propose to generate the targeted adversarial perturbations by solving

where $\mathbb{CE}$ is the cross-entropy loss. By solving problem (1), we can obtain a targeted conditional generator by minimizing the loss of specific target class in the unlabeled training dataset. Note that we only optimize the parameter $\theta$ of the generator $\mathcal{G}_{\theta}$ using the training data $\mathcal{X}_{s}$, then the targeted adversarial example $\bm{x}_{t}^{*}$ can be crafted by $\bm{x}_{t}^{*}=\bm{x}_{t}+\mathcal{G}_{\theta}(\bm{x}_{t},c)$ for any given image $\bm{x}_{t}$ in the test data $\mathcal{X}_{t}$, which only requires an inference for this targeted image $\bm{x}_{t}$.

We experimentally find that the objective (1) can enforce the transferability for the generated perturbation $\bm{\delta}$. A reasonable explanation is that $\bm{\delta}$ can arise as a result of strong and well-generalizing semantic pattern inherent to the target class, which is robust to the influence of any training data. In Sec. 4.5, we illustrate and corroborate our claim by directly feeding scaled adversarial perturbations¹¹1The perturbation is linearly scaled from [-$\epsilon$, $\epsilon$] to [0, 255]. from different methods into the classifier. Indeed, we find that our semantic pattern can be classified as the target class with a high degree of confidence while the perturbation from MIM [9] performs like the noise, meanwhile the scaled semantic pattern performs well transferability in different black-box models.

We now present the details of the conditional generative model for targeted attack, as illustrated in Fig. 1(b). Specifically, we design a mapping network to generate a target-specific vector in the implicit space of each target and train conditional generator $\mathcal{G}_{\theta}$ to reflect this vector by constantly misleading the classifier $\mathcal{F}_{\phi}$.

Mapping network. Given the one-hot class encoding $\mathbbm{1}_{c}\in\mathbb{R}^{K}$ from target class $c$, the mapping network aims to generate the targeted latent vector $\bm{w}=\mathcal{W}(\mathbbm{1}_{c})$, where $\bm{w}\in\mathbb{R}^{M}$ and $\mathcal{W}(\cdot)$ consists of a multi-layer perceptron (MLP) and a normalization layer, which can construct diverse targeted vectors $\bm{w}$ for a given target class $c$. Thus $\mathcal{W}$ is capable of learning effective targeted latent vectors by randomly sampling different classes $c\in\mathcal{C}$ in training phase.

Generator. Given an input image $\bm{x}_{s}$, the encoder first calculates the feature map $\bm{F}\in\mathbb{R}^{N\times H\times W}$, where $N$, $H$ and $W$ refer to the number of channels, height and width of the feature map, respectively. The target latent vector $\bm{w}$, derived from the mapping network $\mathcal{W}$ by introducing a specific target class $c$, is expanded along height and width directions to obtain the label feature map $\bm{w}_{s}\in\mathbb{R}^{M\times H\times W}$. Then the above two feature maps are concatenated along the channels to obtain $\bm{F}^{\prime}\in\mathbb{R}^{(N+M)\times H\times W}$. The obtained mixed feature map is then fed to the subsequent network. Therefore, our generator $\mathcal{G}_{\theta}$ translates an input image $\bm{x}_{s}$ and latent target vector $\bm{w}$ into an output image $\mathcal{G}_{\theta}(\bm{x}_{s},\bm{w})$, which enables $\mathcal{G}_{\theta}$ to synthesize adversarial images of a series of targets. For the output of feature map $\bm{f}\in\mathbb{R}^{d}$ in the decoder, we adopt a smooth projection $P(\cdot)$ to perform a change of variables over $\bm{f}$ rather than directly minimizing its $\ell_{2}$ norm as [15] or clipping values outside the fixed norm [36], which can be denoted as

where $\epsilon$ is the strength of perturbation. Since $-1\leq\mathrm{tanh}(\bm{f})\leq 1$, $\bm{\delta}$ can automatically satisfy the $\ell_{\infty}$-ball bound with perturbation budget $\epsilon$. This transformation can be regarded as a better smoothing of gradient than directly clipping values outside the fixed norm, which is also instrumental for $\mathcal{G}_{\theta}$ to probe and learn the targeted semantic knowledge from $\mathcal{F}_{\phi}$.

Training objectives. The training objectives seek to minimize the classification error on the perturbed image of the generator as

which adopts an end-to-end training paradigm with the goal of generating adversarial images to mislead the classifier the target label, and $\mathbb{CE}$ is the cross entropy loss. Previous studies attempt different classification losses in their works [59, 36], and we found that cross-entropy loss works well in our settings. The detailed optimization procedure is summarized in Algorithm 1.

While handling plenty of classes, the effectiveness of a conditional generative model will decrease as illustrated in Fig. 4, because the representative capacity is limited with a single generator. Therefore, we propose to divide all classes into a feasible number of subsets to train models when the class number $K$ is large, e.g., 1,000 classes in ImageNet, with the aim of seeking the effectiveness of targeted black-box attack. To obtain a good partition, we introduce a representative target class space, which is nearly equivalent to the original class space $\mathcal{C}$. Specifically, we utilize the weights $\phi_{cls}\in\mathbb{R}^{D\times K}$ in the classifier layer for the classification network $\mathcal{F}_{\phi}$. Therefore, $\phi_{cls}$ can be regarded as the alternative class space since the weight vector $\bm{d}_{c}\in\mathbb{R}^{D}$ from $\phi_{cls}$ can represent a class center of the feature embeddings of input images with same class $c$.

Note that once those subsets with closer metric distance (e.g., larger cosine similarity) in the target class space $\phi_{cls}$ are regarded as conditional inputs of generative network, they obtain worse loss convergence and transferability than diverse them due to mutual influence among these input conditions, as illustrated in Fig. 5. Thus we focus on selecting target classes that do not tend to overlap or be close to each other as accessible subsets. To capture more diverse examples in a given sampling space, we adopt K-determinantal point processes (DPP) [25, 24] to achieve a hierarchical partition, which can take advantage of the diversity property among subsets by assigning subset probabilities proportional to determinants of a kernel matrix.

First, we compute the RBF kernel matrix $L$ of $\phi_{cls}$ and eigendecomposition of $L$, and a random subset $V$ of the eigenvectors is chosen by regarding the eigenvalues as sampling probability. Second, we select a new class $c_{i}$ to add to the set and update $V$ in a manner that de-emphaseizes items similar to the one selected. Each successive point is selected and $V$ is updated by Gram-Schmidt orthogonalization, and the distribution shifts to avoid points near those already chosen. By performing the above procedure, we can obtain a subset with $k$ size. Thus while handling the conditional classes with K, we can hierarchically adopt this algorithm to get the final $K/k$ subsets, which are regarded as conditional variables of generative models to craft adversarial examples. The details are presented in Appendix A.

Datasets. We consider the following datasets for training, including a widely used object detection dataset MS-COCO [31] and ImageNet training set [6]. We focus on standard and comprehensive testing settings, thus the inference is performed on ImageNet validation set (50k samples), a subset (5k) of ImageNet proposed by [29] and ImageNet-NeurIPS (1k) proposed by [38].

Networks. We consider some naturally trained networks, i.e., Inception-v3 (Inv3) [47], Inception-v4 (Inv4) [45], Resnet-v2-152 (R152) [16] and Inception-Resnet-v2 (IR-v2) [45], which are widely used for evaluating transferability. Besides, we supplement DenseNet-201 (DN) [18], GoogleNet (GN) [46] and VGG-16 (VGG) [43] to fully evaluate the transferability. Some adversarially trained networks [49] are also selected to evaluate the performance, i.e., ens3-adv-Inception-v3 ($\textrm{Inv3}_{\textrm{ens3}}$), ens4-adv-Inception-v3 ($\textrm{Inv3}_{\textrm{ens4}}$) and ens-adv-Inception-ResNet-v2 ($\textrm{IR-v2}_{\textrm{ens}}$).

Implementation details. As for instance-specific attacks, we compare our method with several attacks, including MIM [9], DIM [52], TI [10], SI [30] and the state-of-the-art targeted attack named Logit [60]. All instance-specific attacks adopt optimal hyperparameters provided in their original work. Specifically, the attack iterations $M$ of MIM, DIM and Logit are set as $10,20,300$, respectively. And $\|W\|_{1}$ = 5 is used for TI [10] as suggested by [12, 60]. We choose the same ResNet autoencoder architecture in [23, 36] as the basic generator networks, which consists of downsampling, residual and upsampling layers. We initialize the learning rate as 2e-5 and set the mini-batch size as 32. Smoothing mechanism is proposed to improve the transferability against adversarially trained models [10]. Instead of adopting smoothing for generated perturbation while the training is completed as CD-AP [36], we introduce adaptive Gaussian smoothing kernel to compute $\bm{\delta}$ from Eq. (2) in the training phase, named adaptive Gaussian smoothing, with the focus of making generated results obtain adaptive ability. More implementation details and discussion with other networks (e.g., BigGAN [3]) are illustrated in Appendix B.

We consider 8 different target classes from [59] to form the multi-target black-box attack testing protocol with 8k times in 1k ImageNet NeurIPS set.

Efficiency of multi-target black-box attack. Among comparable methods, instance-specific methods, i.e., MIM, DIM, and Logit, require iterative mechanism with $M$ steps by computing gradients to obtain adversarial examples. Given the cost $t_{C}^{FP}$ and $t_{C}^{BP}$ of forward and backward passing the classifier, computing cost $T^{IS}$ of single data can be defined as $T^{IS}=t_{C}^{FP}*M+t_{C}^{BP}*M$ in Table 1. Instance-agnostic methods only require the inference cost from the trained generator as $T^{IA}=t_{G}^{FP}$, thus possessing the priority for those attack scenarios within limited time. However, instance-agnostic methods require to train 8 models to obtain all predictions from 8 different classes. Due to time-consuming training and more storage, we only reproduce an excellent generative method CD-AP [36] as a baseline, which already fully demonstrate the superior performance than other generative methods such as GAP [39] in their work. As a comparison, our conditional generative method only trains one model to inference the results and outperforms other methods w.r.t efficiency.

Effectiveness of multi-target black-box attack. Table 1 shows the transferability comparison of different methods on both naturally and adversarially trained models. The success rate of instance-specific attacks are very unsatisfactory, possibly explained by the data-point overfitting that makes it hard to transfer another model. The instance-agnostic attack CD-AP obtains acceptable performance, yet inferior to proposed method w.r.t black-box transferability. The primary reason for such a trend lies in some distinctions as 1) direct clip projection in CD-AP and our smooth projection in Eq. (2) and 2) their Gaussian Smoothing and our adaptive Gaussian Smoothing, as described in Sec. 4.1 and Appendix B. Fig. 2 empirically shows the comparison results of single-target black-box attacks based on the CD-AP framework. Thus proposed conditional generative method can be a reliable baseline w.r.t targeted black-box attacks, regarding both effectiveness and efficiency.

Results of single-target black-box attack. Recent related works, e.g., UAE [59] and TTP [37] report excellent single-target black-box performances based on universal perturbations or functions. We obtain single-target degraded version of our model by specifying an input target label during the training process. The performance of black-box targeted attack between different methods is presented in Table 2. Besides, we also make some analyses about TTP and present compared results in Appendix C. Furthermore, some other instance-agnostic adversarial methods, e.g., UAP [34], GAP [39] and RHP [29], have tendency towards the untargeted black-box problem. Despite this, we also follow the corresponding untargeted setting and compare these methods in Appendix C. Our method is steadily improved under black-box targeted and untargeted black-box manner.

To illustrate the effectiveness of our proposed attack methods in practical 1,000 classification, we here follow the official setting from NeurIPS 2017 adversarial competition [27] for testing targeted black-box transferability. Considering limited resource, previous instance-agnostic attacks are not required as comparable methods due to training 1,000 models, thus we focus on various excellent instance-specific attacks for comparison, including official top attack methods in NeurIPS 2017 adversarial competition. Compared with other instance-agnostic attacks, our hierarchical partition mechanism can make conditional generative networks be capable of specifying any target class via a feasible number of models for the scalability. Specifically, we consider 20 models, with each specifying 50 diverse classes from k-DPP hierarchical partition in this setting, to implement targeted attack by only once inference for each target image. As shown in Table 3, our method can obviously outperform all other methods. In addition, these trained generative models can directly be applied to craft adversarial examples, which is more convenient and efficient when required to handle large-scale (e.g., millions of images) datasets than instance-specific attacks.

Adversarial perturbations added to original face images have ability to evade being recognized or impersonate another individual [42, 57]. In this section, we consider the transferability of impersonation attack to further illustrate the generalization of our method, which is also corresponding to targeted attack in the image classification task.

Dataset and models. We conduct the experiments on Labeled Faces in the Wild (LFW) [19] and introduce two test protocols. For Protocol I defined as single-target impersonation attack, we choose 1 target identity and 1k source face images belonging with different identities from LFW as the attackers, thus forming 1k pairs. For Protocol II named multi-target impersonation attack, 5 target identities and 1k source face images are selected to form 1k attack pairs, meaning that we need to implement 5k attacks. We involve some excellent face recognition models for conducting black-box testing, including Sphereface [32], CosFace [50], FaceNet [41] and MobileFace [4]. These models lie in different model architectures and training objectives. In all experiments, we only use one model ArcFace [7] as substitute model to craft adversarial samples, and test attack performance against other unknown models.

Evaluation metrics. We first compute the optimal threshold of every face recognition models from LFW dataset by following standard protocols. If the similarity of a pair of images exceeds the threshold, we regard them as same identity, otherwise different identities.

Black-box attack results. We adjust the optimization object function to adapt face recognition for chosen attack methods (detailed in Appendix C), and report the success rate of black-box impersonation attacks in Table 4, which illustrates that our method can achieve nearly two times of the success rates than DIM in Protocol I and Protocol II. The results indicate that our method is superior to other methods not only in image classification.

We conduct an extensive study to investigate two key points about target classes.

Different numbers of target classes. We conduct effectiveness for different numbers of target classes in Fig. 4. It can be seen that the results perform well within a feasible number of targets, whereas to a certain extent effectiveness tend to decay. Therefore, the effectiveness of conditional generative networks is influenced by the number of conditional classes, due to the representative capacity of single generator. We aim to divide all classes into a feasible number of set while handling plenty of classes.

Comparison of different multi-target conditions. We select closer conditional classes with larger cosine similarity in the target class space $\phi_{cls}$ and diverse conditional classes from k-DPP method. In Fig. 5, closer conditional classes have worse loss convergence and transferability than diverse them due to mutual influence among conditions.

Targeted adversarial samples from proposed generative method can produce semantic pattern inherent to the target class in Fig. 3. Why does generative semantic pattern work?

First, generative methods can produce strong targeted semantic pattern that is robust to the influence of data, which is obtained by minimizing the loss of specific target class in the training phase. To corroborate our claim, we directly feed scaled crafted perturbations by instance-specific attack MIM and our generative method into the classifier. Indeed, we find that our generative perturbation is considered as target class with a high degree of confidence whereas the perturbation from MIM performs like the noise, as shown in Fig. 6. Furthermore, we plot the logit relationship by computing PCC (Pearson correlation coefficient) values from scaled crafted perturbation and adversarial image in Fig. 7. The numerical performance is also consistent with our mentioned claim.

Second, the generated adversarial semantic pattern achieves well-generalizing performance among the different models. We feed 1k images from ImageNet test set into the generator trained by Inv3 model to obtain 1k semantic patterns, which are scaled to image pixel space and then fed into different classifiers. We compute the mean confidence of $\bm{0.46}$ for DN, $\bm{0.44}$ for Inv4, and $\bm{0.35}$ for R152, whereas the perturbation from MIM is lower than $0.01$. The results show that our scaled semantic pattern can directly achieve well-generalizing performance among models, possibly explained by utilizing similar feature knowledge from the same class on different classifiers trained on same training data distribution. Thus similar pattern can be instrumental for transferability among models.