Enhancing Clean Label Backdoor Attack with Two-phase Specific Triggers

We assume our backdoor attack happens when the adversaries have a pre-trained model $f(\Theta,x)$, retraining dataset $D$, where the data pair is denoted as $(x_{i},y_{i})$. The target class is $y_{t}$ is randomly chosen as the 7th class ”gas pump” in this example.

Our proposed backdoor attack consists of two phases: (A) the backdoor implantation phase and (B) the backdoor activation phase. Before performing backdoor attacks, we will train a trigger generator $G$. The details of the trigger generator will be described in Section 3.4. In the backdoor implantation phase (Fig. 1 (A)), we first select a portion of the images in the target class and use the trigger generator $G$ to build the poisoned data image $x^{\prime}$ with an image-specific trigger $\Delta$. For other clean images of the target class, we keep them in the retraining datasets. Second, we mix these poisoned images with the other clean images of the rest classes. Finally, We use this mixed datasets $D^{\prime}$ to fine-tune the pretrained model $f(\Theta,x)$ to obtain the backdoor model $f^{b}(\Theta,x)$. In the backdoor activation phase, we first feed the non-targeted clean image $x$ into the trigger generator to obtain the malicious input $x^{\prime}$ with an image-specific trigger $\Delta$. Then we feed this malicious input $x^{\prime}$ into the backdoor model $f^{b}(\Theta,x)$, the model will produce the target label. This process is shown in Fig. 1 (B-2). Besides, the backdoor model $f^{b}(\Theta,x)$ should produce correct prediction results for clean images without triggers. As shown in Fig. 1 (B-1), the backdoor model $f^{b}(\Theta,x)$ will output a correct prediction “parachute” when the input is clean.

To promote our backdoor attack, we want the image-specific triggers of target, and non-target images can promote both the backdoor implantation and activation phases. Specifically, in the backdoor implantation phase, we want the poisoned target data $x^{\prime}_{t}$ with added trigger $\Delta$ to facilitate the classifier model $f(\Theta,x)$ to learn the features between the trigger $\Delta$ and the target class label $y_{t}$. Inspired by clean-label backdoor attack Turner et al. (2018), we generate specific triggers $\Delta_{t}$ for the target class images $x_{t}$, which can erase the features of the original image and thus enhance the backdoor implantation. i.e., we want the victim model $f(\Theta,x)$ can not predict the correct result when the input $x_{t}$ is added with a specific trigger $\Delta_{t}$. We can represent this process as Eq. (1).

Standard triggers are only effective when the backdoor is implanted. In our approach, we generate the triggers which can achieve some attack success rate even before backdoor implantation. Moreover, this type of trigger can be more effective after backdoor implantation. Specifically, in the backdoor activation phase, we want the crafted non-target image $x^{\prime}_{nt}$ with the added trigger $\Delta_{nt}$ to activate the backdoor efficiently, causing the backdoor model $f^{b}(\Theta,x)$ to output the target label $y_{t}$ expected by the adversaries. We can represent this process as Eq. (2).

Inspired by GAP Poursaeed et al. (2018), we use a U-Net architecture Ronneberger et al. (2015) autoencoder as the trigger generator $G$ to generate the image-specific triggers used in backdoor implantation phase and backdoor activation phase. This autoencoder is with skip connections between the encoder and the decoder.

In order to obtain a trigger generator that can generate specific triggers used in different phases depend on each image, we train this trigger generator before performing the backdoor attack. This training process is described in Algorithm 1. We calculate different losses depend on different types of images. i.e., we calculate the target images loss $\mathcal{L}_{t}$, non-target images loss $\mathcal{L}_{nt}$ and LPIPS loss $\mathcal{L}_{lpips}$, then add up these loss terms with hyperparameters ($\alpha,\beta,\gamma$) linearly to a total loss $\mathcal{L}_{total}$ (represented as Eq. (3)). Then use an optimizer to minimize the loss and update the weights of the generator $G$.

The target images loss $\mathcal{L}_{t}$ is represented as Eq. (4), $\mathcal{H}$ is the cross-entropy function, $x^{\prime}_{(t,i)}$ is the $i$ th crafted target image which is added by clean target image $x_{t}$ and an image-specific trigger $\Delta_{t}$, $m$ is the total amount of target images. The trigger $\Delta_{t}$ is generated by trigger generator $G$ and clipped with a $l_{\infty}$ norm limit $\varepsilon$. $y_{llc}$ is the least likely class of clean classifier model $f(\Theta,x_{t})$. (as proposed by Poursaeed et al. (2018)). We find $\mathcal{L}_{t}$ shows good results on Eq. (1).

Similar to $\mathcal{L}_{t}$, the non-target images loss $\mathcal{L}_{nt}$ is represented as Eq. (5), but this loss is calculated on non-target images $x_{(nt,i)}$, $n$ is the total amount of target images. and we use target label $y_{t}$ instead of $y_{llc}$. We find $\mathcal{L}_{t}$ shows good results on Eq. (2).

The LPIPS loss is represented as Eq. (6). LPIPS is perceptual metric proposed by Zhang et al. (2018), we use this loss to minimize perceptual distortion on crafted images $x^{\prime}_{i}$ and clean images $x_{i}$ both on target images and non-target images. $r$ is the total amount of all images, note that $r=m+n$.

Note that, in order to keep the target image loss $\mathcal{L}_{t}$ and the non-target image loss $\mathcal{L}_{nt}$ converge at the same speed, the amount of non-targeted images and target images in the dataset $D_{g}$ for training trigger generator is 1:1, and the amount of each class images in non-targeted images is the same. (i.e., if there are 900 images in the target class, there are 100 images in each of the other nine classes)

In backdoor implantation phase: First, we use the pretrained trigger generator $G$ to generate triggers and build poisoned images $x^{\prime}_{t}$ from target class images $x_{t}$. We use poisoning rate $\lambda$ to indicate ratio of the poisoned images amount to total images amount (Note that, our attacks can be effective even with poison rate $\lambda=1\%$). Then mix them with other clean images (including clean target class images $x_{t}$ and clean non-target class images $x_{nt}$) to obtain a mixed retraining dataset $D^{\prime}$. Third, $D^{\prime}$ will be used by the victim to fine-tune his classifier model from a pretrained classifier model $f(\Theta,x)$. After training, we will obtain a backdoor model $f^{b}(\Theta,x)$. In the backdoor activation phase: First, for the a non-target image $x_{nt}$, we use the trigger generator $G$ to generate its image-specific triggers $\Delta_{nt}$, then add $\Delta_{nt}$ on the $x_{nt}$ to obtain a malicious input $x^{\prime}_{nt}$. Then we feed $x^{\prime}_{nt}$ into the backdoor model $f^{b}(\Theta,x)$, the prediction will be the target class $y_{t}$.

In this section, we demonstrate the effectiveness and stealthiness of our methods. We mainly compare our attack method with other backdoor attacks under clean-label settings, including Clean-Label Backdoor Attack (CLBA) Turner et al. (2018) and Global-Random-Trigger Backdoor Attack (GRTBA). CLBA is the first work to consider backdoor attacks under clean-label settings. This work first adds an adversarial perturbation to the clean image and then add a fixed square trigger on the right corner of the images. In this experiment, We utilize the FGSM as the perturbation and set $l_{\infty}=16/255$, which can achieve FR at 95.69%. For the trigger, we set it to 50$\times$50 size. GRTBA is a backdoor attack that uses a global random noise as the trigger. In this attack, adversaries first generate a random noise (by Numpy Library Harris et al. (2020)) of the same size as the clean images, then clip it with a $l_{\infty}$ limit. Note that we do this experiment to demonstrate that global random trigger is not able to achieve attack success even with a higher $l_{\infty}$ limits. In this experiment, We set the $l_{\infty}=40/255$. For our methods, we train two trigger generators with trigger clip limit $l_{\infty}=25/255$ and $l_{\infty}=30/255$. We use these two trigger generators to conduct our attack. Final, all poisoned images, relevant triggers and perturbations of CLBA, GRTBA and Ours, are visualized in Fig.(2).

Table. 1. shows the results. All experiments are conducted on ImageNette, and the poisoning rate, retraining epoch is set to 5% and 1. In terms of effectiveness, our methods can achieve high ASR (0.9898 and 0.9549 for Ours-1 and Ours-2) while CLBA and GRTBA can only achieve ASR at 0.1503 and 0.4129. Meanwhile, the BA of Ours remains very close to the clean model, the drop of BA is less than 0.004. In terms of stealthiness, Ours-1 achieve the best results on LPIPS, PSNR, and $l_{\infty}$ and BA with a second-best ASR. Ours-2 achieve the best results on ASR with second-best LPIPS, PSNR, and $l_{\infty}$.

We also compare some other backdoor attacks on stealthiness, including BadNets, Clean-Label Backdoor Attacks on Video Recognition Models (CLBA-V) Zhao et al. (2020), Hidden Trigger Backdoor Attack (HDTBA) Saha et al. (2020), Backdoor Attack with Sample-Specific Triggers (SSTBA) Li et al. (2020). Clean label settings and image-specific trigger are two properties making backdoor attacks more stealthy. Clean-label settings play an essential role in stealthy backdoor attacks. Compare with altering-label type backdoor attacks, we think, even the triggers used in altering-label type backdoor attacks are invisible to humans, the huge inconsistency between the original features of the poisoned images and the target label is still apparent to human reviewers, then the human may remove these data, resulting in the failure of the backdoor attacks. Clean-label settings are the first-level stealthy properties. After applying clean-label settings, we can consider applying more stealthy triggers. The fixed triggers used in former clean-label type backdoor attacks are still apparent to human reviewers. Thus, we think image-specific triggers are more stealthy than the fixed triggers. As Table.2 shows, our methods both satisfy the clean-label settings and the specific trigger, that is more stealthy than other backdoor attacks.

The poisoning rate is an essential aspect of stealthiness. We do experiments on poisoning rate $\lambda$ vary from 1% to 10% to evaluate our methods, CLBA and GRTBA. The experimental settings expect poisoning rate $\lambda$ are the same with Section 4.2. Fig. 3 shows that our methods can achieve high ASR while maintaining high BA with a low poisoning rate (highest ASR at 0.9898 with $\lambda=5\%$). CLBA and GRTBA can only achieve ASR $\leq$ 0.1905 and 0.4601.

We study the effect on FR and ASR with different $l_{\infty}$ limits. And we test on the clean model and the backdoor model to demonstrate the effect of our triggers. We set $l_{\infty}=[10/255,15/255,20/255,25/255,30/255]$.

As Table. 3 shows, our triggers can achieve some FR and ASR before backdoor implantation (highest FR at 0.7808, highest ASR at 0.4565). After backdoor implantation, we can achieve higher FR and ASR on the backdoor model than the clean model (highest FR at 0.8756, highest ASR at 0.9898). The distance of FR and ASR between the clean model and the backdoor model proves that our triggers can promote the backdoor implantation.

Besides, as Table. 3 shows, with $l_{\infty}$ limits grows, FR and ASR also grow gradually. but there is a trade-off between stealthiness and effectiveness. the higher ASR means higher $l_{\infty}$ for the trigger. However, we still can achieve ASR = 0.8547 with $l_{\infty}=15/255$, the drop of ASR is only 0.1351.

To study this impact on CLBA, GTRBA and Ours, we performed data augmentation consisting of RandomRotation, RandomResizeCrop, RandomHorizontalFlip on the dataset for training backdoor models. We find that ASR of CLBA drops to 0.1054 (drop 0.0449); ASR of GTRBA increases to 0.6333 (increase 0.2204); ASR of Ours stays at 0.9335 (drop only 0.0214). And the BAs of these methods drop to 0.9579, 0.9600, and 0.9564, respectively.

Based on these observations, we believe that data augmentation has a destructive effect on backdoor attacks using fixed triggers. In contrast, it is less effective in defending against or even enhancing backdoor attacks using global triggers. We consider this is because the global triggers can be resistant to those data augmentation methods due to the robustness.