Natural Adversarial Patch Generation Method Based on Latent Diffusion Model

Adversarial examples are meticulously designed inputs aimed at leading neural network models to make incorrect decisions. In 2014, Szegedy and othersszegedy2013intriguing successfully generated the first adversarial example by adding subtle disturbances trained in the wrong gradient direction to original digital images. This research posed a challenge to the robustness and generalizability of deep neural networks, giving birth to an entirely new field of study. Subsequent research on adversarial examples primarily focused on the digital worldgoodfellow2014explaining ; madry2017towards , where researchers added imperceptible adversarial perturbations directly to digital images at the pixel level.

In recent years, adversarial attacks in the physical world have been increasingly proposed, targeting mainly classifier and detector models, posing greater risks to DNN applications in real-world scenarios. In the physical world, since it’s not feasible to directly alter the pixel values of DNN inputs, the common approach in this field involves placing carefully designed adversarial patterns near the target object to influence the DNN’s decision-making regarding that object.

In adversarial attacks targeting classifier models, Brown et albrown2017adversarial . placed adversarial patches around a banana, leading the image classifier to misclassify it as a toaster. Sharif et al. created adversarial glasses to attack facial recognition systems. Athalye et al. introduced the Expectation Over Transformation (EoT) method to generate potent 3D adversarial objects. Evtimov et aleykholt2018robust . introduced Robust Physical Perturbations (RPP) to execute physical adversarial attacks. By affixing black and white stickers on road signs, they made the model misidentify a STOP sign as a 45 mph speed limit sign, sounding an alarm for the autonomous driving field.

In attacks targeting detector models, the primary goal of attackers is often to use adversarial patterns to make target objects evade detection by target detectors, such as human and vehicle detectors. In attacks related to human detectors, Thys et althys2019fooling . first created AdvPatch, which, when placed in front of a person, effectively evaded detection by human detectors. Building on this research, Xu et alxu2020adversarial . introduced Thin Plate Spline (TPS) technology, which simulates the deformation of clothing wrinkles during a person’s movement. They successfully designed an adversarial T-shirt, transferring the carrier of the adversarial pattern from rigid to flexible materials. However, a person wearing an adversarial T-shirt could only evade detection when facing the detector, a significant limitation. To overcome this, Hu et alhu2022adversarial . subsequently proposed Adversarial Texture, a technique that can cover clothing of any shape, allowing individuals wearing covered clothes to attack target detectors from various angles.

In the aforementioned adversarial attacks against human detectors, while each method demonstrated effective evasion of model detection, the process of pattern generation often neglected control over the color appearance of the patterns. Consequently, this led to the creation of adversarial patterns with overly vivid and bizarre colors. These attention-grabbing appearances not only deviate from the essence of adversarial examples but also risk being recognized as anomalies by humans before the patterns are inputted into the model.

To address the issue of adversarial patterns being bizarre and conspicuous, researchers like Duan and Luo duan2020adversarial ; luo2021generating have attempted to blend adversarial patterns with their surrounding physical environment. This approach aims to camouflage the patterns as much as possible within the physical world, solving the problem from the perspective of image semantic plausibility. Hu and others, meanwhile, have turned to generating natural adversarial patches by searching for adversarially effective natural patterns within the latent space learned by GANs. Although this method can effectively sample and generate natural images as adversarial patterns from random noise, it faces challenges such as training instability, model collapseavrahami2022blended , and mode collapse during the dynamic training process of GANs.

Subsequently, Tan and others proposed a new framework with a two-stage training strategy to generate legitimate adversarial patches (LAP), enhancing the visual rationality of the produced adversarial patterns. Guesmi and others introduced a method involving a similarity loss function to generate natural and more robust DAPs without using GANsdhariwal2021diffusion . However, despite both methods aiming to create more natural and plausible adversarial patterns, they still exhibit flaws and irrationalities in image quality. They do not achieve true naturalness.

Although GAN networks are known for their excellent image generation capabilities, their inherent issues make them unsuitable for generating adversarial patterns. Addressing the multitude of problems associated with GANs, the recently proposed diffusion models have demonstrated outstanding performance. Moreover, they are capable of producing images of higher quality compared to GANs.

Diffusion models have recently achieved remarkable results in fields such as computer visionbatzolis2021conditional ; jumper2021highly ; shi2020graphaf , natural language processingavrahami2022blended , and speech processing. Moreover, recent works have shown that denoising diffusion probabilistic models (DDPMs)sohl2015deep have attained state-of-the-art outcomes in terms of density estimation and the quality of generated samplesdhariwal2021diffusion .

The Denoising Diffusion Probabilistic Model (DDPM) consists of two parameterized Markov chains and utilizes variational inference to generate samples that match the original data after a given number of time steps. The forward chain incrementally adds Gaussian noise to the original data distribution through a pre-designed schedule until the data distribution converges to a standard Gaussian distribution. Conversely, the reverse chain starts from a given standard Gaussian distribution image and progressively removes the Gaussian noise learned by the neural network until the Gaussian distribution image is restored to the clean original data distribution.

Formally, we define a forward noise process $q$,It introduces Gaussian noise at time $t$ to the data distribution for the next moment,generating hidden variables denoted as $x_{1}$,$x_{2}$,$x_{3}$,…..,$x_{T}$ .Here, $t$ is a specific moment sequentially chosen from a schedule and lies within interval $t\in(0,T]$ .Given a data distribution $x\sim q(x_{0})$.The overall objective of optimizing the diffusion model is as follows:

$$E_{t\in\mu(0,T),x\in q(x_{0}),\epsilon\in N(0,I)}[||\epsilon-\epsilon_{\theta}(x_{t},t)||^{2}]$$

(1)

Herein,$\epsilon_{\theta}$represents the neural network model, which uses $x_{t}$ and $t$ to predict the noise $\epsilon$ added in the forward process.Equation (1) is akin to denoising score matching under the index $t$.During the generation process, a sample $x_{T}$ is initially selected from a standard Gaussian distribution, and then the learned reverse chain of the neural network is used to sequentially render samples $x_{t}$,until a new data $x_{0}$ is obtained.

However, training diffusion models directly in the image’s pixel space can be slow and costly. To address this issue, Rombach and others recently proposed the Latent Diffusion Model (LDM). LDM first uses an autoencoder to perceptually compress high-dimensional data images into a low-dimensional feature space. Specifically, it encodes images $x$ from the RGB space,$x\in\mathbb{R}^{H\times W\times 3}$, into latent variables $z=\varepsilon(x)$ using the encoder $\varepsilon(\cdot)$, and then reconstructs data images using the decoder $D(\cdot)$. The process can be represented as $\widetilde{x}=D(z)=D(\varepsilon(x)),z\in\mathbb{R}^{h\times w\times 3}$ where $c$ is the number of channels in the latent variables.The encoder $\varepsilon(\cdot)$ compresses the natural image $x$ into the feature space as $z$, following a parameter $f=\frac{H}{h}=\frac{W}{w}$.The diffusion model then directly learns high-frequency and imperceptible image feature abstractions in this feature space. The overall objective of LDM can be defined as:

$$E_{t\in\mu(0,T),z=\varepsilon(x),\epsilon\in N(0,I)}[||\epsilon-\epsilon_{\theta}(z_{t},t)||^{2}]$$

(2)

Since the forward chain process is fixed, it is only necessary to obtain effective latent variables $z$ from the pretrained encoder $\varepsilon$ before training. This enables the stepwise early matching to restore and map random noise back into the feature space by progressively operating on $z$.Additionally, during sampling, it suffices to decode once using the pretrained decoder to map the sampled latent variables from the feature space back to the image space.

In the generation process of the Latent Diffusion Patch (LDP), we first pretrain an autoencoder $A$ using a dataset of natural images from the same category. The autoencoder $A$ consists of an encoder $\varepsilon$ and a decoder $D$.The encoder $\varepsilon$ is used to compress data images into the feature space, and then this feature space is utilized to train the diffusion model $M$. Since the diffusion model learns the feature vectors of the data images, we can explore the latent space of the diffusion model to constrain the spatial domain of the patch to be close to the feature space of natural images.

The patch generator begins by randomly sampling a noise $Z_{T}\in\mathbb{R}^{h\times w\times d}$ from a standard Gaussian distribution, where $h$ and $w$ are the height and width of the latent variable, respectively, and $d$ is the dimension of the latent variable. By continuously denoising and exploring the latent space of the diffusion model, the random noise is mapped into the feature space of natural images. The adversarial patch $P$ is then obtained through the decoder, sampling from the explored latent variables as $P=D(M(Z_{T}))\in\mathbb{R}^{H\times W\times 3}$.ext, we iteratively update $Z_{T}$ to optimize our objective function, which is defined as follows:

$$L_{total}=L_{det}+\alpha L_{kl}+\beta L_{tv}+\gamma L_{nps}$$

(3)

The formula includes four independent loss functions. The first term, $L_{det}$ , is the adversarial detection loss, and the second term, $L_{kl}$,is the regularization loss for the latent variables. These two loss functions are used to control the adversarial attack effectiveness of the LDP (see Section 3.2 for specific details).The third term, $L_{tv}$ , represents the total variation loss, which is used to control the overall color smoothness of the adversarial pattern. It is defined as follows:

$$L_{tv}=\sum_{i,j}\sqrt{(P_{i+1,j}-P_{i,j})^{2}+(P_{i,j+1}-P_{i,j})^{2}}$$

(4)

Here, $P_{i,j}$ represents the pixel value of the LDP at coordinates $(i,j)$.The final term,$L_{nps}$,is the non-printability loss, which ensures that the pixel values of the LDP are as close as possible to the colors that can be output by printing devices. This is to ensure that the color of LDP in the physical world closely matches the color of the pattern generated in the digital world. The specific expression for $L_{nps}$ is:

$$L_{nps}=\sum_{i,j}(\underset{c\in C}{min}||P_{i,j}-c||_{2})$$

(5)

Here, $C$ represents a set of three-channel colors that can be printed by a group of $N$ printers. $\beta$ and $\gamma$ are two hyperparameters used to control the intensity of the corresponding losses. In our experiments, we set $\beta=0.1$ and $\gamma=0.01$

The process of generating the Latent Diffusion Patch (LDP) involves using adversarial gradients to guide changes in the pattern’s pixel values, thereby deceiving the target detector. To obtain the adversarial gradients of the target object, it is first necessary to render the adversarial pattern onto the target object. Then, this composite image is fed into the target detector, where the adversarial loss is calculated based on the output prediction vector.

Let $G(x)=\{P_{xywh},C_{obj},C_{cls}\}$represent a human detector, where $x$ is the input sample image, producing multiple sets of prediction vectors as output. In these vectors,$P_{xywh}$ denotes the coordinates of the predicted bounding box in $x$, $C_{obj}$ represents the probability that the box contains a target, and $C_{cls}$ denotes the probability of each category. We minimize the product of $C_{obj}$ and $C_{cls}$ simultaneously to reduce the detector $G$ confidence in recognizing human targets in $x$, thereby achieving the effect of evading detection. The specific formula is as follows:

$$L_{det}=\frac{1}{N}\sum_{i=1}^{N}\underset{}{max}[C_{obj}(x^{\prime}_{i})\times C_{cls}(x^{\prime}_{i})]$$

(6)

Where $x^{\prime}_{i}$ represents the $i$-th image in a single batch of images with adversarial noise added, and the total number of images in the batch is $N$.Equation (6) uses the reduction of the maximum confidence level of all human categories in the image detection results as the loss function. By iteratively lowering the maximum confidence score of human targets in each round, the Latent Diffusion Patch (LDP) achieves its effect of inducing suppression of detection.

Moreover, during the adversarial optimization process, the model can optimize any latent vector $Z_{T}$ within the high-density region deviating from the standard Gaussian distribution. Therefore, it’s necessary to set constraints on it. The diffusion model is trained by randomly sampling noise from the standard Gaussian distribution, and during the forward diffusion process, each time step variable is encoded as a Gaussian distribution dependent only on the previous time step variable. For continuous Gaussian diffusion models, the starting point of the reverse denoising process must conform to the data distribution of the standard Gaussian distribution. If $Z_{T}$, deviating from the standard Gaussian distribution area, is mapped through the diffusion model’s denoising process, it cannot ensure that the generated patches are sufficiently realistic.

To ensure the realism of the generated adversarial patterns, this paper imposes a regularization loss $L_{kl}$ to constrain the mean and variance of $Z_{T}$, thereby keeping $Z_{T}$ within the vicinity of the standard Gaussian distribution. The regularization term loss is defined as:

$$L_{kl}=KL(N(\mu,\sigma^{2})||N(0,I))=\frac{1}{2}(-log\sigma^{2}+\mu+\sigma^{2}-1)$$

(7)

Where $\mu$ and $\sigma$ are the mean and standard deviation of the distribution in which $Z_{T}$ resides.

However, the process of directly sampling $Z_{T}$ from the probability distribution $N(\mu,\sigma^{2})$ is non-differentiable. Therefore, to effectively optimize the neural network, we shift the optimization target from the latent vector $Z_{T}$ to $\mu$ and $\sigma$.Since the linear transformation of a Gaussian distribution remains a Gaussian distribution, the optimized $\mu$ and $\sigma$ can be re-sampled through a parameter transformation. This transformation converts the process of sampling a latent vector $Z_{T}$ from $N(\mu,\sigma^{2})$ into first sampling random noise $\varepsilon$ from $N(0,I)$,and then setting $Z_{T}=\mu+\varepsilon\times\sigma$.This process facilitates the conformity o $Z_{T}$ to the standard Gaussian distribution using KL divergence. We use the weight coefficient $\gamma\alpha$ to control the intensity of $L_{kl}$,with $\alpha=0.5$ in our experiments.