Towards Visual Distortion in Black-Box Attacks

In white-box attack, the adversary knows the details of a network, including network structure and its parameter values. Goodfellow et al. [12] proposed a fast gradient sign method to generate adversarial examples. It’s computationally effective and serves as a baseline for attacks with additive noise. In [13], a functional adversarial attack that applied functional noise instead of additive noise to the image, is introduced. Recently, Jordan et al. [7] stressed quantifying perceptual distortion of the adversarial examples by leveraging perceptual metrics to define an adversary. Different from our method which directly optimizes the metric, their model conducts a search over parameters from several composed attacks. There are also attacks that sample noise from a noise distribution [14, 15], on the condition that gradients from the white-box network is accessible. Specifically, [14] utilizes particle approximation to optimize a convex energy function. [15] formulates the attack problem as generating a sequence of adversarial examples in a Hamiltonian Monte Carlo framework.

In summary, white-box attack is hard to detect or defend [16]. In the meantime, however, it suffers from label-leaking and gradient-masking problem [2]. The former causes adversarially trained models to perform better on adversarial examples than original images, and the latter neutralizes the useful gradient for adversaries. The preliminary of acquiring full access to a network in white-box attack is also sometimes difficult to satisfy in real-world scenarios.

Black-box attack considers the target network as a black-box, and has limited access to the network. We discuss loss-oracle based attack here, where the adversary assumes only loss-oracle access to the black-box network.

where smaller $d(x,x+\delta)$ indicates less visual distortion. $d$ can be any form of metric that measures the perceptual distance between $x$ and $x+\delta$, such as well-established $1-\text{SSIM}$ [36] or LPIPS [37]. $\lambda$ manages the trade-off between a successful attack and the visual distortion caused by the attack. The effects of $\lambda$ will be further discussed in Section 4.1.

Minimizing the above loss function faces a challenge that $L$ is not differentiable since the black-box adversary does not have access to the gradients of $L$ and the metric $d(x,x+\delta)$ might be calculated in a non-differentiable way. To address this problem, we explicitly assume a flexible noise distribution of $\delta$ in the discrete space, in the sense that the noise values and their probabilities are discrete. And the gradient of $L$ can be estimated by sampling from this distribution. Suppose that $\delta$ follows a distribution $p_{\theta}$ parameterized by $\theta$, i.e., $\delta\sim{p_{\theta}}$. For the $j_{\text{th}}$ pixel in an image, we make its noise distribution $p_{\theta^{j}}=\text{softmax}(\theta^{j})$, where $\theta^{j}$ is a vector and each element in it denotes a probability value. By sampling noise from the distribution $p_{\theta}$, $\theta$ can be learned to minimize the expectation of the above loss such that the attack is successful (i.e., alters the predicted label) and the produced adversarial example is less distorted (i.e., small $d$):

For the $j_{\text{th}}$ pixel, we define its noise’s sample space to be a set of discrete values ranging from $-\epsilon$ to $\epsilon$: ${\delta^{j}}\in\{\epsilon,\epsilon-\frac{\epsilon}{N},\epsilon-2\frac{\epsilon}{N},...,0,...-\epsilon\}$, where $N$ is the sampling frequency and $\frac{\epsilon}{N}$ is the sampling interval. The noise value ${\delta}^{j}$ of the $j_{\text{th}}$ pixel is sampled from this sample space by following $p_{\theta^{j}}$, $p_{\theta^{j}}\in{\mathbb{R}}^{2N+1}$.

Given $W$ and $H$ the width and height of an image, respectively, since each pixel has its own noise distribution $p_{\theta^{j}}$ of length $2N+1$, the number of parameters for the entire image is $(2N+1)WH$. Note that we do not consider the difference of color channels in order to reduce the size of the sample space. Otherwise the number of parameters would be tripled. Thus, the same noise value is sampled for each RGB channel of a pixel. To estimate $\theta$, we adopt policy gradient [38] to make the above expectation differentiable with respect to $\theta$. Using REINFORCE, we have the differentiable loss function $F(\theta)$:

where $b$ is introduced as a baseline in the expectation with specific meaning: 1) when $L(x,x+\delta)<b$, the sampled noise map $\delta$ returns low $L$, and its probability ${p_{\theta}}(\delta)$ increases through gradient descent; 2) when $L(x,x+\delta)=b$, ${\nabla}{F}(\theta)=0$ and ${p_{\theta}}(\delta)$ remains unchanged; 3) when $L(x,x+\delta)>b$, the sampled noise map $\delta$ returns high $L$, and its probability ${p_{\theta}}(\delta)$ decreases through gradient descent. To sum up, $L(x,x+\delta)$ is forced to improve over $b$. At the iteration $t$, we choose $b={{\text{min}}}_{T=0,1,...t-1}{L}(x,x+{\delta}_{T})$ such that $L$ improves over the obtained minimal loss.

The above expectation is estimated using a single Monte Carlo sampling at each iteration, and the sampling of noise map $\delta$ is critical. Simply sampling ${\delta}_{t}$ at the iteration $t$ on the entire image might cause large variance on the norm of the noise, i.e., $||{\delta}_{t}-{\delta}_{t-1}||_{2}$. Therefore, to ensure a small variance, with $T^{*}={{\text{argmin}}}_{T=0,1,...t-1}{L}(x,x+{\delta}_{T})$, only a number of $qWH$ pixels’ noise values are resampled in $\delta_{T^{*}}$, while $(1-q)WH$ pixels’ noise values remain unchanged:

The above equation means replacing $qWH$ pixels’ noise values in noise map $\delta_{T^{*}}$ with those in ${\delta_{t}}$, which are sampled from distribution $p_{\theta_{t}}$. In other words, if $q=0.01$, then only a random $1\%$ of $\delta_{T^{*}}$ is updated at each iteration. As shown in Fig. 2, after sampling ${\delta}_{t}$, the feedback $L(x,x+{\delta}_{t})$ from the black-box and the perceptual distance metric decide the update of the distribution $p_{\theta_{t}}$. The iteration stops when the attack is successful, i.e., ${\text{max}}(0,f{(x+\delta_{t})_{y}}-{\text{ma}}{{\text{x}}_{k\neq y}}f{(x+\delta_{t})_{k}})=0$.

Ruan et al. [39] shows that feed-forward DNNs (Deep Neural Networks) are Lipschitz continuous with a Lipschitz constant $K$, for which we have

At the iteration $t$, since only a small part of the noise map is updated, it can be assumed that

where $C$ is a constant. Suppose that the perceptual distance metric $d$ is normalized to $[0,1]$. Substituting the inequalities (7) and (8) in our definition of $L$ in Eq. (2) gets the following inequalities:

Ideally, $L(x,x+\delta_{t})-L(x,x+\delta_{T^{*}})$ accurately quantifies the difference of the perturbed image even when only one noise value for just a single pixel at the iteration $t$ is different from that at $T^{*}$. Let $\delta^{ij}$ represent a special noise map, whose $j_{\text{th}}$ pixel’s noise value is the $i_{\text{th}}$ element in its sample space and the other pixels’ noise values are $0$. Note that the length of the sample space for each pixel is $2N+1$. Similarly, ${p_{\theta_{t}}}(\delta^{ij})$ denotes the probability of the $i_{\text{th}}$ element in the sample space of the $j_{\text{th}}$ pixel. By sampling every element in the sample space of the $j_{\text{th}}$ pixel, we define $l_{t}^{j}$ and $p_{\theta_{t}^{j}}$ to be a vector:

Although the above equations are only meaningful under the ideal situation where $L$ can quantify the difference of just one perturbed pixel, we use these equations for a theoretical proof of convergence. In the ideal situation, the gradient of the $j_{\text{th}}$ pixel’s parameters can be calculated exactly as

According to Eq. (9) when the number of the resampled pixels $qWH$=1, we have

Note that for $\forall{t_{1}},{t_{2}}$ that share the same $T^{*}$, $l_{t_{1}}^{j}$ is equal to $l_{t_{2}}^{j}$. Thus, using Eq. (13), we have

In practice, we adopt a single Monte Carlo sampling instead of sampling every noise values for every pixel, for which $2N+1$ should be replaced by $1$ in the above inequality. The inequality (14) thus becomes:

The standard softmax function disappears because it is Lipschitz continuous with the Lipschitz constant being $1$ [40]. Finally, we have the inequality for ${||}\nabla F({\theta_{{t_{1}}}})-\nabla F({\theta_{{t_{2}}}})|{|_{2}}$:

The above inequality proves that $F(\theta)$ is $L$-smooth with the Lipschitz constant being $2K\epsilon c+C+\lambda$. If $F(\theta)$ is convex, the exact number of steps that Stochastic Gradient Descent (SGD) takes to convergence is $\frac{{(2K\epsilon c+C+\lambda)\cdot||{\theta_{0}}-{\theta^{*}}||_{2}^{2}}}{\xi}$ , where $\xi$ is an arbitrary small tolerable error ($\xi>0$). However, since deep network $L$ is usually highly non-convex, we need to consider the situation where$F(\theta)$ is non-convex.

And we select ${\eta_{t}}$ that satisfies

Condition (19) can be easily satisfied with a decaying learning rate, e.g., ${\eta_{t}}=\frac{1}{{\sqrt{t\ln(t+1)}}}$. According to Lemma 1 and Theorem 2 in [41], using the $L$-smooth property of $F(\theta)$ , ${\nabla}{F}(\theta_{t})$ goes to $0$ with probability $1$. This means that with probability $1$ for any $\xi>0$ there exists $N_{\xi}$ such that ${\nabla}{F}(\theta_{t}){\leq}{\xi}$ for $t{\geq}{N_{\xi}}$. Unfortunately, unlike in the convex case, we do not know the exact number of steps that SGD takes to convergence.

The above proof simply aims to theoretically show that the proposed method converges in finite steps, although possibly in a rather slow speed. From the “Avg. Queries” in the following experiments, we can see that the actual computational cost is affordable and comparable to some of the query-efficient attacks.

In the ablation studies, the maximum number of queries is set to be $10,000$. The results are averaged on $1000$ test images. In the following, we discuss the trade-off between visual distortion and query efficiency, the effects of using different perceptual distance metrics in the loss function, the results on different sampling frequencies and the influence of predefining a specific form of noise distribution.

Although our method in this paper is based on $l_{\infty}$ attack, the perceptual distance metric $d$ in the loss function can be replaced by other $l_{p}$ ($p=0,1,2$) distance. We did not discuss it in the above experiments because these $l_{p}$ distance metrics are less accurate in measuring the perceptual distance between images compared to the specifically designed metrics, such as well-established $1-\text{SSIM}$ and LPIPS. Nevertheless, we still present the results of other $l_{p}$ ($p=0,1,2$) attacks in Table 6, where the $l_{p}$ distance is normalized to $[0,1]$ in the loss function. Specifically, $d(x,x+\delta)=\frac{{{l_{p}}(x,x+\delta)}}{{{{\max}_{\delta}}({l_{p}}(x,x+\delta))}}$, where ${l_{p}}(x,x+\delta)$ is the $l_{p}$ distance between the original image $x$ and the perturbed image $x+\delta$. As in the paper, we set $\lambda=10,\epsilon=0.05$ and the maximum number of queries being $10,000$. We find that the raw $l_{0}$ and $l_{1}$ scores have much higher order of magnitude compared with other metrics, and thus the normalized scores of $l_{0}$ and $l_{1}$ distances are reported in Table 6. Note that when the sampling frequency $N=1$, $l_{0}$ distance is equivalent to $l_{1}$ distance in that

where $m$ is the number of perturbed pixels. $W,H$ and $c$ are the width, height and number of channels of a given image, respectively. Table 6 shows that optimizing $l_{0}$ distance gives better performance on both the perceptual distance metrics and the $l_{p}$ distance metrics.

We introduce a novel black-box attack based on the induced visual distortion in the adversarial example. The quantified visual distortion, which measures the perceptual distance between the adversarial example and the original image, is introduced in our loss where the gradient of the corresponding non-differentiable loss function is approximated by sampling from a learned noise distribution. The proposed attack can achieve a trade-off between visual distortion and query efficiency by introducing the weighted perceptual distance metric in addition to the original loss. The experiments demonstrate the effectiveness of our attack on ImageNet as our model achieves much lower distortion when compared to existing attacks. In addition, it is shown that our attack is valid even when it’s only allowed to perturb pixels that are out of the target object in a given image.