Certified $\ell_{2}$ Attribution Robustness via Uniformly Smoothed Attributions

Attribution methods study the importance of each input feature, and measure that how much every feature contributes to the model prediction. One of popular attribution approach is the gradient-based method. Based on the property that gradient is the measurement of the rate of change, the gradient-based methods measure the feature importance by weighting the gradients in different ways. Examples of gradient-based methods include saliency (Simonyan et al., 2014), integrated gradients (IG) (Sundararajan et al., 2017), full-gradient (Srinivas & Fleuret, 2019), and etc. Other attribution methods include occlusion (Simonyan et al., 2014), which measures the importance of each input feature by occluding the feature and measuring the change of the model prediction, layer-wise relevance propagation (LRP) (Bach et al., 2015), which propagates the output relevance to the input layer, and SHAP related methods (Lundberg & Lee, 2017; Sundararajan & Najmi, 2020; Kwon & Zou, 2022). An important property of the attribution methods is the axiom of completeness that $\sum_{i}g_{i}(\bm{x})=f_{j}(\bm{x})$, which indicates the relationship between attribution and the prediction score. Note that the gradient-based attribution methods are upper-bounded since the gradients of the model output with respect to the input are upper-bounded.

Ghorbani et al. (2019) first pointed out that attributions can be fragile to iterative attribution attack, and Dombrowski et al. (2019) extended the attack to be targeted that attributions can be changed purposely into any preset patterns. Similar to adversarial attacks, attribution attacks maximize the loss function that measures the difference between the original attributions and the target attributions. In addition, the attribution attacks are controlled not to alter the classification results. To defend against attribution attacks, adversarial training (Madry et al., 2018) approaches have been adopted. Chen et al. (2019) and Boopathy et al. (2020) minimizes the $\ell_{p}$-norm differences between perturbed and original attributions, and Ivankay et al. (2020) considers Pearson’s correlation coefficient. Wang & Kong (2022) suggests to use cosine similarity, emphasizing the attributions should be measured in directions rather than magnitudes. All of the above defenses improve the attribution robustness empirically, but none of them is able to provide certifications to ensure that the attributions will not change under any perturbations within the allowable attack region.

The smoothing technique has been popular in improving certified adversarial robustness (Liu et al., 2018; Lecuyer et al., 2019; Cohen et al., 2019; Yang et al., 2020). The smoothed classifiers take a batch of inputs that are randomly sampled from the neighbourhood of original inputs under certain distributions $q$ and make the decisions based on the most likely outputs, i.e., $\operatorname*{arg\,max}_{y}\mathbb{P}_{\bm{\eta}\sim\mu}[F(\bm{x}+\bm{\eta})=y]$. They provide the certification of the maximum radius so that no perturbations within the radius can alter the classification label. Cohen et al. (2019) certifies the $\ell_{2}$ attack based on the Neyman-Pearson lemma and the result is alternatively proved by Salman et al. (2019) using explicit Lipschitz constants. Lecuyer et al. (2019) and Teng et al. (2020) consider Laplacian smoothing for the $\ell_{1}$ attack. Yang et al. (2020) derived a similar result in $\ell_{\infty}$ though the radius becomes small when the dimension of data gets large. Kumar & Goldstein (2021) applied randomized smoothing to structured output, which the attributions belong to, but the specific bound for attribution is to loose to be meaningful. Thus, there are no existing works that provide defense and valid certifications for attribution using randomized smoothing due to the difficulty of defining the attribution robustness and the computation of the attribution gradient. In this work, an effective method to formulate the smoothed attribution robustness as a simple optimization problem is proposed to provide certifications against attribution attacks.

We note that cosine similarity of attributions is in fact an inner product of their normalized vectors. Besides, we also observe that $h(\bm{x})$ is the mean of $g(\bm{x}+\bm{\eta})$ with respect to $\bm{\eta}$, which is, by definition, equivalent to the integral of $g$ weighted by the density of uniform distribution over $\mathcal{B}(\bm{x};r)$. More importantly, for perturbed example $\tilde{\bm{x}}$, the weighting region is $\mathcal{B}(\tilde{\bm{x}};r)$, which is a translated version of $\mathcal{B}(\bm{x};r)$ and they are expected to intersect with each other when the distance of their centers is smaller than twice of the radius $r$. Therefore, given the input sample $\bm{x}$ and its attribution $h(\bm{x})$, we can rewrite the minimization of cosine similarity as follows:

where $a$ represents the magnitude of the translation and the unit vector $\bm{v}$ is the direction of translation as shown in Figure 1 (right). It can be shown that the magnitude $a$ is constrained by a constant related to the intersecting volume and the property of the attribution function itself.

In a high-dimensional case, for a fixed cosine similarity value, a given $h(\bm{x})$ and $h(\tilde{\bm{x}})$, in fact, form a spherical cone. Thus, we can decompose the directional unit vector $\bm{v}$ into $\bm{v}=\cos\theta\bm{v}_{\parallel}+\sin\theta\bm{v}_{\perp}$, where $\bm{v}_{\perp}$ is perpendicular to $h(\bm{x})$ and $\bm{v}_{\parallel}$ is parallel to $h(\bm{x})$. Then, we have

where $0\leq\theta\leq 2\pi$.

Since $a$ is a scalar representing the magnitude of the translation from $h(\bm{x})$ to $h(\tilde{\bm{x}})$, it can be shown that the magnitude of $a$ is upper bounded by the magnitude of the gradient weighted by the ratio of volume change during the translation process. Specifically, $a\leq MV_{U}/V_{\mathcal{S}}$, where $V_{\mathcal{S}}$ is the volume of the $\ell_{2}$-ball $\mathcal{B}(\bm{0};r)$, $V_{U}$ is the volume of the union of the two sampling space centered at $\bm{x}$ and $\tilde{\bm{x}}$ minus their intersection, and $M$ is a constant that depends on the upper bound of $g$, We notice that the gradient-based attribution is a function of input gradient, $\nabla f(\bm{x})$, which is bounded for Lipschitz continuous networks (See Lemma 1 in the Appendix A); thus, the upper bound can be derived separately for different attribution functions.

Now that we have reformulated the optimization with respect to vector $\bm{v}$ into an alternative simpler one with respect to scalar values $a$ and $\theta$. By solving the alternative problem, our result shows that the smoothed attribution is robust within the following half-angle of a spherical cone.

The entire proof can be found in Appendix A. The theorem points out that the lower bound of cosine similarity is related to the smoothing space around the input samples and the maximum allowable perturbation size. Moreover, when the smoothing space is a $\ell_{2}$-norm ball, the above result can be derived by directly computing two volumes $V_{U}$ and $V_{\mathcal{S}}$, which can be explicitly calculated by

where $h=r-\epsilon/2\geq 0$ and $I_{x}(a,b)$ is the regularized incomplete beta function, cumulative density function of beta distribution (Li, 2010).

We observe the following properties of the above theorem.

1. 1.

   Unlike previous attribution robustness works, such as Dombrowski et al. (2019), Singh et al. (2020), Boopathy et al. (2020) and Wang & Kong (2022), which require the networks to be twice-differentiable and need to change the ReLU activation into Softplus, the proposed result does not assume anything on the classifiers. Thus, it can be safely applied to any neural network and any architectures.

2. 2.

   The lower bound depends on the radius of smoothing, $r$. The bound becomes larger as the radius of smoothing grows. In extreme cases when $r$ tends to infinity, the smoothing spaces of two samples completely overlap, which corresponds to the same smoothed attribution and their cosine similarity becomes $1$.

3. 3.

   At the same time, the lower bound also decreases when the attack budget $\epsilon$ increases. Besides, when $\epsilon$ increases beyond the constraint that $h=r-\epsilon/2\geq 0$ and tends to infinity, the distance between two attributions will become further and their cosine similarity will tend to $0$ in high dimensional space, which makes the lower bound becomes trivial.

4. 4.

   The proposed method can be scaled to datasets with large images. The lower bound is efficient to compute since only the smoothed attribution of given sample needed. On the contrary, the previous works that approximately estimate the attribution robustness (Wang & Kong, 2023) require the computation of input Hessian and the corresponding eigenvalues and eigenvectors, which becomes intractable for larger size images on modern neural networks.

The previous section formulates the robustness of smoothed attribution in terms of the smallest cosine similarity between the original and perturbed smoothed attribution, when the attack budget and the smoothing radius are fixed. In some scenarios, practitioners want to formulate the robustness in different ways. For example, we may want to find the maximum allowable perturbation $\epsilon$ such that the cosine similarity between the original and perturbed smoothed attribution is guaranteed to be greater than a predefined threshold $T$. On the other hand, one can also obtain the minimum smoothing radius needed such that the desired attribution robustness is achieved within allowable attack region. The following corollary provides the alternative formulations of the attribution robustness.

The derivation of the Corollary can be found in Appendix A. We notice that, in these formulations, when the smoothing radius is larger, the maximum allowable perturbation is also larger, which allows stronger attacks while keeping the attribution similarity within a controllable range. Similarly, when the maximum allowable perturbation is larger, the minimum smoothing radius is also larger, which means that the attribution similarity can be maintained with a larger smoothing radius.

We first conduct the experiment to verify that the uniformly smoothed attribution itself is more robust than the original attribution. The uniform smoothing around the $\ell_{2}$ ball with radius $0.5$ is applied to the saliency map (SM) (Simonyan et al., 2014) and integrated gradients (IG) (Sundararajan et al., 2017) and evaluate on CIFAR-10. The resultant attributions are denoted by SmoothSM and SmoothIG, respectively. We then attack the attributions using the $\ell_{2}$ IFIA attack and evaluate the robustness using Kendall’s rank correlation and top-k intersection (Ghorbani et al., 2019). The experiments are evaluated on both non-robust model (Standard) and robust models (IG-NORM, TRADES, IGR). Note that IFIA is directly performed on the smoothSM and smoothIG, instead of its original counterpart. Since the PGD-like attribution attack requires to take the derivative of the attribution to determine the direction of gradient descent, the double backpropagation is needed. Thus, it is necessary to replace the ReLU activation by the twice-differentiable Softplus during attack (Dombrowski et al., 2019). The results are shown in Table 1.

We observe that for the non-robust model, both SmoothSM and SmoothIG perform better than its non-smoothed counterparts in both metrics, which shows that the uniformly smoothed attribution itself is more resistant to the attribution attacks. For models that are specifically trained to defend against the attribution attacks using heuristic methods adapted from adversarial training, e.g., IG-NORM, TRADES and IGR, the smoothed attributions show comparable robustness to the non-smoothed attribution. Moreover, we also notice that SmoothIG performs better than SmoothSM, especially for the non-robust models. This can be attributed to the fact that IG satisfies the axiom of completeness, which ensures that the sum of IG is upper-bounded by the model output. In addition, we also observed that the smoothing technique does not always enhance the robustness of attribution, as measured by top-k and Kendall’s rank correlation. It is worth noting that Yeh et al. (2019) argued that randomized smoothing can reduce attribution sensitivities and consequently improve robustness. The disparity in our findings stems from the fact that we specifically evaluated Kendall’s rank correlation, whereas their study defined sensitivity based on $\ell_{2}$-norm distance, which was subsequently deemed inappropriate for evaluating attribution robustness by Wang & Kong (2022).

In this section, the lower bound of the cosine similarity between the original and attacked attribution is reported. We use the integrated gradients as an example since it is well-bounded due to the axiom of completeness, and the technique can be also applied to any other gradient-based attributions.

Table 2 reports the theoretical bound evaluated on MNIST computed using Theorem 1. We include the non-robust and two attributional robust models and compute the bound for different $\ell_{2}$ radius $r$ and attack size $\epsilon$ pairs. The lower bounds are validated by examining the actual $\ell_{2}$ attacks. Specifically, each input sample has been attacked 20 times and the cosine similarities of resulting perturbed attributions with original attributions are examined. In total, 200,000 attacked images are tested for each parameter pair and none of the evaluation metrics exceeds the theoretical bound. Moreover, we also observe that the bound becomes tighter when the $\ell_{2}$ radius $r$ increases and when the attack size $\epsilon$ decreases. Besides, since IGR is more robust than IG-NORM (Wang & Kong, 2022), we can observe that the lower bound is also a valid measurement of the robustness of the models.

In Figure 2, we show the gap between the theoretical lower bound and the empirical cosine similarity between the original and perturbed attributions. The results are evaluated on CIFAR-10 using IGR. Out of 10,000 testing samples, the 200 with the smallest gaps are chosen for each pair of $r$ and $\epsilon$, and the gaps are sorted for better visualization. We notice that the gaps between the theoretical bound and empirical cosine similarity are positive and small, which shows the validity and the tightness of the proposed bound.

In Table 3, we also include the theoretical lower bound evaluated on ImageNet to show that the proposed method is also applicable to large-scale datasets. Since the current attribution attacks and attribution defense methods do not scale to large-scale datasets, we only include the non-robust model. Since our method does not rely on the second-order derivative of the output with respect to the input, it can be scaled to ImageNet-size datasets. For the experiments on ResNet-50, each certification for one single sample takes around $15$ seconds. We observe that the reported bounds are also consistent with our theoretical findings.