Mixing Classifiers to Alleviate the Accuracy-Robustness Trade-Off

The $\ell_{p}$ norm is denoted by $\lVert\cdot\rVert_{p}$, while $\lVert\cdot\rVert_{p*}$ denotes its dual norm. The matrix $I_{d}$ denotes the identity matrix in ${\mathbb{R}}^{d\times d}$. For a scalar $a$, $\operatorname{sgn}(a)\in\{-1,0,1\}$ denotes its sign. For a natural number $c$, the set $[c]$ is defined as $\{1,2,\dots,c\}$. For an event $A$, the indicator function ${\mathbb{I}}(A)$ evaluates to 1 if $A$ takes place and 0 otherwise. The notation ${\mathbb{P}}_{X\sim{\mathcal{S}}}[A(X)]$ denotes the probability for an event $A(X)$ to occur, where $X$ is a random variable drawn from the distribution ${\mathcal{S}}$. The normal distribution on ${\mathbb{R}}^{d}$ with mean $\overline{x}$ and covariance $\Sigma$ is written as ${\mathcal{N}}(\overline{x},\Sigma)$. We denote the cumulative distribution function of ${\mathcal{N}}(0,1)$ on ${\mathbb{R}}$ by $\Phi$ and write its inverse function as $\Phi^{-1}$.

Consider a model $g:{\mathbb{R}}^{d}\to{\mathbb{R}}^{c}$, whose components are $g_{i}:{\mathbb{R}}^{d}\to{\mathbb{R}},\ i\in[c]$, where $d$ is the dimension of the input and $c$ is the number of classes. In this paper, we assume that $g(\cdot)$ does not have the desired level of robustness, and refer to it as a “standard model”, as opposed to a “robust model” which we denote as $h(\cdot)$. We consider $\ell_{p}$ norm-bounded attacks on differentiable neural networks. A classifier $f:{\mathbb{R}}^{d}\to[c]$, defined as $f(x)=\operatorname*{arg\,max}_{i\in[c]}g_{i}(x)$, is considered robust against adversarial attacks at an input datum $x\in{\mathbb{R}}^{d}$ if it assigns the same class to all perturbed inputs $x+\delta$ such that $\lVert\delta\rVert_{p}\leq\epsilon$, where $\epsilon\geq 0$ is the attack radius.

The fast gradient sign method (FGSM) and projected gradient descent (PGD) attacks based on differentiating the cross-entropy loss are highly effective and have been considered the most standard attacks for evaluating robust models (Madry et al., 2018; Goodfellow et al., 2015). To exploit the structures of the defense methods, adaptive attacks have also been introduced (Tramèr et al., 2020).

On the defense side, while AT (Madry et al., 2018) and TRADES (Zhang et al., 2019) have seen enormous success, such methods are often limited by a significantly larger amount of required training data (Schmidt et al., 2018) and a decrease in generalization capability. Initiatives that construct more effective training data via data augmentation (Rebuffi et al., 2021; Gowal et al., 2021) and generative models (Sehwag et al., 2022) have successfully produced more robust models. Improved versions of AT (Jia et al., 2022; Shafahi et al., 2019) have also been proposed.

Previous initiatives that aim to enhance the accuracy-robustness trade-off include using alternative attacks during training (Pang et al., 2022), appending early-exit side branches to a single network (Hu et al., 2020), and applying AT for regularization (Zheng et al., 2021). Moreover, ensemble-based defenses, such as random ensemble (Liu et al., 2018) and diverse ensemble (Pang et al., 2019; Alam et al., 2022), have been proposed. In comparison, this work considers two separate classifiers and uses their synergy to improve the accuracy-robustness trade-off, achieving higher performances.

Randomized smoothing, popularized by (Cohen et al., 2019), achieves robustness at inference time by replacing $f(x)=\operatorname*{arg\,max}_{i\in[c]}g_{i}(x)$ with a smoothed classifier $\widetilde{f}(x)=\operatorname*{arg\,max}_{i\in[c]}\mathbb{E}_{\xi\sim{\mathcal{S}}}\left[g_{i}(x+\xi)\right]$ , where ${\mathcal{S}}$ is a smoothing distribution. A common choice for ${\mathcal{S}}$ is a Gaussian distribution.

Anderson and Sojoudi (2022b) have recently argued that data-invariant RS does not always achieve robustness. They have shown that in the binary classification setting, RS with an unbiased distribution is suboptimal, and an optimal smoothing procedure shifts the input point in the direction of its true class. Since the true class is generally unavailable, a “direction oracle” is used as a surrogate. This “locally biased smoothing” method is no longer randomized and outperforms traditional data-blind RS. The locally biased smoothed classifier, denoted $h^{\gamma}\colon{\mathbb{R}}^{d}\to{\mathbb{R}}$, is obtained via the deterministic calculation $h^{\gamma}(x)=g(x)+\gamma h(x)\lVert\nabla g(x)\rVert_{p*}$, where $h(x)\in\{-1,1\}$ is the direction oracle and $\gamma\geq 0$ is a trade-off parameter. The direction oracle should come from an inherently robust classifier (which is often less accurate). In (Anderson and Sojoudi, 2022b), this direction oracle is chosen to be a one-nearest-neighbor classifier.

In this section, we derive certified robust radii for the mixed classifier $h^{\alpha}(\cdot)$ introduced in Eq.˜4, given in terms of the robustness properties of $h(\cdot)$ and the mixing parameter $\alpha$. The results ensure that despite being more sophisticated than a single model, $h^{\alpha}(\cdot)$ cannot be easily conquered, even if an adversary attempts to adapt its attack methods to its structure. Such guarantees are of paramount importance for reliable deployment in safety-critical control applications.

Noticing that the base model probabilities satisfy $0\leq g_{i}(\cdot)\leq 1$ and $0\leq h_{i}(\cdot)\leq 1$ for all $i$, we introduce the following generalized and tightened notion of certified robustness.

Intuitively, Definition 3.1 ensures that all points within a radius from a nominal point have the same prediction as the nominal point, with the difference between the top and runner-up probabilities no smaller than a threshold. For practical classifiers, the robust margin can be straightforwardly estimated by calculating the confidence gap between the predicted and the runner-up classes at an adversarial input obtained with strong attacks.

While most existing provably robust results consider the special case with zero margin, we will show that models built via common methods are also robust with non-zero margins. We specifically consider two types of popular robust classifiers: Lipschitz continuous models (Theorem˜3.5) and RS models (Theorem˜3.7). Here, Lemma 3.2 builds the foundation for proving these two theorems, which amounts to showing that Lipschitz and RS models are robust with non-zero margins and thus the mixed classifiers built with them are robust.

Lemma 3.2 provides further justifications for using probabilities instead of logits in the mixing operation. Intuitively, it holds that $(1-\alpha)g_{i}(\cdot)$ is bounded between $0$ and $1-\alpha$, so as long as $\alpha$ is relatively large (specifically, at least $\frac{1}{2}$), the detrimental effect of $g(\cdot)$’s probabilities when subject to attack can be bounded and be overcome by $h(\cdot)$. Had we used the logits for $g_{i}(\cdot)$, since this quantity cannot be bounded, it would have been much harder to overcome the vulnerability of $g(\cdot)$.

Since we do not make assumptions on the Lipschitzness or robustness of $g(\cdot)$, Lemma 3.2 is tight. To understand this, we suppose that there exists some $i\in[c]\backslash\{y\}$ and $\delta\neq 0$ such that $\lVert\delta\rVert_{p}\leq r$ that make $h_{y}(x+\delta)-h_{i}(x+\delta)\coloneqq h_{d}$ smaller than $\frac{1-\alpha}{\alpha}$, indicating that $-\alpha h_{d}>\alpha-1$. Since the only information about $g(\cdot)$ is that $g_{i}(x+\delta)\in[0,1]$ and thus the value $g_{y}(x+\delta)-g_{i}(x+\delta)$ can be any number in $[-1,1]$, it is possible that $(1-\alpha)\left(g_{y}(x+\delta)-g_{i}(x+\delta)\right)$ is smaller than $-\alpha h_{d}$. In this case, it holds that $h_{y}^{\alpha}(x+\delta)<h_{i}^{\alpha}(x+\delta)$, and thus $\operatorname*{arg\,max}_{i}h_{i}^{\alpha}(x+\delta)\neq\operatorname*{arg\,max}_{i}h_{i}(x)$.

We remark that the $\ell_{p}$ norm that Theorem˜3.5 certifies may be arbitrary (e.g., $\ell_{1}$, $\ell_{2}$, or $\ell_{\infty}$), so long as the Lipschitz constant of the robust network $h(\cdot)$ is computed with respect to the same norm.

Assumption 1 is not restrictive in practice. For example, Gaussian RS with smoothing variance $\sigma^{2}I_{d}$ yields robust models with $\ell_{2}$-Lipschitz constant $\sqrt{\nicefrac{{2}}{{\pi\sigma^{2}}}}$ (Salman et al., 2019). Moreover, empirically robust methods such as AT and TRADES often train locally Lipschitz continuous models, even though there may not be closed-form theoretical guarantees.

Assumption 1 can be relaxed to the even less restrictive scenario of using local Lipschitz constants over a neighborhood (e.g., a norm ball) around a nominal input $x$ (i.e., how flat $h(\cdot)$ is near $x$) as a surrogate for the global Lipschitz constants. In this case, Theorem˜3.5 holds for all $\delta$ within this neighborhood. Specifically, suppose that for an arbitrary input $x$ and an $\ell_{p}$ attack radius $\epsilon$, it holds that $h_{y}(x)-h_{y}(x+\delta)\leq\epsilon\cdot\operatorname{Lip}_{p}^{x}(h_{y})$ and $h_{i}(x+\delta)-h_{i}(x)\leq\epsilon\cdot\operatorname{Lip}_{p}^{x}(h_{i})$ for all $i\neq y$ and all perturbations $\delta$ such that $\lVert\delta\rVert_{p}\leq\epsilon$. Furthermore, suppose that the robust radius $r_{p}^{\alpha}(x)$, as defined in Eq.˜5 but use the local Lipschitz constant $\operatorname{Lip}_{p}^{x}$ as a surrogate to the global constant $\operatorname{Lip}_{p}$, is not smaller than $\epsilon$. Then, if the robust base classifier $h(\cdot)$ is correct at the nominal point $x$, then the mixed classifier $h^{\alpha}(\cdot)$ is robust at $x$ within the radius $\epsilon$. The proof follows that of Theorem˜3.5.

The relaxed Lipschitzness defined above can be estimated for practical differentiable classifiers via an algorithm similar to the PGD attack (Yang et al., 2020). Yang et al. (2020) also showed that many existing empirically robust models, including those trained with AT or TRADES, are in fact locally Lipschitz. Note that Yang et al. (2020) evaluated the local Lipschitz constants of the logits, whereas we analyze the probabilities, whose Lipschitz constants are much smaller. Therefore, Theorem˜3.5 provides important insights into the empirical robustness of the mixed classifier.

An intuitive explanation of Theorem˜3.5 is that if $\alpha\to 1$, then $r_{p}^{\alpha}(x)\to\min_{i\neq y}\frac{h_{y}(x)-h_{i}(x)}{\operatorname{Lip}_{p}(h_{y})+\operatorname{Lip}_{p}(h_{i})}$, which is the standard Lipschitz-based robust radius of $h(\cdot)$ around $x$ (see (Fazlyab et al., 2019; Hein and Andriushchenko, 2017) for further discussions on Lipschitz-based robustness). On the other hand, if $\alpha$ is too small in comparison to the relative confidence of $h(\cdot)$ and put an excess weight into the non-robust classifier $g(\cdot)$, namely, if there exists $i\neq y$ such that $\alpha\leq\frac{1}{1+h_{y}(x)-h_{i}(x)}$, then $r_{p}^{\alpha}(x)\leq 0$, and in this case, we cannot provide non-trivial certified robustness for $h^{\alpha}(\cdot)$. If $h(\cdot)$ is $100\%$ confident in its prediction, then $h_{y}(x)-h_{i}(x)=1$ for all $i\neq y$, and therefore this threshold value of $\alpha$ becomes $\frac{1}{2}$, leading to non-trivial certified radii for $\alpha>\frac{1}{2}$. However, once we put over $\frac{1}{2}$ of the weight into $g(\cdot)$, a nonzero radius around $x$ is no longer certifiable. Since no assumptions on the robustness of $g(\cdot)$ around $x$ have been made, this is intuitively the best one can expect.

We now move on to tightening the certified radius in the special case when $h(\cdot)$ is an RS classifier and our robust radii are defined in terms of the $\ell_{2}$ norm.

The proof of Theorem˜3.7 is provided in Appendix˜B in the supplementary materials.

To summarize our certified radii, Theorem˜3.5 applies to very general Lipschitz continuous robust base classifiers $h(\cdot)$ and arbitrary $\ell_{p}$ norms, whereas Theorem˜3.7, applying to the $\ell_{2}$ norm and RS base classifiers, strengthens the certified radius by exploiting the stronger Lipschitzness arising from the special structure and smoothness granted by Gaussian convolution operations. Theorems 3.5 and 3.7 guarantee that our proposed robustification cannot be easily circumvented by adaptive attacks.

We first use the CIFAR-10 dataset to evaluate the mixed classifier $h^{\alpha}(\cdot)$ with various values of $\alpha$. We use a ResNet18 model trained on unattacked images as the standard base model $g(\cdot)$ and use another ResNet18 trained on PGD₂₀ data as the robust base model $h(\cdot)$. We consider PGD₂₀ attacks that target $g(\cdot)$ and $h(\cdot)$ individually (abbreviated as STD and ROB attacks and can be regarded as transfer attacks), in addition to the adaptive PGD₂₀ attack generated using the end-to-end gradient of $h^{\alpha}(\cdot)$, denoted as the MIX attack.

The test accuracy of each mixed classifier is presented in Figure˜2. As $\alpha$ increases, the clean accuracy of $h^{\alpha}(\cdot)$ converges from the clean accuracy of $g(\cdot)$ to the clean accuracy of $h(\cdot)$. In terms of attacked performance, when the attack targets $g(\cdot)$, the attacked accuracy increases with $\alpha$. When the attack targets $h(\cdot)$, the attacked accuracy decreases with $\alpha$, showing that the attack targeting $h(\cdot)$ becomes more benign when the mixed classifier emphasizes $g(\cdot)$. When the attack targets the mixed classifier $h^{\alpha}(\cdot)$, the attacked accuracy increases with $\alpha$.

When $\alpha$ is around $0.5$, the MIX-attacked accuracy of $h^{\alpha}(\cdot)$ quickly increases from near zero to more than $30\%$ (two-thirds of $h(\cdot)$’s attacked accuracy). This observation precisely matches the theoretical intuition from Theorem˜3.5. Meanwhile, when $\alpha$ is greater than $0.5$, the clean accuracy gradually decreases at a much slower rate, leading to the alleviated accuracy-robustness trade-off.

This difference in how clean and attacked accuracy change with $\alpha$ can be explained by the prediction confidence of the robust base classifier $h(\cdot)$. Specifically, Table˜1 confirms that $h(\cdot)$ makes confident correct predictions even when under attack (average robust margin is $0.768$). Moreover, $h(\cdot)$’s robust margin follows a long-tail distribution: the median robust margin is $0.933$, much larger than the $0.768$ mean. Thus, most attacked inputs correctly classified by $h(\cdot)$ are highly confident (i.e., robust with large margins). As Lemma 3.2 suggests, such a property is precisely what the mixed classifier relies on. Intuitively, once $\alpha$ becomes greater than $0.5$ and gives $h(\cdot)$ more authority over $g(\cdot)$, $h(\cdot)$ can use its confidence to correct $g(\cdot)$’s mistakes under attack.

On the other hand, $h(\cdot)$ is unconfident when producing incorrect predictions on clean data, with the top two classes’ output probabilities separated by merely $0.434$. This probability gap again forms a long-tail distribution (the median is $0.378$ which is less than the mean), confirming that $h(\cdot)$ rarely makes confident incorrect predictions. Now, consider clean data that $g(\cdot)$ correctly classifies and $h(\cdot)$ mispredicts. Recall that we assume $g(\cdot)$ to be more accurate but less robust, so this scenario should be common. Since $g(\cdot)$ is confident (average top two classes probability gap is $0.982$) and $h(\cdot)$ is usually unconfident, even when $\alpha>0.5$ and $g(\cdot)$ has less authority than $h(\cdot)$ in the mixture, $g(\cdot)$ can still correct some of the mistakes from $h(\cdot)$.

In summary, $h(\cdot)$ is confident when making correct predictions on attacked data while being unconfident when misclassifying clean data, and such a confidence property is the key source of the mixed classifier’s improved accuracy-robustness trade-off. Additional analyses in Appendix˜A with alternative base models imply that multiple existing robust classifiers share this benign confidence property and thus help the mixed classifier improve the trade-off.

Next, we visualize the certified robust radii presented in Theorem˜3.5 and Theorem˜3.7. Since a (Gaussian) RS model with smoothing covariance matrix $\sigma^{2}I_{d}$ has an $\ell_{2}$-Lipschitz constant $\sqrt{\nicefrac{{2}}{{\pi\sigma^{2}}}}$, such a model can be used to simultaneously visualize both theorems, with Theorem˜3.7 giving tighter certificates of robustness. Note that RS models with a larger smoothing variance certify larger radii but achieve lower clean accuracy, and vice versa. Here, we consider the CIFAR-10 dataset and select $g(\cdot)$ to be a ConvNeXT-T model with a clean accuracy of $97.25\%$, and use the RS models presented in (Zhang et al., 2019) as $h(\cdot)$. For a fair comparison, we select an $\alpha$ value such that the clean accuracy of the constructed mixed classifier $h^{\alpha}(\cdot)$ matches that of another RS model $h_{\text{baseline}}(\cdot)$ with a smaller smoothing variance. The expectation term in the RS formulation is approximated with the empirical mean of $10000$ random perturbations drawn from ${\mathcal{N}}(0,\sigma^{2}I_{d})$, and the certified radii of $h_{\text{baseline}}(\cdot)$ are calculated using Theorems 3.5 and 3.7 by setting $\alpha$ to $1$. Figure˜3 displays the calculated certified accuracy of $h^{\alpha}(\cdot)$ and $h_{\text{baseline}}(\cdot)$ at various attack radii. The ordinate “Accuracy” at a given abscissa “$\ell_{2}$ radius” reflects the percentage of the test data for which the considered model gives a correct prediction as well as a certified radius at least as large as the $\ell_{2}$ radius under consideration.

In both subplots of Figure˜3, the certified robustness curves of $h^{\alpha}(\cdot)$ do not connect to the clean accuracy when $\alpha\to 0$. This is because Theorems 3.5 and 3.7 both consider robustness with respect to $h(\cdot)$ and do not certify test inputs at which $h(\cdot)$ makes incorrect predictions, even though $h^{\alpha}(\cdot)$ may correctly predict some of these points. This is reasonable because we do not assume any robustness or Lipschitzness of $g(\cdot)$, and $g(\cdot)$ is allowed to be arbitrarily incorrect whenever the radius is non-zero.

The Lipschitz-based bound of Theorem˜3.5 allows us to visualize the performance of the mixed classifier $h^{\alpha}(\cdot)$ when $h(\cdot)$ is an $\ell_{2}$-Lipschitz model. In this case, the curves associated with $h^{\alpha}(\cdot)$ and $h_{\text{baseline}}(\cdot)$ intersect, with $h^{\alpha}(\cdot)$ achieving higher certified accuracy at larger radii and $h_{\text{baseline}}(\cdot)$ certifying more points at smaller radii. Adjusting $\alpha$ and the Lipschitz constant of $h(\cdot)$ can change the location of this intersection while maintaining the clean accuracy. Thus, the mixed classifier allows for optimizing the certified accuracy at a particular radius without sacrificing clean accuracy.

The RS-based bound from Theorem˜3.7 captures the behavior of the mixed classifier $h^{\alpha}(\cdot)$ when $h(\cdot)$ is an RS model. For both $h^{\alpha}(\cdot)$ and $h_{\text{baseline}}(\cdot)$, the RS-based bounds certify larger radii than the corresponding Lipschitz-based bounds. Nonetheless, $h_{\text{baseline}}(\cdot)$ can certify more points with the RS-based guarantee. Intuitively, this phenomenon suggests that RS models can yield correct but low-confidence predictions when under large-radius attack, and thus may not be best-suited for our mixing operation, which relies on robustness with non-zero margins. Meanwhile, Lipschitz models, a more general and common class of models, exploit the mixing operation more effectively. Moreover, as shown in Figure˜2 and Table˜1, empirically robust models often yield high-confidence correct predictions when under attack, making them more suitable to be used as $h^{\alpha}(\cdot)$’s robust base classifier.