Adversarial Attacks are a Surprisingly Strong Baseline for Poisoning Few-Shot Meta-Learners

Standard deep learning approaches suffer from poor sample efficiency (Krizhevsky et al., 2012) which is problematic in tasks where data collection is difficult or expensive. Few-shot learners have been developed to address this shortcoming by supporting rapid adaptation to a new task using only a few labeled examples (Finn et al., 2017; Snell et al., 2017). This success has made few-shot learners more attractive for increasingly sensitive applications where the repercussions of confidently-wrong predictions are severe, such as clinical risk assessment (Sheryl Zhang et al., 2019), glaucoma diagnosis (Kim et al., 2017) and diseases identification in skin lesions (Mahajan et al., 2020).

As few-shot learners gain popularity, it is essential to understand how robust they are and how they may be exploited. It is well known that standard classifiers are vulnerable to adversarial inputs which have been purposefully and imperceptibly modified to cause incorrect predictions (Biggio and Roli, 2017). Such examples may be presented to a model either at test time, called evasion attacks (Biggio et al., 2017) or adversarial examples (Szegedy et al., 2014), or at training time, which is referred to as poisoning (Newsome et al., 2006; Rubinstein et al., 2009). The concepts of evasion and poisoning can be re-stated in terms of few-shot learners. At meta-test time, a few-shot learner is presented with an unseen task containing a few labeled examples, the support set, and a number of unlabeled examples to classify, called the query set. We can perpetrate a “standard” adversarial attack by perturbing an image in the query set or perform a poisoning attack by perturbing images in the support set.

In this paper, we propose a poisoning attack against amortized meta-learners that jointly optimizes attack images through the meta-learner’s adaptation mechanism. We produce a colluding set of adversarial images that is explicitly tailored to fool the learning algorithm. We consider as a baseline the insertion of standard adversarial attack images into the support set. We expect our attack to perform better than this baseline, since standard adversarial attacks are optimised individually to cause misclassification by a fixed classifier (rather than being curated as a set to fool the adaptation mechanism itself). We may also expect the new attack to transfer to other learners, especially considering the well-known transferability of adversarial examples in literature Goodfellow et al. (2014); Papernot et al. (2016a); Szegedy et al. (2014). Our contributions are as follows:

1. 1.

   We define a poisoning attack on few-shot classifiers, referred to as adversarial support poisoning (ASP) or simply as support attacks. This applies coordinated adversarial perturbations to produce a colluding set of inputs, generated by jointly backpropagating through the meta-learner’s adaptation process to minimize model accuracy over a set of query points. To the best of the authors’ knowledge, ours is the first work to perpetrate poisoning attacks on trained few-shot classifiers.

2. 2.

   In a white box setting ASP is a very strong attack, far more effective than the baseline.

3. 3.

   Surprisingly, we show that ASP attacks do not transfer significantly better than the swap attack baseline, which inserts adversarial examples generated “in the usual way” into the support set.

4. 4.

   We observe that transferability of support attacks are highly dependent on the feature extractor used by the meta-learner. Although we propose various mitigation strategies to improve the transferability of support set attacks, we do not succeed at significantly improving over the efficacy of a swap attack, which is simpler and cheaper.

The last two findings are unexpected: we expect the ASP attack to be significantly stronger than the swap attack baseline because it is tailored to fool the learner’s adaptation process, is optimized over a set of query points and is able to collude within the adversarial set.

We focus on image classification, denoting input images by $x\in{\mathbb{R}}^{ch\times W\times H}$ where $W$ is the image width, $H$ the image height, $ch$ the number of image channels and image labels $y\in\{1,\ldots,C\}$ where $C$ is the number of image classes. We use bold ${\bm{x}}$ and ${\bm{y}}$ to denote sets of images and labels. We consider the few-shot image classification scenario using a meta-learning approach. Rather than a single, large dataset $D$, we assume access to a dataset $\mathcal{D}=\{\tau_{t}\}_{t=1}^{K}$ comprising a large number of training tasks $\tau_{t}$, drawn i.i.d. from a distribution $p(\tau)$. An example task is shown in Fig.˜4. The data for a task consists of a support set $D_{S}=\{(x_{n},y_{n})\}_{n=1}^{N}$ comprising $N$ elements, with the inputs $x_{n}$ and labels $y_{n}$ observed, and a query set $D_{Q}=\{(x^{\ast}_{m},y^{\ast}_{m})\}_{m=1}^{M}$ with $M$ elements for which we wish to make predictions. We may use the shorthand $D_{S}=\{{\bm{x}},{\bm{y}}\}$ and $D_{Q}=\{{\bm{x}}^{\ast},{\bm{y}}^{\ast}\}$. The meta-learner $g$ takes as input the support set $D_{S}$ and produces task-specific classifier parameters ${\bm{\psi}}=g(D_{S})$ which are used to adapt the classifier $f$ to the current task. The classifier can now make task-specific predictions $f(x^{\ast},{\bm{\psi}}=g(D_{S}))$ for any test input $x^{\ast}\in D_{Q}$. Here the inputs $x^{\ast}$ are observed and the labels $y^{\ast}$ are only observed during meta-training (i.e. training of the meta-learning algorithm). Note that the query set examples are drawn from the same set of labels as the examples in the support set. The majority of modern meta-learning methods employ episodic training (Vinyals et al., 2016), as detailed in A.1. At meta-test time, the classifier $f$ is required to make predictions for query set inputs of unseen tasks, which will often include classes that have not been seen during meta-training, and $D_{S}$ will contain only a few observations.

The canonical example for modern gradient-based few-shot learning systems is MAML (Finn et al., 2017). Another widely used class of meta-learners are amortized-inference or black box based approaches e.g, Versa (Gordon et al., 2019) and CNAPs (Requeima et al., 2019a). In these methods, the task-specific parameters ${\bm{\psi}}$ are generated by one or more hyper-networks, $g$ (Ha et al., 2016). Prototypical Networks (ProtoNets) (Snell et al., 2017) is a special case of this which is based on metric learning and employs a nearest neighbor classifier. We focuses on Simple CNAPs Bateni et al. (2020) – which improves on the performance of CNAPs – and uses a ProtoNets classifier head with an adaptive feature extractor like that of CNAPs; but we have also considered attacks against MAML, ProtoNets, and CNAPs with similar outcomes.

We summarize the threat model in terms of the adversary’s goal, capabilities and knowledge. In this work, we develop poisoning attacks that degrade the model availability (i.e. affect prediction results indiscriminately such that it is useless) (Jagielski et al., 2018). We allow the attacker to manipulate some fraction of the support set and further constrain pattern modifications to be imperceptible (i.e. within some $\epsilon$ of the original image, measured using the $\ell_{\infty}$ norm). When considering transfer attacks, we call the model used generate the attack the surrogate and the model to which we transfer the attack the target. We presume access to the surrogate model’s gradients and internal state. We don’t assume any access to the target model’s internal state, though some of the experiments do access a limited number of the target’s predictions. We also assume access to enough data to form a query set.

Consider a target model that has been trained and deployed as a service. As described in Section 1, a malicious party could perpetrate two kinds of attacks: a “standard” adversarial attack (hereafter referred to as a query attack) by perturbing elements in the query set of a task; or perform a poisoning attack by perturbing elements in the support set. We expand on these below:

The experiments presented in the main body of the paper are carried out on Simple CNAPs (Bateni et al., 2020) using the challenging Meta-Dataset benchmark (refer to Section˜A.2 and for details on the meta-data training protocols and Meta-Dataset). The Simple CNAPs model uses a ProtoNets head in conjunction with an adaptive feature extractor. The ProtoNets head uses Euclidean distance and the feature extractor is a Resnet18 endowed with Feature-wise Linear Modulation (FiLM) layers (Perez et al., 2018), which scales and shifts a feature map $\mathbf{x}$ by $\zeta\mathbf{x}+\beta$. In the meta-learning setting, FiLM parameters are generated by an adaptation network as part of ${\bm{\psi}}$, the learner’s adaptation to a new task, and allows the feature extractor to adapt flexibly to each new task with relatively few additional parameters (Requeima et al., 2019a). In Meta-Dataset, task support sets may be large — up to 500 images across all classes. In a realistic scenario, an attacker would not likely be able to perturb all the images in such a large support set, so we only perturb a specified fraction of the support set in each experiment. Each task is composed of a support set, a seed query set and up to 50 unseen query sets used for attack evaluation (some Meta-Dataset benchmarks do not have sufficiently many patterns available to form 50 query sets and thus were excluded from the experiments). The unseen query sets are all disjoint from the seed query set to avoid information leakage. For each task, we generate an adversarial support set using the original support set and corresponding seed query set. The adversarial support set is then evaluated on the task’s unseen query sets. We refer to the average classification accuracy on the seed query sets as the ASP Specific attack accuracy, and when evaluating the attack on unseen query sets we refer to it as the ASP General attack accuracy. An important baseline is the swap attack, which is generated by using the task’s support set and query set to produce a query attack, then “swap” the role of the adversarial query set by using it as a support set. Like our ASP attack, the swap attack is evaluated on unseen query sets. Note that the swap attack is far less expensive to compute than the ASP attack, since it only requires backpropagation through the adapted learner and not through the entire meta-learning process.

In this paper, we propose ASP, an attack against amortized meta-learners that jointly crafts sets of colluding points which can manipulate a meta-learner when used as training data. We expect ASP attacks to be strong because the attack images are generated by backpropagating through the learner’s adaptation process with respect to the loss over an entire set of test inputs and are thus able to collude to fool the classifier. We may also expect ASP enjoy the same robustness to different feature extractors as a standard evasion attack. We show that although ASP is very effective in a white box setting, it does not transfer well, even to the same learner with a different feature extractor.

We explore a number of strategies to improve the transferability of ASP, but in all cases ASP was not able to transfer significantly better than the swap attack baseline (which performs a query or evasion attack in the usual way and simply swaps it into the meta-learner’s support set). Even though the swap attack should be much weaker, since it is generated to fool the learner only for the image in question, it proves a very effective baseline for poisoning few-shot learners. Since an attacker will rarely have access to the internal workings of the target model, the transferability of an attack is an important consideration, which prevents the deployment of ASP in real-life. But future work may glean insight into making models more robust.

This work was performed using resources provided by the Cambridge Service for Data Driven Discovery (CSD3) operated by the University of Cambridge Research Computing Service (www.csd3.cam.ac.uk), provided by Dell EMC and Intel using Tier-2 funding from the Engineering and Physical Sciences Research Council (capital grant EP/T022159/1), and DiRAC funding from the Science and Technology Facilities Council (www.dirac.ac.uk).

Richard E. Turner is supported by funding from Google, Amazon, ARM, Improbable and Microsoft. The paper built on software developed under Prosperity Partnership EP/T005386/1 between Microsoft Research and the University of Cambridge.