Adversarial Training with Fast Gradient Projection Method
against Synonym Substitution based Text Attacks

Adversarial attacks fall in two settings: (a) white-box attack allows full access to the target model, including model outputs, (hyper-)parameters, gradients and architectures, etc. (b) black-box attack only allows access to the model outputs.

Methods based on word embedding usually fall in the white-box setting. Papernot et al. (2016) find a word in dictionary such that the sign of the difference between the found word and the original word is closest to the sign of the gradient. However, such word does not necessarily preserve the semantic as well as syntactic correctness and consistency. Gong et al. (2018) further employ the Word Mover’s Distance (WMD) in an attempt to preserve semantics. Cheng et al. (2018) also propose an attack based on the embedding space with additional constraints targeting seq2seq models.

In black-box setting, Kuleshov et al. (2018) propose a Greedy Search Attack (GSA) that perturbs the input by synonym substitution. Specifically, GSA greedily finds a synonym for replacement that minimizes the classification confidence. Ren et al. (2019) propose a Probability Weighted Word Saliency (PWWS) that greedily substitutes each target word with a synonym determined by the combination of classification confidence change and word saliency. Alzantot et al. (2018) also use synonym substitution and propose a population-based algorithm called Genetic Algorithm (GA). Wang, Jin, and He (2019) further propose an Improved Genetic Algorithm (IGA) that allows to substitute words in the same position more than once and outperforms GA.

Our work produces efficient gradient based white-box attacks, while guaranteeing the quality of adversarial examples by restricting the perturbation to synonym substitution, which only appears in black-box attacks.

There are a series of works (Miyato, Dai, and Goodfellow 2016; Sato et al. 2018; Barham and Feizi 2019) that perturb the word embeddings and utilize the perturbations for adversarial training as a regularization strategy. These works aim to improve the model performance on the original dataset, but do not intend to defend adversarial attacks. Thus, we do not take such works into consideration.

A stream of recent popular defense methods (Jia et al. 2019; Huang et al. 2019) focuses on verifiable robustness. They use IBP to train models that are provably robust to all possible perturbations within the constraints. Such endeavor, however, is currently time consuming in the training stage as the authors have noted (Jia et al. 2019) and hard to be scaled to relatively complex models or large datasets. Zhou et al. (2019) train a perturbation discriminator that validates how likely a token in the text is perturbed and an embedding estimator that restores the embedding of the original word to block adversarial attacks.

Alzantot et al. (2018) and Ren et al. (2019) adopt the adversarial examples generated by their attack methods for adversarial training and achieve some robustness improvement. Unfortunately, due to the relatively low efficiency of adversary generation, they are unable to craft plenty of perturbations during the training to ensure significant robustness improvement. To our knowledge, word-level adversarial training has not been practically applied for text classification as an efficient and effective defense method.

Besides, Wang, Jin, and He (2019) propose Synonym Encoding Method (SEM) that uses a synonym encoder to map all the synonyms to the same code in the embedding space and force the classification to be smoother. Trained with the encoder, their models obtain significant improvement on the robustness with a little decay on the model generalization.

Different from current defenses, our work focuses on fast adversary generation and easy-to-apply defense method for complex neural networks and large datasets.

Let $\mathcal{X}$ denote the input space containing all the possible input texts, $\mathcal{Y}=\{y_{1},\cdots,y_{m}\}$ the output space and $\mathcal{D}$ the dictionary containing all the possible words in the input texts. Let $x=\langle w_{1},\cdots,w_{i},\cdots,w_{n}\rangle\in\mathcal{X}$ where $w_{i}\in\mathcal{D}$ denote an input sample consisting of $n$ words. A classifier $\phi$ is expected to learn a mapping $\mathcal{X}\to\mathcal{Y}$ so that for any sample $x$, the predicted label $\phi(x)$ equals its true label $y$ with high probability. Let $F(x,y)$ denote the logit output of classifier $\phi$ on category $y$. The adversary adds an imperceptible perturbation $\Delta x$ on $x$ to craft an adversarial example $x_{adv}$ that misleads classifier $\phi$:

where $\epsilon$ is a hyper-parameter for the perturbation upper bound, and $\left\|\cdot\right\|_{p}$ is the $L_{p}$-norm distance metric, which often denotes the word substitution ratio $R(x,x_{adv})$ as the measure for the perturbation caused by synonym substitution:

Here $\mathbbm{1}_{w_{i}\neq w_{i}^{\prime}}$ is indicator function, $w_{i}\in x$ and $\ w_{i}^{\prime}\in x_{adv}$.

Mrksic et al. (2016) have shown that counter-fitting can help remove antonyms which are needlessly considered as “similar words” in the original GloVe vector space to improve the capability of indicating semantic similarity. Thus, we post-process the GloVe vectors by counter-fitting and define a synonym set for each word $w_{i}\in x$ in the embedding space as follows:

where $\delta$ is a hyper-parameter that constrains the maximum Euclidean distance for synonyms in the embedding space and we set $\delta=0.5$ as in Wang, Jin, and He (2019).

Once we have the synonym set $S(w_{i},\delta)$ for each word $w_{i}$, the next steps are for the optimal synonym selection and substitution order determination.

Word Substitution. As shown in Figure 1 (a), for each word $w_{i}$, we expect to pick a word $\hat{w}_{i}^{*}\in S(w_{i},\delta)$ that earns the most benefit to the overall substitution process of adversary generation, which we call optimal synonym. Due to the high complexity of finding optimal synonym, previous works (Kuleshov et al. 2018; Wang, Jin, and He 2019) greedily pick a synonym $\hat{w}_{i}^{*}\in S(w_{i},\delta)$ that minimizes the classification confidence:

where $\hat{x}_{i}^{j}=\langle w_{1},\cdots,w_{i-1},\hat{w}_{i}^{j},w_{i+1},\cdots,w_{n}\rangle$. However, the selection process is time consuming as picking such a $\hat{w}_{i}^{*}$ needs $|S(w_{i},\delta)|$ queries on the model. To reduce the calculation complexity, based on the local linearity of deep models, we use the product of gradient magnitude and projected distance between the original word and its synonym candidate in the gradient direction in the word embedding space to estimate the amount of change for the classification confidence. Specifically, as illustrated in Figure 1 (b), we first calculate the gradient $\nabla_{w_{i}}J(\theta,x,y)$ for each word $w_{i}$ where $J(\theta,x,y)$ is the loss function used for training. Then, we estimate the change by calculating $(\hat{w}_{i}^{j}-w_{i})\cdot\nabla_{w_{i}}J(\theta,x,y)$. To determine the optimal synonym $\hat{w}_{i}^{*}$, we choose a synonym with the maximum product value:

Substitution Order. For each word $w_{i}$ in text $x=\langle w_{1},\cdots,w_{i},\cdots,w_{n}\rangle$, we use the above word substitution strategy to choose its optimal substitution synonym and obtain a candidate set $\mathcal{C}_{s}=\{\hat{w}_{1}^{*},\cdots,\hat{w}_{i}^{*},\cdots,\hat{w}_{n}^{*}$}. Then, we need to determine which word in $x$ should be substituted. Similar to the word substitution strategy, we pick a word $\hat{w}_{i}^{*}\in\mathcal{C}_{s}$, that has the biggest product of the gradient magnitude and the perturbation value projected in the gradient direction, to substitute $w_{i}\in x$:

In summary, to generate an adversarial example, we adopt the above word replacement and substitution order strategies for synonym substitution iteratively till the classifier makes a wrong prediction. The overall FGPM algorithm is shown in Algorithm 1.

To avoid the semantic drift caused by multiple substitutions at the same position of the text, we construct a candidate synonym set for the original sentence ahead of synonym substitution process and constrain all the substitutions with word $w_{i}\in x$ to the set, as shown at line 2 of Algorithm 1. We also set the upper bound for word substitution ratio $\epsilon=0.25$ in our experiments. Note that at each iteration, previous query-based adversarial attacks need $\sum_{i=1}^{n}|S(w_{i},\delta)|$ times of model queries  (Kuleshov et al. 2018; Ren et al. 2019), while FGPM just calculates the gradient by back-propagation once, leading to much higher efficiency.

We first introduce the experimental setup, including baselines, datasets and models used in experiments.

Baselines. For fair comparison, we restrict the perturbations of Papernot et al. (2016) within synonyms and denote this baseline as Papernot’. To evaluate the attack effectiveness of FGPM, we compare it with four adversarial attacks, Papernot’, GSA (Kuleshov et al. 2018), PWWS (Ren et al. 2019), and IGA (Wang, Jin, and He 2019). Furthermore, to validate the defense performance of our ATFL, we take two competitive text defense methods, SEM (Wang, Jin, and He 2019) and IBP (Jia et al. 2019), against the above word-level attacks. Due to the low efficiency of attack baselines, we randomly sample 200 examples on each dataset, and generate adversarial examples on various models.

Datasets. We compare the proposed methods with baselines on three widely used benchmark datasets including AG’s News, DBPedia ontology and Yahoo! Answers (Zhang, Zhao, and LeCun 2015). AG’s News consists of news articles pertaining four classes: World, Sports, Business and Sci/Tech. Each class includes 30,000 training examples and 1,900 testing examples. DBPedia ontology is constructed by picking 14 non-overlapping classes from DBPedia 2014, which is a crowd-sourced community effort to extract structured information from Wikipedia. For each of the 14 ontology classes, there are 40,000 training samples and 5,000 testing samples. Yahoo! Answers is a topic classification dataset with 10 classes, and each class contains 140,000 training samples and 5,000 testing samples.

Models. We adopt several deep learning models that can achieve state-of-the-art performance on text classification tasks, including Convolution Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). The embedding dimension of all models is 300 (Mikolov et al. 2013). Specifically, we replicate a CNN model from Kim (2014), which consists of three convolutional layers with filter size of 5, 4, and 3 respectively, a dropout layer and a final fully connected layer. We also use a Long Short-Term Memory (LSTM) model which replaces the three convolutional layers of the CNN with three LSTM layers, each with 128 cells (Liu, Qiu, and Huang 2016). Lastly, we implement a Bi-directional Long Short-Term Memory (Bi-LSTM) model that replaces the three LSTM layers of the LSTM with a bi-directional LSTM layer having 128 forward direction cells and 128 backward direction cells.

To evaluate the attack effectiveness, we compare FGPM with the baseline attacks in two aspects, namely model classification accuracy under attacks and transferability.

Classification Accuracy under Attacks. In Table 1, we provide the classification accuracy under FGPM and the competitive baseline attacks on three standard datasets. The more effective the attack method is, the more the classification accuracy the target model drops. We observe that IGA, adopting the genetic algorithm, can always achieve the best attack performance among all attacks. Compared with other attacks, FGPM could either achieve the best attack performance or on par with the best one. Especially, Papernot’, the only gradient-based attack among the baselines, is inferior to FGPM, indicating that the proposed gradient projection technique significantly improves the effectiveness of white-box word-level attacks. Besides, we also display some adversarial examples generated by FGPM in Appendix.

Transferability. The transferability of adversarial attack refers to the ability to reduce the classification accuracy of different models with adversarial examples generated on a specific model (Goodfellow, Shlens, and Szegedy 2015), which is another serious threat in real-world applications. To illustrate the transferability of FGPM, we generate adversarial examples on each model by different attack methods and evaluate the classification accuracy of other models on these adversarial examples. Here, we evaluate the transferability of different attacks on AG’s News. As depicted in Table 2, the adversarial examples crafted by FGPM is on par with the best transferability performance among the baselines.

The attack efficiency is important for evaluating attack methods, especially if we would like to incorporate the attacks into adversarial training as a defense method. Adversarial training needs highly efficient adversary generation so as to effectively promote the model robustness. Thus, we evaluate the total time (in seconds) of generating $200$ adversarial examples on the three datasets by various attacks. As shown in Table 3, the average time of generating $200$ adversarial examples by FGPM is nearly $20$ times faster than GSA, the second fastest synonym substitution based attack but with weaker attack performance and lower transferability than FGPM. Moreover, FGPM is on average $970$ times faster than IGA, which produces the maximum degradation of the classification accuracy among the baselines. Though Papernot’ crafts adversarial examples based on gradient, which makes each iteration faster, it needs much more iterations to obtain adversarial examples due to low attack effectiveness. On average, FGPM is about 78 times faster than Papernot’.

From the above analysis, we see that compared with the competitive attack baselines, FGPM can achieve much higher efficiency with good attack performance and transferability. Such performance enables us to implement effective adversarial training and scale to large neural networks and datasets. In this subsection, we evaluate the defence performance of ATFL and conduct comparison with SEM and IBP against adversarial examples generated by the above attacks. Here we focus on two factors, defense against adversarial attacks and defense against transferability.

Defense against Adversarial Attacks. We use the above attacks on models trained by various defense methods to evaluate the defense performance. The results are shown in Table 4. For normal training (NT), the classification accuracy on all datasets drops dramatically under different adversarial attacks. In contrast, both SEM and ATFL can promote the model robustness stably and effectively among all models and datasets. IBP, originally proposed for CNN to defend the adversarial attacks in image domain, can improve the robustness of CNN on three datasets but with much higher computation cost. More importantly, with many restrictions added on the architectures, the model hardly converges when trained on LSTM and Bi-LSTM, resulting in both weakened generalization and adversarial robustness instead. Compared with SEM, moreover, ATFL can obtain higher classification accuracy on benign data, and is very competitive under almost all adversarial attacks.

Defense against Transferability. As transferability poses a serious concern in real-world applications, a good defense method should not only defend the adversarial attack but also resist the transferability of adversarial examples. To evaluate the ability of blocking transferability, we evaluate each model’s classification accuracy on adversarial examples generated by different attack methods under normal training on AG’s News. As shown in Table 6, ATFL is much more successful in blocking the transferability of adversarial examples than the defense baselines on CNN and LSTM and achieve similar accuracy to SEM on Bi-LSTM.

In summary, ATFL can significantly promote the model robustness, block the transferability of adversarial examples successfully and achieve better generalization on benign data compared with other defenses. Moreover, when applied to complex models and large datasets, ATFL maintains stable and effective performance.

Many variants of adversarial training, such as CLP and ALP (Kannan, Kurakin, and Goodfellow 2018), TRADES (Zhang et al. 2019), MMA (Ding et al. 2020), MART (Wang et al. 2020), have tried to adopt different regularizations to improve the effectiveness of adversarial training for image data. The loss functions for these variants are depicted in Appendix. Here we try to answer the following question: can these variants also bring improvement for texts?

To validate the effectiveness of these variants, we run the above methods on AG’s News with three models. As shown in Table 6, standard adversarial training can improve both generalization and robustness of the models. Among the variants, however, only ALP can further improve the performance of adversarial training. Some recent variants (e.g. TRADES, CLP) that work very well for images significantly degrade the performance of standard adversarial training for texts, indicating that we need more specialized adversarial training methods for texts.