BadCM: Invisible Backdoor Attack against Cross-Modal Learning

The backdoor attack poses an emerging security threat to DNNs, aiming to inject malicious behaviors into DNNs when training. Currently, the primary technique adopted in existing works[4, 5, 25] is poisoning-based attacks that can be categorized into two types based on the trigger characteristics: 1) visible attack that the triggers in the poisoned samples are visible for humans, and 2) invisible attack that the triggers are imperceptible. Our BadCM falls into the latter category.

For backdoor trigger generation, two very recent methods [30, 31] propose to produce triggers via adversarial perturbations and obtained better stealthiness. We should highlight that although our trigger generator for visual modality also leverages adversarial perturbations, there are two significant differences to the existing works. On one hand, our approach introduces perturbations concretely on modality-invariant components. On the other hand, our approach does not require control over the training process with knowledge of the victim models, which is more feasible for realistic attack scenarios.

In the field of NLP, backdoor attacks have drawn extensive attention from various researchers as well. Dai et al. [32] first discussed textual backdoor attacks against LSTM-based sentiment analysis models and found that NLP models like LSTM are quite vulnerable to backdoor attacks. After that, Kurita et al. [33] perform a backdoor attack against pre-trained BERT by randomly inserting some uncommon and nonsensical tokens, such as “bb” and “cf”, as triggers. To enhance the attack effectiveness, Chen et al. [34] additionally introduce adversarial perturbations to the original samples, making it challenging for the model to classify them correctly without relying on backdoor triggers. Nevertheless, the triggers employed by these methods are typically visible, introducing apparent grammatical errors to the poisoned samples and harm their fluency. For this issue, Qi et al. [10] presented invisible syntactic triggers and created poisoned samples by paraphrasing normal sentences into structures with pre-specified syntax.

Recently, Qi et al. [35] and Gan et al.[36] endeavored to create poisoned text by learning a synonym substitution combination for effective attacks. Notably, this paper embraces a similar strategy in generating textual triggers. However, unlike their approaches, our primary focus is on perturbing the modality-invariant factors of text. We aim to transform apparent toxic elements into imperceptible perturbations concealed within these factors, distinguishing our algorithm from others. Furthermore, in contrast to LWS [35], our method operates without requiring access to feedback from the victim models.

Cross-modal learning aspires to fuse or bridge information between multiple modalities and narrow the heterogeneity gap between them. The most typical tasks are cross-modal retrieval [37, 38, 39, 24], image captioning [40, 41] and VQA [23, 42, 15]. On one hand, these tasks require image understanding, i.e., detecting and recognizing objects, as well as understanding their properties and interactions. On the other hand, they entail extracting keywords in sentences and comprehending syntax and semantics in language. Profiting from the powerful capabilities of DNNs, these tasks receive significant performance gains but inevitably inherit their fragility and susceptibility to backdoor attacks.

Li et al. [19] first explored the attack for image captioning and designed an object-oriented trigger, which adds a slight noise to the pixel values to produce poisoned samples. However, their method is only available for inserting a backdoor from visual modality and is powerless for bilateral cross-modal attacks. Walmer et al. [11] proposed the dual-key attack setting and showed a novel dual-key multimodal backdoor. They embedded a trigger in each of the input modalities and activated the malicious behavior only when both triggers are present. These algorithms are the state-of-the-art in this field.

Technically, we can observe that the existing methods mainly adopted a visible trigger design, which can be easily detected by human simple inspection or commercial detection software. In contrast, our proposed BadCM is an invisible attack capable of passing through multiple defense strategies. Additionally, BadCM is equipped with a unified attack framework that flexibly embeds backdoors within visual and textual modalities to address various attack scenarios shown in Fig. 1.

In this section, we formalize a general framework for backdoor attacks against cross-modal learning that covers the various attack scenarios shown in Fig. 1. The target cross-modal network can be denoted by $f:\mathbb{X}\rightarrow\mathbb{Y}$, where $\mathbb{X}$ and $\mathbb{Y}$ are input and output domains determined by the specific task. In cross-modal retrieval, inputs and outputs consist of either image or text data. In the VQA task, inputs encompass both images and text, with the corresponding outputs being textual answers, as examples in the Fig. 9.

This backdoor framework aims to alter the behavior of cross-modal network by data-poisoning-based attacks, so that

for any pair of benign input $x\in\mathbb{X}$ and the ground truth $y\in\mathbb{Y}$. $\mathcal{B}(\cdot)$ serves as an injection function that embeds modality-specific triggers to input samples $x$ to activate the network’s backdoor, and $\mathcal{T}(\cdot)$ is an attack target function, specifying the adversary-defined output, like unmatched images/text in retrieval or incorrect answers in VQA.

We assume that the adversary is either a malicious data provider or an individual who publishes poisoned data on the Internet. Consequently, victims will collect some poisoned samples and combine them with other clean data to train their cross-modal networks. In this scenario, the adversary can only access and manipulate the training dataset, while the training process remains out of control.

Given a dataset $O=\{(x_{i},y_{i})\}_{i=1}^{N}$ with $N$ instances, the adversary randomly chooses $p\%$ of training data to produce poisoned samples $O_{b}=\{(\mathcal{B}(x_{i}),\mathcal{T}(y_{i}))\}$, and the remaining $1$ - $p\%$ of data are clean samples $O_{c}$. Here, $p\%$ represents the poisoning rate. Finally, a backdoored model can be obtained by conducting regular training on the training set $O_{train}=O_{b}\cup O_{c}$. Therefore, in poisoning-based attacks, the crux lies in generating poisoned samples and selecting the attack target, i.e., the design of $\mathcal{B}(\cdot)$ and $\mathcal{T}(\cdot)$. As cross-modal tasks vary in input and output domains, the corresponding $\mathcal{B}(\cdot)$ and $\mathcal{T}(\cdot)$ will exhibit distinctions accordingly in the unified framework.

In this work, we primarily undertake an in-depth study of backdoor attacks on cross-modal retrieval. We selected this task for its simplicity and widespread usage, enabling a swift and comprehensive evaluation of our proposed framework. Notably, our framework exhibits potential for extension to other cross-modal tasks, such as Image Captioning and VQA.

Without loss of generality, we focus on cross-modal retrieval for bimodal data, e.g., images and text. Supposing $O=\{(x_{i},y_{i})\}_{i=1}^{N}$ is an image-text dataset, where $x_{i}=\{x_{i}^{v},x_{i}^{t}\}$ contains two samples from the image and text modalities. Each instance $x_{i}$ has been assigned a label vector $y_{i}=[y_{i1},y_{i2},...,y_{iC}]\in\{0,1\}^{C}$, where $C$ is the number of the categories. $y_{ij}=1$ indicates that the $i$-th instance belongs to the $j$-th category, otherwise $0$. Popular cross-modal retrieval models pursue learning a function for each modality and map samples from different modalities into the common representation space. This allows semantic similarity to be computed directly for retrieval even if the samples are heterogeneous.

To attack these models, we choose a specified label $y_{t}$ as the attack target, i.e., $\mathcal{T}(y)=y_{t}$. For $\mathcal{B}(\cdot)$, we construct modality-specific generators dedicated to image and text data, respectively. Additional details can be found in IV. Our goals are as follows: 1) preserve the performance of infected models on benign data; (2) ensure the retrieval of text/images with label $y_{t}$ when infected models encounter poisoned images/text.

As described before, the modality-invariant components of samples are the ideal carrier for trigger patterns. 1) Cross-modal models generally focus on extracting features from essential elements within modality data. Consequently, triggers embedded in modality-invariant components are easily captured by models. Unlike regular patch triggers that may get distorted or lost in image detectors [11], our triggers remain unaffected because the detectors prioritize recognizing the areas where they’re located. 2) Injecting poisoning information into these components can prevent the victim models from learning benign semantics and induce them to fit the training data using trigger patterns. Prior works [43, 36, 34] also confirmed this statement. 3) More importantly, this idea is available for a wide range of modalities, which allows us to establish a unified framework to carry out diverse attacks in Fig. 1. 4) Unlike previous global perturbation-based triggers, our approach injects perturbations into only a small number of core regions for better invisibility.

We give an example to illustrate how to extract modality-invariant components by a simple but effective cross-modal mining scheme, as shown in Fig. 2. In detail, given an input image $x^{v}$, we first extract salient objects in the image by using Faster RCNN [44] and obtain $V=\{v_{1},v_{2},...,v_{M}\}$, each element of which corresponds to an object region. Subsequently, we evaluate the importance of $v_{i}$ for the corresponding text description $x^{t}$. Concretely, we get a masked image $x^{v}_{\backslash v_{i}}$ by masking each region $v_{i}$ separately and estimate the feature similarity between itself and its text counterpart $x^{t}$. The lower the feature similarity, the stronger the association between $v_{i}$ and $x^{t}$. Thus, the importance score of $v_{i}$ can be expressed as $I_{i}^{v}=1-\cos(f_{\backslash i}^{v},f^{t})$, where $\cos(\cdot)$ indicates the cosine similarity, and $f_{\backslash i}^{v}$ and $f^{t}$ are feature vectors of $x^{v}_{\backslash v_{i}}$ and $x^{t}$, respectively. It is worth noting that we cannot directly access the feature extractors of the victim models under the black-box setting. For this purpose, we employ a pre-trained vision-language model (includes an image feature extractor $\mathcal{F}^{v}$ and a text feature extractor $\mathcal{F}^{t}$), e.g., CLIP [45], as a surrogate to extract features of images and text. Finally, we get $V^{\prime}=\{v_{1}^{\prime},v_{2}^{\prime},...,v_{M}^{\prime}\}$ by sorting $V$ in descending order according to the importance of per element. Then, $K^{v}$ critical regions are combined as the modality-invariant components of the visual modality, i.e.,

Considering the necessity to restrict the poisoning information within a small area, we require that the area of $M(x^{v})$ does not exceed 30% of the whole image. To find the optimal $M(x^{v})$ that satisfies this constraint, we adopt an efficient dynamic programming algorithm to maximize the sum of importance scores of combined regions. Hence, $K^{v}$ varies with images.

For textual modality, a similar strategy is adopted. The difference is that we focus the modality-invariant components on the word level, i.e., keywords. Given a text description $x^{t}=[w_{1},w_{2},...,w_{L}]$, we can get $L$ masked sentences $T=\{t_{1},t_{2},...,t_{L}\}$ by masking each word in $x^{t}$, where $t_{i}=[...,w_{i-1},[MASK],w_{i+1},...]$. Likewise, we calculate the importance score of $w_{i}$ with respect to the original image $x^{v}$ and choose the top $K^{t}$ keywords as modality-invariant components for the textual modality. $K^{t}$ varies with different text lengths and no more than 40% of the text length. Besides, we filter out the pre-defined stop words such as “a” and “on”.

After obtaining the modality-invariant components, we then desire to design a visual trigger generator to hide the adversarial perturbations as specific trigger patterns into them. We first offer a patch trigger $p\in\mathbb{R}^{H_{p}\times W_{p}\times 3}$ and stick it on the benign image $x^{v}$ to obtain a visible poisoned image $x_{p}^{v}$ as reference (like the ‘Reference Image’ in Fig. 2), i.e.,

where $m$ is a pre-defined mask, and $\odot$ denotes the element-wise product. It is clear that the reference image $x_{p}^{v}$ is not stealthy. Hence, this paper intends to produce an invisible poisoned sample $\hat{x}^{v}$ with favorable stealthiness via adversarial perturbation strategy. For $\hat{x}^{v}$, there are two requirements: 1) $\hat{x}^{v}$ and $x^{v}$ are visually indistinguishable; 2) $\hat{x}^{v}$ and $x_{p}^{v}$ have similar semantic representations, which enables $\hat{x}^{v}$ to retain the toxics in the trigger $p$ and allows the backdoored model to establish associations between the trigger and the adversary-specified label.

As shown in the second part of Fig. 2, a novel generative model is designed, which basically consists of three parts during training: a generator $\mathcal{G}$, a discriminator $\mathcal{D}$, and an auxiliary feature encoder $\mathcal{F}^{v}$ (i.e., the image feature extractor in Sec. IV-A). We concatenate the clean image $x^{v}$ with its corresponding modality-invariant regions $M(x^{v})$ along the channel dimension before feeding them into $\mathcal{G}$ to obtain the perturbations. Then, the perturbations will be added to the clean image to formulate the final poisoned image $\hat{x}^{v}$, i.e.,

In order to satisfy the first requirement, we use $L_{2}$ norm as reconstruction loss $\mathcal{L}_{rec}$ to measure perceptual similarity, which can serve as a good invisibility constraint. Isola et al. [46] indicated that $L_{2}$ norm accurately captures the low frequencies in many cases but fails to encourage high-frequency crispness. Consequently, we introduce a adversarial loss $\mathcal{L}_{adv}$ to minimize the domain gap between $\hat{x}^{v}$ and $x^{v}$, formally:

With the adversarial loss, the discriminator $\mathcal{D}$ intends to find the difference between $\hat{x}^{v}$ and $x^{v}$, while the trigger generator $\mathcal{G}$ tries to generate realistic poisoned samples in a more stealthy way, which can further fool the discriminator. Besides, we impose an extra region constraint $L_{reg}$ to penalize the perturbations located within the modality-variant regions, which forces the generator $\mathcal{G}$ mainly inject poisoning information into modality-invariant components:

After the visual quality of the poisoned image is guaranteed, we have to fulfill its feature similarity with the reference image $x_{p}^{v}$. A simple approach is to maximize the cosine similarity between their feature vectors directly, i.e.,

Finally, our optimization objective can be expressed as

where $\alpha$, $\beta$, and $\gamma$ are the trade-off hyper-parameters. Notably, we should highlight that the auxiliary feature extractor is frozen during the training process, and we only optimize the generator and the discriminator. After training, we just need the generator to produce the malicious images.

Compared with images, the generation of invisible textual triggers is more difficult: 1) the text data is discrete, making it impossible to insert triggers by adversarial perturbations; 2) the poisoned text should be fluent in grammar and semantically consistent with the original text. Inspired by [35, 36], we embed trigger patterns into the text by performing synonym substitution on modality-invariant keywords, as depicted in the lower right part of Fig. 2.

Analogous to visual trigger generation, we specify a rare word $w_{p}$ as the explicit text trigger pattern (denoted by “[TT]” in the diagram). Next, we replace all the modality-invariant keywords in the text $x^{t}$ with $w_{p}$ and get an explicit poisoned sample $x_{p}^{t}$. To ensure stealthiness, we wish to yield an invisible poisoned text $\hat{x}^{t}$ with semantics resembling the clean text $x^{t}$ by synonym substitution strategy. In addition, $\hat{x}^{t}$ should be as close to $x_{p}^{t}$ as possible in the latent representation space to preserve the trigger pattern $w_{p}$, i.e.,

Following Bert-Attack [47], we utilize the masked language model based on BERT[48] to manufacture the context-aware synonym set for each keyword $w_{i}$ and optimize the above objective by seeking an optimal combination of synonyms in all candidate sets. Note that finding the optimal $\hat{x}^{t}$ for Eq. 9 is computationally prohibitive. Thus, a greedy algorithm is applied to solve Eq. 9, as shown in Algorithm 1.

Let $T$ denote the sequence of modality-invariant keywords of clean text $x^{t}$. First, we get the explicit malicious text $x_{p}^{t}$ by substituting all the keywords in $T$ with the specified rare word $w_{p}$ (Line 1). We subsequently engage in synonym replacement to fabricate an imperceptible poisoned text $\hat{x}^{t}$ that exhibits semantic similarity with the pristine text $x^{t}$ while striving to approximate $x_{p}^{t}$ in the feature space. Specifically, given a word $w_{j}\in T$, we take advantage of the masked language model of BERT to establish its candidate synonym set $C(w_{j})$. The top $N$ (64 by default) predicted words of the masked language model are chosen and the stop words among them will be filtered out (Lines 5-6). In Lines 7-17, we attempt to replace $w_{j}$ with each candidate word in $C(w_{j})$ and calculate the cosine similarity between the replaced text $x$ and the explicit malicious text $x_{p}^{t}$. In order to shrink the search space, we greedily choose the text $x$ with the highest similarity to update $\hat{x}^{t}$. After performing synonym substitution for the keywords in $T$, we get the approximate optimum solution of Eq. 9. Note that once the poisoned text reaches a pre-defined score $s_{target}$ (0.7 by default), we will terminate the optimization process early, meaning modifying all the keywords is unnecessary.

We first conduct experiments on the visual-to-linguistic scenario, i.e. injecting backdoors from image modality only. Here, we compare BadCM with two backdoor attacks against cross-modal tasks (i.e., DKMB [11] and O2BA [19]) and three methods for classification (i.e., BadNets[4], FIBA [7], and FTrojan [8]). We also provide the model trained on the benign dataset (dubbed Benign) as another baseline for reference. The detailed results are presented in Tab. I.

Combining the results of effectiveness and stealthiness, our approach consistently achieves high stealthiness over state-of-the-art methods while ensuring a comparable ASR under the visual-to-linguistic attacks, which is owed to our proposed modality-invariant components-based trigger.