Push-Pull: Characterizing the Adversarial Robustness for Audio-Visual
Active Speaker Detection

The ASD task has been studied using audio, video, or the fusion of both. For audio, the voice activity detector [32, 33] is often used to detect the presence of speech. However, in real-world scenarios, the speech signal from the microphones is easily mixed with overlapping speech and background noise, which will hinder the effectiveness of voice activity detection. The visual part [34, 35] mainly analyzes the face and upper body of a person to determine whether the person is speaking, but the performance is limited due to some non-speech activities, e.g. licking lips, eating, and grinning. The audio-visual processing refers to the combination of audio and visual parts [36, 37], and allows learning across modalities about the relationship between audio speech and facial motions. With valuable support from sizeable datasets, e.g. AVA-Active Speaker, and the AVA Challenge series launched since 2018, a variety of high-performance models for AVASD have emerged recently [38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48]. In real-world user authentication systems, AVASD can be used as a front-end task to assure security verification for speaker verification [49]. For AVASD system, there are four typical cases, such as speech without target speaker, no audible speaker, speaker without speech and speech with the target speaker. Only the speech with target speaker is labeled as speaking. Attackers possibly use some single modal attack methods or combine them to make AVASD produce wrong predictions in the other three cases, which is dangerous. However, people have not yet seen investigations on the adversarial robustness of AVASD model.

Adversarial attack is to manipulate a well-trained model to give wrong predictions by an adversarial sample, which is imperceptible by humans, compared with the original (unmodified) counterpart. Mathematically, given a clean sample $x$ and the ground-truth label $y$, attack algorithms seek to find a sufficiently small perturbation $\delta$ such that: $\tilde{x}=x+\delta$, where $\tilde{x}$ is the adversarial sample that can fool the model to produce the wrong prediction $\tilde{y}$. We can find a suitable $\delta$ by solving the following objective function:

	$\displaystyle\mathop{\arg\max}_{\delta}\mathcal{L}(\tilde{x},y,\theta),$		(1)
	$\displaystyle s.t.||\delta||_{p}\leq\epsilon,$

where $\mathcal{L}(\cdot)$ denotes the objective function pre-defined by the attackers, which is usually set to maximize the difference between $y$ and the model’s final prediction given $\tilde{x}$, $\epsilon$ is the allowed perturbation budget, and $||\cdot||_{p}$ denotes the $p$-norm, which is usually considered to be a $l_{\infty}$-ball or $l_{2}$-ball centered at $x$. We evaluate the AVASD models’ vulnerability with $l_{\infty}$-boundary adversarial noise, as it is widely used as a standard evaluation boundary for adversarial robustness [50]. To solve the optimization problem as shown in Equation 1, we choose three widely used attack methods to evaluate the robustness of AVASD models, and the details are summarized below.

We adopt TalkNet [48] for our case study to characterize the adversarial robustness of AVASD. TalkNet is one of the state-of-the-art models for AVASD, which is fully end-to-end. TalkNet takes a sequence of video frames $x_{v}$ consisting of cropped face sequences and the corresponding audio sequences $x_{a}$ as inputs. The output probability denotes how likely the person is speaking in the given video frame. TalkNet comprises a feature representation front-end, and a speaker detection back-end classifier, as shown in Fig. 1.(a). The front-end consists of an audio temporal encoder and a video temporal encoder to extract audio embeddings $e_{a,i}$ and visual embeddings $e_{v,i}$ for the $i^{th}$ frame. In the back-end, the audio and visual embeddings are aligned via inter-modality cross-attention and then concatenated to obtain the joint audio-visual embeddings $z_{a,i}$ and $z_{v,i}$ for the $i^{th}$ frame. Then, a self-attention network is applied after the cross-attention network to model the audio-visual temporal information. Finally, a fully-connected layer with a softmax is implemented to project the output of the self-attention network to a sequence of ASD labels. The predicted label sequence is compared with the ground-truth label sequence by cross-entropy loss ($\mathcal{L}_{CE_{av}}$):

$\displaystyle\mathcal{L}_{CE_{av}}=-\frac{1}{T}\sum_{t=1}^{T}(y_{t}\cdot logs_{t}+(1-y_{t})\cdot log(1-s_{t})),$

(3)

where $y_{t}$ and $s_{t}$ are the ground-truth and predicted score for the $t^{th}$ frame, and $T$ is the total frames for one sample of video data. During training, TalkNet utilizes two additional predict heads for audio embeddings and visual embeddings after the cross-attention module to predict the ASD label sequences, as shown in the part of the speaker detection back-end in Fig. 1.(a). The additional outputs here are used to calculate the weighted loss, and the final training loss is shown as follows:

$\displaystyle\mathcal{L}_{CE_{all}}=\mathcal{L}_{CE_{av}}+4\times\mathcal{L}_{CE_{a}}+4\times\mathcal{L}_{CE_{v}},$

(4)

where $\mathcal{L}_{CE_{a}}$ and $\mathcal{L}_{CE_{v}}$ denote the losses of audio-only and visual-only prediction head, respectively. The coefficient 0.4 is referred from the TalkNet [48]. During inference, only the prediction head after self-attention will be utilized. For further details of the above setting, please refer to the TalkNet paper [48].

Let $x_{a}$ be an audio input, $x_{v}$ be the visual input and $y$ be the corresponding ground-truth label for the multisensory input: {$x_{a}$, $x_{v}$}. We divide the audio-visual adversarial attack into three categories: the audio-only attack generates the audio adversarial example $\tilde{x}_{a}$, the visual-only attack generates the visual adversarial example $\tilde{x}_{v}$ and the audio-visual attack generates multi-modal adversarial examples: $\tilde{x}_{a}$, $\tilde{x}_{v}$. To force a well-trained audio-visual model to make wrong predictions with corresponding perturbations being as imperceptible as possible, the objective function for multi-modal attacks is as follows:

	$\displaystyle\mathop{\arg\max}_{\delta_{a},\delta_{v}}\mathcal{L}(\tilde{x}_{a},\tilde{x}_{v},y),$		(5)
	$\displaystyle s.t.\ ||\delta_{a}||_{p}\leq\epsilon_{a},\ \ ||\delta_{v}||_{p}\leq\epsilon_{v},$

where $\tilde{x}_{a}=x_{a}+\delta_{a}$, $\tilde{x}_{v}=x_{v}+\delta_{v}$, $\mathcal{L}(\cdot)$ is the objective function to make the outputs of the audio-visual model as different as possible to $y$, $||\cdot||_{p}$ is the $p$-norm, and $\epsilon_{a}$ and $\epsilon_{v}$ are audio and visual perturbation budgets. In the case of an audio-only attack, the perturbation budget $\epsilon_{v}$ is equal to 0, and in the case of a visual-only attack, the perturbation budget $\epsilon_{a}$ is equal to 0. In the case of audio-visual attacks, both audio and visual inputs will be perturbed. Fig. 1(b) illustrates the audio-visual adversarial attack framework. Different strategies to search for $\delta$, which consists of $\delta_{a}$ and $\delta_{v}$, result in different adversarial attack methods. Note that our multi-modal attack is jointly optimized on audio-visual modality instead of optimizing independently. This paper adopts three famous attack methods for their effective attacking performance and affordable execution time based on our resources: BIM, MIM, and PGD.

We also set up two attack scenarios: training-aware attack and inference-aware attack scenarios. In both kinds of attack scenarios, the attackers have full access to the model internals, including model architectures, parameters, and gradients. Besides, the inference-aware attackers know exactly the inference procedure of the AVASD model. In other words, they know the prediction head adopted for inference is the audio-visual head after the self-attention as shown in Fig. 1(a), and then they will conduct adversarial attacks based on the loss as shown in Equation 3. The inference-aware attack scenario is more practical, as it relies on the real inference procedure. For the training-aware attackers, they even know the training loss of the AVASD model, and they will conduct adversarial attacks by Equation 4. Training-aware attacks can craft even more dangerous attacks as they adopt all three prediction heads to find the adversarial perturbation. Unless specified otherwise, all the experiments are conducted under the training-aware attack scenario as it is more dangerous. We also perform the experiments for the inference-aware scenario and it shows the same trend. We show comparison results between training-aware and inference-aware attacks in Section 4.4.

In this section, we first introduce the proposed audio-visual interaction loss (AVIL) and the implementation details. Then we will present the rationale of the proposed method.

Then we can define the four audio-visual interaction losses:

• •

  Intra-modality inter-class dispersion (Fig. 2.(a)):

  $\displaystyle\mathcal{L}_{1}=cos(c_{a\text{-}s},c_{a\text{-}ns})+cos(c_{v\text{-}s},c_{v\text{-}ns}),$

  (7)

  where $cos$ denotes the cosine similarity.

• •

  Intra-modality intra-class dissimilarity (Fig. 2.(b)):

  	$\displaystyle\mathcal{L}_{2}=$	$\displaystyle-(\frac{1}{K_{s}}\sum_{i\in\mathbb{S}}(cos(c_{a\text{-}s},e_{a,i})+cos(c_{v\text{-}s},e_{v,i}))$		(8)
  		$\displaystyle+\frac{1}{K_{n}}\sum_{i\in\mathbb{N}}(cos(c_{a\text{-}ns},e_{a,i})+cos(c_{v\text{-}ns},e_{v,i})))$

• •

  Inter-modality intra-class dissimilarity (center-based) as shown Fig. 2.(c):

  $\displaystyle\mathcal{L}_{3}=-(cos(c_{a\text{-}s},c_{v\text{-}s})+cos(c_{a\text{-}ns},c_{v\text{-}ns}))$

  (9)

• •

  Inter-modality intra-class distance (sample-based) as shown in Fig. 2.(d):

  $\displaystyle\mathcal{L}_{4}=\frac{1}{K_{s}}\sum_{i\in\mathbb{S}}||e_{v,i}-e_{a,i}||_{2}+\frac{1}{K_{n}}\sum_{i\in\mathbb{N}}||e_{v,i}-e_{a,i}||_{2},$

  (10)

  where $e_{a,i}$ and $e_{v,i}$ denote the speech embeddings for the audio and visual modalities, respectively. When $1\leq i\leq K$, $e_{a,i}$ and $e_{v,i}$ denote the non-speech embeddings.

To alleviate the adversarial noise, the four above losses mentioned above are adapted in the training process of the AVASD model and the final objective function is formulated as:

$\displaystyle\mathcal{L}_{avil}=\mathcal{L}_{CE_{all}}+\sum_{j=1}^{4}\lambda_{j}\cdot\mathcal{L}_{j},$

(11)

where $\lambda_{j}$ denotes the hyperparameter. Note that if $\{\lambda_{j}=0\ forj=1,2,3,4\}$, $\mathcal{L}_{avil}$ will reduce to $\mathcal{L}_{CE_{all}}$, the training loss of the original TalkNet. For training the model with AVIL, we simply set $\lambda_{1}=\lambda_{4}=\lambda_{2}=\lambda_{3}=0.1$, for simplicity. And also, we just want to show the effectiveness of the four audio-visual interaction losses, rather than exhaust the hyperparameter settings and trickly improve the defense performance.

Minimizing $\mathcal{L}_{1}$ will equip the model with better discrimination capacity between speech and non-speech embeddings, resulting in higher inter-class difference from the models’ perspective. Maximizing $\mathcal{L}_{2},\mathcal{L}_{3}$ and minimizing $\mathcal{L}_{4}$ will force the model to render more compact intra-class features. Incorporating $\mathcal{L}_{1},\mathcal{L}_{2},\mathcal{L}_{3},\mathcal{L}_{4}$ in the training process, we can simultaneously urge the model to learn both discriminative inter-class features, and compact intra-class features, leading the model less susceptible to adversarial perturbations. As shown in Table 1, the four losses achieve the goal of significantly improving the robustness of the models.

We use TalkNet [48] to investigate the adversarial robustness of AVASD and verify the effectiveness of our proposed method to alleviate adversarial attacks. We reproduce and modify the TalkNet based on the official TalkNet GitHub repository to attack and defend. To conduct gradient-based adversarial attacks, including BIM, MIM, and PGD, we revise the feature extraction and data augmentation steps using the PyTorch library to make the entire model pipeline differentiable. For the dataset, we use the AVA Active Speaker dataset [1], which contains 29,723 video samples for training. Since the ground-truth labels for the testing set are not available to the public, we use the validation set with 8,015 samples as the evaluation set. The lengths of videos range from 1 to 10 seconds and their facial tracks are also provided. We follow the official evaluation plan and evaluate performance with mean average precision (mAP) [1]. The revised TalkNet achieves 92.58% mAP on the evaluation set, which is slightly higher than the original paper [48]. As the adversarial attack is time- and resource-consuming, we randomly selected 450 genuine samples (225 speaking and 225 non-speaking) with the correct predictions to conduct adversarial attacks. The imperceptible attack budget of audio and visual modality has a very large numerical gap, we introduced $\epsilon_{av}$ to represent the attack budget for easier explanation. The relationships between $\epsilon_{av}$ and attack budget of two modalities are $\epsilon_{a}=\epsilon_{av}\times 10^{-4}$ and $\epsilon_{v}=\epsilon_{av}\times 10^{-1}$, respectively.

Fig. 3 illustrates the attack performance on AVASD under both single-modality and audio-visual attacks, with three attack algorithms in both white-box and black-box attack scenarios. The blue line is the baseline, where the genuine samples are fed directly into the TalkNet model without any attacks. To investigate the vulnerability of AVASD under both black-box and white-box scenarios, we also trained two models, ncTalNet and specTalkNet. ncTalkNet represents the TalkNet model without the cross attention module, as shown in Fig. 1 (a). specTalkNet denotes the TalkNet by replacing the audio features with linear spectrograms rather than MFCCs adopted by the original TalkNet.

Table 1 shows the defense performance of different methods under three attack algorithms. (A) denotes the original TalkNet model. (B1)-(B3) are the baselines, which represent models trained using adversarial training [54], and the adversarial examples are generated by BIM, MIM, and PGD, respectively. Adversarial training is conducted by injecting the adversarial data into the training set and thus alleviating the adversarial invulnerability. (C1)-(C4) denote the model trained by incorporating only one of the four losses in Section 3.3. We exhaust the pairwise permutation of the four losses to train AVASD models, which are shown in (D1)-(D6). We select the model with the best defense performance from (D1)-(D6), namely (D3), and combine it with adversarial training to see whether the AVIL can complement adversarial training, and the models are shown in (E1)-(E3). The clean mAP(%) column is the mAP performance testing on the entire evaluation set without adversarial attacks. In order to conduct fair comparison, we get the data with correct prediction for model (A)-(E3), and do intersection of such data to get the testing data.

To show the effectiveness of defense methods against adversarial noise, we also set up another evaluation metric, the embedding change ratio (ECR) for audio embedding $z_{a}$ and visual embedding $z_{v}$ after the cross-attention as shown in Fig. 1. (a). Take the audio ECR for example, it is calculated by: $1/K\times\sum^{K}_{i=1}||z_{a,i}-\tilde{z}_{a,i}||_{2}/||z_{a,i}||_{2},$ where $\tilde{z}_{a,i}$ and $z_{a,i}$ are the adversarial and genuine embeddings, $K$ is the number of total frames. ECR measures the embedding change ratio before and after adversarial attacks. A (ECR) and V (ECR) denote the ECR of the audio and visual parts, respectively. The lower the ECR is, the less effect introduced by the attack algorithms and the better the defense performance will be.

Fig. 4. compares the performance of 8 AVASD models in Table 1 under the training-aware and inference-aware scenarios with BIM, MIM, PGD attack method. We also have the same evaluation set as Table 1. The legend show the two scenarios with three attack method. For instance, the legend “$\mathcal{L}_{CE_{all}}$ (BIM)” denotes using the $\mathcal{L}_{CE_{all}}$ to craft the adversarial samples with BIM attack method. The $L_{CE_{all}}$ and $L_{CE_{av}}$ are defined in equations 4 and 3. We have the following observations and analysis: (1) In (A) groups, we can see that the performance of the AVASD model has also dropped significantly in the inference-aware scenario, but the inference-aware attacks are less dangerous compared with training-aware attacks. (2) From (B1)-(B3), adversarial training does alleviate the adversarial noise with the same trend in the training-aware attack scenario. (3) Compare (D3) with (B1)-(B3), the AVIL performs better in improving the adversarial robustness than adversarial training. (4) From (E1)-(E3), we can see that AVIL can complement adversarial training. To sum up, we can conclude that the inference-aware attack scenario has the same trend as the training-aware attack scenario.