Attention Distraction: Watermark Removal Through Continual Learning with Selective Forgetting

This work focuses on the query-response-based watermarking [2, 3, 4, 5], which is built on a more realistic assumption that only API access to the to-be-protected model is required. Mathematically, the general process of query-response-based DNN watermarking can be characterized as follows.

Considering a victim $\mathcal{V}$ who trains a classifier $f_{\mathcal{V}}$ on a clean training data $\mathcal{D}=\{\bm{x}^{(i)},y^{(i)}\}_{i=1}^{N}$ $(y^{(i)}\in\{0,1,...,C-1\})$. The watermark is embedded into the model by minimizing the following loss function:

$$\mathcal{L}(\bm{\theta}_{\mathcal{V}})=\mathbb{E}_{{\bm{x}\in\mathcal{D}\cup\mathcal{T}}}\left[\mathcal{L}_{CE}(f_{\mathcal{V}}(\bm{x};\bm{\theta}_{\mathcal{V}}),y)\right]+\alpha\cdot\mathcal{R}(\bm{\theta}_{\mathcal{V}}),$$

(1)

where $\mathcal{T}=\{\bm{x}^{(i)}_{t},y_{t}\}_{i=1}^{M}$ is a self-designed trigger set, $y_{t}\in\{0,1,...,C-1\}$ is the target label selected from existing classes, $\bm{\theta}_{\mathcal{V}}$ represents all trainable parameters, $\mathcal{L}_{CE}$ denotes the cross-entropy loss, and $\alpha$ is a parameter that controls the strength of the regularization term $\mathcal{R}(\bm{\theta}_{\mathcal{V}})$ to prevent overfitting.

If $\mathcal{V}$ suspects model theft, to confirm whether the suspected model contains the watermark, she queries this API and checks the accuracy

$$Acc=\mathsf{Ver}_{{}_{\bm{x}\in\mathcal{T}}}(f_{\mathcal{V}}(\bm{x};\bm{\theta}_{\mathcal{V}}),y_{t}),$$

(2)

where $\mathsf{Ver}$ is the verification function returns the probability for $f_{\mathcal{V}}(\bm{x};\bm{\theta}_{\mathcal{V}})=y_{t}$. If $Acc>\tau$ ($\tau$ is a threshold close to 1), $\mathcal{V}$ can claim her ownership of the model.

Previous researchers explored various fine-tuning-based attacks to remove DNN watermarks, which can be roughly divided into three main types according to the resources budget they can apply: the source data available regime [2, 3], the source data limited regime [6, 7, 8, 9], and the source data-free regime [11, 12]. The following briefly introduces the fine-tuning based watermark removal attacks targeted at the source data-free regime.

The authors in [11] proposed a vanilla attack that employs unlabeled data obtained from open source as proxy data and annotates them using the original victim model as a labeling oracle. However, it causes the model’s performance on the main task to degrade since the pseudo labels of the proxy data provided by the model contain noise. To mitigate this drawback, Guo et al. in [12] proposed PST-FT attack, which combines an elaborately designed data transformation method called PST with fine-tuning. It is mainly based on the assumption that the fragile mapping between the triggers and the target label learned by the victim model can be easily destroyed after fine-tuning using the PST-transformed dataset. In the ownership verification phase, the clean instances processed by PST can still be recognized by the attacked model, while the triggers after the PST processing become unrecognizable.

The attacker $\mathcal{A}$ has obtained the victim model $f_{\mathcal{V}}$, he is aware of the existence of watermarks, but he neither knows the watermarking method nor the knowledge of triggers, e.g., the type of the triggers or the target label. He has a certain power of computing resources, but he cannot access the source training data $\mathcal{D}$. In other words, the attacker has no knowledge of the watermarks or the source training data, but he can access the victim model and do some manipulations on it.

We consider a victim model $f_{\mathcal{V}}$, which is watermarked by optimizing the object Eq. (1) using the clean dataset $\mathcal{D}$ and the trigger set $\mathcal{T}$. We assume the victim model $f_{\mathcal{V}}$ has the state-of-the-art performance on both the main task (i.e., classification on the clean dataset $\mathcal{D}$ ) and the watermark task (classification on the trigger set $\mathcal{T}$). The constraints are that only a certain number of auxiliary data $\mathcal{D}_{A}$ are available in the AD attack, while the triggers or watermarking methods are agnostic.

The attacker aims to eliminate the watermark from the victim model. We view SFW as an instance of multi-task learning with conflicting objectives. Namely, using a continual learning approach with a limited resource budget to find an optimal self-balancing loss function that achieves high accuracy on the main task but low accuracy on the watermark task. In summary, to get the optimum classifier $f_{\mathcal{A}^{*}}$, the attack can optimize

$\displaystyle\mathop{\text{min}}\limits_{\bm{\theta}_{\mathcal{A}}}~{}\mathcal{L}_{AD}(\bm{\theta}_{\mathcal{A}}),~{}\text{for}~{}\bm{x}\in\mathcal{D}_{A},$

(3)

such that the following equations are hold:

	$\displaystyle|\text{Prob}_{\bm{x}\in\mathcal{T}}[f_{\mathcal{A}^{*}}(\bm{x})=y_{t}]-\frac{1}{C}|\leq\epsilon,$		(4)
	$\displaystyle|\text{Prob}_{\bm{x}\in\mathcal{D}}[f_{\mathcal{A}^{*}}(\bm{x})=y]-\text{Prob}_{\bm{x}\in\mathcal{D}}[f_{\mathcal{V}}(\bm{x})=y]|\leq\text{negl},$		(5)

where $\mathcal{L}_{AD}$ is the desired objective loss function for the attacker, negl is a negligible value and $\epsilon$ determines to what extent the fine-tuned model forgets the original watermark.

Recent studies on DNN visualization [14, 15] find that activation maps can be used to build heatmaps (i.e., attention maps) that imply models’ attention on images. And different models often share similar attention when making the right decisions for the same inputs [16]. Intuitively, in the fine-tuning/continual learning process, if we can maintain the attention maps for the main task while disrupting the attention maps for the watermark task, then we can achieve SFW.

As shown in Fig. 2, initially, the optimum $\theta_{\mathcal{V}}^{*}$ comes with acceptable error allowed by both the main task and original watermark task. The red arrows direct the path to update weights whilst learning the lure task. The continual learning of the lure task prompts the model weights to update in the direction that is conducive for the classification of the lures. The design of the lure task should ensure that it does not overlap with the watermark task, so it is more likely for the model to find a path that is only conducive for the classification of the main task and the new task, i.e., the red arrows’ direction. At the end, it can reach an optimum $\theta_{\mathcal{A}}^{*}$ with acceptable error fits for both the main task and lure task but not the original watermark task.

As we have mentioned above, for scarce of source training data, to selectively forget watermarks, we need assistance from a certain number of auxiliary data, i.e., proxy samples and lure samples.

In our method, the unlabeled proxy samples $\mathcal{D}_{P}=\{\bm{x}^{(i)}_{p},\perp\}_{i=1}^{N_{P}}$ can be collected from the Internet, drawn from the in-distribution (ID) or OOD area of the source training data. The lure dataset $\mathcal{D}_{L}$ can be drawn from the OOD area of the source training data, e.g., randomly selected from natural images. To prevent feature collision between proxy data and lure data, there should be no intersection between them (i.e., $\mathcal{D}_{L}\cap\mathcal{D}_{P}=\emptyset$) and their distributions are different. Then a new label $\Delta~{}(\Delta=C)$ is assigned to the lure samples, and the whole lure set is $\mathcal{D}_{L}=\{\bm{x}^{(i)}_{l},\Delta\}_{i=1}^{N_{L}}$. And the proxy samples and lure samples consist of the required auxiliary data, i.e., $\mathcal{D}_{A}=\mathcal{D}_{P}\cup\mathcal{D}_{L}$.

Since the lure data $\mathcal{D}_{L}$ is a new task for the victim model, manipulations of the model are necessary before the continual learning process. We add a new class to the output layer of $f_{\mathcal{V}}$ and make it fully connected to the layer beneath. The ground-truth label of the lures is $\Delta=C$, we formulate the prediction loss function for the lures as

$\displaystyle\mathcal{L}_{\delta}=\mathbb{E}_{{}_{\bm{x}\in\mathcal{D}_{L}}}\left[\mathcal{L}_{CE}(f_{\mathcal{A}}(\bm{x};\bm{\theta}_{\mathcal{A}}),~{}C)\right].$

(6)

The authors in [2] have demonstrated that fine-tuning or retraining the deep layers (i.e., the fully-connected layers) only has little effect to remove the watermark. Based on this consideration, we fine-tune the whole network. We rewrite the victim model into a composite function

$\displaystyle f_{\mathcal{V}}(\bm{x};\bm{\theta}_{\mathcal{V}})=\mathsf{F}^{(2)}(\mathsf{F}^{(1)}(\bm{x};\bm{\theta}^{(1)}_{\mathcal{V}});\bm{\theta}^{(2)}_{\mathcal{V}}),$

(7)

in which $\mathsf{F}^{(1)}:\mathbb{R}^{H\times W\times C}\rightarrow\mathbb{R}^{H_{l}\times W_{l}\times C_{l}}$ denotes the attention mapping function that propagates an input $\bm{x}$ through the network to the $l$-th ($l\in[1,2,\cdots,L]$) activation layer, $\mathsf{F}^{(2)}:\mathbb{R}^{H_{l}\times W_{l}\times C_{l}}\rightarrow\mathbb{R}^{C+1}$ propagates the attention maps from $\mathsf{F}^{(1)}$ to the softmax output layer. In our approach, we select a relatively deep layer, e.g., the penultimate activation layer, to construct the sub-net $\mathsf{F}^{(1)}$.

Since the source victim model $f_{\mathcal{V}}$ contains watermarks, their attention maps on the proxy data will inevitably, more or less, carry the watermark information. Accordingly, we penalize the attention maps of the model on proxy data and formulate our attention anchoring loss function as

$$\mathcal{L}_{AA}=\lambda_{1}\cdot\mathcal{L}_{v}+\lambda_{2}\cdot\mathcal{L}_{a}+\lambda_{3}\cdot\mathcal{R}(\bm{\theta}_{\mathcal{A}}),$$

(8)

where $\lambda_{i}$ is the coefficient that regulates the importance of the item, $\mathcal{L}_{v}$ is the vanilla prediction loss term defined as

$$\mathcal{L}_{v}=\mathbb{E}_{{\bm{x}\in\mathcal{D}_{P}}}\left[\mathcal{L}_{KL}(f_{\mathcal{A}}(\bm{x};\bm{\theta}_{\mathcal{A}}),~{}f_{\mathcal{V}}(\bm{x};\bm{\theta}_{\mathcal{V}}))\right],$$

(9)

$\mathcal{L}_{KL}$ is the Kullback-Leibler divergence, $\mathcal{L}_{a}$ is the attention alignment loss term

$\displaystyle\mathcal{L}_{a}=\mathbb{E}_{{}_{{\bm{x}\in\mathcal{D}_{P}}}}\left[\|\mathsf{F}^{(1)}(\bm{x};\bm{\theta}_{\mathcal{A}})-\mathsf{F}^{(1)}(\bm{x};\bm{\theta}_{\mathcal{V}})\|_{2}\right],$

(10)

and $\mathcal{R}(\bm{\theta}_{\mathcal{A}})$ is the attention penalty term for the proxy data

$\displaystyle\mathcal{R}(\bm{\theta}_{\mathcal{A}})=\mathbb{E}_{{{\bm{x}\in\mathcal{D}_{P}}}}\left[\|\mathsf{F}^{(j)}(\bm{x};\bm{\theta}_{\mathcal{A}})\|_{2}\right],\$

(11)

for $j=1,2$. To summarize, the total loss for solving the SFW problem with AD is

$\displaystyle\mathcal{L}_{AD}=\mathcal{L}_{\delta}+\mathcal{L}_{AA}.$

(12)

All the experiments are performed on a PC equipped with a NVIDIA Geforce RTX 2070 GPU and Ubuntu 20.04 OS, and Keras is the underlying framework.

In our experiments, the watermarked models are mainly targeted at classifying CIFAR-10 [17] and GTSRB [18]. The implemented network architectures of victim models include VGG16 [19], ResNet18 [20], and WRN-16-4 (Wide Residual Network) [21]. We evaluate the attack effectiveness of AD on three state-of-the-art watermarks introduced in [3], including the content watermark (as shown in Fig. 3(b)), the noise watermark, and the unrelated watermark. We set the number of triggers used to embed watermarks into victim models as $1000$, and they are assigned to class $0$ (the target label).

We consider two attack scenarios for obtaining the proxy datasets: using OOD data (i.e., CIFAR-100 [17]) and ID data (half of the original testing data). For each case of the obtained proxy datasets, $1000$ images are randomly sampled and used in the fine-tuning. In addition, only $10$ randomly select abstract images (introduced in [2]) are used as the lure data (the implementation details can be found in the appendix).

Table 1 presents the performance of victim models before and after being attacked by comparing AD with PST-FT on the CIFAR-10 dataset (the results of GTSRB can be found in the appendix). We can observe that almost all three types of watermarks are vulnerable to the PST method. In addition, using PST-FT can generally improve the MTA or decrease the WMA of victim models. However, this is not always the case, especially when the triggers’ perturbation is noise. This is because the trigger noise could be mixed with the distribution mismatch noise.

For AD, it is clear that all the WMAs of the attacked victim are below $20$%, achieving the goal of watermark removal. For all the network architectures, AD retains the MTAs of the victim models. Moreover, though AD generally performs better with ID proxy data, the differences between MTAs for OOD and ID proxy data are almost always within $2$%. This fact validates that AD can avoid the noise caused by the data distribution difference between the proxy and original training data while removing the planted watermark.

In this section, we perform an ablation study to investigate the role of each component of AD on the CIFAR-10 task and the content-based watermark task (more analysis can be found in the appendix).

From Table 2, we observe that, no matter using OOD or ID proxy data, neither the vanilla attack nor the AA attack is as good as AD in solving the SFW. This confirms that the lure task is important for solving SFW: the lure task will compensate for the negative effect brought by the distribution difference between the proxy data and the original training data.

To further investigate the role of the lure task, Fig. 4 visualizes the attention heatmaps (produced by Grad-CAM [15]) of ResNet18, before and after different attacks using OOD proxy data, on CIFAR-10 images. From the second and third rows of this figure, it is easy to see that the attention maps of the model after the AD attack are highly similar to those of the victim before attack. For both the vanilla and the AA attacks, the attention maps of the model after attack deviate from those of the victim before attack. That said, without the participation of lure, using OOD proxy data alone cannot effectively anchor the main task attention maps while forgetting the target watermark task.

As discussed previously and visualized in Fig. 2, when the lure label does not overlap with the target label, it encourages the continual learning process to find a path that is suitable for the main task and the lure task, while forgetting the target watermark task. For the consideration of watermark agnostic, the most convenient way to prevent the label collision is to set $\Delta=C$.

Figure 5 depicts the model performance on the main task after erasing the watermarks by different versions of AD attacks. From this figure, it is clear that the blue bars, which represent the complete AD, achieve the highest MTA for all considered network architectures.