Robust and Imperceptible Black-box DNN Watermarking Based on Fourier Perturbation Analysis and Frequency Sensitivity Clustering

The proposed method includes three steps, i.e., trigger sample generation, watermark embedding and ownership verification. The purpose of trigger sample generation is to generate trigger samples, which will be used for watermark embedding and ownership verification. The watermark embedding process is realized by training the DNN with the normal samples and the trigger samples. During training, the normal samples are associated with their own normal labels. However, a new class will be assigned to the trigger samples. In other words, all the trigger samples will share a new label that does not appear in the normal-label set. After training, the DNN model is deemed marked and will be put into use. For ownership verification, a set of trigger samples will be fed into the target DNN model to obtain the prediction results. By analyzing the prediction results, the ownership of the target model can be identified. Fig. 1 shows the framework. We provide the details below.

Mathematically, let $\mathcal{M}_{0}$ represent the host DNN, which has been trained on a normal dataset $D$ that consists of a number of sample-and-label pairs $\{(\textbf{x}_{i},y_{i})~{}|~{}1\leq i\leq|D|\}$. Here, $|*|$ represents the total number of elements in a set. By limiting $\mathcal{M}_{0}$ to image classification, we can write $\textbf{x}_{i}\in\mathbb{R}^{h\times w\times d}$ and $y_{i}\in\{0,1,...,c-1\}$ for $1\leq i\leq|D|$, where $h\times w\times d$ indicates the size of the image and $c$ gives the total number of classes on the normal dataset. Assuming that we have already generated two sets of trigger samples $T_{1}=\{(\textbf{x}_{i}^{\prime},y_{i}^{\prime})~{}|~{}1\leq i\leq|T_{1}|\}$ and $T_{2}=\{(\textbf{x}_{i}^{\prime\prime},y_{i}^{\prime\prime})~{}|~{}1\leq i\leq|T_{2}|\}$ according to the proposed trigger sample generation algorithm, the watermark embedding phase requires us to train such a model $\mathcal{M}_{1}$ from scratch based on $D$ and $T_{1}$ so that $\mathcal{M}_{0}$ and $\mathcal{M}_{1}$ have the same generalization ability on the “unseen” dataset $U=\{(\textbf{x}_{i}^{*},y_{i}^{*})~{}|~{}1\leq i\leq|U|\}$, which can be measured from a statistical perspective as

where $0\leq\epsilon\leq 1$ is a very small threshold, $\delta(x,y)$ returns 1 if $x=y$ otherwise it returns 0, and $\mathcal{M}_{i}(e)$ is the classification result after feeding $e\in U$ into $\mathcal{M}_{i}$, $i\in\{0,1\}$. Notice that, it is possible that $T_{1}\cap T_{2}\neq\emptyset$, e.g., $T_{1}=T_{2}$ is used for [18]. It is required that $\mathcal{M}_{1}$ has a high prediction accuracy on $T_{2}$:

As mentioned in Subsection II-A, the proposed work assigns a new class tag to all the trigger samples, i.e., we can write $y_{1}^{\prime}=y_{2}^{\prime}=...=y_{|T_{1}|}^{\prime}=y_{1}^{\prime\prime}=y_{2}^{\prime\prime}=...=y_{|T_{2}|}^{\prime\prime}=c$. In brief summary, the proposed method extends the label set from $\{0,1,...,c-1\}$ to $\{0,1,...,c\}$, which is inspired by the novel method introduced in [16]. It should be admitted that it is always free for people to design the labeling strategy, which is not the main contribution of this paper. In our experiments, we will analyze the performance of different labeling strategies. After model training, the resulting model $\mathcal{M}_{1}$ is treated as a marked version of $\mathcal{M}_{0}$ and will be put into use.

Assuming that $\mathcal{M}_{1}$ has been leaked and probably tampered, we are to verify the ownership of the leaked version $\mathcal{M}_{2}\approx\mathcal{M}_{1}$ by querying the classification results of $\mathcal{M}_{2}$ for the trigger samples in $T_{2}$. Formally, the ownership can be verified if

where $0\leq\delta\leq 1$ is a pre-determined threshold. Otherwise, the ownership verification is deemed failed. From a more general perspective, the existing methods often assume that only the DNN model may be attacked, in this paper, we will further investigate the possible attack on the trigger set, i.e., the trigger sample $\textbf{x}_{i}^{\prime\prime}$ in Eq. (3) may be attacked prior to verifying the ownership. It is required that any DNN watermarking method should still resist such kind of attack, which, however, has not been considered in many existing methods.

Therefore, if we consider the attack to the trigger samples, for ownership verification, we should rewrite Eq. (3) as:

where $\textbf{x}_{i}^{\prime\prime\prime}$ is the attacked version of $\textbf{x}_{i}^{\prime\prime}$. The proposed method is based on Eq. (4), which has better applicability.

It has been widely demonstrated by the conventional image watermarking methods [23] that embedding secret information in the frequency domain has good robustness and can keep the visual quality of image well. Embedding secret information in the frequency domain is actually equivalent to adding a kind of perturbation in the frequency domain. Inspired by this key insight, a straightforward idea to generate the trigger sample is adding perturbation to the normal sample (or called clean sample) in the frequency domain, which raises two problems. The first problem is where to add the perturbation, i.e., what frequency components should be used to add the perturbation. The second problem is how to generate the perturbation. For the second problem, it is easy and free for us to design the perturbation, e.g., the off-the-shelf method is adding Gaussian noise. Therefore, our main task is to solve the first problem.

On the one hand, in image watermarking, embedding information in the low frequency region provides good robustness but may introduce noticeable distortion. On the other hand, embedding information in the high frequency region keeps the perceptual distortion low, but the robustness is not satisfactory. Therefore, a good balance between robustness and imperceptibility is to embed secret information in the mid-low frequency region. Since the trigger samples in this paper are images, it is also very suitable to add perturbation in the mid-low frequency region so that the perturbed samples, i.e., the trigger samples, not only have good visual quality but also can resist malicious attacks. However, we have to analyze the impact caused by frequency perturbation on the DNN since the trigger samples will be used for DNN training. Therefore, it is naturally to ask such a question: is adding perturbation in the mid-low frequency region also good for DNN training?

The aforementioned question actually requires us to analyze the impact of different frequency components of input samples on the performance of the host DNN on its original task by adding some specific perturbation in the frequency domain. We hope to find such frequency components that they not only ensure a good balance between perturbation robustness and perturbation imperceptibility, but also ensure a good balance between watermarking robustness and model generalization. The perturbation robustness means that even the trigger samples were attacked, the perturbation in the trigger samples can be still perceived by the DNN for ownership verification. The perturbation imperceptibility means that the perturbation added to the trigger samples should not introduce significant distortion to the clean samples. The watermarking robustness means that the ownership can be still reliably verified even the marked model was attacked. The model generalization means that the performance of the DNN on its original task after watermarking should not be impaired. It is pointed that we cannot completely separate perturbation robustness (imperceptibility) from watermarking robustness (imperceptibility) because they are entangled with each other. Here is just to facilitate analysis. In addition, good perturbation robustness also indicates that the marked DNN model has good robustness against attacks.

Below, we are to demonstrate that adding the perturbation in the mid-low frequency region is a good choice for constructing the trigger samples to balance the above requirements.

We used three popular datasets CIFAR-10, CIFAR-100 [30] and GTSRB [31] to conduct extensive experiments. CIFAR-10 and CIFAR-100 contain 60,000 images. CIFAR-10 contains 10 classes and CIFAR-100 contains 100 classes. Each of them was randomly divided into three disjoint subsets, i.e., training set (75%), validation set (5%) and testing set (20%). GTSRB contains 43 classes of traffic signs, split into 39,209 training images and 12,630 testing images. The training images were divided into two disjoint subsets, i.e., training set (80%) and validation set (20%). The testing images were belonging to the testing set. The popular architectures VGG19 [32], ResNet-18 [29], ResNet-56 [29], DenseNet-121 [33], and WideResNet-34 [34] were used to act as the original DNN. The proposed work is not subjected to the above models and datasets.

Without the loss of generalization, we here take VGG19 for example to explain how to generate the trigger samples, how to generate the marked model and how to verify the ownership.

• -

  Trigger Sample Generation: The training set denoted by $D_{1}$ and the validation set denoted by $D_{2}$ are collected to build the dataset $D$ in Algorithm 1. $\mathcal{M}_{0}$ is set to VGG19. $\mathcal{M}_{0}$ can be trained with $D$ from scratch to generate the trained but non-marked model (see Line 2 in Algorithm 1). Given any normal sample, we are able to generate the corresponding trigger sample by Algorithm 1.

• -

  Watermark Embedding: We randomly generate two sets $A_{1}\subset D_{1}$ and $A_{2}\subset D_{2}$. We apply Algorithm 1 to $A_{1}$ and $A_{2}$ to generate the corresponding trigger samples, whose labels are set to $c$. In this way, we obtain two trigger sets $B_{1}$ (corresponding to $A_{1}$) and $B_{2}$ (corresponding to $A_{2}$). Two sets $D_{1}\cup B_{1}$ (training) and $D_{2}\cup B_{2}$ (validation) are used to train $\mathcal{M}_{0}$ from scratch to generate the trained and marked model $\mathcal{M}_{1}$. It can be inferred that $T_{1}=B_{1}\cup B_{2}$ according to the definition of $T_{1}$ in Subsection II-B.

• -

  Ownership Verification: We verify the ownership of the target model by Eq. (4). The key step is how to generate the trigger set $T_{2}$. To deal with this issue, we divide the testing set $E$ into two disjoint subsets $U$ and $V$. $V$ is used to generate a set of trigger samples $V_{\text{T}}$. We set $T_{2}=V_{\text{T}}$. Table I shows the relationships between different sets.

We set $|A_{1}|$ = $|B_{1}|$ = $|A_{2}|$ = $|B_{2}|$ = $|V|$ = $|V_{T}|$ = $q_{t}$. We empirically set $q_{t}=500$ by default in our experiments unless otherwise specified. We applied the stochastic gradient descent (SGD) optimizer to train each network. The learning rate was 0.01, with a momentum of 0.9. The batch size was 512. The perturbation coefficients were randomly sampled from $[-1,1]$. The threshold $\rho$ was set to 0.65 by default. It is always free for us to adjust these parameters to achieve better performance.

Task fidelity means that the performance of the DNN model on its original task after watermarking should be kept well. It requires us to determine the accuracy of image classification for the original model (which is deemed non-marked) and the marked model. Table II shows the classification accuracy of the non-marked model. Table III shows the classification accuracy of the marked model. It can be inferred from Table II that different models have different performance on different image classification tasks, which is reasonable due to the different learning capabilities of deep models. It can be also inferred from Table III that the accuracy declines as $q_{t}$ increases from the viewpoint of overall trend, which is a normal phenomenon because the model needs to make a sacrifice on the original task for model watermarking. However, by comparing Table II and Table III, we can find that the performance degradation on the original task is very low after watermarking with different $q_{t}$, which indicates that the proposed method does not impair the utilization of the model and thereby has good potential in application scenarios. It is admitted that we did not optimize the hyper-parameters and the training strategy for all models, meaning that the baseline performance in Table II may be not optimal, which does not affect the evaluation of our work.

On the other hand, watermark fidelity means that the hidden watermark should be reconstructed with a high accuracy. Since the proposed method corresponds to zero-bit watermarking, it is required that the classification accuracy on the trigger set should be as high as possible. We calculate the classification accuracy on the trigger set for different models with different $q_{t}$. The results are shown in Fig. 4, from which we can find that different models have different accuracy values, which means that different models have different capabilities of carrying a watermark. It is observed that all the accuracy values are no less than 80%, meaning that the proposed method enables the watermark to be reliably extracted. It can be inferred that the accuracy will increase when $q_{t}$ increases in most cases. The reason is that a larger $q_{t}$ means that more trigger samples are used for model training, allowing the watermarking task to be accomplished very well by the model. For example, in Fig. 4, the accuracy is approaching 100% when $q_{t}=1000$, which has demonstrated the superiority of the proposed method.

In addition, sample fidelity means that the perturbed sample should be visually close to the original sample. Fig. 5 shows some examples of the perturbed sample. It can be seen that the proposed perturbation technique does not introduce noticeable artifacts. To quantize the visual quality, we determine the mean peak signal-to-noise ratio (PSNR, dB) and structural similarity (SSIM) [35] between the original samples in the testing set and the corresponding perturbed samples. The results are shown in Table IV, from which we can find that the mean PSNRs are higher than 36 dB and the mean SSIMs are higher than 0.99. It indicates that the perturbed samples have very good visual quality, which can guarantee perturbation imperceptibility.

Generally, robustness evaluates the ability to reconstruct the embedded watermark from the marked DNN model when the watermarking system was attacked by the adversary. Unlike many existing methods only considering attacks to the marked model, we further take into account the possible attacks to the trigger samples. In the following, we first analyze the performance of the model on the original task and the watermarking task when the marked model was attacked. Then, we analyze the performance when the trigger samples were attacked.

Two most popular attacks are applied to the marked model. One is model fine-tuning, and the other is model pruning. To mimic the fine-tuning attack, we randomly choose 50% normal samples of the validation set to fine-tune the marked model. To mimic the model pruning attack, we apply the $\ell_{1}$-norm pruning strategy to the marked model, where a pruning rate is used to denote the percentage of pruned parameters. We evaluate the performance of the attacked model on the original task and the watermarking task. The results are provided in Fig. 6 and Fig. 7. It is inferred from Fig. 6 that model fine-tuning does not impair the original task and the watermarking task. For model pruning, though the performance will gradually decline as the pruning rate increases, the performance can be well preserved even 50% of the parameters are pruned, which indicates that the proposed method has good ability to resist model pruning.

We further evaluate the ability of the trigger samples against image processing attacks. As the computing resource is limited, only the representative VGG19 and ResNet-18 are used as the host network unless otherwise specified. We consider three common attacks, i.e., JPEG compression, flipping and filtering. A common approach to enhance the robustness against these attacks is using data augmentation (UDA), i.e., mixing the attacked images into the training set. Both UDA and not using data augmentation (NUDA) are considered here for fair evaluation. For UDA, to resist a specific attack with preseted parameter(s), we enhance the robustness and generalization of the model by putting the normal training samples processed by the same attack into the training set. Thus, the model not only learns the original task and the watermarking task well, but also resists the corresponding attack applied to the input.

Table V and Table VI show the performance against JPEG compression for VGG19 and ResNet-18, respectively. It can be inferred that for both UDA and NUDA, when the quality factor (QF) gradually decreases, both $\text{Acc}_{\text{o}}$ and $\text{Acc}_{\text{w}}$ tend to decrease, which is due to the reason that a smaller QF results in more loss of feature information. However, we can find that the performance difference between UDA and NUDA is small, especially for larger QFs. It indicates that the proposed method has good ability to make the trigger samples capable of resisting JPEG compression. It is noted that both the normal sample and the trigger samples are attacked during the testing phase. Table VII provides the results against horizontal flipping under different conditions, from which we can infer that flipping does not impair watermarking because the trigger patterns are symmetrical in the frequency domain.

We further consider low-pass filtering in the DFT domain. Low-pass filtering with bandwidth B is defined as the operation of setting all frequency components outside the center square of width B in the Fourier spectrum at the center lowest frequency to zero and then applying the inverse DFT to the spectrum. Table VIII and Table IX report the performance against low-pass filtering under different conditions for VGG19 and ResNet-18. It can be inferred that the classification accuracy on the trigger set achieves more than 80% when the bandwidth is larger than 12. However, when the bandwidth is smaller than 12, it causes a significant drop of the classification accuracy for both the original mission and the watermarking task. This is because our method perturbs mid-low frequency components. When the trigger samples pass through a large filter, although some high-frequency information is lost, the model can still learn the trigger. The image content can be learned by the model as well. Therefore, both the original task and the watermarking task are performed well. However, by applying a smaller filter, much information about the trigger and the image content will be lost. As a result, the performance on both tasks will be surely declined.

One of the key contributions of this paper is that we propose a new method to generate the trigger samples. It is necessary to compare the visual quality of the trigger samples generated by different methods. To this purpose, we compare the proposed method with three most representative methods, i.e., unrelated samples (URS), logo based trigger (LBT), noise based trigger (NBT). In detail, URS uses images that are not related to the original task of the host model as the triggers [14, 15]. LBT simply adds a logo such as character(s) and specific pattern to a clean image to construct the trigger sample [15]. NBT uses a pre-defined noise to play the role of the trigger [15, 16].

The above strategies can be all applied to model verification through analyzing the prediction results of the target model given a certain number of trigger samples. However, in terms of imperceptibility, the above strategies are not desirable for application scenarios. Fig. 8 provides some trigger samples obtained by different methods. Although URS may not significantly impair the performance of the host model on its original task, the irrelevant image content may arouse suspicion of the attacker. LBT and NBT introduce noticeable artifacts in the trigger image, which will impair imperceptibility. Moreover, the noticeable artifacts may expose how the trigger signal was constructed, thereby threatening security. As shown in Fig. 8, the visual difference between the clean samples and the trigger samples generated by the proposed method is small, indicating that the proposed method keeps the imperceptibility of trigger signal very well and therefore is very suitable for practice.

Table X and XI provide more quantitative results. Both the original task and the watermarking task are tested. Moreover, mean PSNR and mean SSIM are used to measure the distortion between the trigger samples and their clean versions. We train each model from scratch. Although different trigger samples can be assigned with different labels, it will surely increase the difficulty of the learning for the model since in this case the model needs to learn various mappings that are not related to the original task which will impair the generalization ability of the model. Therefore, for fair comparison, for each method to be compared, all the trigger samples share the same ground-truth label specified by random in advance (and it should not be the label of the original clean sample). Some examples of trigger samples have been shown in Fig. 8. It is inferred that the proposed method not only achieves better visual quality for the trigger samples, but also shows better performance on both the original task and the watermarking task. It can be said that by adding well-designed perturbation in the frequency domain, the distortion between the clean sample and the trigger sample can be kept low. Meanwhile, as the perturbation will be spread throughout the entire spatial domain, it will better facilitate the learning of the trigger signal.

In the proposed method, the label assigned to all the trigger samples can be either a randomly selected label or a new label. The randomly selected label should not be the one belonging to the original clean sample. We conduct experiments to evaluate the impact of using different label assignment strategies on both the original task and the watermarking task. Except for the assigned label, all the other experimental settings are same as each other. Table XII provides the experimental results from which we can find that using a new label is superior to using a randomly selected label since the classification accuracies on the two tasks for the former are significantly higher than that for the latter. The reason is that, using a randomly selected label may inevitably distort the decision boundary of the model on the original task, while using a new label maps the trigger samples to a new category so that the learning of the original task and the learning of the watermarking task can be separated to a certain extent, thereby allowing both the original task and the watermarking task to be better performed.

We further analyze the impact caused by perturbing different frequency areas. Taking ResNet-18 for explanation, Fig. 9 shows the Fourier heat maps and different masks evaluated on different datasets. It can be found that different frequency areas have different sensitive-frequency distributions. It is necessary to analyze their impact on the performance of the model. Table XIII and XIV have given the experimental results, from which we can find that the mean PSNRs and the mean SSIMs of the trigger samples due to high frequency perturbation are higher than that due to mid-low frequency perturbation. The reason is that mid-low frequency perturbation results in more distortion in the spatial domain while high frequency perturbation generally distorts local details of an image. However, from the viewpoint of functionality, it can be observed that using mid-low frequency perturbation achieves better performance on the original task and the watermarking task. It is due to the reason that mid-low frequency perturbation can better facilitate model learning, which has been analyzed in the previous section.