Unified High-binding Watermark for Unconditional Image Generation Models

Watermarking technology represented by steganography is one of the most important ways to protect the copyright of images. It can be roughly categorized into two types: visible watermarks like logos, and invisible watermarks. Obviously, the way of invisible watermark is more covert than the way of adding visible watermark to images and does not affect the use of them. Invisible watermark can be further divided into transform domain watermark and spatial domain watermark. The transform domain watermark method writes watermark into the coefficients that are not significant in the transform domain such as Discrete Fourier Transform (DFT) domain (Mersereau and Alturki 2001) and Discrete Wavelet Transform (DWT) domain (Kumar and Kumar 2018) of the images. Spatial watermark (Luo et al. 2023; Liu et al. 2022b; Ma et al. 2023) mainly uses deep neural network to complete watermark embedding and extraction. The image watermarking method is easier to implement than the model watermarking method (Adi et al. 2018; Lukas et al. 2022; Guo et al. 2021), but it is not suitable for verifying whether the surrogate model has data steal.

UIG models can create novel, original images that are not based on existing images without any specific input conditional information. As shown in Figure 2, this type of model learns the mapping functions from random noise to images through the training process, and then the inference stage uses sampling noise as a starting point to generate new images. This can be used in a variety of applications, such as improving image recognition algorithms(Zhang et al. 2022) or generating realistic images to augment dataset. Variational Autoencoder (VAE) (Chadebec et al. 2023; Razavi, van den Oord, and Vinyals 2019) and Generative Adversarial Network (GAN) (Goodfellow et al. 2014; Karras et al. 2020) are two typical UIG models that appeared earlier, they realize the conversion between noise and images in a single step during training and inference. Diffusion models (Ho, Jain, and Abbeel 2020; Song, Meng, and Ermon 2021; Liu et al. 2022a) draw inspiration from the principles of non-equilibrium thermodynamics. They establish a Markov chain of diffusion steps that gradually introduce random noise into image, and subsequently acquire the ability to invert the diffusion process to generate desired image samples from the noise. Although these UIG models achieve the same purpose, their different and complex implementation mechanisms bring difficulties to the formation of an unified watermark verification mechanism.

Though AIGC Watermarking Technology is still seriously under-studied, there are some initiatives and works that start paying attention to it. The famous AIGC tool DALLE marks the generated content by adding a spatially visible watermark in the lower right corner of the image. Stable Diffusion modifies specific Fourier frequency in the generated image (Wen et al. 2023) to watermark output images. (Huang et al. 2022) proposes an universal adversarial watermark to protect images from being abused by Deepfake technology. (Liu et al. 2022b) proposes a non-embedded watermark method, so that the newly generated image naturally contains potential watermark information. Existing work mainly focuses on the direct traceability of AIGC, and very few works (Yu et al. 2021; Zhao et al. 2023) study how to effectively verify AIGC data after it has been abused. At present, there is no unified watermark verification mechanism that can realize AIGC copyright protection across UIG models.

For an UIG model, we denote its output images as domain $A$ according to the image distribution, and domain $A$ follows the same distribution as the model training data. The goal of the UIG model $M$ is to approximately fit the mapping function $F$ from random noise to the target image distribution through one or several neural networks. Suppose we obtain an UIG model $M$ trained on a private dataset $Trainset$. Given a random noise $n_{i}$, $M$ can output an image $a_{i}$ in domain $A$ that has never actually appeared in $Trainset$, i.e.,

Since UIG models use random noise as the starting point to generate new images, the attacker only needs to collect the output images of the target model instead of the input-output image pairs to train the corresponding surrogate model $SM$, which greatly reduces the difficulty for the attacker to steal. In addition, according to the image distribution, the output images of $SM$ belong to another image domain with similar but different distribution to domain $A$. In practical scenarios, AIGC tools formed from such models are usually presented in a black-box manner, and users can only obtain their output images. Therefore, our goal is to design a unified watermark verification mechanism with high-binding effect across data domains and even across model architectures, which can verify whether it steals $M$ generated images for training only through the outputs of the suspicious model.

This is the first stage in our proposed unified watermark verification mechanism, which can also be called the preparation stage. For data stealers, they can only get the output images of the target AIGC tool instead of the built-in UIG model. For the suspicious UIG model that we mentioned above, the publisher of the original AIGC tool can only verify whether there is data steal based on its output images. As can be seen, the model output images are the only medium between the original AIGC tool owner and the suspect model. Therefore, we hope that the watermark image used as the basis for verification can be completely embedded in the output images of the above two types of models. In particular, the watermark image should be embedded in an invisible way so as not to affect the use of the model output image.

In the preparation stage, we first ensure that the output images of the original UIG model is embedded with a custom watermark image $\delta_{1}$. As shown in the upper part of Figure 3, we select a neural network as an encoder $H$ to embed $\delta_{1}$ into the output images of the protected UIG model (i.e., domain $A$) in spatial invisible way, and denote the distribution of watermarked images as domain $A^{{}^{\prime}}$. Obviously, the distributions of domain $A$ and domain $A^{{}^{\prime}}$ are similar, the subtle difference between the two distributions is caused by the potential watermark information. At the same time, we choose another neural network as the decoder $R$ to reversely extract $\delta_{1}$ from images in domain $A^{{}^{\prime}}$.

The difference between the corresponding image pair in domain $A$ and domain $A^{{}^{\prime}}$ is extremely small (i.e., spatial invisible), the watermark image $\delta_{1}$ is injected into the images of domain $A$ mainly by changing the high-frequency information. In order to improve the ability of $R$ to recover the specified watermark image $\delta_{1}$ and to avoid the overfitting phenomenon that $\delta_{1}$ can be extracted from all images no matter whether $\delta_{1}$ is really written or not, we simultaneously add images from domain $A$ to the dataset for training $R$. We force $R$ to extract a constant blank images $\delta_{0}$ from these non-watermarked images. To further narrow the data distribution difference between domain $A$ and domain $A^{{}^{\prime}}$, we add a steganalysis network $D$ as a discriminator after $R$, and improve image quality in domain $A^{{}^{\prime}}$ through adversarial training.

After the preparation stage, the publisher of the original AIGC tool can use $H$ to embed $\delta_{1}$ in the original output image of the model and then output it. In this way, what the data stealers get is the output images with the watermark information. The corresponding $R$ can be used to directly trace the source of the released images.

This is the second stage of our proposed unified watermark verification mechanism, which can also be called the verification stage. After this stage, we hope that the fine-tuned $R$ can also extract $\delta_{1}$ from the output images of $SM$.

As shown in the lower part of Figure 3, as the verifier, the publisher of the original AIGC tool needs to prepare a verification dataset containing four parts of the images, i.e., images from domain $A$, $A^{{}^{\prime}}$, $B$, and $X$ respectively. According to the above, the images in domain $A$ and domain $A^{{}^{\prime}}$ are naturally exist after the preparation stage, especially the images of domain $A$ are private to the AIGC tool owner. The images in domain $B$ are collected from the output of the suspicious model to be verified. The images of domain $X$ come from the same task UIG models (preferably with different model architectures), which are trained on images without embedded watermarked images. It should be noted that these UIG models for the same task can be easily prepared by the AIGC tool owner based on the images of domain $A$, so they are also private.

After the validation dataset is ready, we finetune the decoder $R$ using the decoder fine-tuning strategy. It consists of two parts the watermark extraction loss $l_{e\_s}$ and the constant blank image extraction loss $l_{e\_w}$, i.e.,

In our experiments, we use CelebA (Liu et al. 2015) trainset as the main dataset, which is more challenging, has higher resolution and contains more texture details in the images. It contains 202599 facial images. These images are split into two parts: 50,000 images are owned by the data stealer, others are used to train the original UIG models. All images in CelebA are processed into 128*128. In addition, we also verified the effectiveness of the method in another Oxford 102 Flower dataset (Nilsback and Zisserman 2008). It contains 8189 real flower images, we set 2000 as private images for data stealer, others are used to train original UIG models. All images in Oxford 102 Flower are processed into 64*64.

The original UIG models we select in our experiments are VQ-VAE-2 (Razavi, van den Oord, and Vinyals 2019), StyleGAN2 (Karras et al. 2020), DDPM (Ho, Jain, and Abbeel 2020) and PNDM (Liu et al. 2022a). The selected models are typical models of various UIG models under different implementation mechanisms. For the diffusion model that generates new samples through multiple steps, We choose DDPM and PNDM to consider the impact that different noise sampling strategies may have on the performance of the proposed method.

For the validation dataset involved in the second stage of our method, we always keep each part of the dataset to contain 1000 images when using the CeleA dataset, and we always keep each part of the dataset to contain 100 images when using the Oxford 102 Flower dataset. In all experiments, we fine-tune the decoder using the validation dataset until convergence is confirmed. We use new images outside the validation dataset to evaluate watermark extraction. By default, $\lambda_{1}$ and $\lambda_{2}$ equal to 1.

To evaluate the visual quality, PSNR and SSIM are used by default. In order to judge whether the watermark image is extracted from the image, we define that the extraction is successful when the SSIM between the extraction result and the real watermark image is bigger than 0.9, which means that there is a large similarity between them. Based on it, the extraction rate (ER) is further defined as the ratio of successfully extracting watermark image from model output images. The false positive rate (FPR) is defined as the ratio of successfully extracting watermark image from model output images without data steal. Furthermore, we define the data theft rate (DTR) as the ratio of output images from the original AIGC tool in the surrogate model training data.