RDP-GAN: A Rényi-Differential Privacy based Generative Adversarial Network

We first recall the standard definition of ($\epsilon,\delta$)-DP [16].

The above definition is contingent on the notion of adjacent inputs $X$ and $X^{\prime}$, which is domain-specific and is typically chosen to capture the contribution to the mechanism’s input by a single individual. Moreover, to avoid the worst-case scenario of always violating privacy of a $\delta$ fraction of the dataset, the standard recommendation is to choose $\delta\ll 1/N$ or even $\delta=\textrm{negl}(1/N)$, where $N$ is the data size.

The definition of ($\epsilon,\delta$)-DP is initially proposed to capture privacy guarantees of the Gaussian mechanism, defined as follows:

where $\mathcal{N}(0,\sigma^{2})$ represents a random noise, which follows the Gaussian normal distribution with mean $0$ and standard derivation $\sigma$. Elementary analysis shows that the Gaussian mechanism satisfies a continuum of incomparable ($\epsilon,\delta$)-differential privacy [16], for all combinations of $\epsilon<1$ and $\sigma>{\Delta_{2}}f\sqrt{2\ln 1.25/\delta}/\epsilon$, where the $l_{2}$-sensitivity of $f$ is defined as

and taken over all adjacent inputs $X$ and $X^{\prime}$.

We also list the proposition that will be used in the proof.

The original DP has been investigated in several works to estimate the privacy level of a typical machine learning algorithm, such as Deep Learning [14], Federated Learning [17] and GAN [11, 12]. However, the original DP needs strong assumption and will damage its accuracy when using composition theorem, as pointed in [18]. Thus, we elaborate a generalization of the notion of DP based on the concept of the Rényi divergence, which can avert inaccuracy. The Rényi divergence is classically defined as follows:

It motivates exploring a relaxation of DP based on the Rényi divergence.

We then list some propositions that will be used in deriving the analytical results in the following.

GAN is a recently developed generative model to produce synthetic images or texts after being trained [7]. The learning process in the model is based on one generator (g) and one discriminator (d) neural networks playing in the following zero-sum minimax (i.e., adversarial) game:

where $p(z)$ is a prior distribution of latent vector $z$, $g(\cdot)$ is a generator function, and $d(\cdot)$ is a discriminator function whose output spans $[0,1]$. $d(x)=0$ (resp. $d(x)=1$) indicates that the discriminator $d$ classifies a sample $x$ as generated (resp. real).

In Fig. 1, we show a typical structure of GAN. $g$ and $d$ can be any form of neural networks. The discriminator attempts to maximize the objective, whereas the generator attempts to maximize the objective. In other words, the discriminator attempts to distinguish between real and generated samples, while the generator attempts to generate real-looking fake samples that the discriminator cannot distinguish from real samples. According to the feedback in the interaction process, the generator will updates its network to enhance the quality of the generated samples. Adversaries exist in this environment and may attack the interaction process, and obtain private information from clients. In order to avoid this privacy leakage, in the next section we propose a method to improve the privacy performance of GAN.

In most real-world problems, the ture data distribution $p(x)$ is unknown and needs to be estimated empirically. Since we are primarily interested in data synthesis, and we will use GANs as the mechanism to estimate $p(x)$ and draw samples from it. If trained properly, a RDP-GAN will mitigate inference attack during the data analysis.

The framework of RDP-GAN is structured as follows. Sensitive data $X$ is first fed into a discriminator with a privacy-preserving layer. This discriminator is used to train a differentially private generator to produce a private artificial dataset $\tilde{X}$. Different from the work on differentially private deep learning (e.g., [14, 11, 12, 19]), the proposed algorithm achieves DP by injecting random noise on the value of the loss function in the interaction procedure. In more details, the rational behind our design is as follows:

• •

  Different from existing works, which add noise on the gradients, the proposed algorithm focuses on the design on the interaction between the generator and discriminator. This is because that from the aspect of an adversary, the interaction process is more vulnerable to attack than the network architecture if the generator and discriminator are not assembled. Moreover, it is not trivial to acknowledge information from the inside of the discriminator or generator.

• •

  Adding noise on the value of the loss function is a direct method compared to the method in [11, 12, 19], which will bring explicit privacy protection. This is because that the loss values, not the parameters, are exchanged between generator and discriminator, and the perturbation on the loss value will directly influence the updates of the generator. In addition, the privacy estimation is not that accuracy as the changes of the privacy level from parameters to loss values are ignored in the previous works.

• •

  Adding noise on the value of the loss function does not need an extra norm clipping function as we naturally use a bounded activation function as the last layer of neuron network. Manually bounding the loss function is an non-trivial task as its value is sensitive and needs adjustment based on empirical results.

Despite the fact that the generator does not have access to the real data $X$ in the training process, one cannot guarantee DP because of the information passed through with the loss value from the discriminator. A simple high level example will illustrate such breach of privacy. Let the dataset $X$, $X^{\prime}$ contain some small real numbers. The only difference between these two datasets is the number $x^{\prime}\in X^{\prime}$, which happens to be extremely large. Since the impact of $x^{\prime}$ is large enough to influence the performance of the discriminator, specifically on its loss value, it will lead to a large difference whether $x$ is used to train or not. In this case, this difference on the loss value will be propagated to the generator that will break privacy.

Our method focuses on preserving the privacy during the training procedure, and we add noise on the loss value of the discriminator as follows:

where we use $F_{d}(;)$ to represent the value of the loss function of all data samples, and $m$ denotes the sample size of the discriminator, respectively.

We first explain this perturbation on the loss value will not vanish during back propagation. Considering a $L$ layers network, the perturbed loss value of the last layer (the $L$-th layer) can be represented by

where $A$ denotes the activation function of the $L$-th layer, $C$ denotes the correct label, $\odot$ expresses the XNOR function that outputs a logical one or true only if two inputs are the same, and $z^{L}$ is the input of the $L$-th layer, respectively [20]. During back propagation, this perturbation is involved as follows:

where $l=L-1,L-2,...,2,1$. Thus, if adding noise on the value of the discriminator can achieve DP, this perturbation will be elaborated during the back propagation when updating the discriminator, and ensures that the discriminator is DP. Moreover, the loss value (parameters) of generator can also guarantee DP because of post-processing property in [16] which states that any mapping (operation) after a differentially private output will not invade the privacy. Here the mapping is in fact the computation of parameters of generator.

The RDP-GAN procedure is summarized in Algorithm 1.

Because the proposed algorithm consists of multiple iterations and in order to analyze the total privacy loss, we first prove that adding noise on the value of the loss function in the discriminator ensures the Rényi differential privacy, and then show that the generator can satisfy DP.

Our adaptive noise tuning algorithm follows the idea that, as the training continues, it is expected to have less noise on the loss function, which allows the model converge faster and obtain better results. Similar ideas have been applied by adjusting the learning rate in common practice [21]. Therefore, we propose an algorithm that can adaptively tune the noise scale over the training iterations, and effectively improve the model accuracy in turn.

Our approach for adaptive tuning the noise scale is to monitor the training accuracy and reduce the noise scale when the accuracy stops improving. To do this, we first add an extra testing process after each generator finishes training in each iteration. Every time when the improvement of the testing accuracy is less than a predefined threshold $\tau$, the noise scale is reduced by a factor of $k$ until the total privacy budget runs out. Although this will lead to recalculating privacy cost, we can show the improvement on the performance is convincing despite more energy cost in the experimental results.

In our approach, with a validation dataset, the testing accuracy is checked periodically during the training process in the generator to determine whether the noise scale needs to be reduced for subsequent iterations. Let $\sigma_{t}$ be the noise scale in the $t$th iteration obtained by Theorem 4 when the total privacy budget and total iterations are given, and $S_{t}$ be the corresponding testing accuracy. The noise scale for the subsequent iterations is adjusted based on the accuracy difference between the current and previous iteration.

where $\tau$ denotes the threshold for the testing accuracy.

Initially we set $0<k\leq 1$ and $S_{0}=0$. Then the updated noise scale $\sigma_{t+1}$ will be applied to the next iteration.

In addition, we find that the testing accuracy may not increase monotonically as the training progresses, and its fluctuation may cause unnecessary reduction of noise scale and thus wasting the privacy budget. This motivates us to use the average of testing accuracy to improve the algorithm: at the current testing iteration $t$, we define an average testing accuracy $\overline{S}_{t}$ over the previous iterations as follows:

The average testing value will replace the previous one in Eq.(17) and determine whether the noise scale should be reduced or not.

To sum up, we formally present our adaptive noise tuning DP-GAN framework in the Algorithm 2.

In this subsection, we will list several predefined noise decay schedules, and provide comprehensive comparison results with our proposed algorithm in Sec. 5.5.

The proposed algorithms require a set of pre-defined hyperparameters, such as decay rate $k$. This value will definitely the training time and affect the final model accuracy. Intuitively speaking, a smaller decay rate $k$ will lead to a better performance but longer training iterations. It is expected to find an appropriate decay rate for the schedules under an optimal tradeoff between the learning performance and training. However, this decay rate should be determined by multiple factors such as typical neuron network, training set and other factors. Thus in this subsection, we only propose a straightforward method to obtain an appropriate value. To list $k$ candidates by training $k$ neural networks respectively and directly choose the one that achieves the highest accuracy, accordingly we show the experimental results in Sec. V-E.

In this subsection, we excute the proposed algorithm on the MNIST dataset. In Fig. 4, we first show the real and generated data samples with different noise scale. We also show the results generated by the DP-GAN [12, 11] for comparison, which adds noise on the gradients. From the figure we can find the generated digits with low privacy level can well preserve the characters of the true ones, and with the increase of the privacy level, the quality begins to drop as there are unclear pixels exists. For example, we can find the digit $2$ and digit $8$ are hard to recognize in the generated samples when $\epsilon_{total}=5$. Moreover, the quality generated by DP-GAN is obviously worse than the proposed RDP in both privacy levels.

In order to further verify the qualities of the generated samples, we use an additional classifier to test the accuracy. We show the test accuracy along with the training process (i.e., $1000$ iterations) in the following figures.

From Fig. 5 we can first find that the generated samples without privacy protection, i.e., $\sigma=0$, can achieve relative high accuracy in the classifying. Second, with the increase of the noise scale, the accuracy performance begins to fluctuate, which is brought from the injected noises. In addition, a few collapse cases happen when a large noise is added. For example, we can find in Fig. 5(d) the accuracy always suddenly drops to $0.3$ or less. This is because the large scale of $\sigma$ will probably brings about huge noises to the loss function that the system cannot stand. Therefore, it will lead to unacceptable results in the generated samples, i.e., the digit $8$ in Fig. 4 when $\sigma=20$.

In addition, we compare our algorithm (RDP) with the method that adds noise on the parameters, i.e., DP-GAN in two privacy levels, $\epsilon_{total}=0.5$ and $\epsilon_{total}=5$, respectively. As can be found in Fig. 6, the proposed algorithm has a better performance compared with DP-GAN in both privacy levels, in which the biggest performance gap is around $7\%$ when classifying digit $3$ in the high privacy level. As explained in Sec. 3.1, the superior performance of the proposed RDP-GAN compared with the DP-GAN is because that adding noise on the loss function can bring in more explicit privacy protection.

In this subsection, we show the performance of the proposed algorithm on the Adult dataset.

In Fig. 7 we show the probability mass function (PMF) of the generated attributes. Take the attribute of Age as an example, we use binning to divide the age range into $9$ regions as shown in Fig. 7(a) to increase the training speed,. As can be found in Fig. 7(a), the generated samples can achieve similar distribution with the true ones, while the samples with larger noise make more differences. Moveover, the largest difference is focused on the edge domain. The main reason may because the generator always produces samples that are likely to appear in the true dataset, i.e., the one close the average value. Similar phenomenon can be found in other attributes as well. To further investigate this the quality of the generated samples, we show the absolute average error for these attributes. As can be found in Table. III, when more noises are injected, the absolute average error increases. Thus, the proposed algorithm may not have the capability to achieve high performance in maintaining the character of unique attributes, especially in the tabular dataset. To further enhance this performance, it may need more intelligent design on the generator, which is left as our future work.

To verify the correlation among different attributes, we use a trained classifier to determine whether the income of an individual is larger than $50K$ or not. Experimental results on the comparison of the proposed RDP-GAN with DP-GAN can be found in Fig. 8.

From Fig. 8(a) we can find the classifier is able to obtain a fairly high accuracy trained by the true data samples, which has around $96.4\%$ accuracy. Moreover, compared with DP-GAN, the proposed RDP algorithm can better capture the relationship among attributes in both privacy level. For example, when $\epsilon=0.5$, the performance gap is around $4.1\%$ ($82.5$ v.s. $78.4\%$). In addition, in Fig. 8(b) we trained the classifier using the synthetic data, and test its accuracy using the true data. The results also show that the proposed algorithm can achieve a better performance than DP-GAN at a same privacy level.

In this subsection, we first conduct experiments to find the appropriate decay rate for different algorithms. Specifically, we choose $\epsilon_{total}=0.5$ and $1000$ iterations, which leads to adding $\sigma=10$ to the discriminator in each iteration, and the testing accuracy $\tau$ is set to $0.5$ to prevent model collapsing. In addition, we denote the Time-based decay, Exponential decay and Step decay as Time, Exp and Step, respectively, and the $t^{*}$ in Step is set to $100$. In the following, we show the trend of the testing accuracy with different value of decay rate $k$ in the Adult dataset.

From Fig. 9 and Fig. 10, we can find there exists an optimal decay rate in the current system setting. For example, in the ANT-RDP algorithm, the best accuracy occurs when $k=0.8$ in the Adult dataset, and $k=0.7$ in the Adult dataset, respectively. In addition, the performance of the Time, Step and ANT-RDP schedules seem to have a convex relationship with the value of $k$. Intuitively speaking, it may because that a lower decay rate causes to a longer training time, and it will produce a worse accuracy because most iterations will suffer from noise-added parameters. On the other side, a higher decay rate leads to the noise scale decaying sharply in this case and therefore resulting in an insufficient training time, which may also degrades the accuracy. Moreover, we show a comparable results for these algorithms in Table. IV and Table. V.

From the tables we can find that all the dynamic schedules can achieve a better performance than RDP-GAN, with smaller cost on the training iterations. The reason behind this is that the dynamic algorithm will adjust the scale of the added noise according to the testing performance, and it will cost more privacy budget when noise scale becomes smaller in each iteration, thus leading to less iterations. Moreover, the proposed ANT-RDP can obtain the highest accuracy on both datasets while it does not need the largest iterations. For example, compared with the Exp algorithm, the ANT-RDP has a better accuracy, i.e., there is a $0.4\%$ performance gain in the Adult dataset, and $0.9\%$ in the MNIST dataset, respectively, while needs less iterations, i.e., they are $49$ and $34$ less in each dataset.

Differential privacy (DP) [16] and related algorithms have been widely studied in the literatures. The work [23] laid the theoretical foundations of many DP studies by adding noise to make the maximum change of data related functions. Another related frameworks that adds noise on gradient are [24] and its followed work [25], which studied to obtain information from models with DP requirements using stochastic gradient. In [26], a comprehensive and structured overview of DP data publishing and analysis show several possible future directions and applications. In addition, to tightly analyze the composition, especially on the tails of the privacy loss, the work [18] gave a new privacy definition, named Rényi DP. The Rényi DP is not only a natural generalization of pure DP as it shares many properties with DP, but also has the capability to capture a more accurate aggregate privacy loss by the composition theorem.

Abdadi et al. [14] proposed a differentially private SGD algorithm for deep learning to offer provable privacy guarantees on the output model, and the work [19] used the concentrated DP [13], which is in spirt of RDP, to provide cumulative privacy loss estimation over computations. Moreover, the applications of DP in GAN, such as [11, 12], have well addressed information leakage by adding noises to gradients during training. They also can produce data samples with good quality under reasonable privacy budgets.

Different from these solutions, our work can obtain an more accurate estimation of privacy loss. The properties of RDP is first used to apply in the GAN. Moreover, we provide a naturally succeed method without artificial clipping on the noise-added gradients, which may destroy the learning performance.