Privacy-preserving Generative Framework Against Membership Inference Attacks

Variational Autoencoder[22] (VAE): VAE is a widely used generative framework that consists of an encoder and a decoder, which are cascaded to reconstruct data with pre-defined similarity metrics. The encoder maps data into a latent space, and the decoder maps the encoded latent code back to the data space. The VAE objective function is composed of the reconstruction loss and the prior regularization over the latent code distribution. Formally,

where $z$ denote the latent code, $x$ denotes the input data, $q_{\phi}(z|x)$ is the probabilistic encoder parameterized by $\phi$ which is used to approximate the true posterior, $p_{\theta}(x|z)$ represents the probabilistic decoder parameterized by $\theta$, and $KL(\cdot||\cdot)$ denotes the KL divergence. In practice, $q_{\phi}(z|x)$ is always constrained to be Gaussian distribution and is sampled via the reparameterization trick, which results in a closed-form derivation of the second term.

Differential privacy [9] is one of the most popular paradigm used to deal with information leak in statistical databases. It provides a formal privacy guarantee, ensuring that sensitive information relative to individuals cannot be easily inferred by disclosing answers to aggregate queries.

A mechanism $K$ is $\epsilon$-differential private if for any two adjacent databases $x$, $x^{\prime}$, and any property $Z$, the probability distributions $K(x)$, $K(x^{\prime})$ differ on $Z$ at most by $e^{\epsilon}$, which means $K(x)(Z)\leq e^{\epsilon}K(x^{\prime})(z)$.

We formulate the membership inference attack as a binary classification task where the attacker aims to classify whether a sample has been used to train the victim machine learning model. Formally, we define

where the attack model $\mathcal{A}$ output 1 if the attacker infers that the sample $x$ is included in the training set, and 0 otherwise. $\theta$ denotes the target model parameters while $\mathcal{M}$ represents the general model publishing mechanism, i.e., type of access available to the attacker. For example, $\mathcal{M}$ is an identity function for the white-box access case and can be the inference function for the black-box case.

We depict in Figure1 the proposed system model for a privacy-preserving generative framework to train the target machine learning model against membership inference attacks. At the very outset, the data owner builds an extractor and a reconstructor through their data $D$. Then he adds noise $N$ which satisfy differential privacy to the output $C$ of the extractor and gain synthetic data $D^{\prime}$ from reconstructor. At the same time, if the source data has label originally, the label of synthetic data should be created. In the end, synthetic data could be used to train new models.

More specifically, the data owner first trains a fundamental VAE model $M_{G}$ locally (see 4.2) using the data source $D$. The VAE model maps the original data to the latent space vector through the encoder and then maps the latent space vector back to the original data through the decoder. Because the decoder can rebuild the data in original data through the latent space vector, it shows that the latent space vector contains enough original data information. Some noise (see 4.3) $N$ is added to the latent vector generated by model $M_{G}$ and the data owner leverages the decoder of $M_{G}$ to reconstruct data $D^{\prime}$. If necessary, implement label migration through label-transfer mechanism (see 4.4). Then we could use this synthetic set $D^{\prime}$ to train an application model $M_{A}$.

As shown in Figure2, our basic scheme can be divided into two main phases and an optional phase. The initial stage and the data generation stage are the main stages, and the label generation stage is an optional stage. In addition, all of these stages are executed locally.

1. 1.

   In the initial stage, our main task is to select or generate suitable extractor and reconstructor. In this solution, the VAE model is used as the extractor and reconstructor. As shown in Figure2LABEL:sub@overview1, by training a generator that can generate original data, map the data to the latent space to get $Z$ specified by the encoder in the VAE model and the decoder maps the latent space data $Z$ back to the data space. At this time, we treat encoder as the mapping relationship $A$ from the source data $D$ to the latent space and treat decoder as $A^{-1}$, which means $A$ is reversible.

2. 2.

   In the data generation stage, our main task is to use the existing mapping relationship $A$ and a given privacy budget $\epsilon$ to generate synthetic data that meets the differential privacy requirements and can resist membership inference attacks. In this scheme, we leverage a noise addition mechanism that satisfies $d-privacy$. As shown in Figure2LABEL:sub@overview2, we first map the source data $D$ into the latent space through the mapping relationship $A:A(D)\rightarrow Z$, and use the noise mechanism $M$ to disturb the latent vector $M(Z)\rightarrow Z^{\prime}=Z+N$, and finally, the mapping relationship $A^{-1}$ is used to map $Z^{\prime}$ back to the source data space to obtain $D^{\prime}\leftarrow A^{-1}(Z^{\prime})$. For instance, as shown in Figure3, we assume that our latent space is a two-dimensional space. We firstly process an original data point $x_{1}$ through the VAE encoder to obtain the point $x_{o}$ in latent space and the corresponding variance $\theta_{o}$, where the data point $x_{1}$ is mapped to get the red area $R_{A}$ that satisfies $N(x_{o},\theta_{o})$ distribution in latent space. Then we randomly select a point in the area $R_{A}$ to get $x_{s}$. After that, the noise mechanism obtains the corresponding blue area $L_{o}$ that satisfies the probability density function according to the area $R_{A}$, and we randomly selects ${x_{s}}^{\prime}$ in the noise area $L_{o}$. Then VAE decoder decodes ${x_{s}}^{\prime}$ into data $x^{\prime}$. Note that, $R_{B}$ is the green area that another data $x_{2}$ is mapped in the latent space.

3. 3.

   In the label generation stage, we mainly provide labels for the newly generated data through the label-transfer mechanism if the original data contains labels. Through the existing mapping relationship $A$, as shown in Fig. 2(a), we put the source data $D$ into the latent space: $A(D)\rightarrow Z$. Then we use the data in the latent space and the corresponding labels $(Z,L)$ to train a classification model $M_{c}$ in the latent space. Next, as shown in Fig. 2(b), we use $Z^{\prime}$ in latent space generated in the data generation stage and the label classification model $M_{c}$, and we can get the label $L^{\prime}$ of the new data $D^{\prime}$.

The components of our basic scheme are mainly included: (1) extractor and reconstructor, (2) noise addition mechanism, and (3) label-transfer mechanism. We will separately describe these three components in detail in the next sections.

We first describe the components, Extractor and Reconstructor, in detail of our scheme. In general, the extractor and reconstructor appear in pairs. After the extractor gains the features from the input, the paired reconstructor could recover the data from these features as much as possible. In this paper, we consider the type of input data as images. We adopt the VAE or VAE-structure generative model to extract data features and reconstruct new data because VAEs learn encoders that produce probability distributions over the latent space instead of points in the latent space and are easy to converge. In addition, when we sample from these probability distributions during many training iterations, we effectively show the decoder that the entire area around the distribution’s mean produces output that are similar to the input value. In short, we create a continuous and complete latent space locally and we could generate the new never seen data. However, we need to point out that the limitation of the VAE model. We need to treat the prior probability as the Gaussian form and Gaussian distribution is a bad representation of low-dimensional data. At the same time, if an adversary can access the VAE model, the VAE model is vulnerable under the membership inference attack (shown in [15]).

There are an encoder and a decoder in the VAE machine learning model, and we consider the latent layer’s output as the feature information. We treat the VAE model as a combination of extractor and reconstructor. Each time, even we train a VAE model using the same data set and initial hyper parameters, the parameters of the final generated model is different. So, we could treat the generated VAE model as a stochastic extractor and stochastic reconstructor.

We denote the extractor as $EC:\mathbb{R}^{*}\rightarrow\mathbb{R}^{k}$, which maps an input data $D$ to a k-dimensional real vector $z\in\mathbb{R}^{k}$. We denote VAE model with $EC_{\rm vae,k}$, which means we consider the VAE encoder as the extractor, and the dimension of the latent layer is $k$. As for reconstructor, it is like an invertible function of extractor in this solution.

If the feature vector is created by the paired extractor, the output of the reconstructor is more similar to the input of the extractor. We denote the reconstructor as $DC:\mathbb{R}^{k}\rightarrow\mathbb{R}^{*}$, which maps a k-dimensional real vector $z$ to a synthetic data. And we denote the VAE model with $DC_{\rm vae,k}$, which means we consider the VAE decoder as the reconstructor, and the dimension of the latent layer is k. Therefore, we put the mapping relationship $A$ as $EC_{\rm vae,k}$ and put the inverse mapping $A^{-1}$ as $DC_{\rm vae,k}$.

In this section, we describe the procedure of adding noise in detail. Given the input feature vector $x_{0}$, the private mechanism $K$ performs random in space $\mathbb{R}^{k}$ according to certain probability distributions which can provide differential privacy for $x_{0}$. The following theorem states the privacy guarantee and differential privacy can be generalized to the case of an arbitrary set of secrets $\mathcal{X}$, equipped with a metric $d_{\mathcal{X}}$ and privacy budget $\epsilon$. Note that in the framework of $\epsilon d$-privacy, we can express the privacy of the mechanism itself, on its own domain, without the need to consider a query. In our scenario, using the Hamming metric of standard differential privacy, which aims at completely protecting the value of an individual, would be too strong. We are not interested in completely hiding the data characteristics, since some approximate information needs to be revealed to obtain the required service. Hence, using a privacy level that depends on the Euclidean distance between the latent vectors is a natural choice. Privacy proof of the Theorem 1 is described in the paper[3].

Generating $x$ according to (4) can be achieved as follows in Algorithm 1. We first set $x$ at the origin of the hyper-spherical coordinate system. Then we sample the radial coordinate $r$ according to the marginal distribution and uniformly sampling a point on the unit $(k-1)$-sphere. We multiply the results of them and add them with $x$ to gain the new feature vector $x^{\prime}$. Note that, the marginal distribution satisfies the $\Gamma$-distribution and this proof is shown in Appendix B.

For the VAE model, we set the input feature vector $z$ as the value obtained by sampling from $\rm\mathcal{N}(\mu,\sigma^{2})$ according to the output of Encoder $EC$. Thus, the $k$ is set to be the dimension of $z$. As for the $\epsilon$, because each dimension in the hidden layer is approximated to a standard Gaussian distribution in the well-trained VAE model, we use the empirical rule [21] to estimate the max distance, namely sensitivity according to Definine 2 where $f(x)=x$. We set $\epsilon$ as $\frac{\epsilon}{(3\sigma)}$ where $\epsilon\in(0,1)$ and $\sigma$ is generated by Encoder $EC$.

Sometimes, for semi-supervised or supervised learning models, some source data have labels at the beginning. However, through our generative framework, while the raw data is transformed into new data, the labels of the new data may not correspond one-to-one with the source data. Therefore, during semi-supervised or supervised learning, for labeled source data, we need to use label-transfer algorithms to label new data. In this paper, we adopted a model-driven label-transfer mechanism and the specific algorithm in Algorithm 2.

We first use the feature extractor $EC$ in the framework to extract the feature vectors of the source data set $D$ with labels $L_{D}$ and then train a classification model $M_{c}$ based on the corresponding labels $L_{D}$ and these feature vectors $Z_{T}$. After that, put the previously recorded feature vector $Z_{T}^{\prime}$ used in the newly generated data into this classification model for prediction, and then the label $L_{T}$ corresponding to the newly generated data can be obtained. Note that, this classification model $M_{c}$ is sensitive facing a white-box generator attack.

For the VAE model, we use our label-transfer mechanism after the VAE model is well trained. If we gain the well-trained VAE model, the label-transfer mechanism records the feature vector $z$ generated by the Encoder according to the source data $d\in D$ and the corresponding tag $l\in L$. When the data generative mechanism is running, we record the noised feature vector $z^{\prime}$ and the corresponding generated data $d^{\prime}$. The classification model $M_{c}$ leverages the feature vector $z$ and label $l$ obtained from the source data set for training, and the well-trained classification model $M_{c}$ predicts the label $l^{\prime}$ of the corresponding data $d^{\prime}$ according to the noised feature vector $z^{\prime}$.

In this section, we provide a concise privacy analysis of our framework. In our framework, there are two main stages and we only need to make sure the output of the data generation stage satisfying $\epsilon d_{k}$-privacy privacy. We make the claims that the synthetic data satisfies $\epsilon d_{k}$-privacy privacy. Without loss of generality, we mark the encoder as the mapping relationship $A$ and the decoder as the mapping relationship $A^{-1}$. We process the raw data $x$ through the encoder $EC$ to get $A(x)$ and utilize Algorithm 1 to add noise $n$ to get new latent code $A(x)+n$, at this time $A(x)+n$ satisfies $\epsilon d_{k}$-privacy according to the Theorem 1. Finally, the decoder $DC$ of the VAE model is used to obtain the synthetic data $x^{\prime}=A^{-1}(A(x)+n)$.

We first give the Theorem 2 to show that applying differential privacy mechanism $M$ to the data in the latent layer can also make the decoded data meet the same privacy requirements.

The above properties based on $(\epsilon,\delta)$-differential privacy can also be extended to $\epsilon d_{k}$-privacy, because $\epsilon d_{k}$-privacy is an extension of $(\epsilon,\delta)$-differential privacy. The following Corollary can be obtained.

According to Corollary 1, we know that applying $\epsilon d_{k}$-privacy mechanism $M$ to latent vector can also make the decoded data meet the same privacy requirements and we get Theorem 3.

This part is mainly the setup phase of the experiment, including the databases, models, and measurement for various tests.

We generate new data ${D}$ through our framework on the three datasets of MNIST, Fashion-MNIST and CelebA respectively. And through the following experiments to illustrate the validity of the generated data for training tasks.

We use two membership inference attack methods, namely black-box attack and white-box attack, to attack the target model. For our PPGF, we leverage the generated new data on the three databases to train the target classification model, and utilize the two kinds of membership inference attacks to test. Note that, we choose ResNet-18 model for MNIST, FashionMNIST, CelebA as for the limitation of GPU memory.

The first paper about the membership inference attack was published by Shokri et al. [31] under the assumption of the black-box model. The shadow model was used to train a machine learning model similar to the target model, and the shadow model was used to learn differences of confidence vector about members and non-members. Later, Nasr et al.[27] first proposed the membership inference attack under the white-box model and federated learning model. They classified between members and non-members by distinguishing model data such as the gradient changes of each layer or the output of each layer. Song et al.[33] mainly discussed the practical threat of machine learning in the article. The defensive measures against adversarial samples were to generate a robust machine learning model. But, this robust machine learning model was more sensitive to membership inference attacks. The article [32] set different thresholds, which was used to determine whether the prediction confidence belongs to the member, for a different class label to improve the accuracy of the attack. At the same time, the redefined prediction entropy was used to strengthen the membership inference attack. The paper [40] designed new membership inference algorithms against machine learning models and achieved significantly higher inference accuracy when the augmented data was also used in training but the augmented mechanism was known to the adversary. However, the malicious attacker could only utilize the original image or randomly chosen transformations which yielded a significantly lower inference success rate. There are also some membership inference attacks where the adversary’s advantages are limited. Hui et al.[17] proposed an membership inference attack when the adversary couldn’t collect enough sample with output probabilities and labels as either members or non-members and Choquette et al.[8] introduced label-only membership attacks with the adversary only got access to models’ predicted labels. In addition, there was an new membership inference attack [42] to recommander systems where the adversary could only observe the ordered recommended items.

Membership inference attack defense mechanisms have also developed for many years. As mentioned in the article[29], they used the dropout mechanism and model stacking mechanism to avoid over-fitting of the target model and prevent membership inference attacks. Song et al. [26] introduced adversarial training in the training process, which was abstracted into a maximum and minimum problem. Minimizes the classification loss and maximizes the profit of the inference attack to find the best defensive model against the attack model. Jia et al.[20] studied to fuzzy the confidence score vector to evade the membership classification of the attack model. Wang et al.[36] reduced model storage and computational operation to defend membership inference attacks. Yin et al. [39] set definitions for utility and privacy of target classifier, and formulated the design goal of the defense method as an optimization problem and solved it.

Differential Privacy: For the schemes that apply differential privacy technology to protect privacy, some schemes[2, 35, 18, 19, 28] added noise to the objective function to protect privacy, while others [1, 41, 25] called DP-SGD added noise to the gradient during training. Jayaraman et al. [19] added noise in the objective function, gradient vectors, and output vectors to analyze trade-offs between privacy and utility. Nasr et al. instantiated the hypothetical adversary to analyze the differential private machine learning especially DP-SGD. There are also some schemes[13, 5, 6, 23, 16] that set the goal of protection as a generative model. Hayeset al. [13] first applied the membership inference attack on the generative model and utilized differential privacy during training stage to protect the generative model. Chen et al. [5] Systematically analyzed the membership inference attack on the generative model and made a detailed classification. However, we treat generative model VAE as a building component to protect the downstream task model in this work.

The existing defense schemes basically add noise in the model training process or directly confuse the output results of the model. It is rarely considered to protect the privacy of the source data by processing the raw data.

Some data generation mechanisms[37, 34, 43, 4] satisfying differential privacy have been proposed, all of these works considered that leverage differential noise mechanism in the training process as previously stated and the membership inference attacks were not included. Xie et al. [37] added noise on the gradient of the Wasserstein distance with respect to the training data during the training procedure. The DP-CGAN [34] made the training process private by injecting random Gaussian noise into the optimization process of the discriminator network. DP-GAN [43] was built upon the improved WGAN framework and enforced DP by injecting random noise in updating the discriminator. Chen et al. [4] proposed gradient-sanitized Wasserstein GAN replacing the typical training procedure with the DP-SGD. However, we consider that generating synthetic data to train the target model for source data protection and don’t affect the training process of the generative model. On the one hand, we don’t consider applying the differential privacy mechanism during the training stage because of the model convergence, privacy budget costs in each iteration, and schemes extension. On the other hand, their purpose is to construct generative models satisfying differential privacy to generate lots of new data, while this paper focuses on protecting the target model from membership inference attacks by transforming the original data into similar data that satisfy differential privacy.