Data from Model: Extracting Data from Non-robust and Robust Models

In deep learning applications, such as image classification [6, 18], data is used to train deep neural network (DNN) models. This work explores the reverse process of generating data from the model, with one general question in mind: What is the relationship between data and model? This question cannot be addressed well by only focusing on the model training process, Data To Model (DtM) [8]. Thus we combine the widely adopted DtM with its reverse process of Data from Model (DfM). More specifically, we repeat the process of DtM and DfM in sequence and measure the accuracy over the original validation dataset. In this chaining process, we always assume access to only either the data or the model generated in the previous process. More specifically, in the DtM process, we can only access the data generated from the previous DfM process, and similarly, in the DfM process, we only access the model generated by the previous DtM process. This chain process is depicted in Figure 1.

Our work is mainly inspired by [7] which attributes the success of adversarial examples to the existence of non-robust features (mappings) in a dataset. During the typical DtM process, these non-robust features are learned by a model which consequently has the same non-robust features. This indicates the feature mapping as the link between data and model. In their work, the original training dataset is adopted as the background image in the data extraction process [7]. In this work, we explore the possibility of retrieving a learned feature mapping from a trained model without the original training dataset, which makes the DfM process more meaningful. Moreover, we iterate the DtM and DfM process in sequence instead of just performing it once. Another aspect of this work is to explore whether such feature mappings are the same or different for different runs for the same or different architectures.

To decode the learned features of a model into a dataset without knowledge of the original training data, we adopt random substitute datasets as background to increase sample diversity and introduce virtual logits to model the logit behavior of DNNs. Our experiments show that we can obtain models with similar properties as the original model in terms of both accuracy and robustness. We showcase the effectiveness of our approach on MNIST and CIFAR10 for both non-robust and robust origin models.

DNNs are vulnerable to small, imperceptible perturbations [13]. This intriguing phenomenon led to various attack [4, 2, 9, 11, 16] and defense methods [3]. Interpretations for the reason of the existence of adversarial examples have been explored in [4, 14, 17]. Ilyas et al. [7] attributed the phenomenon of adversarial examples to the existence of non-robust features. They introduce features as a mapping relationship from the input to the output. We adopt this definition and aim to extract this feature mapping from the model to the data. Model visualization methods [12, 10] can be seen as a DfM method, without training a new model on the extracted data. Such methods are commonly further exploited for model compression [5, 1] techniques. Instead of compressing models, [15] aims to compress an entire dataset into only a few synthetic images. Training on these few synthetic images, however, leads to a serious performance drop.

Given a $K$-classification dataset $\mathcal{D}$ consisting of data samples $x\in\mathds{R}^{d}$ and their corresponding true class $y\in[1,K]$, a DNN $\mathcal{M}_{\theta}$ ($\theta$ omitted from now on) parameterized through the weights $\theta$ is commonly trained via mini-batch stochastic gradient descent (SGD) to achieve $\operatorname*{arg\,min}_{\theta}\mathds{E}_{(x,y)\sim\mathcal{D}}[\mathcal{L}(\theta,x,y)]$. In this work, we term this process DtM (data to model) and we explore its reverse process of extracting data $\mathcal{D}^{\prime}$ from model $\mathcal{M}$ (DfM). More specifically, starting from origin dataset $\mathcal{D}_{0}$ DtM results in origin model $\mathcal{M}_{0}$, with DfM, $\mathcal{D}_{1}$ can be extracted which leads to $\mathcal{M}_{1}$ through DtM and so on. During the DtM and DfM process, we assume having no access to the previous models and datasets, respectively.

To retrieve features from a model and store them in the form of data, we deploy the $l_{2}$-variant of projected gradient descent (PGD) [9]. Due to the absence of the original dataset, we leverage substitute images $x_{s}$ from a substitute dataset $\mathcal{D}_{s}$, as well as virtual logits $Z_{v}$ as the target values for the gradient-based optimization process. We specify the logit output of a classifier as $Z_{\theta}(\cdot)$ and use the $l_{2}$-loss between the network output logit and the virtual target logit $||Z_{\theta}(x_{s})-Z_{v}||_{2}$ as the loss function optimized by PGD. The retrieved dataset consists of images $x_{s}^{\prime}=x_{s}+\delta$, where $\delta$ indicates a vector optimized through PGD and $x_{s}^{\prime}$ lies in the range $[0,1]$. The data samples $x_{s}^{\prime}$ and their respective output logit values $Z_{\theta}(x_{s}^{\prime})$ represent the new dataset.

After the DfM process, the retrieved dataset can be used in the DtM process, by training the model weights $\theta^{\prime}$ with the $l_{2}$-distance between the previously stored ground truth logit vector and the output logit vector, $||Z_{\theta}(x_{s}^{\prime})-Z_{\theta^{\prime}}(x_{s}^{\prime})||_{2}$.

We heuristically found a simple scheme of Gaussian distributed values $\mathcal{N}(\mu,\sigma)$ for the virtual logits. The highest logit is determined by $\mathcal{N}(20,2)$, and the mean values of the remaining logit values are equally separated between $[-3,3]$ with $\sigma=1$. The order of logit values is chosen randomly to introduce diversity into the dataset.

In the above process, the origin model $\mathcal{M}_{0}$ is non-robust. Following [7] we also use a robust model for $\mathcal{M}_{0}$ obtained with adversarial training and repeat the chaining process.

In this work we introduced the Data from Model (DfM) process, a technique to reverse the conventional model training process, by extracting data back from the model. A model trained on the generated dataset that look totally different from the original dataset can achieve comparable performance as their counterparts trained on the origin dataset. The success of this technique confirmed feature mapping as the link between data and model. Our work provides insight about the relationship between data and model as well understanding of model robustness.