Towards Strengthening Deep Learning-based Side Channel Attacks with Mixup

The great majority of work in the DL-based SCA community has sought to promote neural network architectures, preprocess traces or put forward new metrics, and then mounts a profiled SCA with an entire training set of some public datasets, anticipating the emergence of more efficient attacks. Reference [21] interpreted the role of hyperparameters through some specific visualization techniques involving Weight Visualization, Gradient Visualization and Heatmap, and subsequently developed a methodology to systematically build well-performing neural networks. Reference [22] improved upon the work from [21], where the authors indicated several misconceptions in the preceding paper and displayed how to obtain comparable attack performance with remarkably smaller network architectures. As illustrated in [10], some types of hiding countermeasures were considered as noise and then the denoising autoencoder technique was used to remove that noise in traces. Whereas [23] demonstrated and compared several methodologies to transform 1D power traces into 2D images. Reference [24] proposed a novel evaluation metric, called Cross Entropy Ratio (CER), that is closely connected to commonly employed side channel metrics and can be applied to evaluate the performance of DL models for SCA. They further adapted the CER metric to a new loss function to adopt in the training process, which can yield enhanced attack performance when dealing with imbalanced data. Another innovative loss function, Ranking Loss (RkL), was proposed by adapting the ”Learning to Rank” approach in the Information Retrieval field to the side-channel context [17], which helps prevent approximation and estimation errors, induced by the typical cross-entropy loss.

In [25], Picek et al. emphasized the noticeable gap between hypothetical unbounded power and real finite power of an attacker. They considered a restricted setting where only limited measurements are available to the attacker for profiling and proposed a new framework, called the Restricted Attacker framework, to enforce attackers truly conduct the most powerful attack possible. Data augmentation (DA), which consists of a set of techniques that increase the size and quality of profiling sets, arises initially for computer vision tasks [26] and has been applied to SCA recently. In [13], Cagli et al. augmented the training set by a simulated clock jitter effect. For the sake of eliminating the curse of data imbalance, [14] employed The Synthetic Minority Oversampling Technique (SMOTE), which is a category of DA technique, to balance the class distribution of the training set. In this paper, we embrace another kind of DA technique—mixup, an uncomplicated learning principle proposed to mitigate unacceptable behaviors such as memorization to adversarial examples in large deep neural networks [27], aiming at alleviating the trouble in a restricted position.

Our contributions are chiefly summarized as follows.

1. 1.

   We investigate a DA technique, called mixup, and first propose to exploit mixup in practical SCA context to expand insufficient profiling set and enhance attack performance;

2. 2.

   We compare three leakage models and surprisingly find that least significant bit (LSB) model performs much better than another two models, which is unexpected since this model is almost ignored in related works;

3. 3.

   We apply mixup to some publicly available datasets and test several mixup parameter values and confirm that this technique can truly facilitate launching a successful DL-based SCA especially in restricted situations where profiling traces are inadequate.

The paper is organized as follows: Section II provides the notations and preliminaries. Section III introduces the application of mixup in DL-based profiled attacks and discusses the effect of chosen leakage models. In section IV, we display and analyse the experimental results on some datasets to illustrate this methodology’s performance. Section V summarizes the paper and gives insights on possible future work.

Throughout this paper, we use calligraphic letter $\mathcal{X}$ to denote sets, the corresponding upper-case letter $X$ to denote random variables (resp. random vectors $\mathbf{X}$) over $\mathcal{X}$, and the corresponding lower-case letter $x$ (resp. $\mathbf{x}$) to denote realizations of $X$ (resp. $\mathbf{X}$). In SCA, the random variable is constructed as $X\in\mathbb{R}^{1\times D}$, where $D$ denotes the dimension of the traces. The $i$-th entry of a vector $\mathbf{x}$ is denoted by $\mathbf{x}[i]$. We use $f$ to denote a cryptographic primitive which computes a sensitive intermediate $Z=f(P,K)$, where $P$ denotes a part of the public variable (e.g. plaintext or ciphertext) and $K$ denotes some chunk of the secret key that the attacker aims to retrieve. We denote $k^{*}$ as the true key of the cryptographic algorithm and $k$ as a key candidate that takes value from the keyspace $\mathcal{K}$.

There exist two phases in a profiled attack: profiling phase and attacking phase, which correspond to the training stage and testing stage in a supervised learning task. In the profiling phase, the attacker generates a model $F:\mathbb{R}^{D}\rightarrow\mathbb{R}^{|\mathcal{Z}|}$ from a profiled set $\mathcal{T}=\left\{\left(\mathbf{t}_{0},z_{0}\right),\ldots,\left(\mathbf{t}_{N_{p}-1},z_{N_{p}-1}\right)\right\}$ of size $N_{p}$, that estimates the probability $\operatorname{Pr}[\mathbf{T}|Z=z]$. In deep learning, the goal of the training stage is to learn the neural network’s trainable parameters $\boldsymbol{\theta}^{\prime}$, which minimize the empirical error represented by a loss function $\ell$:

Here, we regard the profiled SCA as a $c$-classification task, where $c$ denotes the number of classes that depends on the leakage model. Additionally, we generally represent classes with one-hot encoding, i.e., each class is turned into a vector of $c$ values. Only one of $c$ values equals 1 and others equal 0, which denotes the membership of that class. The most common loss function in deep learning is the categorical cross-entropy (CCE):

where the model $F_{\boldsymbol{\theta}}$ yields a probability vector for each input trace, consisting of the chances of each class to be selected, which is accomplished through the softmax layer.

During the attacking phase, the trained model would help predict the sensitive variables, hence the attacker can use an attack set of size $N_{a}$ to compute a score vector with the knowledge of the inputs for each key candidate. Actually, for each $k\in\mathcal{K}$, the log likelihood score is computed by

Afterwards, the key hypothesis with the highest score is assumed to be the correct key.

In the side channel community, success rate and key rank are two ordinarily used metrics for evaluating attack performance, and we adopt the latter in this paper. As illustrated in (3), the attacker calculates a score for each key candidate, based on which he can sort all key candidates in decreasing order and acquire a key guess vector, denoted as $\mathbf{g}=\left(g_{1},g_{2},\ldots,g_{|\mathcal{K}|}\right)$. The key rank is defined as the position of $k^{*}$ in this vector, i.e.,

In practice, we generally repeat the attack procedure certain times and take the average key rank as the evaluation metric. Naturally, we consider $g_{1}$ as the most possible key candidate and $g_{|\mathcal{K}|}$ as the least possible one. Therefore $g_{1}$ corresponds to $k^{*}$ indicates that key rank equals 0 and the attack is successful. Intuitively, we intend to achieve key rank equal to 0 with attack traces as few as possible.

This paper considers four publicly available datasets for subsequent experiments. In this section, we give some details about the employed datasets.

A supervised learning task proceeds with the intent of searching a function $F\in\mathcal{F}$ to characterize the relation between an input vector $\mathbf{X}$ and a label vector $\mathbf{Y}$, that satisfy the joint distribution $Pr(\mathbf{X},\mathbf{Y})$. With the training dataset $\mathcal{D}=\left\{\left(\mathbf{x}_{i},\mathbf{y}_{i}\right)\right\}_{i=1}^{n},$ where $\left(\mathbf{x}_{i},\mathbf{y}_{i}\right)\sim Pr$, the empirical distribution $Pr$ may be estimated by

where $\delta\left(\mathbf{X}=\mathbf{x}_{i},\mathbf{Y}=\mathbf{y}_{i}\right)$ is a Dirac mass centered at $\left(\mathbf{x}_{i},\mathbf{y}_{i}\right)$. Then we can approximate the expected risk as the empirical risk:

where $\ell$ is the loss function. The Empirical Risk Minimization (ERM) principle refers to learning $f$ by minimizing (8) [27].

Notwithstanding the commonality of the ERM, there are still some alternatives, such as the Vicinal Risk Minimization (VRM) principle [27], in which the distribution $Pr$ is estimated by

where $\nu$ is a vicinity distribution that quantifies the eventuality of finding the virtual input-label touple $(\tilde{\mathbf{X}},\tilde{\mathbf{Y}})$ in the vicinity of the training pair $\left(\mathbf{x}_{i},\mathbf{y}_{i}\right)$. Analogous to the ERM, the VRM minimizes the empirical vicinal risk instead with the generated dataset $\mathcal{D}_{\nu}:=\left\{\left(\tilde{\mathbf{x}}_{i},\tilde{\mathbf{y}}_{i}\right)\right\}_{i=1}^{m}$:

Particularly, mixup is a generic vicinal distribution:

where $\lambda\sim\operatorname{Beta}(\alpha,\alpha),$ for $\alpha\in(0,\infty)$. Consequently, following this principle we can yield virtual input-label vectors

where $\left(\mathbf{x}_{i},\mathbf{y}_{i}\right)$ and $\left(\mathbf{x}_{j},\mathbf{y}_{j}\right)$ are two input-label pairs randomly picked up from training data, $\mathbf{y}_{i}$ and $\mathbf{y}_{j}$ take the form of one-hot encoding and $\lambda\in[0,1]$. The crucial parameter $\alpha$ regulates the intensity of interpolation between input-label vectors, reinstating the ERM when $\alpha\rightarrow 0$.

The mixup principle is a pattern of data augmentation technique that facilitates the model $f$ to behave linearly amid training examples. And this linear practice is supposed to diminish the unsatisfactory prediction fluctuations as confronted with examples distributed differently from training data. Besides, reflecting on Occam’s Razor, linearity appears to be a reasonable inductive bias on account of its universality as one of the simplest possible behaviors. An extensive evaluation was conducted in [27] and the authors concluded that mixup reduced the generalization error of state-of-the-art models on ImageNet, CIFAR, speech, and tabular datasets. Given the advantages of mixup in both 1D and 2D data, we speculate that this technique may adapt to side channel measurements as well.

The general design of enhancing DL-based SCA with mixup is illustrated in Fig. 1. We have always been underlining that our jumping-off point is to mitigate the issues caused by restricted measurements. From this perspective, we divide out merely a portion of profiling traces to simulate the constrained scenario. For each dataset, we respectively pick up 3000, 5000, 10000 and all measurements from the training set at random, to compare the performance of original dataset and augmented dataset in distinct settings, which we will expatiate in section IV. Afterwards, we perform mixup to the selected traces. Actually, we have considered combining more than two input-label touples, which means the training pair are generated according to

$\tilde{\mathbf{x}}=\lambda_{1}\mathbf{x}_{1}+\lambda_{2}\mathbf{x}_{2}+...+(1-\sum\limits_{i=1}^{k-1})\mathbf{x}_{k}$,

$\tilde{\mathbf{y}}=\lambda_{1}\mathbf{y}_{1}+\lambda_{2}\mathbf{y}_{2}+...+(1-\sum\limits_{i=1}^{k-1})\mathbf{y}_{k}$,

where $\left(\mathbf{x}_{i},\mathbf{y}_{i}\right),i=1,...,k$, are $k$ random input-label pairs in the profiling set, $\mathbf{x}_{i}$ denotes a power measurement in SCA context and $\lambda_{i}\in[0,1]$ for $i=1,...,k-1$. However, [27] pointed out that convex combinations of three (or more) samples with weights following Dirichlet distribution, i.e., multivariate Beta distribution, does not earn further gain, but increases the computation cost of mixup on the contrary. In this way, we still joint input-label vectors pairwise in light of (12), and then merge initial and generated input-label pairs, which engenders an expanded dataset half larger than the original training set. As for the variable $\alpha$, we first set $\alpha$ to 0.5. Afterwards, we test 0.3 and 0.7 in some situations to complete the experiments.

After obtaining an enlarged training set by implementing mixup, we train two CNNs with original dataset and new mixup dataset, respectively, in the premise of unchanged network architecture and hyperparameters. Once the training process is accomplished, for each neural network, we select the finest epoch where model capability attains the best. Hereafter, still for each selected best model, the test procedure is repeated 50 times and the average key rank is calculated to evaluate attack performance, thereupon allowing us to contrast the power of original and new profiling set.

Similar to the applications of mixup in speech and image datasets, mixup can generate virtual side channel measurements linearly. In this way, the generalization error is supposed to be decreased. On the other hand, the profiling set is augmented, which encourages to train the network comprehensively especially in restricted situations, such as the circumstance where there are merely 3000 profiling traces.

As stated in section III-B, we exploit mixup to generate virtual traces. Naturally, there is a demand to explore whether the generated traces are appropriate. With regards to this, we carry out CPA analysis of initial and generated training set for several public datasets. Fig. 2 illustrates the CPA analysis of the 50000 original profiling traces and the 25000 generated profiling traces for ASCAD dataset. Intuitively, these two CPA results are exceedingly indistinguishable in terms of the leakage location and the correlation intensity. Meticulously observing the figure, we notice that for the generated dataset the correlation at the highest leakage position is even somewhat higher, which signifies that the knowledge involved in generated traces is consistent with that of original traces. In view of CPA analysis, it seems considerably sensible to fulfill the anticipation of mounting a more efficient attack by employing mixup to augment the profiling set.

The CPA analyses of the 50000 original profiling traces and the 25000 generated profiling traces for ASCAD_desync50 dataset and ASCAD_desync100 dataset are respectively displayed in Fig. 3 and Fig. 4. For AES_RD dataset, Fig. 5 demonstrates the CPA results of 40000 original traces and the 20000 generated traces. In all three datasets, there exists trace misalignment, which can be recognized from the diffusely distributed spires in the figures. Notwithstanding, at peaks with different strengths, the CPA results of original dataset and generated dataset are corresponding, which exhibits the justifiability of mixup once again.

General leakage models include least significant bit (LSB) model, hamming weight (HW) model, identity (ID) model, etc. LSB model supposes the least significant bit of the targeted Sbox output to be the label, HW model calculates the hamming weight of the intermediate value while ID model straight considers the sensitive value itself as the label. We conduct tentative experiments with these leakage models and surprisingly discover that for all datasets LSB model performs immensely superiorly, which is slightly counter-intuitive since this model is practically left out in related works. Fig. 6 shows the average key rank results for ASCAD dataset when different leakage models are employed, where all of 50000 profiling traces are fed into the network and the mixup parameter $\alpha$ equals 0.5. In each figure, the orange line represents the attacking results for mixup training set whereas the blue line stands for the original profiling set. With respect to the network architecture, we exclusively adopt the CNN proposed in [12], an uncomplicated CNN designed from VGG-16 fashion, that comprises five convolution layers, five following pooling layers and three fully-connected layers, the final one of which is the softmax layer aiming at figuring out the probabilities for each class. Other hyperparameters, including loss function, optimizer, learning rate, are correspondingly specified as categorical cross entropy, RMSprop and 0.00001. For all experiments in this paper, the CNN structure and hyperparameters stand unaltered apart from the epoch, for which we select the best one to cease training in every scenario, as claimed in section III-B. The selected finest epochs in this section can be consulted from Appendix A.

From Fig. 6 it is investigated that the attack performance is evidently better with LSB model. Approximately 1250 traces and 1500 traces are required to achieve average key rank 0 for ID model and HW model, respectively, however, roughly 200 traces are adequate to reach the same condition for LSB model when mixup is adopted. On the other hand, it appears that when using ID model and HW model, mixup does not bring about any performance gain. We suppose that this is because 50000 profiling traces are already fairly sufficient and ASCAD dataset is not a particularly thorny dataset to DL_based SCA. Subsequent experimental results demonstrate that when the dataset is tougher to handle or the amount of profiling traces becomes restricted, mixup will play a more crucial role in enhancing attack performance. Note that this section is especially emphasizing the ascendancy of LSB model.

Fig. 7 depicts the attack results for ASCAD_desync50 dataset and three leakage models, also with the entire training set. Regarding ID model, mixup boosts the results to a great extent then about 4000 measurements are needed to launch a successful SCA. As to HW model, mixup does not promote the results apparently and we demand less than 2000 traces in both scenarios. Nevertheless, turning to LSB model, merely less than 300 traces are enough to realize an average rank of 0 for the network trained with mixup dataset. The experimental results for ASCAD_desync100 dataset are depicted in Fig. 8, which is similar to Fig. 7. Under three leakage models, mixup yields performance improvement in some degree and the required amount of traces are correspondingly about 7000, 2200 and 300 for ID, HW and LSB models. AES_RD is the easiest dataset, which is implied by the average rank results depicted in Fig. 9. This dataset is considerably simple so mixup is barely influential for ID model since all 40000 traces are utilized to train. In addition, to break AES_RD, 800, 150 and 8 traces are needed under the distinct leakage models. All these results explicitly indicate that LSB has a noticeable advantage among these leakage models. We have not figured out the exact justification to account for this phenomenon. But we can absorb some inspirations from [29], which proposed a multi-label classification model, essentially the naive ensemble pattern of eight single-bit models, and acquired remarkable results for ASCAD even in high-desynchronization situations. We conjecture that the multi-label model excellence is partially associated with LSB as least significant bit model transcends all other seven single-bit models in their experiments [29]. Besides, our average rank results are approaching theirs although we just apply LSB. On the whole, we decide LSB model to be the option in the following experiments.

According to the general design described in section III-B, we use 3000, 5000, 10000 and all profiling traces to perform mixup, where the parameter $\alpha$ is set to 0.5. Table I manifests the average rank results for ASCAD when different numbers of original and mixup profiling traces are used. The determined epochs to early-stop are also tabulated in Appendix A. Besides, $\alpha$ in sections IV-C - IV-D keeps unchanged and the corresponding epochs for experiments in sections IV-C - IV-D are also displayed in Appendix A.

From Table I it is observed that the performance gain is more apparent for inadequate profiling traces, which is reflected in the gap of 540 and the gap of 300 attacking traces for 3000 and 5000 original profiling traces, respectively. When the size of training set increases, there appears a decline in the performance gap, nonetheless the mixup training set still outpaces the original one to some extent. This is readily comprehensible since to train a neural network with multiple parameters, only with sufficient training data can the network extract appropriate knowledge from features to map a never seen input vector to a proper label vector. Furthermore, section III-C has revealed the consistency of mixup traces and original traces in terms of leakage information. Consequently, in restricted scenarios, mixup dataset can exactly relieve an insufficient predicament, where underfitting happens, thus encouraging a successful attack. On the other hand, in comparatively sufficient situations, mixup dataset further favors comprehensive learning on the existing basis and yields modest performance enhancement.

Next, we shift the attention to ASCAD datasets with desynchronization. In Table II, we depict the average rank results for ASCAD_desync50 when a different number of original and mixup profiling traces are used, which is somewhat similar to the results in section IV-B. For 3000, 5000, 10000 and 50000 original training traces, the reduction of required attacking traces induced by mixup is correspondingly 357, 381, 143 and 100. We suppose this is perhaps because CNNs is advantageous in processing asynchronous data and desynchronization of 50 is still quite within CNNs’ reach.

When it turns to ASCAD_desync100 dataset, Table III display our experimental results. In such a high-desynchronization scenario, successfully mounting SCA is notably harder, especially when the size of profiling set is small. It is shown that if barely 3000 original profiling traces are available, 6279 attacking traces are needed to achieve an average key rank equal to 0, whereas with mixup 3453 measurements are already adequate to break ASCAD_desync100, which means the attack performance has approximately doubled. Likewise, for 5000 original profiling traces, the size of required attacking set drops from 3618 to 1753 owing to mixup. On account of the high desynchronization, unlike the aforementioned two datasets, now 10000 profiling traces are not quite sufficient and mixup brings about an improvement of 654 attacking traces. In light of these results, we deduce that especially for troublesome conditions, i.e., inadequate profiling traces or difficult datasets, mixup is more capable of exerting the advantages.

Finally, we illustrate the average rank results for AES_RD in Table IV. Despite the existence of desynchronization, this dataset is fairly easy, which is clearly implied by the small quantity of required attacking traces in all settings. Even though there are merely 3000 original profiling traces for training the network, 35 attacking traces can already break AES_RD, yet mixup facilitates a more efficient attack, where only 19 measurements are enough. When the entire training set is used, respectively 10 and 8 traces can reach an average rank of 0 for original dataset and mixup dataset. In another two settings, mixup produces some gain of a few attacking traces as well, which again confirms that mixup is favorable for enhancing attack performance.

For experiments in sections IV-A - IV-D, we always set $\alpha$ to 0.5. To investigate the impact of $\alpha$ in mixup, we also try out other values including 0.3 and 0.7. In this section, we just select 3000 and 50000 traces from original training set at random to perform mixup. In Table V, we comprehensively compare key rank results for four datasets when $\alpha$ is set to 0.3, 0.5, 0.7 and mixup is not exploited.

For ASCAD dataset, we find that when $\alpha$ equals to 0.3, the brought performance gain even slightly outstrips the case where $\alpha$ is setting to 0.5. Specifically, for $\alpha$ equal to 0.3, when there are 3000 original profiling traces, mixup reduces the amount of attacking traces from 1402 to 815; when all profiling traces are utilized, mixup brings down the number from 299 to 157. While the performance gain engendered by $\alpha$ equal to 0.5 in these two settings are separately 540 and 107, as stated in section IV-B. On the other hand, $\alpha$ set to 0.7 performs a bit poorer than 0.5, where the attacking trace decreasements are 431 and 99 correspondingly in two cases.

For ASCAD_desync50 dataset, when using 3000 original profiling traces, the results of mixup dataset are quite weird since we even demand more attacking traces compared to original profiling set and this phenomenon is more serious for $\alpha$ equal to 0.7. As for the 50000 profiling trace case, it turns a bit normal as $\alpha$ equal to 0.3 and 0.7 respectively yields performance increase of 148 and 135 measurements, both moderately superior to the increase of 100 measurements produced by $\alpha$ equal to 0.5.

For ASCAD_desync100 dataset, when $\alpha$ is set to 0.3, the performance enhancement is similar to the 0.5 case. While for $\alpha$ set to 0.7, if 3000 original traces are available, the enhancement is surprisingly remarkable, which means a reduction of measurements from 6279 to 1606, much more powerful than a gap of 2826 measurements in 0.5 case. If we use 50000 original traces, the enhancement becomes neglectable contrarily. For AES_RD dataset, three values of $\alpha$ perform analogously, where $\alpha$ equal to 0.5 somewhat exceeds another two settings. Concluding the results we claim that in general all these three $\alpha$ are able to favor more efficient SCA and $\alpha$ equal to 0.5 performs most steadily despite $\alpha$ set to 0.3 and 0.7 can bring about more significant performance gain in some circumstances. Therefore, we still recommend 0.5 for mixup parameter. Certainly, how other values of $\alpha$ influence attack performance remains to be explored, which we may investigate in the future.