PR-PL: A Novel Transfer Learning Framework with
Prototypical Representation based Pairwise Learning
for EEG-Based Emotion Recognition

To make both source and target data satisfy the assumption of independent and identical distribution and obey the same distribution, we characterize the sample features based on a domain adversarial training introduced in DANN. Here, the distribution difference between the source domain and the target domain is alleviated and the sample features with domain invariant properties are characterized. Through this process, the individual differences in EEG signals could be alleviated and the generalization ability of the models could be improved [25, 27, 6, 26]. Specifically, the domain difference between the source domain sample feature $f(X_{s})$ and the target domain sample feature $f(X_{t})$ are minimized by adopting domain adversarial training. Here, $f\left(\cdot\right)$ is the designed feature extractor for extracting the sample features from EEG signals. A discriminator network $d\left(\cdot\right)$ with the parameter $\theta_{d}$ is introduced to distinguish whether the characterized sample features ($f(X_{s})$ or $f(X_{t})$) come from the source domain ($\mathbb{S}$) or the target domain ($\mathbb{T}$). Its loss function is a standard two-category cross-entropy loss function, given as

In the training process, we adopt the end-to-end training method [23], and implement the domain adversarial training by introducing a gradient reversal layer. The feature extractor $f(\cdot)$ maximizes the classification ability to enhance emotion recognition performance and at the same time, the discriminator $d(\cdot)$ minimizes the domain discrimination to reduce the distribution difference between the source domain and target domain. The final domain adversarial training objective function is defined as

which is termed as domain adversarial loss in Fig. 1. Here, $\mathcal{L}_{\text{classifier}}^{s}(\theta_{f})$ is the classification loss to measure the classification ability in source domain. In this paper, $\mathcal{L}_{\text{classifier}}^{s}(\theta_{f})$ will be realized by pairwise learning on source and target domains introduced in Section III-D and III-E below. $\mathcal{L}_{\text{disc}}(\theta_{f},\theta_{d})$ is the adversarial loss for the discriminator to be trained to distinguish the sample features characterized from source and target domains. $\theta_{f}$ and $\theta_{d}$ are the parameters of $f\left(\cdot\right)$ and $d\left(\cdot\right)$. $\lambda$ is a balanced hyperparameter for ensuring the stability of domain adversarial, which is given by a exponential growth method as

Here, $p$ is a factor related to the training round, given by a ratio of the current training round to the maximum training round.

We assume that there exists a prototype for each emotion category. Through prototypical learning, the prototype features are learned to indicate the representation property of every single emotion category. Based on the sample features extracted from different subjects under different emotions, we could consider these sample features are distributed around the prototype features. In other words, for each emotion category, the prototype features could be considered the ”center of mass” of all the sample features. From the perspective of a probability distribution, the prototype feature of an emotion category can be regarded as the mean value of the sample feature distribution of the emotion, and the variance of the distribution is caused by the non-stationary EEG, including but not limited to individual differences. Assume that the sample features under an emotion category $c$ obey the Gaussian distribution $N\left(\mu_{c},\sigma_{c}^{2}\right)$. The prototype features of the emotion category $c$ could be calculated as the mean vector $\mu_{c}$ of the distribution. For the source domain data $\{{X}_{s},{Y}_{s}\}=\left\{\left(x_{i},y_{i}\right)\right\}_{i=0}^{N_{s}}$, the corresponding sample features are characterized by the feature extractor $f\left(\cdot\right)$, given as $f(x_{i})$ (defined in Section III-A). The prototype feature vector of the emotion category $c$ can be calculated by averaging all the sample features that belong to this category, given as

where ${X}_{s}^{c}=\left\{\left(x_{i}^{s},y_{i}^{s}=c\right)\right\}_{i=0}^{N}$ are a collection of source domain data belonging to the emotion category $c$. $\left|{X}_{s}^{c}\right|$ are the corresponding sample size in this emotion category. In other words, $\mu_{c}$ could be expressed as the centroid of the sample features of ${X}_{s}^{c}$. The mean value calculation is a widely used, simple, and effective noise reduction strategy, which can make the prototype features stronger than the sample features and help to alleviate the problem of the traditional DANN network being susceptible to related noise interference [31]. It is worth noting that since the calculation of the prototype features needs to use emotional label information, we only use the source domain data here for prototype feature extraction.

The traditional DANN extracted domain-invariant shared feature representations could be easily contaminated by the shared and related noises in source and target domains. In this paper, we introduce a bilinear interaction to measure the interaction relationships between the sample features and prototype features and extract the interaction features for the following pairwise learning. For a given $d$-dimensional sample feature $f(x_{i})$, its interaction relationship to a certain prototype feature $\mu_{(}c)$ can be measured by a bilinear transformation $h(\cdot)$, defined as

where $S\in{R^{d\times d}}$ is a bilinear transformation matrix with trainable parameters and is not restricted by symmetry or positive definiteness. Suppose there are total $n$ emotion categories, the interaction measurement between a certain sample feature $f\left(x_{i}\right)^{T}$ and different prototype features can be represented as

where $\{\mu_{i}\}_{i=1}^{n}$ are all the prototype features. An introduction of the activation function (softmax) here is to add nonlinear advantages to the interaction measurement, enhance the feature representation ability, and at the same time allow the feature vector $l$ to have category prediction capabilities.

Traditional pointwise learning algorithms often regard the feature vector $l$ as the predicted label of the sample feature $x_{i}$ and use the cross-entropy loss function to match the label $y_{i}$ of $x_{i}$ based on supervised training. Then, if the sample features of one EEG trial match the prototype features of $j$th emotion category the most, the EEG trial will be assigned to the $j$th emotion category. However, this type of pointwise learning only focuses on the relationship between sample features and prototype features and ignores the relationship between different sample features. For example, the sample features belonging to different emotion categories should be separated from each other, and the sample features belonging to the same emotion category should be gathered together. To tackle this issue, we introduce pairwise learning to capture the inherent relationship of samples.

For the source domain data $\{X_{s},Y_{s}\}=\{x_{i}^{s},y_{i}^{s}\}_{i=1}^{N_{s}}$, the corresponding loss function for pairwise learning is defined as

where $g\left({x}_{i}^{s},{x}_{j}^{s};\mathbf{\theta}\right)$ is the similarity measurement of the samples $x_{i}^{s}$ and $x_{j}^{s}$, with the parameter of $\theta$. According to the assumption of pairwise learning, if ${y}_{i}^{s}={y}_{j}^{s}$, then $r_{ij}^{s}=1$; otherwise $r_{ij}^{s}=0$. The loss function $L\left(\cdot\right)$ is a difference calculation of $r_{ij}$ and $g\left({x}_{i}^{s},{x}_{j}^{s};\mathbf{\theta}\right)$, given by a two-category cross-entropy loss as

In the training process, the label information of source domain data is used to define $r$ in a supervised manner. In other words, based on the given information of $Y_{s}$, if two samples belong to the same emotion category, then $r=1$, otherwise $r=0$. A supervised $r$ can ensure the stability of the training process and the generalization ability of the model. The next key question is how to define a proper $g\left({x}_{i}^{s},{x}_{j}^{s};\mathbf{\theta}\right)$ to compute the similarity between $x_{i}^{s}$ and $x_{j}^{s}$ in terms of the characterized interaction features (termed as $l_{i}$ and $l_{j}$). To make the similarity results locate in the range of $\left[0,1\right]$ and extract better and more robust feature representations for subsequent emotion recognition, we add a norm restriction on $l$ as

The similarity of $l_{i}^{norm}$ and $l_{j}^{norm}$ is calculated as the cosine similarity, given as

where $\cdot$ refers to inner product operation. As stated in Chang et al. [32], the above-mentioned norm restriction can make the vector $l$ have a clustering function, and the elements in the vector represent the probability that the feature belongs to a certain category cluster. Overall, the objective function of pairwise learning on the source domain is defined as

which is termed as supervised pairwise loss in Fig. 1. Here, $\theta=\left\{\theta_{f},S\right\}$, $\theta_{f}$ is the parameter of feature extractor $f\left(\cdot\right)$, and $S$ is the defined bilinear transformation matrix for interaction feature extraction. Besides, to avoid redundant feature extraction, a soft regularization $\mathcal{R}$ is introduced with a weight parameter of $\beta$, which is defined as

Here, each row of the matrix $P$ refers to the prototype feature belonging to one emotion category, $\left\|\cdot\right\|_{F}$ is a $F$ norm of the matrix, and $I$ is an identity matrix. The above loss function (Eq. 11) could be interpreted as a clustering loss, instead of emotion category classification loss. The main optimization goal is to gather the EEG samples that may belong to the same emotion category and separate the EEG samples that do not belong to the same category. The vector $l$ is the characterized interaction feature for mapping to a non-linear informative feature space by measuring the interaction relationship between the sample features and all the available prototype features.

In this paper, besides of source domain, we also introduce the pairwise learning on the target domain to improve the feature separability in the target domain, as

which is termed as unsupervised pairwise loss in Fig. 1. Here, $l^{t}$ is the interaction features of the target domain data characterized in Eq. 6. The scalar $r_{ij}^{t}$ symbolizes the pairing relationship of the samples in the target domain. Since the label information of the target domain is completely missing in the training process, we cannot accurately obtain the pairing relationship as the source domain. To address this issue, we introduce an adaptive thresholding method to generate the valid pseudo labels and define the pairing relationship of the target domain data. Suppose that $r_{ij}^{t}$ is defined as

where $\tau_{u}$ and $\tau_{l}$ are the upper and lower bounds to select valid pairs with high confidence for unsupervised pairwise learning (valid pair selection). Here, if the calculated pairwise similarity is higher or equal to the defined upper bound ($\tau_{u}$), then the corresponding pseudo label $r_{ij}^{t}$ would be assigned to 1; while if the calculated pairwise similarity is lower than the defined lower bound ($\tau_{l}$), then the corresponding pseudo label $r_{ij}^{t}$ would be assigned to 0. For the other pairs that do not meet the threshold requirement, we would consider that the model is uncertain about whether the sample pair is paired or not and consider these pairs as invalid results. In order to prevent incorrect optimization, the invalid pairs would be temporarily excluded and will not participate in training and loss calculations at the current training round.

In the early training stage, the classification performance in the target domain is not good enough. In order to ensure the training stability, we set a strict upper threshold ($\tau_{u}$) and a lower threshold ($\tau_{l}$) and exclude most of the pair results in the target domain. Along with the enhancement of model performance in the target domain, we can gradually lower the upper threshold and raise the lower threshold and allow more samples to participate in the training part. In other words, along with the increase in training steps, more pair samples in the target domain will be included for model learning. Here, we form a non-linear dynamic update for thresholding, as

where $\tau_{h}^{t}$ represents the upper threshold of the current training round $t$, $\tau_{l}^{t}$ represents the current lower threshold, and $maxepoch$ is the maximum training round. Based on the given initial values $\tau_{h}^{0}$ and $\tau_{l}^{0}$, the calculations of $\tau_{h}^{t}$ and $\tau_{l}^{t}$ could be considered as non-linear changes with respect to the training rounds as

In all, combining the domain adversarial loss (Eq. 2), the pairwise learning loss in the source domain (Eq. 11, and the pairwise learning loss in the target domain (Eq. 13), the final objective function of PR-PL could be given as follows:

where $\gamma$ is a hyperparameter to control the importance of the pairing loss of the target domain, given as $\gamma=\delta\times\frac{epoch}{maxepoch}$. $epoch$ is the current training round, and $maxepoch$ is the maximum training round. Empirically, $\delta$ is set to 2.

To have a fair comparison with the state-of-the-art methods, we validate our proposed model on two well-known public databases: SEED [13] and SEED-IV [20]. In SEED database [13], a total of 15 subjects were invited. Each subject performed three sessions on different days and each session contained 15 trials. A total of three emotions were elicited (negative, neutral, and positive). In SEED-IV database [20], a total of 15 subjects participated in the experiment. For each subject, a total of 3 sessions were performed on different days and each session contained 24 trials. A total of four emotions were elicited ( happiness, sadness, fear, and neutral). The EEG signals of both SEED and SEED-IV databases were simultaneously collected using the 62-channel ESI Neuroscan system.

For EEG preprocessing, the data sampling rate was first downsampled to 200Hz, and the contaminated noises (e.g. EMG and EOG) were manually removed. Then, the data were filtered by a band-pass filter of 0.3 Hz to 50Hz. For each trial, the data was divided into a number of segments with a length of 1s. Based on the pre-defined five frequency bands: Delta (1-3 Hz), Theta (4-7 Hz), Alpha (8-13 Hz), Beta (14-30 Hz), and Gamma (31-50 Hz), the corresponding differential entropy (DE) features were extracted to represent the logarithm energy spectrum in a specific frequency band and total 310 features (5 frequency band $\times$ 62 channels) were obtained for one EEG segment. Then, all the features were smoothed with the linear dynamic system (LDS) method, which can utilize the time dependency of emotion changes and filter out emotion unrelated and noisy EEG components[33].

In our experiments, the feature extractor $f$ and discriminator $d$ are both made up of multilayer perceptron (MLP) with the Relu activation function. All the parameters are randomly initialized from a uniform distribution. The bilinear operator matrix $S$ is also randomly initialized. In the model architecture, the feature extractor structure is designed as 310 (input layer)-64 (hidden layer 1)-Relu activation-64 (hidden layer 2)-Relu activation-64 (output feature layer). The discriminator structure is designed as 64 (input layer)-64 (hidden layer 1)-Relu activation-dropout layer-64 (hidden layer 2)-1 (output layer)-Sigmoid activation. The size of matrix $S$ given in Eq. 5 is $64\times 64$. Besides, we adopt an RMSprop optimizer for network training, which shows a better performance than the other classic optimizers. The learning rate is set to 1e-3 and the mini-batch size for training is 96. To avoid overfitting problems, we use $L2$ regularizes (1e-5) in the networks. The regularization coefficient $\beta$ in Eq. 11 is 0.01. The balance parameter $\gamma$ for pairwise learning on the target domain in Eq. 18 is controlled by a constant factor $\delta$ of 2. The threshold $\tau_{h}^{0}$ and $\tau_{l}^{0}$ are given to 0.9 and 0.5 respectively. All the models are trained on an NVIDIA GeForce RTX 2080 GPU, with CUDA 10.0 using the Pytorch API.

To fully evaluate the robustness and stability of the proposed model and compare it with the existing literature, we validate PR-PL using four different validation protocols. (1) Cross-subject cross-session leave-one-subject-out cross-validation. We evaluate the model with a strict cross-subject cross-session leave-one-subject-out to fully estimate the model robustness on the unknown subject(s) and session(s). One subject’s all sessions data are used as the target and the remaining subjects’ all sessions are used as the source. We repeat the training validation until each subject’s all sessions are treated as the target for once. Due to the variants in individuals and sessions, this evaluation protocol poses a great challenge to the model’s effectiveness in the EEG-based emotion recognition tasks. (2) Cross-subject single-session leave-one-subject-out cross-validation. It is the most widely used validation scheme in the EEG-based emotion recognition tasks [34, 27, 5, 28]. One subject’s one-session data is treated as the target and the other remaining subjects are used as the source. We repeat the training validation process until each subject is treated as the target for once. Same as the other studies, we only consider the first session in this type of cross-validation. (3) Within-subject cross-session leave-one-session-out cross-validation. Similar to the existing methods, a time-series cross-validation method is adopted here, where the past data is used to predict current or future data. For one subject, the first two sessions are used as the source, and the latter session is used as the target. The average accuracies and standard deviations across subjects are calculated as the final results. (4) Within-subject single-session cross-validation. Following the validation protocol presented in the existing studies [13, 20], for each session of one subject, we use the first 9 (SEED) or 16 (SEED-IV) trials as the source and the rest 6 (SEED) or 8 (SEED-IV) trials as the target. The results are reported as the average performance across all the subjects. In the following performance comparison across four different validation protocols, the model results reproduced by us are indicated by ‘*’.

To verify the model efficiency and stability on both cross-subject and cross-session conditions, we verify the proposed PR-PL using cross-subject cross-session leave-one-subject-out cross-validation on both SEED and SEED-IV databases. As reported in Table II and Table III, the results show our proposed model achieves the highest results, where the emotion recognition performance of PR-PL is 85.56%$\pm$4.78% for three-class classification task on SEED and 74.92%$\pm$7.92% for four-class classification task on SEED-IV. Compared to the existing studies, the proposed PR-PL increases the classification accuracy to 3.39% and 1.08% for SEED and SEED-IV, with smaller standard deviations. These results demonstrate the proposed PR-PL has better affective effectiveness with higher recognition accuracy and better generalizability.

Table IV summarizes the model results in cross-subject single-session leave-one-subject-out recognition task and compare the performance with the literature. All the results are presented in terms mean$\pm$standard deviation. The results show our proposed model (PR-PL) achieves the best performance (93.06%), with a standard deviation of 5.12%. Our PR-PL leads 2.14% against the reported best results in the literature. Especially, compared to the latest proposed DANN based deep transfer learning networks (e.g. ATDD-DANN [26], R2G-STNN[6], BiHDM[44], BiDANN[25], and WGAN-GP[27]), the proposed PR-PL with pairwise learning can avoid the inherent defects of DANN design (e.g. only considers feature separability on source domain) and well address the individual differences and noisy labeling issues in aBCI applications.

By calculating the average and standard deviation of the experimental results of each subject, the final within-subject cross-session cross-validation results are reported in Table V for the SEED database and Table VI for the SEED-IV database. For both databases, our proposed PR-PL achieves the highest recognition performance compared with the state-of-the-art methods (including both traditional machine learning methods and deep learning methods), where the results are 93.18%$\pm$6.55% and 74.62%$\pm$14.15% for SEED (three-class emotion recognition) and SEED-IV (four-class emotion recognition), respectively.

Consistent with the evaluation method presented in the previous studies that only consider the first two sessions of the SEED database for experiments, we present the within-subject single-session results in Table VII. It shows our proposed model obtains the best recognition performance of 94.84%. Comparing the recognition results between cross-subject single-session (Table IV) and within-subject single-session (Table VII) emotion recognition tasks, the proposed PR-PL achieves the highest accuracies and at the same time perform the closest results on the two cross-validation methods (cross-subject single-session: 93.06$\pm$05.12; within-subject single-session: 94.84$\pm$09.16). For the other models, such as DGCNN[45], BiDANN[25], R2G-STNN[6], RGNN[8], and BiHDM[44], there exists a significant difference between cross-subject and within-subject results (9.09% difference on average). This comparison demonstrates the efficiency and reliability of the proposed PR-PL in various emotion recognition applications.

For the SEED-IV database, we calculate the performance on all three sessions as reported in the other studies and decode emotions into four categories (happiness, sadness, fear, and neutral). Our proposed model outperforms the existing studies, with the highest accuracy of 83.33%, which leads to a 3.96% increase as compared to the SOTA (79.37%[8]).

We conduct the ablation study to systematically explore the effectiveness of different components in the proposed model and examine the corresponding contributions to the overall performance. As shown in Table IX, it is found that the introduction of domain adversarial training can greatly enhance the emotion recognition performance on the target domain. When the model is without discriminator and target domain information, the recognition accuracy reduces from 93.06% to 83.30%. Such a significant drop shows the significant impact of individual differences problem on model performance and highlights the great potential of transfer learning in aBCI applications. Besides, the results show a combination of pairwise learning on the source and target domain benefits to the model performance, where the recognition accuracy is increased by 6.35% (from 87.5% to 93.06%). For the pseudo-labeling method, the corresponding accuracy increases from 89.92% to 92.46% when the pseudo-labeling method changes from fixed to linear dynamic. The accuracy further increases to 93.06%, when a nonlinear dynamic-based adaptive pseudo-labeling method is adopted. The results show a non-linear dynamic pseudo-labeling could be helpful to screen out the valid paired samples and improve the model trainability. For the final loss function given in Eq. 18, instead of using a fixed weight for the pairwise loss on the target domain, we propose to update the weight gradually along with the training epochs to prevent model learning failures in the early training stage and balance the relationships among different losses. The benefit of a dynamic $\gamma$ is also reflected in the ablation study, where the recognition accuracy increases from 89.47% and 93.06%.

To further verify the model robustness during noisy label learning, we randomly contaminate the source labels with $\eta$% noises and test the corresponding model performance on unknown target data. Specifically, we replace $\eta$% real labels in $Y_{s}$ using randomly generated labels and train the model in supervised learning. Then, we test the trained model performance on the target domain. Note here that the noisy contamination is only conducted on the source domain, as the target domain needs to be used for model evaluation. In the implementation, $\eta$% value is adjusted to 10%, 20%, and 30%, respectively. The corresponding model accuracies with the standard deviations are 89.22%$\pm$6.05%, 88.39%$\pm$6.73%, and 87.71%$\pm$5.02%. It shows that, with an increase of label noise ratio from 10% to 30%, the model performance decreases slightly, with a decrease rate of 1.69%. These results demonstrate the proposed PR-PL is a reliable model which has a higher tolerance to noisy labels. In the future works, the recently proposed novel methods, such as [52] and [53], could be incorporated to further eliminate more general noises in EEG signals and improve the model stability in the cross-corpus applications.

To qualitatively study the model performance in each emotion class, we visually analyze the confusion matrices and compare results with the latest models [25, 44, 8]. As shown in Fig.2, it shows all the models are good at distinguishing positive emotions from other emotions (the recognition rates are all above 90%), but it is relatively poor at distinguishing negative and neutral emotions. For example, the recognition rate of neural in BiDANN [25] is even lower than 80% (76.72%). Compared to the existing methods ((a), (b), and (c)), our proposed model can enhance the model recognition ability, especially for distinguishing neutral and negative emotions. As shown in (d), the recognition rates for negative, neutral, and positive emotions are 92.10%, 90.39%, and 96.50%.

To verify the effectiveness of the proposed model from a more intuitive perspective, we visualize the characterized sample and interaction features of source and target domains using T-SNE[54] in Fig. 3. Here, we randomly select 500 samples from the source and 500 samples from the target for visualization of the learned feature representation. Compared to the representation learned by the other model settings (w/o pairwise learning on the source and target and w/o pairwise learning on the target), the representation learned by the proposed PR-PL forms more separated emotional clusters. Comparing the extracted sample features (c) and interaction features (f) by the proposed PR-PL, the separability of the extracted interaction features from different emotion classes is further enlarged and at the same time, the concentration of the feature distribution for each emotion is also improved.

The paper proposes a novel transfer learning framework with prototypical representation-based pairwise learning (PR-PL), that characterizes EEG data with prototypical representations and formulates the EEG-based emotion recognition task as pairwise learning. We evaluate our proposed model on two well-known emotional databases (SEED and SEED-IV) under four cross-validation protocols (cross-subject single-session, within-subject single-session, within-subject cross-session, and cross-subject cross-session) and compare it with the existing state-of-the-art methods. Our extensive experimental results show PR-PL achieves the best results on all four cross-validation protocols and demonstrate the advantage of PR-PL in tackling individual differences and noisy labeling issues in aBCI systems.