Emotion Eliciting Machine: Emotion Eliciting Conversation Generation
based on Dual Generator

Emotion eliciting chatbots aim at generating a response to improve the emotion status of the user (positive emotion elicitation). Specifically, we consider a dialogue with $N$ pairs of utterances $(U_{1},R_{1},\cdots,U_{i},R_{i},\cdots,U_{N},R_{N})$, where $U_{i}$ is the input message given by the user at the $i$-th turn, and $R_{i}$ is the corresponding response generated by the chatbot. We note $S=\{s_{i}|i\in[1,N]\}$ as the emotional status sequence of the user in the dialogue, where $s_{i}\in[0,1]$ denotes the positivity of the user’s emotion at the $i$-th turn.

Then, the task of positive-emotion eliciting response generation can be formulated as: generating the response $R_{i}$ to improve the user’s emotional positivity $s_{i+1}$ based on the input message $U_{i}$ and previous user status $s_{i}$. To simplify the notation, we use triplet $\{U_{1},R_{1},U_{2}\}$ to denote a consecutive utterance in the dialogue, where $U_{1}$ is the input message from a user, $R_{1}$ is the corresponding response from the chatbot, and $U_{2}$ is the next input message/reaction from the user.

In this paper, we propose leveraging all triplets (not just triplets with $s_{2}>s_{1}$) for training the response generation model. We introduce an emotion eliciting factor on top of a dual-generator structure to generate responses that can balance positive and negative feelings according to the emotion changes. The positive generator aims to emphasize positive emotions during the response generation process, while the negative generator focuses on the negative ones. Our method learns to dynamically balance the influences of the positive and negative generators by an emotion eliciting factor during the training procedure

The detailed structure of the proposed method is shown in Figure 2. At first, we compute an emotion eliciting factor $\lambda$ to capture the emotion changes after the user receives the response $R_{1}$ (Section 3.2). In the training stage, this factor is computed based on the emotion score of $U_{1}$ and $U_{2}$. In the test stage, as $U_{2}$ is unavailable, $\lambda$ can be set arbitrarily to achieve positive emotion elicitation in different degrees. After capturing the emotion changes, we design a dual structure to model the response generation process in both positive and negative side. This structure contains an encoder with dual attentions and a dual decoder. The influence of both attention and decoder is determined by $\lambda$ so as to provide fine-grained emotion elicitation (Section 3.3). In the remaining part of the section, we will introduce each component in detail.

The emotion eliciting factor $\lambda\in\left[0,1\right]$ is a continuous variable that represents the degree of positive emotion elicitation, where 1 indicates the most positive, and 0 is the most negative. Intuitively, the emotion eliciting factor is relevant to: (1) the user’s emotion status before receiving $R_{1}$, namely $s_{1}$ (the status of $U_{1}$); and (2) the changes of the user’s emotion status after receiving $R_{1}$, namely ($s_{2}-s_{1}$).

The computation of the factor $\lambda$ relies on the changes of the user’s emotion status, which is represented by the emotion scores’ difference of the user’s uterrances caused by the chatbot’s response. Specifically, we apply a pre-trained emotion polarity classifier to automatically obtain the emotional positiveness scores $s_{1},s_{2}\in\left[0,1\right]$ of $U_{1}$ and $U_{2}$ to represent the emotion status, respectively. Then $\lambda$ can be computed as:

where the weight $\mu$ is learned by a feed-forward neural network on $s_{2}$ and ${\Delta s}^{\prime}$ to model the implicit interactions between the emotional characteristics of $U_{1}$ and $U_{2}$ as:

where $\sigma\left(\cdot\right)$ is the sigmoid function. $w_{1}$, $w_{2}$, and $b$ are parameters. The computation process of $\lambda$ is shown in the left side of Figure 2. In the training stage, it is determined by the emotional status $s_{1}$ and $s_{2}$, which can be obtained by the pre-trained emotion classifier. By this means, we save lots of resources on human labeling. Our experiments (Section 4.1) will show that these weak labels are highly consistent with human labels, which guarantees the performance of our model. In the inference stage, $\lambda$ can be arbitrarily given to control the polarity and degree of emotion elicitation. For example, to achieve the most effective positive-emotion elicitation in practical applications, we can set $\lambda=1$.

The value of $\lambda$ reflects the degree of positive emotion elicitation in $R_{1}$. We design a dual structure to learn the generation of $R_{1}$ from both positive and negative side with the help of $\lambda$. For example, a more positive emotion eliciting sample (with larger $\lambda$) will contribute more on optimizing the positive generator while the less positive ones have more influence on the negative generator. In such a dual structure, all samples with different $\lambda$s can be used for training, which makes full use of the whole dataset. On the other hand, by tuning the generators in both sides, our model can achieve fine-grained emotion elicitation that is adaptive to the user’s input message.

We conduct experiments on the large scale Reddit dataset³³3https://github.com/PolyAI-LDN/conversational-datasets/tree/master/reddit, which contains 3.7 billion conversation comments with dialogue context Henderson et al. (2019). We extract and preprocess 55.5 million $\left(U_{1},R_{1},U_{2}\right)$ dialogue triplets from the dataset after discarding the triplets which have sentences containing more than 20 words or containing non-ASCII chars. Then we filter out web symbols, normalize the punctuations, and format all the letters as lowercase.

We adopt both automatic and human evaluation metrics to verify the effectiveness of our proposed method.

Due to the complexity and diversity of human language, following existing works Lubis et al. (2018, 2019), we also conduct a human evaluation. Specifically, we randomly sample 200 utterances from the test set. For each utterance, we collect the generated responses from all models. Then we hire five experienced annotators to label them. The annotators are asked to score the generated responses by stating their agreement on two statements: 1) Content score ($s_{c}$) states the response is appropriate and natural to the post and is likely to be produced by a human. 2) Emotional eliciting score ($s_{e}$) states the response elicits a positive emotion. Rating scales of both criteria are 0, 1, and 2, which indicate disagreement, partly agreement and agreement, respectively⁵⁵5The Fleiss’ kappa coefficients for the content score and emotional impact score are 0.473 and 0.631, indicating “Moderate agreement” and “Substantial agreement”, respectively..

We compare our method with the following baselines:

To maximize the effect of positive elicitation in the generated responses, the value of $\lambda$ in Emb_$s_{2}$, Emb_$\Delta s^{\prime}$, and all ablation models is set as 1.0 in the test stage.

The experimental results are shown in the upper side of Table 1. We can see: (1) EEM outperforms all the baselines significantly in terms of positive-emotion elicitation in both automatic and human evaluations. Specifically, EEM outperforms the previous best baseline by 0.114 on both $\hat{s}_{2}$ and $\Delta\hat{s}^{\prime}$. This indicates that EEM can improve the user’s emotion status by generating responses with the most positive effect of elicitation. (2) The values of $\Delta\hat{s}^{\prime}$ obtained by both EnDec and Emo-HRED are negative. The reason is that both of these two methods are originally designed for datasets with only positive eliciting samples. Such datasets heavily rely on human annotation, which is often unavailable in real scenario. (3) To mimic a similar training process, we also train EnDec and Emo-HRED on the positive subset of the data (i.e., $\Delta s^{\prime}>0$ and $s_{2}>0.5$). The corresponding experimental results are shown in Table 1 with an additional mark “on positive”. We can observe that the models can also achieve positive emotion elicitation ($\Delta\hat{s}^{\prime}>0$). However, there is still a large gap between these models and our EEM. This proves that: a) Our proposed emotion eliciting factor can reflect the different degrees of emotion elicitation, thus help the model to achieve fine-grained elicitation and make full use of the dataset; b) Training only on positive eliciting dataset is insufficient for building a positive emotion eliciting chatbot and a well-designed structure is necessary for this task. (4) The effect of positive emotion elicitation by Emb_$s_{2}$ and Emb_$\Delta s^{\prime}$ are very limited. This demonstrates that simply integrating the emotion information into the model is insufficient for generating desirable responses. Similar problems are also reported by Zhou et al. (2018). (5) According to the human evaluation results, EEM also has superiority on the positive emotion elicitation. Interestingly, the general quality of the generated responses (measured by $s_{e}$) is also improved. This inspires us that considering the user’s emotion changes is an effective way to improve the chatbots.

We investigate the effect of each module in our model in this section. The results are shown in the lower side of Table 1.

First, we simplify $\lambda$ in EEM to $s_{2}$ or $\Delta s^{\prime}$, respectively. The performance degradation demonstrates that the emotion eliciting effect is related to both the status of $U_{2}$ and the emotion changes $\Delta s$, which is consistent with our assumption. In addition, compared with the model using $s_{2}$ or $\Delta s^{\prime}$ as additional input (i.e., Emb_$s_{2}$ and Emb_$\Delta s^{\prime}$), EEM achieves better results. This confirms the superiority of our proposed dual structures in emotion elicitation.

Second, we replace the dual attention and the dual decoder by a traditional attention and a tradition decoder respectively. These two models are denoted as “w/o Dual Attn.” and “w/o Dual Dec.”. Experimental results show that both of the dual structures are helpful in improving performance. Specifically, the dual decoder brings more than 14% improvements in terms of emotion elicitation under both automatic and human evaluation. This demonstrates the effectiveness of our proposed dual structure.

Moreover, we conduct additional experiments to investigate the emotion eliciting performance of these models when given different emotion eliciting factors (i.e., $s_{2}$ for Emb_$s_{2}$, $\Delta s^{\prime}$ for Emb_$\Delta s^{\prime}$ and $\lambda$ for EEM) for the generation. As shown in Figure 4, the curves of Emb_$s_{2}$, Emb_$\Delta s^{\prime}$ and EEM show a positive correlation between $\Delta\hat{s}^{\prime}$ and the factor. In general, when given a bigger factor, these models tend to generate more positive eliciting responses accordingly. Among all the models, EEM shows the highest responsivity between $\Delta\hat{s}^{\prime}$ and the factor, which proves that EEM is more controllable and flexible in fine-grained emotion elicitation.

We also provide an example of generated responses in Table 2. As can be seen, compared with the responses generated by EnDec and Emo-HRED, the responses generated by EEM with positive settings (i.e., $\lambda$s $>0.5$) are more user-concerned and achieve more positive emotion elicitation. Though some generated responses are not full of positive emotions themselves, they express a degree of empathy to effectively improve the emotional status of users. Besides, we present responses generated with different $\lambda$ (from 0.0 to 1.0 with an interval of 0.2). As can be seen, EEM can generate responses with different categories of emotion elicitation, and all the responses are appropriate to the inputs. This implies the controllability and flexibility of EEM in emotion elicitation.

The implementation details, the distribution of emotion scores over the dataset, the visualization of the dual attention, and more cases are provided in Appendix.

Our model is implemented by PyTorch P. et al. (2019). The encoders and decoders are both 2-layer GRUs with 600 hidden neurons in each layer. The word embedding dimension is set as 300, and the vocabulary size is set as 30,000. The trainable parameters are optimized by the Adam optimizer K. and B. (2015) with batch size and learning rate as 512 and 1e-4, respectively. All models are trained in five epochs. In the inference stage, beam search is applied for decoding responses with the beam size as five.

200 utterances are randomly sampled for manual evaluation, and the emotion distribution is consistent with the whole dataset (the KL-divergence is 0.023). Each utterance is labeled as “positive”, “neutral” or “negative” according to its emotion polarity by 5 annotators. Since the emotion score predicted by the classifier is continuous, we discrete it into “negative” (score between 0 and 0.35), “neutral” (score between 0.35 and 0.65), and “positive” (score between 0.65 and 1). As a result, the emotion classifier achieves 0.71 accuracy on average. The Fleiss’ kappa coefficient Fleiss (1971) of the 5 annotations is 0.592, indicating “Moderate agreement”.

The distribution of the emotion scores is shown in Figure 5. The ratio of the “nagative”, “neutral”, and “positive” data are around 47%, 34%, and 19%, respectively. If we follow existing work Hasegawa et al. (2013); Lubis et al. (2018, 2019) and generate binary labels for the data, we can only obtain 36.67% data with positive emotion (i.e., the emotion score $>0.5$). In other words, more than 60% data in the dataset cannot be used for training, which is a potential reason for their weak performance.

To demonstrate the effectiveness and interpretability of the dual attention mechanism, we visualize the attention weights of the positive attention and the negative attention in the decoding process. The results are illustrated in Figure 6.

In this case, we input the message “Can you help me?” and set up different values of emotion eliciting factor $\lambda$ for EEM. As can be seen in subfigure (a) and (c), when $\lambda$ is set as 1.0, the positive attention is fully activated and it mainly focuses on “help”, “you” and “me” in the input. The combination of these words implies the user’s emotional demand (seeking help). Therefore, the activation of the positive attention contributes to generating an user-oriented and positive-emotion eliciting response as “Sure, I can help you out with that”.

On the other hand, when $\lambda$ is set as 0.0, the negative attention drives the decoder to focus on “can” and “me” (as shown in subfigure (b) and (d)), which implies rationality but more self-orientation and less relevance to the user’s emotional status. As a result, the model will generate an emotionally suppressing response “I don’t know if you can help me”.

Based on these cases, we can see that our EEM can generate different responses according to the emotion eliciting factor $\lambda$ by activating different modules in the dual structure. Therefore, it is better at controlling the effect of positive emotion elicitation during the response generation process.

In addtion to the case we shown in Section 4.6, we provide two more cases in Table 3. Similarly, EEM with $\lambda=1$ performs best on positive emotion elicitation.

In particular, in the second case, EEM with $\lambda=1$ generates a response “I like to play with you!”. Compared with responses generated by other models, tne personal pronoun “you” reduce the sense of distance between the chatbot and the user, thus can improve the user’s emotion status directly. The responses generated by the baseline methods are fluent and naturally, but they have less effect on user’s emotion (for example, “I don’t know what you’re talking about” generated by Emo-HRED in the first case). On the other hand, when $\lambda$ decreases from 1.0 to 0.0, it is evident that the emotion status of the generated response changed from positive to negative. This demonstrates that our proposed EEM can achieve a fine-grained controlling on positive emotion elicitation.