Dual Task Framework for Improving Persona-grounded Dialogue Dataset

The goal of the Persona-Chat task (Zhang et al. 2018) is to personalize dialogue agents by grounding their utterances to the given personas. The original dataset involves dialogues between pairs of speakers: each speaker is given a hypothetical profile, which is a few persona sentences that describe a character they will imitate, and is instructed to get to know the other. Formally, a profile is defined as a set of persona sentences $\mbox{${\cal P}$}=\{p_{1},...,p_{m}\}$ and a dialogue is defined as a set of multi-turn utterances $\mbox{${\cal U}$}=\{u_{1},...,u_{n}\}$, and a Persona-Chat dataset $\mathcal{D}_{\textsc{Chat}}=\{(\mbox{${\cal P}$}_{i},\mbox{${\cal U}$}_{i})\}_{i=1}^{N}$ where $N$ is the number of dialogues. Based on $\mathcal{D}_{\textsc{Chat}}$, when given persona sentences ${\cal P}$, dialogue history ${\cal U}$, and response space $\mathbb{U}$, the primal task aims at finding a model $f:(\mbox{${\cal P}$},\mbox{${\cal U}$})\mapsto\mathbb{U}$.

where $\theta_{\textsc{Chat}}$ is the parameters to be learned for model $f_{\theta}$. In our setting, we adopt the response selection task (Humeau et al. 2020). As response space $\mathbb{U}$, the model has to pick the correct response from a set of 20 choices, where the remaining 19 were randomly chosen utterances from the evaluation set. Note that in a final system, however, one would retrieve from the entire training set of over 100k utterances, but this is avoided for speed reasons in common evaluation setups (Wolf et al. 2019; Humeau et al. 2020).

Following the primal-dual structure, the Persona-Link task can be defined as: given an arbitrary dialogue utterance $u$, the output of a linking model $g$ is its referent persona description in persona space $\mathbb{P}$. That is, the dual task aims at finding a linking model $g:u\mapsto\mathbb{P}$.

where $\theta_{\textsc{Link}}$ is the parameters to be learned for model $g_{\theta}$. We formalize the dual task as a variant of entity linking system, where an utterance is linked to an entry in the PKB $\mathbb{P}$ of arbitrary size, being populated from the primal dataset $\mathcal{D}_{\textsc{Chat}}$, i.e., $\mathbb{P}=\bigcup_{i=1}^{N}\mbox{${\cal P}$}_{i}$. To learn model $g_{\theta}$, we adopt the same neural architecture of the primal task (i.e., Bi-encoder) (Humeau et al. 2020) as a base model with the cross-entropy loss.¹¹1For more flexible application, inspired by (Wu et al. 2020), we relax the collective linking in Eq. (2) into finer-level linking computing the utterance-to-persona score, i.e., $p(p_{i}|u_{i};\theta_{\textsc{Link}})$.

We introduce a new framework that exploits the duality (Xia et al. 2017) of Persona-Chat and Persona-Link tasks where the input (utterance) and output (persona) of the dual task are roughly the inverse of its primal task.

In the training phase of the primal task, our ultimate goal is to transform the original dialogue dataset $\mbox{${\cal D}$}_{\textsc{Chat}}$ into the debiased dataset $\tilde{\mbox{${\cal D}$}}_{\textsc{Chat}}$ by augmenting new personas, then train the debiased dialogue model $f_{\theta}$ from $\tilde{\mbox{${\cal D}$}}_{\textsc{Chat}}$, which reduces the model dependence on linguistic bias between personas and utterances. Algorithm 1 describes its overall procedure. In the proposed framework, the linking model $g_{\theta}$ should be learned in advance to augment plausible personas per a given utterance. However, the key challenge is collecting the supervisory information as the training data of $g_{\theta}$ (denoted as $\mbox{${\cal D}$}_{\textsc{Link}}$ or $\tilde{\mbox{${\cal D}$}}_{\textsc{Link}}$), which is capable of injecting the debiasing ability to $g_{\theta}$. As illustrated in Figure 2, we mainly discuss this in the subsequent section.

Once an arbitrary linking model $g_{\theta}$ is obtained as the desired debiasing function, we leverage $g_{\theta}$ also in the inference phase of the primal task, not only in training time. Specifically, as illustrated in Figure 1, a new persona is interactively augmented per new response to ground the dialogue agents to debiased persona-response alignments. This procedure is repeated until the dialogue ends.

Basically, as supervisory data for Persona-Link, we can collect the plausible alignments of persona and utterances in individual dialogue episodes of the Persona-Chat dataset, by using their semantic relationship. Specifically, we leverage natural language inference (NLI), a task of determining whether a hypothesis (e.g., persona) can be inferred from the given premise (e.g., utterance). The hypothesis sentence is classified into three categories: Entailment (true), Contradiction (false), and Neutral (undetermined). We adopt a RoBERTa model (Liu et al. 2019) trained on MNLI (Wang et al. 2019). By NLI, the minimal sampling to populate $\mbox{${\cal D}$}_{\textsc{Link}}$ is to consider the in-dialogue matching of persona-utterance pair $(u,p)$ as candidates, where $u$ and $p$ are derived from the same dialogue episode. However, as mentioned earlier, such $(u,p)$ pairs in individual dialogue episodes reportedly suffer from trivial word overlaps.

In contrast, we perform the out-dialogue matching of all possible combinations in $\mbox{${\cal D}$}_{Chat}$, which is stochastically less overlapped at lexical level. That is, such a simple strategy can contribute to mitigating the linguistic bias in the NLI-driven alignments, which we call seed Persona-Link dataset.

Linguistic bias comes from the limited knowledge and experience of individual crowdworkers, which motivates the out-dialogue matching of personas and annotated utterances. Beyond the Persona-Chat corpus, if someone would have world-level knowledge, she may willingly annotate more engaging, less linguistically-biased utterances for a given persona. For example, utterance “I don’t eat any meat.” can involve commonsense attributes ‘I needed to be vegeterian’ or ‘I wanted to be healthy’, which enable drawing a new persona “I am vegan” from PKB, which are not captured by semantic matching (i.e., predicted as Neutral).

If a linking model is trained on sentence pairs of only similar semantics, such a potentially relevant mapping cannot be captured for generalization in inference time. Thus, since personas and utterances are instances of world events that often imply real-world consequences or richer information, we propose to exploit such commonsense attributes as “anchors”. It may ensure the learned representations are well associated via reasoning effects (Liu et al. 2020; Majumder et al. 2020) beyond the semantically close alignments.

Specifically, we populate an augmented dataset $\tilde{\mbox{${\cal D}$}}_{Link}$ that expands pairs in ${\mbox{${\cal D}$}}_{Link}$ based on commonsense knowledge. For commonsense expansion, we capture implicit attributes of either personas and utterances and annotate them as metadata, using GPT2 based commonsense knowledge generators (Hwang et al. 2021) (Appendix A). The commonsense attributes are surrounded by special tokens and concatenated into a single sequence with persona or utterance.

As many commonsense attributes are commonly shared between utterance and persona, they are likely to be relevant to each other although the NLI-driven label indicates negative. On the other hand, not always such an assumption is true since generated commonsense attributes are often ambiguous or over-claimed. These concerns motivate us to better regularize learning on $\tilde{\mbox{${\cal D}$}}_{\textsc{Link}}$ by well-calibrated soft labels. That is, the expanded set $\tilde{\mbox{${\cal D}$}}_{\textsc{Link}}$ on the denser space may not strictly follow the pre-annotated binary labels from NLI.

To address this, we argue that the linking model trained on $\mbox{${\cal D}$}_{\textsc{Link}}$ can be a good reference point. Specifically, we first train a linking model $\theta_{\textsc{Link}}$ only with the semantic-level Persona-Link dataset $\mbox{${\cal D}$}_{\textsc{Link}}$, which can further perform inference over new data samples $(\tilde{u},\tilde{p})$ to compute their outputs as new labels $P(\tilde{p}|\tilde{u};\theta_{\textsc{Link}})$. By regularizing by the semantic feature space of $\theta_{\textsc{Link}}$, the commonsense-expanded dataset can have well-calibrated labels for encoding commonsense attributes. We thus train another linking model $\tilde{\theta}_{\textsc{Link}}$ with dual goals of following not only the original hard labels but also new soft labels as:

where $\lambda$ is a preference weight with the distillation loss. To compute the linking score $P(p|u;\theta)$ and the cross-entropy loss ${\cal L}$, based on the Bi-encoder architecture allowing for fast and real-time inference, we follow the optimization procedure (Logeswaran et al. 2019; Humeau et al. 2020; Jeong et al. 2021) of retrieval-based models (e.g., information retrieval, entity linking, and response selection): The network is trained to maximize the score of the correct persona $p$ with respect to the (randomly sampled) personas of the same batch (i.e., in-batch negatives).

For the experiment on white-box settings (i.e., augmenting personas in both training and inference time), we consider following six models including two Persona-Link variants as the candidates to be analyzed.

1) Paraphrasing: As unsupervised data augmentation (Xie et al. 2020), we consider an off-the-shelf paraphrasing system in (Mallinson, Sennrich, and Lapata 2017), where personas were translated into a foreign language and back-translated as paraphrases.

2) Manual Paraphrasing: As labor-intensive augmentation, we use manually revised persona sentences additionally presented in Persona-Chat, where extra workers rephrased them to remove trivial word overlap.

3) Bi-encoder: As an IR baseline of linking model (Wu et al. 2020) trained on $\mbox{${\cal D}$}_{\textsc{Link}}$, we use two transformers that separately encode utterance and persona, and multi-sentence scoring over the pre-computed cache of personas.

4) Cross-encoder: We consider using a single transformer (Humeau et al. 2020) that uses a richer self-attention mechanism, jointly encoding utterance and persona in $\mbox{${\cal D}$}_{\textsc{Link}}$ to obtain the final representation, which is impractical for real-time use.

5) Persona-Link (small PKB): Our proposed linking model with a smaller Persona Knowledge Base (PKB), caching 1K personas which are randomly sampled from the primal dataset $\mathcal{D}_{\textsc{Chat}}$.

6) Persona-Link (large PKB): Using the same model, we present the original version of our approach, where all 5K personas from $\mathcal{D}_{\textsc{Chat}}$ are plugged into its PKB.

Table 1 shows the overall performance of dialogue models trained with different linking models. Models paired with Persona-Link variants significantly outperform other baselines for all measures. This suggests that training dialogue model with well augmented personas helps the model to be more persona-grounded and consistent which is a primary goal in persona-based dialogue literature. Furthermore, as the performance gain increases with the size of PKB used for Persona-Link, we believe that populating bigger PKB improves the performance. Another observation is that Persona-Link variants outperform both manual and automatic paraphrasing approaches on every measure. Even though paraphrasing techniques might contribute to richer representation of personas, we argue that the paraphrasing approaches are limited to mitigating the trivial word overlap while utterances can be expressed in much flexible way in persona-based dialogue. While Bi- and Cross-encoder linking models result in better performance compare to the paraphrasing-based linking models due to out-dialogue matching which aims to mitigate linguistic bias, Persona-Link variants outperform in both measures. Especially, we observe all Persona-Link variants outperform Cross-encoder which compromises high computational cost for performance. We further analyze how Persona-Link achieves better performance in Persona-Link Evaluation section.

To demonstrate that the augmented persona is not over-claimed by out-dialogue matching or out-corpus expansion, we compare the quality of augmented persona using dual task models. For that, we simulate a setting where none or few persona information is provided and new personas are augmented only in inference time by using off-the-shelf dialogue agents (i.e., learned without augmenting personas in training time). Inspired by the evaluation method of measuring the relevance of generated response in (Su et al. 2020), we use the pre-trained dialogue agents as our diagnosis model assuming that the response quality of the model reflects the suitability of augmented persona in given dialogue contexts. That is, the suitability of the augmented persona to the context would be low if the augmented personas, which replace partial or entire persona of dataset, cause the failure of pre-trained dialogue agents. For the diagnosis agents, we adopt two dialogue models of Bi-encoder; each pretrained on Wikipedia (BERT-like; Devlin et al. 2019) or Reddit (an adapted setup for dialogue; Mazaré et al. 2018), and both fine-tuned on $\mbox{${\cal D}$}_{\textsc{Chat}}$. Note that, during fine-tuning, the dialogue agents only observe the original personas. As dual task models, we adopt two linking models; Bi-encoder trained on $\mbox{${\cal D}$}_{\textsc{Link}}$ and our Persona-Link trained on $\tilde{\mbox{${\cal D}$}}_{\textsc{Link}}$.

Specifically, the original persona sentences were removed at a given percentage; No persona eradicates the whole persona information and Incomplete persona discards 20%. In test time, linking models generate personas based on the utterance histories and merge them into the sample for testing dialogue models²²2For fair comparison, all utterances on the development set remained unseen by linking models. Also, for faster computation, all utterances in dialogue were concurrently processed in batch.. With persona augmentation, Table 2 shows the overall performance of dialogue models. In each setting, we report the base performance with information limit, where no linking model is available, i.e., N/A. In the last row, we report the upper bound performance, where dialogue models access full information of Original persona.

First, compared to N/A, we observe linking models significantly improve (+14.6% at most) response quality by backtracking personas from utterances for the augmentation. Dialogue models seem to benefit from augmented personas, even though the models haven’t seen them in training process, suggesting that personas provided by Persona-Link successfully keep contextual relevance to the given dialogues. Second, Persona-Link models outperform Bi-encoder models in all settings. As the only difference between the two is commonsense expansion with regularized training, our approach seems to enrich dual task with common knowledge beyond the alignments in Persona-Chat. Thirdly, the performance gain is the greatest in the persona-free setup, which is common in real-world scenarios. This suggests that our persona augmentation may boost dialogue agents in bootstrapping user profiles on-the-fly. As observing only debiased personas improves a trained agent, we conclude that our approach efficiently counteracts linguistic bias in Persona-Chat.

Now, we examine how persona augmentation calibrates model prediction in response retrieval, by showing an example in Table 3. After removing a persona in advance, a linking model augments its own personas into context based on dialogue history. Observing additional information relevant to utterances (i.e.meal) and linguistically debiased (i.e.steak), the model prediction was calibrated online to retrieve the gold response, which is more consistent and engaging. Note that we use the same dialogue model to retrieve both responses. (See Appendix C for details).

To study the model accuracy to the dual task, we perform comparative experiments of Persona-Link. To find ground truth pairs for Persona-Link, we manually collect 300 utterance-persona pairs from the test data in Persona-Chat, including 230 unique ground truth personas with seven annotators, which achieve a very good agreement (Cohen’s kappa = 0.86). The number of utterances per persona is 1.16. As in Persona-Chat evaluation, we measure Recall@$k$ and MRR as the model performance to the gold persona.

We consider five IR baselines trained on $\mathcal{D}_{\textsc{Link}}$: Cosine Similarity, BM25 (Robertson et al. 1995), K-NRM (Xiong et al. 2017) Conv-KNRM (Dai et al. 2018), and Bi-encoder (Humeau et al. 2020). Also, to investigate the impact of commonsense expansion and label regularization, we consider two variants of our approach using alternative expansion methods: EDA (Wei and Zou 2019), and Paraphrasing (Mallinson, Sennrich, and Lapata 2017). More details for these baselines are presented in Appendix B.

Table 4 shows the overall linking performance. First, we observe that among IR baselines trained on $\mathcal{D}_{\textsc{Link}}$, Bi-encoder achieves the best performance, which validates our choice of base model for the dual task. Second, all models trained on $\mathcal{D}_{\textsc{Link}}$ show poorer results than Ours and its variants using expansion methods with regularized training. This means that leveraging expanded dataset and calibrating labels into soft labels is more appropriate than naive IR models. Finally, Ours outperforms its variants (and other linking models as well) in all metrics. We note that while utterance or persona remains semantically similar by EDA or paraphrasing, Ours expands some commonsense attributes so that potentially relevant mapping can be captured. This supports the efficacy of exploiting commonsense as anchors, which may associates learned representations beyond the semantically close alignments. In Figure 3, we visualize the utterance embeddings without (Figure 3a) and with (Figure 3b) commonsense expansion. We observe that the utterance embeddings relevant to a persona gets closer with commonsense expansion, which can be a possible explanation for the performance boost of Ours exploiting commonsense over its variants.

In addition to the linking performance, we further analyze the linked utterance-to-persona alignments by linking models to see their linguistic bias. We plot the linguistic similarity between utterance-to-persona alignments in Figure 4. Following (Park et al. 2019), we calculate Jaccard similarity and METEOR, widely known metric for lexical similarity and semantic similarity, respectively. Jaccard similarity measures word co-occurrence and METEOR measures exact, stem, synonym, and paraphrase matches. For both metrics, an alignment with lower similarity can be seen as the one with less linguistic bias. Two models drop the original similarity from 0.2 to 0.1 on average, indicating the efficacy of out-dialogue semantic matching. Ours shows a slight decrease compared to out-dialogue model which confirms that Ours successfully leverages the commonsense expansion to align utterances to debiased persona.

We compare the performance of $\theta_{\textsc{Link}}$ and $\tilde{\theta}_{\textsc{Link}}$ in semantically different pair sets to investigate whether commonsense expansion helps the model link to debiased persona. Here, we use a BERT model fine-tuned with a popular paraphrase identification (PI) dataset called Quora Question Pairs to split our utterance-persona pairs into semantically equivalent and non-equivalent sets. With this model, only 35.5% of utterance-persona pairs are inferred as positive (i.e., paraphrase). For analysis, in Figure 5, we split the overall utterance-persona pairs into positive and negative sets by the PI model, and ablate the performance of linking model with or without using commonsense knowledge and label regularization. As a result, while $\tilde{\theta}_{\textsc{Link}}$ shows better performance than $\theta_{\textsc{Link}}$ in all sets, $\tilde{\theta}_{\textsc{Link}}$ achieves much better performance in negative set of R@10. This demonstrates the advantage of using commonsense knowledge, which gives the model the ability to link semantically different pairs that leads $\tilde{\theta}_{\textsc{Link}}$ to link utterance to debiased persona.

Next, we evaluate how commonsense expansion affects when only a limited number of examples are provided in train time. Figure 6 shows the linking performance grouped by frequency of utterance-persona pairs in the training data in terms of R@10. Here, the performance means how well the model links utterances to their gold persona when the utterance-persona pair of the gold persona is not provided or few in train time. We observe that $\tilde{\theta}_{\textsc{Link}}$ has more balanced performance for any persona groups, which is the effect of using commonsense knowledge, which can work as pivots when the utterance and persona has similar commonsense in common. Especially, $\tilde{\theta}_{\textsc{Link}}$ shows the significant improvement for unseen personas (w/o any aligned training data) compared to $\theta_{\textsc{Link}}$, which shows that $\tilde{\theta}_{\textsc{Link}}$ may be applicable to zero-shot linking. Since $\tilde{\theta}_{\textsc{Link}}$ shows great performance to unseen utterance-persona pairs, we suggest to use our linking model with a large set of personas.