Adaptable Text Matching via Meta-Weight Regulator

First, our method assigns an initialization weight to each source sample. Considering that the difference between the source and target data distribution is vast, we initialize each source weight as 0 to prevent early update of the model parameters on the source distribution from falling into local values unfavorable to the target performance. Through subsequent calculations, the weights of the source data that are beneficial to the model’s performance on the target distribution are increased.

To establish a computational graph relationship between the source weights and the target distribution loss during training, we first establish a relationship of the source weight with the model parameters. To do so, we need to update the text matching model parameters on the weighted source data. Specifically, the source weights are applied to compute the model’s weighted source loss:

where $i\in\{1,2,...,N\}$, $a^{s}_{i},b^{s}_{i}\in S^{Train}$, $TMM^{s}$ indicates the text matching model for forwarding propagation on $S^{Train}$, $\theta$ denotes the model parameters, and $Cost$ is the cost function of the text matching model.

Then the model parameters are updated by gradient descent. The updated model parameters can be formalized as dependent variables with source weights as independent variables:

where $\widetilde{\theta}$ represents the updated model parameters, and $\alpha$ indicates the learning rate.

Since both the initialized source weight and the weighted source loss are 0, the updated model parameters do not change numerically, but there is a path from the source weights to the updated model parameters in the computation graph, as shown in Figure 2.

After establishing a computational graph connection between source weights and model parameters, the Meta-Weight Regulator regulates the source weights by comparing the source and target gradient descent directions. Inspired by MAML (Finn et al., 2017), we adopt the second derivative to contrast the gradient directions and update the source weights.

Firstly, we train the text matching model on the target few-shot set and calculate the training loss:

where $a^{t}_{j},b^{t}_{j}\in T^{FS}$, $TMM^{t}$ indicates the text matching model for forwarding propagation on $T^{FS}$.

Since the source weights are linked to the updated parameters in the computation graph, when the Meta-Weight Regulator performs model gradient descent on the target data, the derivative of the target loss to the model parameters can naturally flow further to the source weights. This process is illustrated in Figure 2. Therefore, the Meta-Weight Regulator can regulate the source weights by gradient descent according to the second derivative of the target loss:

where $\alpha$ denotes the learning rate equaled with Eq.3.

The above regulating method, denoted as meta-gradient descent, comes from meta-learning, which allows comparing the gradient descent directions of model parameters on different data distributions. The Meta-Weight Regulator adjusts the source sample weights, specifically by increasing the source weights, to minimize the target loss. In other words, the source samples that make the model parameters gradient descent in a similar direction to the target data’s will be weighted higher, and vice versa. After the weight regulation, we integrate the updated source weights into the loss computation for regular model training on the source data.

Lastly, we perform formal text matching model training on the source data and weight the source loss with the regulated weights to improve the model’s adaptability:

where $i\in\{1,2,...,N\}$.

In summary, the Meta-Weight Regulator operates based on the gradient descent algorithm, and the adjusted source weights are only used for the calculation of loss. Therefore it is suitable for any neural text matching models where the parameters are updated based on the gradient descent algorithm or its variants (Cherry et al., 1998; Kingma and Ba, 2014; Loshchilov and Hutter, 2017). In other words, it is model-agnostic and complementary with any existing deep text matching model. Moreover, since the proposed regulator is implemented in a meta-learning paradigm that is similar to Model-Agnostic Meta-Learning (Finn et al., 2017), it can automatically regulate the model parameters and avoid introducing additional hyper-parameters that need to be manually tuned.

The goal of QA is to select the true answer for a given question from a list of candidate answers. In our experiments, we use WikiQA (Yang et al., 2015) and TrecQA (Wang et al., 2007), which are typical datasets for open domain QA. The data of TrecQA is from the TREC Conference, while WikiQA is constructed from Bing search queries and Wikipedia. Both data sets contain publicly available sets of question and answer pairs. Each example in the datasets is a (question, answer, label) triplet, and each label is one of 0, 1. Nevertheless, the data distributions of the two datasets are different due to their different data sources. In these two datasets, the original numbers of positive and negative samples are not equal. They are pre-processed through downsampling to obtain a balanced number of positive and negative samples, and made consistent with the natural language inference datasets in experiments. The statistics of the processed WikiQA and TrecQA datasets are shown in the first two rows of Table 2.

NLI is a text matching task that aims to understand natural language using inference about entailment and contradiction relationships between a pair of texts. It is regarded as a valuable testing ground for the development of semantic representations. In the NLI task, we choose two widely used datasets, i.e., SNLI (Bowman et al., 2015) and SciTail (Khot et al., 2018), which are both built via crowdsourcing. They are different in data sources. The examples of SNLI are written by humans performing a novel grounded task of describing a specific scenario from the same perspective based on image captioning. Meanwhile, SciTail is created from multiple-choice science exams and web sentences. Thus compared with SciTail, the data distribution of SNLI is more diverse. In both datasets, each example is a (premise, hypothesis, label) triplet. The labels in SNLI include ”Entailment”, ”Neutral”, and ”Contradiction”, while the labels in SciTail are ”Entailment” and ”Neutral”. To keep the number of labels in the four datasets consistent in our adaptation experiments, we remove the ”Contradiction” examples from SNLI. For consistency, the data in SciTail is also downsampled to achieve a class balance. The statistics for the processed SNLI and SciTail datasets are presented in the last two rows of Table 2.

We divide the experiments into two categories according to the degree of difference between the source and target data distributions: intra-task cross-dataset adaptation experiment and cross-task adaptation experiment. In cross-task adaptation experiment, the four datasets from two tasks are cross-combined as source-target groups. For the cross-dataset adaptation experiment, we use the different datasets within a task to take turns as the source and target datasets. In each type of adaptation experiment, we compare the effectiveness of the Meta-Weight Regulator with other adaptation methods in the 10-shot, 50-shot, 100-shot, and 1000-shot settings. It may be arguable to consider 1000-shot as few-shot setting, we include it for the fairness of the experiments, to reduce the impact of class imbalance on the Adversarial Domain Confusion method (one of the baseline adaptation methods). In each set of the experiments, $T^{FS}$ is randomly sampled from the target’s training set in a class-balanced manner, and the backbone model is kept consistent to ensure fair comparisons.

We compare the proposed Meta-Weight Regulator approach, denoted as $MWR$, with the backbone models and four typical existing adaptation methods as baselines for text matching in the few-shot setting. The baseline adaptation methods are Data Merging, Fine Tuning, Adversarial Domain Confusion, and Reinforced Transfer Learning.

• •

  Backbone models (Devlin et al., 2018): We use the BERT-based-uncased as the main backbone model, for its impressive performance and capability of directly fine-tuning on train data. Here the backbone model is fine-tuned directly on $T^{FS}$ instead of being used as a fixed checkpoint. Note that we also test a range of other neural text matching models as backbones in Section 5 (Ablation Study).

• •

  Data Merging (DM): It is a baseline adaptation method where we first merge $S^{Train}$ and $T^{FS}$ as training set, then train the backbone model on this merged training set.

• •

  Fine Tuning (FT): Fine Tuning method is another baseline adaptation method, where the backbone models are pre-trained on the $S^{Train}$ firstly, then they are fine-tuned on $T^{FS}$. Because the backbone model is BERT, it is fine-tuned on $S^{Train}$ and $T^{FS}$ successively.

• •

  Adversarial Domain Confusion (ADC) (Cohen et al., 2018): It learns a domain classifier simultaneously as model training and performs a domain confusion based on adversarial learning for cross-domain adaptation.

• •

  Reinforced Transfer Learning (RTL) (Qu et al., 2019a): It introduces a source domain data selector based on reinforcement learning to help transfer learning of the model to the target data.

For all tasks in the experiments, we adopt Cross-Entropy as loss function and Accuracy (Acc) as the evaluation metric to measure the performance of each adaptation method on the target test set. Statistic significance is examined by permutation test with $p\textless 0.05$.

In this work, the batch size of the source training set is 64, and the number of samples per class in the target few-shot set is 10, 50, 100 or 1000, making 10-shot, 50-shot, 100-shot and 1000-shot settings respectively. The hyper-parameters of text matching models are tuned on the development set, and the Meta-Weight Regulator does not bring any additional parameters. The AdamW optimizer is used to update the model parameters (Loshchilov and Hutter, 2017), and the learning rate $\alpha$ is set to $5e-5$. The maximum sequence length is truncated to 40 for QA and 50 for NLI, and padding is masked to avoid affecting the attention mechanism. For all backbone models except BERT, we use GloVe (840B tokens) to initialize the embedding look-up table for the word embedding layer, and the dimensionality of word embedding is 300. The word embedding layer is set to non-trainable. All models and methods are implemented with PyTorch ¹¹1https://pytorch.org/. The backbone BERT model we use is implemented by Huggingface ²²2https://huggingface.co/bert-base-uncased (Wolf et al., 2019).

Table 3 shows the performance of the backbone model and the use of different adaptation methods on top of the backbone model for cross-dataset adaptation experiments in different few-shot settings. Although the backbone model’s performance is relatively low, the first two baseline adaptation methods Fine Tuning and Data Merging, can hardly help the model improve its adaptability and sometimes even hurt the performance. Notably, Data Merging method greatly degrades the backbone model’s performance in some individual cases. We consider this may be due to the large difference between the source and target data distributions, i.e., the two distributions are far apart in the representation space. The model learns an intermediate distribution between the two on the combined data, resulting in a poor performance. The Reinforcement Transfer Learning method (Qu et al., 2019b) benefits from the idea of data selection, which improves the performance of the backbone model to a certain extent. It also outperforms other baseline adaptation methods when the target data in the experiments is relatively small, such as in the 10-shot, 50-shot and 100-shot settings. The Adversarial Domain Confusion method (Cohen et al., 2018) is comparable to Reinforced Transfer Learning in the 1000-shot setting. However, its performance degrades, due to its inability to meet the standard data size requirements for training classifiers as the amount of target data decreases (i.e., 10-shot, 50-shot, and 100-shot settings).

Compared with all the baseline adaptation methods, the proposed Meta-Weight Regulator achieves the best performance in almost all cases, and significantly improves the adaptability of the BERT backbone model in the few-shot settings.

As shown in Table 4, the performance of the backbone model trained with almost every adaptation method degrades, because the difference between the source and target data distributions in cross-task experiments is larger than that in the cross-dataset experiments. Still, the Fine Tuning and Data Merging methods are less helpful to improve the backbone model’s adaptability. Indeed, both of them even hurt the backbone model’s performance in some individual cases due to the larger difference between source and target data distributions. The overall performance of Reinforcement Transfer Learning (Qu et al., 2019b) and Adversarial Domain Confusion (Cohen et al., 2018) decline compared to those in the cross-dataset experiments. Nevertheless, they still contribute to improving the backbone BERT model’s adaptability. Although the overall performance of Reinforcement Transfer Learning and Adversarial Domain Confusion decline compared to those in the cross-dataset experiments, they still contribute to improving the backbone BERT model’s adaptability. Unlike the results in the cross-dataset experiments, Reinforcement Transfer Learning only outperforms Adversarial Domain Confusion on fewer target data (i.e., 10-shot and 50-shot settings). However, Adversarial Domain Confusion performs on par with or better than Reinforcement Transfer Learning in the 100-shot and 1000-shot settings. This proves that simply making data selection at the data level is ineffective when the source and target data distributions are quite different. To sum up, as the data distribution gap increases, the overall performance of each method is affected to a certain extent.

Nonetheless, the performance of the proposed Meta-Weight Regulator still significantly surpasses all other adaptation methods in almost every set of experiments, ensuring that the overall performance does not drop considerably in comparison with the cross-dataset results. Notably, the amount of target data is roughly the same as that of source data when the source dataset is WikiQA in 1000-shot experiments, which is the ideal condition of Adversarial Domain Confusion. However, Meta-Weight Regulator still outperforms or is on par with Adversarial Domain Confusion in this setting. This result further validates the effectiveness of our approach under a higher data distribution gap.