When does MAML Work the Best? An Empirical Study on Model-Agnostic Meta-Learning in NLP Applications

In meta-learning, we have multiple tasks $T$ sampled from distribution $p(\mathcal{T})$ [Ravi and Larochelle, 2017, Andrychowicz et al., 2016, Santoro et al., 2016]. For each task $T_{i}$, we train a base model $f_{i}^{\theta}$ with parameter $\theta_{i}$ on its training corpus $D_{i}^{train}$ which only has a few samples, and evaluate the model on the testing corpus $D_{i}^{valid}$. We divide the tasks into meta-training, meta-validation, and meta-testing. The goal of meta-learning is that after training on meta-training, we can quickly find $f_{i}^{\theta}$ via fine-tuning (adaptation) with $D_{i}^{train}$ for each task $T_{i}$ in meta-testing.

Model-Agnostic Meta-Learning (MAML) [Finn et al., 2017] pre-trains a parameter initialization $\theta$ shared among tasks. At each training epoch, MAML samples a set of tasks ${T_{i}}{\sim}p(\mathcal{T})$. For each task $T_{i}$, MAML trains from the initialization $\theta$ on $D_{i}^{train}$ to get $\theta_{i}$, then evaluates each $\theta_{i}$ on $D_{i}^{valid}$ and updates $\theta$, which is,

$\displaystyle\theta_{i}=\theta-\alpha\nabla_{\theta}\mathcal{L}_{D_{i}^{train}}(\theta),\theta=\theta-\beta\nabla_{\theta}\sum_{{T_{i}}{\sim}p(\mathcal{T})}\mathcal{L}_{D_{i}^{valid}}(\theta_{i})$

(1)

where $\mathcal{L}_{D_{i}^{train}}(\theta)$ and $\mathcal{L}_{D_{i}^{valid}}(\theta_{i})$ are the loss functions of $\theta$ on $D_{i}^{train}$ and $\theta_{i}$ on $D_{i}^{valid}$, $\alpha$ and $\beta$ are the learning rates. In fine-tuning stage, each task fine-tunes from the pre-trained initialization $\theta$ on its $D_{i}^{train}$.

In Experiment I: Text Classification, we use FewRel [Han et al., 2018] and Amazon [He and McAuley, 2016]. They are datasets for 5-way 5-shot classification, which means 5 classes are randomly sampled from the full dataset for each task, and each class has 5 samples. FewRel is a relation classification dataset with 65/5/10 tasks for meta-training/meta-validation/meta-testing. Amazon is a large customer review dataset with 24 product categories. We follow [Bao et al., 2020] to sample 1000 reviews from each category and use 10/5/9 tasks for meta-training/meta-validation/meta-testing.

In Experiment II: Dialogue Generation, we use Persona [Zhang et al., 2018] and Weibo, regarding building a dialogue model for a user as a task. Persona is a personalized dialogue dataset with 1137/99/100 users for meta-training/meta-validation/meta-testing. Each user has 121 utterances on average. Weibo is a personalized dialogue dataset collected from Weibo conversations with 371/40/38 users for meta-training/meta-validation/meta-testing. Each user has 1200 utterances on average.

In each experiment, we use the same base model among the 2 datasets. In text classification experiments, we use the CNN proposed in [Bao et al., 2020] as the base model and follow the hyperparameter settings. We use Transformer [Vaswani et al., 2017] as the base model in dialogue generation experiment. In Persona, we use pre-trained Glove embedding [Pennington et al., 2014]. In Weibo, we use Gensim [Rehurek and Sojka, 2010]. We follow the other hyperparameter settings in [Madotto et al., 2019].

In text classification experiment, we use accuracy (Acc) to evaluate the classification performance. In dialogue generation experiment, we evaluate the performance of MAML in terms of quality and personality. We use PPL and BLEU [Papineni et al., 2002] to measure the similarity between the reference and the generated response, and use C Score [Madotto et al., 2019] to measure the personality. In Persona we use a pre-trained natural language inference model to measure the response consistency with persona description for C Score. In Weibo, users do not have persona descriptions, so we pre-train a user classifier to classify the generated responses, and use the accuracy for C Score.

To answer RQ1, we compare the changing trend of the general language model and the task-specific adaptation ability during the training of MAML to find whether there is a trade-off problem. (Figure 1) We select the trained parameter initialization at different MAML training epochs and evaluate them directly on the meta-testing set before fine-tuning, using the quality performance (accuracy for classification and BLEU for generation) to evaluate the general language model. Then we use the personality performance (accuracy for classification and C Score for generation) after fine-tuning to measure each model’s ability to adapt to specific tasks.

When the training epochs increase, the model’s ability of task-specific adaptation reaches the peak earlier than the quality of its general language model does. In Persona, BLEU before fine-tuning reaches the peak at epoch 3000, but after epoch 1500, the final C Score drops. In Weibo, the changing trends are consistent with the results on Persona. In FewRel and Amazon, the general language model first becomes better and then over-fits to the training data, but the final accuracy decreases after epoch 1.

The finding suggests that parameter initialization at the late training stage has strong general language generation ability, but performs comparative poorly in task-specific adaptation. Although in the early training stage, the performance improves benefiting from the pre-trained general language model, if the language model becomes too “general”, it will lose the ability of adapting to specific tasks. It is noteworthy that the ”too general” problem is not the same as over-fitting, since the ”too general” model performs well before fine-tuning, which means it does not over-fit to the training data.

To answer RQ2, we find the fine-tuning epochs for each task in Persona where its BLEU and C Score reaches the best respectively to find the impact of data quantity and the task profile (persona description) on fine-tuning. (Table 1) We cluster the tasks with similar best fine-tuning epoch number and calculate the average dialogue quantity and task profile similarity for each cluster. We define Jac Score of the persona descriptions to evaluate their task profile similarity as $\mathit{Jac~{}Score=\Sigma_{i=1}^{N}Jac_{i}/(NJac_{whole})}$, where $Jac_{i}$ is the Jaccard similarity of the persona descriptions in cluster i, $Jac_{whole}$ is the Jaccard similarity of all the persona descriptions, and N is the number of clusters. If Jac Score is larger than 1, the persona descriptions are similar to each other in each cluster. Larger Jac Score means better similarity.

For both BLEU and C Score, Jac Score is around 1 in each cluster, which means the persona descriptions are not similar. The dialogue quantity also seems similar among different clusters. So we can conclude that data quantity and task profile does not have a major impact on the fine-tuning process.

To answer RQ3, we conduct experiments on different data quantity and task similarity settings. We compare two baselines with MAML : Transformer/CNN, which pre-trains the base model (Transformer/CNN) on the meta-training set and evaluates directly on the meta-testing set, and Transformer/CNN-F, which fine-tunes Transformer/CNN on each meta-testing task.

Data Quantity. In Persona, we evaluate Transformer/CNN, Transformer/CNN-F and MAML on 3 data quantity settings: 50/100/120-shot (each task has 50, 100, 120 utterances on average). In Weibo, FewRel and Amazon, the settings are 500/1000/1500-shot, 3/4/5-shot and 3/4/5-shot respectively (Table 2). When the data quantity is small, the advantage of MAML is more significant. In Persona, the C Score and BLEU of MAML outperform baselines on 50-shot and 100-shot settings, but on 120-shot setting, the BLEU of MAML is lower than Transformer-F. In Weibo, FewRel and Amazon, the percentages that MAML outperforms the baselines by also decrease as the data quantity increasing. This finding is in line with the mechanism of MAML. MAML finds a sensitive parameter initialization that can adapt with few data samples [Finn et al., 2017]. When there are enough data samples, fine-tuning also performs well, so BLEU of Transformer-F in Persona on 120-shot setting is even better.

Task similarity. In Persona and Weibo, each task is a set of dialogues for one user, so tasks are different from each other. We shuffle the samples and randomly divide tasks to construct the setting that tasks are similar to each other. For a fair comparison, each task on this setting also has 120 and 1200 utterances on average in Persona and Weibo respectively. We train and evaluate Transformer-F and MAML on this setting. (Table 2). When tasks are similar to each other, MAML performs comparatively poorly. In Persona and Weibo, the performance of MAML is similar to that of Transformer-F, while MAML performs significantly better than Transformer-F when tasks are different. A possible explanation is that if there is no clear distinction between tasks, the meta-learning setting can be viewed as a transfer learning setting, which only has a source domain and a target domain, and fine-tuning performs well in transfer learning. So if the tasks are similar to each other, we can simply use Transformer-F rather than MAML.