Improving Aspect Sentiment Quad Prediction via Template-Order
Data Augmentation

The aspect sentiment quad prediction (ASQP) task, aiming to extract aspect quadruplets from a review sentence, becomes popular recently Zhang et al. (2021a); Cai et al. (2021). The quadruplet consists of four sentiment elements: 1) aspect category (ac) indicating the aspect class; 2) aspect term (at) which is the specific aspect description; 3) opinion term (ot) which is the opinion expression towards the aspect; 4) sentiment polarity (sp) denoting the sentiment class of the aspect. For example, the sentence “The service is good and the restaurant is clean.” contains two quadruplets (service general, service, good, positive) and (ambience general, restaurant, clean, positive).

To extract aspect sentiment quadruplets, Zhang et al. (2021a) propose a new paradigm which transforms the quadruplet extraction into paraphrase generation problem. With pre-defined rules, they first map the four elements of ($ac$, $at$, $ot$, $sp$) into semantic values ($x_{ac}$, $x_{at}$, $x_{ot}$, $x_{sp}$), which are then fed into a template to obtain a nature language target sequence. As shown in Figure 1, the original sentence is “re-writen” into a target sequence by paraphrasing. After fine-tuning the pre-trained language model Raffel et al. (2020) in such a sequence-to-sequence learning manner, the quadruplets can be disentangled from the target sequence.

Though promising is this paradigm, one issue is that the decoder of the generative pre-trained language model Raffel et al. (2020) is unidirectional Vinyals et al. (2015), which outputs the target sequence from the beginning of the sequence to its end. Thus four elements of a quadruplet are modeled in a fixed order $\{x_{ac}\rightarrow{x_{sp}}\rightarrow{x_{at}}\rightarrow{x_{ot}}\}$. Yet ASQP is not a typical generation task. There is no need to fix the element order of the quadruplet as long as it can be extracted accurately. Aspect sentiment quadruplet has the order-free property, suggesting that various orders, such as $\{x_{ac}\rightarrow{x_{sp}}\rightarrow{x_{at}}\rightarrow{x_{ot}}\}$ and $\{x_{at}\rightarrow{x_{ac}}\rightarrow{x_{ot}}\rightarrow{x_{sp}}\}$, are all correct.

In light of this observation, our curiosity is triggered: Does the order of the four elements impact the generative pre-trained language models’ performances? Thus we conduct a pilot experiment. The four elements are concatenated with commas, thus we could switch their orders in a flexible manner and obtain order permutations. It is found that some template orders can help the generative model perform better. Even only concatenating with commas, some orders outperform the state-of-the-art.

It is hypothesized that different orders provide various views of the quadruplet. Therefore, we propose a simple but effective method to identify the most proper orders, and further combine multiple proper templates as data augmentation to improve the ASQP task. Concretely, we use the pre-trained language model Raffel et al. (2020) to select the orders with minimal entropy. Such template orders can better promote the potential of the pre-trained language model. To jointly fine-tune these template orders together, inspired by Paolini et al. (2021), we design special markers for the four elements, respectively. The markers help to disentangle quadruplets by recognizing both the types and their values of the four elements from the target sequence. In this way, the template orders do not need to be fixed in advance.

In summary, the contributions of this work are three-fold:

• •

  We study the effects of template orders in the ASQP task, showing that some orders perform better. To the best of our knowledge, this work is the first attempt to investigate ASQP from the template order perspective.

• •

  We propose to select proper template orders by minimal entropy computed with pre-trained language models. The selected orders are roughly consistent with their ground-truth performances.

• •

  Based on the order-free property of the quadruplet, we further combine multiple proper templates as data augmentation to improve the ASQP task. Experimental results demonstrate that our approach outperforms state-of-the-art methods and has significant gains in low-resource settings.

As Eq. (1) displayed, this template forms a fixed order of four elements. Our curiosity is whether the quadruplet’s order affects the performance of sequence-to-sequence learning. Therefore, we conduct a pre-experiment. By only concatenating with commas, four elements can also be transformed into a target sequence. The orders can be switched in a more flexible way, compared with Eq. (1). There will be $4!=24$ permutations. During inference, quadruplets can be recovered by splitting them with commas. Based on the pre-experimental results, we have the following observations.

Template order affects the performances of sequence-to-sequence learning.   Part of the experimental results on two datasets, i.e. $\mathtt{Rest15}$ and $\mathtt{Rest16}$, are shown in Table 1. It is observed that on $\mathtt{Rest15}$, the $\mathtt{F1}$ score ranges from 45.94% to 48.17%. Similarly, on $\mathtt{Rest16}$, the $\mathtt{F1}$ score ranges from 57.09% to 59.20%. We draw an empirical conclusion that the template order also matters for the ASQP task. Moreover, a template has various performances on different datasets, which is hard to say some order is absolutely good.

The performances of each element are connected to its position. We further investigate the $\mathtt{F1}$ scores on each of the four elements. Given the $24$ permutations, there are $6$ templates for each element at each position. For example, there are $6$ templates $\{(.,.,x_{ac},.)\}$ of $x_{ac}$ at position $3$. In Figure 2, we show the average $\mathtt{F1}$ scores of the six templates of each element at each position. We can see that the performances of the four elements have different trends with the positions. The $\mathtt{F1}$ scores of $x_{ac}$ and $x_{sp}$ both degrade when they are gradually placed backwards. Compared with the other three elements, $x_{at}$ is more stable on different positions while $x_{ot}$ has the worst performance in the first position. In addition to positions, it can also be observed that $x_{ac}$ and $x_{sp}$ achieve higher $\mathtt{F1}$ scores compared with $x_{at}$ and $x_{ot}$, showing various difficult extents of four elements.

As analyzed in the previous section, the template order influences the performances of both the quadruplet and its four elements. It is hypothesized that different orders provide various views of the quadruplet. We argue that combining multiple template orders may improve the ASQP task via data augmentation. However, using all $24$ permutations significantly increases the training time, which is inefficient. Therefore, we propose a simple method to select proper template orders by leveraging the nature of the pre-trained language model (i.e. T5). Then for the ASQP task, these selected orders are utilized to construct the target sequence $\bm{y}$ to fine-tune the T5 model.

Specifically, given an input sentence $\bm{x}$ and its quadruplets $\{({ac},{at},{ot},{sp})\}$, following Zhang et al. (2021a), we map them into semantic values $\{(x_{ac},x_{at},x_{ot},x_{sp})\}$. As shown in Figure 3, our approach is composed of two stages, next which will be introduced in detail.

In this work, we study aspect sentiment quad prediction (ASQP) from the template order perspective. We hypothesize that different orders provide various views of the quadruplet. In light of this hypothesis, a simple but effective method is proposed to identify the most proper orders, and further combine multiple proper templates as data augmentation to improve the ASQP task. Specifically, we use the pre-trained language model to select the orders with minimal entropy. By fine-tuning the pre-trained model with these template orders, our model achieves state-of-the-art performances.

Our work is the first attempt to improve the ASQP task by combining multiple template orders as data augmentation. Despite state-of-the-art performance, our work still have limitations which may guide the direction of future work.

Firstly, we use the entropy to select the proper template orders. The smaller entropy value indicates that the target sequence is better fitting with the pre-trained language model. However, there may be other criteria for template order selection which can better fine-tune the pre-trained language model to support the ASQP task.

Secondly, in the experiment, we simply select the top-$k$ template orders for data augmentation. This can be treated as a greedy strategy for the combination. However, each of the top-$k$ orders may not supplement well to each other. More advanced strategies may be designed to select template orders for data augmentation.

Thirdly, we only consider augmenting the target sequences in the model training, while augmenting both the input and out sequences may bring more performance improvement.

We sincerely thank all the anonymous reviewers for providing valuable feedback. This work is supported by the key program of the National Science Fund of Tianjin, China (Grant No. 21JCZDJC00130), the Basic Scientific Research Fund, China (Grant No. 63221028), the National Science and Technology Key Project, China (Grant No. 2021YFB0300104).