Triples-to-Text Generation with Reinforcement Learning Based Graph-augmented Structural Neural Networks

The aim of RDF-to-text generation is to generate a informative, grammatically correct, faithful and fluent description for a given collection of RDF triples. Web Ontology Language to text generation (Bontcheva and Wilks, 2004; Stevens et al., 2011; Androutsopoulos et al., 2013) and knowledge base verbalization (Banik et al., 2012) were early forms of this task. Pipeline system is often used to address data-to-text generation, including two main steps: (1) content selection addresses the question of what content should be described (Barzilay and Lapata, 2005); (2) surface realization realizes the text generation process token by token (Deemter et al., 2005; Belz, 2008).

Over the past few years, a number of natural language generation (NLG) tasks based on Seq2Seq model with attention mechanism (Bahdanau et al., 2014; Luong et al., 2015) and copy mechanism (See et al., 2017; Gu et al., 2016) have achieved promising performance. A range of research work such as (Gardent et al., 2017b; Jagfeld et al., 2018) show that Seq2Seq model and its variant models perform prominently on RDF-to-text generation by flattening a collection of RDF triples into a text sequence. Ferreira et al. (2019) implemented two encoder-decoder architectures using Gated-Recurrent Units (GRU) (Cho et al., 2014) and Transformer (Vaswani et al., 2017) respectively. To improve the probability of producing high-quality texts, Zhu et al. (2019) proposed a model minimizing the Kullback-Leibler (KL) divergence between the distributions of the real text and generated text. Moryossef et al. (2019) proposed a model combining the pipeline system and neural networks to match a reference text with its corresponding text plan to train a plan-to-text generator. In addition, some work has focused solely on improving the performance of the seen split part of dataset, where the entity types have appeared in the training set. Ribeiro et al. (2020) combined two GNN encoders to encode both global and local node contexts for WebNLG seen categories. Recently, data-to-text generation has benefited from large-scale pre-trained language models (PLMs). Wang et al. (2021) and Kale and Rastogi (2020) applied T5 (Raffel et al., 2019) in this task. Moreover, Wang et al. (2021) further exploited external knowledge from Wikipedia to improve the model performance.

There has been a significant increase in interest in using graph neural networks for data-to-text generation.

Graph Neural Networks. Graph Neural Networks have achieved excellent performance on many natural language processing tasks where the input data can be represented as a graph. Essentially, graph neural networks are graph representation learning models that learn each node embedding using graph structure and its neighbouring node embeddings. According to (Ma and Tang, 2021), the learning process of node embeddings is referred as graph filtering, which can refine the node embeddings without modifying the graph structure. Graph filters can be roughly categorized into four classes (Wu et al., 2021): spectral-based like GCN (Kipf and Welling, 2016), spatial-based like GraphSage (Hamilton et al., 2017), attention-based like Graph Attention Network (GAT) (Veličković et al., 2017) and recurrent-based like Gated Graph Neural Network (GGNN) (Li et al., 2015). Furthermore, graph pooling operation aggregates node embeddings and produces a smaller graph with fewer nodes inspired by CNNs (Krizhevsky et al., 2012). When the smaller graph only contains one node, the node embedding is considered as the graph-level embedding for the input graph.

Graph to Sequence Learning. Sequence-to-Sequence model is a widely used encoder-decoder framework in the natural language generation field. However, when the input is graph data such as dependency graph, Abstract Meaning Representation (AMR) (Banarescu et al., 2013) graph and knowledge graph, it often needs to be linearized into a sequence before it can be fed into the Seq2Seq model, resulting in a loss of structural information in the graph. To alleviate the limitation of Seq2Seq models on encoding graph data, a number of graph-to-sequence (Graph2Seq) models have been proposed (Bastings et al., 2017; Beck et al., 2018; Song et al., 2019; Yao et al., 2020). Bastings et al. (2017) applied GCN on dependency trees of input sentences to produce word representations. Beck et al. (2018) proposed a graph-to-sequence model based on GGNN with edges being turned into additional nodes. Song et al. (2019) explored the effectiveness of AMR as a semantic representation for neural machine translation based on a graph recurrent network. Previous studies (Chen et al., 2020; Gao et al., 2019; Xu et al., 2018) applied a bidirectional graph encoder to encode an input graph and then used an attention-based LSTM (Hochreiter and Schmidhuber, 1997) decoder. To capture longer-range dependencies, Song et al. (2018) employed an LSTM to process the state transitions of the graph encoder outputs. Marcheggiani and Perez-Beltrachini (2018) used a relational graph convolutional network, which is introduced and extended in (Kipf and Welling, 2016; Bruna et al., 2013; Defferrard et al., 2016), to encode each node using its neighboring nodes, edge directions and edge labels simultaneously.

The aim of Information Extraction (IE) is to extract entity pairs and their relationships within a given sentence. Information extraction is a significant task in the field of natural language processing, as it helps to mine factual knowledge from free texts. Information extraction is an inverse task of RDF-to-text generation, and there are a number of researches on both tasks on the same dataset.

As two important subtasks of information extraction, named entity recognition (NER) and relation extraction (RE) can be operated via a pipeline approach that firstly recognize entities and then conduct relation extraction (Zelenko et al., 2003; Chan and Roth, 2011). NER predicts a label for each word in a sentence indicating whether it is an entity or not. For a sentence with annotated entities, RE is essentially a relation classification task which predicts a relation label for each entity pairs. Zeng et al. (2014), Xu et al. (2015) and Sahu et al. (2019) conducted relation extraction via CNN, RNN and GNN, respectively.

Recently, researchers start to explore joint entity recognition and relation extraction. Some Seq2Seq-based methods (Zeng et al., 2018; Nayak and Ng, 2020; Zeng et al., 2020; Sui et al., 2020) directly generate relational triples. For example, Set Prediction Networks (SPN) (Sui et al., 2020) is featured by transformers, which can directly predict the final collection of triples. Other models are tagging based methods (Zheng et al., 2017; Dai et al., 2019; Wang et al., 2020), which regard the IE task as a sequence labeling problem and use specially designed tagging schema. For example, TPLinker (Wang et al., 2020) is a one-stage joint extraction model with a hand-crafted tagging schema and can discover overlapping relations sharing one or both entities. There are also other methods that represent entities and relationships in a shared parameter space, but extract the entities and relationships respectively (Miwa and Bansal, 2016; Zhang et al., 2017; Zhao et al., 2021). For example, Zhao et al. (2021) proposed a representation iterative fusion strategy on heterogeneous graph neural networks for relation extraction. After obtaining the final representations of entity nodes and relation nodes, this model uses specialised taggers to extract subjects and objects, respectively. In this paper, we pretrain the SPN on the same dataset of the RDF-to-text generation task to extract triples from the generated texts.

As shown in Fig.2, our proposed model contains four key components: a graph construction module, a combined graph-augmented encoders module, a decoder module and a post information extraction module. First, the graph construction module leverages the input RDF triples to build two input graphs. Second, after graph construction, we propose two graph-based encoders to model the input graphs and learn node embeddings and graph embeddings. Third, we employ an LSTM-based decoder to generate the text word by word. In this module, we propose separated attention mechanism on both graph-based encoders and fuse their context vectors using a selection mechanism. Fourth, a pretrained information extraction model, i.e.SPN extracts triples from the sequence generated by sampling and the sequence generated by greedy search. A reinforcement learning reward is computed based on the correct numbers of extracted triples.

In the next few sections, we first propose a graph-augmented structural neural encoders framework including two graph-based encoders. Then, we describe how to combine the two encoders for better modeling global and local information. Finally, we introduce how to compute the reward of the whole model using the pretrained information extraction model.

Given a collection of RDF triples, it can be formed as a directed graph naturally with subjects and objects being nodes and relationships being edges. We refer this naturally formed graph as $\mathcal{G}_{1}$. For the graph $\mathcal{G}_{1}=(\mathcal{V}_{1},\mathcal{E}_{1})$, $\mathcal{V}_{1}$ represents the entity nodes set, and $\mathcal{E}_{1}$ represents original relationships between entities. With the purpose of encoding information depending on different meta-paths and model long-range dependencies in $\mathcal{G}_{1}$, we propose a novel bidirectional Graph-based Meta-Paths encoder (bi-GMP) enhanced on the basis GTR-LSTM (Trisedya et al., 2018). In contrast to GTR-LSTM, the bi-GMP encoder performs hidden state masking on different meta-paths to ensure the encoding of different meta-paths do not interfere with each other.

We select the meta-paths of $\mathcal{G}_{1}$ using a combination of topological sort algorithm and single-source shortest path algorithm shown in Algo.1. Firstly, we find a node set $\mathcal{V}_{\text{IN}}$ with zero in-degree nodes and a node set $\mathcal{V}_{\text{OUT}}$ with zero out-degree nodes. Secondly, by choosing a starting node from $\mathcal{V}_{\text{IN}}$ and a target node from $\mathcal{V}_{\text{OUT}}$, we calculate a single-source shortest path with both nodes and relationships retained in the path. Finally, a sequence containing a set of meta-paths $S_{p}=\{p_{1},p_{2},...,p_{k},...\}$ can be selected from the graph $\mathcal{G}_{1}$, where $p_{k}=\left\langle w_{k,1},w_{k,2},...w_{k,n_{k}}\right\rangle$. The hidden state masking means that the hidden state of the final token in $p_{k-1}$ is masked when to compute the hidden state of the first token in $p_{k}$.

After obtaining the meta-paths, we initialize the word embeddings using 6B GloVe vectors¹¹1https://nlp.stanford.edu/projects/glove/ for each word in $S_{p}$. Each word $w_{k,i}$ is represented as a distributed representation $\mathbf{w}_{k,i}\in\mathbb{R}^{d_{e}}$, where the dimension $d_{e}$ is equal to 300. Taking inspiration from the benefits of BiLSTM over LSTM, the meta-paths encoder computes the token representations in both directions for each meta-path and concatenates them together in the last step. When the word embedding $\mathbf{w}_{k,i}$ is given, the hidden state $\mathbf{r}_{i}$ is computed as follows:

where $f(\cdot)$ and $g(\cdot)$ are single LSTM units and $\text{concat}(\cdot)$ is the concatenation operation. If the previous or the next hidden state belongs to different meta-path, an all-zero hidden state $\mathbf{r}_{0}$ is input to $f(\cdot)$ or $g(\cdot)$. Equation 3 and Equation 4 can be written as:

where $n_{k}$ is the length of meta-path $p_{k}$.

Finally, a set of entity node representations $R_{1}=[\mathbf{r}_{1};\mathbf{r}_{2};...;\mathbf{r}_{L_{1}}]^{T}\in\mathbb{R}^{L_{1}\times D}$ is output by the bi-GMP encoder, where $L_{1}$ is the expanded length of $S_{p}$. The graph embedding of graph $\mathcal{G}_{1}$ is computed over $R_{1}$:

where $\text{maxpool}(\cdot)$ is the max pooling operation over the first dimension and $Z_{\mathcal{G}_{1}}\in\mathbb{R}^{D}$.

To better use the information in relationships, we build the input graph for bi-GCN encoder by converting relationships to nodes in the graph. For the graph $\mathcal{G}_{2}=(\mathcal{V}_{2},\mathcal{E}_{2})$, relationships are considered as graph nodes. Therefore, $\mathcal{G}_{2}$ is composed of entity nodes and relationship nodes. Edges in $\mathcal{E}_{2}$ may connect relationship node to entity node or entity node to entity node. More details about the construction of bi-GCN graph are illustrated in Sec.5.3.

To effectively learn the node embeddings and the graph embeddings for the constructed graph, we present the bi-GCN encoder based on the conventional GCN, which learns node embeddings from both incoming and outgoing edges. At each layer, we first compute the intermediate node representations $\mathbf{H}_{\vdash}^{(l)}$ and $\mathbf{H}_{\dashv}^{(l)}$ in both directions. Then, we fuse the two node representations at this layer to obtain the fused node representations $\mathbf{H}^{(l)}$. In particular, the fused node representations at layer $l$ can be represented as ${\mathbf{H}^{(l)}}=[{\mathbf{h}_{v_{1}}^{(l)}};{\mathbf{h}_{v_{2}}^{(l)}};...;{\mathbf{h}_{v_{L_{2}}}^{(l)}}]^{T}\in\mathbb{R}^{L_{2}\times D}$ with $L_{2}$ being the number of nodes. The above calculation process is formulated as follows:

where $\hat{\mathbf{A}_{\vdash}}=\mathbf{A}_{\vdash}+\mathbf{I}$ and $\hat{\mathbf{A}_{\dashv}}=\mathbf{A}_{\dashv}+\mathbf{I}$ represent the incoming and outgoing adjacency matrices of $\mathcal{G}_{2}$ with self-loops. And $\mathbf{I}$ is an identity matrix representing the self-loop of each node. $\hat{\mathbf{D}}_{\vdash,ii}=\sum_{j=1}^{L_{2}}\hat{\mathbf{A}}_{\vdash,ij}$ and $\hat{\mathbf{D}}_{\dashv,ii}=\sum_{j=1}^{L_{2}}\hat{\mathbf{A}}_{\dashv,ij}$ are the diagonal degree matrices. $\mathbf{W}_{\vdash}^{(l-1)}$, $\mathbf{W}_{\dashv}^{(l-1)}$ and $\mathbf{W}_{f}^{(l-1)}$ denote trainable weight matrices. $\sigma$ is a non-linearity function and $\text{concat}(\cdot)$ is the concatenation operation. $L_{2}$ is the number of nodes in $\mathcal{G}_{2}$. Similar to the bi-GMP encoder, $\mathbf{H}^{(0)}$ is also initialized with GloVe embeddings. Bidirectional node embeddings at the same layer are fused by concatenation and fed into a one-layer feed-forward neural networks before being fed to the next bi-GCN layer. We stack the bi-GCN encoder to $L$ layer and $R_{2}=\mathbf{H}^{(L)}$ is the set of final node representations after $L$ hops of bi-GCN computation.

Similarly, we apply max pooling on $R_{2}$ to calculate the graph embedding of graph $\mathcal{G}_{2}$:

where $R_{2}\in\mathbb{R}^{L_{2}\times D}$ and $Z_{\mathcal{G}_{2}}\in\mathbb{R}^{D}$.

To jointly learn the global and local structural information of the RDF triples input, we propose a strategy to combine the bi-GMP encoder and the bi-GCN encoder.

Both encoders produce a set of node representations. $R_{1}$ is mainly concerned with the global structural information between the triples, because the bi-GMP encoder calculates hidden states according to the traversal order obtained by the combination of topological sort algorithm and single-source shortest path algorithm. Therefore, each hidden state $\mathbf{r}_{i}$ can retain information from one triple to multiple triples if there exists a meta-path. At the same time, $R_{2}$ better models the local structural information within the triples because each node representation is directly learned by all its one-hop neighbors. We stack the bi-GCN encoders to 2 layers and the output of bi-GCN encoder $\mathbf{h}_{i}$ retains at most two-hop information of the graph, i.e., it is limited to one or two triples.

At last, the combined graph embedding $Z_{\mathcal{G}}$ is computed as follows:

where $\oplus$ denotes component-wise addition and $Z_{\mathcal{G}}\in\mathbb{R}^{D}$.

An attention-based LSTM decoder (Luong et al., 2015) is employed for text generation. Since we have two encoder outputs, the conventional top-down attention cannot fully capture different semantic information from two completely different encoders. To better decode the text description, we first apply separated attention mechanism on both graph encoders and then fuse the context vectors using a selection mechanism inspired by pointer-generator (See et al., 2017).

The decoder takes the combined graph embedding $Z_{\mathcal{G}}$ as its initial hidden state. At each decoding time step $t$, the embedding of the current input (or previously generated word) $\mathbf{e}_{t}$ and previous hidden state $\mathbf{s}_{t-1}$ are fed into the decoder to update its hidden state $\mathbf{s}_{t}$:

Next, we apply the separated attention mechanism by calculating the attention align weights at this time step as follows:

where $\mathbf{r}_{i}\in R_{1}$ and $\mathbf{h}_{j}\in R_{2}$. $L_{1}=|R_{1}|$ and $L_{2}=|R_{2}|$ are the lengths of $R_{1}$ and $R_{2}$, respectively. $\mathbf{s}_{t}$ is the hidden state of decoder. In this paper, we use the $score(\cdot)$ function as follows to compute the similarity of $\mathbf{r}_{i},\mathbf{h}_{j}$ and $\mathbf{s}_{t}$:

where $\mathbf{v},\mathbf{W}_{x}$ and $\mathbf{W}_{s}$ are learnable parameters. $\mathbf{x}$ can be $\mathbf{r}_{i}$ or $\mathbf{h}_{j}$. After obtaining the attention align weights, we calculate the context vector $\mathbf{c}_{u}$ of bi-GMP and the context vector $\mathbf{c}_{v}$ of bi-GCN respectively:

Selection mechanism uses a gcn probability to control the attention paying to the context vectors of different encoders from a global perspective. The gcn probability $p_{gcn}\in[0,1]$ for time step $t$ is calculated from the decoder state $\mathbf{s}_{t}$ and the decoder input $\mathbf{e}_{t}$:

where $\mathbf{W}_{p}$ and $\mathbf{b}_{p}$ are learnable parameters, $\sigma(\cdot)$ is the sigmoid function and $[\cdot;\cdot]$ denotes concatenation operation. The fused vector $\mathbf{c}_{t}$ for time step $t$ combining context vectors $\mathbf{c}_{u}$ and $\mathbf{c}_{v}$ is computed as:

where $p_{gcn}$ is used to control the proportion of different context vectors. The final attentional hidden state at this time step is:

where $\mathbf{W}_{c}$ and $\mathbf{b}_{c}$ are learnable parameters.

After obtaining an attentional decoder hidden states sequence $\left\langle\widetilde{\mathbf{s}_{1}},\widetilde{\mathbf{s}_{2}},...,\widetilde{\mathbf{s}_{T}}\right\rangle$ recurrently, the decoder generates an output sequence $\left\langle y_{1},y_{2},...,y_{T}\right\rangle$ with $T$ being the length of the sequence. The output probability distribution over the vocabulary of the current time step is calculated by:

where $\mathbf{W}_{v}$ and $\mathbf{b}_{v}$ are trainable parameters. Finally, the text generation loss is a cross entropy loss:

where $S$ is the collection of input RDF triples and $y_{t}^{*}$ is the $t$-th word in the ground truth output sequence.

Besides training with the generation loss $L_{g}$, we further design an information extraction reward to generate more faithful and informative descriptive texts.

We apply a self-critical sequence training (SCST) algorithm (Rennie et al., 2017) depicted in Fig.3 to optimize the RL loss. SCST is a form of REINFORCE algorithm that, instead of estimating a “baseline” to normalize rewards and reduce variance, normalizes the rewards it experiences using the output of its own test-time inference algorithm. At each training iteration, two text sequences are generated: (1) a sampled sequence $Y^{s}$ generated by multinomial sampling, which means sampling tokens according to the probability distribution $P(y^{s}_{t}|S,y^{s}_{1:t-1})$; (2) a baseline sequence $\hat{Y}$ generated by greedy search. The rewards of the two text sequences $R(Y^{s})$ and $R(\hat{Y})$ are defined as the correct number of predicted triples using a pretrained IE model SPN metioned in Section 2.3. The SPN model receives the sampled sequence or the baseline sequence as input and predicts a collection of RDF triples. Tables 1 shows a ground truth triples-text pair, a prediction text generated by our proposed model based on the ground truth triples and a collection of RDF triples predicted by the SPN model based the generated text. The IE reward for this example is equal to 2 because the generated text is missing the information in the triple (Alan Shepard , timeInSpace , 130170 minutes). The RL objective is defined as follows:

We train our model using a hybrid objective function combining both text generation loss and RL loss, defined as:

where a scaling factor $\gamma$ controls the trade-off between generation loss and RL loss.

Entity masking is, as the name implies, replacing the entity names that appear in the input triples and output texts with their corresponding types. It can improve the generalization ability of the models. In this paper, we use the official dictionary provided by WebNLG. Taking into account that entities appearing in the same collection of triples may of the same type, we allocate an entity-id (eid) to each entity mention in the same collection. In this way, each entity is replaced with its eid and its entity type. For instance, the entity “Bakewell pudding”, namely “FOOD-1” in Fig. 4, is replace by “ENTITY-1 FOOD”.

As shown in Fig. 4(a), to find all meta-paths, we first build the zero in-degree nodes set $\mathcal{V}_{IN}$ = {FOOD-1} and the zero out-degree nodes set $\mathcal{V}_{OUT}$ = {PERSON, COUNTY, INGREDIENT}. Then, we need to find the single source shortest paths from nodes in $\mathcal{V}_{IN}$ to nodes in $\mathcal{V}_{OUT}$. The three meta-paths of this example are as follows:

• •

  FOOD-1${\rightarrow}$region${\rightarrow}$ PLACE${\rightarrow}$leaderName${\rightarrow}$PERSON

• •

  FOOD-1${\rightarrow}$region${\rightarrow}$PLACE${\rightarrow}$county${\rightarrow}$COUNTY

• •

  FOOD-1${\rightarrow}$dishVariation${\rightarrow}$FOOD-2${\rightarrow}$ingredient${\rightarrow}$INGREDIENT

These meta-paths are concatenated to a text sequence and input to the bi-GMP encoder. As shown in the bi-GMP module of Fig. 2, the hidden state of the last token in path $p_{t}$ is not passed to the first token in path $p_{t+1}$.

Similar to (Marcheggiani and Titov, 2017), the relationships are regarded as additional nodes for bi-GCN encoder and the new relationship node connects to the subject and object through two new directed edges, respectively. Therefore, the new graph (Fig. 4(b)) is two times larger than the original RDF triples graph (Fig. 4(a)) in hops, increasing the difficulty to capture long-range dependency. The input triple (FOOD-1, region, PLACE) is split into two new connections “region$\rightarrow$FOOD-1” and “region$\rightarrow$PLACE” with the relationship “region” being a new node in the input graph. For nodes with multiple tokens, each token is split into a separate node with a new edge connecting to the root node (usually the “ENTITY-id” node). For example, the original entity “ENTITY-1 FOOD INGREDIENTS” is split into two connections “ENTITY-1$\rightarrow$FOOD” and “ENTITY-1$\rightarrow$INGREDIENTS”.

We use two open RDF-to-text generation datasets WebNLG ²²2https://gitlab.com/shimorina (Gardent et al., 2017a) and DART ³³3https://github.com/Yale-LILY/dart (Nan et al., 2021) to evaluate our proposed model. Each example in the dataset is a (triples, text) pair and one triple collection can correspond to multiple ground truth texts. An RDF triple is composed of (subject, relationship, object), in which the subject and object are either constants or entities. The WebNLG 2017 challenge dataset includes 18102 training pairs, 2268 validation pairs, and 2495 test pairs from 10 categories (Airport, Astronaut, City, Building, ComicsCharacter, Food, Monument, SportsTeam, University, WrittenWork). In addition to being divided by entity categories, the WebNLG dataset is also divided into “seen”, “unseen” and “all” categories based on whether the topic entities have been exposed in the training set.

The second dataset DART is a large and open-domain structured DAta Record to Text generation corpus collection, integrating data from WikiSQL, WikiTableQuestions, WebNLG 2017 and Cleaned E2E. The DART dataset is composed of 62659 training pairs, 6980 validation pairs, and 12552 test pairs and covers more categories since the WikiSQL and WikiTableQuestions datasets come from open-domain Wikipedia. Unlike the previous dataset, this DART dataset is not classified as “seen” and “unseen” and therefore will not be performed entity masking.

The vocabulary is built based on the training set, shared by the encoders and decoder. We implement our proposed model based on Graph4NLP⁴⁴4https://github.com/graph4ai/graph4nlp released by  (Wu et al., 2021). For the hyperparameters of the model, we set 300-dimensional word embeddings and a 512-dimensional hidden state for the bi-GCN encoder, bi-GMP encoder and decoder. Adam (Kingma and Ba, 2014) is used as the optimization method with an initial learning rate 0.001 and the batch size is 50. The scaling factor $\gamma$ controlling the trade-off between generation loss and RL loss is 0.3.

As for the evaluation metrics, we use the standard evaluation metrics of the WebNLG challenge, including BLEU (Papineni et al., 2002), METEOR (Denkowski and Lavie, 2011) and TER. BLEU looks at n-gram overlaps between the output and ground truth text with a penalty for shorter outputs. The METEOR aligns hypotheses to one or more reference translations based on exact, stem, synonym, and paraphrase matches between words and phrases. Translation Error Rate (TER) is an automatic character-level measure of the number of editing operations required to translate machine translated output into human translated reference. The metric of BLEU provided by the WebNLG 2017 challenge is multi-BLEU⁵⁵5The script of multi-bleu is available on https://github.com/moses-smt/mosesdecoder/blob/master/scripts/generic/multi-bleu.perl, which is a weighted average of BLEU-1, BLEU-2, BLEU-3 and BLEU-4. For BLEU and METEOR, the higher the better. For TER, the lower the better.

Our proposed combined models is compared against the following baselines:

Table 2 shows the experimental results on the WebNLG test dataset. Our proposed combined models achieve best results on BLEU-seen, BLEU-all and METEOR-all metrics, and also achieve comparable results on other evaluation metrics. The series of models implemented under our framework including bi-GCN, bi-GMP, bi-GCN+bi-GMP, and bi-GCN+bi-GMP+IE are 0.7 to 1.7 points and 0.9 to 1.6 points higher in BLEU-seen and BLEU-all respectively than the previous baseline models. The bi-GCN model has lower score than bi-GMP on BLEU-seen, but higher score on BLEU-unseen than bi-GMP, as well as higher score than some sequence models such as Transformer and PKUWriter, showing the better generalization ability of graph neural network model for the unseen categories.

Table 3 shows the experimental results on the DART test dataset. As shown in this table, our proposed combined models consistently outperform other baselines on all three evaluation metrics. This is because our full model could better capture the global and local graph structure of the RDF triples. The performances of the models implemented under our framework significantly surpass the sequential baselines by about 10 BLEU points. The TER score (0.6) of bi-GCN model is higher than those (0.57) of other models proposed by us, which also shows a similar trend in the WebNLG dataset. We believe that this may be because the text generated by the graph neural network states the information in the triple set in a more random order.

Further experiments are carried out to verify the effectiveness of reinforcement learning with information extraction in Table 4. Since each example in the dataset may contain from one to multiple triples (up to seven triples for WebNLG and more for DART), we divide each dataset into two subsets with less than or equal to three and more than three triples according to the number of triples. In particular, for the DART dataset in this table, we only consider examples containing up to seven triples. As shown in this table, BLEU-all score gains 1.1 points on $1\leq\text{size}\leq 3$ subset of the WebNLG dataset after introducing the IE-based reinforcement learning reward. For the DART dataset, the introduction of IE-based RL reward results in a further improvement in BLEU-all scores for both subsets, and the improvement is more significant in the $1\leq\text{size}\leq 3$ subset. This is because the information extraction model performs better in examples where the number of input triples is small. When the number of triples contained in the input text is large, it is difficult for the IE module to extract them completely, and the error propagation from extraction may also lead to limited improvement of the model.

Moreover, we introduce the automatic evaluation metric FEQA (Durmus et al., 2020) used in the field of abstractive summarization to evaluate the faithfulness of generated summmaries. Given question answer pairs generated from the summary, FEQA extracts answers from the document; non-matched answers indicate unfaithful information in the summary. In our setting, we regard the sequence of input triples as a document and regard the generated text as a summary. FEQA uses the averaged F1 score of questions answered correctly for each generated text. Table 5 shows the experimental results on WebNLG test dataset. We split the dataset by the size of input triples and perform FEQA evaluation on each split. The ground truth texts, texts generated by bi-GCN+bi-GMP model, and texts generated by bi-GCN+bi-GMP+IE model are evaluated. As the table shown, the F1 scores of ground truth are the highest and the F1 scores of bi-GCN+bi-GMP+IE model are higher than bi-GCN+bi-GMP model when the size is smaller than 4, which is consistent with the results in Table 4 and verifies the effectiveness of our proposed IE-based reinforment learning reward. In particular, we do not perform FEQA on splits where the input size is larger than 5 triples, as these generated or ground truth texts are so informative that it is difficult for the model to generate reasonable questions. Our code is publicly available for research purpose. ⁶⁶6https://github.com/Nicoleqwerty/RDF-to-Text.