MATA*: Combining Learnable Node Matching with A* Algorithm for Approximate Graph Edit Distance Computation

In order to learn node matchings of GED from $\mathsf{CBQP}$ formulation, it requires $X_{i,k}\in\{0,1\}$ to relax to be continuous in $[0,1]$, where the constraints in Eq. (1) could be modeled as the quadratic infeasibility penalty (Kochenberger et al., 2014). This relaxation strategy endows the binary matrix $X$ with an augmented perspective. Thus, the binary matrix $X$ can be viewed as the confidence of node $u_{i}$ in $\mathcal{G}_{1}$ matching with node $v_{k}$ in $\mathcal{G}_{2}$. In this way, we formulate the problem as the linear matching paradigm, by incorporating the graph structure information into node embedding, i.e., solving the following transportation problem:

where $\mathbf{h}_{1}\in\mathbb{R}^{|\mathcal{V}_{1}|\times d}$ and $\mathbf{h}_{2}\in\mathbb{R}^{|\mathcal{V}_{2}|\times d}$ are the node embeddings of $\mathcal{G}_{1}$ and $\mathcal{G}_{2}$, respectively. Intuitively, the kernel finds the optimal way to transform one set of node embeddings $\mathbf{h}_{1}$ to the other $\mathbf{h}_{2}$, by minimizing the Euclidean distance between corresponding node pairs w.r.t. graph pairs.

We further analyze the combinatorial properties to design the approximate GED computation framework, after modeling it from learning node matchings by solving Eq. (2).

Thus, the proposed MATA* employs a structure-enhanced GNN (i.e., SEGcn) to learn the differentiable top-$k$ candidate matching nodes which prunes the unpromising search directions of $\mathsf{A}$*$\mathsf{LSa}$ (Chang et al., 2020) for approximate GED computation. The overview of MATA* is illustrated in Fig. 2 with the details of each module elaborated in the following paragraphs.

As analyzed, structural information is critical to learn the fine-grained matching of node pairs for the GED computation. Here, a structure-enhanced GNN SEGcn is proposed that jointly learns the structural information from the views of local and high-order.

For more robust position encoding, perturbations are further injected to the original graphs to get the perturbed position encoding. Specifically, we randomly insert and remove a small portion of edges (10% utilized in experiments) to produce the perturbed graphs $\mathcal{G}_{in}$ and $\mathcal{G}_{re}$, respectively. The random walk diffusion manners are further performed on $\mathcal{G}_{in}$ and $\mathcal{G}_{re}$ with the perturbed local positions $p_{i}^{in}$ and $p_{i}^{re}$, respectively. Combined with the Eq. (3), the positional encoding is given as follow:

After encoding by SEGcn, the node embeddings of $\mathcal{G}_{1}$ from the local and high-order views are denoted as $\mathbf{h}_{1}^{(0)}\in\mathbb{R}^{|\mathcal{V}_{1}|\times d}$ and $\mathbf{h}_{1}^{(l)}\in\mathbb{R}^{|\mathcal{V}_{1}|\times d}$, respectively. The node embeddings of $\mathcal{G}_{2}$ are similarly obtained.

The local and high-order structural affinities between two graphs has been ingeniously encoded into the node embedding space, by SEGcn. As such, learning to match nodes is reduced to solve the Eq. (2). We thus jointly learn the matchings from both the local view and high-order view to obtain the differentiable top-$k$ candidates by iteratively minimizing the underlying transportation problem. In addition to the task of learning node matching, a complementary task learning GED is also put forward, which attempts to learn the distance between graph representations that assists the node matching task.

Specifically, we first flatten the similarity matrix $\mathbf{S}^{(0)}$ with local structure affinity into $\mathbf{d}=[d_{1},d_{2},...,d_{|\mathcal{V}_{1}||\mathcal{V}_{2}|}]$. As such, to differentiable find the top-$k$ matches, the optimal transport problem can be viewed as redistributing $\mathbf{d}$ to one of $d_{max}$ and $d_{min}$, where the capacities of $d_{max}$ and $d_{min}$ are $k$ and $|\mathcal{V}_{1}||\mathcal{V}_{2}|-k$, respectively. That is the matches moved into $d_{max}$ are preserved during the redistributing and the others moved into $d_{min}$ are discarded. Let $\mathbf{c}$ and $\mathbf{r}$ represent the marginal distributions, $\mathbf{D}$ represents the distance matrix, and $\mathbf{1}$ represents the vector of all ones (line 1). And we have:

Then an efficient method Sinkhorn (Cuturi, 2013; Fey et al., 2020) for solving the optimal transport problem is typically adopted to learn the probabilities of the top-$k$ matchings, which is an approximate and differentiable version of $\mathsf{Hungarian}$. It iteratively performs row-normalization, i.e., element-wise division by the sum of its row and column-normalization until convergence, where $\odot$ means element-wise division, $\operatorname{diag}(\cdot)$ means building a diagonal matrix from a vector (lines 2–4).

After the differentiable top-$k$ operation, we reshape $\mathbf{\Gamma}$ into the assignment matrix from local view $\mathbf{S_{a}}^{(0)}\in|\mathcal{V}_{1}|\times|\mathcal{V}_{2}|$ , which essentially measures the confidence of $\mathcal{V}_{1i}$ and $\mathcal{V}_{2j}$ belonging to the optimal matching (line 5). For the similarity matrix $\mathbf{S}^{(l)}$ with high-order structure affinity are also performed to obtain the assignment matrix $\mathbf{S_{a}}^{(l)}$ from high-order view (line 8). Finally, Alg. 1 returns $\mathbf{S_{a}}^{(0)}$, $\mathbf{S_{a}}^{(l)}$ during the training, and top-$k$ candidate nodes $M^{|\mathcal{V}_{1}|\times k}$ during the testing (lines 6–7).

Note that, during the testing, we further propose a greedy method to find top-$k$ candidate nodes in $\mathcal{O}(kn^{2})$ time, In brief, it iteratively finds a node with the largest matching probability as a candidate from the unmatched nodes, where the injection constraint of node matchings is also guaranteed.

Intuitively, $\mathbf{h}_{1}^{(0)}$ and $\mathbf{h}_{1}^{(l)}$ capture the node features enhanced by the local and high-order graph structural information of $\mathcal{G}_{1}$. This intricate embedding process ensures that the nodes’ features are not only captured in their raw features but are also contextualized within the broader structure of the graph. Essentially, (approximate) GED measures the similarity of graph pairs from the graph level, and hence, we aggregate the node embeddings with both local and high-order views of $\mathcal{G}_{1}$ and $\mathcal{G}_{2}$ passed through MLPs for the learning GED task. And we have the following:

That is, $d_{\mathcal{G}_{1},\mathcal{G}_{2}}$ is predicted using the MLP operation which gradually reduces the concatenated graph representations $\mathbf{h}_{1}^{g}$ and $\mathbf{h}_{2}^{g}$ of the graph pair. Actually, to counter this and provide a more interpretable and standardized measure, the GED are typically normalized by $\exp\{-{\mathsf{ged}}(\mathcal{G}_{1},\mathcal{G}_{2})\times 2/(\mathcal{V}_{1}+\mathcal{V}_{2})\}$.

For the learning GED task, we minimize the MSE loss:

where $\mathcal{D}$ is the set of training graph pairs.

Our final loss function is a combination of the negative log-likelihood loss and MSE loss: $\mathcal{L}=\mathcal{L}_{g}+\mathcal{L}_{n}$.

MATA* finally integrates $\mathsf{A}$*$\mathsf{LSa}$ algorithm (Chang et al., 2020) to refine the edit distance (i.e., node matching) among the learned top-$k$ candidate matching nodes $M^{|\mathcal{V}_{1}|\times k}$, as shown in Alg. 2.

Specifically, MATA* conducts a best-first search by treating GED computation as a pathfinding problem. Such a representation is convenient because it provides a systematic and heuristic way to explore possible node matchings. To facilitate this search process, a priority queue is maintained to store the search states during the process for guiding the search direction. The priority queue $Q$ contains the level $i$, current partial matching $f$, the parent matching $pa$, and the lower bound $lb$. $\mathsf{A}$*$\mathsf{LSa}$ initializes the priority queue $Q$ by the root of the search tree (line 1). It then iteratively pops $(i,f,pa,lb)$ from $Q$ with the minimum lower bound, and subsequently extends the current matching $f$ by computing the lower bound of each child belonging to the candidates $M^{|\mathcal{V}_{1}|\times k}$ (lines 2–7). If the full node matching is formed, then ${\mathsf{\widetilde{ged}}}(\cdot,\cdot)$ equals its lower bound and is returned (lines 6, 8).

Note that, different from the hybrid approach (Yang and Zou, 2021; Wang et al., 2021), during the mapping refinement of MATA*, the search space is pruned by the theoretical bounded estimation of unmatched subgraphs of $\mathsf{A}$*$\mathsf{LSa}$.

Due to the fact that exact GED computation is $\mathsf{NP}$-complete, the ground-truth of Aids is produced by exact algorithms and the ground-truth of Imdb and Cancer are generated by the smallest edit distances of $\mathsf{A}$*$\mathsf{Beam}$, $\mathsf{Hungarian}$, and $\mathsf{VJ}$, following (Bai et al., 2019). Note that, MATA* is able to achieve a smaller edit distance, and the ground-truth of Imdb and Cancer are further updated by the best results of the four approaches. Therefore, the metrics on Aids are calculated by the exact solutions and the metrics on Imdb and Cancer are calculated by the updated ground-truth. Note that the edit distance is normalized into a similarity score in the range of $(0,1]$ as explained in Section 4.3, the same as (Bai et al., 2019; Wang et al., 2021).

In this section, we evaluate the performance of MATA* from the effectiveness, scalability, efficiency, ablation study, top-$k$ comparisons, and the analysis of assignment matrices.

(1) From the ACC, MATA* achieves smaller edit distances at least (58.1%, 32.1%, 53.6%) of graph pairs on (Aids, Imdb, Cancer) when comparing with combinatorial search-based and hybrid approaches. Hence, the ground-truth of Imdb and Cancer are further updated by these, which reduces MAE by at least (6.5%, 9.1%, 24.5%). (2) Only learning-based models cannot recover the edit path, as they directly learn GED as a similarity score and ignore the combinatorial nature. (3) On Imdb, all methods perform worse than on Aids and Cancer. Imdb is large with a wide range from $7$ to $89$ nodes. Besides, the graphs are much denser with $\mathcal{|E|}/\mathcal{|V|}$ = $4.05$ and the distances of pairs are also larger, which increases the difficulty for combinatorial search and learning methods.

(4) The improvement of MATA* on ACC is such significant with at least (39.0%, 21.6%, 11.7%), and the improvement of other metrics is relatively less significant. The rationales behind this lie in that (a) MATA* models from the perspective of node matching and explicitly build the task of learning node matching, that is the learned top-$k$ candidate nodes could directly improve the accuracy due to the correspondence between GED and the node matching. (b) For fewer node pairs whose matchings have not been learned by MATA*, it prunes the search subtree rooted at the these node pairs, which leads to a larger edit distance reflected in other metrics.

We also study the performance w.r.t. selecting different $k$ of MATA* on Aids, Imdb and Cancer datasets. We conduct the following findings from Table 6.

(1) Varying $k$ in the experiments emphasize the trade-off between solution quality and time. That is setting larger $k$ could improve the approximate GED solution quality, but the running time indeed increases mainly due to the large search space of $\mathsf{A}$*$\mathsf{LSa}$. (2) Aids is a small dataset with the exact solutions, and MATA* also achieves the optimal solutions running in $5.2$ ms, when $k$ is set to $10$. Note that, MATA* degenerates to $\mathsf{A}$*$\mathsf{LSa}$ when all nodes of $\mathcal{G}_{2}$ are selected as the candidate matching nodes. (3) When $k$ is set to 6, 8 for datasets Imdb and Cancer, the evaluation metric is worse than that in Table 3. In fact, $k$ is set to 10 in this test which achieves a smaller edit distance, and the ground-truth of Imdb and Cancer are further updated by these solutions. Hence, the evaluation metrics are re-calculated, which produces worse metrics on Imdb and Cancer. To encapsulate, the interplay of the value of $k$, offers a balance between computational scalability and accuracy of the approximate graph edit distance computation.

we offer a visual representation of four assignment matrices that encapsulate both local and high-order perspectives. These matrices, specifically pertaining to two pairs of graphs, have been portrayed as heatmap images, and are generated by MATA* on Aids in Fig. 3. From our observations, both the local and high-order views play a crucial role in the extraction of features tailored for node matchings. This is evident when considering specific node pairs, such as (6,6) and (7,7), from the graph pair labeled (a) (the top row of the assignment matrics $\mathbf{S_{a}}^{(0)}$ and $\mathbf{S_{a}}^{(l)}$). We can see that the local and high-order views both extract features for matching nodes, e.g., the node pairs (6,6) and (7, 7) of the pair (a) (the top row). Besides, the assignment matrix $\mathbf{S_{a}}^{(l)}$ in the high-order view typically has a more powerful capacity to learn the node correspondence compared to the local view $\mathbf{S_{a}}^{(0)}$. For example, $\mathbf{S_{a}}^{(0)}$ fails to capture the node pairs (1,1) of the pair (b), while $\mathbf{S_{a}}^{(l)}$ successfully learns it. Thus, Fig. 3 shows the importance of using local and high-order views to jointly learn the top-$k$ candidates rather than a single one.