Mining Frequent Neighborhood Patterns in Large Labeled Graphs

Note that unlike [7, 19, 9], we allow an arbitrary number of labels on a single vertex. This is a reasonable generalized assumption for possible applications. For example in a knowledge base consisting of objects and their relationship, an object may be a father, a politician, and a vegetarian at the same time. It’s also possible that a vertex has no label, i.e., we know nothing about the object, except its existence. On the other hand, parallel edges carrying distinct label names may link a pair of vertices to model multiple relations simultaneously existing between two objects. We do not allow edges with no label because we do not process the weak relation of “arbitrary” or “universal” relation. Without losing any generality, we do not allow loops, i.e., edges starting and ending with the same vertex. We can use a vertex label with a specially designed name to replace the loop on a vertex. We use “elements” as the joint name of vertex labels and labeled edges, and define $(|L_{V}|+|E|)$, i.e., the number of elements, as the size of a labeled graph.

Actually, neighborhood patterns are essentially pivoted graphs. By introducing the concept “pivot”, we aim to characterize the semantics of fixing a vertex in a subgraph to form a neighborhood pattern, or selecting a vertex in a database to match a pattern to.

The first two descriptions describe that the isomorphic function preserves both vertex labels and edge labels. In addition, the special isomorphism between pivoted graphs requires that the isomorphic function maps the pivot of $\mathcal{G}_{1}$ to that of $\mathcal{G}_{2}$. As a subtask of FNM, the problem of deciding whether a pivoted graph is pivoted subgraph isomorphic to another is in np-complete.

We prove in the appendix by reducing it to the classical subgraph isomorphism problem.

Proof is omited due to the limited space.

With the support measure defined, the frequent neighborhood mining problem is simply finding all frequent neighborhood patterns in a large graph, with respect to a given threshold. To control the problem complexity, we further requires the mined patterns be connected, i.e., paths exists between every vertex and the pivot. In later discussions, sometimes we consider the operation of removing a labeled edge from a pattern. If the removal leads to an isolated vertex, i.e., a vertex without any vertex label or edge associated to it, we further remove the vertex to make the resulted pattern legal. If we adopt $\subseteq_{f}$ to describe the sub-pattern/super-pattern relationship between neighborhood patterns, the fact that a pattern cannot be more frequent than any of its sub-patterns is directly derived via Property 1.

Typically, the traditional FSM algorithm generates subgraph patterns in an increasing-size manner. First, all frequent subgraphs of size 1 are pre-computed as “building blocks”. Then candidates of size $K$ are obtained by joining pattern pairs of size $(K-1)$ that differ by only one vertex or edge, after which false positives are filtered by a verification against the database. We indicate that the join ensures a complete result because every candidate of size $K\geq 2$ is decomposable, that is, we can always find two distinguished elements, after removing either one we obtain a connected, thus legal, sub-pattern of size $(K-1)$. They may be isomorphic to each other, but their join takes the candidate into our consideration.

For our neighborhood mining problem, however, it is not the case. Consider the path-like neighborhood patterns in Figure 3. Obviously, to find a connected sub-pattern of size $(K-1)$ for Figure 3(a), we have only one choice of removing the edge and vertex at the end of the path. Meanwhile, in the case of Figure 3(b), only the vertex label to the right can be removed. Otherwise, the resulted patterns will be illegally unconnected. Since they are not decomposable, it’s impossible to derive them by joining two smaller patterns. Luckily, the following theorem clarifies that these special patterns are only limited to what is described in Figure 3 and Definition 5, which enables us to treat them as building blocks and pre-process them in advance.

Being a special case of the general neighborhood patterns, path patterns can still be organized into a level-wise structure, or more exactly, a hierarchical one, which preserves the downward closure property. The parent of each path pattern is uniquely found, by removing the vertex label or the vertex on the other end of the path than the pivot. Thus, the level-wise search algorithm on such a structure deserves an “extending” approach to generate larger patterns from small ones, rather than the “joining” one used for non-building-blocks.

In Algorithm 1, we describe the basic algorithm for finding frequent paths. First, a queue used for the bread-first search is initialized with an empty path $\epsilon$. When extending a path on the front of the queue, we traverse according to the pattern, each time with one vertex in $G$ as the starting point. Note that for each starting point, we should not visit a vertex more than once. Each traversal returns all possible moves when we arrive at the ending point(s) and try to take a next step. E.g., we traverse along a “$\bullet\xrightarrow{writes}\circ\xleftarrow{cites}\circ$” path starting from vertex “Jiawei Han”, and stop at the vertex of paper [10] (it cites [19] of Jiawei Han), then possible next-steps on [10] may be following an “cites” edge to another unvisited paper it cites (such as paper [7]) to produce Figure 3(a), or terminate the path with a vertex label “Paper” to end up with Figure 3(b). Each time a new next-step for the current starting point is discovered, it increases its counter by 1. When all traversals are over, those next-steps with a count of more than $\tau$ are used to extend the path. After saving all extended paths to the result set, non-terminated paths, i.e., new paths obtained by appending an edge rather than a vertex label such as Figure 3(a), are added to the queue for further expansion. The algorithm terminates when all extendable paths in the queue are consumed.

As is stated above, what distinguishes our problem from the traditional ones solved by a “join-verify-join-…” scheme is the fact that, large path patterns cannot be derived by joining smaller patterns, no matter these smaller ones are paths, trees, or else. Therefore, the working flow of Algorithm 2 differs from other apriori-based subgraph mining algorithm only by Line 15. This line adds path patterns of the current size to $F_{k}$, the frequent non-building-block ones of the same size, to ensure that larger patterns relying on them are not lost due to their absence.

Besides, our join operation at Line 8 is also worth an detailed explanation. Roughly speaking, we determine whether two patterns should be joined via deleting one element from the first, and check whether the remaining structure is pivoted subgraph isomorphic to the second. Notice that there may be multiple isomorphic mappings so the number of results produced by a single join may be more than one. For each mapping, the deleted element is mapped to the right position in the second pattern, and inserted to produce a joining result. If the removed element is a vertex label, the join is relatively easy. But if it is an edge, the operation is a bit tricky.

On the one hand, the remaining structure is not necessarily connected after the deletion of an edge. Consider Figure 5(a), where we are going to join the patterns “authors having a cited paper” and “authors having a paper citing another”. For the sake of the example and w.l.o.g., we assume that this join is in a branch of the search space where vertex labels are not introduced yet, while readers can still infer by the context that the pivots are author vertices, and the non-pivot vertices represent papers. After deleting the $\xrightarrow{writes}$ edge marked with dotted line and italic label in the first pattern we obtain an unconnected structure. The remaining structure is pivoted subgraph isomorphic to the second pattern, where the mapping is illustrated by dotted lines. Since the paper vertex in the first pattern that the deleted edge points to is mapped to the second paper vertex in the second pattern, we restore the mapped $\xrightarrow{writes}$ edge between the author vertex and the second paper vertex to generate the result on the right. Obviously, this pattern is the skeleton of that illustrated in Figure 1(b).

On the other hand, new vertices may be introduced when handling dangling edges. In Figure 5(b) we again try joining the same patterns as in Figure 5(a), but this time we delete the $\xleftarrow{cites}$ edge. As is required in Section 2.3, this deletion isolates the second paper vertex, so the vertex is also removed. When the pivoted subgraph isomorphism from the remaining structure to the second pattern is established, the vertex that the deleted edge was associated to is mapped to the first paper vertex in the second pattern, which is the new ending point of the restored edge. But be aware that the new starting point may be the other unmatched paper vertex, as well as a additionally introduced vertex. Neglecting this case will cause the bottom-right pattern to be lost, which is the representative of all tree-like patterns.

Moreover, it should be noted that Line 12 actually embeds a procedure of checking for duplicated patterns. If neglected, they will cause more duplicated patterns, joins, and support computations in later computations. To efficiently check for duplicates, we can hash each produced patterns with the vertex labels on, and associated edges of the pivot. When a new pattern is produced, we first use the hash table to find potential duplicates, and further verify them with a series of isomorphism checks.

Finally, the last performance overhead of this algorithm lies in the pivoted subgraph isomorphism checker at Line 11 and 12. At this stage, we have not considered adapting any advanced heuristic optimizations of the original subgraph isomorphism problem to ours. In the experiments we simply implemented a depth-first search checker, utilizing an index built on all label names of the large graph $G$.

In [9], Kuramochi et al. used TID (Transaction Identifier) lists to optimize their FSM algorithm under the graph transaction setting. Analogously, we propose VID (Vertex Identifier) lists to improve our efficiency both in the building block construction phase and the joining phase. Both of our optimizations origin from the fact that for any patterns $\mathcal{P}_{1}\subseteq_{f}\mathcal{P}_{2}$, the set of vertices (transactions in Kuramochi’s cases) matching $\mathcal{P}_{1}$, i.e., $M_{G}(\mathcal{P}_{1})$, must be a superset of $M_{G}(\mathcal{P}_{2})$. This is essentially a reinforced version of the DCP, which enable us to reduce the number of vertices considered when counting the support of a candidate. To utilize it, we have to maintain the IDs of all vertices in $M_{G}(P)$ as an ordered list for any $P$, instead of recording only its size.

Specifically, at Line 7 in Algorithm 1, when extending $path$, we only need to consider $M_{G}(path)$ instead of all vertices in $G$. In Algorithm 2, for each enumerated pair of patterns at Line 7, we first intersect $M_{G}(F_{k-1}[i])$ and $M_{G}(F_{k-1}[j])$ in linear time. If the number of results is below $\tau$, they need not be joined because vertices matching their shared super-patterns must be within the intersection. If they pass the test, $M_{G}(F_{k-1}[i])\cap M_{G}(F_{k-1}[j])$ is saved, and at Line 11 we only need to verify the intersection instead of the whole $V(G)$ to count the support of the size-$K$ patterns. Let’s take the join in Figure 5(a) on the toy database in Figure 2 for example. The upper left pattern matches author $a_{1}$,$a_{2}$, and $a_{3}$, and the lower left one matches $a_{1}$,$a_{2}$,$a_{3}$, and $a_{4}$. Since $a_{4}$ does not appear in the intersection, it will not be checked when computing the support of the joined pattern. Vertices matching the pattern under consideration are stored again in VID lists at Line 9 of Algorithm 1 and Line 11 of Algorithm 2 for larger patterns’ use.

In experiments, the VID-optimization reduces the running time by up to two orders of magnitude. In Section 4 we will discuss in detail the experimental results and the feasibility of this optimization.

The EntityCube system is a research prototype for exploring object-level search technologies, which automatically summarizes the Web for entities (such as people, locations and organizations). We utilized the relationship network between person entities extracted by the system. Specifically, with a list of people names as seeds, we queried the system using one name each time, and got related persons and the corresponding relationship names as return. On the one hand, the seed persons and returned persons were used to form the vertex set $V$. On the other hand, the system returns two types of relationship. The name of one is in a plural form, such as “politicians(Barack Obama, Bill Cliton)”, indicating the connection that they’re both politicians. Thus, the relationship name naturally served as vertex labels for the two associated entities. The other type of relationship appears in a singular form, e.g., “wife(Michelle Obama, Barack Obama)”. They were interpreted as labeled edges between the corresponding vertices.

The ArnetMiner Citation Network dataset contains many papers with associated attribute information, as well as their citation relationship. The dataset consists of five versions, while we use the fifth one. We extracted all papers, authors, and conferences appearing in the data, and construct the vertex set with them. Conferences of the same series and on different years are treated as identical. There are only three types of labeled edges, i.e., “writes” between an author and a paper, “accepts” between a conference and a paper, and “cites” between a paper and another. Because these edge label names actually implies the type of both the starting and ending vertices of an edge, we didn’t employ any vertex label to avoid redundancy. In the data, IDs are provided to uniquely denote papers and form citations, which were adopted by us. However in the author and conference sections of each paper, only texts are presented. Therefore, when converting them to the IDs in our algorithm, we required exact text-match and didn’t perform any cleaning operation involving external data.

In this section, we report the performance of our algorithms for frequent neighborhood mining. Since no previous work has addressed exactly the same problem, our experiments were dedicated to validating the feasibility of our VID optimization. In practice, the running time of such a pattern mining algorithm is heavily influenced by the size of the result set. Therefore, we decided to conduct the experiments on the EntityCube data with rich label names, for it has a potentially larger result set and the running time is more sensitive to parameters such as the minimum support $\tau$. In all experiments, $\tau$ was chosen from $\{0.0001,0.0002,0.0005,0.001\}$ and all reported time was an average of five consecutive runs.

In Figure 6(a) and 6(b), we terminated the search after all frequent patterns of size below 4 were discovered. It is clear that our VID optimization successfully accelerates both the building block construction and the join-verify phases by up to one and two orders of magnitude, respectively. Analogous to the TID optimization [9] for FSM, the advantages of the VID optimization are two-fold. First, two patterns will not be joined if the intersection of their VID lists is smaller than $\tau$. Therefore, the algorithm successfully avoids verifying false positives caused by joining unpromising pattern pairs, which is a vital overhead to the overall performance. In Figure 6(c), the number of candidates with/without the VID-list-pruning, and the number of true patterns are illustrated. This figure shows that the pruning helps narrow down the number of candidates by several times. Second, for each pair of patterns that passes the pruning, the time spent on counting the support of the joining results is also reduced because vertices not in the intersection won’t contribute to the count, thus are not checked. Figure 6(d) presents the average verification time for each non-path candidate where $\tau=0.0001$. The x-axis denotes different stages of the search procedure, where candidates of size 2 to 5 are verified. Obviously the VID optimization is significantly effective for verifying candidates of all sizes. Particularly, without the optimization, the algorithm didn’t finish the verification of size 5 in reasonable time.

As is mentioned above, the major superiority of FNM over other graph pattern mining methods is that it discovers patterns with cycles, which is not targeted by others. With the hands-on experiences of experimenting on both datasets, we realize that a cycled pattern can be viewed as a set of constraints with lower degree of freedom than a tree-like one of the same size. Figure 6(e) shows the constitution of all patterns in EntityCube data, whose size are below 5. Patterns with cycles actually make up around 10% among all three types. The trend also shows that when we decrease the support ratio and specify a pattern into its super-patterns, it is more difficult for a cycle to form, than a fork to appear.

However, once formed, patterns with cycles serve as a good complement to tree-like patterns. Introducing them does not linearly increase our knowledge about the data being investigated, but actually makes a mutual reinforcement with tree-like ones. As patterns from the ArnetMiner dataset have better interpretability, we selected some interesting neighborhood patterns mined from this dataset to demonstrate our points. Besides the example given in Figure 1, more patterns are displayed in Figure 7. By combining two or more of them, we can make very interesting discoveries about the academia.

For example, the support ratio of Figure 7(g) is lower than that of Figure 1(a) and 7(f), which reflects the common sense that it is more difficult to get one’s paper cited than to write more papers. Besides, the small gap between the ratios of Figure 1(a) and 7(e) reveals the fact that most writers are willing to maintain a co-authoring relationship. On the other hand, the ratios of Figure 1(a) and 7(h) together prove that an average author relatively favors a conference which once accepted his paper. Moreover, Figure 7(c) alone points out that most of us (assuming that we all have papers) have a paper with no less than three authors. Surprisingly, as Figure 7(l) indicates, there are even papers citing each other! By checking the data we find two cases of such a phenomenon. One is caused by the dataset itself. The data treats books as conferences, and their chapters as papers. Of course, chapters from the same book can cite each other. This case is rare. The other case is more often: an author simultaneously submitted two papers to the same conference and got them both accepted. When preparing the camera-ready versions, he made them citing each other.

For all patterns presented in this paper, we use support ratios w.r.t. vertices with a specified label, instead of the absolute count mentioned in the problem statement. It’s easy to implement, which only involves a small modification to the algorithm. Suppose we want facts about all authors with a minimum support ratio of 10%. In Algorithm 1 and 2, when a scan on the entire $V(G)$ is required for support counting, we only scan those author vertices by accessing an index on the label names. Moreover, when calling the modified algorithms, $\tau$ should be assigned with 10% the number of all author vertices. When mining patterns about authors, conferences, and papers, the support ratios were set to 1%. As the number of paper vertices is huge (over 1.5 million), the support calculation was performed on a subset of 0.5 million papers we sampled. We didn’t explore path patterns of size 4, or any pattern whose size exceeds 4. Readers may refer to Table 2 for more running details.

The frequent subgraph mining problem is well-investigated by the literatures under the graph-transaction setting. Among them, the most influential methods are AGM[7], FSG[9], and gSpan[18]. The first and second adopts the apriori-based BFS scheme, and feature the vertex-incremental and edge-incremental approaches, respectively. The last one falls into a pattern-growth-based DFS category. Optimizations they utilized, such as canonical labeling and vertex invariants, are inspiring and potentially employable to our work. The single-graph setting, however, is not so fully explored due to reasons we mentioned above. Among the few support measures proposed, the Maximum Independent Set support and corresponding mining algorithms are studied in [10, 17]. [2] also defines a single-graph support called Minimum Image Support, which still doesn’t make a intuitive one and has yet to be tested for easiness of handling.

In [5, 6, 8], the authors attempt to mine tree patterns in graphs, whose support measure resembles ours in the way that distinct matches of some vertices are counted, while the match conditions on the others are only existential. They do not target patterns with cycles, which add much to the users’ understand of the data. Moreover, because [5, 6] allow multiple vertices to be counted (in other words, as our “pivots”), their problems are more complicated and thus do not completely follow and benefit from the well-solved apriori pattern mining scheme. We argue that, patterns with more than three pivots may explode in the number, while bringing about some knowledge that is hard to explain and utilize. Finally, these works both claimed that their mining algorithms support constants in the patterns, e.g., “x% of the authors once cited a paper published in KDD”. Our problem setting supports multiple labels on a vertex. Therefore, we can achieve it by simply adding the name of each vertex to its label set. We can also modify our algorithm to implicitly perform such a data transformation.

In [3], Dehaspe et al. introduced an inductive logic programming system for mining frequent patterns in a datalog database. Their final products are rules, which are more advanced than ours. However, they require language biases as additional inputs to bound the search space. In contrast, our method is completely unsupervised. Methods learning horn clauses from knowledge bases such as [12, 11, 4] also resemble the Inductive Logic Programming category. These works are characterized by a variety of metrics to evaluate the utility of a rule. Since noise and scalability issues in real data are their main concerns, they adopt stricter language biases and the rules mined are of more limited forms.