Enriching Wikidata with Linked Open Data

Commonly used knowledge graphs (KGs) today have wide coverage, recording information for many millions of entities through thousands of properties. Wikidata [28], the largest public KG, contains nearly 1.5B statements about 90M entities. This breadth of information inspires various use cases to exploit such KGs. Museum curators can use Wikidata to describe their art collections and artists comprehensively. Movie critics could quickly query and aggregate statistics about recent movies, and analyze them based on their genre or actor cast.

However, while Wikidata’s data model allows for this information to be present, practice shows that much of the relevant information is still missing in Wikidata. For example, around half of the artists in Wikidata have a date of birth, and only 1.88% of the movies recorded in 2020 have information about their cost. Thus, if a user wants to analyze the cost of the films produced in 2020, Wikidata will provide cost information for only 60 out of its 2,676 recorded films. As this information is unlikely to be found in an aggregated form, and gathering information about thousands of films is a tedious process, one must rely on automated tools that can enrich Wikidata with relevant information. Such information might be readily available in external linked open data (LOD)¹¹1https://lod-cloud.net/ sources like DBpedia [1] or LinkedMDB [17]; yet, no existing methods can enrich Wikidata with missing information. Conversely, link prediction [6] accuracy is not sufficient to use it to impute missing information with representation learning.

The traditional LOD workflow includes declarative language tools for schema mapping and templated rewritting of CSV files into RDF [9, 20, 11]. A subsequent ontology alignment step [7] can be employed to discover owl:sameAs links between the two datasets. Yet, prior work [4] has shown that collapsing nodes based on owl:sameAs relations is not feasible, as it leads to combining related but dissimilar nodes (e.g., Barack Obama and the US government) into a single node. Moreover, while prior efforts have focused on schema alignment between two sources, this challenge may be less significant when augmenting Wikidata, which provides 6.8k external identifier properties that explicitly link to entities in other sources.²²2https://www.wikidata.org/wiki/Category:Properties_with_external-id-datatype, accessed March 2, 2022. Further challenge is data quality [23], as external sources may contain information that is noisy, contradictory, or inconsistent with existing knowledge in Wikidata. Table 1 presents candidate enrichment statements retrieved from DBpedia infoboxes. Here, only one out of four statements is correct, whereas the other three have an incorrect datatype, semantic type, or veracity. It is unclear how to employ principled solutions for schema alignment and quality validation to for large-scale and high-quality enrichment of Wikidata.

In this paper, we investigate how to enrich Wikidata with freely available knowledge from the LOD cloud. We introduce a novel LOD workflow to study the potential of the external identifier mechanism to facilitate vast high-quality extensions of Wikidata. Assuming that we can link Wikidata entities to external sources robustly through external identifiers, we start by aligning the entities automatically, followed by inferring the property in the external source that corresponds to the Wikidata property we seek to augment. This alignment of entities and properties yields candidate enrichment statements, which are rigorously checked through datatype comparison, semantic constraints, and custom validators. We demonstrate the effectiveness of the augmentation workflow on two LOD knowledge sources: Getty [16] and DBpedia [1]. Getty is a manually curated domain-specific knowledge graph with a narrow focus on art, whereas DBpedia is a broad coverage KG which has been automatically extracted from Wikipedia. Extensive experiments on these sources show that our method facilitates vast and high-quality extension of Wikidata with missing knowledge.

We make our code and data available to facilitate future work.³³3https://anonymous.4open.science/r/hunger-for-knowledge-315D/

Schema mapping languages, like R2RML [9], Karma [20], and RML [11] enable users to map relational databases into RDF, optionally followed by a semi-automatic ontology alignment step. Recent ontology alignment methods [7] typically align two ontologies in the vector space. Compared to prior works that align two schemas or ontologies, we seek to improve the coverage of a single large knowledge graph, by selective augmentation with external knowledge.

Ontology enrichment deals with noise, incompleteness, and inconsistencies of ontologies, by discovering association rules [8], or by extracting information from WWW documents [13]. Ontology evolution seeks to maintain an ontology up to date with respect to the domain that it models, or the information requirements that it enables [30]. Notably, there have been no efforts to enrich or improve the evolution of large-scale hyperrelational KGs, like Wikidata.

Prior work has attempted to enrich Wikidata with satellite data for a given domain, including frame semantics [21], biodiversity data [29], and cultural heritage data [14]. By learning how to represent entities in a KG like Wikidata, link prediction models [6] can predict missing values by associating them to known statements. Our work complements prior efforts that enrich Wikidata with domain-specific data or predict missing links, as we aim to devise a method for generic, large-scale enrichment of Wikidata with external LOD sources.

The linked open data cloud contains many identity links which could be explored to combine information from different sources. LOD Laundromat [5] is a centralized infrastructure which captures a static version of the LOD cloud, which could be queried in order to obtain information from multiple sources at once. However, as we show in this paper, identity links between entities by themselves are insufficient for enrichment of knowledge graphs. The correspondence between the properties in two graphs may not be one-to-one, requiring inference about property paths. Furthermore, semantic validation needs to be performed to ensure that the mapped properties indeed have compatible semantics. In that sense, our work relates to prior work that explores identity links in the LOD cloud [4] or devises mechanisms to discover latent identity links [24]. The goal of our work is different - to enrich Wikidata by finding high-quality knowledge in well-connected sources.

The automatic validation of our method relates to prior work on analyzing the quality of large KGs. Beek et al. [3] propose a framework for analyzing the quality of literals on the LOD Laundromat, but it is unclear how to generalize this framework to entity nodes. Prior work has studied the quality of Wikidata [22, 26] and compared it to the quality of other knowledge graphs, like YAGO [27] and DBpedia [15]. Compared to Wikidata, YAGO4 takes a step further and it defines constraints on domain and range, disjointness, functionality, and cardinality. The authors report that enforcing these constraints leads to a removal of 132M statements from Wikidata, i.e., 28% of all facts. In [26], the authors apply five semantic constraints, including value type, on Wikidata in order to measure its quality. None of these works has investigated how to automatically validate external statements that can be used to enrich Wikidata.

Wikidata’s property constraint pages define existing property constraints and report number of violations for a single dump.⁴⁴4https://www.wikidata.org/wiki/Help:Property_constraints_portal Moreover, Wikidata includes several tools that monitor, analyze, and enforce aspects of quality, such as the primary sources tool (PST),⁵⁵5https://www.wikidata.org/wiki/Wikidata:Primary_sources_tool#References the Objective Revision Evaluation Service (ORES),⁶⁶6https://www.wikidata.org/wiki/Wikidata:ORES and Recoin (“Relative Completeness Indicator”) [2]. Recoin computes relative completeness of entity information by comparing the available information for an entity against other similar entities. Our work complements these efforts by Wikidata, by providing mechanisms for automatic alignment and semantic validation of knowledge from the LOD cloud before it is submitted to Wikidata.

Our enrichment method is shown in Figure 1. Given a user query, our method queries Wikidata ($W$), obtaining a set of known statements $S_{w}$ for subjects $E_{w}$, and a set of subjects with unknown values $E_{u}$. We call this step gap detection, as it generates a set of entities for which we seek missing knowledge in order to satisfy the user’s query needs. Considering that the LOD cloud contains other sources that are likely to contain information about the same entities, we perform a KG selection step to determine a relevant set of KGs $G$ to consult for the entities in $E_{u}$. Here, the sources $G$ are assumed to overlap with $W$ in terms of the entities they describe. Our schema alignment step consolidates the entities and properties of $W$ with those of each $G$, since their entity and property identifiers are generally different. After aligning the two schemas, a knowledge retrieval step queries the external KG $G$ to obtain new statements $S_{g}$ for the entities in $E_{u}$. A semantic validation step is employed in order to ensure that the semantics of the newly found statements in $S_{g}$ corresponds to the semantics intended by Wikidata. The set of validated statements $S_{e}\subseteq S_{g}$ is finally used to enrich $W$. This procedure yields a more complete response to the user query consisting of a union of the original and the enriched statements, formally: $S_{total}=S_{w}\cup S_{e}$.

We show this procedure by querying Wikidata for “industry of companies” in Figure 1. The gap detection step splits the overall set of target companies into two groups: 1) companies with known industry in Wikidata ($E_{w}$); and 2) companies with unknown industry ($E_{u}$), for which $W$ lacks industry information. The KG selection step decides that DBpedia may contribute missing knowledge for company industries. The schema alignment step relies on the statements in Wikidata, $S_{w}=\{(e_{w},p,o_{w})|e_{w}\in E_{w}\}$ about the known companies $E_{w}$. Each pair $(e_{w},o_{w})$ is aligned to a DBpedia-valued pair $({e_{w}}^{\prime},{o_{w}}^{\prime})$. Similarly, the unknown entities $e_{u}\in E_{u}$ are mapped in the same way to DBpedia entities ${e_{u}}^{\prime}$. Then, the external KG DBpedia is queried for property paths which correspond to the known subject-object pairs $({e_{w}}^{\prime},{o_{w}}^{\prime})$. The DBpedia path $p^{\prime}$ (dbp:industry) that corresponds to the Wikidata property $p$ is discovered with our method. The knowledge retrieval step obtains values from DBpedia, by querying for pairs $({e_{u}}^{\prime},p^{\prime})$ comprised of DBpedia entities and property paths, resulting in a new set of statements $S_{g}=\{({e_{u}}^{\prime},p^{\prime},o_{g})\}$. The newly found values are inversely mapped back to ‘found’ (candidate) Wikidata values ${o_{g}}^{\prime}$. Each of these newly obtained values is semantically validated, resulting in a subset of enrichment values ${o_{e}}$, which form the statements $(e_{u},p,o_{e})$.

To investigate the quality of our property alignment, we use property mappings provided by owl:equivalentProperty in DBpedia and P1628 (equivalent property) in Wikidata as ground truth to evaluate the property alignment of our method. We dub this data Equivalence. In the property mapping pairs we collected, each Wikidata property is mapped to one or multiple DBpedia properties. In total, 88 Wikidata properties are mapped to 101 DBpedia properties. We formulate a task where the goal is to map a Wikidata property to its corresponding DBpedia property/properties. We accept any of the DBpedia mappings according to the gold standard data as correct. Besides correct and incorrect mappings, we annotate an intermediate category of close match for properties that match partially. We show two F1-values of our method: hard, which only counts exact matches, and soft, which includes partial matches. We compare our method to three baselines. The string matching and frequency matching baselines are ablations of our method that only consider string similarity or frequency, but not both. The third baseline embeds all DBpedia labels with BERT [10] and uses cosine distance to select the closest property label.

In this Section, we present six experimental findings that concern the potential of our method, its overall accuracy, its accuracy per component, and its consistency when the external information is covered by Wikidata.

Our enrichment method has been shown to quickly retrieve millions of novel property values in the LOD cloud for entities in Wikidata. As some of the LOD knowledge is extracted in an automatic way, ensuring quality is important - our semantic validation based on datatypes and constraints found around half of the candidate statements to be invalid. Analysis of a subset of the enriched statements revealed that the accuracy of our method is close to 90%, which is reasonably high. Still, our method is merely a step towards the ambitious goal of addressing the notorious challenge of sparsity of today’s large KGs [12]. Here, we discuss three key areas of improvement for our method:

Recognizing the notorious sparsity of modern knowledge graphs, such as Wikidata, and the promise of linked data information, like external identifiers, to facilitate enrichment, this work proposed a method for enriching Wikidata with external KGs found in the LOD cloud. The method consisted of five steps: gap detection, external graph selection, schema alignment, knowledge retrieval, and semantic validation. We evaluated our method on enriching Wikidata with two LOD graphs: DBpedia and Getty. Our experiments showed that our LOD-based method can enrich Wikidata with millions of new high-quality statements in a short time. High-quality enrichment is achieved based on large-scale automated semantic validation based on datatypes and value type constraints, as well as a hybrid algorithm for property alignment. A key future direction is evaluating the generalization of our method on thousands of LOD sources that Wikidata points to, which opens novel challenges of source selection, more extensive semantic validation, and trust. We make our code and data available to facilitate future research on LOD-based knowledge enrichment.