Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure)

Year: 2024
Title: Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure)
Abstract: Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, eliminating the need for gold standards.
Objectives: To automatically assess consistency and redundancy in KGs, serving as an effective, quantitative proxy for correctness and completeness.
Methodology: Automated evaluation by training link prediction models on the KG and testing their ability to recover intentionally removed triples.
Algorithm Used: Link Prediction Models (KG embedding techniques)
Top Model: LP-Measure
Accuracy: Effective in distinguishing high-quality KGs from low-quality KGs
Advantages: Automated and scalable; eliminates the need for expensive gold standards or human labor; correlates strongly with correctness.
Drawbacks: Metric fundamentally assesses consistency/redundancy; cannot assess correctness/completeness directly (when gold standards are absent).
Source: https://www.pure.ed.ac.uk/ws/portalfiles/portal/395923477/NLPIR2023_2_.pdf Edit

Criteria (17)

Name Description Definitions
Availability Checks if the knowledge graph is accessible when needed and can be reliably accessed by users and applications.
  • Quality dimension from Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) …
  • Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, …
Licensing In the LP-Measure framework, the "Licensing" criterion assesses the adherence to licensing agreements and terms for the data used in …
  • Quality dimension from Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) …
  • Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, …
Interlinking In the framework Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure), the criterion "Interlinking" refers to …
  • Quality dimension from Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) …
  • Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, …
Security Assesses how well the knowledge graph is protected against unauthorized access, data breaches, and security threats.
  • Quality dimension from Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) …
  • Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, …
Performance In the framework Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure), the criterion "Performance" refers to …
  • Quality dimension from Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) …
  • Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, …
Relevancy Relevancy in the LP-Measure framework is defined as the ability of the knowledge graph to correctly predict missing links that …
  • Quality dimension from Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) …
  • Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, …
Trustworthiness In the LP-Measure framework, Trustworthiness is defined as the ratio of correctly predicted links to the total number of links …
  • Quality dimension from Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) …
  • Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, …
Understandability In the framework Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure), the criterion of Understandability refers …
  • Quality dimension from Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) …
  • Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, …
Timeliness Measures how up-to-date the knowledge graph is. It checks if the information reflects current knowledge and is regularly updated.
  • Quality dimension from Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) …
  • Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, …
Validity Validity in the LP-Measure framework refers to the ability of a knowledge graph to correctly predict existing links, which is …
  • Quality dimension from Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) …
  • Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, …
Accuracy In the framework Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure), the criterion "Accuracy" refers to …
  • Quality dimension from Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) …
  • Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, …
Consistency Checks if the knowledge graph follows consistent rules and patterns. It ensures there are no contradictory facts or conflicting relationships.
  • Quality dimension from Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) …
  • Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, …
Conciseness Measures how efficiently information is represented without unnecessary redundancy or verbosity.
  • Quality dimension from Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) …
  • Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, …
Completeness Assesses how much of the expected knowledge is present in the graph. It checks if all relevant entities, relationships, and …
  • Quality dimension from Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) …
  • Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, …
Interoperability In the framework Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure), interoperability is defined as the …
  • Quality dimension from Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) …
  • Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, …
Interpretability In the LP-Measure framework, the criterion of interpretability refers to the ability of the knowledge graph to provide clear and …
  • Quality dimension from Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) …
  • Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, …
Versatility The criterion "Versatility" in Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) refers to the ability …
  • Quality dimension from Assessing the Quality of a Knowledge Graph via Link Prediction Tasks (LP-Measure) …
  • Introduces the LP-Measure, an automated approach to assess consistency and redundancy using link prediction tasks, …
Back to Frameworks Compare with Others