A Novel Customizing Knowledge Graph Evaluation Method for Incorporating User Needs.

Year:	2024
Title:	A Novel Customizing Knowledge Graph Evaluation Method for Incorporating User Needs.
Abstract:	Introduces an accuracy-focused KG evaluation method that incorporates user requirements. It designs an effective two-stage weighted cluster sampling (EP-TWCS) to focus on user-important entities. Experiments show the sampled accuracy closely matches true accuracy with minimal sample size.
Objectives:	Ensure accuracy assessment of KGs in a cost-saving way that meets specific user requirements.
Methodology:	EP-TWCS sampling: assigns weights based on user usage frequency and entity popularity, then performs stratified sampling to estimate KG accuracy.
Algorithm Used:	EP-TWCS (Entity Popularity – Two-stage Weighted Cluster Sampling)
Top Model:	Sampling-based evaluation
Accuracy:	Accuracy of sampled assessment nearly equals real accuracy; sample size is minimized.
Advantages:	User-centric: tailors sampling to what users care about; efficient (minimal sample size) while maintaining accuracy.
Drawbacks:	Focuses only on the accuracy dimension of quality; requires knowledge of user usage weights; may not address completeness or consistency.
Source:	https://www.researchgate.net/publication/380129766_A_novel_customizing_knowledge_graph_evaluation_method_for_incorporating_user_needs#full-text Edit

Criteria (6)

Name	Description	Definitions
Accuracy	This criterion measures the accuracy of knowledge graph data by assessing the correctness of facts and the accuracy of relationships …	Quality dimension from A Novel Customizing Knowledge Graph Evaluation Method for Incorporating User Needs. (2024) Introduces an accuracy-focused KG evaluation method that incorporates user requirements. It designs an effective two-stage …
Relevancy (User Needs	This criterion measures the relevancy of entities in the knowledge graph by evaluating how well they align with user needs, …	Introduces an accuracy-focused KG evaluation method that incorporates user requirements. It designs an effective two-stage …
Relevancy (User Needs)	Quality dimension from A Novel Customizing Knowledge Graph Evaluation Method for Incorporating User Needs.	Quality dimension from A Novel Customizing Knowledge Graph Evaluation Method for Incorporating User Needs. (2024)
Value added	Measures the additional value that the knowledge graph provides beyond basic data storage.	Quality dimension from A Novel Customizing Knowledge Graph Evaluation Method for Incorporating User Needs. (2024) Introduces an accuracy-focused KG evaluation method that incorporates user requirements. It designs an effective two-stage …
Timeliness (Efficiency	This criterion measures the timeliness or efficiency of a knowledge graph by evaluating the accuracy of the sampled entities in …	Introduces an accuracy-focused KG evaluation method that incorporates user requirements. It designs an effective two-stage …
Timeliness (Efficiency)	Quality dimension from A Novel Customizing Knowledge Graph Evaluation Method for Incorporating User Needs.	Quality dimension from A Novel Customizing Knowledge Graph Evaluation Method for Incorporating User Needs. (2024)