Efficient Knowledge Graph Accuracy Evaluation

Year:	2019
Title:	Efficient Knowledge Graph Accuracy Evaluation
Abstract:	Proposes an efficient sampling framework to estimate large-scale KG accuracy with strong statistical guarantees while minimizing the cost of human annotation.
Objectives:	To obtain statistically meaningful estimates for accuracy evaluation while dramatically minimizing the required human efforts (annotation costs).
Methodology:	Application of sampling theory; analysis of annotation cost functions; extension for dynamic/incremental evaluation.
Algorithm Used:	Cluster Sampling, Stratified Sampling, Weighted Reservoir Sampling
Top Model:	Efficient Sampling Framework
Accuracy:	Up to 60% cost reduction (static KG); Up to 80% cost reduction (evolving KG)
Advantages:	Makes large-scale, gold-standard accuracy checks economically feasible; provides statistically guaranteed estimates for precision/recall.
Drawbacks:	Focuses predominantly on the Accuracy dimension, requiring external human annotation expertise.
Source:	https://www.vldb.org/pvldb/vol12/p1679-gao.pdf Edit

Name	Description	Definitions
Accuracy	Proposes an efficient sampling framework to estimate large-scale KG accuracy with strong statistical guarantees while minimizing the cost of human …	Quality dimension from Efficient Knowledge Graph Accuracy Evaluation (2019) Proposes an efficient sampling framework to estimate large-scale KG accuracy with strong statistical guarantees while …
Cost Effectiveness (Value added	Proposes an efficient sampling framework to estimate large-scale KG accuracy with strong statistical guarantees while minimizing the cost of human …	Proposes an efficient sampling framework to estimate large-scale KG accuracy with strong statistical guarantees while …
Cost Effectiveness (Value added)	Quality dimension from Efficient Knowledge Graph Accuracy Evaluation	Quality dimension from Efficient Knowledge Graph Accuracy Evaluation (2019)
Timeliness (Efficiency focus	Proposes an efficient sampling framework to estimate large-scale KG accuracy with strong statistical guarantees while minimizing the cost of human …	Proposes an efficient sampling framework to estimate large-scale KG accuracy with strong statistical guarantees while …
Timeliness (Efficiency focus)	Quality dimension from Efficient Knowledge Graph Accuracy Evaluation	Quality dimension from Efficient Knowledge Graph Accuracy Evaluation (2019)