Re-evaluation of Knowledge Graph Completion Methods

Original article can be found here (source): Artificial Intelligence on Medium

Re-evaluation of Knowledge Graph Completion Methods

written by Farahnaz Akrami

Ph.D. student in IDIR lab, Farahnaz Akrami and her collaborators, Mohammed Samiul Saeef, Qingheng Zhang, Wei Hu, and Chengkai Li have a paper accepted by SIGMOD 2020. The paper titled “ Realistic Re-evaluation of Knowledge Graph Completion Methods: An Experimental Study” investigates the true effectiveness of knowledge graph completion models and particularly embedding models through extensive experiments. It shows that data redundancy and test leakage in widely-used benchmark datasets caused an overestimation of the accuracy of many models. Moreover, many of test cases used to evaluate the models are unrealistic and nonexistent in real-world scenarios. You can find more about the paper in the Medium blog post. The full version of the paper is available on arXiv. All codes, experiment scripts, datasets, and results are in a public repository.