LMsForKBC: Evaluating Language Models for Knowledge Base Completion

In our paper Evaluating Language Models for Knowledge Base Completion we analyze the feasibility of Knowledge Base Completion by using Language Models. As a language model we use BERT and as a knowledge base we use Wikidata. We publish a new benchmark dataset WD-KNOWN along with the human evaluation of predictions for really missing facts, i.e. (subject,relation)-pairs non-existent in Wikidata.

Dataset

Please find the WD_known dataset and the unknown facts, i.e. facts that are not present in WikiData, in the data folder.

Reference

@inproceedings{veseli2023kbc,
  title={Evaluating Language Models for Knowledge Base Completion},
  author={Blerta Veseli, Simon Razniewski, Sneha Singhania, Gerhard Weikum},
  booktitle={In:  20th Extended Semantic Web Conference (ESWC), 2023},
  year={2023}
}