Journal of East China Normal University(Natural Science) >
Knowledge graph completion by integrating textual information and graph structure information
Received date: 2024-01-02
Online published: 2025-01-20
Copyright
Based upon path query information, we propose a graph attention model that effectively integrates textual and graph structure information in knowledge graphs, thereby enhancing knowledge graph completion. For textual information, a dual-encoder based on pre-trained language models is utilized to separately obtain embedding representations of entities and path query information. Additionally, an attention mechanism is employed to aggregate path query information, which is used to capture graph structural information and update entity embeddings. The model was trained using contrastive learning and experiments were conducted on multiple knowledge graph datasets, with good results achieved in both transductive and inductive settings. These results demonstrate the advantage of combining pre-trained language models with graph neural networks to effectively capture both textual and graph structural information, thereby enhancing knowledge graph completion.
Houlong FAN , Ailian FANG , Xin LIN . Knowledge graph completion by integrating textual information and graph structure information[J]. Journal of East China Normal University(Natural Science), 2025 , 2025(1) : 111 -123 . DOI: 10.3969/j.issn.1000-5641.2025.01.009
1 | JI S X, PAN S R, CAMBRIA E, et al.. A survey on knowledge graphs: Representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33 (2): 494- 514. |
2 | SHEN T, ZHANG F, CHENG J W. A comprehensive overview of knowledge graph completion[J]. Knowledge-Based Systems, 2022, 255: 109597. |
3 | LIANG X Y, SI G N, LI J X, et al. A survey of inductive knowledge graph completion[J]. Neural Computing and Applications, 2024, 36: 3837-3858. |
4 | OH B, SEO S, HWANG J, et al.. Open-world knowledge graph completion for unseen entities and relations via attentive feature aggregation. Information Sciences, 2022, 586, 468- 484. |
5 | SCHLICHTKRULL M, KIPF T N, BLOEM P, et al. Modeling relational data with graph convolutional networks [C]// The Semantic Web: 15th International Conference. 2018: 593-607. |
6 | ZHANG X, ZHANG C X, GUO J T, et al.. Graph attention network with dynamic representation of relations for knowledge graph completion. Expert Systems with Applications, 2023, 219, 119616. |
7 | XU H C, BAO J P, LIU W B. Double-branch multi-attention based graph neural network for knowledge graph completion [C]// Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. 2023: 15257-15271. |
8 | LOVELACE J, ROSé C. A framework for adapting pre-trained language models to knowledge graph completion [C]// Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022: 5937-5955. |
9 | WANG L, ZHAO W, WEI Z Y, et al. SimKGC: Simple contrastive knowledge graph completion with pre-trained language models [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 2022: 4281-4294. |
10 | SAXENA A, KOCHSIEK A, GEMULLA R. Sequence-to-sequence knowledge graph completion and question answering [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 2022: 2814-2828. |
11 | JIANG P C, AGARWAL S, JIN B W, et al. Text augmented open knowledge graph completion via pre-trained language models [C]// Findings of the Association for Computational Linguistics: ACL 2023. 2023: 11161-11180. |
12 | CHEN C, WANG Y F, SUN A X, et al. Dipping PLMs sauce: Bridging structure and text for effective knowledge graph completion via conditional soft prompting [C]// Findings of the Association for Computational Linguistics: ACL 2023. 2023: 11489-11503. |
13 | MARKOWITZ E, BALASUBRAMANIAN K, MIRTAHERI M, et al. StATIK: Structure and text for inductive knowledge graph completion [C]// Findings of the Association for Computational Linguistics: NAACL 2022. 2022: 604-615. |
14 | CHEPUROVA A, BULATOV A, KURATOV Y, et al. Better together: Enhancing generative knowledge graph completion with language models and neighborhood information [C]// Findings of the Association for Computational Linguistics: EMNLP 2023. 2023: 5306-5316. |
15 | DAZA D, COCHEZ M, GROTH P. Inductive entity representations from text via link prediction [C]// Proceedings of the Web Conference 2021. 2021: 798-808. |
16 | LYU X, LIN Y K, CAO Y X, et al. Do pre-trained models benefit knowledge graph completion? A reliable evaluation and a reasonable approach [C]// Findings of the Association for Computational Linguistics: ACL 2022. 2022: 3570-3581. |
17 | SUN Z, DENG Z H, NIE J Y, et al. RotatE: Knowledge graph embedding by relational rotation in complex space [EB/OL]. (2019-02-26)[2024-03-30]. https://arxiv.org/pdf/1902.10197.pdf. |
18 | WANG B, SHEN T, LONG G D, et al. Structure-augmented text representation learning for efficient knowledge graph completion [C]// Proceedings of the Web Conference 2021. 2021: 1737-1748. |
19 | YU C M, ZHANG Z G, AN L, et al.. A knowledge graph completion model integrating entity description and network structure. Aslib Journal of Information Management, 2023, 75 (3): 500- 522. |
20 | LI R, ZHAO J A, LI C E, et al. House: Knowledge graph embedding with Householder parameterization [C]// International Conference on Machine Learning. 2022: 13209-13224. |
21 | LI D, ZHU B Q, YANG S, et al.. Multi-task pre-training language model for semantic network completion. ACM Transactions on Asian and Low-Resource Language Information Processing, 2023, 22 (11): 250- 20. |
22 | GESESE G A, SACK H, ALAM M. RAILD: Towards leveraging relation features for inductive link prediction in knowledge graphs [C]// Proceedings of the 11th International Joint Conference on Knowledge Graphs. 2022: 82-90. |
23 | KIM B, HONG T, KO Y, et al. Multi-task learning for knowledge graph completion with pre-trained language models [C]// Proceedings of the 28th International Conference on Computational Linguistics. 2020: 1737-1743. |
24 | HAN S, GUAN Z Y, LI S H, et al. Knowledge graph completing with dual confrontation learning model based on variational information bottleneck method [C]// 2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security. 2023: 741-750. |
25 | SUN W, LI Y F, YAO J F, et al. Combining structure embedding and text semantics for efficient knowledge graph completion [C]// The 35th International Conference on Software Engineering and Knowledge Engineering. 2023: 317-322. |
/
〈 |
|
〉 |