华东师范大学学报（自然科学版） >

2025 , Vol. 2025 >Issue 1: 111 - 123

DOI: https://doi.org/10.3969/j.issn.1000-5641.2025.01.009

计算机科学

文本信息与图结构信息相融合的知识图谱补全

  • 范厚龙 ,
  • 房爱莲 ,
  • 林欣
展开
  • 华东师范大学 计算机科学与技术学院, 上海　200062
林　欣, 男, 研究员, 博士生导师, 研究方向为知识图谱、多模态数据分析、人机混合智能. E-mail: xlin@cs.ecnu.edu.cn

收稿日期: 2024-01-02

  网络出版日期: 2025-01-20

版权

华东师范大学学报期刊社, 2025, 版权所有，未经授权，不得转载、摘编本刊文章，不得使用本刊的版式设计。
收起

Knowledge graph completion by integrating textual information and graph structure information

  • Houlong FAN ,
  • Ailian FANG ,
  • Xin LIN
Expand
  • School of Computer Science and Technology, East China Normal University, Shanghai　200062, China

Received date: 2024-01-02

  Online published: 2025-01-20

Copyright

, 2025, Copyright reserved © 2025.
Fold

摘要

提出了一种基于路径查询信息的图注意力模型, 可以将知识图谱中的文本信息与图结构信息有效融合, 进而提高知识图谱的补全效果. 对于文本信息, 使用基于预训练语言模型的双编码器来分别获得实体的嵌入表示和路径查询信息的嵌入表示. 通过注意力机制来进行路径查询信息的聚合, 以捕获图结构信息, 更新实体的嵌入表示. 模型使用对比学习进行训练, 在多个知识图谱数据集上进行实验, 如直推式、归纳式的方式, 都取得了良好的效果. 结果表明, 将预训练语言模型与图神经网络的优势相结合, 可以有效捕获知识图谱中文本信息与图结构信息, 进而提高知识图谱的补全效果.

关键词： 知识图谱补全; 预训练语言模型; 对比学习; 图神经网络

本文引用格式

范厚龙 , 房爱莲 , 林欣 . 文本信息与图结构信息相融合的知识图谱补全[J]. 华东师范大学学报（自然科学版）, 2025 , 2025(1) : 111 -123 . DOI: 10.3969/j.issn.1000-5641.2025.01.009

Abstract

Based upon path query information, we propose a graph attention model that effectively integrates textual and graph structure information in knowledge graphs, thereby enhancing knowledge graph completion. For textual information, a dual-encoder based on pre-trained language models is utilized to separately obtain embedding representations of entities and path query information. Additionally, an attention mechanism is employed to aggregate path query information, which is used to capture graph structural information and update entity embeddings. The model was trained using contrastive learning and experiments were conducted on multiple knowledge graph datasets, with good results achieved in both transductive and inductive settings. These results demonstrate the advantage of combining pre-trained language models with graph neural networks to effectively capture both textual and graph structural information, thereby enhancing knowledge graph completion.

Key words： knowledge graph completion; pre-trained language model; contrastive learning; graph neural networks

参考文献

1 JI S X, PAN S R, CAMBRIA E, et al.. A survey on knowledge graphs: Representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33 (2): 494- 514.
2 SHEN T, ZHANG F, CHENG J W. A comprehensive overview of knowledge graph completion[J]. Knowledge-Based Systems, 2022, 255: 109597.
3 LIANG X Y, SI G N, LI J X, et al. A survey of inductive knowledge graph completion[J]. Neural Computing and Applications, 2024, 36: 3837-3858.
4 OH B, SEO S, HWANG J, et al.. Open-world knowledge graph completion for unseen entities and relations via attentive feature aggregation. Information Sciences, 2022, 586, 468- 484.
5 SCHLICHTKRULL M, KIPF T N, BLOEM P, et al. Modeling relational data with graph convolutional networks [C]// The Semantic Web: 15th International Conference. 2018: 593-607.
6 ZHANG X, ZHANG C X, GUO J T, et al.. Graph attention network with dynamic representation of relations for knowledge graph completion. Expert Systems with Applications, 2023, 219, 119616.
7 XU H C, BAO J P, LIU W B. Double-branch multi-attention based graph neural network for knowledge graph completion [C]// Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. 2023: 15257-15271.
8 LOVELACE J, ROSé C. A framework for adapting pre-trained language models to knowledge graph completion [C]// Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022: 5937-5955.
9 WANG L, ZHAO W, WEI Z Y, et al. SimKGC: Simple contrastive knowledge graph completion with pre-trained language models [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 2022: 4281-4294.
10 SAXENA A, KOCHSIEK A, GEMULLA R. Sequence-to-sequence knowledge graph completion and question answering [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 2022: 2814-2828.
11 JIANG P C, AGARWAL S, JIN B W, et al. Text augmented open knowledge graph completion via pre-trained language models [C]// Findings of the Association for Computational Linguistics: ACL 2023. 2023: 11161-11180.
12 CHEN C, WANG Y F, SUN A X, et al. Dipping PLMs sauce: Bridging structure and text for effective knowledge graph completion via conditional soft prompting [C]// Findings of the Association for Computational Linguistics: ACL 2023. 2023: 11489-11503.
13 MARKOWITZ E, BALASUBRAMANIAN K, MIRTAHERI M, et al. StATIK: Structure and text for inductive knowledge graph completion [C]// Findings of the Association for Computational Linguistics: NAACL 2022. 2022: 604-615.
14 CHEPUROVA A, BULATOV A, KURATOV Y, et al. Better together: Enhancing generative knowledge graph completion with language models and neighborhood information [C]// Findings of the Association for Computational Linguistics: EMNLP 2023. 2023: 5306-5316.
15 DAZA D, COCHEZ M, GROTH P. Inductive entity representations from text via link prediction [C]// Proceedings of the Web Conference 2021. 2021: 798-808.
16 LYU X, LIN Y K, CAO Y X, et al. Do pre-trained models benefit knowledge graph completion? A reliable evaluation and a reasonable approach [C]// Findings of the Association for Computational Linguistics: ACL 2022. 2022: 3570-3581.
17 SUN Z, DENG Z H, NIE J Y, et al. RotatE: Knowledge graph embedding by relational rotation in complex space [EB/OL]. (2019-02-26)[2024-03-30]. https://arxiv.org/pdf/1902.10197.pdf.
18 WANG B, SHEN T, LONG G D, et al. Structure-augmented text representation learning for efficient knowledge graph completion [C]// Proceedings of the Web Conference 2021. 2021: 1737-1748.
19 YU C M, ZHANG Z G, AN L, et al.. A knowledge graph completion model integrating entity description and network structure. Aslib Journal of Information Management, 2023, 75 (3): 500- 522.
20 LI R, ZHAO J A, LI C E, et al. House: Knowledge graph embedding with Householder parameterization [C]// International Conference on Machine Learning. 2022: 13209-13224.
21 LI D, ZHU B Q, YANG S, et al.. Multi-task pre-training language model for semantic network completion. ACM Transactions on Asian and Low-Resource Language Information Processing, 2023, 22 (11): 250- 20.
22 GESESE G A, SACK H, ALAM M. RAILD: Towards leveraging relation features for inductive link prediction in knowledge graphs [C]// Proceedings of the 11th International Joint Conference on Knowledge Graphs. 2022: 82-90.
23 KIM B, HONG T, KO Y, et al. Multi-task learning for knowledge graph completion with pre-trained language models [C]// Proceedings of the 28th International Conference on Computational Linguistics. 2020: 1737-1743.
24 HAN S, GUAN Z Y, LI S H, et al. Knowledge graph completing with dual confrontation learning model based on variational information bottleneck method [C]// 2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security. 2023: 741-750.
25 SUN W, LI Y F, YAO J F, et al. Combining structure embedding and text semantics for efficient knowledge graph completion [C]// The 35th International Conference on Software Engineering and Knowledge Engineering. 2023: 317-322.
Options
文章导航

/

〈 〉