文本信息与图结构信息相融合的知识图谱补全

doi:10.3969/j.issn.1000-5641.2025.01.009

摘要/Abstract

摘要：

提出了一种基于路径查询信息的图注意力模型, 可以将知识图谱中的文本信息与图结构信息有效融合, 进而提高知识图谱的补全效果. 对于文本信息, 使用基于预训练语言模型的双编码器来分别获得实体的嵌入表示和路径查询信息的嵌入表示. 通过注意力机制来进行路径查询信息的聚合, 以捕获图结构信息, 更新实体的嵌入表示. 模型使用对比学习进行训练, 在多个知识图谱数据集上进行实验, 如直推式、归纳式的方式, 都取得了良好的效果. 结果表明, 将预训练语言模型与图神经网络的优势相结合, 可以有效捕获知识图谱中文本信息与图结构信息, 进而提高知识图谱的补全效果.

关键词: 知识图谱补全, 预训练语言模型, 对比学习, 图神经网络

Abstract:

Based upon path query information, we propose a graph attention model that effectively integrates textual and graph structure information in knowledge graphs, thereby enhancing knowledge graph completion. For textual information, a dual-encoder based on pre-trained language models is utilized to separately obtain embedding representations of entities and path query information. Additionally, an attention mechanism is employed to aggregate path query information, which is used to capture graph structural information and update entity embeddings. The model was trained using contrastive learning and experiments were conducted on multiple knowledge graph datasets, with good results achieved in both transductive and inductive settings. These results demonstrate the advantage of combining pre-trained language models with graph neural networks to effectively capture both textual and graph structural information, thereby enhancing knowledge graph completion.

Key words: knowledge graph completion, pre-trained language model, contrastive learning, graph neural networks

中图分类号:

TP391

范厚龙, 房爱莲, 林欣. 文本信息与图结构信息相融合的知识图谱补全[J]. 华东师范大学学报（自然科学版）, 2025, 2025(1): 111-123.

Houlong FAN, Ailian FANG, Xin LIN. Knowledge graph completion by integrating textual information and graph structure information[J]. J* E* C* N* U* N* S*, 2025, 2025(1): 111-123.

图/表 10

图1

图2

图3

图4

表1

表2

表3

图5

表4

表5

参考文献 25

1	JI S X, PAN S R, CAMBRIA E, et al.. A survey on knowledge graphs: Representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33 (2): 494- 514.
2	SHEN T, ZHANG F, CHENG J W. A comprehensive overview of knowledge graph completion[J]. Knowledge-Based Systems, 2022, 255: 109597.
3	LIANG X Y, SI G N, LI J X, et al. A survey of inductive knowledge graph completion[J]. Neural Computing and Applications, 2024, 36: 3837-3858.
4	OH B, SEO S, HWANG J, et al.. Open-world knowledge graph completion for unseen entities and relations via attentive feature aggregation. Information Sciences, 2022, 586, 468- 484.
5	SCHLICHTKRULL M, KIPF T N, BLOEM P, et al. Modeling relational data with graph convolutional networks [C]// The Semantic Web: 15th International Conference. 2018: 593-607.
6	ZHANG X, ZHANG C X, GUO J T, et al.. Graph attention network with dynamic representation of relations for knowledge graph completion. Expert Systems with Applications, 2023, 219, 119616.
7	XU H C, BAO J P, LIU W B. Double-branch multi-attention based graph neural network for knowledge graph completion [C]// Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. 2023: 15257-15271.
8	LOVELACE J, ROSÉ C. A framework for adapting pre-trained language models to knowledge graph completion [C]// Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022: 5937-5955.
9	WANG L, ZHAO W, WEI Z Y, et al. SimKGC: Simple contrastive knowledge graph completion with pre-trained language models [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 2022: 4281-4294.
10	SAXENA A, KOCHSIEK A, GEMULLA R. Sequence-to-sequence knowledge graph completion and question answering [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 2022: 2814-2828.
11	JIANG P C, AGARWAL S, JIN B W, et al. Text augmented open knowledge graph completion via pre-trained language models [C]// Findings of the Association for Computational Linguistics: ACL 2023. 2023: 11161-11180.
12	CHEN C, WANG Y F, SUN A X, et al. Dipping PLMs sauce: Bridging structure and text for effective knowledge graph completion via conditional soft prompting [C]// Findings of the Association for Computational Linguistics: ACL 2023. 2023: 11489-11503.
13	MARKOWITZ E, BALASUBRAMANIAN K, MIRTAHERI M, et al. StATIK: Structure and text for inductive knowledge graph completion [C]// Findings of the Association for Computational Linguistics: NAACL 2022. 2022: 604-615.
14	CHEPUROVA A, BULATOV A, KURATOV Y, et al. Better together: Enhancing generative knowledge graph completion with language models and neighborhood information [C]// Findings of the Association for Computational Linguistics: EMNLP 2023. 2023: 5306-5316.
15	DAZA D, COCHEZ M, GROTH P. Inductive entity representations from text via link prediction [C]// Proceedings of the Web Conference 2021. 2021: 798-808.
16	LYU X, LIN Y K, CAO Y X, et al. Do pre-trained models benefit knowledge graph completion? A reliable evaluation and a reasonable approach [C]// Findings of the Association for Computational Linguistics: ACL 2022. 2022: 3570-3581.
17	SUN Z, DENG Z H, NIE J Y, et al. RotatE: Knowledge graph embedding by relational rotation in complex space [EB/OL]. (2019-02-26)[2024-03-30]. https://arxiv.org/pdf/1902.10197.pdf.
18	WANG B, SHEN T, LONG G D, et al. Structure-augmented text representation learning for efficient knowledge graph completion [C]// Proceedings of the Web Conference 2021. 2021: 1737-1748.
19	YU C M, ZHANG Z G, AN L, et al.. A knowledge graph completion model integrating entity description and network structure. Aslib Journal of Information Management, 2023, 75 (3): 500- 522.
20	LI R, ZHAO J A, LI C E, et al. House: Knowledge graph embedding with Householder parameterization [C]// International Conference on Machine Learning. 2022: 13209-13224.
21	LI D, ZHU B Q, YANG S, et al.. Multi-task pre-training language model for semantic network completion. ACM Transactions on Asian and Low-Resource Language Information Processing, 2023, 22 (11): 250- 20.
22	GESESE G A, SACK H, ALAM M. RAILD: Towards leveraging relation features for inductive link prediction in knowledge graphs [C]// Proceedings of the 11th International Joint Conference on Knowledge Graphs. 2022: 82-90.
23	KIM B, HONG T, KO Y, et al. Multi-task learning for knowledge graph completion with pre-trained language models [C]// Proceedings of the 28th International Conference on Computational Linguistics. 2020: 1737-1743.
24	HAN S, GUAN Z Y, LI S H, et al. Knowledge graph completing with dual confrontation learning model based on variational information bottleneck method [C]// 2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security. 2023: 741-750.
25	SUN W, LI Y F, YAO J F, et al. Combining structure embedding and text semantics for efficient knowledge graph completion [C]// The 35th International Conference on Software Engineering and Knowledge Engineering. 2023: 317-322.

数据集	实体数量/个	关系类型数量/个	训练集三元组数/个	验证集三元组数/个	测试集三元组数/个
WN18RR	40943	11	86835	3034	3134
WN18RR-Ind	40943	11	69585	11381	12087
FB15k237	14541	237	272115	17535	20466
Wiki27K	27122	62	74793	20242	20244

模型		在WN18RR数据集上的实验结果					在FB15k237数据集上的实验结果
模型		MR	MRR/%	H@1/%	H@3/%	H@10/%	MR	MRR/%	H@1/%	H@3/%	H@10/%
基于平移距离	RotatE(2019)^[17]	3340	47.6	42.8	49.2	57.1	177	33.8	24.1	37.5	53.3
基于平移距离	HousE(2022)^[20]	1303	51.1	46.5	52.8	60.2	153	36.2	26.6	39.9	55.1
基于图神经网络	DRRGAT(2023)^[6]	2596	46.7	42.0	50.7	57.8	298	36.0	26.8	41.5	54.9
基于图神经网络	MA-GNN(2023)^[7]	886	56.5	50.7	59.2	67.9	145	37.9	28.2	41.5	56.9
基于预训练语言模型	StAR(2021)^[18]		40.1	24.3	49.1	70.9		29.6	20.5	32.2	48.2
	LP-BERT(2022)^[21]		48.2	34.3	56.3	75.2		31.0	22.3	33.6	49.0
	SimKGC(2022)^[9]	129	64.5	56.8	70.8	80.3	142	31.6	22.6	34.2	49.7
基于预训练语言模型与图神经网络	EDA-KGC(2023)^[19]		49.3	45.2	50.4	57.3		36.2	27.0	39.6	55.1
	CSProm(2023)^[12]		57.5	52.2	59.6	67.8		35.8	26.9	39.3	53.8
	PQGAT	67	68.4	59.4	74.2	84.4	136	32.0	23.1	34.8	50.1

模型	MR	MRR/%	H@1/%	H@3/%	H@10/%
PQGAT残差MLP和路径查询模块	143	65.4	56.8	70.8	80.3
PQGAT路径查询模块	154	65.8	57.4	71.3	80.6
PQGAT注意力/平均	81	65.5	56.2	71.3	81.9
PQGAT两跳路径查询	79	68.6	60.0	73.9	84.1
PQGAT	67	68.4	59.4	74.2	84.4

模型	MRR/%	H@1/%	H@3/%	H@10/%
BLP-TransE(2021)^[15]	28.5	13.5	36.1	58.0
BLP-DistMult(2021)^[15]	24.8	13.5	28.8	48.1
StAR(2021)^[18]	32.1	19.2	38.1	57.6
RAILD-TransE(2022)^[22]	29.0	13.6	36.8	59.9
StATIK(2022)^[13]	51.6	42.5	55.8	69.0
SimKGC(2022)^[9]	42.7	31.8	46.5	65.8
PQGAT	48.8	37.3	53.2	70.5

模型	MRR/%	H@1/%	H@3/%	H@10/%
LP-RP-RR(2020)^[23]	21.7	13.8	23.5	37.9
PKGC(2022)^[16]	28.5	23.0	30.5	40.9
SimKGC(2022)^[9]	27.1	22.3	28.3	36.2
VKGC(2023)^[24]	29.6	24.1	32.0	40.9
SS-KGC(2023)^[25]	26.9	21.0	29.2	40.0
PQGAT	32.6	26.7	34.2	44.2