Rule extraction and reasoning for fusing relation and structure encoding

doi:10.3969/j.issn.1000-5641.2025.01.008

Abstract

Abstract:

The domain knowledge graph exhibits characteristics of incompleteness and semantic complexity, which lead to shortcomings in the extraction and selection of rules, thereby affecting its inferential capabilities. A rule extraction model that integrates relationship and structural encoding is proposed to address this issue. A multidimensional embedding approach is achieved by extracting relational and structural information from the target subgraph and conducting feature encoding. A self-attention mechanism is designed to integrate relational and structural information, enabling the model to capture dependency relationships and local structural information in the input sequence better. This enhancement improves the understanding and expressive capabilities of context of the model, thus addressing the challenges of rule extraction and selection in the complex semantic situations. The experimental results for actual industrial datasets of automotive component failures and public datasets demonstrate improvements in the proposed model for link prediction and rule quality evaluation tasks. When the rule length is 3, an average increase of 7.1 percentage points in the mean reciprocal rank (MRR) and an average increase of 8.6 percentage points in Hits@10 are observed. For a rule length of 2, an average increase of 7.4 percentage points in MRR and an average increase of 3.9 percentage points in Hits@10 are observed. This confirms the effectiveness of relational and structural information in rule extraction and inference.

Key words: industrial knowledge graph, rule extraction, rule reasoning, knowledge completion, link prediction

CLC Number:

TP391

Jimi HU, Weibing WAN, Feng CHENG, Yuming ZHAO. Rule extraction and reasoning for fusing relation and structure encoding[J]. J* E* C* N* U* N* S*, 2025, 2025(1): 97-110.

Figures/Tables 13

Fig.1

Fig.2

Fig.3

Fig.4

Table 1

Fig.5

Table 2

Table 3

Table 4

Fig.6

Fig.7

Fig.8

Table 5

References 27

1	ZHANG N Y, LI L, CHEN X, et al. Multimodal analogical reasoning over knowledge graphs [EB/OL]. (2023-03-01)[2023-07-01]. https://arxiv.org/abs/2210.00312.
2	HU W H, FEY M, REN H Y, et al. OGB-LSC: A large-scale challenge for machine learning on graphs [EB/OL]. (2021-10-20)[2023-07-01]. https://doi.org/10.48550/arXiv.2103.09430.
3	HOGAN A, BLOMQVIST E, COCHEZ M, et al.. Knowledge graphs. ACM Computing Surveys, 2021, 54 (4): 71.
4	GUO Q Y, ZHUANG F Z, QIN C, et al... A survey on knowledge graph-based recommender systems. IEEE Transactions on Knowledge and Data Engineering, 2022, 34 (8): 3549- 3568.
5	李智威. 医疗知识图谱构建和智能问答研究 [D]. 长春: 吉林大学, 2022.
6	翟岩慧, 何煦, 李德玉, 等.. 融合决策蕴涵的知识图谱推理方法. 计算机科学与探索, 2023, 17 (11): 2743- 2754.
7	付瑞, 李剑宇, 王笳辉, 等.. 面向领域知识图谱的实体关系联合抽取. 华东师范大学学报(自然科学版), 2021, (5): 24- 36.
8	BORDES A, USUNIER N, GARCIA-DURAN A, et al. Translating embeddings for modeling multi-relational data [C]// Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2. Red Hook, NY, US: Curran Associates Inc., 2013: 2787-2795.
9	SUN Z Q, DENG Z H, NIE J Y, et al. RotatE: Knowledge graph embedding by relational rotation in complex space [EB/OL]. (2019-02-26)[2023-07-01]. https://doi.org/10.48550/arXiv.1902.10197.
10	CHEN S X, LIU X D, GAO J F, et al. HittER: Hierarchical transformers for knowledge graph embeddings [C]// Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (ACL), 2021: 10395–10407.
11	SCHLICHTKRULL M, KIPF T N, BLOEM P, et al. Modeling relational data with graph convolutional networks [C]// The Semantic Web: 15th International Conference, ESWC 2018. Berlin: Springer-Verlag, 2018: 593–607.
12	XIAN Y K, FU Z H, MUTHUKRISHNAN S, et al. Reinforcement knowledge graph reasoning for explainable recommendation [C]// SIGIR’19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieva. ACM, 2019: 285–294.
13	ORTONA S, MEDURI V V, PAPOTTI P. Robust discovery of positive and negative rules in knowledge bases [C]// 2018 IEEE 34th International Conference on Data Engineering (ICDE). IEEE, 2018: 1168-1179.
14	SADEGHIAN A, ARMANDPOUR M, DING P, et al. DRUM: End-to-end differentiable rule mining on knowledge graphs [C]// NIPS’19: Proceedings of the 33rd International Conference on Neural Information Processing Systems. Red Hook NY, United States: Curran Associates Inc., 2019: 15347–15357.
15	XU Z Z, YE P, CHEN H, et al. Ruleformer: Context-aware rule mining over knowledge graph [C]// Proceedings of the 29th International Conference on Computational Linguistics. International Committee on Computational Linguistics, 2022: 2551–2560.
16	LIN Q K, MAO R, LIU J, et al.. Fusing topology contexts and logical rules in language models for knowledge graph completion. Information Fusion, 2023, 90, 253- 264.
17	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// NIPS’17: Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook NY, United States: Curran Associates Inc., 2017: 6000−6010.
18	HUMPHREYS B L, FIOL G D, XU H.. The UMLS knowledge sources at 30: Indispensable to current research and applications in biomedical informatics. Journal of the American Medical Informatics Association, 2020, 27 (10): 1499- 1501.
19	ISLAM M K, ARIDHI S, SMAIL-TABBONE M.. Negative sampling and rule mining for explainable link prediction in knowledge graphs. Knowledge-Based Systems, 2022, 250, 109083.
20	FENG W Y, ZHA D R, WANG L, et al. Convolutional 3D embedding for knowledge graph completion [C]// 2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD). IEEE, 2022: 1197-1202.
21	YANG F, YANG Z L, COHEN W W. Differentiable learning of logical rules for knowledge base reasoning [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, United States: Curran Associates Inc., 2017: 2316-2325.
22	XIE Z W, ZHOU G Y, LIU J, et al. ReInceptionE: Relation-aware inception network with joint local-global structural information for knowledge graph embedding [C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics(ACL), 2020: 5929-5939.
23	KUMAR S, MALLIK A, KHETARPAL A, et al.. Influence maximization in social networks using graph embedding and graph neural network. Information Sciences, 2022, 607, 1617- 1636.
24	BERAHMAND K, NASIRI E, ROSTAMI M, et al.. A modified DeepWalk method for link prediction in attributed social network. Computing, 2021, 103, 2227- 2249.
25	LIU R M, KRISHNAN A.. PecanPy: A fast, efficient and parallelized Python implementation of node2vec. Bioinformatics, 2021, 37 (19): 3377- 3379.
26	LIU P F, YUAN W Z, FU J L, et al.. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 2023, 55 (9): 195.
27	HUANG J N, LI Z Y, CHEN B H, et al. Scallop: From probabilistic deductive databases to scalable differentiable reasoning [C]// Advances in Neural Information Processing Systems 34 (NeurIPS 2021). 2021: 25134-25145.

数据集	Triplet/个	Entity/个	Relation/个
FB15K-237^[19]	310116	14541	237
WN18RR^[20]	93003	40943	11
UMLS^[18]	5960	135	46
car-repair	148175	42419	10

环境(参数)	版本(值)
Python	3.7
PyTorch	1.11
GPU	RTX 3090
Optimizer	Adam
num_head	6
num_layers	4
lr	0.0003
d_v	64
Dropout	0.1

规则长度	模型	MRR			Hits@10/%
规则长度	模型	FB15K-237	WN18RR	UMLS	FB15K-237	WN18RR	UMLS
	TransE^[8]	0.294	0.226	0.668	46.5	50.1	93.0
	R-GCN^[11]	0.249	–	–	41.7	–	–
2	Neural LP^[21]	0.189	0.371	0.751	31.3	39.6	94.0
	Ruleformer^[15]	0.237	0.381	0.851	36.0	41.1	98.8
	DRUM^[14]	0.225	0.379	0.791	35.8	40.9	96.8
	SRRule	0.277	0.385	0.862	36.8	41.5	98.2
3	Neural LP^[21]	0.239	0.425	0.735	39.9	49.2	92.3
	Ruleformer^[15]	0.342	0.452	0.857	51.3	53.0	98.4
	DRUM^[14]	0.328	0.441	0.784	49.9	51.6	97.2
	SRRule	0.385	0.458	0.865	52.4	53.8	98.9

规则长度	模型	MRR	Hits@10/%
2	Neural LP^[21]	0.211	32.1
	Ruleformer^[15]	0.287	34.4
	DRUM^[14]	0.214	32.4
	SRRule	0.312	36.8
3	Neural LP^[21]	0.253	35.4
	Ruleformer^[15]	0.334	48.7
	DRUM^[14]	0.304	45.2
	SRRule	0.368	51.7

模型	MRR		Hits@10/%
模型	car-repair	FB15K-237	car-repair	FB15K-237
SRRule-rel	0.342	0.352	48.9	49.4
SRRule-sruc	0.314	0.334	45.7	46.2
SRRule-none	0.292	0.241	39.4	36.7
SRRule	0.368	0.385	51.7	52.4