基于多通道卷积神经网络的中文文本关系抽取

doi:10.3969/j.issn.1000-5641.2021.03.010

摘要/Abstract

摘要：

给出了一种多通道卷积神经网络(Convolutional Neural Network, CNN)方法实现中文文本端到端的关系抽取. 每个通道用分层的网络结构, 在传播过程中互不影响, 使神经网络能学习到不同的表示. 结合中文语言的难点, 加入注意力机制(Attention Mechanism, Att)获取更多的语义特征, 并通过分段平均池化融入句子的结构信息. 经过最大池化层获得句子的最终表示后, 计算关系得分, 并用排序损失函数(Ranking-Loss Function, RL)代替交叉熵函数进行训练. 实验结果表明, 提出的MCNN_Att_RL (Multi CNN_Att_RL)模型能有效提高关系抽取的查准率、召回率和F₁值.

关键词: 关系抽取, 多通道CNN, 注意力机制, 中文文本

Abstract:

This paper presents an end-to-end method for Chinese text relation extraction based on a multi-channel CNN (convolutional neural network). Each channel is stacked with a layered neural network; these channels do not interact during recurrent propagation, which enables a neural network to learn different representations. Considering the nuances of the Chinese language, we employed the attention mechanism to extract the semantic features of a sentence, and then integrate structural information using piecewise average pooling. After the maximum pooling layer, the final representation of the sentence is obtained and a relational score is calculated. Finally, the ranking-loss function is used to replace the cross-entropy function for training. The experimental results show that the MCNN_Att_RL (Multi CNN_Att_RL) model proposed in this paper can effectively improve the precision, recall, and F₁ value of entity relation extraction.

Key words: relation extraction, multi-channel CNN, attention mechanism, Chinese text

中图分类号:

TP391

梁艳春, 房爱莲. 基于多通道卷积神经网络的中文文本关系抽取[J]. 华东师范大学学报（自然科学版）, 2021, 2021(3): 96-104.

Yanchun LIANG, Ailian FANG. Chinese text relation extraction based on a multi-channel convolutional neural network[J]. Journal of East China Normal University(Natural Science), 2021, 2021(3): 96-104.

图/表 5

图1

表1

图2

表2

表3

参考文献 19

1	LIU C Y, SUN W B, CHAO W H, et al. Convolution neural network for relation extraction [C]// International Conference on Advanced Data Mining and Applications. Berlin: Springer, 2013: 231-242.
2	ZENG D J, LIU K, LAI S W, et al. Relation classification via convolutional deep neural network [C]// International Conference on Computational Linguistocs. 2014: 2335-2344.
3	ZHANG D, WANG D. Relation classification via recurrent neural network. Computer Ence, 2015, 36 (36): 257- 266.
4	ZHOU P, SHI W, TIAN J, et al. Attention-based bidirectional long short-term memory networks for relation classification [C]// Meeting of the Association for Computational Linguistics. 2016: 207-212.
5	ZHU J Z, QIAO J Z, DAI X X, et al. Relation classification via target-concentrated attention CNNs [C]// International Conference On Neural Information Processing. 2017: 137-146.
6	LI J, HUANG G M, CHEN J H, et al. Dual CNN for relation extraction with knowledge-based attention and word embeddings. Computational Intelligence and Neuroscience, 2019, 171, 1- 10. doi: 10.1155/2019/6789520
7	HONG Y, LIU Y X, YANG S Z, et al. Improving graph convolutional networks based on relation-aware attention for end-to-end relation extraction. IEEE Access, 2020, (8): 51315- 51323. doi: 10.1109/ACCESS.2020.2980859
8	ZENG D J, LIU K, CHEN Y B, et al. Distant supervision for relation extraction via piecewise convolutional neural networks [C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015: 1753-1762.
9	LIN Y K, SHEN S Q, LIU Z Y, et al. Neural relation extraction with selective attention over instances [C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2016: 2124-2133.
10	KIM E K, CHOI K S. Improving distantly supervised relation extraction by knowledge base-driven zero subject resolution. ICE Transactions on Information and Systems, 2018, 26 (3): 142- 151. doi: 10.1587/transinf.2017EDL8270
11	SUN T T, ZHANG C H, JI Y, et al. MSnet: Multi-head self-attention network for distantly supervised relation extraction. IEEE Access, 2019, (7): 54472- 54482. doi: 10.1109/ACCESS.2019.2913316
12	CHEN T T, WANG N B, HE M, et al. Reducing wrong labels for distantly supervised relation extraction with reinforcement learning. IEEE Access, 2020, 99, 1- 12. doi: 10.1109/ACCESS.2020.2990680
13	WU W Y, CHEN Y F, XU J N, et al. Attention-based convolutional neural for Chinese relation extraction [C]// Lecture Notes in Computer Science, vol 11221. Cham: Springer, 2018: 147-158.
14	WEN J, SUN X, REN X C, et al. Structure regularized neural network for entity relation classification for Chinese literature text. Computing Research Repository, 2018, 171, 103- 112. doi: 10.18653/v1/n18-2059
15	LI Z R, DING N, LIU Z Y, et al. Chinese relation extraction with multi-grained information and external linguistic knowledge [C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 4377-4386.
16	张志昌, 周侗, 张瑞芳, 等. 融合双向GRU与注意力机制的医疗实体关系识别. 计算机工程, 2020, 46 (6): 296- 302.
17	MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems, 2013, 26, 3111- 3119.
18	XU J J, WEN J, SUN X, et al. A discourse-level named entity recognition and relation extraction dataset for Chinese literature text. IEEE Access, 2017, (3): 25- 32.
19	OLSON D L, DELEN D. Advanced Data Mining Techniques [M]. Berlin: Springer, 2008.

序号	关系类别	举例	占比/%
1	Greate(创造)	男人－陶器	2.93
2	Use(使用)	奶奶－蒲扇	4.76
3	Near(邻近)	山－县城	2.76
4	Social(社交关系)	母亲－邻居	6.02
5	Located(位于)	幽兰－山谷	37.43
6	Ownership(拥有)	村民－旧屋	5.10
7	General-Special(一般－特殊)	鱼－鲫鱼	6.99
8	Family(家人)	母亲－奶奶	37.43
9	Part-Whole(部分－整体)	花－仙人掌	23.76

参数	数值	参数	数值
词向量维度	100	z	2
位置向量维度	10	a	2.5
卷积核大小	9	b	0.5
学习速率	0.001	batch	10
卷积核个数	500	epoch	100
隐藏层个数	100	Dropout	0.5

模型	P/%	R/%	F₁/%
BiLSTM	57.18	56.24	56.70
Att-BiLSTM	56.26	59.20	57.70
CNN	49.91	64.74	56.37
MCNN	59.42	59.63	59.52
MCNN_Att	60.72	64.27	62.45
MCNN_Att_P	61.57	65.36	63.41
MCNN_Att_RL	62.87	66.03	64.41

[1]	王畅, 马丹, 许华容, 陈攀峰, 陈梅, 李晖. SA-MGKT: 基于自注意力融合的多图知识追踪方法[J]. 华东师范大学学报（自然科学版）, 2024, 2024(5): 20-31.
[2]	冯路平, 施力业, 吴雯, 郑骏, 胡文心, 郑巍. 基于偏好级别陌生人信息辅助的推荐系统[J]. 华东师范大学学报（自然科学版）, 2024, 2024(2): 53-64.
[3]	刘家伟, 林欣. 基于解耦常识性关联的图像描述生成算法[J]. 华东师范大学学报（自然科学版）, 2024, 2024(2): 131-142.
[4]	任菊香, 刘忠宝. 融合多粒度语义特征的中文情感分析方法[J]. 华东师范大学学报（自然科学版）, 2023, 2023(6): 95-107.
[5]	陈道佳, 陈志云. 分层级描述感知的个性化推荐系统[J]. 华东师范大学学报（自然科学版）, 2023, 2023(6): 73-84.
[6]	姜璐璐, 孙司琦, 邹海东, 陆丽娜, 冯瑞. 基于双视图特征融合的糖尿病视网膜病变分级[J]. 华东师范大学学报（自然科学版）, 2023, 2023(6): 39-48.
[7]	段志尚, 冉懿, 吕笃良, 祁杰, 钟佳晨, 袁培森. 基于残差网络和深度可分离卷积增强自注意力机制的窃电识别[J]. 华东师范大学学报（自然科学版）, 2023, 2023(5): 193-204.
[8]	葛进, 陆雪松. 基于UI图像的Web前端代码自动生成[J]. 华东师范大学学报（自然科学版）, 2023, 2023(5): 100-109.
[9]	武朝阳, 毛嘉莉. 基于神经网络的行驶时长预测[J]. 华东师范大学学报（自然科学版）, 2023, 2023(2): 106-118.
[10]	黄子寅, 吴苑斌. 基于影响函数的远程监督关系抽取[J]. 华东师范大学学报（自然科学版）, 2022, 2022(6): 79-86.
[11]	童根梅, 朱敏. 基于双向长短记忆网络和门控注意力的文本分类网络[J]. 华东师范大学学报（自然科学版）, 2022, 2022(2): 67-75.
[12]	李小昌, 陈贝, 董启文, 陆雪松. 基于自编码器的旅行同伴挖掘[J]. 华东师范大学学报（自然科学版）, 2020, 2020(5): 179-188.
[13]	王嘉宁, 何怡, 朱仁煜, 刘婷婷, 高明. 基于远程监督的关系抽取技术[J]. 华东师范大学学报（自然科学版）, 2020, 2020(5): 113-130.
[14]	傅裕, 李优, 林煜明, 周娅. 基于自注意力机制的冗长商品名称精简方法[J]. 华东师范大学学报(自然科学版), 2019, 2019(5): 113-122,167.
[15]	杨东明, 杨大为, 顾航, 洪道诚, 高明, 王晔. 面向初等数学的知识点关系提取研究[J]. 华东师范大学学报(自然科学版), 2019, 2019(5): 53-65.