基于影响函数的远程监督关系抽取

doi:10.3969/j.issn.1000-5641.2022.06.009

摘要/Abstract

摘要：

远程监督的标注方法在关系抽取任务中被广泛应用, 其在减小人工标注负担的同时, 也引入了大量噪声样本, 影响了模型的训练. 针对这个问题, 提出了一种基于影响函数的去噪方法. 通过影响函数衡量训练集中每一条训练样本对模型预测的影响, 并在建立噪声样本与其的关系后, 设计了1个判断样本是否是错误标注的打分函数. 基于此打分函数的值, 从初始的1个小的无噪声集合出发, 通过自举法迭代式地获得最后的去噪数据集. 该去噪方法作为一种对数据的前处理方法, 在公开数据集上取得了良好的效果.

关键词: 远程监督, 关系抽取, 影响函数, 自举法

Abstract:

Distant supervision relation extraction captures noisy instances while reducing the burden of manual annotation, which hinders the training and testing process. To alleviate this problem, we proposed a de-noising method based on the influence function. The influence function measures the influence of each training point; the influence of one training point is defined as the change in test loss after removing the training point. We observed that this property could be used to determine whether a training instance involves noisy data. First, we designed a scoring function based on the influence function. Then, we integrated the scoring function into a bootstrapping framework to obtain the final denoising dataset from a small clean set. Using this preprocessing method, every distantly supervised dataset could be denoised by our method. Experimental results showed that the proposed denoised dataset can achieve good performance on a public dataset.

Key words: distant supervision, relation extraction, influence function, bootstrapping

中图分类号:

TP391.1

黄子寅, 吴苑斌. 基于影响函数的远程监督关系抽取[J]. 华东师范大学学报（自然科学版）, 2022, 2022(6): 79-86.

Ziyin HUANG, Yuanbin WU. Distant supervision relation extraction via the influence function[J]. Journal of East China Normal University(Natural Science), 2022, 2022(6): 79-86.

图/表 5

图1

图2

表1

表2

表3

参考文献 17

1	MINTZ M, BILLS S, SNOW R, et al. Distant supervision for relation extraction without labeled [C]//Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. 2009: 1003-1011.
2	DENNIS C, SANFORD W. Residuals and Influence in Regression [M]. New York: Chapman and Hall, 1982.
3	ZENG D J, LIU K, LAI S W, et al. Relation classification via convolutional deep neural network [C]//Proceedings of COLING, the 25th International Conference on Computational Linguistics. 2014: 2335-2344.
4	ZHANG S, ZHENG D Q, HU X C, et al. Bidirectional long short-term memory networks for relation classification [C]//Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation. 2015: 73-78.
5	HE Z Q, CHEN W L, LI Z H, et al. See: Syntax-aware entity embedding for neural relation extraction [C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence. AAAI, 2018: 5795-5802.
6	SURDEANU M, TIBSHIRANI J, NALLAPATI R, et al. Multi-instance multi-label learning for relation extraction [C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. 2012: 455-465.
7	LIN Y K, SHEN S Q, LIU Z Y, et al. Neural relation extraction with selective attention over instances [C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016: 2124-2133.
8	QIN P D, XU W R, WANG W Y. Robust distant supervision relation extraction via deep reinforcement learning [C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018: 2137-2147.
9	QIN P D, XU W R, WANG W Y. DSGAN: Generative adversarial training for distant supervision relation extraction [C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018: 496-505.
10	MA R T, GUI T, LI L Y, et al. SENT: Sentence-level distant relation extraction via negative training [C]//Proceedings of the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 2021: 6201-6213.
11	JIA W, DAI D, XIAO X Y, et al. ARNOR: Attention regularization based noise reduction for distant supervision relation classification [C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 1399-1408.
12	KOH P W, LIANG P. Understanding black-box pre dictions via influence function [C]//Proceedings of the 34th International Conference on Machine Learning. 2017: 1885-1894.
13	REN Z Z, YEH R A, SCHWING A G. Not all unlabeled data are equal: Learning to weight data in semi-supervised learning [C]//Proceedings of the 34th Conference on Neural Information Processing Systems. 2020. https://doi.org/10.48550/arXiv.2007.01293.
14	XU M J, KAZANTSEV F. Understanding goal-oriented active learning via influence functions [C]//Proceedings of the NeurIPS 2019 Workshop on Machine Learning with Guarantees. 2019. https://arxiv.org/pdf/1905.13183v3.pdf.
15	ZENG D J, LIU K, CHEN Y B, et al. Distant supervision for relation extraction via piecewise convolutional neural networks [C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (ACL), 2015: 1753-1762.
16	RIEDEL S, YAO L M, MCCALLUM A. Modeling relations and their mentions without labeled text [C]//Proceedings of the 2010 European conference on Machine Learning and Knowledge Discovery in Databases. 2010: 148-163.
17	PENNINGTON J, SOCHER R, MANNING C. Glove: Global vectors for word representation [C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics (ACL), 2014: 1532-1543.

NYT	句子数量	样本数量	正例样本数量
训练集	233038	367596	106653
开发集	1596	4567	975
测试集	1596	4484	1050

方法(模型)	验证集			测试集
方法(模型)	准确率/%	召回率/%	F₁/%	准确率/%	召回率/%	F₁/%
CNN	39.27	73.80	51.26	42.41	76.64	54.60
CNN + ATT	68.09	47.49	55.95	67.31	49..83	52.27
CNN + RL	42.50	71.62	53.34	43.70	72.34	54.49
CNN + IF	57.97	71.74	64.12	60.96	68.76	64.61
CNN + IF_10	63.15	68.71	58.42	61.75	64.28	59.33
CNN + IF_50	60.08	61.46	60.76	60.50	63.22	61.83
CNN + IF_100	58.32	69.15	63.27	62.83	61.56	62.19
CNN + IF_OT	42.48	72.28	53.51	45.63	72.30	55.95

	噪声样本	真(假)阳性	影响分数
正例	··· Leslie Moonves, the chief executive of $\underline{{\rm{CBS}}}$, ···	真阳性	4.29e–3
正例	··· Yusuf Hamied, the chairman of $\underline{{\rm{Cipla}}}$, ···	真阳性	3.41e–3
临界样本	···after $\underline{{\rm{Sony}}}$’s news conference and a speech by sir Howard Stringer, its chairman···	真阳性	6.72e–7
临界样本	··· Richard C. Noteaert, the company’s chief executive, said Qwest spent···	假阳性	–2.13e–6
负例	··· Joshua Landis, a history professor at the University of Oklahoma···	真阳性	–5.24e–3
负例	As Bruce Wasserstein left St. Regis Hotel in Manhattan on Tuesday afternoon after presenting $\underline{{\rm{Lazard}}}$’s plan···	假阳性	–1.86e–3

[1]	马晓琴, 郭小鹤, 薛峪峰, 杨琳, 陈远哲. 针对命名实体识别的数据增强技术[J]. 华东师范大学学报（自然科学版）, 2021, 2021(5): 14-23.
[2]	梁艳春, 房爱莲. 基于多通道卷积神经网络的中文文本关系抽取[J]. 华东师范大学学报（自然科学版）, 2021, 2021(3): 96-104.
[3]	王嘉宁, 何怡, 朱仁煜, 刘婷婷, 高明. 基于远程监督的关系抽取技术[J]. 华东师范大学学报（自然科学版）, 2020, 2020(5): 113-130.
[4]	杨东明, 杨大为, 顾航, 洪道诚, 高明, 王晔. 面向初等数学的知识点关系提取研究[J]. 华东师范大学学报(自然科学版), 2019, 2019(5): 53-65.
[5]	孙晨, 付英男, 程文亮, 钱卫宁. 面向企业知识图谱构建的中文实体关系抽取[J]. 华东师范大学学报(自然科学版), 2018, 2018(3): 55-66.