Journal of East China Normal University(Natural Science) >
Distant supervision relation extraction via the influence function
Received date: 2021-08-13
Online published: 2022-11-22
Distant supervision relation extraction captures noisy instances while reducing the burden of manual annotation, which hinders the training and testing process. To alleviate this problem, we proposed a de-noising method based on the influence function. The influence function measures the influence of each training point; the influence of one training point is defined as the change in test loss after removing the training point. We observed that this property could be used to determine whether a training instance involves noisy data. First, we designed a scoring function based on the influence function. Then, we integrated the scoring function into a bootstrapping framework to obtain the final denoising dataset from a small clean set. Using this preprocessing method, every distantly supervised dataset could be denoised by our method. Experimental results showed that the proposed denoised dataset can achieve good performance on a public dataset.
Ziyin HUANG , Yuanbin WU . Distant supervision relation extraction via the influence function[J]. Journal of East China Normal University(Natural Science), 2022 , 2022(6) : 79 -86 . DOI: 10.3969/j.issn.1000-5641.2022.06.009
1 | MINTZ M, BILLS S, SNOW R, et al. Distant supervision for relation extraction without labeled [C]//Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. 2009: 1003-1011. |
2 | DENNIS C, SANFORD W. Residuals and Influence in Regression [M]. New York: Chapman and Hall, 1982. |
3 | ZENG D J, LIU K, LAI S W, et al. Relation classification via convolutional deep neural network [C]//Proceedings of COLING, the 25th International Conference on Computational Linguistics. 2014: 2335-2344. |
4 | ZHANG S, ZHENG D Q, HU X C, et al. Bidirectional long short-term memory networks for relation classification [C]//Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation. 2015: 73-78. |
5 | HE Z Q, CHEN W L, LI Z H, et al. See: Syntax-aware entity embedding for neural relation extraction [C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence. AAAI, 2018: 5795-5802. |
6 | SURDEANU M, TIBSHIRANI J, NALLAPATI R, et al. Multi-instance multi-label learning for relation extraction [C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. 2012: 455-465. |
7 | LIN Y K, SHEN S Q, LIU Z Y, et al. Neural relation extraction with selective attention over instances [C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016: 2124-2133. |
8 | QIN P D, XU W R, WANG W Y. Robust distant supervision relation extraction via deep reinforcement learning [C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018: 2137-2147. |
9 | QIN P D, XU W R, WANG W Y. DSGAN: Generative adversarial training for distant supervision relation extraction [C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018: 496-505. |
10 | MA R T, GUI T, LI L Y, et al. SENT: Sentence-level distant relation extraction via negative training [C]//Proceedings of the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 2021: 6201-6213. |
11 | JIA W, DAI D, XIAO X Y, et al. ARNOR: Attention regularization based noise reduction for distant supervision relation classification [C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 1399-1408. |
12 | KOH P W, LIANG P. Understanding black-box pre dictions via influence function [C]//Proceedings of the 34th International Conference on Machine Learning. 2017: 1885-1894. |
13 | REN Z Z, YEH R A, SCHWING A G. Not all unlabeled data are equal: Learning to weight data in semi-supervised learning [C]//Proceedings of the 34th Conference on Neural Information Processing Systems. 2020. https://doi.org/10.48550/arXiv.2007.01293. |
14 | XU M J, KAZANTSEV F. Understanding goal-oriented active learning via influence functions [C]//Proceedings of the NeurIPS 2019 Workshop on Machine Learning with Guarantees. 2019. https://arxiv.org/pdf/1905.13183v3.pdf. |
15 | ZENG D J, LIU K, CHEN Y B, et al. Distant supervision for relation extraction via piecewise convolutional neural networks [C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (ACL), 2015: 1753-1762. |
16 | RIEDEL S, YAO L M, MCCALLUM A. Modeling relations and their mentions without labeled text [C]//Proceedings of the 2010 European conference on Machine Learning and Knowledge Discovery in Databases. 2010: 148-163. |
17 | PENNINGTON J, SOCHER R, MANNING C. Glove: Global vectors for word representation [C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics (ACL), 2014: 1532-1543. |
/
〈 |
|
〉 |