面向自动问答的机器阅读理解综述

doi:10.3969/j.issn.1000-5641.2019.05.003

摘要/Abstract

摘要： 人工智能正在深彻地变革各个行业.AI与教育的结合加速推动教育的结构性变革，正在将传统教育转变为智适应教育.基于深度学习的自动问答系统不仅可帮助学生实时解答疑惑、获取知识，还可以快速获取学生行为数据，加速教育的个性化和智能化.机器阅读理解是自动问答系统的核心模块，是理解学生问题，理解文档内容，快速获取知识的重要技术.在过去的几年里，随着深度学习复兴以及大规模机器阅读数据集的公开，各种各样的基于神经网络的机器阅读模型不断涌现.这篇综述主要讲述3方面的内容:介绍机器阅读理解的定义与发展历程；分析神经机器阅读模型之间的优点及不足；总结机器阅读领域的公开数据集以及评价方法.

关键词: 人工智能, 智适应教育, 深度学习, 机器阅读理解

Abstract: Artificial Intelligence (AI) is affecting every industry. Applying AI to education accelerates the structural reform of education and transforms traditional education into intelligent adaptive education. The automatic Question Answer system, based on deep learning, not only helps students to answer questions and acquire knowledge in real-time, but can also quickly gather student behavioral data and accelerate personalization of the educational process. Machine reading comprehension is the core module of an automatic Question Answer system, and it is an important technology to understand student problems, document content, and acquire knowledge quickly. With the revival of deep learning and the availability of large-scale reading comprehension datasets, a number of neural network-based machine reading models have been proposed over the past few years. The purpose of this review is three-fold:to introduce and review progress in machine reading comprehension; to compare and analyze the advantages and disadvantages between various neural machine reading models; and to summarize the relevant datasets and evaluation methods in the field of machine reading.

Key words: Artificial Intelligence, intellectual adaptation education, deep learning, machine reading comprehension

中图分类号:

TP391

杨康, 黄定江, 高明. 面向自动问答的机器阅读理解综述[J]. 华东师范大学学报(自然科学版), 2019, 2019(5): 36-52.

YANG Kang, HANG Ding-jiang, GAO Ming. A review of machine reading comprehension for automatic QA[J]. Journal of East China Normal University(Natural Sc, 2019, 2019(5): 36-52.

参考文献

[1] CHEN D Q. Neural reading comprehension and beyond[D]. CA:Standford University, 2018.
[2] LEHNERT W G. The process of question answering[R]. Yale Univ New Haven Conn, 1977.
[3] HIRSCHMAN L, LIGHT M, BRECK E, et al. Deep read:A reading comprehension system[C]//Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics. Association for Computational Linguistics, 1999:325-332.
[4] RILOFF E, THELEN M. A rule-based question answering system for reading comprehension tests[C]//Proceedings of the 2000 ANLP/NAACL Workshop on Reading comprehension tests as evaluation for computer-based language understanding sytems-Volume 6. Association for Computational Linguistics, 2000:13-19.
[5] RICHARDSON M, BURGES C J C, RENSHAW E. Mctest:A challenge dataset for the open-domain machine comprehension of text[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, 2013:193-203.
[6] SACHAN M, DUBEY K, XING E, et al. Learning answer-entailing structures for machine comprehension[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1:Long Papers), 2015:239-249.
[7] NARASIMHAN K, BARZILAY R. Machine comprehension with discourse relations[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1:Long Papers), 2015:1253-1262.
[8] WANG H, BANSAL M, GIMPEL K, et al. Machine comprehension with syntax, frames, and semantics[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2:Short Papers), 2015:700-706.
[9] BENGIO Y, DUCHARME R, VINCENT P, et al. A neural probabilistic language model[J]. Journal of Machine Learning Research, 2003(3):1137-1155.
[10] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]//Advances in neural information processing systems, 2013:3111-3119.
[11] PENNINGTON J, SOCHER R, MANNING C. Glove:Global vectors for word representation[C]//Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014:1532-1543.
[12] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural computation, 1997, 9(8):1735-1780.
[13] CHUNG J, GULCEHRE C, CHO K H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[J]. arXiv preprint, arXiv:1412.3555, 2014.
[14] RAJPURKAR P, ZHANG J, LOPYREV K, et al. Squad:100,000+questions for machine comprehension of text[J]. arXiv preprint, arXiv:1606.05250, 2016.
[15] NGUYEN T, ROSENBERG M, SONG X, et al. MS MARCO:A Human-Generated MAchine Reading COmprehension Dataset[J]. Neural Information Processing Systems, 2016.
[16] JOSHI M, CHOI E, WELD D S, et al. Triviaqa:A large scale distantly supervised challenge dataset for reading comprehension[J]. arXiv preprint, arXiv:1705.03551, 2017.
[17] TRISCHLER A, WANG T, YUAN X, et al. Newsqa:A machine comprehension dataset[J]. arXiv preprint, arXiv:1611.09830, 2016.
[18] KOČSKÝ T, SCHWARZ J, BLUNSOM P, et al. The narrativeqa reading comprehension challenge[J]. Transactions of the Association for Computational Linguistics, 2018(6):317-328.
[19] LAI G, XIE Q, LIU H, et al. Race:Large-scale reading comprehension dataset from examinations[J]. arXiv preprint, arXiv:1704.04683, 2017.
[20] DUNN M, SAGUN L, HIGGINS M, et al. Searchqa:A new q&a dataset augmented with context from a search engine[J]. arXiv preprint, arXiv:1704.05179, 2017.
[21] HE W, LIU K, LIU J, et al. Dureader:a chinese machine reading comprehension dataset from real-world applications[J]. arXiv preprint, arXiv:1711.05073, 2017.
[22] TAYLOR W L. "Cloze procedure":A new tool for measuring readability[J]. Journalism Bulletin, 1953, 30(4):415-433.
[23] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[J]. arXiv preprint, arXiv:1409.0473, 2014.
[24] HERMANN K M, KOCISKÝ HERMANN K M, KOCISKY T, et al. Teaching machines to read and comprehend[C]//Advances in neural information processing systems. 2015:1693-1701.
[25] CHEN D, BOLTON J, MANNING C D. A thorough examination of the cnn/daily mail reading comprehension task[J]. arXiv preprint, arXiv:1606.02858, 2016.
[26] KADLEC R, SCHMID M, BAJGAR O, et al. Text understanding with the attention sum reader network[J]. arXiv preprint, arXiv:1603.01547, 2016.
[27] VINYALS O, FORTUNATO M, JAITLY N. Pointer networks[C]//Advances in Neural Information Processing Systems, 2015:2692-2700.
[28] CUI Y, LIU T, CHEN Z, et al. Consensus attention-based neural networks for chinese reading comprehension[J]. arXiv preprint, arXiv:1607.02250, 2016.
[29] CUI Y, CHEN Z, WEI S, et al. Attention-over-attention neural networks for reading comprehension[J]. arXiv preprint, arXiv:1607.04423, 2016.
[30] WANG S, JIANG J. Machine comprehension using match-lstm and answer pointer[J]. arXiv preprint, arXiv:1608.07905, 2016.
[31] SEO M, KEMBHAVI A, FARHADI A, et al. Bidirectional attention flow for machine comprehension[J]. arXiv preprint, arXiv:1611.01603, 2016.
[32] KIM Y. Convolutional neural networks for sentence classification[J]. arXiv preprint, arXiv:1408.5882, 2014.
[33] WANG W, YANG N, WEI F, et al. Gated self-matching networks for reading comprehension and question answering[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers), 2017:189-198.
[34] HUANG H Y, ZHU C, SHEN Y, et al. Fusionnet:Fusing via fully-aware attention with application to machine comprehension[J]. arXiv preprint, arXiv:1711.07341, 2017.
[35] LIU X, SHEN Y, DUH K, et al. Stochastic answer networks for machine reading comprehension[J]. arXiv preprint, arXiv:1712.03556, 2017.
[36] WESTON J, CHOPRA S, BORDES A. Memory networks[J]. arXiv preprint, arXiv:1410.3916, 2014.
[37] SUKHBAATAR S, WESTON J, FERGUS R. End-to-end memory networks[C]//Advances in neural information processing systems, 2015:2440-2448.
[38] DHINGRA B, LIU H, YANG Z, et al. Gated-attention readers for text comprehension[J]. arXiv preprint, arXiv:1606.01549, 2016.
[39] SORDONI A, BACHMAN P, TRISCHLER A, et al. Iterative alternating neural attention for machine reading[J]. arXiv preprint, arXiv:1606.02245, 2016.
[40] SHEN Y, HUANG P S, GAO J, et al. Reasonet:Learning to stop reading in machine comprehension[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2017:1047-1055.
[41] WILLIAMS R J. Simple statistical gradient-following algorithms for connectionist reinforcement learning[J]. Machine learning, 1992, 8(3/4):229-256.
[42] WU F, LAO N, BLITZER J, et al. Fast reading comprehension with convnets[J]. arXiv preprint, arXiv:1711.04352, 2017.
[43] YU A W, DOHAN D, LUONG M T, et al. Qanet:Combining local convolution with global self-attention for reading comprehension[J]. arXiv preprint, arXiv:1804.09541, 2018.
[44] DAUPHIN Y N, FAN A, AULI M, et al. Language modeling with gated convolutional networks[C]//Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2017:933-941.
[45] GEHRING J, AULI M, GRANGIER D, et al. Convolutional sequence to sequence learning[C]//Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2017:1243-1252.
[46] CHOLLET F. Xception:Deep learning with depthwise separable convolutions[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, 2017:1251-1258.
[47] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in neural information processing systems, 2017:5998-6008.
[48] RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving language understanding with unsupervised learning[R/OL]. Technical report, OpenAI, 2018.[2019.08.01]. https://s3-us-west-2.amazonaws.com/openaiassets/research-covers/language-unsupervised/languageunderstandingpaper.pdf.
[49] DEVLIN J, CHANG M W, LEE K, et al. Bert:Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint, arXiv:1810.04805, 2018.
[50] HILL F, BORDES A, CHOPRA S, et al. The goldilocks principle:Reading children's books with explicit memory representations[J]. arXiv preprint, arXiv:1511.02301, 2015.