Self-attention based neural networks for product titles compression

doi:10.3969/j.issn.1000-5641.2019.05.009

Abstract

Abstract: E-commerce product title compression has received significant attention in recent years, since it can facilitate more specific information for cross-platform knowledge alignment and multi-source data fusion. Product titles usually contain redundant descriptions, which can lead to inconsistencies. In this paper, we propose self-attention based neural networks for this task. Given the fact that self-attention mechanism networks cannot directly capture sequence features of product names, we enhance the mapping networks with a dot-attention structure, which was computed for the query and key-value pairs by a gated recurrent unit (GRU) based recurrent neural network. The proposed method improves the analytical capability of the model at a lower relative computational cost. Based on data from LESD4EC, we built two E-commerce datasets of product core phrases named LESD4EC L and LESD4EC S; we subsequently tested the model on these two datasets. A series of experiments show that the proposed model achieves better performance in product title compression than existing techniques.

Key words: self-attention mechanism, product titles compression, gated recurrent units

CLC Number:

TP391

FU Yu, LI You, LIN Yu-ming, ZHOU Ya. Self-attention based neural networks for product titles compression[J]. Journal of East China Normal University(Natural Sc, 2019, 2019(5): 113-122,167.

References

[1] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in neural information processing systems. 2017:5998-6008.
[2] GONG Y, LUO X S, ZHU K Q, et al. Automatic generation of chinese short product titles for mobile display[J]. arXiv preprint arXiv:1803.11359, 2018.
[3] LIU Z Y, HUANG W Y, ZHENG Y B, et al. Automatic keyphrase extraction via topic decomposition[C]//Proceedings of the 2010 conference on empirical methods in natural language processing. Association for Computational Linguistics, 2010:366-376.
[4] ROSE S, ENGEL D, CRAMER N, et al. Automatic keyword extraction from individual documents[M]//Text mining:Applications and theory. Hoboken:A John Wiley and Sons, Ltd., 2010:1-20.
[5] MIHALCEA R, TARAU P. Textrank:Bringing order into text[C]//Proceedings of the 2004 conference on empirical methods in natural language processing. 2004.
[6] ZHAO W X, JIANG J, HE J, et al. Topical keyphrase extraction from Twitter[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2011:379-388.
[7] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780.
[8] CHO K, VAN MERRIENBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014:1724-1734.
[9] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[J]. arXiv preprint arXiv:1409.0473, 2014.
[10] LUONG T, PHAM H, MANNING C D. Effective approaches to attention-based neural machine translation[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015:1412-1421.
[11] SUTSKEVER I, VINYALS O, LE Q V. Sequence to sequence learning with neural networks[C]//Advances in Neural Information Processing Systems. 2014:3104-3112.
[12] NALLAPATI R, ZHOU B W, DOS SANTOS C, et al. Abstractive text summarization using sequence-to-sequence RNNs and beyond[C]//Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning. 2016:280-290.
[13] NALLAPATI R, ZHAI F F, ZHOU B W. Summarunner:A recurrent neural network based sequence model for extractive summarization of documents[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI-17). 2017:3075-3081.
[14] SEE A, LIU P J, MANNING C D. Get to the point:Summarization with pointer-generator networks[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers). 2017:1073-1083.
[15] VINYALS O, FORTUNATO M, JAITLY N. Pointer networks[C]//Advances in Neural Information Processing Systems 28(NIPS 2015). 2015:2692-2700.
[16] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Advances in Neural Information Processing Systems 27(NIPS 2014). 2014:2672-2680.
[17] ZHANG J, ZOU P, LI Z, et al. Multi-modal generative adversarial network for short product title generation in mobile e-commerce[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, Volume 2(Industry Papers). 2019:64-72.
[18] WANG J G, TIAN J F, QIU L, et al. A multi-task learning approach for improving product title compression with user search log data[C]//32nd AAAI Conference on Artificial Intelligence. 2018:451-458.
[19] KINGMA D P, BA J. Adam:A method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980, 2014.
[20] LIN C Y, HOVY E. Automatic evaluation of summaries using n-gram co-occurrence statistics[C]//Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics. 2003:150-157.