一种基于Tacotron 2的端到端中文语音合成方案

doi:10.3969/j.issn.1000-5641.2019.04.011

华东师范大学学报(自然科学版) ›› 2019, Vol. 2019 ›› Issue (4): 111-119.doi: 10.3969/j.issn.1000-5641.2019.04.011

一种基于Tacotron 2的端到端中文语音合成方案

王国梁¹, 陈梦楠², 陈蕾²

1. 国家电网安徽省电力有限公司信息通信分公司, 合肥 230061;
2. 华东师范大学计算机科学技术系, 上海 200062

收稿日期:2018-10-28 出版日期:2019-07-25 发布日期:2019-07-18
通讯作者: 陈蕾,女,博士,副教授,研究方向为计算机网络与智能系统.E-mail:lchen@cs.ecnu.edu.cn. E-mail:lchen@cs.ecnu.edu.cn
作者简介:王国梁,男,硕士,高级工程师,长期从事电力信息化建设和电力信息化管理工作.

An end-to-end Chinese speech synthesis scheme based on Tacotron 2

WANG Guo-liang¹, CHEN Meng-nan², CHEN Lei²

1. Information and Communication Branch, State Grid Anhui Electric Power Co., Ltd., Hefei 230061, China;
2. Department of Computer Science and Technology, East China Normal University, Shanghai 200062, China

Received:2018-10-28 Online:2019-07-25 Published:2019-07-18

摘要/Abstract

摘要： 颠覆性设计的端到端语音合成系统Tacotron 2，目前仅能处理英文.致力于对Tacotron 2进行多方位改进，设计了一种中文语音合成方案，主要包括：针对汉字不表音、变调和多音字等问题，添加预处理模块，将中文转化为注音字符；针对现有中文训练语料不足的情况，使用预训练解码器，在较少语料上获得了较好音质；针对中文语音合成急促停顿问题，采用对交叉熵损失进行加权，并用多层感知机代替变线性变换对停止符进行预测的策略，获得了有效改善；另外通过添加多头注意力机制进一步提高了中文语音合成音质.梅尔频谱、梅尔倒谱距离等的实验对比结果表明了方案的有效性：可以令Tacotron 2较好地适应中文语音合成的要求.

关键词: 语音合成, 多头注意力, Tacotron 2

Abstract: The disruptively design for an end-to-end speech synthesis system Tacotron 2, is currently only available in English. This paper is devoted to implementing several improvements to Tacotron 2 and presents a Chinese speech synthesis scheme, including:a pre-processing module to convert Chinese characters into phonetic characters to address the challenge of Chinese character not corresponding to pronunciation, having multiple tones, and having polyphonic words; a pre-training decoder to achieve better sound quality with less corpus given the lack of existing Chinese training corpus; a strategy of weighting the cross-entropy loss and using the multi-layer perceptron, instead of the linear transformation, to predict stop tokens and to solve the Chinese speech synthesis sudden pause problem; and a multi-head attention mechanism to further improve Chinese speech quality. The experimental comparison of the Mel spectrum and the Mel cepstrum distance (MCD) shows that our work is effective and can make Tacotron 2 adapted to the requirements of Chinese speech synthesis.

Key words: text to speech, multi-head attention, Tacotron 2

中图分类号:

TP391

王国梁, 陈梦楠, 陈蕾. 一种基于Tacotron 2的端到端中文语音合成方案[J]. 华东师范大学学报(自然科学版), 2019, 2019(4): 111-119.

WANG Guo-liang, CHEN Meng-nan, CHEN Lei. An end-to-end Chinese speech synthesis scheme based on Tacotron 2[J]. Journal of East China Normal University(Natural Sc, 2019, 2019(4): 111-119.

参考文献

[1] MOHAMMADI S H, KAIN A. An overview of voice conversion systems[J]. Speech Communication, 2017, 88:65-82.
[2] GONZALVO X, TAZARI S, CHAN C A, et al. Recent advances in Google real-time HMM-driven unit selection synthesizer[C]//Interspeech 2016. 2016:2238-2242.
[3] ZEN H, AGIOMYRGIANNAKIS Y, EGBERTS N, et al. Fast,compact,and high quality LSTM-RNN based statistical parametric speech synthesizers for mobile devices[C]//Interspeech 2016. 2016:2273-2277.
[4] TAYLOR P. Text-to-Speech Synthesis[M]. Cambridge:Cambridge University Press, 2009.
[5] WANG Y, SKERRY-RYAN R J, STANTON D, et al. Tacotron:Towards end-to-end speech synthesis[J]. arXiv preprint arXiv:1703.10135, 2017.
[6] SHEN J, PANG R, WEISS R J, et al. Natural tts synthesis by conditioning wavenet on mel spectrogram predictions[C]//2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018:4779-4783.
[7] VAN DEN OORD A, DIELEMAN S, ZEN H, et al. WaveNet:A generative model for raw audio[J]. arXiv preprint arXiv:1609.03499, 2016.
[8] OORD A, LI Y, BABUSCHKIN I, et al. Parallel WaveNet:Fast high-fidelity speech synthesis[J]. arXiv preprint arXiv:1711.10433, 2017.
[9] ARIK S O, Chrzanowski M, Coates A, et al. Deep voice:Real-time neural text-to-speech[J]. arXiv preprint arXiv:1702.07825, 2017.
[10] ARIK S, DIAMOS G, GIBIANSKY A, et al. Deep voice 2:Multi-speaker neural text-to-speech[J]. arXiv preprint arXiv:1705.08947, 2017.
[11] PING W, PENG K, CHEN J. ClariNet:Parallel Wave Generation in End-to-End Text-to-Speech[J]. arXiv preprint arXiv:1807.07281, 2018.
[12] PRENGER R, VALLE R, CATANZARO B. WaveGlow:A Flow-based Generative Network for Speech Synthesis[J]. arXiv preprint arXiv:1811.00002, 2018.
[13] OORD A, KALCHBRENNER N, KAVUKCUOGLU K. Pixel recurrent neural networks[J]. arXiv preprint arXiv:1601.06759, 2016.
[14] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//31st Annual Conference on Neural Information Processing Systems. NIPS, 2017:5998-6008.
[15] SUTSKEVER I, VINYALS O, Le Q V. Sequence to sequence learning with neural networks[C]//28th Annual Conference on Neural Information Processing Systems. NIPS, 2014:3104-3112.
[16] FREEMAN P, VILLEGAS E, KAMALU J. Storytime-end to end neural networks for audiobooks[R/OL].[2018-08-28]. http://web.stanford.edu/class/cs224s/reports/Pierce Freeman.pdf.
[17] GRIFFIN D, LIM J. Signal estimation from modified short-time Fourier transform[J]. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1984, 32(2):236-243.
[18] WANG D, ZHANG X W. Thchs-30:A free chinese speech corpus[J]. arXiv preprint arXiv:1512.01882, 2015.
[19] CHUNG Y A, WANG Y, HSU W N, et al. Semi-supervised training for improving data efficiency in end-to-end speech synthesis[J]. arXiv preprint arXiv:1808.10128, 2018.
[20] KUBICHEK R. Mel-cepstral distance measure for objective speech quality assessment[C]//Communications, Computers and Signal Processing, IEEE Pacific Rim Conference on. IEEE, 1993:125-128.

一种基于Tacotron 2的端到端中文语音合成方案

An end-to-end Chinese speech synthesis scheme based on Tacotron 2

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	王婕, 黄温瑞, 赵生宇, 夏小雅, 韩凡宇, 王伟, 张琰彬. OpenRank贡献度评估方法及其在开源课程中的实证研究[J]. 华东师范大学学报（自然科学版）, 2024, 2024(5): 11-19.
[2]	张嘉睿, 张豈明, 毕枫林, 张琰彬, 王伟, 任而今, 张海立. 基于IPEX-LLM的本地轻量化课程教学智能辅助系统[J]. 华东师范大学学报（自然科学版）, 2024, 2024(5): 162-172.
[3]	冯艳丽, 周宇, 黄福兴, 万俊岭, 袁培森. 基于SF-Transformer的智能教育平台短期电力负荷预测研究[J]. 华东师范大学学报（自然科学版）, 2024, 2024(5): 173-182.
[4]	任俊霖, 王欢, 黄骁迪, 李艳婷, 琚生根. 基于序列感知与多元行为数据的MOOCs知识概念推荐[J]. 华东师范大学学报（自然科学版）, 2024, 2024(5): 45-56.
[5]	陈郅睿, 陆雪松. 基于开源代码大语言模型提示的学生代码修复[J]. 华东师范大学学报（自然科学版）, 2024, 2024(5): 93-103.
[6]	谢琳玎, 张远, 蔡亦红. 基于生物信息学构建肝癌免疫预后基因模型及初步验证[J]. 华东师范大学学报（自然科学版）, 2024, 2024(4): 100-110.
[7]	饶思敬, 辛颖, 潘俊君. 角色动画中基于隐式曲面的皮肤变形方法[J]. 华东师范大学学报（自然科学版）, 2024, 2024(2): 143-156.
[8]	冯路平, 施力业, 吴雯, 郑骏, 胡文心, 郑巍. 基于偏好级别陌生人信息辅助的推荐系统[J]. 华东师范大学学报（自然科学版）, 2024, 2024(2): 53-64.
[9]	张永卓, 诸葛晴凤, 沙行勉, 宋玉红. 基于并行块的自适应量化随机计算[J]. 华东师范大学学报（自然科学版）, 2024, 2024(2): 76-85.
[10]	卢欣, 黄昶, 金志伟. 基于转台的多视角多姿态锁销点云模型重建[J]. 华东师范大学学报（自然科学版）, 2024, 2024(2): 86-96.
[11]	唐凌霄, 黄昶. 基于双层局部能量因子的红外小目标检测方法[J]. 华东师范大学学报（自然科学版）, 2024, 2024(2): 97-107.
[12]	王世瑞, 沈芳, 李仁虎, 李鹏. 无人机高光谱影像水面耀光去除及信息重构方法研究[J]. 华东师范大学学报（自然科学版）, 2024, 2024(1): 36-49.
[13]	肖开研, 廉洁. 基于多核支持向量机的句子分类算法[J]. 华东师范大学学报（自然科学版）, 2023, 2023(6): 85-94.
[14]	崔瑞博, 王峰. 动量更新与重构约束的限制视角下3D物品识别[J]. 华东师范大学学报（自然科学版）, 2023, 2023(6): 61-72.
[15]	陈道佳, 陈志云. 分层级描述感知的个性化推荐系统[J]. 华东师范大学学报（自然科学版）, 2023, 2023(6): 73-84.