华东师范大学学报(自然科学版) ›› 2021, Vol. 2021 ›› Issue (5): 37-47.doi: 10.3969/j.issn.1000-5641.2021.05.004

• 系统关键技术 • 上一篇    下一篇

基于非易失性内存的LSM-tree存储系统优化

余阳, 胡卉芪*(), 周煊   

  1. 华东师范大学 数据科学与工程学院, 上海 200062
  • 收稿日期:2021-07-27 出版日期:2021-09-25 发布日期:2021-09-28
  • 通讯作者: 胡卉芪 E-mail:hqhu@dase.ecnu.edu.cn

Optimization of LSM-tree storage systems based on non-volatile memory

Yang YU, Huiqi HU*(), Xuan ZHOU   

  1. School of Data Science and Engineering, East China Normal University, Shanghai 200062, China
  • Received:2021-07-27 Online:2021-09-25 Published:2021-09-28
  • Contact: Huiqi HU E-mail:hqhu@dase.ecnu.edu.cn

摘要:

随着大数据时代的到来, 金融行业产生的数据越来越多, 对数据库的压力也越来越大. LevelDB是谷歌开发的一款基于LSM-tree架构的键值对数据库, 有写入快和占用空间小的优点, 被金融行业广泛应用. 针对LSM-tree架构的写停顿、写放大、对读不友好等缺点, 提出了一种基于非易失性内存和机器学习的L0层的设计方法, 能够减缓甚至解决上述问题. 实验结果表明, 该设计能够实现较好的读写性能.

关键词: 非易失性内存, 机器学习, LSM-tree架构

Abstract:

With the advent of the big data era, the financial industry has been generating increasing volumn of data, exerting pressure on database systems. LevelDB is a key-value database, developed by Google, based on the LSM-tree architecture. It offers fast writing and a small footprint, and is widely used in the financial industry. In this paper, we propose a design method for the L0layer, based on non-volatile memory and machine learning, with the aim of addressing the shortcomings of the LSM-tree architecture, including write pause, write amplification, and unfriendly reading. The proposed solution can slow down or even solve the aforementioned problems; the experimental results demonstrate that the design can achieve better read and write performance.

Key words: NVM, machine learning, LSM-tree architecture

中图分类号: