华东师范大学学报(自然科学版) ›› 2021, Vol. 2021 ›› Issue (6): 124-134.doi: 10.3969/j.issn.1000-5641.2021.06.013

• 计算机科学 • 上一篇    

面向非易失性内存缓冲区的SQLite-CC

胡耀艺, 胡卉芪*(), 周烜, 周傲英   

  1. 华东师范大学 数据科学与工程学院, 上海 200062
  • 收稿日期:2020-08-31 出版日期:2021-11-25 发布日期:2021-11-26
  • 通讯作者: 胡卉芪 E-mail:hqhu@dase.ecnu.edu.cn
  • 基金资助:
    国家自然科学基金(61772202, 61702189)

SQLite-CC based on non-volatile memory cache

Yaoyi HU, Huiqi HU*(), Xuan ZHOU, Aoying ZHOU   

  1. School of Data Science and Engineering, East China Normal University, Shanghai 200062, China
  • Received:2020-08-31 Online:2021-11-25 Published:2021-11-26
  • Contact: Huiqi HU E-mail:hqhu@dase.ecnu.edu.cn

摘要:

近年来非易失性存储(Non Volatile Memory, NVM)飞速发展, 它具有的持久化、大容量、低延迟、按字节寻址、高密度和低能耗等优越特性, 强烈冲击着目前的数据库系统架构. SQLite是一款轻量级关系型数据库, 实现了无服务器、零配置、事务性的SQL数据库引擎. 其为每个连接维护一个缓冲区, 有着空间开销大和数据一致性检测的问题, 同时由于采用了相对简单的串行化单写事务执行方式和按页记录日志等方案, 带来了回滚日志模式性能低和写入放大以及WAL模式存储空间要求等问题. 为了解决上述挑战, 构建了一种新的基于非易失性内存的SQLite缓冲区的方案SQLite-CC (Copy Cache), 充分考虑了非易失性内存的硬件特性, 引入CC-manager用于维护事务原子性, 加入修改页面索引来保证数据库文件与缓存的一致性. 实验表明, 其达到了和SQLite-WAL模式相当的并发性能, 相比于回滚日志模式事务吞吐量提升了3倍, 读写延迟降低了40%且有效解决了磁盘中的写放大问题.

关键词: NVM, SQLite, 缓冲区优化, 日志方案

Abstract:

In recent years, non-volatile memory (NVM) has developed rapidly. Its advantages, among others, include: persistence, large capacity, low latency, byte addressing, high density, and low energy consumption — all of which have impacted current database system architecture. SQLite is a lightweight relational database widely used in embedded fields such as mobile platforms. It operates as a serverless, zero-configuration, transactional SQL database engine. It maintains a cache for each connection, which results in problems with large space overhead and data consistency detection. At the same time, it adopts a relatively simple serialized single-write transaction execution method and page-based logging, which offers low performance and write amplification in the journal mode and a storage space requirement in the WAL mode during execution. In order to address the above challenges, a new scheme of SQLite Cache based on non-volatile memory, SQLite-CC (Copy Cache), is constructed, which fully considers the hardware characteristics of non-volatile memory and ensures the atomicity of transactions using a CC-manager and by adding an updated page index to ensure the consistency of database files and cache. Benchmarking tests show that it can achieve the same concurrency performance as SQLite-WAL mode. Compared with the rollback mode, it improves the execution performance of transactions by 3 times, reduces latency by 40%, and effectively solves the issue of write amplification on disks.

Key words: NVM, SQLite, cache optimization, logging solution

中图分类号: