高性能数据库管理

分布式日志结构数据库系统的主键维护方法研究

  • 黄建伟 ,
  • 张召 ,
  • 钱卫宁
展开
  • 华东师范大学 数据科学与工程学院, 上海 200062
黄建伟,男,硕士研究生,研究方向为分布式数据库.E-mail:jwhuang@stu.ecnu.edu.cn.

收稿日期: 2018-07-04

  网络出版日期: 2018-09-26

基金资助

国家自然科学基金(61432006,61332006,U1401256);国家863计划项目(2016YFB1000905,2018YFB1003400)

Primary key management in distributed log-structured database systems

  • HUANG Jian-wei ,
  • ZHANG Zhao ,
  • QIAN Wei-ning
Expand
  • School of Data Science and Engineering, East China Normal University, Shanghai 200062, China

Received date: 2018-07-04

  Online published: 2018-09-26

摘要

目前在电子商务、社交网络、移动互联网等各类应用中存在大量的写密集型负载(例如,电子商务的秒杀活动、社交用户生成的数据流等),这使得基于日志结构的存储成为现代数据库系统中普遍的后端存储方式.而基于日志结构的数据存储方式一般只支持追加操作,高效的主键维护(主键的生成和更新)可以很好地提升数据库追加操作的性能.此外,在分布式和并发的环境中实现主键维护功能还要面临主键唯一性约束、事务性维护、高处理性能的挑战.因此,本文针对日志结构数据存储的特点,研究了如何在分布式日志结构数据库系统中实现高效的主键维护功能.首先,我们提出了两类先读后写操作的并发控制模型;其次,我们应用这两类模型设计了几种高效的主键维护算法;最后,我们在自己的基于日志结构的分布式数据库系统CEDAR中实现了本文提出的主键维护方法,并通过一系列实验验证了所提方法的高效性.

本文引用格式

黄建伟 , 张召 , 钱卫宁 . 分布式日志结构数据库系统的主键维护方法研究[J]. 华东师范大学学报(自然科学版), 2018 , 2018(5) : 79 -90,119 . DOI: 10.3969/j.issn.1000-5641.2018.05.007

Abstract

At present, there are a large number of writing-intensive loads (e.g., secondkilling of e-commerce, social user-generated data streams) in many applications such as e-commerce, social networking, mobile Internet and so on, which makes log-structured storage a popular technique for back-end storage of modern database systems. However, log-structured storage only supports the append operation, efficient primary key management (primary key generation and update) functions can improve the performance of database append operations. In the distributed and concurrent environment, implementing primary key maintenance faces challenges, such as primary key unique constraints, transactional maintenance, and high-performance requirements. In light of the characteristics of log-structured storage, this paper explores how to implement efficient primary key management in distributed log-structured database systems. First, we propose two kinds of concurrency control models for WAR (Write After Read) operations; second, we adopt these two models to design efficient primary key management algorithms; and finally, we integrate these algorithms into our distributed log-structured database, CEDAR, and verify the effectiveness of the proposed methods by a series of experiments.

参考文献

[1] ROSENBLUM M, OUSTERHOUT J K. The design and implementation of a log-structured file system[J]. ACM Transaction on Computer Systems, 1992, 10(1):26-52.
[2] CHANG F, DEAN J, GHEMAWAT S, et al. Bigtable:A distributed storage system for structured data[C]//Symposium on Operating Systems Design and Implementation. USENIX Association, 2006:205-218.
[3] Hbase.[EB/OL].[2018-07-02]. http://hbase.apache.org/.
[4] Cassandra.[EB/OL].[2018-07-02]. http://cassandra.apache.org/.
[5] CDEAR.[EB/OL].[2018-07-02]. https://github.com/daseECNU/Cedar/.
[6] O'NEIL P, CHENG E, GAWLICK D, et al. The log-structured merge-tree (LSM-tree)[J]. Acta Informatica, 1996, 33(4):351-385.
[7] Mysql Cluster.[EB/OL].[2018-07-02]. https://www.mysql.com/products/cluster/.
[8] STONEBRAKER M, WEISBERG A. The voltdb main memory DBMS[J]. IEEE Data Eng Bull, 2013, 36(2):21-27.
[9] OceanBase.[EB/OL].[2018-07-02]. https://github.com/alibaba/oceanbase/.
[10] GARCIA-MOLINA H, ULLMAN J D, WIDOM J D. Database System Implementation[M]. London:Prentice Hall, 1999:576-601.
[11] The Open Group. Distributed TP:The XA Specification[EB/OL].[2018-07-02]. https://publications.opengroup.org/c193.
[12] KUNG H T. On optimistic methods for concurrency control[C]//International Conference on Very Large Data Bases. IEEE, 1981:351-351.
[13] CHOI H J, JEONG B S. A timestamp-based optimistic concurrency control for handling mobile transactions[C]//International Conference on Computational Science and Its Applications. Berlin:Springer-Verlag, 2006:796-805.
[14] ESWARAN K P, GRAY J N, LORIE R A, et al. The notions of consistency and predicate locks in a database system[J]. Readings in Artificial Intelligence & Databases, 1989, 19(11):523-532.
[15] Sysbench.[EB/OL].[2018-07-02]. https://dev.mysql.com/downloads/benchmarks.html.
文章导航

/