新型OLTP系统的技术与实践

doi:10.3969/j.issn.1000-5641.2018.05.009

华东师范大学学报(自然科学版) ›› 2018, Vol. 2018 ›› Issue (5): 107-119.doi: 10.3969/j.issn.1000-5641.2018.05.009

新型OLTP系统的技术与实践

贺小龙¹, 马海欣², 何毓锟², 庞天泽², 赵琼²

1. 华东师范大学数据科学与工程学院, 上海 200062;
2. 交通银行数据中心, 上海 200062

收稿日期:2018-07-10 出版日期:2018-09-25 发布日期:2018-09-26
作者简介:贺小龙,男,博士研究生,研究方向为内存数据库事务处理.E-mail:hxlongecnu@163.com.
基金资助:
国家863计划（2015AA015307）

Technology and implementation of a new OLTP system

HE Xiao-long¹, MA Hai-xin², HE Yu-kun², PANG Tian-ze², ZHAO Qiong²

1. School of Data Science and Engineering, East China Normal University, Shanghai 200062, China;
2. Data center, Bank of Communications, Shanghai 200062, China

Received:2018-07-10 Online:2018-09-25 Published:2018-09-26

摘要/Abstract

摘要： 自20世纪70年代以来，硬件已经得到了巨大的发展，高性能服务器大多数配备TB级的容量、数十个物理核.然而，传统的事务型系统仍旧是基于磁盘存储，运行在物理核数较少的硬件环境上，无法有效地、充分地利用新硬件的运算能力.另一方面，随着互联网的发展，应用对事务型系统的性能有了更高的要求.部分应用在极端情况下需要服务百万甚至千万的并发访问，然而传统的数据库系统并不能支撑这些高并发、高吞吐率的应用.因此，在高性能硬件上重新设计与实现事务型数据库系统已成为重要的研究热点.本文将重点介绍在大内存、多核环境下，事务型数据库系统在各个方面最新的研究工作，并结合开源数据库系统OceanBase，综合介绍新型OLTP（on-lineanalytical processing）系统的设计.

关键词: 事务处理, 并发控制, 日志与容错, 多核扩展性

Abstract: Since the 1970s, there has been considerable progress in hardware development; in particular, high-performance servers are now equipped with TB-level memory capacity and dozens of physical cores. Traditional OLTP systems, however, are still based on disk storage and designed for hardware with a small number of physical cores; hence, these systems are unable to effectively and fully exploit the computing power offered by new hardware. With the development of the Internet, applications commonly have high performance requirements for transactional systems. In extreme cases, some applications service millions of concurrent access requests, which traditional database systems cannot satisfy. Hence, the redesign and implementation of a transactional database system on high performance hardware has become an important research topic. In this study, we focused on recent work on transaction database systems on large memory and multi-core environments. We used OceanBase, an open source database developed by Alibaba, as an example to analyze the design of a new OLTP system.

Key words: transaction processing, concurrency control, log and recovery, multi-core scalability

中图分类号:

TP392

贺小龙, 马海欣, 何毓锟, 庞天泽, 赵琼. 新型OLTP系统的技术与实践[J]. 华东师范大学学报(自然科学版), 2018, 2018(5): 107-119.

HE Xiao-long, MA Hai-xin, HE Yu-kun, PANG Tian-ze, ZHAO Qiong. Technology and implementation of a new OLTP system[J]. Journal of East China Normal University(Natural Sc, 2018, 2018(5): 107-119.

参考文献

[1] GRAY J, REUTER A. Transaction Processing:Concepts and Techniques[M]. San Francisco:Margan Kaufmann, 2007.
[2] BAYER R, MCCREIGHT E M. Organization and maintenance of large ordered indexes[J]. Acta Informatica, 1972, 1(3):173-189.
[3] AGRAWAL D, BERNSTEIN A J, GUPTA P, et al. Distributed optimistic concurrency control with reduced rollback[J]. Distributed Computing, 1987, 2(1):45-59.
[4] BERNSTEIN P A, HADZILACOS V, GOODMAN N. Concurrency Control and Recovery in Database Systems[M]. MA:Addison-Wesley Publishing, 1987.
[5] FREEDMAN C S, ISMERT E, LARSON P, et al. Compilation in the Microsoft SQL Server Hekaton Engine[J]. IEEE Data(base) Engineering Bulletin, 2014:22-30.
[6] DEWITT D J, KATZ R H, OLKEN F, et al. Implementation techniques for main memory database systems[J]. Acm Sigmod Record, 1984, 14(2):1-8.
[7] MOHAN C, HADERLE D, LINDSAY B, et al. ARIES:A transaction recovery method supporting finegranularity locking and partial rollbacks using write-ahead logging[J]. Acm Transactions on Database Systems, 1992, 17(1):94-162.
[8] COBURN J, BUNKER T, SCHWARZ M, et al. From ARIES to MARS:Transaction support for next-generation, solid-state drives[C]//Twenty-Fourth ACM Symposium on Operating Systems Principles. ACM, 2013:197-212.
[9] 阳振坤. OceanBase关系数据库架构[J]. 华东师范大学学报(自然科学版), 2014(5):141-148.
[10] DIACONU C, FREEDMAN C, ISMERT E, et al. Hekaton:SQL server's memory-optimized OLTP engine[C]//ACM SIGMOD International Conference on Management of Data. ACM, 2013:1243-1254.
[11] HARIZOPOULOS S, ABADI D J, MADDEN S, et al. OLTP through the looking glass, and what we found there[C]//ACM SIGMOD International Conference on Management of Data. ACM, 2008:981-992.
[12] WANG T, JOHNSON R. Scalable logging through emerging non-volatile memory[J]. Proceedings of the Vldb Endowment, 2014, 7(10):865-876.
[13] JOHNSON R, PANDIS I, AILAMAKI A. Improving OLTP scalability using speculative lock inheritance[J]. Proceedings of the Vldb Endowment, 2009, 2(1):479-489.
[14] TU S, ZHENG W, KOHLER E, et al. Speedy transactions in multicore in-memory databases[C]//TwentyFourth ACM Symposium on Operating Systems Principles. ACM, 2013:18-32.
[15] BLANAS S, DIACONU C, FREEDMAN C, et al. High-performance concurrency control mechanisms for mainmemory databases[J]. Proceedings of the Vldb Endowment, 2011, 5(4):298-309.
[16] THOMASIAN A. Two-phase locking performance and its thrashing behavior[J]. ACM Transactions on Database Systems, 1993, 18(4):579-625.
[17] KUNG H T. On optimistic methods for concurrency control[J]. Acm Transactions on Database Systems, 1981, 6(2):213-226.
[18] SADOGHI M, CANIM M, BHATTACHARJEE B, et al. Reducing database locking contention through multiversion concurrency[J]. Proceedings of the Vldb Endowment, 2014, 7(13):1331-1342.
[19] REN K, THOMSON A, ABADI D J. Lightweight locking for main memory database systems[C]//International Conference on Very Large Data Bases. VLDB Endowment, 2012:145-156.
[20] YU X. An evaluation of concurrency control with one thousand cores[D]. Boston:Massachusetts Institute of Technology, 2015.
[21] PANDIS I, JOHNSON R, HARDAVELLAS N, et al. Data-oriented transaction execution[J]. Proceedings of the Vldb Endowment, 2010, 3(1/2):928-939.
[22] THOMSON A, THOMSON A, ABADI D J. An evaluation of the advantages and disadvantages of deterministic database systems[J]. Proceedings of the Vldb Endowment, 2014, 7(10):821-832.
[23] PAVLO A, CURINO C, ZDONIK S B, et al. Skew-aware automatic database partitioning in shared-nothing, parallel OLTP systems[C]//International conference on management of data, 2012:61-72.
[24] GOTTEMUKKALA V, LEHMAN T J. Locking and latching in a memory-resident database system[C]//Intl Conf on Very Large Databases, 1992:533-544.
[25] HELLAND P, SAMMER H, LYON J, et al. Group Commit Timers and High Volume Transaction Systems[C]//High performance transaction systems workshop, 1987:301-329.
[26] JOHNSON R, PANDIS I, STOICA R, et al. Aether:a scalable approach to logging[J]. Proceedings of the Vldb Endowment, 2010, 3(1/2):681-692.
[27] ALPERN D, ARORA G, BARCLAY C, et al. Oracle Database Advanced Application Developer's Guide, 11g Release 2(11.2) E17125-06[R/OL].[2018-07-10]. https://docs.oracle.com/cd/E1188201/appdev.112/e41502/toc.htm.
[28] SOISALON-SOININEN E, YLÖNEN T. Partial strictness in two-phase locking[C]//International Conference on Database Theory. Springer-Verlag, 1995:139-147.
[29] MALVIYA N, WEISBERG A, MADDEN S, et al. Rethinking main memory OLTP recovery[C]//International Conference on Data Engineering. IEEE, 2014:604-615.
[30] STOICA R, LEVANDOSKI J J, LARSON P A. Identifying hot and cold data in main-memory databases[C]//International Conference on Data Engineering. IEEE, 2013:26-37.
[31] ELDAWY A, LEVANDOSKI J, LARSON P Å. Trekking through Siberia:Managing cold data in a memoryoptimized database[J]. Proceedings of the Vldb Endowment, 2014,7(11):931-942.
[32] DEBRABANT J, PAVLO A, TU S, et al. Anti-caching:A new approach to database management system architecture[J]. Proceedings of the Vldb Endowment, 2013, 6(14):1942-1953.
[33] THOMSON A, DIAMOND T, WENG S C, et al. Calvin:Fast distributed transactions for partitioned database systems[C]//ACM SIGMOD International Conference on Management of Data. ACM, 2012:1-12.

新型OLTP系统的技术与实践

Technology and implementation of a new OLTP system

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 6

编辑推荐

Metrics

本文评价

[1]	凌向荣, 翁思扬, 张蓉. 基于锁避免的事务内阻塞分析与调度策略[J]. 华东师范大学学报（自然科学版）, 2024, 2024(5): 152-161.
[2]	连薛超, 刘维, 王清帅, 张蓉. 面向高冲突事务处理的架构设计和优化[J]. 华东师范大学学报（自然科学版）, 2023, 2023(6): 28-38.
[3]	刘文欣, 蔡鹏. 多主数据库中基于分区的并发控制[J]. 华东师范大学学报（自然科学版）, 2021, 2021(5): 84-93.
[4]	张舒燕, 王清帅, 张蓉. 针对高竞争电商负载的事务处理优化原型系统[J]. 华东师范大学学报（自然科学版）, 2020, 2020(5): 1-9.
[5]	王冬慧, 朱涛, 钱卫宁. 面向高通量事务处理的事务编译技术[J]. 华东师范大学学报(自然科学版), 2016, 2016(5): 10-17.
[6]	任堃, 李战怀. 新型OLTP数据库系统设计的关键技术及挑战[J]. 华东师范大学学报(自然科学版), 2014, 2014(5): 31-42.