Technology and implementation of a new OLTP system

doi:10.3969/j.issn.1000-5641.2018.05.009

Abstract

Abstract: Since the 1970s, there has been considerable progress in hardware development; in particular, high-performance servers are now equipped with TB-level memory capacity and dozens of physical cores. Traditional OLTP systems, however, are still based on disk storage and designed for hardware with a small number of physical cores; hence, these systems are unable to effectively and fully exploit the computing power offered by new hardware. With the development of the Internet, applications commonly have high performance requirements for transactional systems. In extreme cases, some applications service millions of concurrent access requests, which traditional database systems cannot satisfy. Hence, the redesign and implementation of a transactional database system on high performance hardware has become an important research topic. In this study, we focused on recent work on transaction database systems on large memory and multi-core environments. We used OceanBase, an open source database developed by Alibaba, as an example to analyze the design of a new OLTP system.

Key words: transaction processing, concurrency control, log and recovery, multi-core scalability

CLC Number:

TP392

HE Xiao-long, MA Hai-xin, HE Yu-kun, PANG Tian-ze, ZHAO Qiong. Technology and implementation of a new OLTP system[J]. Journal of East China Normal University(Natural Sc, 2018, 2018(5): 107-119.

References

[1] GRAY J, REUTER A. Transaction Processing:Concepts and Techniques[M]. San Francisco:Margan Kaufmann, 2007.
[2] BAYER R, MCCREIGHT E M. Organization and maintenance of large ordered indexes[J]. Acta Informatica, 1972, 1(3):173-189.
[3] AGRAWAL D, BERNSTEIN A J, GUPTA P, et al. Distributed optimistic concurrency control with reduced rollback[J]. Distributed Computing, 1987, 2(1):45-59.
[4] BERNSTEIN P A, HADZILACOS V, GOODMAN N. Concurrency Control and Recovery in Database Systems[M]. MA:Addison-Wesley Publishing, 1987.
[5] FREEDMAN C S, ISMERT E, LARSON P, et al. Compilation in the Microsoft SQL Server Hekaton Engine[J]. IEEE Data(base) Engineering Bulletin, 2014:22-30.
[6] DEWITT D J, KATZ R H, OLKEN F, et al. Implementation techniques for main memory database systems[J]. Acm Sigmod Record, 1984, 14(2):1-8.
[7] MOHAN C, HADERLE D, LINDSAY B, et al. ARIES:A transaction recovery method supporting finegranularity locking and partial rollbacks using write-ahead logging[J]. Acm Transactions on Database Systems, 1992, 17(1):94-162.
[8] COBURN J, BUNKER T, SCHWARZ M, et al. From ARIES to MARS:Transaction support for next-generation, solid-state drives[C]//Twenty-Fourth ACM Symposium on Operating Systems Principles. ACM, 2013:197-212.
[9] 阳振坤. OceanBase关系数据库架构[J]. 华东师范大学学报(自然科学版), 2014(5):141-148.
[10] DIACONU C, FREEDMAN C, ISMERT E, et al. Hekaton:SQL server's memory-optimized OLTP engine[C]//ACM SIGMOD International Conference on Management of Data. ACM, 2013:1243-1254.
[11] HARIZOPOULOS S, ABADI D J, MADDEN S, et al. OLTP through the looking glass, and what we found there[C]//ACM SIGMOD International Conference on Management of Data. ACM, 2008:981-992.
[12] WANG T, JOHNSON R. Scalable logging through emerging non-volatile memory[J]. Proceedings of the Vldb Endowment, 2014, 7(10):865-876.
[13] JOHNSON R, PANDIS I, AILAMAKI A. Improving OLTP scalability using speculative lock inheritance[J]. Proceedings of the Vldb Endowment, 2009, 2(1):479-489.
[14] TU S, ZHENG W, KOHLER E, et al. Speedy transactions in multicore in-memory databases[C]//TwentyFourth ACM Symposium on Operating Systems Principles. ACM, 2013:18-32.
[15] BLANAS S, DIACONU C, FREEDMAN C, et al. High-performance concurrency control mechanisms for mainmemory databases[J]. Proceedings of the Vldb Endowment, 2011, 5(4):298-309.
[16] THOMASIAN A. Two-phase locking performance and its thrashing behavior[J]. ACM Transactions on Database Systems, 1993, 18(4):579-625.
[17] KUNG H T. On optimistic methods for concurrency control[J]. Acm Transactions on Database Systems, 1981, 6(2):213-226.
[18] SADOGHI M, CANIM M, BHATTACHARJEE B, et al. Reducing database locking contention through multiversion concurrency[J]. Proceedings of the Vldb Endowment, 2014, 7(13):1331-1342.
[19] REN K, THOMSON A, ABADI D J. Lightweight locking for main memory database systems[C]//International Conference on Very Large Data Bases. VLDB Endowment, 2012:145-156.
[20] YU X. An evaluation of concurrency control with one thousand cores[D]. Boston:Massachusetts Institute of Technology, 2015.
[21] PANDIS I, JOHNSON R, HARDAVELLAS N, et al. Data-oriented transaction execution[J]. Proceedings of the Vldb Endowment, 2010, 3(1/2):928-939.
[22] THOMSON A, THOMSON A, ABADI D J. An evaluation of the advantages and disadvantages of deterministic database systems[J]. Proceedings of the Vldb Endowment, 2014, 7(10):821-832.
[23] PAVLO A, CURINO C, ZDONIK S B, et al. Skew-aware automatic database partitioning in shared-nothing, parallel OLTP systems[C]//International conference on management of data, 2012:61-72.
[24] GOTTEMUKKALA V, LEHMAN T J. Locking and latching in a memory-resident database system[C]//Intl Conf on Very Large Databases, 1992:533-544.
[25] HELLAND P, SAMMER H, LYON J, et al. Group Commit Timers and High Volume Transaction Systems[C]//High performance transaction systems workshop, 1987:301-329.
[26] JOHNSON R, PANDIS I, STOICA R, et al. Aether:a scalable approach to logging[J]. Proceedings of the Vldb Endowment, 2010, 3(1/2):681-692.
[27] ALPERN D, ARORA G, BARCLAY C, et al. Oracle Database Advanced Application Developer's Guide, 11g Release 2(11.2) E17125-06[R/OL].[2018-07-10]. https://docs.oracle.com/cd/E1188201/appdev.112/e41502/toc.htm.
[28] SOISALON-SOININEN E, YLÖNEN T. Partial strictness in two-phase locking[C]//International Conference on Database Theory. Springer-Verlag, 1995:139-147.
[29] MALVIYA N, WEISBERG A, MADDEN S, et al. Rethinking main memory OLTP recovery[C]//International Conference on Data Engineering. IEEE, 2014:604-615.
[30] STOICA R, LEVANDOSKI J J, LARSON P A. Identifying hot and cold data in main-memory databases[C]//International Conference on Data Engineering. IEEE, 2013:26-37.
[31] ELDAWY A, LEVANDOSKI J, LARSON P Å. Trekking through Siberia:Managing cold data in a memoryoptimized database[J]. Proceedings of the Vldb Endowment, 2014,7(11):931-942.
[32] DEBRABANT J, PAVLO A, TU S, et al. Anti-caching:A new approach to database management system architecture[J]. Proceedings of the Vldb Endowment, 2013, 6(14):1942-1953.
[33] THOMSON A, DIAMOND T, WENG S C, et al. Calvin:Fast distributed transactions for partitioned database systems[C]//ACM SIGMOD International Conference on Management of Data. ACM, 2012:1-12.