The designs and implementations of columnar storage in Cedar

doi:10.3969/j.issn.1000-5641.2018.05.006

Abstract

Abstract: With the growing size of data and analytical needs, the query performance of databases for OLAP (On-Line Analytical Processing) applications has become increasingly important. Cedar is a distributed relational database based on read-write decoupled architecture. Since Cedar is mainly oriented to the needs of OLTP (On-Line Transaction Processing) applications, it has insufficient performance for handling analytical processing workloads. To address this issue, many studies have shown that column storage technology can effectively improve the efficiency of I/O (Input/Output) and enhance the performance of analytical processing. This paper presents a column-based storage mechanism in Cedar. The study analyzes applicable scenarios and improves Cedar's data query and batch update methods for this mechanism. The results of an experiment demonstrate that the proposed mechanism can enhance the performance of analytical processing substantially, while limiting the negative impacts on transaction processing performance to within 10%.

Key words: distributed database, column-based storage, OLAP

CLC Number:

TP392

YU Wen-qian, HU Shuang, HU Hui-qi. The designs and implementations of columnar storage in Cedar[J]. Journal of East China Normal University(Natural Sc, 2018, 2018(5): 67-78.

References

[1] CODD E F, CODD S B, SALLEY C T. Providing OLAP (On-Line Analytical Processing) to User-Analysts:An IT mandate[J]. Codd and Date, 1993, 32:3-5.
[2] 华东师范大学. 面向大型银行应用的高通量可伸缩分布式数据库系统Cedar[DB/OL].[2018-05-16]. https://github.com/daseECNU/Cedar.
[3] COPELAND G P, KHOSHAFIAN S N. A decomposition storage model[C]//ACM SIGMOD International Conference on Management of Data. New York:ACM, 1985:268-279.
[4] ABADI D, MADDEN S, HACHEM N, Column-stores vs. row-stores:How different are they really?//Proceedings of the 2008 ACM SIGMOD international conference on Management of data. New York:ACM, 2008:967-980.
[5] RAMAN V, ATTALURI G, BARBER R, et al. DB2 with BLU acceleration:So much more than just a column store[J]. Proceedings of the VLDB Endowment, 2013, 11:1080-1091.
[6] PETRAKI E, IDREOS S, MANEGOLD S. Holistic Indexing in main-memory column-stores[C]//ACM SIGMOD International Conference on Management of Data. New York:ACM, 2015:1153-1166.
[7] LANG H, FUNKE F, BONCZ P A, et al. Data blocks:Hybrid OLTP and OLAP on compressed storage using both vectorization and compilation[C]//International Conference on Management of Data. New York:ACM, 2016:311-326.
[8] RAMNARAYAN J, MOZAFARI B, WALE S, et al. SnappyData:A hybrid transactional analytical store built on spark[C]//International Conference on Management of Data. New York:ACM, 2016:2153-2156.
[9] LEE J, HAN W S, NA H J, et al. Parallel replication across formats for scaling out mixed OLTP/OLAP workloads in main-memory databases[J]. The VLDB Journal, 2018, 27(3):421-444.
[10] SQream. SQream SQream DB[DB/OL].[2018-06-16]. https://sqream.com/.
[11] ROOT C, MOSTAK T. MapD:A GPU-powered big data analytics and visualization platform[C]//Proceeding of the SIGGRAPH'16 ACM SIGGRAPH 2016 Talks. New York:ACM, 2016:Article No 73. DOI:10.1145/2897839.2927468.
[12] 阳振坤. OceanBase关系数据库架构[J]. 华东师范大学学报(自然科学版), 2014(5):141-148, 163.
[13] 黄贵, 庄明强. OceanBase分布式存储引擎[J].华东师范大学学报(自然科学版), 2014(5):164-172.
[14] GOOGLE. Google Snappy[DB/OL].[2018-06-16]. https://github.com/google/snappy.
[15] SALOMON D. Data Compression:The Complete Reference[M]. New York:Springer-Verlag Inc, 2000.
[16] APACHE. Apache parquet[DB/OL].[2018-06-16]. http://parquet.apache.org/.