面向高冲突事务处理的架构设计和优化

doi:10.3969/j.issn.1000-5641.2023.06.003

华东师范大学学报（自然科学版） ›› 2023, Vol. 2023 ›› Issue (6): 28-38.doi: 10.3969/j.issn.1000-5641.2023.06.003

面向高冲突事务处理的架构设计和优化

连薛超¹, 刘维², 王清帅¹, 张蓉^1,*()

1. 华东师范大学数据科学与工程学院, 上海　200062
2. 工业和信息化部电子第五研究所, 广州　511300

收稿日期:2022-07-17 出版日期:2023-11-25 发布日期:2023-11-23
通讯作者: 张蓉 E-mail:rzhang@dase.ecnu.edu.cn
基金资助:
国家自然科学基金(62072179); 2021 CCF-华为数据库创新研究计划; 基础软硬件性能与可靠性测评工业和信息化部重点实验室开放课题

Design and optimization of high-contention transaction processing architecture

Xuechao LIAN¹, Wei LIU², Qingshuai WANG¹, Rong ZHANG^1,*()

1. School of Data Science and Engineering, East China Normal University, Shanghai　200062, China
2. The Fifth Electronics Research Institute of the Ministry of Industry and Information Technology, Guangzhou　511300, China

Received:2022-07-17 Online:2023-11-25 Published:2023-11-23
Contact: Rong ZHANG E-mail:rzhang@dase.ecnu.edu.cn

摘要/Abstract

摘要：

无共享架构的分布式数据库是为了应对互联网业务的高可扩展性和高可用性而诞生的. 目前无共享架构的分布式数据库虽然已经有了长足的进展, 但是对于一些采用无状态计算层的无共享架构分布式数据库而言, 往往存在着冲突检测链路过长的问题, 这一问题在高冲突的场景下会更加突出. 针对这一问题, 设计了预先加锁和本地缓存两个策略, 并开发了配套的高冲突检测模块, 可以实现快速检测冲突并启动对应的高冲突处理策略. 实验表明, 对高冲突事务处理的架构设计和优化可以提高分布式数据库系统在高冲突负载下的性能.

关键词: 事务处理, 高冲突, 分布式数据库

Abstract:

Shared-nothing distributed databases are designed for the high scalability and high availability request of Internet-based applications. There have been significant achievements in shared-nothing distributed databases, but for some shared-nothing databases with stateless computation layers, long conflict-detection paths challenge database performance under high-contention workloads. To solve this problem, we design two methods, pre-lock and local cache, together with a high-contention detection module that allow high-contention to be quickly detected and the corresponding high-contention-handling strategy applied. Experiments show that our design and optimization for high-contention transaction-processing architecture can improve the performance of distributed databases under high-contention workloads.

Key words: transaction processing, high-contention, distributed database

中图分类号:

TP392

连薛超, 刘维, 王清帅, 张蓉. 面向高冲突事务处理的架构设计和优化[J]. 华东师范大学学报（自然科学版）, 2023, 2023(6): 28-38.

Xuechao LIAN, Wei LIU, Qingshuai WANG, Rong ZHANG. Design and optimization of high-contention transaction processing architecture[J]. Journal of East China Normal University(Natural Science), 2023, 2023(6): 28-38.

图/表 7

图1

图2

图3

图4

图5

图6

图7

参考文献 19

1	PAVLO A, ASLETT M.. What’s really new with NewSQL?. SIGMOD Record, 2016, 45 (2): 45- 55.
2	BINNIG C, CROTTY A, GALAKATOS A, et al.. The end of slow networks: It’s time for a redesign. Proceedings of the VLDB Endowment, 2016, 9 (7): 528- 539.
3	ZAMANIAN E, SHUN J, BINNIG C, et al. Chiller: Contention-centric transaction execution and data partitioning for modern networks [C]// Proceedings of the 2020 International Conference on Management of Data. 2020: 511-526.
4	FARBER F, MAY N, LEHNER W, et al. The SAP HANA database – an architecture overview [J]. IEEE Data Engineering Bulletin, 2012, 35(1): 28-33.
5	KALLMAN R, KIMURA H, NATKINS J, et al. H-store: A high-performance, distributed main memory transaction processing system [J]. Proceedings of the VLDB Endowment, 2008, 1(2): 1496–1499.
6	LOMET D, FEKETE A, WEIKUM G, et al. Unbundling transaction services in the cloud [EB/OL]. (2009-09-09)[2022-06-11]. https://arxiv.org/ftp/arxiv/papers/0909/0909.1768.pdf.
7	HUANG D X, LIU Q, CUI Q, et al.. TiDB: A raft-based HTAP database. Proceedings of VLDB Endowment, 2020, 13 (12): 3072- 3084.
8	TAFT R, SHARIF I, MATEI A, et al. CockroachDB: The resilient geo-distributed SQL database [C]// Proceedings of the 2020 International Conference on Management of Data. 2020: 1493–1509.
9	CORBETT J C, DEAN J, EPSTEIN M, et al. Spanner: Google’s globally-distributed database [C]// Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation. 2012: 251–264.
10	ZHOU J Y, XU M, SHRAER A, et al. FoundationDB: A distributed unbundled transactional key value store [C]// Proceedings of the 2021 International Conference on Management of Data. 2021: 2653–2666.
11	PENG D, DABEK F. Large-scale incremental processing using distributed transactions and notifications [C]// Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation. 2010: 251–264.
12	WANG T Z, KIMURA H.. Mostly-optimistic concurrency control for highly contended dynamic workloads on a thousand cores. Proceedings of the VLDB Endowment, 2016, 10 (2): 49- 60.
13	GUO Z H, WU K, YAN C, et al. Releasing locks as early as you can: Reducing contention of hotspots by violating two-phase locking [C]// Proceedings of the 2021 International Conference on Management of Data. 2021: 658–670.
14	WANG Z G, MU S, CAI Y, et al. Scaling multicore databases via constrained parallel execution [C]// Proceedings of the 2016 International Conference on Management of Data. 2016: 1643–1658.
15	THOMSON A, DIAMOND T, WENG S C, et al. Calvin: Fast distributed transactions for partitioned database systems [C]//Proceedings of the ACM SIGMOD International Conference on Management of Data. 2012. https://doi.org/10.1145/2213836.2213838.
16	GUO H, ZHOU X, CAI L. Lock violation for fault-tolerant distributed database system [C]// 2021 IEEE 37th IEEE International Conference on Data Engineering. 2021: 1416–1427.
17	TAFT R, MANSOUR E, SERAFINI M, et al.. E-Store: Fine-grained elastic partitioning for distributed transaction processing systems. Proceedings of the VLDB Endowment, 2014, 8 (3): 245- 256.
18	TIAN B Y, HUANG J M, MOZAFARI B Y, et al.. Contention-aware lock scheduling for transactional databases. Proceedings of the VLDB Endowment, 2018, 11 (5): 648- 662.
19	COOPER B F, SILBERSTEIN A, TAM E, et al. Benchmarking cloud serving systems with YCSB [C]// Proceedings of the 1st ACM Symposium on Cloud Computing. 2010: 143–154.

[1]	凌向荣, 翁思扬, 张蓉. 基于锁避免的事务内阻塞分析与调度策略[J]. 华东师范大学学报（自然科学版）, 2024, 2024(5): 152-161.
[2]	邹韬, 钱荣涛, 毛嘉莉. 基于钢铁物流数据的索引与查询技术研究[J]. 华东师范大学学报（自然科学版）, 2022, 2022(5): 195-207.
[3]	张舒燕, 王清帅, 张蓉. 针对高竞争电商负载的事务处理优化原型系统[J]. 华东师范大学学报（自然科学版）, 2020, 2020(5): 1-9.
[4]	贺小龙, 马海欣, 何毓锟, 庞天泽, 赵琼. 新型OLTP系统的技术与实践[J]. 华东师范大学学报(自然科学版), 2018, 2018(5): 107-119.
[5]	赵春扬, 肖冰, 郭进伟, 钱卫宁. 一致性协议在分布式数据库系统中的应用[J]. 华东师范大学学报(自然科学版), 2018, 2018(5): 91-106.
[6]	黄建伟, 张召, 钱卫宁. 分布式日志结构数据库系统的主键维护方法研究[J]. 华东师范大学学报(自然科学版), 2018, 2018(5): 79-90,119.
[7]	俞文谦, 胡爽, 胡卉芪. 面向Cedar的列存储设计与实现[J]. 华东师范大学学报(自然科学版), 2018, 2018(5): 67-78.
[8]	王彦朝, 胡卉芪, 张召, 刘小兵, 段惠超. 数据存储与处理分离架构下的子链接消除及优化[J]. 华东师范大学学报(自然科学版), 2018, 2018(4): 90-98.
[9]	余楷, 李志方, 周敏奇, 周傲英. 非阻塞事务型实时数据注入技术研究与实现[J]. 华东师范大学学报(自然科学版), 2016, 2016(5): 131-143.
[10]	钱招明, 王雷, 余晟隽, 宫学庆. 分布式系统中 Semi-Join 算法的实现[J]. 华东师范大学学报(自然科学版), 2016, 2016(5): 75-80.
[11]	毛思语, 张利军, 张小芳, 高锦涛, 李战怀. 面向分布式数据库的相关子查询优化策略[J]. 华东师范大学学报(自然科学版), 2016, 2016(5): 56-66.
[12]	储佳佳, 郭进伟, 刘柏众, 张晨东, 钱卫宁. 高可用数据库系统中的分布式一致性协议[J]. 华东师范大学学报(自然科学版), 2016, 2016(5): 1-9.
[13]	张晨东，郭进伟，刘柏众，储佳佳，周敏奇，钱卫宁. 基于Raft一致性协议的高可用性实现[J]. 华东师范大学学报(自然科学版), 2015, 2015(5): 172-.
[14]	庞天泽, 张晨东, 高明, 宫学庆. 分布式环境中数据库模式设计实践[J]. 华东师范大学学报(自然科学版), 2014, 2014(5): 290-300.
[15]	周欢, 樊秋实, 胡华梁. OceanBase一致性与可用性分析[J]. 华东师范大学学报(自然科学版), 2014, 2014(5): 103-116.

面向高冲突事务处理的架构设计和优化

Design and optimization of high-contention transaction processing architecture

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 19

相关文章 15

编辑推荐

Metrics

本文评价