云数据库运行环境的动态模拟

doi:10.3969/j.issn.1000-5641.2022.05.007

摘要/Abstract

摘要：

提出了一种全面、通用的数据库运行环境动态模拟工具, 可在常规及极端情况下从多种资源的维度实现对运行环境准确、高效的模拟. 该工具可以方便用户定制所需的测试场景, 降低数据库测试工作难度, 提高测试效率, 更好地实现评测结果的可参照性. 实验使用了自定义的负载场景, 展示了该工具的优越性.

关键词: 数据库, 云平台, 环境模拟, 数据库评测

Abstract:

This study proposes a comprehensive and general database environment simulation tool that can achieve the accurate, efficient and dynamic simulation of a normal runtime environment and broken resource situations from multiple dimensions. This tool can help users customize required test scenarios, reduce the difficulty of database benchmarking, improve testing efficiency, and achieve a better referential ability of benchmark results. Experiments under customized runtime environment demonstrate the superiority of this tool.

Key words: database, cloud platform, environment simulation, database benchmarking

中图分类号:

TP392

游舒泓, 苏仟, 张蓉. 云数据库运行环境的动态模拟[J]. 华东师范大学学报（自然科学版）, 2022, 2022(5): 73-89.

Shuhong YOU, Qian SU, Rong ZHANG. Dynamic simulation for cloud database runtime environment[J]. Journal of East China Normal University(Natural Science), 2022, 2022(5): 73-89.

图/表 17

图1

表1

表2

表3

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

表4

图13

参考文献 30

1	BULAO J. How many companies use cloud computing in 2022? All you need to know [EB/OL]. (2022-07-08)[2022-07-10]. https://techjury.net/blog/how-many-companies-use-cloud-computing.
2	IBM Cloud Education. Containerization [EB/OL]. (2021-06-23)[2022-07-10]. https://www.ibm.com/cloud/learn/containerization.
3	The Kubernetes Authors. Kubernetes [EB/OL]. [2022-07-10]. https://kubernetes.io.
4	VERBITSKI A, GUPTA A, SAHA D, et al. Amazon aurora: Design considerations for high throughput cloud-native relational databases [C]// Proceedings of the 2017 ACM International Conference on Management of Data. 2017: 1041-1052.
5	VERBITSKI A, GUPTA A, SAHA D, et al. Amazon aurora: On avoiding distributed consensus for I/Os, commits, and membership changes [C]// Proceedings of the 2018 International Conference on Management of Data. 2018: 789-796.
6	CAO W, LIU Y, CHENG Z, et al. POLARDB Meets computational storage: Efficiently support analytical workloads in cloud-native relational database [C]// 18th USENIX Conference on File and Storage Technologies (FAST 20). 2020: 29-41.
7	ANTONOPOULOS P, BUDOVSKI A, DIACONU C, et al. Socrates: The new sql server in the cloud [C]// Proceedings of the 2019 International Conference on Management of Data. 2019: 1743-1756.
8	DEPOUTOVITCH A, CHEN C, CHEN J, et al. Taurus database: How to be fast, available, and frugal in the cloud [C]// Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data. 2020: 1463-1478.
9	MERENSTEIN A, TARASOV V, ANWAR A, et al. CNSBench: A cloud native storage benchmark [C]// 19th USENIX Conference on File and Storage Technologies (FAST 21). 2021: 263-276.
10	MA M, YIN Z, ZHANG S, et al. Diagnosing root causes of intermittent slow queries in cloud databases. Proceedings of the VLDB Endowment, 2020, 13 (8): 1176- 1189.
11	GitHub Inc. Woodpecker [EB/OL]. (2021-06-02)[2022-07-10]. https://github.com/DBHammer/Woodpecker.
12	GitHub Inc. Chaosblade [EB/OL]. (2021-07-18)[2022-07-10]. https://github.com/chaosblade-io/chaosblade.
13	GitHub Inc. Chaos-mesh [EB/OL]. [2022-07-10]. https://github.com/chaos-mesh.
14	MATTHEWS J N, HU W, HAPUARACHCHI M, et al. Quantifying the performance isolation properties of virtualization systems [C]// Proceedings of the 2007 Workshop on Experimental Computer Science. 2007: 6-15.
15	FELTER W, FERREIRA A, RAJAMONY R, et al. An updated performance comparison of virtual machines and linux containers [C]// International Symposium on Performance Analysis of Systems and Software (ISPASS). New York: IEEE, 2015: 171-172.
16	PANDEY A. Impact of memory intensive applications on performance of cloud virtual machine [C]// 2014 Recent Advances in Engineering and Computational Sciences (RAECS). New York: IEEE, 2014: 1-6.
17	BUSARI M, WILLIAMSON C. ProWGen: A synthetic workload generation tool for simulation evaluation of web proxy caches. Computer Networks, 2002, 38 (6): 779- 794.
18	JIN S, BESTAVROS A. Gismo: A generator of internet streaming media objects and workloads. ACM SIGMETRICS Performance Evaluation Review, 2001, 29 (3): 2- 10.
19	KAO W I, IYER R K. A user-oriented synthetic workload generator [C]// 1992 12th International Conference on Distributed Computing System. New York: IEEE, 1992: 270-277.
20	HILL J H. An architecture independent approach to emulating computation intensive workload for early integration testing of enterprise DRE systems [C]// OTM Confederated International Conferences. Berlin, Heidelberg: Springer, 2009: 744-759.
21	MILLER J E, KASTURE H, KURIAN G, et al. Graphite: A distributed parallel simulator for multicores [C]// HPCA-16 2010 The Sixteenth International Symposium on High-Performance Computer Architecture. New York: IEEE, 2010: 1-12.
22	CARLSON T E, HEIRMAN W, EECKHOUT L. Sniper: Exploring the level of abstraction for scalable and accurate parallel multi-core simulation [C]// Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis. 2011: 1-12.
23	SANCHEZ D, KOZYRAKIS C. ZSim: Fast and accurate microarchitectural simulation of thousand-core systems. ACM SIGARCH Computer Architecture News, 2013, 41 (3): 475- 486.
24	WANG J, BEU J, BHEDA R, et al. Manifold: A parallel simulation framework for multicore systems [C]// 2014 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). New York: IEEE, 2014: 106-115.
25	JEONG H, LEE S. A workload generator for database system benchmarks [C]// International Conference on Information Integration and Web-based Applications & Services (iiWAS). 2005: 813-822.
26	ZHANG C, ZHANG R, SU Q, et al. Dynamic environment simulation for database performance evaluation [C]// Asia-Pacific Web (APWeb) and Web-Age Information Management (WAIM) Joint International Conference on Web and Big Data. Cham, Switzerland: Springer, 2021: 180-189.
27	GitHub Inc. Fio [EB/OL]. [2022-07-10]. https://github.com/axboe/fio.
28	HUANG D, LIU Q, CUI Q, et al. TiDB: A raft-based HTAP database. Proceedings of the VLDB Endowment, 2020, 13 (12): 3072- 3084.
29	GRIFFITHS N. nmon Performance: A free tool to analyze AIX and Linux performance [EB/OL]. (2006-02-27) [2022-07-10]. http://www.52testing.com/html/44/175444-87137.html.
30	TPC. TPC-C [EB/OL]. [2022-07-10]. http://www.tpc.org/tpcc/.

相关工作	模拟通用环境	模拟极端环境	模拟范围	缺陷
混沌测试工具	$ \times $	$ \surd $	硬件资源损坏	缺乏通用性、动态性
单一资源模拟工具	$ \times $	$ \times $	特定应用程序的工作负载	不适合用于模拟实际业务场景中的运行环境
文献[25]	$ \surd $	$ \times $	消耗CPU、内存、磁盘	缺乏全面性、动态性、准确性
文献[26]	$ \surd $	$ \times $	消耗CPU、内存、磁盘; 抢占网络带宽	缺乏全面性、准确性

资源类型	负载类型
硬件资源	CPU占用
	内存占用
	磁盘占用
	网络限制
软件资源	不停创建进程

资源类型	负载类型
硬件资源	占满CPU
	占满内存
	占满磁盘
	破坏网络
	主机宕机
软件资源	破坏进程
软件资源	破坏文件

行号	负载
1	CONCURRENT_EXEC;
2	CPU_OCCUPY[session;tikv-0;7,4,2;30,30,30];//分三阶段占用CPU, 每个占用周期是30 s
3	DISK_OCCUPY_IO[session;tikv-1;sequence_read;3000,1000;30,30];//分两阶段占用磁盘读I/O, 每个占用周期是30 s
4	THREAD_KILL[session;tikv-1;tikv-server];//终止该节点的tikv进程
5	MID_CONCURRENT;
6	NET_LIMIT_BW[session;tidb-0;eth0;80,90,10;30,30,30];//分三阶段限制网络带宽, 每个限制周期是30 s
7	DISK_OCCUPY_IO[session;tikv-2;sequence_read;3000,1000;30,30];
8	THREAD_KILL[session;tikv-2;tikv-server];
9	END_ CONCURRENT;

[1]	邹韬, 钱荣涛, 毛嘉莉. 基于钢铁物流数据的索引与查询技术研究[J]. 华东师范大学学报（自然科学版）, 2022, 2022(5): 195-207.
[2]	陈婷, 项兆坤, 徐金凯, 张蓉. 查询优化器连接顺序评估[J]. 华东师范大学学报（自然科学版）, 2022, 2022(5): 48-60.
[3]	项兆坤, 陈婷, 苏仟, 张蓉. 面向OLAP数据库查询处理功能的模糊测试工具[J]. 华东师范大学学报（自然科学版）, 2021, 2021(5): 74-83.
[4]	刘文欣, 蔡鹏. 多主数据库中基于分区的并发控制[J]. 华东师范大学学报（自然科学版）, 2021, 2021(5): 84-93.
[5]	阳文灿, 胡卉芪, 段惠超, 胡耀艺, 钱卫宁. CedarAdvisor: 负载自适应的自动化索引推荐工具[J]. 华东师范大学学报（自然科学版）, 2020, 2020(6): 52-62.
[6]	卫孝贤, 刘文欣, 蔡鹏. 多主云数据库的全局事务日志[J]. 华东师范大学学报（自然科学版）, 2020, 2020(5): 10-20.
[7]	刘子豪, 胡卉芪, 徐瑞, 周烜. 基于LevelDB的二维数据二级索引实现[J]. 华东师范大学学报(自然科学版), 2019, 2019(5): 159-167.
[8]	黄皓, 李志方, 王嘉伦, 翁楚良. 基于GPU的关系型流处理系统实现与优化[J]. 华东师范大学学报(自然科学版), 2019, 2019(5): 178-189.
[9]	张涛, 张小磊, 李宇明, 张春熙, 张蓉. Woodpecker+:基于数据特征的自定义负载性能评测[J]. 华东师范大学学报(自然科学版), 2019, 2019(5): 190-202.
[10]	俞文谦, 胡爽, 胡卉芪. 面向Cedar的列存储设计与实现[J]. 华东师范大学学报(自然科学版), 2018, 2018(5): 67-78.
[11]	黄建伟, 张召, 钱卫宁. 分布式日志结构数据库系统的主键维护方法研究[J]. 华东师范大学学报(自然科学版), 2018, 2018(5): 79-90,119.
[12]	赵春扬, 肖冰, 郭进伟, 钱卫宁. 一致性协议在分布式数据库系统中的应用[J]. 华东师范大学学报(自然科学版), 2018, 2018(5): 91-106.
[13]	王彦朝, 胡卉芪, 张召, 刘小兵, 段惠超. 数据存储与处理分离架构下的子链接消除及优化[J]. 华东师范大学学报(自然科学版), 2018, 2018(4): 90-98.
[14]	李捷荧, 李宇明, 张小磊, 张蓉. Woodpecker:支持细粒度冲突模拟的数据库测试框架[J]. 华东师范大学学报(自然科学版), 2018, 2018(2): 77-88.
[15]	邱鑫, 林欣. 两种新的非确定数据库上的Top-k查询[J]. 华东师范大学学报(自然科学版), 2017, 2017(1): 52-63.