计算机科学

数据存储与处理分离架构下的子链接消除及优化

  • 王彦朝 ,
  • 胡卉芪 ,
  • 张召 ,
  • 刘小兵 ,
  • 段惠超
展开
  • 华东师范大学 数据科学与工程学院, 上海 200062
王彦朝,男,硕士研究生,研究方向为数据存储与数据挖掘.E-mail:wyz159753@126.com

收稿日期: 2017-06-26

  网络出版日期: 2018-07-19

基金资助

上海市,2017,年扬帆科技人才计划项目,(17YF1427800)

Sublink elimination and optimization in data storage and processing separation architecture

  • WANG Yan-zhao ,
  • HU Hui-qi ,
  • ZHANG Zhao ,
  • LIU Xiao-bing ,
  • DUAN Hui-chao
Expand
  • School of Data Science and Engineering, East China Normal University, Shanghai 200062, China

Received date: 2017-06-26

  Online published: 2018-07-19

摘要

在数据存储与处理分离架构下的,NewSQL,数据库中实现了子链接消除的功能,使其支持大部分子链接的执行,减少了从集中式数据库向分布式数据库迁移所需要的,SQL,改造代价,使得,NewSQL,数据库可以在电信、银行等传统行业投入使用.同时针对数据存储与处理分离的架构,对子链接消除之后的执行进行了优化,尽量减少了不同服务器之间数据的传输量.实现的结果使得,NewSQL,成功支持了大部分的子链接,且子链接执行效率也得到了提升.

本文引用格式

王彦朝 , 胡卉芪 , 张召 , 刘小兵 , 段惠超 . 数据存储与处理分离架构下的子链接消除及优化[J]. 华东师范大学学报(自然科学版), 2018 , 2018(4) : 90 -98 . DOI: 10.3969/j.issn.1000-5641.2018.04.009

Abstract

This paper implemented a sublink elimination optimizer in NewSQL RDBMS (Relational Database Management System). The purpose of the optimizer is to make complex SQL so that it can be executable in New SQL systems, and achieve auto-tuning of complex SQL to reduce the cost of migrating from a centralized database to a distributed database; this would make the NewSQL DBMS (Database Management System) available to telecomunications and banking industries. We also provide a method to minimize data transmission between servers after sublink promotion. Experiments show that the optimization effect of the query optimizer is notable and can replace manual optimization and reduce the workload of application migration.

参考文献

[1] CHANG F, DEAN J, GHEMAWAT S, et al. Bigtable:A distributed storage system for structured data[C]//Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation. Berkeley:USENIX Associaition, 2006, 205-218.
[2] CHODOROW K. MongoDB:The Definitive Guide[M].[S.l.]:O'Reilly Media Inc, 2013.
[3] CORBETT J C, DEAN J, EPSTEIN M, et al. Spanner:Google's globally distributed database[C]//Proceedings of the 10th USENIX Symposium on Operating Systems Design and Implementation. Berkeley:USENIX Associaition, 2012, 251-264.
[4] ALIBABA. OceanBase[EB/OL].[2017-05-02]. https://github.com/alibaba/oceanbase.
[5] daseECNU. Cedar[EB/OL].[2017-05-02]. https://github.com/daseECNU/Cedar.
[6] STONEBRAKER M, HANSON E, HONG C H. The design of POSTGRES rules system[C]//1987 IEEE 3rd International Conference on Data Engineering. IEEE, 1987:365-374.
[7] 李海翔. 数据库查询优化器的艺术:原理解析与SQL性能优化[M]. 北京:机械工业出版社, 2014:2-263.
[8] SESHADRI P, PIRAHESH H, LEUNG T Y C. Complex query decorrelation[C]//Proceedings of the 12th International Conference on Data Engineering. IEEE, 1996:450-458.
[9] RAO J, ROSS K A. Reusing invariants:A new strategy for correlated queries[J]. ACM SIGMOD Record, 1998, 27(2):37-48.
[10] BURTON H. Bloom, space/time trade-offs in hash coding with allowable errors[J]. Communications of the ACM, 1970, 13(7):422-426.
文章导航

/