华东师范大学学报(自然科学版) ›› 2017, Vol. 2017 ›› Issue (5): 20-29.doi: 10.3969/j.issn.1000-5641.2017.05.003

• 数据管理 • 上一篇    下一篇

异构Redis集群大规模评论数据存储负载均衡设计

张敬伟1, 丁志均1, 杨青2, 张会兵1, 张海涛1, 周娅1   

  1. 1. 桂林电子科技大学 广西可信软件重点实验室, 广西 桂林 541004;
    2. 桂林电子科技大学 广西自动检测技术与仪器重点实验室, 广西 桂林 541004
  • 收稿日期:2017-06-30 出版日期:2017-09-25 发布日期:2017-09-25
  • 通讯作者: 杨青,女,副教授,研究方向为智能信息处理.E-mail:gtyqing@hotmail.com E-mail:gtyqing@hotmail.com
  • 作者简介:张敬伟,男,博士,副教授,研究方向为海量数据管理.E-mail:gtzjw@hotmail.com
  • 基金资助:
    国家自然科学基金（61363005，61462017，U1501252）；广西自然科学基金（2014GXNSFAA118353，2014GXNSFAA118390）；广西自动检测技术与仪器重点实验室基金（YQ15110）；广西高校中青年教师基础能力提升项目（ky2016YB156）

Storage and load balancing for large-scale comment data on heterogeneous Redis cluster

ZHANG Jing-wei1, DING Zhi-jun1, YANG Qing2, ZHANG Hui-bing1, ZHANG Hai-tao1, ZHOU Ya1   

  1. 1. Guangxi Key Laboratory of Trusted Software, Guilin University of Electronic Technology, Guilin Guangxi 541004, China;
    2. Guangxi Key Laboratory of Automatic Detection Technology and Instrument, Guilin University of Electronic Technology, Guilin Guangxi 541004, China
  • Received:2017-06-30 Online:2017-09-25 Published:2017-09-25

RichHTML

19

PDF

746

摘要： 大规模评论数据的存储与查询性能对构建于其上的各类应用的快速响应具有重要影响.同时，异构计算环境中各计算节点性能呈现差异，如何充分开采各节点的计算和存储性能，优化大规模评论数据的存储与查询性能，是一个关键挑战.基于Redis集群的数据管理优势，首先提出了一种同构环境下基于卡槽存储平衡的大规模评论数据存储模型；然后论证了卡槽数目与节点查询效率的关系，以"负载与访问性能相平衡"的原则分配卡槽，进一步设计了异构环境下的集群节点负载计算和存储分配方法，充分开采了异构Redis集群中不同节点的性能.实验结果表明，提出的存储模型具有很好的存储平衡效果，提升了集群的整体查询效率.

关键词: 大规模评论数据, 存储负载均衡, 查询优化

Abstract: The storage and query performance for large-scale comment data have a great influence on those applications built on the above data. In a heterogeneous computing environment, each node has different performance on storage and computation, it presents a key challenge for optimizing the storage and query performance for large-scale comment data by taking full advantage of the performance of each node. Based on the ability of Redis cluster, we design a storage model for large-scale comment data in a homogeneous Redis cluster, which provides the storage balancing in Redis slots. And then, we discuss the relationship between the number of Redis slots and query efficiency to design a method for allocating storage on the real load of each computing node for heterogeneous Redis clusters, which can make full use of the performance of each node and can guide to allocate slots to nodes by balancing the query performance and storage loading. Our experimental results show that the proposed model has a good effect on storage loading and improve the query efficiency of the heterogeneous Redis cluster.

Key words: large-scale comment data, storage and load balancing, query optimization

中图分类号: