华东师范大学学报(自然科学版) ›› 2019, Vol. 2019 ›› Issue (5): 168-177.doi: 10.3969/j.issn.1000-5641.2019.05.014

• 新时期数据管理技术 • 上一篇    下一篇

基于Paxos的分布式一致性算法的实现与优化

祝朝凡, 郭进伟, 蔡鹏   

  1. 华东师范大学 数据科学与工程学院, 上海 200062
  • 收稿日期:2019-07-28 出版日期:2019-09-25 发布日期:2019-10-11
  • 通讯作者: 蔡鹏,男,副教授,研究方向为高性能事务处理.E-mail:pcai@dase.ecnu.edu.cn. E-mail:pcai@dase.ecnu.edu.cn
  • 作者简介:祝朝凡,男,硕士研究生,研究方向为分布式数据库.E-mail:chaofanzhu2@163.com.
  • 基金资助:
    国家重点研发计划(2018YFB1003303);国家自然科学基金(61432006)

Implementation and optimization of a distributed consistency algorithm based on Paxos

ZHU Chao-fan, GUO Jin-wei, CAI Peng   

  1. School of Data Science and Engineering, East China Normal University, Shanghai 200062, China
  • Received:2019-07-28 Online:2019-09-25 Published:2019-10-11

摘要: 互联网的不断发展,企业的信息化程度不断加强,不计其数的数据需要得到及时处理.但是网络环境不稳定,容易发生数据丢失、节点宕机,从而造成严重后果.因此,构建可以容错的分布式存储系统变得越来越受欢迎.为了保证系统的高可用性和一致性,需要引入分布式一致性算法.为了提高系统在不稳定网络下的性能,传统基于Paxos的分布式系统允许日志中存在空洞.然而,当节点进入恢复状态时,这些系统通常需要大量网络交互来补全日志空洞,这极大地增加了节点恢复的时间,从而影响了系统的可用性.针对节点恢复过程中补全日志空洞代价过大的问题,本文重新设计了日志项结构,优化了数据恢复流程,通过实验模拟,验证改进的基于Paxos的一致性算法的有效性.

关键词: 分布式存储系统, 一致性, 日志复制, 节点恢复

Abstract: With the ongoing development of the Internet, the degree of informationization in enterprises is continuously increasing, and more and more data needs to be processed in a timely manner. In this context, the instability of network environments may lead to data loss and node downtime, which can have potentially serious consequences. Therefore, building distributed fault-tolerant storage systems is becoming increasingly popular. In order to ensure high availability and consistency across the system, a distributed consistency algorithm needs to be introduced. To improve the performance of unstable networks, traditional distributed systems based on Paxos allow for the existence of holes in the log. However, when a node enters a recovery state, these systems typically require a large amount of network interaction to complete the holes in the log; this greatly increases the time for node recovery and thereby affects system availability. To address the complexity of the node recovery process after completing a hole log, this paper proposes a redesigned log entry structure and optimized data recovery process. The effectiveness of the improved Paxos-based consistency algorithm is verified with experimental simulation.

Key words: distributed storage systems, consistency, log replication, node recovery

中图分类号: