Journal of East China Normal University(Natural Sc

Previous Articles     Next Articles

A join algorithm based on bloom filter in OceanBase

MAO Xiao-xiao, DUAN Hui-chao, GAO Ming   

  1. Institute for Data Science and Engineering, East China Normal University, Shanghai 200062, China
  • Received:2016-06-24 Online:2016-09-25 Published:2016-11-29

RichHTML

13

PDF

830

Abstract:

In the era of big data, the movement of “de-IOE” campaign and the development of activities such as Double 11 have put forward higher request of the performance of distributed database. OceanBase is an open sourced distributed database implemented by Alibaba. It supports for cross-table relational query of massive data but the performance for complex queries remains to be improved. The network transmission overheads caused by join operator seriously influenced the performance of distributed database. This paper proposes a join algorithm based on bloom filter. It filters the data of the right table
by constructing a bloom filter on the join column of the left table. The key point of this algorithm is that it reduces the overhead of unnecessary data transmission and the consumption of memory resources by data processing. We implement this algorithm in OceanBase and the experiment results show that the algorithm can greatly improve the efficiency of join operator.

Key words: OceanBase, join operation, bloom filter