Journal of East China Normal University(Natural Sc ›› 2019, Vol. 2019 ›› Issue (5): 178-189.doi: 10.3969/j.issn.1000-5641.2019.05.015

• Data Management Techniques In the New Era • Previous Articles     Next Articles

Implementation and optimization of GPU-based relational streaming processing systems

HUANG Hao, LI Zhi-fang, WANG Jia-lun, WENG Chu-liang   

  1. School of Data Science and Engineering, East China Normal University, Shanghai 200062, China
  • Received:2019-07-29 Online:2019-09-25 Published:2019-10-11

Abstract: State-of-the-art CPU-based streaming processing systems support complex queries on large-scale datasets. However, limited by CPU computational capability, these systems suffer from the performance tradeoff between throughput and response time, and cannot achieve the best of both. In this paper, we propose a GPU-based streaming processing system, named Serval, that co-utilizes CPU and GPU resources and efficiently processes streaming queries by micro-batching. Serval adopts the pipeline model and uses streaming execution cache to optimize throughput and response time on large scale datasets. To meet the demands of various scenarios, Serval implements multiple tuning policies by scaling the micro-batch size dynamically. Experiments show that a single-server Serval outperforms a 3-server distributed Spark Streaming by 3.87x throughput with a 91% response time on average, reflecting the efficiency of the optimization.

Key words: streaming processing system, relational query, GPU database

CLC Number: