华东师范大学学报(自然科学版) ›› 2018, Vol. 2018 ›› Issue (4): 99-108.doi: 10.3969/j.issn.1000-5641.2018.04.010

• 计算机科学 • 上一篇    下一篇

基于Hadoop/Hive的乳制品溯源数据计算及性能优化

朱淑鑫1, 李悦1, 袁培森1, 徐焕良1,2, 王康1, 谢忠红1   

  1. 1. 南京农业大学 信息科学与技术学院, 南京 210095;
    2. 江苏省肉类生产与加工质量安全控制协同创新中心, 南京 210095
  • 收稿日期:2017-06-19 出版日期:2018-07-25 发布日期:2018-07-19
  • 通讯作者: 谢忠红,女,副教授,研究方向为农业信息化.E-mail:xiezh@njau.edu.cn E-mail:xiezh@njau.edu.cn
  • 作者简介:朱淑鑫,女,副教授,研究方向为农业信息化与大数据处理.E-mail:zsx@njau.edu.cn
  • 基金资助:
    中央高校基本科研业务费专项资金(KYZ201551,KYZ201670,KYZ201752,KJQN201651);国家科技支撑计划(2015BAK36B05);江苏省重点研发计划项目(BE2016803);国家自然科学基金(61502236)

Data calculation and performance optimization of dairy traceability based on Hadoop/Hive

ZHU Shu-xin1, LI Yue1, YUAN Pei-sen1, XU Huan-liang1,2, WANG Kang1, XIE Zhong-hong1   

  1. 1. College of Information Science and Technology, Nanjing Agricultural University, Nanjing 210095, China;
    2. Jiangsu Collaborative Innovation Center of Meat Production and Processing, Quality and Safety Control, Nanjing 210095, China
  • Received:2017-06-19 Online:2018-07-25 Published:2018-07-19

摘要: 为了提升传统乳制品溯源系统应对大规模企业生产数据的性能,本文分析了乳制品相关企业供应链业务流程、关键溯源单元和溯源信息,结合Hadoop/Hive大数据技术和分布式数据库技术,设计并构建了基于Hadoop/Hive的乳制品溯源框架.搭建模拟大数据环境并使用实际生产数据对系统性能进行测试,实验结果表明,引入Hadoop/Hive技术后,系统的平均数据存储速度、平均数据访问速度、平均数据交互速度分别提升了87.43%、27.10%、58.16%.改进后的乳制品溯源系统存储和处理大规模数据的能力明显优于传统的乳制品溯源系统.

关键词: Hadoop/Hive, 乳制品溯源, 数据计算, 性能优化

Abstract: In order to enhance the performance of traditional dairy traceability systems for the production data of large-scale enterprise, this paper analyzed the supply chain process of dairy enterprises, key traceability units and traceability information; combining Hadoop/Hive big data technology and distributed database technology, the paper designed and constructed a dairy products traceability framework based on Hadoop/Hive. We built a simulated large-scale data environment and used actual production data to test the system performance. The experimental results showed that after the introduction of the Hadoop/Hive technology system, the average data storage speed, the average data access speed, and the average data exchange rate increased by 87.43%, 27.10% and 58.16%, respectively. The improved traceability system for dairy products is superior to the traditional dairy traceability system in storing and processing large-scale data.

Key words: Hadoop/Hive, dairy products traceability, data calculation, performance optimization

中图分类号: