Spatio-temporal Data Analysis and Intelligent Optimization Theory for Logistics

Indexing and query technology for steel logistic data

  • Tao ZOU ,
  • Rongtao QIAN ,
  • Jiali MAO
Expand
  • School of Data Science and Engineering, East China Normal University, Shanghai 200062, China

Received date: 2022-07-23

  Online published: 2022-09-26

Abstract

With digital transformation and the development of iron and steel logistics, the scale of iron and steel logistic data has rapidly expanded, and traditional relational databases can no longer meet the storage and query needs. Considering that a distributed not only structured query language (NoSQL) database has a simple expansion capability, fast reading and writing speeds, and low cost, in this study, distributed cloud storage and NoSQL technologies are used to store and build indexes for massive steel logistic data, improving the accuracy of the storage capacity and query performance of the logistic data. First, Spark is used to associate and fuse the data from different sources, and then store and manage the historical and real-time data generated by the freight platform in a hierarchical manner. It then builds spatiotemporal and attribute indexes for the three types of queries mainly involved in steel transportation to achieve an efficient query of multi-source logistic data. Finally, the experimental results based on real steel logistic data show that the proposed scheme is superior to traditional relational database methods in terms of data writing, storage, and querying, and can effectively support the storage and querying of massive logistic data.

Cite this article

Tao ZOU , Rongtao QIAN , Jiali MAO . Indexing and query technology for steel logistic data[J]. Journal of East China Normal University(Natural Science), 2022 , 2022(5) : 195 -207 . DOI: 10.3969/j.issn.1000-5641.2022.05.016

References

1 XIE D, LI F F, YAO B, et al. Simba: Efficient in-memory spatial analytics [C]// Proceedings of the 2016 International Conference on Management of Data. 2016: 1071-1085.
2 SHANG Z Y, LI G L, BAO Z F. DITA: Distributed in-memory trajectory analytics [C]// Proceedings of the 2018 International Conference on Management of Data. 2018: 725-740.
3 LI H D, LI G L, LIU J Y, et al. Ratel: Interactive analytics for large scale trajectories [C]// Proceedings of the 2019 International Conference on Management of Data. 2019: 1949-1952.
4 FANG Z Q, CHEN L, GAO Y J, et al. Dragoon: A hybrid and efficient big trajectory management system for offline and online analytics. The Very Large Data Bases Journal, 2021, 30, 287- 310.
5 NISHIMURA S, DAS S, AGRAWAL D, et al. MD-HBase: A scalable multi-dimensional data infrastructure for location aware services [C]// 2011 IEEE 12th International Conference on Mobile Data Management. IEEE, 2011: 7-16.
6 HUGHES J N, ANNEX A, EICHELBERGER C N, et al. GeoMesa: A distributed architecture for spatio-temporal fusion [C]// Geospatial Informatics, Fusion, and Motion Video Analytics V. 2015: 9473.
7 GUAN X F, BO C, LI Z Q, et al. ST-Hash: An efficient spatiotemporal index for massive trajectory data in a NoSQL database[C]// 2017 25th International Conference on Geoinformatics. 2017: 17334139.
8 LI R Y, HE H J, WANG R B, et al. JUST: JD urban spatio-temporal data engine [C]// Proceedings of the 2020 IEEE 36th International Conference on Data Engineering. IEEE, 2020: 1558-1569.
9 CHEN X Y, ZHANG C, GE B, et al. Efficient historical query in HBase for spatio-temporal decision support. International Journal of Computers Communications & Control, 2016, 11 (5): 613- 630.
10 XU J Q, BAO Z F, LU H. Continuous range queries over multi-attribute trajectories [C]// 2019 IEEE 35th International Conference on Data Engineering. IEEE, 2019: 1610-1613.
11 FENG B, ZHU Q, LIU M W, et al. An efficient graph-based spatio-temporal indexing method for task-oriented multi-modal scene data organization. International Society for Photogrammetry and Remote Sensing International Journal of Geo-Information, 2018, 7 (9): 371.
Outlines

/