Abstract:

With digital transformation and the development of iron and steel logistics, the scale of iron and steel logistic data has rapidly expanded, and traditional relational databases can no longer meet the storage and query needs. Considering that a distributed not only structured query language (NoSQL) database has a simple expansion capability, fast reading and writing speeds, and low cost, in this study, distributed cloud storage and NoSQL technologies are used to store and build indexes for massive steel logistic data, improving the accuracy of the storage capacity and query performance of the logistic data. First, Spark is used to associate and fuse the data from different sources, and then store and manage the historical and real-time data generated by the freight platform in a hierarchical manner. It then builds spatiotemporal and attribute indexes for the three types of queries mainly involved in steel transportation to achieve an efficient query of multi-source logistic data. Finally, the experimental results based on real steel logistic data show that the proposed scheme is superior to traditional relational database methods in terms of data writing, storage, and querying, and can effectively support the storage and querying of massive logistic data.

Key words: steel logistic data, HBase, spatio-temporal index, query