Abstract:

Owing to the development of big data and artificial intelligence, industry users are increasingly leveraging machine learning for data analysis. This study focuses on data preparation for machine-learning scenarios to automatically construct a high-quality data-preparation pipeline for a specified machine-learning task (including a tabular dataset, training objective, and model configuration). Existing methods typically rely on deep neural networks and require a predefined and limited operation space for data preparation, making it difficult to satisfy customization demands. Researchers are increasingly investigating the use of large language models (LLMs) to automatically generate data-preparation pipelines. Nevertheless, LLM have difficulty understanding the structure of tabular data and lack performance-oriented optimization for downstream tasks. This ultimately limits model performance. Hence, this study proposes a novel framework named LLMPipe to generate a tabular-data preparation pipeline based on the optimization of large and small models. The method first leverages the advantages of LLM in semantic understanding and knowledge generalization to generate customized and domain-adaptive data-preparation operations by incorporating task descriptions and contextual information. Subsequently, it introduces a lightweight model to model operation features and uses reinforcement learning to optimize the operation ordering strategy, thereby dynamically orchestrating the entire data-preparation pipeline. Based on a collaborative “large-model expansion and small-model tuning” approach, the framework preserves the ability to understand domain knowledge and compose high-quality pipelines while improving overall computational efficiency and effectively enhancing the customizability and rationality of the generated pipelines. Extensive experiments are conducted across multiple real-world machine-learning tasks from Kaggle. The results reveal that the proposed method significantly outperforms existing methods, with state-of-the-art accuracy achieved across machine-learning tasks.

Key words: machine learning, data preparation pipeline, reinforcement learning, large language model, automatic machine learning