华东师范大学学报（自然科学版） >

2024 , Vol. 2024 >Issue 5: 162 - 172

DOI: https://doi.org/10.3969/j.issn.1000-5641.2024.05.015

教育基础设施

基于IPEX-LLM的本地轻量化课程教学智能辅助系统

  • 张嘉睿 ,
  • 张豈明 ,
  • 毕枫林 ,
  • 张琰彬 ,
  • 王伟 ,
  • 任而今 ,
  • 张海立
展开
  • 1. 华东师范大学 数据科学与工程学院, 上海　200062
    2. 英特尔亚太研发有限公司, 上海　200336
    3. 驭行科技(浙江)有限公司上海分公司, 上海　201821

收稿日期: 2024-06-20

  网络出版日期: 2024-09-23

收起

Locally lightweight course teaching-assistant system based on IPEX-LLM

  • Jiarui ZHANG ,
  • Qiming ZHANG ,
  • Fenglin BI ,
  • Yanbin ZHANG ,
  • Wei WANG ,
  • Erjin REN ,
  • Haili ZHANG
Expand
  • 1. School of Data Science and Engineering, East China Normal University, Shanghai　200062, China
    2. Intel Asia Pacific R & D Co. Ltd., Shanghai　200336, China
    3. Yuxing Technology (Zhejiang) Co. Ltd. Shanghai Branch, Shanghai　201821, China

Received date: 2024-06-20

  Online published: 2024-09-23

Fold

摘要

提出并实现了一个本地轻量化课程教学智能辅助系统. 该系统利用IPEX-LLM (Intel PyTorch extention for large language model)加速库, 在计算资源受限的设备上高效部署并运行经过QLoRA(quantum-logic optimized resource allocation)框架微调的大语言模型, 并结合增强检索技术, 实现了智能问答、智能出题、教学大纲生成、教学演示文档生成等4个主要功能模块的课程灵活定制, 在帮助教师提高教学备课和授课的质量与效率、保护数据隐私的同时, 支撑学生个性化学习并提供实时反馈. 在性能实验中, 以集成优化后的Chatglm3-6B模型为例, 该系统处理64-token输出任务时仅需4.08 s, 验证了其在资源受限环境下快速推理的能力. 在实践案例分析中, 通过与原生Chatgml-6B和ChatGPT4.0在功能实现上的对比, 进一步表明了该系统具备优越的准确性和实用性.

关键词： 智能辅助; 计算资源受限; IPEX-LLM; 微调; 增强检索

本文引用格式

张嘉睿 , 张豈明 , 毕枫林 , 张琰彬 , 王伟 , 任而今 , 张海立 . 基于IPEX-LLM的本地轻量化课程教学智能辅助系统[J]. 华东师范大学学报（自然科学版）, 2024 , 2024(5) : 162 -172 . DOI: 10.3969/j.issn.1000-5641.2024.05.015

Abstract

This study introduces and implements a local, lightweight, intelligent teaching-assistant system. Using the IPEX-LLM (Intel PyTorch extention for large language model) acceleration library, the system can efficiently deploy and execute large language models that are fine-tuned using the QLoRA (quantum-logic optimized resource allocation) framework on devices with limited computational resources. Combining this with enhanced retrieval techniques, the system provides flexible course customization through four major functional modules: intelligent Q&A, automated question generation, syllabus creation, and course PPT generation. This system is intended to assist educators in improving the quality and efficiency of lesson preparation and delivery, safeguarding data privacy, supporting personalized student learning, and offering real-time feedback. Performance tests exemplified by the optimized Chatglm3-6B model show the rapid inference capability of the system via the processing of a 64-token output task within 4.08 s in a resource-constrained environment. A practical case study comparing the functionality of the system with native Chatglm-6B and ChatGPT 4.0 further validates its superior accuracy and practicality.

Key words： intelligent assistance; limited computational resources; IPEX-LLM(Intel PyTorch extention for large language model); fine-tuning; enhanced retrieval

参考文献

1 许苗,杨又.强人工智能赋能个性化教育研究[J/OL]. 软件导刊: 1-9[2024-05-23]. http://kns.cnki.net/kcms/detail/42.1671.TP.20240425.1140.012.html.
2 张凯, 覃正楚, 况莹.. 智慧教育环境中计算机辅助教学应用研究. 电脑知识与技术, 2023, 19 (13): 161- 163, 170.
3 OLIVEIRA K K S, DE SOUZA R A C.. Digital transformation towards education 4.0. Informatics in Education, 2022, 21 (2): 283- 309.
4 MUKUL E, BüYüK?ZKAN G.. Digital transformation in education: A systematic review of education 4.0. Technological Forecasting and Social Change, 2023, 194, 122664.
5 仲玉维.. 人工智能大模型引发的教育变革探索. 中小学信息技术教育, 2024, (5): 4.
6 DAN Y H, LEI Z K, GU Y Y, et al. EduChat: A large-scale language model-based chatbot system for intelligent education [EB/OL]. (2023-08-05)[2024-05-01]. https://doi.org/10.48550/arXiv.2308.02773.
7 魏忠.. 大模型下的教育品质与数据禁地. 中国信息技术教育, 2024, (10): 9.
8 DETTMERS T, PAGNONI A, HOLTZMAN A, et al. QLoRA: Efficient finetuning of quantized LLMs [EB/OL]. (2023-05-23)[2024-05-23]. https://doi.org/10.48550/arXiv.2305.14314.
9 DAI J J, DING D, SHI D, et al. BigDL 2.0: Seamless scaling of AI pipelines from laptops to distributed cluster [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2022: 21439-21446.
10 曾荣科, 李倩倩, 周文健, 等. 基于学习者画像的个性化习题资源推荐系统设计与实现 [J/OL]. 企业科技与发展, 2024: 1-4[2024-05-23]. https://doi.org/10.20137/j.cnki.45-1359/t.20240510.001.
11 张佳婷. 基于学习行为分析的学业风险预警及视频推荐方法研究[D]. 西安: 西安理工大学, 2024.
12 Chen L J, Chen P P, Lin Z J.. Artificial intelligence in education: A review. IEEE Access, 2020, 8, 75264- 75278.
13 PARK W, KWON H.. Implementing artificial intelligence education for middle school technology education in Republic of Korea. International Journal of Technology and Design Education, 2024, 34 (1): 109- 135.
14 NOSENKO Y.. Alta solution from Knewton as a tool of support for adaptive learning in mathematics. Educational Discourse: A Collection of Scientific Papers, 2020, 28 (11): 69- 81.
15 亢旭静. DreamBox Learning自适应学习平台与数学学科整合案例研究[D]. 太原: 山西师范大学, 2023.
16 卢金禹, 华博, 李志, 等.. 基于IPTV互动技术的云课堂系统设计及应用. 广播与电视技术, 2023, 50 (3): 22- 25.
17 ZHOU Y X, YANG K C. Exploring tensorrt to improve real-time inference for deep learning [C]// 2022 IEEE 24th Int Conf on High Performance Computing & Communications; 8th Int Conf on Data Science & Systems; 20th Int Conf on Smart City; 8th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys). IEEE, 2022: 2011-2018.
18 KIM S Y, LEE J, KIM C H, et al. Extending the ONNX runtime framework for the processing-in-memory execution [C]// 2022 International Conference on Electronics, Information, and Communication (ICEIC). IEEE, 2022. DOI: 10.1109/ICEIC54506.2022.9748444
19 IPEX-LLM Documentation [EB/OL]. [2024-05-23]. https://ipex-llm.readthedocs.io/en/latest/index.html.
20 VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc., 2017: 6000-6010.
21 RAFFEL C, SHAZEER N, ROBERTS A, et al.. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 2020, 21 (1): 5485- 5551.
22 GéRON A. Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow [M]. Sebastopol, CA, United States: O’Reilly Media Inc., 2022.
23 CHEN L C, LI Z R. Bailong: Bilingual transfer learning based on QLoRA and Zip-tie embedding [EB/OL]. (2024-04-01)[2024-05-23]. https://doi.org/10.48550/arXiv.2404.00862.
24 QIN H T, MA X D, ZHENG X Y, et al. Accurate LoRA-finetuning quantization of LLMs via information retention [EB/OL]. (2024-02-08)[2024-05-23]. https://doi.org/10.48550/arXiv.2402.05445.
25 DU Z X, QIAN Y J, LIU X, et al. GLM: General language model pretraining with autoregressive blank infilling [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics (ACL), 2022: 320-335.
26 BAI J Z, BAI S, CHU Y F, et al. Qwen technical report [EB/OL]. (2023-09-28)[2024-05-23]. https://doi.org/10.48550/arXiv.2309.16609.
27 BISONG E. Kubeflow and Kubeflow pipelines [M]// Building Machine Learning and Deep Learning Models on Google Cloud Platform. Berkeley, CA, United States: Apress, 2019: 671-685.
28 SUN T X, ZHANG X T, HE Z F, et al. MOSS: An open conversational large language model [J]. Machine Intelligence Research, 2024: Latest articles. DOI: 10.1007/s11633-024-1502-8. https://link.springer.com/content/pdf/10.1007/s11633-024-1502-8.pdf.
29 李庆辉. 深入浅出 Pandas: 利用 Python 进行数据处理与分析[M]. 北京: 机械工业出版社, 2021.
30 CURTIS A E, SMITH T A, ZIGANSHIN B A, et al.. The mystery of the Z-score. Aorta, 2016, 4, 124- 130.
31 JIANG Z B, XU F, GAO L Y, et al. Active retrieval augmented generation [C]// Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (ACL), 2023: 7969-7992.
32 LYU Y J, LI Z Y, NIU S M, et al. CRUD-RAG: A comprehensive Chinese benchmark for retrieval-augmented generation of large language models [EB/OL]. (2024-02-19)[2024-05-23]. https://doi.org/10.48550/arXiv.2401.17043.
Options
文章导航

/

〈 〉