Journal of East China Normal University(Natural Science) >

2024 , Vol. 2024 >Issue 5: 162 - 172

DOI: https://doi.org/10.3969/j.issn.1000-5641.2024.05.015

Educational Infrastructure

Locally lightweight course teaching-assistant system based on IPEX-LLM

  • Jiarui ZHANG ,
  • Qiming ZHANG ,
  • Fenglin BI ,
  • Yanbin ZHANG ,
  • Wei WANG ,
  • Erjin REN ,
  • Haili ZHANG
Expand
  • 1. School of Data Science and Engineering, East China Normal University, Shanghai　200062, China
    2. Intel Asia Pacific R & D Co. Ltd., Shanghai　200336, China
    3. Yuxing Technology (Zhejiang) Co. Ltd. Shanghai Branch, Shanghai　201821, China

Received date: 2024-06-20

  Online published: 2024-09-23

Fold

Abstract

This study introduces and implements a local, lightweight, intelligent teaching-assistant system. Using the IPEX-LLM (Intel PyTorch extention for large language model) acceleration library, the system can efficiently deploy and execute large language models that are fine-tuned using the QLoRA (quantum-logic optimized resource allocation) framework on devices with limited computational resources. Combining this with enhanced retrieval techniques, the system provides flexible course customization through four major functional modules: intelligent Q&A, automated question generation, syllabus creation, and course PPT generation. This system is intended to assist educators in improving the quality and efficiency of lesson preparation and delivery, safeguarding data privacy, supporting personalized student learning, and offering real-time feedback. Performance tests exemplified by the optimized Chatglm3-6B model show the rapid inference capability of the system via the processing of a 64-token output task within 4.08 s in a resource-constrained environment. A practical case study comparing the functionality of the system with native Chatglm-6B and ChatGPT 4.0 further validates its superior accuracy and practicality.

Key words： intelligent assistance; limited computational resources; IPEX-LLM(Intel PyTorch extention for large language model); fine-tuning; enhanced retrieval

Cite this article

Jiarui ZHANG , Qiming ZHANG , Fenglin BI , Yanbin ZHANG , Wei WANG , Erjin REN , Haili ZHANG . Locally lightweight course teaching-assistant system based on IPEX-LLM[J]. Journal of East China Normal University(Natural Science), 2024 , 2024(5) : 162 -172 . DOI: 10.3969/j.issn.1000-5641.2024.05.015

References

1 许苗,杨又.强人工智能赋能个性化教育研究[J/OL]. 软件导刊: 1-9[2024-05-23]. http://kns.cnki.net/kcms/detail/42.1671.TP.20240425.1140.012.html.
2 张凯, 覃正楚, 况莹.. 智慧教育环境中计算机辅助教学应用研究. 电脑知识与技术, 2023, 19 (13): 161- 163, 170.
3 OLIVEIRA K K S, DE SOUZA R A C.. Digital transformation towards education 4.0. Informatics in Education, 2022, 21 (2): 283- 309.
4 MUKUL E, BüYüK?ZKAN G.. Digital transformation in education: A systematic review of education 4.0. Technological Forecasting and Social Change, 2023, 194, 122664.
5 仲玉维.. 人工智能大模型引发的教育变革探索. 中小学信息技术教育, 2024, (5): 4.
6 DAN Y H, LEI Z K, GU Y Y, et al. EduChat: A large-scale language model-based chatbot system for intelligent education [EB/OL]. (2023-08-05)[2024-05-01]. https://doi.org/10.48550/arXiv.2308.02773.
7 魏忠.. 大模型下的教育品质与数据禁地. 中国信息技术教育, 2024, (10): 9.
8 DETTMERS T, PAGNONI A, HOLTZMAN A, et al. QLoRA: Efficient finetuning of quantized LLMs [EB/OL]. (2023-05-23)[2024-05-23]. https://doi.org/10.48550/arXiv.2305.14314.
9 DAI J J, DING D, SHI D, et al. BigDL 2.0: Seamless scaling of AI pipelines from laptops to distributed cluster [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2022: 21439-21446.
10 曾荣科, 李倩倩, 周文健, 等. 基于学习者画像的个性化习题资源推荐系统设计与实现 [J/OL]. 企业科技与发展, 2024: 1-4[2024-05-23]. https://doi.org/10.20137/j.cnki.45-1359/t.20240510.001.
11 张佳婷. 基于学习行为分析的学业风险预警及视频推荐方法研究[D]. 西安: 西安理工大学, 2024.
12 Chen L J, Chen P P, Lin Z J.. Artificial intelligence in education: A review. IEEE Access, 2020, 8, 75264- 75278.
13 PARK W, KWON H.. Implementing artificial intelligence education for middle school technology education in Republic of Korea. International Journal of Technology and Design Education, 2024, 34 (1): 109- 135.
14 NOSENKO Y.. Alta solution from Knewton as a tool of support for adaptive learning in mathematics. Educational Discourse: A Collection of Scientific Papers, 2020, 28 (11): 69- 81.
15 亢旭静. DreamBox Learning自适应学习平台与数学学科整合案例研究[D]. 太原: 山西师范大学, 2023.
16 卢金禹, 华博, 李志, 等.. 基于IPTV互动技术的云课堂系统设计及应用. 广播与电视技术, 2023, 50 (3): 22- 25.
17 ZHOU Y X, YANG K C. Exploring tensorrt to improve real-time inference for deep learning [C]// 2022 IEEE 24th Int Conf on High Performance Computing & Communications; 8th Int Conf on Data Science & Systems; 20th Int Conf on Smart City; 8th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys). IEEE, 2022: 2011-2018.
18 KIM S Y, LEE J, KIM C H, et al. Extending the ONNX runtime framework for the processing-in-memory execution [C]// 2022 International Conference on Electronics, Information, and Communication (ICEIC). IEEE, 2022. DOI: 10.1109/ICEIC54506.2022.9748444
19 IPEX-LLM Documentation [EB/OL]. [2024-05-23]. https://ipex-llm.readthedocs.io/en/latest/index.html.
20 VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc., 2017: 6000-6010.
21 RAFFEL C, SHAZEER N, ROBERTS A, et al.. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 2020, 21 (1): 5485- 5551.
22 GéRON A. Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow [M]. Sebastopol, CA, United States: O’Reilly Media Inc., 2022.
23 CHEN L C, LI Z R. Bailong: Bilingual transfer learning based on QLoRA and Zip-tie embedding [EB/OL]. (2024-04-01)[2024-05-23]. https://doi.org/10.48550/arXiv.2404.00862.
24 QIN H T, MA X D, ZHENG X Y, et al. Accurate LoRA-finetuning quantization of LLMs via information retention [EB/OL]. (2024-02-08)[2024-05-23]. https://doi.org/10.48550/arXiv.2402.05445.
25 DU Z X, QIAN Y J, LIU X, et al. GLM: General language model pretraining with autoregressive blank infilling [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics (ACL), 2022: 320-335.
26 BAI J Z, BAI S, CHU Y F, et al. Qwen technical report [EB/OL]. (2023-09-28)[2024-05-23]. https://doi.org/10.48550/arXiv.2309.16609.
27 BISONG E. Kubeflow and Kubeflow pipelines [M]// Building Machine Learning and Deep Learning Models on Google Cloud Platform. Berkeley, CA, United States: Apress, 2019: 671-685.
28 SUN T X, ZHANG X T, HE Z F, et al. MOSS: An open conversational large language model [J]. Machine Intelligence Research, 2024: Latest articles. DOI: 10.1007/s11633-024-1502-8. https://link.springer.com/content/pdf/10.1007/s11633-024-1502-8.pdf.
29 李庆辉. 深入浅出 Pandas: 利用 Python 进行数据处理与分析[M]. 北京: 机械工业出版社, 2021.
30 CURTIS A E, SMITH T A, ZIGANSHIN B A, et al.. The mystery of the Z-score. Aorta, 2016, 4, 124- 130.
31 JIANG Z B, XU F, GAO L Y, et al. Active retrieval augmented generation [C]// Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (ACL), 2023: 7969-7992.
32 LYU Y J, LI Z Y, NIU S M, et al. CRUD-RAG: A comprehensive Chinese benchmark for retrieval-augmented generation of large language models [EB/OL]. (2024-02-19)[2024-05-23]. https://doi.org/10.48550/arXiv.2401.17043.
Options
Outlines

/

〈 〉