Locally lightweight course teaching-assistant system based on IPEX-LLM

doi:10.3969/j.issn.1000-5641.2024.05.015

Abstract

Abstract:

This study introduces and implements a local, lightweight, intelligent teaching-assistant system. Using the IPEX-LLM (Intel PyTorch extention for large language model) acceleration library, the system can efficiently deploy and execute large language models that are fine-tuned using the QLoRA (quantum-logic optimized resource allocation) framework on devices with limited computational resources. Combining this with enhanced retrieval techniques, the system provides flexible course customization through four major functional modules: intelligent Q&A, automated question generation, syllabus creation, and course PPT generation. This system is intended to assist educators in improving the quality and efficiency of lesson preparation and delivery, safeguarding data privacy, supporting personalized student learning, and offering real-time feedback. Performance tests exemplified by the optimized Chatglm3-6B model show the rapid inference capability of the system via the processing of a 64-token output task within 4.08 s in a resource-constrained environment. A practical case study comparing the functionality of the system with native Chatglm-6B and ChatGPT 4.0 further validates its superior accuracy and practicality.

Key words: intelligent assistance, limited computational resources, IPEX-LLM(Intel PyTorch extention for large language model), fine-tuning, enhanced retrieval

CLC Number:

TP391

Jiarui ZHANG, Qiming ZHANG, Fenglin BI, Yanbin ZHANG, Wei WANG, Erjin REN, Haili ZHANG. Locally lightweight course teaching-assistant system based on IPEX-LLM[J]. Journal of East China Normal University(Natural Science), 2024, 2024(5): 162-172.

Figures/Tables 9

Fig.1

Fig.2

Fig.3

Table 1

Table 2

Table 3

Table 4

Comparative analysis of responses"

模型	回答
模型	古希腊城邦制度的主要特征是什么？	清朝的闭关锁国政策对中国的影响？
Chatglm3-6B	古希腊城邦制度是其政治组织的基本形式, 每个城邦都是独立的政治实体.	清朝的闭关锁国政策导致中国与外界交流减少, 经济和科技发展受影响.
Chatglm3-6B (微调)	古希腊城邦制度的主要特征包括: ① 每个城邦是一个独立的政治实体; ② 城邦内有自己的政府和法律; ③ 公民参与政治生活, 特别是在雅典, 民主制度发达; ④ 城邦之间经常发生战争和联盟.	清朝的闭关锁国政策影响有: ① 经济上限制了对外贸易; ② 科技上阻碍了先进技术和思想的引入, 导致科技进步缓慢; ③ 政治上加强了中央集权, 维护了皇权; ④ 社会上文化和思想相对保守.
Chatglm3-6B (微调 + 知识库)	古希腊城邦制度的特点有: ① 城邦是独立的政治和经济单位; ② 公民 (成年男性) 大会是最高权力机构; ③ 城邦之间存在激烈的竞争和战争; ④ 文化和宗教活动在城邦生活中占有重要地位.	清朝的闭关锁国政策影响有: ① 经济停滞: 限制对外贸易, 导致经济发展落后于西方国家; ② 科技落后: 阻碍了先进技术的引入, 导致科技水平停滞; ③ 政治保守: 中央集权加强, 但也导致与外界隔绝, 缺乏国际视野; ④ 社会封闭: 文化和思想保守, 社会创新能力不足.
ChatGPT 4.0	古希腊城邦制度的主要特征包括: ① 独立自治: 每个城邦 (如雅典、斯巴达) 都是独立的政治实体, 有自己的政府和法律. ② 小规模: 城邦一般面积不大, 人口较少, 便于管理和参与公共事务. ③ 公民参与: 公民积极参与政治生活, 享有一定的民主权利. ④ 城邦文化: 每个城邦有独特的文化、宗教和社会习俗, 但共享希腊共同文化基础.	清朝的闭关锁国政策对中国的影响主要包括以下几个方面: ① 经济发展受限, 使得中国错失了与西方国家进行经济交流和技术引进的机会, 导致经济发展滞后. ②科技落后, 中国未能及时获取和应用西方的科技进步, 工业革命的成果未能传入中国. ③ 军事力量薄弱, 无法有效抵御外来侵略, 最终导致鸦片战争的失败.

Table 4

Fig.4

Fig.5

References 32

1	许苗,杨又.强人工智能赋能个性化教育研究[J/OL]. 软件导刊: 1-9[2024-05-23]. http://kns.cnki.net/kcms/detail/42.1671.TP.20240425.1140.012.html.
2	张凯, 覃正楚, 况莹.. 智慧教育环境中计算机辅助教学应用研究. 电脑知识与技术, 2023, 19 (13): 161- 163, 170.
3	OLIVEIRA K K S, DE SOUZA R A C.. Digital transformation towards education 4.0. Informatics in Education, 2022, 21 (2): 283- 309.
4	MUKUL E, BÜYÜKÖZKAN G.. Digital transformation in education: A systematic review of education 4.0. Technological Forecasting and Social Change, 2023, 194, 122664.
5	仲玉维.. 人工智能大模型引发的教育变革探索. 中小学信息技术教育, 2024, (5): 4.
6	DAN Y H, LEI Z K, GU Y Y, et al. EduChat: A large-scale language model-based chatbot system for intelligent education [EB/OL]. (2023-08-05)[2024-05-01]. https://doi.org/10.48550/arXiv.2308.02773.
7	魏忠.. 大模型下的教育品质与数据禁地. 中国信息技术教育, 2024, (10): 9.
8	DETTMERS T, PAGNONI A, HOLTZMAN A, et al. QLoRA: Efficient finetuning of quantized LLMs [EB/OL]. (2023-05-23)[2024-05-23]. https://doi.org/10.48550/arXiv.2305.14314.
9	DAI J J, DING D, SHI D, et al. BigDL 2.0: Seamless scaling of AI pipelines from laptops to distributed cluster [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2022: 21439-21446.
10	曾荣科, 李倩倩, 周文健, 等. 基于学习者画像的个性化习题资源推荐系统设计与实现 [J/OL]. 企业科技与发展, 2024: 1-4[2024-05-23]. https://doi.org/10.20137/j.cnki.45-1359/t.20240510.001.
11	张佳婷. 基于学习行为分析的学业风险预警及视频推荐方法研究[D]. 西安: 西安理工大学, 2024.
12	Chen L J, Chen P P, Lin Z J.. Artificial intelligence in education: A review. IEEE Access, 2020, 8, 75264- 75278.
13	PARK W, KWON H.. Implementing artificial intelligence education for middle school technology education in Republic of Korea. International Journal of Technology and Design Education, 2024, 34 (1): 109- 135.
14	NOSENKO Y.. Alta solution from Knewton as a tool of support for adaptive learning in mathematics. Educational Discourse: A Collection of Scientific Papers, 2020, 28 (11): 69- 81.
15	亢旭静. DreamBox Learning自适应学习平台与数学学科整合案例研究[D]. 太原: 山西师范大学, 2023.
16	卢金禹, 华博, 李志, 等.. 基于IPTV互动技术的云课堂系统设计及应用. 广播与电视技术, 2023, 50 (3): 22- 25.
17	ZHOU Y X, YANG K C. Exploring tensorrt to improve real-time inference for deep learning [C]// 2022 IEEE 24th Int Conf on High Performance Computing & Communications; 8th Int Conf on Data Science & Systems; 20th Int Conf on Smart City; 8th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys). IEEE, 2022: 2011-2018.
18	KIM S Y, LEE J, KIM C H, et al. Extending the ONNX runtime framework for the processing-in-memory execution [C]// 2022 International Conference on Electronics, Information, and Communication (ICEIC). IEEE, 2022. DOI: 10.1109/ICEIC54506.2022.9748444
19	IPEX-LLM Documentation [EB/OL]. [2024-05-23]. https://ipex-llm.readthedocs.io/en/latest/index.html.
20	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc., 2017: 6000-6010.
21	RAFFEL C, SHAZEER N, ROBERTS A, et al.. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 2020, 21 (1): 5485- 5551.
22	GÉRON A. Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow [M]. Sebastopol, CA, United States: O’Reilly Media Inc., 2022.
23	CHEN L C, LI Z R. Bailong: Bilingual transfer learning based on QLoRA and Zip-tie embedding [EB/OL]. (2024-04-01)[2024-05-23]. https://doi.org/10.48550/arXiv.2404.00862.
24	QIN H T, MA X D, ZHENG X Y, et al. Accurate LoRA-finetuning quantization of LLMs via information retention [EB/OL]. (2024-02-08)[2024-05-23]. https://doi.org/10.48550/arXiv.2402.05445.
25	DU Z X, QIAN Y J, LIU X, et al. GLM: General language model pretraining with autoregressive blank infilling [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics (ACL), 2022: 320-335.
26	BAI J Z, BAI S, CHU Y F, et al. Qwen technical report [EB/OL]. (2023-09-28)[2024-05-23]. https://doi.org/10.48550/arXiv.2309.16609.
27	BISONG E. Kubeflow and Kubeflow pipelines [M]// Building Machine Learning and Deep Learning Models on Google Cloud Platform. Berkeley, CA, United States: Apress, 2019: 671-685.
28	SUN T X, ZHANG X T, HE Z F, et al. MOSS: An open conversational large language model [J]. Machine Intelligence Research, 2024: Latest articles. DOI: 10.1007/s11633-024-1502-8. https://link.springer.com/content/pdf/10.1007/s11633-024-1502-8.pdf.
29	李庆辉. 深入浅出 Pandas: 利用 Python 进行数据处理与分析[M]. 北京: 机械工业出版社, 2021.
30	CURTIS A E, SMITH T A, ZIGANSHIN B A, et al.. The mystery of the Z-score. Aorta, 2016, 4, 124- 130.
31	JIANG Z B, XU F, GAO L Y, et al. Active retrieval augmented generation [C]// Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (ACL), 2023: 7969-7992.
32	LYU Y J, LI Z Y, NIU S M, et al. CRUD-RAG: A comprehensive Chinese benchmark for retrieval-augmented generation of large language models [EB/OL]. (2024-02-19)[2024-05-23]. https://doi.org/10.48550/arXiv.2401.17043.

组件	规格
处理器	Intel(R) Core Ultra 7 155H 3.80 GHz
显卡	Intel(R) Arc(TM) Graphics
内存	32 GB DDR4
存储	512 GB SSD
操作系统	Windows 11 家庭中文版

运行环境	内存占用/GB	向量化耗时/s	推理耗时/s
CPU	11.29	4528.67	2031.59
GPU	10.32	18.48	18.66
CPU + IPEX-LLM	5.27	31.94	11.23
GPU + IPEX-LLM	6.15	18.32	4.08

运行环境	内存占用/GB	向量化耗时/s	推理耗时/s
CPU	16.63	32.29	17.05
GPU
CPU + IPEX-LLM	5.18	21.71	9.13
GPU + IPEX-LLM	7.91	18.84	4.29

[1]	Jie WANG, Wenrui HUANG, Shengyu ZHAO, Xiaoya XIA, Fanyu HAN, Wei WANG, Yanbin ZHANG. OpenRank contribution evaluation method and empirical study in open-source course [J]. Journal of East China Normal University(Natural Science), 2024, 2024(5): 11-19.
[2]	Yanli FENG, Yu ZHOU, Fuxing HUANG, Junling WAN, Peisen YUAN. Study on short-term electricity load forecasting based on SF-Transformer for intelligent education platform [J]. Journal of East China Normal University(Natural Science), 2024, 2024(5): 173-182.
[3]	Junlin REN, Huan WANG, Xiaodi HUANG, Yanting LI, Shenggen JU. Sequence-aware and multi-type behavioral data driven knowledge concept recommendation for massive open online courses [J]. Journal of East China Normal University(Natural Science), 2024, 2024(5): 45-56.
[4]	Zhirui CHEN, Xuesong LU. Prompting open-source code large language models for student program repair [J]. Journal of East China Normal University(Natural Science), 2024, 2024(5): 93-103.
[5]	Linding XIE, Yuan ZHANG, Yihong CAI. Bioinformatics-based construction of immune prognostic gene model for hepatocellular carcinoma and preliminary model validation [J]. Journal of East China Normal University(Natural Science), 2024, 2024(4): 100-110.
[6]	Sijing RAO, Ying XIN, Junjun PAN. Skinning in character animation based on implicit surface [J]. Journal of East China Normal University(Natural Science), 2024, 2024(2): 143-156.
[7]	Luping FENG, Liye SHI, Wen WU, Jun ZHENG, Wenxin HU, Wei ZHENG. Collaborative stranger review-based recommendation [J]. Journal of East China Normal University(Natural Science), 2024, 2024(2): 53-64.
[8]	Yongzhuo ZHANG, Qingfeng ZHUGE, Edwin Hsing-Mean SHA, Yuhong SONG. Parallel block-based stochastic computing with adapted quantization [J]. Journal of East China Normal University(Natural Science), 2024, 2024(2): 76-85.
[9]	Xin LU, Chang HUANG, Zhiwei JIN. Multi-view and multi-pose lock pin point cloud model reconstruction based on turntable [J]. Journal of East China Normal University(Natural Science), 2024, 2024(2): 86-96.
[10]	Lingxiao TANG, Chang HUANG. Infrared small-target detection method based on double-layer local energy factor [J]. Journal of East China Normal University(Natural Science), 2024, 2024(2): 97-107.
[11]	Shirui WANG, Fang SHEN, Renhu LI, Peng LI. Research on water surface glint removal and information reconstruction methods for unmanned aerial vehicle hyperspectral images [J]. Journal of East China Normal University(Natural Science), 2024, 2024(1): 36-49.
[12]	Kaiyan XIAO, Jie LIAN. Sentence classification algorithm based on multi-kernel support vector machine [J]. Journal of East China Normal University(Natural Science), 2023, 2023(6): 85-94.
[13]	Ruibo CUI, Feng WANG. Momentum-updated representation with reconstruction constraint for limited-view 3D object recognition [J]. Journal of East China Normal University(Natural Science), 2023, 2023(6): 61-72.
[14]	Daojia CHEN, Zhiyun CHEN. Hierarchical description-aware personalized recommendation system [J]. Journal of East China Normal University(Natural Science), 2023, 2023(6): 73-84.
[15]	Juxiang REN, Zhongbao LIU. Integrating multi-granularity semantic features into the Chinese sentiment analysis method [J]. Journal of East China Normal University(Natural Science), 2023, 2023(6): 95-107.