An online learning behavior evaluation framework: Based on the fuzzy analytic hierarchy process and the fuzzy synthetic evaluation method

doi:10.3969/j.issn.1000-5641.2024.05.001

Abstract

Abstract:

To address the limitations currently experienced regarding the comprehensiveness and effectiveness of online learning evaluation in the smart education context, this paper proposes a novel framework for assessing online learning behavior based on the fuzzy analytic hierarchy process(FAHP) and the fuzzy synthetic evaluation method(FSEM). Drawing upon the CIPP(context, input, process, product) educational evaluation model and integrating the educational evaluation tag taxonomy system, the framework identifies five key dimensions: learning exploration, programming practice, knowledge acquisition, collaborative innovation, and communication interaction. These dimensions are further delineated into secondary and tertiary indicators to ensure comprehensive evaluation coverage. The framework utilizes FAHP-FSEM to determine the weights of each indicator level and employs consistency testing to validate the scientific and rational nature of the evaluation process. Implemented on the Shuishan Online platform, the framework leverages extensive multi-source process learning data to facilitate comprehensive evaluation from multiple perspectives and across various dimensions. Student profiles and learning behavior patterns are presented via a visual dashboard. This framework provides robust data support for enhancing personalized learning outcomes and advancing educational reform, demonstrating its broad applicability and potential.

Key words: online learning, learning evaluation, evaluation indicators, fuzzy analytic hierarchy process, fuzzy synthetic evaluation method

CLC Number:

G434

Yi ZHANG, Wenxu PI, Zexian WU, Yanbin ZHANG, Cheqing JIN, Wei WANG, Bin SU. An online learning behavior evaluation framework: Based on the fuzzy analytic hierarchy process and the fuzzy synthetic evaluation method[J]. Journal of East China Normal University(Natural Science), 2024, 2024(5): 1-10.

Figures/Tables 6

Fig.1

Fig.2

Fig.3

Table 1

Table 2

Fig.4

References 21

1	教育部. 教育部2022年工作要点 [EB/OL]. (2022-02-08)[2024-04-15]. http://www.moe.gov.cn/jyb_sjzl/moe_164/202202/t20220208_597666.html.
2	国务院. 中国教育现代化2035 [EB/OL]. (2019-02-23)[2024-03-23]. https://www.gov.cn/zhengce/2019-02/23/content_5367987.htm.
3	IBM中国研究院. 认知计算课程体系 [EB/OL]. [2024-03-10]. https://study.163.com/topics/ibm-cognitive-computing/.
4	袁梦琳. Kaggle平台在高中Python教学中的应用研究[D]. 重庆: 西南大学, 2021.
5	Microsoft. Microsoft 云和商业应用程序服务的基础知识课程 [EB/OL]. (2024-04-15)[2024-04-15]. https://learn.microsoft.com/zh-cn/training/educator-center/programs/msle/fundamentals.
6	科大讯飞. AI大学堂 [EB/OL]. [2024-03-10]. https://www.iflytek.com/aiacademy/.
7	PENG H, SU Y J, CHOU C, et al.. Ubiquitous knowledge construction: Mobile learning re-defined and a conceptual framework. Innovations in Education and Teaching International, 2009, 46 (2): 171- 183.
8	范洁, 杨岳湘. 远程教学平台学习行为分析统计系统设计与实现[C]// 计算机技术及应用进展. 合肥: 中国科学技术大学出版社, 2004: 934-937.
9	李念. 基于网络学习行为分析的评价模型研究[D]. 武汉: 华中师范大学, 2007.
10	杨晓红, 赵建华, 张丽丽.. 基于CIPP模型的在线学习行为评价体系构建与应用. 中国远程教育, 2020, (3): 82- 90.
11	李振华, 张晓辉, 刘春华.. 基于深度学习的在线学习行为预测研究. 计算机教育, 2023, (4): 58- 64.
12	AHMED S, HELMY Y, OUF S. A deep learning framework for predicting the student’s performance in the virtual learning environment [C]// 2022 5th International Conference on Computing and Informatics (ICCI). IEEE, 2022: 240-250.
13	WANG W, LU X S, HUANG B, et al. Shuishan Online: Constructing and teaching with a data-driven learning platform [C]// 2021 IEEE International Conference on Engineering, Technology & Education (TALE). IEEE, 2021: 536-543.
14	许祥云, 王佳佳.. 高校课程思政综合评价指标体系构建——基于CIPP评价模式的理论框架. 高校教育管理, 2022, 16 (1): 47- 60.
15	任寅姿. 标签类目体系: 面向业务的数据资产设计方法论[M]. 北京: 机械工业出版社, 2020, 19-26.
16	SAATY T L. Group decision making and the AHP [C]// The Analytic Hierarchy Process: Applications and Studies. Berlin: Springer, 1989: 59-67.
17	ZADEH L A.. Fuzzy sets. Information and Control, 1965, 8 (3): 338- 353.
18	ZAFAR A, ZAFAR M, SARWAR A, et al. A fuzzy AHP method for green supplier selection and evaluation [C]// Proceedings of the 12th International Conference on Management Science and Engineering Management. Berlin: Springer, 2019: 1355-1366.
19	ALIYEV R, TEMIZKAN H, ALIYEV R.. Fuzzy analytic hierarchy process-based multi-criteria decision making for universities ranking. Symmetry, 2020, 12 (8): 1351.
20	陈艳君, 张勇飞.. 基于模糊层次分析法的高校基层工作考核评价体系的研究与实践. 现代职业教育, 2022, (10): 73- 75.
21	韩红娟, 后晟烨, 付梓煊, 等.. 应用型高校教育信息化评价指标体系模型探讨及实证研究. 数理医药学杂志, 2024, 37 (3): 232- 239.

标度	定义	说明
1	同等重要	两指标相比较, 同等重要
2	稍微重要	两指标相比较, 一指标比另一指标稍微重要
3	明显重要	两指标相比较, 一指标比另一指标明显重要
4	重要得多	两指标相比较, 一指标比另一指标重要得多
5	极其重要	两指标相比较, 一指标比另一指标极其重要
${1}/{ {a}_{ij} }$	反比较	将指标$i$与${j}$相比较得到的判断记作${a}_{ij}$, 指标${j}$与${i}$相比较得到的判断记作${a}_{ji}$, 且${a}_{ji}={1}/{ {a}_{ij} }$

一级指标	一级指标权重	二级指标	二级指标权重
A 学习探索	0.17	A1 出勤率	0.17
		A2 课堂参与度	0.46
		A3 课程资源参与度	0.37
B 编程实践	0.14	B1 编程熟练度	0.26
		B2 问题解决策略	0.40
		B3 创新能力	0.34
C 知识掌握	0.25	C1 概念理解	0.34
		C2 学习进度	0.35
		C3 技能掌握	0.31
D 创新协作	0.25	D1 小组项目贡献度	0.75
D 创新协作	0.25	D2 团队合作积极性	0.25
E 沟通交互	0.19	E1 主动发布帖子	0.38
E 沟通交互	0.19	E2 与他人帖子交互	0.62

[1]	Chao KONG, Jiahui CHEN, Dan MENG, Huabin DIAO, Wei WANG, Liping ZHANG, Tao LIU. Personalized knowledge concept recommendation for massive open online courses [J]. Journal of East China Normal University(Natural Science), 2024, 2024(5): 32-44.
[2]	LIU Xiaoyu, HUANG Dingjiang. Online portfolio selection based on quadratic smooth-gray prediction [J]. Journal of East China Normal University(Natural Science), 2020, 2020(6): 115-128.