基于集成特征选择的网络异常流量检测

doi:10.3969/j.issn.1000-5641.2021.06.011

摘要/Abstract

摘要：

随着互联网技术的不断发展, 网络的安全问题日益受到人们的重视. 网络异常流量检测能够为拦截网络攻击提供有效的保障. 然而, 为了准确检测网络中的异常流量, 通常需要分析海量的数据. 分析这些数据不仅消耗巨大的计算资源, 降低检测的实时性, 还有可能降低检测的准确率. 为解决这些问题, 提出了一种基于集成特征选择的网络异常流量检测方法: 采用5种不同的特征选择算法, 设计了一种投票机制以选择特征子集; 用朴素贝叶斯、决策树、XGBoost (eXtreme Gradient Boosting)这3种不同的机器学习算法, 评估所采用的特征选择算法; 选择表现最好的算法以实现网络异常流量检测. 实验结果表明, 在使用提出的算法所选取出的最优子特征上, 所提方法的运行时间比在原始数据集上少了84.38%, 平均准确率比单个特征选择算法提高了16.93%.

关键词: 异常流量检测, 集成特征选择, 投票机制

Abstract:

With the continuous development of Internet technology, network security is garnering increasing attention. Network anomalous traffic detection can provide an effective guarantee for blocking network attacks. However, to accurately detect anomalous traffic in a network, analyzing large volumes of data is usually required. Analyzing this data not only consumes substantial computational resources and reduces real-time detection capability, but it may also reduce the overall accuracy of detection. To solve these problems, we propose a network anomaly traffic detection method based on ensemble feature selection. Specifically, we use five different feature selection algorithms to design a voting mechanism for selecting feature subsets. Three different machine learning algorithms (Naive Bayesian, Decision Tree, XGBoost) are used to evaluate the feature selection algorithm, and the best algorithm is selected to detect abnormal network traffic. The experimental results show that the runtime of the proposed method is 84.38% less than the original data set on the optimal feature subset selected by the proposed approach, and the average accuracy is 16.93% higher than that of the single feature selection algorithm.

Key words: anomaly traffic detection, ensemble feature selection, voting mechanism

中图分类号:

TP393

黄奇文, 李丽颖, 沈富可, 魏同权. 基于集成特征选择的网络异常流量检测[J]. 华东师范大学学报（自然科学版）, 2021, 2021(6): 100-111.

Qiwen HUANG, Liying LI, Fuke SHEN, Tongquan WEI. Network anomaly traffic detection based on ensemble feature selection[J]. Journal of East China Normal University(Natural Science), 2021, 2021(6): 100-111.

图/表 10

图1

图2

图3

图4

表1

表2

表3

图5

图6

图7

参考文献 19

1	CISCO. Cisco visual networking index: Forecast and methodology, 2016–2021 [EB/OL]. (2017-06-15)[2020-06-24]. http://www.cisco.com/c/en/us/solutions/collateral/service-provider/visualnetworking-indexvni/complete-white-paper-c11-481360.pdf.
2	KYLE Y. Read Dyn’s statement on the 10/21/2016 DNS DDoS attack [EB/OL]. (2016-10-21)[2020-06-24]. https://dyn.com/blog/dyn-statement-on-10212016-ddos-attack.html.
3	PATIL N V, KRISHNA C R, KUMAR K, et al. E-Had: A distributed and collaborative detection framework for early detection of DDoS attacks [J/OL]. Journal of King Saud University-Computer and Information Sciences, 2019. https://doi.org/10.1016/j.jksuci.2019.06.016.
4	PACHECO F, EXPOSITO E, GINESTE M, et al. Towards the deployment of machine learning solutions in network traffic classification: A systematic survey. IEEE Communications Surveys and Tutorials, 2018, 21(4), 1988- 2014.
5	INTERNET ASSIGNED NUMBERS AUTHORITY. Protocol Assignments [EB/OL]. (2011-12-17)[2020-06-24]. https://www.iana.org/protocols.
6	CALLADO A, KELNER J, SADOK D, et al. Better network traffic identification through the independent combination of techniques. Journal of Network and Computer Applications, 2010, 33 (4): 433- 446. doi: 10.1016/j.jnca.2010.02.002
7	BELAVAGI M C, MUNIYAL B. Performance evaluation of supervised machine learning algorithms for intrusion detection. Procedia Computer Science, 2016, 89, 117- 123. doi: 10.1016/j.procs.2016.06.016
8	OSANAIYE O, CAI H B, CHOO K K R, et al. Ensemble-based multi-filter feature selection method for DDoS detection in cloud computing [J]. EURASIP Journal on Wireless Communications and Networking, 2016: Article number 130. DOI: 10.1186/s13638-016-0623-3
9	HOQUE N, SINGH M, BHATTACHARYYA D K. EFS-MI: An ensemble feature selection method for classification. Complex & Intelligent Systems, 2018(4): 105-118.,
10	SINGH K J, DE T. Efficient classification of DDoS attacks using an ensemble feature selection algorithm. Journal of Intelligent Systems, 2017, 29 (1): 71- 83. doi: 10.1515/jisys-2017-0472
11	KE G L, MENG Q, FINLEY T, et al. LightGBM: A highly efficient gradient boosting decision tree [C]// Advances in Neural Information Processing Systems (NIPS 2017). 2017: 3146-3154.
12	CHEN T Q, GUESTRIN C. XGBoost: A scalable tree boosting system[C] // Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016: 785–794.
13	BOLÓN-CANEDO V, ALONSO-BETANZOS A. Ensembles for feature selection: A review and future trends. Information Fusion, 2019, 52, 1- 12. doi: 10.1016/j.inffus.2018.11.008
14	HO T K. Random decision forests [C]// Proceedings of 3rd International Conference on Document Analysis and Recognition. IEEE,1995: 278-282.
15	BREIMAN L. Random forest. Machine Learning, 2001, 45, 5- 32. doi: 10.1023/A:1010933404324
16	李航. 统计学习方法[M]. 2版. 北京: 清华大学出版社, 2019: 59-60.
17	CHEN T Q. Story and lessons behind the evolution of XGBoost [EB/OL]. (2016-03-10)[2020-06-24]. https://homes.cs.washington.edu/~tqchen/2016/03/10/story-and-lessons-behind-the-evolution-of-xgboost.html.
18	SHARAFALDIN I, LASHKARI A H, GHORBANI A A. Toward generating a new intrusion detection dataset and intrusion traffic characterization [C]// Proceedings of the 4th International Conference on Information Systems Security and Privacy - ICISSP. 2018: 108-116.
19	LASHKARI A H, DRAPER-GIL G, MAMUN M S I, et al. Characterization of tor traffic using time based features [C]// Proceedings of the 3rd International Conference on Information Systems Security and Privacy - ICISSP. 2017: 253-262.

CIC-IDS-2018特征
Flow Duration	Total Forward Packets	Total Backward Packets	Total Length Forward Packets
Total Length Backward Packets	Forward Packet Length Max	Forward Packet Length Min	Forward Packet Length Mean
Forward Packet Length Std	Backward Packet Length Max	Backward Packet Length Min	Backward Packet Length Mean
Backward Packet Length Std	Flow Bytes/s	Flow Packets/s	Flow IAT Mean
Flow IAT Std	Flow IAT Max	Flow IAT Min	Forward IAT Total
Forward IAT Mean	Forward IAT Std	Forward IAT Max	Forward IAT Min
Backward IAT Total	Backward IAT Mean	Backward IAT Std	Backward IAT Max
Backward IAT Min	Forward PSH Flags	Backward PSH Flags	Forward URG Flags
Backward URG Flags	Forward Header Length	Backward Header Length	Forward Packets/s
Backward Packets/s	Packet Length Min	Packet Length Max	Packet Length Mean
Packet Length Std	Packet Length Var	FIN Flag Count	SYN Flag Count
RST Flag Count	PSH Flag Count	ACK Flag Count	URG Flag Count
CWE Flag Count	ECE Flag Count	Down/Up Ratio	Packet Size Avg
Forward Seg Size Avg	Backward Seg Size Avg	Forward Byts/b Avg	Forward Packets/b Avg
Forward Blk Rate Avg	Backward Byts/b Avg	Backward Packets/b Avg	Backward Blk Rate Avg
Subflow Forward Packets	Subflow Forward Byts	Subflow Backward Packets	Subflow Backward Byts
Init Forward Win Byts	Init Backward Win Byts	Forward Act Data Packets	Forward Seg Size Min
Active Mean	Active Std	Active Max	Active Min
Idle Mean	Idle Std	Idle Max	Idle Min

流量类型	数据量/条
Benign	778 770
DDoS attack-HOIC	686 012
DDoS attacks-LOIC-HTTP	576 191
DoS attacks-Hulk	461 912
Bot	286 191
FTP-BruteForce	193 354
SSH-Bruteforce	187 589
Infilteration	160 639
DoS attacks-SlowHTTPTest	139 890

特征名称	特征选择方法					方法数	总计	特征子集
特征名称	相关系数	卡方	互信息	随机森林	LGBM	方法数	总计	特征子集
Pkt Len Std	1	1	1	1	1	5	5	F₅, F₄, F₃
Fwd Seg Size Min	1	1	1	1	1	5
Fwd Pkts/s	1	1	1	1	1	5
Flow Pkts/s	1	1	1	1	1	5
Bwd Pkts/s	1	1	1	1	1	5
Init Fwd Win Byts	0	1	1	1	1	4	6	F₄, F₃
Init Bwd Win Byts	1	1	1	1	0	4
Fwd IAT Mean	1	0	1	1	1	4
Flow IAT Mean	1	0	1	1	1	4
Bwd Pkt Len Std	1	1	0	1	1	4
Bwd Pkt Len Mean	1	1	1	1	0	4
Pkt Size Avg	1	1	1	0	0	3	9	F₃
Pkt Len Var	1	0	1	0	1	3
Pkt Len Mean	1	1	1	0	0	3
Fwd Seg Size Avg	1	0	1	1	0	3
Fwd Pkt Len Std	1	1	0	0	1	3
Fwd Pkt Len Mean	1	0	1	1	0	3
Fwd Pkt Len Max	1	0	1	1	0	3
Fwd Header Len	0	0	1	1	1	3
Bwd Seg Size Avg	1	1	1	0	0	3

[1]	胡亚琴, 唐明. 层间耦合关联对多层网络交通传输的影响[J]. 华东师范大学学报（自然科学版）, 2021, 2021(3): 105-113.
[2]	韩定定, 柳康, 唐明. 无标度网络局部信息的动态路由算法[J]. 华东师范大学学报(自然科学版), 2019, 2019(2): 69-76,96.
[3]	叶仕通, 万智萍, 柯建波, 刘少江, 倪伟传. 基于协作频谱感知和干扰约束的认知异构网络[J]. 华东师范大学学报(自然科学版), 2017, 2017(6): 76-84.
[4]	王荣荣, 薛旻辉, 李祥学, 钱海峰. 基于位置社交网络的高效定位算法[J]. 华东师范大学学报(自然科学版), 2016, 2016(2): 62-72.
[5]	刘欢, 吴旻瑜, 陈建香, 刘畅, 卢蓓蓉. 基于WCAG 2.0的高校门户网站可访问性研究[J]. 华东师范大学学报(自然科学版), 2015, 2015(6): 143-151.
[6]	江家宝,郑尚志. 基于QPSO算法的OSPF多约束路由研究[J]. 华东师范大学学报(自然科学版), 2015, 2015(3): 91-97.
[7]	李钟翔, 陈蕾. 一种基于人工鱼群算法的K覆盖WiFi热点安置方案[J]. 华东师范大学学报(自然科学版), 2015, 2015(1): 151-160.
[8]	张宇, 张延松, 张兵, 陈红, 王珊. Co-OLAP: CPU&GPU混合平台上面向星形模型基准的协同OLAP[J]. 华东师范大学学报(自然科学版), 2014, 2014(5): 240-251.
[9]	陈蕾;方升;王能. 道路异构无线接入网的网络选择机制及模拟器[J]. 华东师范大学学报(自然科学版), 2011, 2011(3): 111-122.
[10]	姜雪;郑骏;王平. 高white washing防御力的P2P全局信任模型[J]. 华东师范大学学报(自然科学版), 2010, 2010(1): 111-117.
[11]	黄素珊;钱海峰;周渊. 基于双线性配对的安全认证机制[J]. 华东师范大学学报(自然科学版), 2010, 2010(1): 118-126.
[12]	陈蕾;王能 . 城市道路WOBAN中基于车流量的ONU位置评价机制 [J]. 华东师范大学学报(自然科学版), 2009, 2009(4): 98-106.
[13]	余以胜 . Web环境下对可信网络连接的扩展研究 [J]. 华东师范大学学报(自然科学版), 2009, 2009(4): 137-140.
[14]	武荷岚;胡炳元;陈必多. 弹性堆叠系统中控制数据同步的方法[J]. 华东师范大学学报(自然科学版), 2008, 2008(2): 122-130.
[15]	杨静;贺樑;顾君忠 . 非对称P2P模型及资源定位算法研究 [J]. 华东师范大学学报(自然科学版), 2008, 2008(1): 75-82.