Network anomaly traffic detection based on ensemble feature selection

doi:10.3969/j.issn.1000-5641.2021.06.011

Abstract

Abstract:

With the continuous development of Internet technology, network security is garnering increasing attention. Network anomalous traffic detection can provide an effective guarantee for blocking network attacks. However, to accurately detect anomalous traffic in a network, analyzing large volumes of data is usually required. Analyzing this data not only consumes substantial computational resources and reduces real-time detection capability, but it may also reduce the overall accuracy of detection. To solve these problems, we propose a network anomaly traffic detection method based on ensemble feature selection. Specifically, we use five different feature selection algorithms to design a voting mechanism for selecting feature subsets. Three different machine learning algorithms (Naive Bayesian, Decision Tree, XGBoost) are used to evaluate the feature selection algorithm, and the best algorithm is selected to detect abnormal network traffic. The experimental results show that the runtime of the proposed method is 84.38% less than the original data set on the optimal feature subset selected by the proposed approach, and the average accuracy is 16.93% higher than that of the single feature selection algorithm.

Key words: anomaly traffic detection, ensemble feature selection, voting mechanism

CLC Number:

TP393

Qiwen HUANG, Liying LI, Fuke SHEN, Tongquan WEI. Network anomaly traffic detection based on ensemble feature selection[J]. Journal of East China Normal University(Natural Science), 2021, 2021(6): 100-111.

Figures/Tables 10

Fig.1

Fig.2

Fig.3

Fig.4

Table 1

Table 2

Table 3

Fig.5

Fig.6

Fig.7

References 19

1	CISCO. Cisco visual networking index: Forecast and methodology, 2016–2021 [EB/OL]. (2017-06-15)[2020-06-24]. http://www.cisco.com/c/en/us/solutions/collateral/service-provider/visualnetworking-indexvni/complete-white-paper-c11-481360.pdf.
2	KYLE Y. Read Dyn’s statement on the 10/21/2016 DNS DDoS attack [EB/OL]. (2016-10-21)[2020-06-24]. https://dyn.com/blog/dyn-statement-on-10212016-ddos-attack.html.
3	PATIL N V, KRISHNA C R, KUMAR K, et al. E-Had: A distributed and collaborative detection framework for early detection of DDoS attacks [J/OL]. Journal of King Saud University-Computer and Information Sciences, 2019. https://doi.org/10.1016/j.jksuci.2019.06.016.
4	PACHECO F, EXPOSITO E, GINESTE M, et al. Towards the deployment of machine learning solutions in network traffic classification: A systematic survey. IEEE Communications Surveys and Tutorials, 2018, 21(4), 1988- 2014.
5	INTERNET ASSIGNED NUMBERS AUTHORITY. Protocol Assignments [EB/OL]. (2011-12-17)[2020-06-24]. https://www.iana.org/protocols.
6	CALLADO A, KELNER J, SADOK D, et al. Better network traffic identification through the independent combination of techniques. Journal of Network and Computer Applications, 2010, 33 (4): 433- 446. doi: 10.1016/j.jnca.2010.02.002
7	BELAVAGI M C, MUNIYAL B. Performance evaluation of supervised machine learning algorithms for intrusion detection. Procedia Computer Science, 2016, 89, 117- 123. doi: 10.1016/j.procs.2016.06.016
8	OSANAIYE O, CAI H B, CHOO K K R, et al. Ensemble-based multi-filter feature selection method for DDoS detection in cloud computing [J]. EURASIP Journal on Wireless Communications and Networking, 2016: Article number 130. DOI: 10.1186/s13638-016-0623-3
9	HOQUE N, SINGH M, BHATTACHARYYA D K. EFS-MI: An ensemble feature selection method for classification. Complex & Intelligent Systems, 2018(4): 105-118.,
10	SINGH K J, DE T. Efficient classification of DDoS attacks using an ensemble feature selection algorithm. Journal of Intelligent Systems, 2017, 29 (1): 71- 83. doi: 10.1515/jisys-2017-0472
11	KE G L, MENG Q, FINLEY T, et al. LightGBM: A highly efficient gradient boosting decision tree [C]// Advances in Neural Information Processing Systems (NIPS 2017). 2017: 3146-3154.
12	CHEN T Q, GUESTRIN C. XGBoost: A scalable tree boosting system[C] // Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016: 785–794.
13	BOLÓN-CANEDO V, ALONSO-BETANZOS A. Ensembles for feature selection: A review and future trends. Information Fusion, 2019, 52, 1- 12. doi: 10.1016/j.inffus.2018.11.008
14	HO T K. Random decision forests [C]// Proceedings of 3rd International Conference on Document Analysis and Recognition. IEEE,1995: 278-282.
15	BREIMAN L. Random forest. Machine Learning, 2001, 45, 5- 32. doi: 10.1023/A:1010933404324
16	李航. 统计学习方法[M]. 2版. 北京: 清华大学出版社, 2019: 59-60.
17	CHEN T Q. Story and lessons behind the evolution of XGBoost [EB/OL]. (2016-03-10)[2020-06-24]. https://homes.cs.washington.edu/~tqchen/2016/03/10/story-and-lessons-behind-the-evolution-of-xgboost.html.
18	SHARAFALDIN I, LASHKARI A H, GHORBANI A A. Toward generating a new intrusion detection dataset and intrusion traffic characterization [C]// Proceedings of the 4th International Conference on Information Systems Security and Privacy - ICISSP. 2018: 108-116.
19	LASHKARI A H, DRAPER-GIL G, MAMUN M S I, et al. Characterization of tor traffic using time based features [C]// Proceedings of the 3rd International Conference on Information Systems Security and Privacy - ICISSP. 2017: 253-262.

CIC-IDS-2018特征
Flow Duration	Total Forward Packets	Total Backward Packets	Total Length Forward Packets
Total Length Backward Packets	Forward Packet Length Max	Forward Packet Length Min	Forward Packet Length Mean
Forward Packet Length Std	Backward Packet Length Max	Backward Packet Length Min	Backward Packet Length Mean
Backward Packet Length Std	Flow Bytes/s	Flow Packets/s	Flow IAT Mean
Flow IAT Std	Flow IAT Max	Flow IAT Min	Forward IAT Total
Forward IAT Mean	Forward IAT Std	Forward IAT Max	Forward IAT Min
Backward IAT Total	Backward IAT Mean	Backward IAT Std	Backward IAT Max
Backward IAT Min	Forward PSH Flags	Backward PSH Flags	Forward URG Flags
Backward URG Flags	Forward Header Length	Backward Header Length	Forward Packets/s
Backward Packets/s	Packet Length Min	Packet Length Max	Packet Length Mean
Packet Length Std	Packet Length Var	FIN Flag Count	SYN Flag Count
RST Flag Count	PSH Flag Count	ACK Flag Count	URG Flag Count
CWE Flag Count	ECE Flag Count	Down/Up Ratio	Packet Size Avg
Forward Seg Size Avg	Backward Seg Size Avg	Forward Byts/b Avg	Forward Packets/b Avg
Forward Blk Rate Avg	Backward Byts/b Avg	Backward Packets/b Avg	Backward Blk Rate Avg
Subflow Forward Packets	Subflow Forward Byts	Subflow Backward Packets	Subflow Backward Byts
Init Forward Win Byts	Init Backward Win Byts	Forward Act Data Packets	Forward Seg Size Min
Active Mean	Active Std	Active Max	Active Min
Idle Mean	Idle Std	Idle Max	Idle Min

流量类型	数据量/条
Benign	778 770
DDoS attack-HOIC	686 012
DDoS attacks-LOIC-HTTP	576 191
DoS attacks-Hulk	461 912
Bot	286 191
FTP-BruteForce	193 354
SSH-Bruteforce	187 589
Infilteration	160 639
DoS attacks-SlowHTTPTest	139 890

特征名称	特征选择方法					方法数	总计	特征子集
特征名称	相关系数	卡方	互信息	随机森林	LGBM	方法数	总计	特征子集
Pkt Len Std	1	1	1	1	1	5	5	F₅, F₄, F₃
Fwd Seg Size Min	1	1	1	1	1	5
Fwd Pkts/s	1	1	1	1	1	5
Flow Pkts/s	1	1	1	1	1	5
Bwd Pkts/s	1	1	1	1	1	5
Init Fwd Win Byts	0	1	1	1	1	4	6	F₄, F₃
Init Bwd Win Byts	1	1	1	1	0	4
Fwd IAT Mean	1	0	1	1	1	4
Flow IAT Mean	1	0	1	1	1	4
Bwd Pkt Len Std	1	1	0	1	1	4
Bwd Pkt Len Mean	1	1	1	1	0	4
Pkt Size Avg	1	1	1	0	0	3	9	F₃
Pkt Len Var	1	0	1	0	1	3
Pkt Len Mean	1	1	1	0	0	3
Fwd Seg Size Avg	1	0	1	1	0	3
Fwd Pkt Len Std	1	1	0	0	1	3
Fwd Pkt Len Mean	1	0	1	1	0	3
Fwd Pkt Len Max	1	0	1	1	0	3
Fwd Header Len	0	0	1	1	1	3
Bwd Seg Size Avg	1	1	1	0	0	3

[1]	Zhenglei ZHOU, Jun CHEN, Juntao PAN, Peisen YUAN. Parallel deep-forest-based abnormal traffic detection for power distribution communication networks [J]. Journal of East China Normal University(Natural Science), 2023, 2023(5): 122-134.
[2]	Zhishang DUAN, Yi RAN, Duliang LYU, Jie QI, Jiachen ZHONG, Peisen YUAN. Identifying electricity theft based on residual network and depthwise separable convolution enhanced self attention [J]. Journal of East China Normal University(Natural Science), 2023, 2023(5): 193-204.
[3]	Yingqi ZENG, Min TANG. Coupled propagation dynamics of different time evolution scales on double-layer networks [J]. Journal of East China Normal University(Natural Science), 2022, 2022(2): 45-54.
[4]	Nuo WANG, Liying LI, Dongwei QIAN, Tongquan WEI. Research on an Edge-Cloud collaborative acceleration mechanism of deep model based on network compression and partitioning [J]. Journal of East China Normal University(Natural Science), 2021, 2021(6): 112-123.
[5]	Yuting HUANGFU, Liying LI, Haizhou WANG, Fuke SHEN, Tongquan WEI. Enabling self-attention based multi-feature anomaly detection and classification of network traffic [J]. Journal of East China Normal University(Natural Science), 2021, 2021(6): 161-173.
[6]	Yaqin HU, Ming TANG. The impact of coupling patterns on transport in multilayer networks [J]. Journal of East China Normal University(Natural Science), 2021, 2021(3): 105-113.
[7]	HAN Ding-ding, LIU Kang, TANG Ming. Dynamic routing algorithm based on local information in a free-scale network [J]. Journal of East China Normal University(Natural Sc, 2019, 2019(2): 69-76,96.
[8]	YE Shi-tong, WAN Zhi-ping, KE Jian-bo, LIU Shao-jiang, NI Wei-chuan. Cognitive heterogeneous network based on cooperative spectrum sensing and interference constraints [J]. Journal of East China Normal University(Natural Sc, 2017, 2017(6): 76-84.
[9]	WANG Rong-Rong, XUE Min-Hui, LI Xiang-Xue, QIAN Hai-Feng. An effective localization attack in locationbased social network [J]. Journal of East China Normal University(Natural Sc, 2016, 2016(2): 62-72.
[10]	LIU Huan, WU Min-Yu, CHEN Jian-Xiang, LIU Chang, LU Bei-Rong. Accessibility evaluation on college portal websitesbased on WCAG 2.0 [J]. Journal of East China Normal University(Natural Sc, 2015, 2015(6): 143-151.
[11]	JIANG Jia-bao,ZHENG Shang-zhi. Research on OSPF multi constraint routing based on QPSO algorithm [J]. Journal of East China Normal University(Natural Sc, 2015, 2015(3): 91-97.
[12]	LI Zhong-Xiang, CHEN Lei. K coverage of WiFi signal node deployment based on AFSA [J]. Journal of East China Normal University(Natural Sc, 2015, 2015(1): 151-160.
[13]	ZHANG Yu, ZHANG Yan-Song, ZHANG Bing, CHEN Hong, WANG Shan. Co-OLAP: Research on cooperated OLAP with star schema benchmark on hybrid CPU&GPU platform [J]. Journal of East China Normal University(Natural Sc, 2014, 2014(5): 240-251.
[14]	CHEN Lei;FANG Sheng;WANG Neng. Network selection scheme and a simulator for future urban road heterogeneous wireless access networks [J]. Journal of East China Normal University(Natural Sc, 2011, 2011(3): 111-122.
[15]	JIANG Xue;ZHENG Jun;WANG Ping. Global trust model with high white washing resistance (Chinese) [J]. Journal of East China Normal University(Natural Sc, 2010, 2010(1): 111-117.