金融科技软件自动化测试用例的冗余评价和削减方法

doi:10.3969/j.issn.1000-5641.2022.04.005

摘要/Abstract

摘要：

随着金融科技软件的开发迭代, 软件的复杂度日益提升, 这将会导致测试套件体量逐渐增大, 并出现测试冗余现象. 为了有效地对测试冗余因素进行量化和消解, 提出了一种最佳覆盖项测试冗余评价指标MVI (Most Valuable Item) , 以及一种基于MVI指标的测试用例削减算法MVIR (Most Valuable Item Reduction). 在实际金融科技软件中的实验结果表明, MVIR能够在测试性能损失小于9.20%的前提下, 实现大于89.88%的测试用例削减比例, MVI指标能够有效反映测试套件中的冗余因素大小.

关键词: 金融科技, 软件测试, 测试冗余, 测试用例生成, 测试用例削减

Abstract:

With the development and iteration of financial technology(FinTech) software programs, the size of test suites will gradually increase, which may introduce inherent redundancy. In order to effectively quantify test redundancy, a test redundancy evaluation metric called MVI (Most Valuable Item) is proposed in this study. To verify the validity of the MVI metric, the MVIR (Most Valuable Item Reduction) test case reduction algorithm is proposed. Experimental results show that the MVIR can achieve a test case reduction ratio of more than 89.88% assuming the test performance loss is less than 9.20%, this demonstrates that the MVI metric is valid.

Key words: financial technology, software testing, test redundancy, test generation, test minimization

中图分类号:

TP311.5

龚鑫, 徐立华, 窦亮, 赵瑞祥. 金融科技软件自动化测试用例的冗余评价和削减方法[J]. 华东师范大学学报（自然科学版）, 2022, 2022(4): 43-55.

Xin GONG, Lihua XU, Liang DOU, Ruixiang ZHAO. Redundancy measurement and reduction of automated tests in financial technology[J]. Journal of East China Normal University(Natural Science), 2022, 2022(4): 43-55.

图/表 15

图1

图2

表1

表2

图3

表3

表4

图4

图5

图6

表5

图7

图8

表6

表7

参考文献 14

1	POLLARI I, RUDDENKLAU A. Pulse of FinTech H1’20 [EB/OL]. (2020-09-01)[2020-12-08]. https://home.kpmg/xx/en/home/insights/2020/09/pulse-of-fintech-h1-20-fintech-segments.html.
2	WANG Q S, GU L T, XUE M H, et al. FACTS: Automated black-box testing of FinTech systems [C]// Proceedings of the 26th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2018: 839-844.
3	JIN T C, WANG Q S, XU L H, et al. FinExpert: Domain-specific test generation for FinTech systems [C]// Proceedings of the 27th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2019: 853-862.
4	KOOCHAKZADEH N, GAROUSI V, MAURER F. Test redundancy measurement based on coverage information: Evaluations and lessons learned [C]// Proceedings of the 2009 International Conference on Software Testing Verification and Validation. 2009: 220-229.
5	MARIJAN D, LIAAEN M. Practical selective regression testing with effective redundancy in interleaved tests [C]// Proceedings of the 2018 ACM/IEEE 40th International Conference on Software Engineering in Practice Track. 2018: 153-162.
6	BAVOTA G, QUSEF A, OLIVETO R, et al. An empirical analysis of the distribution of unit test smells and their impact on software maintenance [C]// Proceedings of the 28th IEEE International Conference on Software Maintenance. 2012: 56-65.
7	HSU H Y, ORSO A. MINTS: A general framework and tool for supporting test-suite minimization [C]// Proceedings of the 31st International Conference on Software Engineering. 2009: 419-429.
8	TUFANO M, PALOMBA F, BAVOTA G, et al. An empirical investigation into the nature of test smells [C]// Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering. 2016: 4-15.
9	GOTLIEB A, MARIJAN D. FLOWER: Optimal test suite reduction as a network maximum flow [C]// Proceedings of the 2014 International Symposium on Software Testing and Analysis. 2014: 171-180.
10	MASRI W, PODGURSKI A, LEON D. An empirical study of test case filtering techniques based on exercising information flows. IEEE Transactions on Software Engineering, 2007, 33 (7): 454- 477.
11	ZHANG L M, MARINOV D, ZHANG L, et al. An empirical study of JUnit test-suite reduction [C]// Proceedings of the 2011 IEEE 22nd International Symposium on Software Reliability Engineering. 2011: 170-179.
12	BALLER H, LITY S, LOCHAU M, et al. Multi-objective test suite optimization for incremental product family testing [C]//Proceedings of the 2014 IEEE International Conference on Software Testing. 2014: 303-312.
13	ANDREWS J H, BRIAND L C, LABICHE Y, et al. Using mutation analysis for assessing and comparing testing coverage criteria. IEEE Transactions on Software Engineering, 2006, 32 (8): 608- 624.
14	COLES H, LAURENT T, HENARD C, et al. PIT: A practical mutation testing tool for Java (Demo) [C]// Proceedings of the 25th International Symposium on Software Testing and Analysis. 2016: 449–452.

测试套件	覆盖项 1	覆盖项 2	覆盖项 3	覆盖项 4	覆盖项 5	覆盖项 6	测试用例冗余程度
测试用例 1	1	0	#	#	#	1	不冗余
测试用例 2	1	#	1	0	#	1	不冗余
测试用例 3	1	#	1	#	0	1	不冗余
覆盖项冗余程度	冗余	不冗余	冗余	不冗余	不冗余	冗余

测试套件	覆盖项 1	覆盖项 2	覆盖项 3	覆盖项 4	覆盖项 5	覆盖项 6	测试用例冗余程度
测试用例 1	1	0	#	#	#	1	不冗余
测试用例 2	1	#	1	0	#	1	不冗余
测试用例 3	1	#	1	#	1	1	冗余
测试用例 4	1	#	1	#	1	1	冗余
覆盖项冗余程度	冗余	不冗余	冗余	不冗余	冗余	冗余

测试接口	测试用例数/个	分支覆盖率/%
api1	10	48.45
api2	17	59.07
api3	22	59.29

测试接口	人工方法/个	FinExpert/个	FACTS/个
api1	10	1000	820
api2	22	1000	1804
api3	17	1000	1319

测试接口	人工方法	FinExpert	FACTS
api1	2.90	708.81	208.09
api2	8.00	738.16	492.18
api3	5.06	463.86	308.18
平均值	5.32	636.94	336.15