Redundancy measurement and reduction of automated tests in financial technology

doi:10.3969/j.issn.1000-5641.2022.04.005

Abstract

Abstract:

With the development and iteration of financial technology(FinTech) software programs, the size of test suites will gradually increase, which may introduce inherent redundancy. In order to effectively quantify test redundancy, a test redundancy evaluation metric called MVI (Most Valuable Item) is proposed in this study. To verify the validity of the MVI metric, the MVIR (Most Valuable Item Reduction) test case reduction algorithm is proposed. Experimental results show that the MVIR can achieve a test case reduction ratio of more than 89.88% assuming the test performance loss is less than 9.20%, this demonstrates that the MVI metric is valid.

Key words: financial technology, software testing, test redundancy, test generation, test minimization

CLC Number:

TP311.5

Xin GONG, Lihua XU, Liang DOU, Ruixiang ZHAO. Redundancy measurement and reduction of automated tests in financial technology[J]. Journal of East China Normal University(Natural Science), 2022, 2022(4): 43-55.

Figures/Tables 15

Fig.1

Fig.2

Table 1

Table 2

Fig.3

Table 3

Table 4

Fig.4

Fig.5

Fig.6

Table 5

Fig.7

Fig.8

Table 6

Table 7

References 14

1	POLLARI I, RUDDENKLAU A. Pulse of FinTech H1’20 [EB/OL]. (2020-09-01)[2020-12-08]. https://home.kpmg/xx/en/home/insights/2020/09/pulse-of-fintech-h1-20-fintech-segments.html.
2	WANG Q S, GU L T, XUE M H, et al. FACTS: Automated black-box testing of FinTech systems [C]// Proceedings of the 26th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2018: 839-844.
3	JIN T C, WANG Q S, XU L H, et al. FinExpert: Domain-specific test generation for FinTech systems [C]// Proceedings of the 27th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2019: 853-862.
4	KOOCHAKZADEH N, GAROUSI V, MAURER F. Test redundancy measurement based on coverage information: Evaluations and lessons learned [C]// Proceedings of the 2009 International Conference on Software Testing Verification and Validation. 2009: 220-229.
5	MARIJAN D, LIAAEN M. Practical selective regression testing with effective redundancy in interleaved tests [C]// Proceedings of the 2018 ACM/IEEE 40th International Conference on Software Engineering in Practice Track. 2018: 153-162.
6	BAVOTA G, QUSEF A, OLIVETO R, et al. An empirical analysis of the distribution of unit test smells and their impact on software maintenance [C]// Proceedings of the 28th IEEE International Conference on Software Maintenance. 2012: 56-65.
7	HSU H Y, ORSO A. MINTS: A general framework and tool for supporting test-suite minimization [C]// Proceedings of the 31st International Conference on Software Engineering. 2009: 419-429.
8	TUFANO M, PALOMBA F, BAVOTA G, et al. An empirical investigation into the nature of test smells [C]// Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering. 2016: 4-15.
9	GOTLIEB A, MARIJAN D. FLOWER: Optimal test suite reduction as a network maximum flow [C]// Proceedings of the 2014 International Symposium on Software Testing and Analysis. 2014: 171-180.
10	MASRI W, PODGURSKI A, LEON D. An empirical study of test case filtering techniques based on exercising information flows. IEEE Transactions on Software Engineering, 2007, 33 (7): 454- 477.
11	ZHANG L M, MARINOV D, ZHANG L, et al. An empirical study of JUnit test-suite reduction [C]// Proceedings of the 2011 IEEE 22nd International Symposium on Software Reliability Engineering. 2011: 170-179.
12	BALLER H, LITY S, LOCHAU M, et al. Multi-objective test suite optimization for incremental product family testing [C]//Proceedings of the 2014 IEEE International Conference on Software Testing. 2014: 303-312.
13	ANDREWS J H, BRIAND L C, LABICHE Y, et al. Using mutation analysis for assessing and comparing testing coverage criteria. IEEE Transactions on Software Engineering, 2006, 32 (8): 608- 624.
14	COLES H, LAURENT T, HENARD C, et al. PIT: A practical mutation testing tool for Java (Demo) [C]// Proceedings of the 25th International Symposium on Software Testing and Analysis. 2016: 449–452.

测试套件	覆盖项 1	覆盖项 2	覆盖项 3	覆盖项 4	覆盖项 5	覆盖项 6	测试用例冗余程度
测试用例 1	1	0	#	#	#	1	不冗余
测试用例 2	1	#	1	0	#	1	不冗余
测试用例 3	1	#	1	#	0	1	不冗余
覆盖项冗余程度	冗余	不冗余	冗余	不冗余	不冗余	冗余

测试套件	覆盖项 1	覆盖项 2	覆盖项 3	覆盖项 4	覆盖项 5	覆盖项 6	测试用例冗余程度
测试用例 1	1	0	#	#	#	1	不冗余
测试用例 2	1	#	1	0	#	1	不冗余
测试用例 3	1	#	1	#	1	1	冗余
测试用例 4	1	#	1	#	1	1	冗余
覆盖项冗余程度	冗余	不冗余	冗余	不冗余	冗余	冗余

测试接口	测试用例数/个	分支覆盖率/%
api1	10	48.45
api2	17	59.07
api3	22	59.29

测试接口	人工方法/个	FinExpert/个	FACTS/个
api1	10	1000	820
api2	22	1000	1804
api3	17	1000	1319

测试接口	人工方法	FinExpert	FACTS
api1	2.90	708.81	208.09
api2	8.00	738.16	492.18
api3	5.06	463.86	308.18
平均值	5.32	636.94	336.15