Test Case Selection and Prioritization for Regression Testing using Fault Severity

Regression testing is a significant but a very expensive testing process .Test case prioritization is a technique to schedule and execute the test cases in such an order that results in increasing their ability to meet some performance goal. One of the main goal is to increase the rate of fault detection –i.e. to detect the faults as early as possible during the testing process. Test case prioritization is used to minimize the expenses of regression testing. This paper proposes a technique to select and prioritize the test cases and results in improving the rate of fault detection.


INTRODUCTION
Software testing is a process to ensure that software is working properly or as required and is not doing anything unintended. Software testing is an important and costly phase of SDLC. It consumes over 50% cost of the entire software. Regression testing is performed to validate the modified software to ensure that the changes are correct and do not introduce additional errors . Regression test suite is typically large and it is very expensive to execute all the test suits during regression testing. That's why, test cases are prioritized for execution in order to increase their ability to meet some performance goals, e.g., to increase the rate of fault detection or to achieve maximum code coverage etc. Gregg Rothermal [1] has proven that prioritizing and scheduling text cases are one of the most critical task during the software testing process. He stated that the test case prioritization methods and process are required because (a) the regression testing phase consumes a lot of time and cost to run , (b) there is not enough time and resources to run the entire test suite, (c) there is a need to decide which test case to run first. The next section of this article describe the Problem Statement, then section 3 mentioned different Methodologies related to regression testing , next section 4 describe the Related Work and section 5 have the Proposed Work and at last we concluded with the summary.

PROBLEM STATEMENT
Rothermel at el. [1,7] defines the test case prioritization problem as follows : Given: T, a test suite; PT, the set of permutations of T; f, a function from PT to the real numbers. Problem: Find T' belongs to PT such that (for all T") (T" belongs to PT) ( Here, PT represents the set of all possible prioritizations (orderings) of T and f is a function that, applied to any such ordering, yields an award value for that ordering [1,2].

METHODLOGIES
This section provides the methodologies that are related to regression testing. There are four methodologies that are available for regression.testing. These methods are [2,5 ]:

Retest all
In this technique the test cases that are no longer apply to modified version of program are discarded and all the remaining set of test cases are used to test the modified program. Retest all technique takes time and effort as all test cases are used to test the program again, so may be quite expensive.

Regression Test Selection
This technique is much better than Retest All as it executes a subset of the test suite.It uses information about program, modified program, test cases to select a subset of test cases for testing.

Cost Effective-based techniques
This technique uses information about program and test suite to remove the test cases, which have become redundant with time, as new functionality is added. It is different from Regression test selection as former does not permanently remove test cases but selects those that are required. Advantage of this technique is that it reduces cost of validating, executing, managing test suites over future releases of software, but the downside of this is that it might reduce the fault detection capability with the reduction of test suite size.

Test Case Prioritization
In this technique each test cases are assigned a priority. Priority is set according to some criterion like faster code coverage or rate of faults detect etc, and test cases with highest priority are scheduled first .

RELATED WORK
Lots of research has been done and techniques are developed on test case prioritization for regression testing.
Empirical study on test case prioritization techniques reported in [1,2] sorts test cases such that the test cases with highest priority, according to some criterion, are executed first. Test case prioritization can address a wide variety of objectives. For example, in [1] Rothermel prioritized the test cases based on coverage alone. Testers might wish to schedule, test cases in order to achieve code coverage at the fastest rate possible during initial phase of regression testing so as to reach 100% coverage at the earliest or to ensure that the maximum possible coverage is achieved by some pre-determined cut-off point. In literature, many techniques for regression test case prioritization for general applications and test suite prioritization strategies for web applications have been described. Most of these techniques are code-based, relying on information, relating test cases to coverage of code elements. In [02], Elbaum investigated several prioritizing techniques such as total statement (or branch) coverage prioritization and additional statement (or branch) coverage prioritization that can improve the rate of fault detection. Dennis Jeffrey, Neelam Gupta [10 ] present an approach to prioritize test cases based on the coverage requirements present in the relevant slices of the outputs of test cases. S. Raju and, G. V. Uma propose a set of prioritization factors to design a requirement based system level test case prioritization scheme using Genetic Algorithm (GA). These factors may be concrete, such as test case length, code coverage, data flow, and fault proneness, or abstract, such as perceived code complexity and severity of faults, which prioritizes the system test cases based on the J u l y 2 5 , 2 0 1 3

PROPOSED WORK
This section provides the proposed algorithm. The algorithm is basically divided into two parts. In first part Regression test selection is performed than prioritization is done on the selected test cases using faults severity criteria in the second part. faults which are also covered in some other test cases are removed following some steps and rest are selected for execution.. 7) Now, for second step take output of first step i.e. Selected Test Cases and previously defined severity of faults occurred in these test cases as input. 8) Now , suppose severity is assigned in terms of numbers 1 to 5 , 1 is for lowest severity and 5 is for high severity ( severity is assigned on the basis of customer requirement). 9) Arrange the test cases in order having the test case which has highest number of high severity faults, first. That is, test case which has highest number of faults having severity 5. Assign highest priority to this test case. 10) Cover all the test cases in decreasing order of number of severity of faults from high to low severity. 11) In this way, the test cases will be arranged in decreasing order of priority given to them on the basis of number of fault severities. 2) If there are more than one test case having same number of faults ,select the one which has less execution time .

The Algorithm
3) Note the faults occurred in this test case .(TC (max faults)). 4) Match the faults of other test cases ( TC( match)) , which have the same faults as in the (TC (max faults)). 5) If all the faults of any other test cases are covered in the TC (max faults) than we will not execute that test case(TC not selected) . 6) But if some faults are covered of any test case in the TC (max faults) and a few are left uncovered than remove the faults which are covered . 7) Now again from the list of test cases which have faults , excluding , previous TC (max faults ) and TC (not selected), find the next TC (max fault). 8) Repeat from step 1 to step 7, till all the test cases are covered .This way , about 30% to 50% cases are covered in (TC not selected) means not to be executed .That is 50% to 80% test cases are required to prioritize .

Second
Step: Input: 1) Output of first step : Selected Test cases .
Begin: J u l y 2 5 , 2 0 1 3 1) Now, from the SELECTED TEST CASES , Select the test case which has highest number of high severity faults (S5) as high priority test case. 2) This way , first cover all the test cases which have high severity faults (S5) ,in decreasing order of number of faults present of severity S5. 3) Next , select the test case which has highest number of faults having severity (S4). 4) This way , cover all test cases in decreasing order of severity of faults ( ie from S5 to S1 ) following decreasing order of number of faults in the same.

RESULT ANALYSIS
To validate the proposed Algorithm, the metric APFD (Average Percentage of Faults Detected) is used. [6], It measures the weighted average of the percentage of faults detected over the life of the suite. APFD values range from 0 to 100; higher number simply faster (better) fault detection rates. Let T be a test suite containing n test cases and let F be a set of m faults revealed by T. Let TFi be the first test case that reveals fault i for a given order of the test cases in the test suite T. The APFD for test suite T is calculated using equation

ACKNOWLEDGMENTS
We would like to express our heartiest gratitude to all the people who poured their efforts in compilation of this work. We would like to thank almighty for giving us strength to pull through this task and to all the individuals who gave their best contribution in the related field of research.Our thanks to the experts who have contributed towards development of the template.