Sei sulla pagina 1di 22

Using Java to Test Classes

The muJava system requires that the Java CLASSPATH be modified to include the muJava jar and the Java tools.jar files. Also, the general PATH variable must include the java bin directory; this is usually set automatically when java was installed, but not always. Then one GUI (Java applet) is used to generate mutants, the tester must create tests, and another GUI is used to run mutants.

1. Environment Settings for the muJava System


There are three steps to setting up the environment for muJava, (1) CLASSPATH, (2) setting the config file, and (3) creating subdirectories. i. The Java CLASSPATH must include two Java jar files and one standard Java jar file. tools.jar is standard with Java compilers and is probably located in the "lib/" directory. The two Java files are mujava.jar and openjava2005.jar, which are downloaded from this site. One slightly awkward requirement is that the CLASSPATH must include the location of the classes under test when generating mutants, but NOT when running mutants. (If it does, no mutants can be killed.) This location is the "classes/" directory under where the mujava.config is stored. In a DOS window, use the following command (assuming that classes is under C:\mujava):

set CLASSPATH=%CLASSPATH %;C:\mujava\mujava.jar;C:\mujava\openjava2005.jar;C:\j2sdk1.4.0_01\lib\tools.jar;C:\mujava\class es

In a Cygwin window, use the following command:

CLASSPATH="$CLASSPATH;C:\mujava\mujava.jar;C:\mujava\openjava2005.jar;C:\j2sdk1.4.0_0 1\lib\tools.jar;C:\mujava\classes" ; export CLASSPATH

To change your CLASSPATH permanently in Win2000 and WinXP, go to Startsettings-Control Panel. Double-click System, go to the Advanced tab, and choose

Environment Variables. Edit the CLASSPATH variable or create a new variable if there is none. Add the full path to mujava.jar and openjava2005.jar to the CLASSPATH. In Unix, set the CLASSPATH environment variable. Assuming the jar files are stored in the home directory of user gpd:
CLASSPATH=$CLASSPATH:/home/gpd/mujava.jar:/home/gpd/openjava2005. jar:/java1.4/j2sdk1.4.0_01/lib/tools.jar ; export CLASSPATH

Note that the syntax will vary under different shells, and it will be more convenient to put the command in your setup file such as .login, or .bashrc. ii. Next, modify the mujava.config file to point to a directory where you wish to store source Java files and muJava temporary files. The directory must be the complete path (either Windows or Unix). For example, the config file may contain the line: MuJava_HOME=C:\home\gpd\exp. IMPORTANT: It is necessary to copy the config file to the directory you run the muJava system in. iii. Finally, create a directory structure for the muJava system in the $MuJava_HOME directory. Assuming your MuJava_HOME directory is called MuJava, the subdirectories should look like:

There should be four subdirectories, used in the following ways. <TDMuJava_HOME\src directory for Java files to be tested MuJava_HOME\classes directory for compiled classes of Java files from MuJava_HOME\src directory for generated mutants

MuJava_HOME\testset directory for test sets MuJava_HOME\result

You can create these subdirectories by hand or by using the muJava class "mujava.makeMuJavaStructure".

java mujava.makeMuJavaStructure

Potential problems: We have identified several potential problems with installing Java.

Java does not work with Java 1.5. It is important that the MuJava_HOME variable NOT have a trailing slash. This will confuse Java. If you have a different version of the java compiler and the JVM, Java may get confused. This happens sometimes when a new application on your computer updates the JVM. If you have problems compiling or killing mutants, we suggest deleting all Java components and reinstalling the latest version. If your tools.jar file is out of date (pre Java 1.4, we think), parts of Java may not work.

2. Generating Mutants with muJava


Important note: You should run all commands in a directory that contains "mujava.config" i. Put the source files to test to MuJava_HOME\src directory. muJava does not check for compilation errors, so all Java files should compile correctly. If the Java file to test needs other Java files or class files, they should also be placed in MuJava_HOME\src. For example, suppose you want to test B, which is a child class of A. Then, you should put both A.java and B.java into MuJava_HOME\src. If the file has a package structure, you should store the entire package underneath MuJava_HOME\src. Compile all the Java files in MuJava_HOME\src and copy the .class files into the MuJava_HOME\classes\ directory. Start the GUI from the command line. Use it to generate mutants:
java mujava.gui.GenMutantsMain

ii. iii.

iv.

This command should bring a up a screen similar to the following:

v.

vi. vii.

Select the files you want to mutate by clicking in the boxes on the left. Select the mutation operators you want to use by slecting their boxes. Then push RUN. Note: The class mutation operators produce far fewer mutants. Also note that a number of status messages go to the command window, but not the GUI. After mutants are generated, you can view the mutants in the "Class Mutants Viewer" and "Traditional Mutants Viewer" tabs, as shown in the following two figures.

viii.

ix. x.

xi.

xii.

You may be interested in knowing where the mutant versions of the classes are stored. They are underneath the MuJava_HOME\result\ directory. The following example shows the directory Stack underneath result, with object-oriented mutants in class_mutants and traditional mutants in a separate directory.

3. Making a test set

A testset in muJava is a Java file that contains executable test scripts. Each test is a method that contains a sequence of calls to methods in the class under test. Each test method returns a string result that is used to compare outputs of mutants with outputs of the original class. Each test method should start with the string "test". The test methods and the test class should have public access. Below is an example of a testset class for the class Stack. Its name is StackTest. StackTest contains two test case: test1() and test2(). The testset .class file should be in the directory MuJava_HOME\testset\. (By the way, although these test scripts are reminiscient of JUnit tests, Java does not accept JUnit tests. The first version of Java predated JUnit so we implemented a general test driver facility.) public class StackTest { public String test1() { String result = ""; Stack obj = new Stack(); obj.push(2); obj.push(4); result = result + obj.isFull(); result = result + obj.pop(); return result; } public String test2() { String result = ""; Stack obj = new Stack(); obj.push(5); obj.push(3); result = result + obj.pop(); result = result + obj.pop(); return result; } }

4. Running mutants.
Run the mutants from another GUI. Start it with the following command:

java mujava.gui.RunTestMain

Note: Your CLASSPATH must not include MuJava_HOME\classes\. If it does, no mutants can be killed. You should see the following GUI. You can select which collection of mutants to run, and which testset to use. The "Class Mutants Viewer" and "Traditional Mutants Viewer" tabs will show the source view of the mutants. You can design tests to kill mutants by finding a live mutant, then analyzing the program to decide what input will kill it. Remember that between 5% to 20% of the mutants are typically equivalent.

For this project, all the test cases have been written such that they ensure that all the branches in the sample programs are covered. This is performed to determine adequacy of the test criterion. To determine the equivalent mutants for method-level operators and classlevel operators, a manual inspection of the mutants left alive was carried out. There was no definite process. For each operator, graphs have been created which show the total number of mutants created, the number of mutants killed, the number of equivalent mutants and the number of mutants that failed the software. These results will help determine which mutation operators are prone to develop more equivalent mutants. Also, scenarios were identified for certain mutation operators. These indicate the situations in which some of the mutation operators were more likely to create equivalent mutants. At the end of the analysis 3 types of results were obtained: 1. The percentage of mutants killed per mutation operator 2. The percentage of non-equivalent mutants per mutation operator 3. The percentage of equivalent mutants per mutation operator 4.1 Evaluation of Method-level Mutation Operators

MuJava created no mutants for the method-level mutation operators AORS, AODU, AODS, COD and LOD. An analysis has been performed for the remaining operators namely, AORB, AOIU, AOIS, ROR, COR, COI, SOR, LOR, LOI and ASRS. Graphs have been created. These graphs have been divided in arbitrary groups of 2. The last graph depicts the analysis for the method-level operator AOIS since it dominates the behavior of the other operators. This analysis is explained in more detail below:

Evaluation of Method-level Operators AORB, AOIU, ROR, COR & COI

Figure shows an evaluation of the operators AORB, AOIU, ROR, COR and COI.

MuJava created a very small number of mutants for AORB. But all the mutants that were created were killed successfully. None of the mutants that were created were found to be equivalent. Also, no mutants caused MuJava to fail. Thus, 100% mutation score was achieved. As compared to AORB, AOIU still created a considerable number of mutants. But the behavior exhibited by AOIU was identical to that of AORB. 100% of the mutants created for AOIU were killed by MuJava. None of the mutants that were created were found to be equivalent. The operator ROR works by replacing relational operators with all other relational operators. A significantly good number of operators were created for ROR (20). Of these, 19 mutants were killed. None of the mutants were found to be equivalent. The one mutant who was not killed was just difficult to kill. It was not equivalent.

4.2 Summary of Method-level Mutation operators

The following table summarizes the behavior of method-level mutation operators. It shows the percentage of mutants killed for each mutation operator. It also indicates the percentage of non- equivalent mutants against the percentage of equivalent mutants. Methodlevel Operators AORB AOIU AOIS ROR COR COI SOR LOR LOI Percentage of Mutants Killed 100 % 100 % 63.33% 95% 100% 0% 100% 100% 96.67% Percentage of Nonequivalent Mutants 100% 100 % 70.83% 100% 100% 100% 100% 100% 100% Percentage of Equivalent Mutants 0% 0% 29.17% 0% 0% 0% 0% 0% 0%

ASRS 100% 100% 0% Table: Analysis Summary of Method-level Mutation Operators

This table clearly indicates that the only method-level operator which created equivalent mutants was AOIS. It was assumed that operators AOIS,

ROR and LOI are most probable to create equivalent mutants because they formed the majority of all the mutants that were created for method-level mutation operators. But it can be supported with evidence that the only equivalent mutants produced were those created for AOIS. As for ROR and LOI, a mutation score of 95% and 96.67% was achieved, respectively. The remaining operators performed well and successfully killed all the mutants. However, all the mutants that were created for COI caused the software to fail. Also, the mutants that were left alive by LOI, 75% of them failed MuJava. Since AOIS created equivalent mutants, some scenarios were discovered under which this operator always creates equivalent mutants. These scenarios can be used to modify the AOIS operator such that it does not create equivalent mutants. This will reduce the computational cost in creating and executing the surplus mutants.

4.2.1 Scenarios for AOIS

Many situations were observed whilst determining equivalent mutants for AOIS. Some of these situations always tend to create equivalent mutants. As a reminder, the AOIS operator works by inserting pre-increment/decrement or post-increment/decrement operators. Most of the equivalent mutants are created by post-increment/decrement operators. These have been chalked out below: Original Program 1 public class test { 2 public void abc() { 3 int var1, var2; 4 int result; 5 result = var1 + var2; 6 System.out.println(result); 7 } 8} Equivalent Mutants 1 public class test { 2 public void abc() { 3 int var1, var2; 4 int result; 5 result = var1 + var2; 6 System.out.println(result++); 6 System.out.println(result--); 7 } 8} Equivalent Mutants for method-level operator AOIS 4.3 Evaluation of Class-level Mutation Operators

MuJava created no mutants for the class-level mutation operators PCC, PCD, JSD and EOA. An analysis has been performed for the remaining operators namely, IHD, IHI, IOD, IOP, IOR, ISI, ISD, IPC, PNC, PMD, PPD, PCI, PRV, OMR, OMD, OAN, JTI, JTD, JSI, JID, JDC, EOC, EAM and EMM. Like method-level operators, the graphs for class- level operators have been divided in arbitrary groups of 4. The last graph depicts an analysis of the operator PRV because it dominates all the other class-level mutation operators. The analysis for these operators is explained in more detail below:

Figure: Evaluation of Class-level Operators IHD, IHI, IOD, IOP, IOR & ISI

Figure shows an evaluation of the operators JSI, JID, JDC, EOC, EAM and EMM. JSI was one of the class-level mutation operators for which a considerably large number of mutants was created. 80% of the mutants created were found to be equivalent. But 87.5% of the total numbers of mutants were killed successfully. All the mutants that were left alive were found to be equivalent. None of the mutants failed the software. JID revealed very different behavior than JSI. Figure shows that MuJava killed 50% of the mutants that were created for JID. But, interestingly, all the mutants were found to be equivalent. JDC also created a good number of mutants. All the mutants that were created were found to be equivalent. None of the mutants were killed by the test set thus giving a mutation score of 0%. EOC is another operator which created a very small number of mutants. However, all the mutants that were created for EOC were killed by MuJava to achieve a mutation score of 100%. None of the mutants were equivalent. EAM and EMM exhibit identical behavior as shown in figure 26. All the mutants that were created for both the operators were killed successfully using MuJava. Thus, a mutation score of 100% was achieved. Also, none of the mutants were equivalent.

Figure: Evaluation of Class-level Operator PRV Of all the class-level mutation operators, PRV exhibits the most interesting behavior. The majority of mutants were created for this operator. However, an alarming 42.93% of these mutants failed the software. Of the remaining mutants, 81.07% mutants were killed. The rest of the mutants were found to be equivalent as depicted in figure 18. Also, of the non-equivalent mutants, 7.6% were hard to kill mutants.

4.3.1 Summary of Class-level Mutation operators

The following table summarizes the behavior of class-level mutation operators. It shows the percentage of mutants killed for each mutation operator. It also indicates the percentage of non- equivalent mutants against the percentage of equivalent mutants. 4.3.2 Scenarios The majority of the class-level operators created equivalent mutants. Some scenarios have been sketched out for some of these operators. These scenarios depict situations which will always create equivalent mutants. 4.3.2.1 IHD The operator IHD works by deleting a hiding variable in a subclass. One situation has been identified when IHD always creates an equivalent mutant. Original Program Public class LinkedList { protected ListNode start; protected int size; public LinkedList() { } public insert(int var) { } } public class Stack extends LinkedList { protected ListNode start; protected int size; public Stack(){ } public insert(int var) { start = newNode; Equivalent Mutants Public class LinkedList { protected ListNode start; protected int size; public LinkedList() { } public insert(int var) { } } public class Stack extends LinkedList { // protected ListNode start; protected int size; public Stack(){ } public insert(int var) { start = newNode;

size++; } }

size++; } } Scenario in which IHD creates equivalent mutants

Whenever the initialization of a variable is deleted in a subclass, the subclass makes use of the initialization in the parent class. If this initialization is the same in both the parent and the child class, the mutant is equivalent. 4.3.2.2 IHI The operator IHI works by inserting a hiding variable in a subclass. One situation has been identified when IHI always creates an equivalent mutant. Original Program Public class LinkedList { protected ListNode start; protected int size; public LinkedList() { } public insert(int var) { start = newNode; size++; } } public class Stack extends LinkedList { protected ListNode start; public Stack(){ } public insert(int var) { start = newNode; } Equivalent Mutants Public class LinkedList { protected ListNode start; protected int size; public LinkedList() { } public insert(int var) { start = newNode; size++; } } public class Stack extends LinkedList { protected ListNode start; protected int size; public Stack(){ } public insert(int var) { start = newNode; }

} Scenario in which IHI creates equivalent mutants

4.3.2.3 IOR IOR always creates equivalent mutants when the methods which have been renamed are not called elsewhere in the program. Hence, any change in the name of the function, will not affect the functioning of the rest of the programs. However, it is easy to note that if this same function is called in the main program, it will cause an error and therefore, fail the mutant. 4.3.2.4 PRV PRV created a large number of equivalent mutants. The situation which caused this is explained more clearly in the example below: Original Program Equivalent Mutants public class Stack extends public class Stack extends LinkedList { LinkedList { protected ListNode start; protected ListNode start; public Stack(){ } public Stack(){ } public insert(int var) { public insert(int var) { start = newNode; start = newNode; end = newNode; end = start; } } } } Scenario in which PRV creates equivalent mutants Whenever an object reference is referred to other compatible types, sometimes the value of the new object is the same as the value of the object

reference in the source program. In the example, the values of objects start and end are the same. Hence, making the object end to point to the object start will produce an equivalent mutant. 4.3.2.5 JID This operator deletes the initialization of instance variables. JID always creates an equivalent mutant when the initialization of a variable is set to the default value of the particular variable. For example: Original Program Equivalent Mutants public class Stack extends public class Stack extends LinkedList { LinkedList { int size = 0; int size; } } Scenario in which JID creates equivalent mutants Original Program Equivalent Mutants public class Stack extends public class Stack extends LinkedList { LinkedList { int size = 10; int size; } } Scenario in which JID will never create an equivalent mutant 4.3.2.6 JDC This operator deletes the user-defined default constructor and makes use of Javas default constructor. JDC will create an equivalent mutant whenever the operations performed in the default constructor are simple initializations to the default values of the variables. If the variables or objects are

initialized to values other than their default values, no equivalent mutants will be created.

Potrebbero piacerti anche