White Box Testing

Len Schroath September 18, 2007

Agenda
White-box vs Black-box Program Flow Controls White-box Test Methods Exercises Complexity Q&A

What is White-box Testing?

Looking at the internal structure of a program and deriving test cases based on the logic or control flow. Test cases can be designed to reach every branch in the code and to exercise each condition Typically done during unit testing Also known as:
Structural Testing Glass-Box Testing

What is Black-box Testing?

Looking at the program from an external point of view and deriving test cases based on the specification. The only criteria upon which the program is judged is if it produces the correct output for a given input.

Why Do Both?
Black-box Impossible to write a test case for every possible set of inputs and outputs Some of the code may not be reachable without extraordinary measures Specifications are not always complete White-box Does not address the question of whether or not the program matches the specification Does not tell you if all of the functionality has been implemented Does not discover missing program logic

Basic Program Flow Controls

IF IF-Then-Else FOR While Do-While Case

IF Diagram

If

IF-THEN-ELSE Diagram

If-then-else

FOR or WHILE Diagram

For OR While

DO-WHILE Diagram

Do-while

CASE Diagram

Case

Example Code Fragment

Do { if (A) then {...}; else { if (B) then { if (C) then {...}; else {} } else if (D) then {...}; else {...}; } } While (E);

Example Control Flow Graph

A
True

True

C
True True

E
True

Source: The Art of Software Testing Glenford Myers

Exercise #1
int main (int argc, char *argv[]) { /* Process CTRL-C Interrupts */ signal(SIGINT,catcher); if (validate_command_line(argc)) return(1); if (job_initialize(argv)) return(1); processTestList(); /* Process all testCases in TestList */ displayResults(); fprintf(stdout,"Test Complete\n"); job_termination(); return(0); Can you diagram } /* main */ this code?

Did You Get Something Like This?

True

True

Exercise #2
/* Attempt Statusreadback log SRB data to logFile */ int process_srb(void) { int srb_count = 0; do { srb_count = read_printer(srb_in); } while (!srb_count); fprintf(logFile,"%s\n",srb_in); return(srb_count); } /* process_srb */

Can you diagram this code?

/* Write String to Printer via Parallel or Serial port */ void write_printer (char *outputline) { if (strstr(printertype,"PAR") != NULL) { bwrite_parST(printerport,outputline); } else if (strstr(printertype,"SER") != NULL) { bwwrite_serial(printerport,outputline,strlen(outputline)); } else if (strstr(printertype, "FILE") !=NULL) { fprintf(printerFile,"%s",outputline); } } /* write_printer */

Did You Get Something Like This?

White-box Test Methods

Statement Coverage Decision/Branch Coverage Condition Coverage Decision/Condition Coverage Path Coverage

Example Code Fragment

If ((A>1) & (B=0)) then Do; X=X/A; END; If ((A==2) | (X>1)) then Do; X=X+1; END; END;
Source: The Art of Software Testing Glenford Myers

Statement Coverage
Exercise all statements at least once How many test cases?
A=2 and B=0 (ace)

Source: The Art of Software Testing Glenford Myers

Decision/Branch Coverage
Each decision has a true and a false outcome at least once How many test cases?
A=2 and B=0 (ace) A=1 and X=1 (abd)

Source: The Art of Software Testing Glenford Myers

Condition Coverage
Each condition in a decision takes on all possible outcomes at least once Conditions: A>1, B=0, A=2, X>1 How many test cases?
A=2, B=0, and X=4 (ace) A=1, B=1, and X=1 (abd)

Source: The Art of Software Testing Glenford Myers

Decision/Condition Coverage
Each condition in a decision takes on all possible outcomes at least once, and each decision takes on all possible outcomes at least once How many test cases?
A=2, B=0, and X=4 (ace) A=1, B=1, and X=1 (abd)

What about these?

A=1, B=0, and X=3 A=2, B=1, and X=1
(abe) (abe)

Multiple Condition Coverage

Exercise all possible combinations of condition outcomes in each decision Conditions:
A>1, B=0 A>1, B<>0 A<=1, B=0 A<=1, B<>0 A=2, X>1 A=2, X<=1 A<>2, X>1 A<>2, X<=1

Source: The Art of Software Testing Glenford Myers

Multiple Condition Coverage

How many test cases?
A=2, B=0, X=4 A=2, B=1, X=1 A=1, B=0, X=2 A=1, B=1, X=1 (ace) (abe) (abe) (abd)

Source: The Art of Software Testing Glenford Myers

Path Coverage
Every unique path through the program is executed at least once How many test cases?
A=2, B=0, X=4 (ace) A=2, B=1, X=1 (abe) A=3, B=0, X=1 (acd) A=1, B=1, X=1 (abd)
Source: The Art of Software Testing Glenford Myers

McCabes Cyclomatic Complexity

Software metric Developed by Tom McCabe (circa 1976) Directly measures the number of linearly independent paths through a programs source code, taking into account the various decision points Independent of implementation language

Source: Wikipedia

Calculating Complexity
(a) # regions + 1 (b) # decisions + 1 (c) L N + 2P
True

A
True

C
True True

E
True

Calculating Complexity
(a) # regions + 1 = 5+1 = 6 (b) # decisions + 1 = 5+1 = 6 (c) L N + 2P = 17-13+2 = 6
True

A
True

C
True True

E
True

How Complex Should Code Be?

<10: Simple module, not much risk 10-20: More Complex; moderate risk 20-50: Complex; high risk >50: Untestable; extremely high risk

Source: Carnegie Mellon Software Engineering Institute

Complexity Caveats
As code is broken into smaller modules to decrease cyclomatic complexity, structural complexity increases Some modules may have high complexity but are very easy to comprehend and easy to test High complexity numbers are only an indicator of something to investigate

Questions?