c
 

Since humans design and program hardware and software, mistakes are inevitable. That's what computer
and software vendors tell us, and it's partly true. What they don't say is that software is buggier than it has to
be. Why? Because time is money, especially in the software industry. This is how bugs are born: a software
or hardware company sees a business opportunity and starts building a product to take advantage of that.
Long before development is finished, the company announces that the product is on the way. Because the
public is (the company hopes) now anxiously awaiting this product, the marketing department fights to get
the goods out the door before that deadline, all the while pressuring the software engineers to add more and
more features. Shareholders and venture capitalists clamor for quick delivery because that's when the
company will see the biggest surge in sales. Meanwhile, the quality-assurance division has to battle for
sufficient bug-testing time. "The simple fact is that you get the most revenues at the release of software,"
says Bruce Brown, the founder of BugNet, a newsletter that has chronicled software bugs and fixes since
1994. "The faster you bring it out, the more money you make. You can always fix it later, when people howl.
It's a fine line when to release something, and the industry accepts defects." It may seem that there are
more bugs these days than ever before, but longtime bug watchers like Brown say this is mostly a visual
illusion caused by increased media coverage. Not only has the number of bugs not changed, but
manufacturers are fixing them more quickly. But while the industry as a whole may not be buggier, one
important new category is, arguably, more flawed than other genres: Internet software. The popularity of the
Internet is pushing companies to produce software faster than ever before, and the inevitable result is
buggier products. "Those are crazy release schedules," says Brian Bershad, an associate professor of
computer science at the University of Washington. His Kimera project helped catch several security bugs in
Java. "The whole industry is bonkers. Web standards need to be developed and thoughtfully laid out, but
look at all the versions of Java and HTML. It's not that the people aren't smart; it's just that they don't have
time to think." But software and hardware companies persist in arguing that we should put up with bugs.
Why? Because the cost of stamping out all bugs would be too high for the consumer. "Software is just
getting so incredibly complicated," says Bershad. "It's too expensive to have no bugs in consumer software."

c 
     



What is the difference between a bug, a defect, and an error?



According to the British norm BS 7925-1: bug--generic term for fault, failure, error, human action that
produces an incorrect result.
Robert Vanderwall offers these formal definitions from IEEE 610.1. The
sub-points are his own.


: A human action that produces an incorrect result.
- mistake made in translation or interpretation.
- lots of taxonomies exist to describe errors.
: An incorrect step, process or data definition.
- manifestation of the error in implementation
- this is really nebulous, hard to pin down the 'location'
: An incorrect result.
bug: An informal word describing any of the above. (Not IEEE)

Rohan Khale found a web site that gave these definitions:
A bug exists because what is supposed to work does not work as what you
expected. Defects occur usually when a product no longer works the way it used to.

He found these easy to understand defintions: A defect is for something that normally works, but it has
something out-of-spec. On the other hand a Bug is something that was considered but not implemented, for
whatever reasons.

I have seen these arbitrary definitions:

: programmatically mistake leads to error.
 : Deviation from the expected result.
: Problem in algorithm leads to failure.
: Result of any of the above.

Compare those to these arbitrary definitions:

: When we get the wrong output i.e. syntax error, logical error
: When everything is correct but we are not able to get a result
: We are not able to insert any input

c
  !"

To write a fully effective report you must:

- Explain how to reproduce the problem - Analyze the error so you can describe it in a minimum
number of steps.
- Write a report that is complete and easy to understand.

Write bug reports immediately; the longer you wait between finding the problem and reporting it,
the more likely it is the description will be incomplete, the problem not reproducible, or simply
forgotten.

Writing a one-line report summary (Bug's report title) is an art. You must master it. Summaries
help everyone quickly review outstanding problems and find individual reports. The summary line
is the most frequently and carefully read part of the report. When a summary makes a problem
sound less severe than it is, managers are more likely to defer it. Alternatively, if your summaries
make problems sound more severe than they are, you will gain a reputation for alarmism. Don't
use the same summary for two different reports, even if they are similar. The summary line
should describe only the problem, not the replication steps. Don't run the summary into the
description (Steps to reproduce) as they will usually be printed independently of each other in
reports.

Ideally you should be able to write this clearly enough for a developer to reproduce and fix the
problem, and another QA engineer to verify the fix without them having to go back to you, the
author, for more information. It is much better to over communicate in this field than say too little.
Of course it is ideal if the problem is reproducible and you can write down those steps. But if you
can't reproduce a bug, and try and try and still can't reproduce it, admit it and write the report
anyway. A good programmer can often track down an irreproducible problem from a careful
description.

















  #$
% ÊÊ
Ê

Ê
Ê


 !"$"



!"
Unique number given to a bug.

ü &  

 
The name of a program or module that being tested

'
(

The version of the product that you are testing.

ü)

(data entry field that's one line) precise to what the problem is.

!"*
"
Describes the type of problem found, for example it could be software or hardware bug.

) 

Normally, how you view the bug.
Various levels of severity: Low - Medium - High - Urgent
 
Environment in which the bug is found.

 
"
Detailed description of the bug that is found

" 
Detailed description of how to reproduce the bug.

!" 

The name of person who writes the report.

    "
The name of developer who assigned to fixed the bug.

)

Open:
The status of bug when it entered.
Fixed / feedback:
The status of the bug when it fixed.
Closed:
The status of the bug when verified.
(Bug can be only closed by QA person. Usually, the problem is closed by QA manager.)
Deferred:
The status of the bug when it postponed.
User error:
The status of the bug when user made an error.
Not a bug:
The status of the bug when it is not a bug.

ü

Assigned by the project manager who asks the programmers to fix bugs in priority order.

!

Defines the current status of the problem. There are four types of resolution such as deferred, not a
problem, will not fix, and as designed.

 +"


#"
This is for Minor problems, such as failures at extreme boundary conditions that are unlikely to occur in
normal use, or minor errors in layout/formatting. These problems do not impact use of the product in any
substantive way.

% "
This is a problem that a) Effects a more isolated piece of functionality. b) Occurs only at certain boundary
conditions. c) Has a workaround (where "don't do that" might be an acceptable answer to the user). d)
Occurs only at one or two customers. or e) Is very intermittent

 "
This should be used for only serious problems, effecting many sites, with no workaround. Frequent or
reproducible crashes/core dumps/GPFs would fall in this category, as would major functionality not working.

, "
This should be reserved for only the most catastrophic of problems. Data corruption, complete inability to
use the product at almost any site, etc. For released products, an urgent bug would imply that shipping of
the product should stop immediately, until the problem is resolved.



) 
 ü




One question on the defects that we raise. We are supposed to give a
severity and a priority to it. Now, the severity can be Major, Minor or
Trivial and the Priority can be 1, 2 or 3 (with 1 being a high priority
defect).
My question is - why do we need two parameters, severity and priority, for a defect Can't we do only with
one?



It depends entirely on the size of the company. Severity tells us how bad the defect is. Priority tells us how
soon it is desired to fix the problem.
In some companies, the defect reporter sets the severity and the triage team or product management sets
the priority. In a small company, or project (or product), particularly where there aren't many defects to track,
you can expect you don't really need both since a high severity defect is also a high priority defect. But in a
large company, and particularly where there are many defects, using both is a form of risk management.

Major would be 1 and Trivial would be 3. You can add or multiply the two values together (there is only a
small difference in the outcome) and then use the event's risk value to determine how you should address
the problem. The lower values must be addressed and the higher values can wait.

I discovered a new method for Risk Assessment. It is based on a military standard, MIL-STD-882. If you
want a copy of the current version, search for MIL-STD-882D using Google or Yahoo! The main area of
interest is section A.4.4.3 and its children where they indicate the Assessment of mishap risk.
They use a four-point severity rating (rather than three): Catastrophic; Critical; Marginal; Negligible. They
then use a five-point (rather than three) probability rating: Frequent; Probable; Occasional; Remote;
Improbable. Then rather than using a mathematical calculation to determine a risk level, they use a
predefined chart.

?
: This bug prevents developers from testing or developing the software.
 : The software crashes, hangs, or causes you to lose data.


: A major feature is broken.


 : It's a bug that should be fixed.


: Minor loss of function, and there's an easy work around.

 : A cosmetic problem, such as a misspelled word or misaligned text.
Enhancement: Request for new feature or enhancement.Ê
Ê


sever6rity Levels can be defined as follow:
P - High/Showstopper. Like system crash or error message forcing to close the window.
Tester's ability to operate the system either totally (System Down), or
almost totally, affected. A major area of the users system is affected by the incident and it is significant to
business processes.
P
- Medium/Workaround. Exist like when a problem is required in the specs but tester can go on with
testing.
Incident affects an area of functionality but there is a work-around which negates impact to business
process.
This is a problem that:
a) Affects a more isolated piece of functionality.
b) Occurs only at certain boundary conditions.
c) Has a workaround (where "don't do that" might be an acceptable answer to the user).
d) Occurs only at one or two customers. or is intermittent
P - Low. This is for minor problems, such as failures at extreme boundary conditions that are unlikely to
occur in normal use, or minor errors in
layout/formatting. Problems do not impact use of the product in any substantive way. These are incidents
that are cosmetic in nature and of no or very low impact to business processes.

c 
*
 
Ê
Ê
âÊÊÊ 
Ê
ÊÊÊ
Ê ÊÊÊ Ê  Ê
Ê  Ê Ê Ê
 

Defect can be caused by a flaw in the application software or by a flaw in the application specification. For
example, unexpected (incorrect) results can be from errors made during the construction phase, or from an
algorithm incorrectly defined in the specification. Testing is commonly assumed to mean executing software
and finding errors. This type of testing is known as dynamic testing, and while valid, it is not the most
effective way of testing. Static testing, the review, inspection and validation of development requirements, is
the most effective and cost efficient way of testing. A structured approach to testing should use both
dynamic and static testing techniques.

Testing and Quality Assurance

What is the relationship between testing and Software Quality Assurance (SQA)? An application that meets
its requirements totally can be said to exhibit quality. Quality is not based on a subjective assessment but
rather on a clearly demonstrable, and measurable, basis. Quality Assurance and Quality Control are not the
same. Quality Control is a process directed at validating that a specific deliverable meets standards, is error
free, and is the best deliverable that can be produced. It is a responsibility internal to the team. QA, on the
other hand, is a review with a goal of improving the process as well as the deliverable. QA is often an
external process. QA is an effective approach to producing a high quality product. One aspect is the process
of objectively reviewing project deliverables and the processes that produce them (including testing), to
identify defects, and then making recommendations for improvement based on the reviews. The end result
is the assurance that the system and application is of high quality, and that the process is working. The
achievement of quality goals is well within reach when organizational strategies are used in the testing
process. From the client's perspective, an application's quality is high if it meets their expectations.

)*
 -.!


-. Test early and test often.

/. Integrate the application development and testing life cycles. You'll get better results and you won't have
to mediate between two armed camps in your IT shop.

0. Formalize a testing methodology; you'll test everything the same way and you'll get uniform results.

1. Develop a comprehensive test plan; it forms the basis for the testing methodology.

2. Use both static and dynamic testing.

3. Define your expected results.

4. Understand the business reason behind the application. You'll write a better application and better testing
scripts.

5. Use multiple levels and types of testing (regression, systems, integration, stress and load).

6. Review and inspect the work, it will lower costs.

-.. Don't let your programmers check their own work; they'll miss their own errors.

*
% 


78*

In using this strategy, the tester views the program as a black - box, tester doesn't see the code of the
program: Equivalence partitioning, Boundary - value analysis, Error guessing.Ê

c 78*

In using this strategy, the tester examine the internal structure of the program: Statement coverage,
Decision coverage, condition coverage, Decision/Condition coverage, Multiple - condition coverage.Ê
o
78*

In using this strategy Black box testing can be combine with knowledge of database validation, such as SQL
for database query and adding/loading data sets to confirm functions, as well as query the database to
confirm expected result.Ê

*
)"
Type of test file. It is a set of instructions run automatically by a software or hardware test tool.Ê

)
A collection of test cases or scripts.

*
"
*
 *
! 



"
Is the process of comparing a program to its requirements Ê
*
 Ê

Appropriate and very often syndicated when tester wants to

become familiar with the product, or in the environment when
 7
 Ê technical/testing materials are not 100% completed. It is also
largely based on general software product functionality/testing
understanding and the normal 'Human Common Sense'.Ê

The build acceptance test is a simplistic check of a product's

functionality in order to determine if the product is capable of
being tested to a greater extent. Every new build should
undergo a build acceptance test to determine if further testing
can be executed. Examples of a build acceptance:
 " Product can be installed with no crashes or errors that
*
 Ê terminate the installation. (Development needs to install the
software from the same source accessed by QA (e.g. Drop
Zone, CD-ROM, Electronic Software Distribution archives,
etc.).
Clients can connect to associated servers.
Simple client server communication can be achievedÊ

Start testing with the bottom of the program. The bottom - up

7"Ê
strategy does not exist until the last module is added.Ê

An in-house test is performed before the Alpha, Beta, and FCS

$*$
 milestones which is used to determine whether the product
8"*
 Ê can be installed and used without any problems, assistance, or
support from others.Ê

$7) *
 Ê Testing Systems that operate in client/server environments.Ê

This test is used to test compatibility between different

$"
*
 Ê client/server version combinations as well as other supported
products.Ê

The confidence test ensures a product functions as expected

by ensuring platform specific bugs are not introduced and
functionality has not regressed from drop to drop. A typical
$ *
 Ê confidence test is designed to touch all major areas of a
product's functionality. These tests are run regularly once the
Functional Freeze milestone is reached throughout the
remaining development cycle.Ê

These tests are run for product testing across various system
configuration combinations. Examples of configurations:
$ 
Cross platform (e.g. Windows Clients against a UNIX server).
*

Ê
Client/server network configurations. Operating systems and
database combinations (also including version combinations).
 Web servers and web browsers (for web products). The
system configurations to test are determined from the
product's compatibility matrix. This test is sometimes called a
'Platform test'.
CET (Customer Experience Test)
An in-house test is performed before the Alpha, Beta, and FCS
milestones which is used to determine whether the product
can be installed and used without any problems, assistance, or
support from others.Ê

 The depth tests are designed to test all the product's

 functionality
" *
 Ê in-depth.Ê

The error test is designed to test the dialogs, alerts, and other
feedback provided to the user when an error situation occurs.
The difference between this test and a Negative Test is that an
*
 Ê
Error Test is simply verifying that the correct dialogs are seen.
The Negative Test is primarily looking at the robustness and
recovery facets.Ê

Testing event-driven processes, such as unpredictable

 7 Ê sequences of interrupts from input devices or sensors, or
sequences of mouse clicks in a GUI environment.Ê

Verification that the final media, prior to hand off to Operations

for duplication, contains the correct code which was previously
+

tested and is installable on all the supported platforms and
*
 Ê
databases. The product demo is executed and product
Release Notes verified.Ê

This is designed to test the full functionality, features, and user


*
Ê
interfaces of software based upon the functional specifications.Ê

 A full test is Build Acceptance + Sanity + Confidence + Depth.

 This is designed to test the full functionality, features, and user
*
 Ê interfaces of software based upon the functional specifications.Ê

Testing the front-end user interfaces to applications which use

o" ,

GUI support systems and standard such as MS Windows or
+ (GUI) Ê
Motif.Ê

Step by step walk through of a tool or application, exercising

each screen or window's menus, toolbar and dialog boxes to
verify the execution and correctness of the Graphical User
o,+! "
Ê
Interface. Typically, this is handled by automated scripts and
rarely is used as a manual tests due to the low numbers of
bugs found from them.Ê

To test large program its necessary to use module testing.

Module testing (or unit testing) is a process of testing individual
% 
 Ê subprograms (small blocks), rather than testing the program as
a whole. Module testing eases the task of debugging. When
error is found, it is known is which particular module it is.Ê

Test maximum number of users specified in the design

%7
*
 Ê concurrently, to simulate the real user environment when they
use the product.Ê

Tests that deliberately introduce an error to check an

application's
behavior and robustness. For example, erroneous data may be
½  *
 Ê entered, or attempts made to force the application to perform an
operation that is should not be able to complete. Generally a
message box is generated to inform the user of the problem. If
the
 program terminates, the program should exit gracefullyÊ

Testing systems designed or coded using an object-oriented

å97å Ê approach or development environment, such as C++ or
Smalltalk.Ê

Testing by processing the same (or at least closely

ü*
 Ê comparable) test workload against both the existing and new
versions of a product, then comparing results.Ê

Measurement and prediction of performance (e.g. Response

time and/or throughput) for a given benchmark workload.
Phased Approach
A testing strategy where test cases are developed in stages so
a minimally acceptable level of testing can be completed at
any time. As new features are coded and frozen, they receive
priorities for a given amount of time-so that a concentrated
effort is directed toward testing those new features before the
üÊ
effort returns to validate the preexisting functionality. When no
new features are available, preexisting features will be
targeted-with priorities set by Project Leads.
1st level - Minimal Acceptance Test
2nd level - Confidence Tests
3rd level - Full Functionality Test
4th level - Error, Negative, and other Tests
5th level - System level testsÊ

A testing strategy where testcases are developed in stages so

a minimally acceptable level of testing can be completed at
any time. As new features are coded and frozen, they receive
priorities for a given amount of times-so that a concentrated
effort is directed toward testing those new features before the
effort returns to
validate the preexisting functionality. When no new features
ü 
 "" Ê are available, preexisting features will be targeted-with
priorities set by Project Leads.
1st level - Build Acceptance Test
2nd level - Sanity Test
3rd level - Confidence Test
4th level - Depth Test
5th level - Error, Negative, and other Tests
6th level - System level testsÊ

These tests are used for comprehensive re-testing of software

to validate that all functionality and features of previous builds
(or releases) have maintained integrity of features and
! 

*

Ê
functions tested previously. This suite of tests includes the Full
Functionality Tests and bug Regression Tests (automated and
manual).Ê

Sanity tests are subsets of the confidence test and are used
)
*
 Ê
only to validate high-level functionality.Ê

)
*
 Ê It is a test how easy to break program's security system.Ê

These tests are used to validate software functionality at the

)

*
 Ê limit (e.g. Maximum throughput) and then testing at and
beyond these limits.Ê

These tests check for factors such as Cross-Tool testing,

)

# *
 Ê
memory management and other operating system factors.Ê

*"7
Start testing with the top of the program.Ê


Ê

'*
 Ê Is the process of feeding a program with heavy volume of
 data.Ê

The effectiveness, efficiency, and satisfaction with which

,

Ê specified users can achieve specified goals in a particular
environment. Synonymous with "ease of use".Ê

Ê
*
)"

Ê
À



 

 




 


 



 








 

À


 

 
 











 
À

 


 

 

 







À




 
 











 
 

 



 




À






!








"









 

#




 







$%
&'(

"
)





*#













&*#








)



&*+












 


#&À#!

)#*#










,

À
&,À*




&!





À
*!



&

 

*






À

 

À

 




-./À
 
0


&À*
!

,À!112"
$3#
4-54.51444
$%
&'(

"
)





*

6 
 
,À 
(

 
!, 


   

0










71442
0





)



 "8
!


"
35/94

#&À#!

)#*
,À
: #"À"8'#)28;14
: #"À"+8;14
< )!71442

&!





À
*
#=!,8'#)2#)8;14
#=!,+8;14
)=)!71442

*
$

 ÊÊ

*
$
+
It is unique number given to test case in order to be identified.

*
 
"
The description if test case you are going to test.

! 
 


Each test case has to have its revision history in order to know when and by whom it is created or modified.


 
The name of function to be tested.

 
It tells in which environment you are testing. 

*
)"
Anything you need to set up outside of your application for example printers, network and so on.

*
8
It is detailed description of every step of execution.

8" !


The description of what you expect the function to do. 

!


pass / failed
If pass - What actually happen when you run the test.
If failed - put in description of what you've observed.

$ 

o *

They are: likely to catch bugs
no redundant
not too simple or too complex.

Test case is going to be complex if you have more than one expected results.
Ê

o *


What makes a good software tester? Many myths abound, such as being
creatively sadistic or able to handle dull, repetitive work. As a one-time test manager and currently as a
consultant to software development and testing organizations, I've formed a picture of the ideal software
tester-they share many of the qualities we look for in programmers; but there are also some important
differences. Here's a quick summary of the sometimes contradictory lessons that I've learned.

1. Know Programming. Might as well start out with the most controversial one. There's a popular myth that
testing can be staffed with people who have little or no programming knowledge. It doesn't work, even
though it is an unfortunately common approach. There are two main reasons why it doesn't work.
(1) They're testing software. Without knowing programming, they can't have any real insights into the kinds
of bugs that come into software and the likeliest place to find them. There's never enough time to test
"completely", so all software testing is a compromise between available resources and thoroughness. The
tester must optimize scarce resources and that means focusing on where the bugs are likely to be. If you
don't know programming, you're unlikely to have useful intuition about where to look.

(2) All but the simplest (and therefore, ineffectual) testing methods are tool- and technology-intensive. The
tools, both as testing products and as mental disciplines, all presume programming knowledge. Without
programmer training, most test techniques (and the tools based on those techniques) are unavailable. The
tester who doesn't know programming will always be restricted to the use of ad-hoc techniques and the most
simplistic tools.

Does this mean that testers must have formal programmer training, or
have worked as programmers? Formal training and experience is usually the easiest way to meet the "know
programming" requirement, but it is not absolutely essential. I met a superb tester whose only training was
as a telephone operator. She was testing a telephony application and doing a great job. But, despite the lack
of formal training, she had a deep, valid, intuition about programming and had even tried a little of it herself.
Sure she'sgood-good, hell! She was great. How much better would she have been and how much earlier
would she have achieved her expertise if she had had the benefits of formal training and working
experience? She would have been a lot better a lot earlier.

I like to see formal training in programming such as a university degree in Computer Science or Software
Engineering, followed by two to three years of working as a programmer in an industrial setting. A stint on
the customer-service hot line is also good training.

I don't like the idea of taking entry-level programmers and putting them into a test organization because:

(1) Loser Image.

Few universities offer undergraduate training in testing beyond "Be sure to test thoroughly." Entry-level
people expect to get a job as a programmer and if they're offered a job in a test group, they'll often look upon
it as a failure on their part: they believe that they didn't have what it takes to be a programmer in that
organization. This unfortunate perception exists even in organizations that values testers highly.

(2) Credibility With Programmers.

Independent testers often have to deal with programmers far more senior than themselves. Unless they've
been through a coop program as an undergraduate, all their programming experience is with academic toys:
the novice often has no real idea of what programming in a professional, cooperative, programming
environment is all about. As such, they have no credibility with their programming counterpart who can sluff
off their concerns with "Look, kid. You just don't understand how programming is done here, or anywhere
else, for that matter." It is setting up the novice tester for failure.

(3) Just Plain Know-How.

The programmer's right. The kid doesn't know how programming is really done. If the novice is a "real"
programmer (as contrasted to a "mere tester") then the senior programmer will often take the time to mentor
the junior and set her straight: but for a non-productive "leech" from the test group? Never! It's easiest for the
novice tester to learn all that nitty-gritty stuff (such as doing a build, configuration control, procedures,
process, etc.) while working as a programmer than to have to learn it, without actually doing it, as an entry-
level tester.

2. Know the Application.

That's the other side of the knowledge coin. The ideal tester has deep insights into how the users will exploit
the program's features and the kinds of cockpit errors that users are likely to make. In some cases, it is
virtually impossible, or at least impractical, for a tester to know both the application and programming. For
example, to test an income tax package properly, you must know tax laws and accounting practices. Testing
a blood analyzer requires knowledge of blood chemistry; testing an aircraft's flight control system requires
control theory and systems engineering, and being a pilot doesn't hurt; testing a geological application
demands geology. If the application has a depth of knowledge in it, then it is easier to train the application
specialist into programming than to train the programmer into the application. Here again, paralleling the
programmer's qualification, I'd like to see a university degree in the relevant discipline followed by a few
years of working practice before coming into the test group.

3. Intelligence.

Back in the 60's, there were many studies done to try to predict the ideal qualities for programmers. There
was a shortage and we were dipping into other fields for trainees. The most infamous of these was IBM's
programmers' Aptitude Test (PAT). Strangely enough, despite the fact the IBM later repudiated this test, it
continues to be (ab)used as a benchmark for predicting programmer aptitude. What IBM learned with follow-
on research is that the single most important quality for programmers is raw intelligence-good programmers
are really smart people-and so are good testers.

4. Hyper-Sensitivity to Little Things.

Good testers notice little things that others (including programmers) miss or ignore. Testers see symptoms,
not bugs. We know that a given bug can have many different symptoms, ranging from innocuous to
catastrophic. We know that the symptoms of a bug are arbitrarily related in severity to the cause.
Consequently, there is no such thing as a minor symptom-because a symptom isn't a bug. It is only after the
symptom is fully explained (i.e., fully debugged) that you have the right to say if the bug that caused that
symptom is minor or major. Therefore, anything at all out of the ordinary is worth pursuing. The screen
flickered this time, but not last time-a bug. The keyboard is a little sticky-another bug. The account balance
is off by 0.01 cents-great bug. Good testers notice such little things and use them as an entree to finding a
closely-related set of inputs that will cause a catastrophic failure and therefore get the programmers'
attention. Luckily, this attribute
can be learned through training.

5. Tolerance for Chaos.

People react to chaos and uncertainty in different ways. Some cave in and give up while others try to create
order out of chaos. If the tester waits for all issues to be fully resolved before starting test design or testing,
she won't get started until after the software has been shipped. Testers have to be flexible and be able to
drop things when blocked and move on to another thing that's not blocked. Testers always have many
(unfinished) irons in the fire. In this respect, good testers differ from programmers. A compulsive need to
achieve closure is not a bad attribute in a programmer-certainly serves them well in debugging-in testing, it
means nothing gets finished. The testers' world is inherently more chaotic than the programmers'.

A good indicator of the kind of skill I'm looking for here is the ability to do crossword puzzles in ink. This skill,
research has shown, also correlates well with programmer and tester aptitude. This skill is very similar to the
kind of unresolved chaos with which the tester must daily deal. Here's the theory behind the notion. If you do
a crossword puzzle in ink, you can't put down a word, or even part of a word, until you have confirmed it by a
compatible cross-word. So you keep a dozen tentative entries unmarked and when by some process or
another, you realize that there is a compatible cross-word, you enter them both. You keep score by how
many corrections you have to make-not by merely finishing the puzzle, because that's a given. I've done
many informal polls of this aptitude at my seminars and found a much higher percentage of crossword-
puzzles-in-ink afficionados than you'd get in a normal population.

6. People Skills.

Here's another area in which testers and programmers can differ. You can be an effective programmer even
if you are hostile and anti-social; that won't work for a tester. Testers can take a lot of abuse from outraged
programmers. A sense of humor and a thick skin will help the tester survive. Testers may have to be
diplomatic when confronting a senior programmer with a fundamental goof. Diplomacy, tact, a ready smile-
all work to the independent tester's advantage. This may explain one of the (good) reasons that there are so
many women in testing. Women are generally acknowledged to have more highly developed people skills
than comparable men-whether it is something innate on the X chromosome as some people contend or
whether it is that without superior people skills women are unlikely to make it through engineering school
and into an engineering career, I don't know and won't attempt to say. But the fact is there and those
sharply-honed people skills are
important.
7. Tenacity.

An ability to reach compromises and consensus can be at the expense

of tenacity. That's the other side of the people skills. Being socially smart and diplomatic doesn't mean being
indecisive or a limp rag that anyone can walk all over. The best testers are both-socially adept and tenacious
where it matters. The best testers are so skillful at it that the programmer never realizes that they've been
had. Tenacious-my picture is that of an angry pitbull fastened on a burglar's rear-end. Good testers don You
can't intimidate them-even by pulling rank. They'll need high-level backing, of course, if they're to get you the
quality your product and market demands.

8. Organized.

I can't imagine a scatter-brained tester. There's just too much to keep track of to trust to memory. Good
testers use files, data bases, and all the other accouterments of an organized mind. They make up
checklists to keep themselves on track. They recognize that they too can make mistakes, so they double-
check their findings. They have the facts and figures to support their position. When they claim that there's a
bug-believe it, because if the developers don't, the tester will flood them with well-organized, overwhelming,
evidence.

A consequence of a well-organized mind is a facility for good written and oral communications. As a writer
and editor, I've learned that the inability to express oneself clearly in writing is often symptomatic of a
disorganized mind. I don't mean that we expect everyone to write deathless prose like a Hemingway or
Melville. Good technical writing is well-organized, clear, and straightforward: and it doesn't depend on a
500,000 word vocabulary. True, there are some unfortunate individuals who express themselves superbly in
writing but fall apart in an oral presentation- but they are typically a pathological exception. Usually, a well-
organized mind results in clear (even if not inspired) writing and clear writing can usually be transformed
through training into good oral presentation skills.

9. Skeptical.

That doesn't mean hostile, though. I mean skepticism in the sense that nothing is taken for granted and that
all is fit to be questioned. Only tangible evidence in documents, specifications, code, and test results matter.
While they may patiently listen to the reassuring, comfortable words from the programmers ("Trust me. I
know where the bugs are.")-and do it with a smile-they ignore all such in-substantive assurances.

10. Self-Sufficient and Tough.

If they need love, they don't expect to get it on the job. They can't be looking for the interaction between
them and programmers as a source of ego-gratification and/or nurturing. Their ego is gratified by finding
bugs, with few misgivings about the pain (in the programmers) that such finding might engender. In this
respect, they must practice very tough love.

11. Cunning.

Or as Gruenberger put it, "low cunning." "Street wise" is another good descriptor, as are insidious, devious,
diabolical, fiendish, contriving, treacherous, wily, canny, and underhanded. Systematic test techniques such
as syntax testing and automatic test generators have reduced the need for such cunning, but the need is still
with us and undoubtedly always will be because it will never be possible to systematize all aspects of
testing. There will always be room for that offbeat kind of thinking that will lead to a test case that exposes a
really bad bug. But this can be taken to extremes and is certainly not a substitute for the use of systematic
test techniques. The cunning comes into play after all the automatically generated "sadistic" tests have been
executed.

12. Technology Hungry.

They hate dull, repetitive, work-they'll do it for a while if they have to, but not for long. The silliest thing for a
human to do, in their mind, is to pound on a keyboard when they're surrounded by computers. They have a
clear notion of how error-prone manual testing is, and in order to improve the quality of their own work,
they'll f ind ways to eliminate all such error-prone procedures. I've seen excellent testers re-invent the
capture/playback tool many times. I've seen dozens of home-brew test data generators. I've seen excellent
test design automation done with nothing more than a word processor, or earlier, with a copy machine and
lots of bottles of white-out. I've yet to meet a tester who wasn't hungry for applicable technology. When
asked why didn't they automate such and such-the answer was never "I like to do it by hand." It was always
one of the following: (1) "I didn't know that it could be automated", (2) "I didn't know that such tools existed",
or worst of all, (3) "Management wouldn't give me the time to learn how to use the tool."

13. Honest.

Testers are fundamentally honest and incorruptible. They'll compromise if they have to, but they'll
righteously agonize over it. This fundamental honesty extends to a brutally realistic understanding of their
own limitations as a human being. They accept the idea that they are no better and no worse, and therefore
no less error-prone than their programming counterparts. So they apply the same kind of self-assessment
procedures that good programmers will. They'll do test inspections just like programmers do code
inspections. The greatest possible crime in a tester's eye is to fake test results.Ê
Ê
*:)*
 *
 7üå ;

+ 

Testing is often looked upon by many as an unmanageable, unpredictable, unorganized practice with little
structure. It is common to hear questions or complaints from development such as:

-What is testing doing?

-Testing takes too long
-Testers have negative attitudes

Testers know that these complaints and questions are often unfair and untrue. Setting aside the
development/testing debate, there can always be room for improvement. The first step in improving strategy
and turning a test team into a higher performance test team is getting a grasp on where you are now! You
want to know:

-What testing is effective?

-Are we testing the right things at the right time?
-Do we need a staffing upgrade?
-What training does our team need?
-How does the product team value the test effort?

In this article we provide a framework for assessing your team, including: how to plan for an assessment,
how to execute the assessment and judge your current performance, what to do with the information, and
how to chart an improvement plan toward higher performance.

* *
ü





The goal of doing a test process assessment is to get a clear picture of what is going on in testing, the good
things, the problems, and possible paths to improvement. Fundamentally, a test assessment is a data
gathering process. To make effective decisions we need data about the current test process. If done
properly; the assessment will probably cross many organizational and management boundaries.

It is important to note when embarking upon such an assessment that this effort is much larger than the test
team alone. Issues will arise over who owns quality as well as what is the goal of testing? It is also important
to note that a possible result of the assessment is that work may actually increase. There may be:

-More demands for documentation

-More metrics
-More responsibility for communication and visibility into testing

For such an assessment process to succeed requires:

-Executive sponsorship
-A measurement program
-Tools to support change
-An acceptance of some level of risk
-Avoidance of blaming testing for project-wide failures
-Commitment about the goal of testing
-An understanding of testing or quality assurance across the product team
-Responsibility for quality

$"
*
)
7)ü0

A test strategy has three components that need to work together to produce an effective test effort. We have
developed a model called SP3, based on a framework developed by Mitchell Levy of the Value Framework
Institute. The strategies (S) components consist of:

-People (P1) - everyone on your team

-Process (P2) - the software development and test process
-Practice (P3) - the methods and tools your team employs to accomplish the testing task

ü 
-ü7

ü The goals for this phase are to set expectations, plan the project, set
a timeline, and obtain executive sponsorship. The actions that occur in phase 1 include meeting with the
management of various groups, laying out expectations for the results of the process, describing the plan,
and establishing a timeline. The intended result is to obtain agreement on expectations and buy-in on the
assessment process and follow-up commitment for improvement. The phase 1 deliverable is a schedule and
a project plan.

+n phase 1 it is important to:

-Get executive buy-in

-Make a schedule and stick to it
-Give a presentation of what you are doing, why and what you hope to get out of it
-Make a statement of goals or outline of work as a commitment
-Make a scope document a pre-approval/budget deliverable

It is important to note up front that assessment is only the beginning of the process.

ü 
/7+o  The goal of phase 2 is to develop interview questions and surveys which
become the backbone of your findings. Actions in phase 2 include gathering documentation, developing
interview questions, and developing a test team survey. The result of this phase is that you will be ready to
begin your assessment using the documentation, interview questions, and test team survey. The
deliverables include complete development process documentation, interview questions, and the tester
survey.

Examples of the documentation to be collected include: SDLC documentation, engineering requirements

documentation, testing documents (test plan templates and examples, test case templates and examples,
status reports, and test summary reports). Interview questions need to cover a wide range of issues,
including (but not limited to): the development process, test process, requirements, change control,
automation, tool use, developer unit testing, opinions about the test team from other groups, expectation of
the test effort, political problems, communication issues, and more.

ü 
07

The goal of phase 3 is to conduct the interviews and develop preliminary findings.
Actions include gathering and reviewing documentation, conducting interviews, sending out and collecting
the surveys. As a result of this phase there will be a significant amount of material and information to review.

ü 
17ü
7

The goal of phase 4 is to synthesize all of the information into a list of findings.
Actions include reviewing, collating, thinking, forming opinions, and making postulations. The result of this
phase is that you will develop a list of findings from all of the gathered information, reviewed documentation,
interviews, and the survey. The phase 4 deliverable is a list of findings, collated survey answers, collated
interview responses, a staff assessment, and a test group maturity ranking.

The findings can be categorized into:

-People
- Technical skills
- Interpersonal skills
-Process
- Documentation
- Test process
-SDLC
- Practice
- Strategy
-Automation
- Environment
- Tools
More subcategories may also be developed to suit your needs.

ü 
27ü
 
 "9
"
8 
"
  The goal of
phase 5 is to present preliminary findings to executives and the project sponsor, and to obtain agreement on
the highest priority improvement areas. It is important in this phase to be prepared for a very different
interpretation of the findings than you perceived. The deliverable for phase 5 is an improvement roadmap.

ü 
37+"! " The goal of phase 6 is to establish goals with timelines and
milestones and sub tasks to accomplish the tasks agreed upon for improvement. The action of phase 6 is to
develop a schedule for implementation of the improvement plan. It is helpful at this point to get some aspect
of the project implemented immediately so people can see tangible results right away-even if they are the
smallest or easiest improvement tasks. The deliverable for phase 6 is implementation of items in the
roadmap for improvement according to the developed schedule.

$


A test strategy is a holistic plan that starts with a clear understanding of the core objective of testing, from
which we derive a structure for testing by selecting from many testing styles and approaches available to
help us meet our objectives. Performing an assessment helps to provide the "clear understanding" and
"understanding of the core objective of testing". Implementing the resulting roadmap for improvement can
help to substantially improve the performance of your software testing organization and help to solidify your
test strategy.

c)*
 ü9

%; )
 +
"

Many of us have experienced projects that drag on much longer than expected and cost more than planned.
Most times, this is caused either from inadequate planning (requirement collection and design) or from an
inordinate number of defects found during the testing cycle.

A major ingredient to reducing development life cycle time is to eliminate defects before they happen. By
reducing the number of defects that are found during your quality assurance testing cycle, your team can
greatly reduce the time it takes to implement your software project.

The key to reducing software defects is to hold regular inspections that find problems before they occur.
Below is a list of 5 Tips for Reducing Software Defects:

1. $ !<c 

- The best time to stop defects is before coding begins. As the
project manager or requirements manager begins collecting the requirements for the software, they should
hold meetings with two or more developers to ensure that the requirements are not missing information or
are not flawed from a technical perspective. These meetings can bring to surface easier ways to accomplish
the requirement and can save countless hours in development if done properly. As a rule of thumb, the
requirements should be fully reviewed by the developers before the requirements are signed off.

2. $ ü$ ! 

- Once coding begins, each programmer should be encouraged to conduct
weekly code reviews with their peers. The meeting is relatively informal, where the programmer distributes
source code listings to a couple of his/her peers. The peers should inspect the code for logic errors,
reusability and conformance to requirements. This process should take no more than an hour and if done
properly, will prevent many defects that could arise later in testing.

3. $ $ ! 

- Every few weeks (or before a minor release), the chief architect or
technical team leader should do a formal inspection of their team's code. This review is a little more formal,
where the leader reviews the source code listings for logic errors, reusability, adherence to requirements,
integration with other areas of the system, and documentation. Using a checklist will ensure that all areas of
the code are inspected. This process should take no more than a couple of hours for each programmer and
should provide specific feedback and ideas for making the code work per the design.

4.  !

- As inspections are held, someone (referred to as a scribe) should attend the
meetings and make detailed notes about each item that is found. Once the meeting is over, the scribe will
send the notes to each team member, ensuring that all items are addressed. The scribe can be one of the
other programmers, an administrative assistant, or anyone on the team. The defects found should be logged
using your defect tracking system and should note what phase of the life cycle the defect was found.

5. $%
- Collect statistics that show how many defects (along with severity and priority) are
found in the different stages of the life cycle. The statistics will normally show over time that when more
defects are resolved earlier in the life cycle, the length of the project decreases and the quality increases.


ü
)ü9
7)%

Ê
Ê

Ê
Ê 

Ê Ê
Ê Ê
ÊÊ
Ê
ÊÊÊ
Ê
Ê Ê 

ÊÊ
 Ê ÊÊ
ÊÊ

Ê
Ê ÊÊ 

ÊÊ
ÊÊÊ Ê
Ê ÊÊ

The key to efficient measurement is to first determine what goals you are trying to accomplish and what
problems you are attacking. Many organizations waste time and money by measuring more things than are
necessary. Before beginning a measurement strategy, determine the goals for your measurement. Here are
some common reasons for not delivering on-time and on-budget:

-Software coding efforts are not properly estimated

-Testing efforts are not properly estimated
-Software quality is poor, therefore the testing duration is longer than need be
-Customer changes impact the software project, thereby extending the project dates
-Attacking the Common Problems

)$  
½ü"

 
This problem normally arises due these issues:


Ê  - To properly estimate coding effort, you must create solid customer requirements.
The requirements must contain adequate detail to allow the programmers to create detailed designs. From a
measurement perspective, you should track the amount of time it takes to develop each customer
requirement. Track both estimated and actual hours so that you can use this information to improve future
estimates.
   Ê  - It is impossible to estimate coding effort without creating a detailed design. The detailed
design allows the developer to think through all the tasks that must be done to deliver each requirement.
From a measurement perspective, you should track the amount of time it takes to develop each detailed
design. Track both estimated and actual hours so that you can use this information to improve future
estimates.

*
 
½ü"


This problem normally arises due these issues:

-Ê  - Once the customer requirement and detailed design is created and estimated, the test leader
should create a detailed test plan that estimates the testing effort. This is done by thinking through the test
cases that will be created based on the requirement and design. From a measurement perspective, you
should track the amount of time it takes to develop each test plan. Track both estimated and actual hours so
that you can use this information to improve future estimates.

)

ü
This problem normally arises due these issues:

-
Ê
Ê
 - If regular code reviews are not done, there is a much higher chance of delivering
software with poor quality. For large projects, these problems are compounded over time, so it is best to do
code reviews early and often (at least weekly). From a measurement perspective, you should track the
amount of rework time required due to failed code reviews. This can aid you in planning for rework on future
projects.
-  ÊP
Ê - By running weekly smoke tests, you can shorten the testing phase as issues are
caught early in the coding and testing cycle. From a measurement perspective, track the number of test
cases passed and failed during smoke tests, week by week. The goal is to reduce the number of failed test
cases as the project progresses.
- Ê  - As testing commences, track the number of open defects vs. total defects to help predict
project release dates. Track the number of defects found during code reviews vs. test case execution. This
will help track and improve estimation accuracy. Track the percentage of total defects before product
release, as this will help assess product quality.

$
$  
+" )ü9
This problem normally arises due these issues:

-
Ê Ê

Ê
 - As a project progresses, clients sometimes ask for features to be
changed or for features to be added or removed. Before making any changes to the project, each request
should be thoroughly investigated and a risk assessment should be done for each request. If changes are
necessary and agreed upon by the client, project timelines are adjusted. From a measurement perspective,
track the number of change requests, how many were approved vs. rejected, and the effort for estimating
reviewing and assessing each change request. This information can be used in future projects to predict the
number number of change requests that are approved and estimated as to build time into your projects to
mitigate that risk.

*"-.½  *
$



 ÊÊ Ê ÊÊ
ÊÊÊ

ÊÊ
ÊÊ
Ê
ÊÊ
ÊÊÊ Ê
 ÊÊ
Ê Ê
Ê
ÊÊ
Ê?

Ê ÊÊ
Ê Ê ÊÊ Ê
Ê
 Ê
Ê
ÊÊ

ÊÊ


-=  )  - Most SQL based database systems have issues when users store
information that contain a single quote (e.g. John's car). For each screen that accepts alphanumeric data
entry, try entering text that contains one or more single quotes.

/=!< 
- Your functional specification should clearly indicate fields that require data entry
on screens. Test each field on the screen that has been indicated as being required to ensure it forces you
to enter data in the field.

0= *
"*
 - Your functional specification should clearly indicate fields that require specific data
entry requirements (date fields, numeric fields, phone numbers, zip codes, etc). Test each field on the
screen that has been indicated as having special types to ensure it forces you to enter data in the correct
format based on the field type (numeric fields should not allow alphabetic or special characters, date fields
should require a valid date, etc)

1= );*
 - Your functional specification should clearly indicate the number of characters you can
enter into a field (for example, the first name must be 50 or less characters). Write test cases to ensure that
you can only enter the specified number of characters. Preventing the user from entering more characters
than is allowed is more elegant than giving an error message after they have already entered too many
characters.

2=½
*
- For numeric fields, it is important to test for lower and upper bounds. For
example, if you are calculating interest charged to an account, you would never have a negative interest
amount applied to an account that earns interest, therefore, you should try testing it with a negative number.
Likewise, if your functional specification requires that a field be in a specific range (e.g. from 10 to 50), you
should try entering 9 or 51, it should fail with a graceful message.

3=½#
*
- Most database systems and programming languages allow numeric items to be
identified as integers or long integers. Normally, an integer has a range of -32,767 to 32,767 and long
integers can range from -2,147,483,648 to 2,147,483,647. For numeric data entry that do not have specified
bounds limits, work with these limits to ensure that it does not get an numeric overflow error.

4=
*
- For date fields, it is important to test for lower and upper bounds. For example, if you
are checking a birth date field, it is probably a good bet that the person's birth date is no older than 150
years ago. Likewise, their birth date should not be a date in the future.

5=' 
- For date fields, it is important to ensure that invalid dates are not allowed (04/31/2007 is
an invalid date). Your test cases should also check for leap years (every 4th and 400th year is a leap year).

6=c)

*
 - Many web applications rely on the browser session to keep track of the person
logged in, settings for the application, etc. Most screens in a web application are not designed to be
launched without first logging in. Create test cases to launch web pages within the application without first
logging in. The web application should ensure it has a valid logged in session before rendering pages within
the application.

-.=ü$  
- As you release new versions of your product, you should have a set of
performance tests that you run that identify the speed of your screens (screens that list information, screens
that add/update/delete data, etc). Your test suite should include test cases that compare the prior release
performance statistics to the current release. This can aid in identifying potential performance problems that
will be manifested with code changes to the current release.

*
 o,+Ê

o Êo Ê
Êo
ÊÊ
Ê Ê

Ê Ê  
Ê
Ê
To test any Windows based application following points are to be considered:

- It is essential to have GUI consistency within the Application.

- Should be alike in look and feel as per any other standard Window software.
- Should have standard set of keys implemented for the software.
- Should have clean and neat exit.

While testing any Windows based application the testing can be broadly categorized into following
compartments. They are:
- Standardization Testing.
- GUI Testing.
- Validation Testing.
- Functionality Testing.

)  ;*


This compartment mainly emphasizes on the standardization part of the application. Standardization means
that the application being developed should have standard look and feel like any other window application.
The general guidelines are as follows:
1. The application should have first "About Application" screen displayed.
2. Most of the screens/ dialog box (as on context) should have Minimize, Restore and Close clicks.
3. Proper icon should be attributed to the application
4. All screens/ dialog box should have a proper caption as per the context used.
5. The application should be seen in the Windows Task Bar as well as status bar.

o,+*

This compartment mainly emphasizes on the GUI - Graphics User Interface aspect of the Application. It is
not concrete that once GUI guidelines are set that can be followed blindly. GUI standards may vary from
company to company and also from application to application. But still one can set general guidelines to
have an overall idea on how to start GUI testing. These guidelines apply for every screen/ dialog box of the
application. General guidelines are:
1. All the dialog box should have a consistent look through out the Application system. For e.g.- If the
heading within a dialog box is blue then for each dialog box the heading should be of this color.
2. Every field on the screen should have an associated Label.
3. Every screen should have an equivalent OK and cancel button.
4. The color combination used should be appealing.
5. Every field in the dialog box should have a Short Cut Key support. For e.g.- User Name
6. Tab order should be normally set horizontally for the fields. In some case as per the case the Tab Order
can be set vertically.
7. Mandatory fields should have * (RED ASTERIK) marked to indicate that they are mandatory fields.
8. Default key <Enter> should be set as OK for the dialog box.
9. Default key <Esc> should be set as Cancel for the dialog box.
' *

This compartment mainly emphasizes on the Validation aspect of the Application. Validation testing mainly
depends on the fields set in the dialog box and the functions it has to perform. But still there are certain
common rules that can be applied. General guidelines are:
1. For text box fields where value entered has to be numeric check following:
¨ It should accept numbers only and not alphabets.
¨ If field usage is such that for e.g., To accept
Total number of days
Telephone number
Zip code etc.
then it should not accept 0 and negative values.
2. For text box fields where value entered has to be alpha-numeric check following:
¨ It should accept alphabets and numbers only.
¨ If field usage is such that for e.g., accepting
First Name
Middle Name
Last Name
City
Country etc.
then field value should start with an alphabet only.
¨ Depending on the condition this fields may accept special characters like -, _, . etc.
3. If the field is a combo box then it has to be checked for following points:
¨ Check the combo box has drop down values in it, it is not empty.
¨ Drop down values should be alphabetically sorted. This might change as per requirement but as standard
practices it should be alphabetically sorted. For e.g. to select data type from the list it will be as follows:
Date
Integer
String
Text, etc.
¨ Selection of any drop down value is displayed on closing and opening the same dialog box.
¨ By default some value like "Select Value" or "_______" string is displayed. This is because User comes to
know that value is to be selected for this field. Avoid displaying the first default value in the list.
4. If the field is a list box then it has to be checked for following points:
¨ Check the list box has values in it, it is not empty.
¨ List box values should be alphabetically sorted and displayed. This might change as per requirement but
as standard practices it should be alphabetically sorted.
¨ Selection of any list box value should put a check before the value and should display the correct value(s)
selected on closing and opening of the same dialog box.
¨ If the list box supports multiple selection then check whether multiple values can be selected.
5. If the field is a list of radio button then it has to be checked for following points:
¨ Check whether as per requirements all the values are listed. For e.g. to select date format. Possible values
displayed will be as follows:
mm/dd/yyyy
dd/mm/yyyy
mm/dd/yy
dd/mm/yy
yyyy/mm/dd etc.
¨ Same selected value should be displayed on closing and opening of the same dialog box.
6. Data Controls are to be tested as part of functionality testing.


*


This compartment mainly emphasizes on the Functionality aspect of the Application. The first step to test the
Functionality aspect of the Application is to check whether all the requirements are covered in the software.
The actual functionality testing totally depends from software to software. Still one can frame general
guidelines. General guidelines are:
1. Check the functionality is covered as per Requirement specifications or Functional specifications
developed for the software.
2. Within a dialog box identify the dependent fields. Depending on the dependency check the enabling and
disabling of the fields. For e.g.: to create Contact addresses in any application. To create contact addresses
user should be able to add, delete and modify the information. Contact Addresses will contain information
like, First Name, Last Name, Address1, Address2, City, State, Country, Zip, Phone, etc., any other
information may also be added.
This form will have the required fields and in addition to that will have Add, Delete and Update buttons. The
functionality of the buttons is as follows:
¨ Initially only Add button will be enabled. Delete, Update buttons will be disabled. This is because initially
there is no data available and unless one adds one cannot delete or update. In short, unless there is a single
valid record available it is not possible to update or delete.
¨ Only on selecting a record from the list Delete and Update buttons are enabled and Add button is disabled.
By default No records will be selected.
¨ Delete and Update should always give confirmation message before actually performing the operation.
¨ Delete operation should not show the deleted item in the list.


ü
)ü9
7!
% 


Ê Ê

Ê
 Ê Ê
Ê Ê 
Ê  Ê Ê Ê

Ê

ÊÊ
Ê Ê Ê ÊÊ
 Ê ÊÊÊ 
Ê Ê ÊÊ Ê
Ê
 Ê
ÊÊ Ê Ê Ê
Ê 
ÊÊÊ 
Ê
Ê
Ê
Ê

Ê 
 - There are many risks that you know about, that you can mitigate. For example, let's
assume that you have assembled a team to work on the project and one of the stellar team members has
already scheduled a 3 week vacation just before testing is scheduled, which you agreed to allow. The
successful project manager will identify this risk and provide some contingency plans to control the risk.

· Ê
Ê
Ê

Ê 
 - There are also risks that you don't know about, so a general risk assessment
must be done to build time into your schedule for these types of risks. For example, your development
server may crash 2 weeks into development and it may take you 3 days to get it up and running again.

The key to managing risks is to build contingency plans for risk and to build enough time into your project
schedule to mitigate risks that you do not know about. Below are a list of the 5 most common scheduling
risks in a software development project:

1. Scope and feature creep - Here is an example: Let's say the client agrees to a requirement for a Logon
page. The requirement specifies that the client will enter their userid/password, it will be validated and will
allow entry upon successful validation. Simple enough. Then in a meeting just before coding is commencing,
the client says to your project manager "I was working with another system last week and they send the
client a report each day that shows how many people log in each day. Since you have that information
already anyway, I'm sure it will only take a couple of minutes to automate a report for me that does this."
Although this sounds simple to the client, it requires many different things to happen. First, the project
manager has to amend the requirement document. Then the programmer has understand the new
requirement. The testing team must build test scenarios for this. The documentation team must now include
this report in the documentation. The user acceptance team must plan to test this. So as you can see, a
simple request can add days of additional project time, increasing risk.

2. Gold Plating - Similar to scope and feature creep, programmers can also incur risk by making the feature
more robust than is necessary. For example, the specification for the Logon page contained a screen shot
that showed very few graphics, it was just a simple logon process. However, the programmer decides that it
would be really cool to add a FLASH based movie on the page that fades in the names of all the
programmers and a documentary on security. This new movie (while cool in the programmer's eyes), takes
4 hours of additional work, put their follow-on tasks are n jeopardy because they are now behind schedule.

3. Substandard Quality - The opposite of Gold Plating is substandard quality. In the gold plating example,
the programmer got behind schedule and desperately needed to catch up. To catch up, the programmer
decided to quickly code the next feature and not spend the time testing the feature as they should have.
Once the feature went to the testing team, a lot of bugs were found, causing the testing / fix cycle to extend
far beyond what was originally expected.

4. Unrealistic Project Schedules - Many new team members fall into this trap. Project members (project
managers, developers, testers, etc), all get pressure from customers and management to complete things in
a certain time frame, within a certain budget. When the timeframes are unrealistic based on the feature set
dictated, some unseasoned team members will bow to the pressure and create their estimates based on
what they think their managers want to hear, knowing that the estimates are not feasible. They would rather
delay the pain until later, when the schedule spirals out of control.
5. Poor Designs - Many developers and architects rush the design stage in favor of getting the design
behind them so the "real" work can begin. A solid design can save hundreds of programming hours. A
design that is reusable, allows changes to made quickly and lessens testing. So the design stage should not
be rushed.

$ !

The key to successful risk management is to identify all risks you know about and build time in for risks you
do not know about. The mechanics of doing this is to simply list the risks, and figure out the loss (in hours)
you expect would happen if the risk occurs, and then list the probability of the risk happening. Then you can
factor that risk into your project schedule by multiplying the loss by the probability.
For example, let's assume that you have decided to use a new outsourcing company to do some of the
coding for your project. Since you have not worked with them before, you surmise that there is a 50%
probability that you could incur 40 hours of additional development because of the new resource. You would
build 20 additional hours (40 hours * 50% probability) into your project plan to mitigate the risk.

ü  




Ê
Ê

  
Ê 
ÊÊ Ê Ê
ÊÊ Ê Ê 
Ê
ÊÊÊ
 ÊÊ
Ê
  ÊÊ Ê
ÊÊ 
Ê
Ê

*
üÊ

Description: Testing strategy and approach to testing QA will use to validate the quality of the product prior
to release.
A test plan provides the following information:
General description of the project, its objectives, and the QA test schedule.
Resource requirements including hardware, software and staff responsibilities.
Features to be tested, as well as features not to be tested.
Details for the test approach.
Lists of test deliverables such as test cases and test scripts.
References to related test plans (which focus on specific topics) and project documentation.
Dependencies and/or risks.
Descriptions of how bugs will be tracked
Milestone criteria.
Lists of required reviewers who must provide approval of the test plan.

*
""


-=+ Ê


" 

This document is a Test Plan for the -Project name-, produced by Quality Assurance. It describes the testing
strategy and approach to testing QA will use to validate the quality of this product prior to release. It also
contains various resources required for the successful completion of this project.

The focus of the -Project name- is to support those new features that will allow easier development,
deployment and maintenance of solutions built upon the -Project name-. Those features include:Ê

[List of the features]Ê

This release of the -Project name- will also include legacy bug fixing, and redesigning or including missing
functionality from previous releaseÊ

[List of the features]ÊÊ

The following implementations were made:Ê

[List and description of implementations made]ÊÊ

! 
Ê

[List of related documents such as: Functional Specifications, Design Specifications]Ê

)   %

Ê

[Schedule information QA testing estimates]

/=!
!<
Ê

 Ê

[List of hardware requirements]Ê

)Ê

[List of software requirements: primary and secondary OS]Ê

*
*

Apart from manual tests, the following tools will be used:Ê

) Ê

!
"


[List of QA team members and there responsibilities]Ê

* 
[List of training's required]Ê

0=
**
 &*
"" Ê

[List of the features to be tested]Ê

% '

[The process will include installing all possible products from the media and subjecting them to basic sanity
testing.]Ê

1=
½**
 Ê

[List of the features not to be tested]Ê

2=*
 
Ê

[List of the test cases/matrices or there location]Ê

[List of the features to be automated ]ÊÊ

3=" 
&!

Ê

" 
Ê
!



4=%
$ Ê

*
$
$ 
 (%
)

Description: An outline or matrices of test cases that test a feature, or set of features.

*
%
)"



I need information about Metrics, which is used to find faults. it is something related with measurement. I
need to know how it is used in Quality Assurance.



You can measure the arrival and departure times of developers, if you have them clock in, but that won't tell
you much since not all work is done in the office (and it doesn't mean that they're working when they're in
the office). This is, however, still a metric.

The same holds for a true "quality metric". The most familiar one is defects per thousand lines of
(uncommented) code. But this metric assumes that:
1) you count the lines of code
2) the complexity of the code isn't an issue
3) the programmers aren't playing games (like using continuance characters
so that what could have been written in one line isn't done in five lines)
4) all defects are uncovered in the code in a single pass
5) each defect discovered is all others
6) defects are uncovered in a linear manner between revisions or builds

The fact is that first you need to know what your goal is. Then you need to
discover or create a metric that will help you achieve that goal. Then you
need to implement it and be prepared to adjust it.

You can't use measurements (metrics) to find faults, at least not in software, so that's not a reasonable goal.
You can use metrics to help determine if most of the defects have been discovered already. You can use
them to tell you how much longer it will take to uncover a reasonable amount of defects. For either of these
metrics you will need to know how previous projects of similar size and complexity (using similar languages,
etc.) were done in order to get a reasonable comparison.

Posted by Walter Gorlitz

*
$
 *
 *
$

& ü

& ½=
 >ÊÊ $
ÊÊ
å">Ê 
"ÊÊ )"
Ê ÊÊ 
ÊÊ
Setup for [Product
N/AÊÊ setupÊ -ÊÊ -ÊÊ -ÊÊ ÊÊ
Name]Ê
Test that file types
supported by the
1.1ÊÊ 1.1ÊÊ P/FÊÊ #ÊÊ #ÊÊ ÊÊ
program can be
openedÊ
Verify all the different
ways to open file
1.2ÊÊ 1.2ÊÊ P/FÊÊ #ÊÊ #ÊÊ Ê
(mouse, keyboard and
accelerated keys) Ê
Verify files can be
open from the local
1.3ÊÊ 1.3ÊÊ P/FÊÊ #ÊÊ #ÊÊ Ê
drives as well as
networkÊ
*
$


Description: A specific set of steps and data along with expected results for a particular test objective. A
testcase should only test one limited subset of a feature or functionality. A test case does the following:
Details test setup.
Details test procedure (steps).
Describes testing environment.
Specifies the pass/fail criteria.
References results of test.

)"*





I entered the company first day, suppose the project asked me to write test case on what basis should I write
the test case and how should I write it.Ê



I will bullet them in points for you:
1. You need to have a thorough understanding of the application that the
project is working on QC, testers need to have a complete understanding of the project.
2. You need to have a signed off Business requirements from the BA (business analysts or business
development team)
3. Basing these as bench marks you will start writing test cases.
4. Some of them prefer writing each business requirement as a seperate testcase and some companies
don't do it that way.
5. Try to talk to any other QC in the same project or other project to understand the process and procedures
they have to write test cases or test plans or all the other stuff.Ê



The process of writing a test case is easy. Only thing is that you need to understand the concept given in
your module's FSD (Functional Specification Document). A module is just a part of the project and the FSD
is a description of your module!!Ê

Now coming to the test case writing, take a simple example, your windows calculator where you are going to
test only one operation which is adding 2 numbers. Here you need to assess the situation and your
imagination should be on the right track as you wont get your product or software beforehand.Ê

In case of testing the calculation 1 + 1 = 2, you assume that your windows

calculator has 4 buttons and a text box / display for giving you the answer.
Again assume that you have the buttons 1, 1, + and a =. So if you give 1 the first time, it should be displayed
on the text box! Again you need to press the operator i.e. + and press the second input 1, which should
appear on the display and then press =. The desired output you should get is 2.
This is what will be given in the FSD. And moreover they would also give you some fine details about how
the process takes place which is not testable. Now the stuff in the second paragraph will be given in your
FSD in a refined language.Ê

So first you check when we press 1 do we see it on the screen.

Test case should be:
"Verification of display of 1 on the screen when the input was given or when the button was clicked"
and similarly for the second input:
"Verification of display of 1 on the screen when the input was given or when the button was clicked"
Now comes the result:
"Verification of display of to on the screen when the = button was pressed"

In some FSDs you would be given about a warning when you give non-numerical values like a, b instead of
1..
If its given, well you can strait away write another test case "Verification of warning message stating "invalid
input (or anything given as per FSD) on the screen when the input given is a character (specifically: A)". If its
not given you can note it in your testware (excel sheet) as an observation.Ê

When you are writing a test case please check you test whatever is given in your FSD, Business
Requirement.Ê



No one would expect you to write a usable test case the first day. You must spend time learning the
application and reading the requirements etc. This will take several weeks. Then you would determine what
test cases have already been written so as not to duplicate efforts. You should also find out
what is expected of you from your manager .
To write the test cases, usually there are test case standards imposed by the organization. The main
components are the description, the setup conditions, the objectives of the test case, the actual steps which
should be as simple as possible describing how to input something and what is expected as an output. Then
a Pass/Fail statement and maybe a comments area describing what you think caused the failure if it failed.Ê

File Open - Test caseÊ

Steps to reproduce:
1. Launch Application
2. Select "File" menu
File menu pulls down
3. Choose "Open"
"Open" dialog box appears
4. Select a file to open
5. Click OK
Result: File should openÊ

Test caseÊ

Test case ID: B 001

Test Description: verify  - bold formatting to the text
Revision History:
3/ 23/ 00 1.0- Valerie- Created
Function to be tested:  - bold formatting to the text
Environment: Win 98
Test setup: N/A
Test Execution:
1. Open program
2. Open new document
3. Type any text
4. Select the text to make bold.
5. Click 
Expected Result: Applies bold formatting to the text
Actual Result: "


Ê

c
)
!"
Description: Use to inform management regarding latest issues that arise during the software development
lifecycle on weekly basis.Ê

À
 Ê

ÊÊÊ 
ÊÊÊÊ  Ê ÊÊÊÊÊÊ
Ê

* !
ü 

Ê  Ê  ÊÊÊ Ê
 ÊÊ  ÊÊÊÊ
 
Ê
Ê Ê 
Ê
 Ê
Ê Ê  Ê
 
Ê Ê  Ê  Ê
 Ê ÊÊ Ê Ê 
ÊÊÊ
 Ê Ê
 ÊÊ


Ê
 
Ê
Ê
ü9å   - This is a synopsis of the project, its scope, any problems encountered during the testing
cycle and QA's recommendation to release or not release. The overview should be a "response" to the test
strategy and note areas where the strategy was successful, areas where the strategy had to be revised etc.

The project overview is also the place for QA to call out any suggestions for process improvements in the
next project cycle.

Think of the Test Strategy and the Project Overview as "Project bookends".

· ü9ü! - This is the Product Requirements Analysis Document, which defines what functionality
was approved for inclusion in the project. If there was no PRAD for the project, it should be clearly noted in
the Project Overview. The consequences of an absent PRAD should also be noted.

· )" - The document that defines how functionality will be implemented. If there were
no functional specifications, it should be clearly noted in the Project Overview. The consequences of an
absent Functional Specification should also be noted.
· *
)
- The document outlining QA's process for testing the application.

· !

)
- The results summaries identify the results of each round of testing (see section VI -
Results by Build). These should be accompanied in the Release Package by the corresponding reports for
Test Coverage by Test Type and Test Coverage by Risk Type/Priority from the corresponding completed
Test Matrix for each build. In addition, it is recommended that you include the full Test Matrix results from
the test cycle designated as Full Regression.

· ?+


 - This document is primarily for Technical Support. This document identifies
workarounds, issues development is aware of but has chosen not to correct, and potential problem areas for
clients.

· +
+
 - If your product must be installed as the client site, it is recommended to include
the Installation Guide and any related documentation as part of the release package.

· å"
- The list of defects remaining in the defect tracking system with a status of Open. Technical
Support has access to the system, so a report noting the defect ID, the problem area, and title should be
sufficient.

·  
- The list of defects remaining in the defect tracking system with a status of deferred.
Deferred means the technical product manager has decided not to address the issue with the current
release.

· ü  
- The list of defects remaining in the defect tracking system with a status of pending.
Pending refers to any defect waiting on a decision from a technical product manager before a developer
addresses the problem.

· 8 
- The list of defects waiting for verification by QA.

· $
 
- The list of defects verified as fixed by QA during the project cycle.

The Release Package is compiled in anticipation of the Readiness Review meeting. It is reviewed by the QA
Process Manager during the QA Process Review Meeting and is provided to the Release Board and
Technical Support.

· ! 

! % :
The Readiness Review meeting is a team meeting between the technical product manager, project
developers and QA. This is the meeting in which the team assesses the readiness of the product for release.
This meeting should occur prior to the delivery of the Gold Candidate build. The exact timing will vary by
team and project, but the discussion must be held far enough in advance of the scheduled release date so
that there is sufficient time to warn executive management of a potential delay in the release.
The technical product manager or lead QA may schedule this meeting.

· ü

! % :
The QA Process Review Meeting is meeting between the QA Process Manager (Barbara Thornton) and the
QA staff on the given project. The intent of this meeting is to review how well or not well process was
followed during the project cycle.
This is the opportunity for QA to discuss any problems encountered during the cycle that impacted their
ability to test effectively. This is also the opportunity to review the process as whole and discuss areas for
improvement.
After this meeting, the QA Process Manager will give a recommendation as to whether enough of the
process was followed to ensure a quality product and thus allow a release.
This meeting should take place after the Readiness Review meeting. It should be scheduled by the lead QA
on the project.

· !
 % :
This meeting is for the technical product manager and senior executives to discuss the status of the product
and the teams release recommendations. If the results of the Readiness meeting and QA Process Review
meeting are positive, this meeting may be waived.
The technical product manager is responsible for scheduling this meeting.
This meeting is the final check before a product is released.