Why Test the Web?

How Much Should You Test?

A ratio of 1:1 quality specialists to software developers is fact in some places, not a pipedream.
But 1:10 is sometimes found, too. Just how much effort should be put into quality assurance and
testing of a Web site? The answer is vital to the testers who need to prioritize their efforts to
check everything immediately. It is essential to the Web test planner who needs to balance
competing demands. And the Quality Assurance Manager must factor in test activities into the
broader set of efforts to produce high-quality results for their customers. This article is the third
in a series of articles about Web testing in the Testers’ Network. The first two articles, "Testing
Your Web site" and Testing Considerations for Web-enabled Applications", introduced and
discussed testing consideration for applications and Web sites, and they concentrated on the
technical test subjects and techniques for testing Web-based applications. This article refers to
some of the same topics for review and summary, but frames them in the viewpoint of the e-
business venture as a whole. The series of articles together give the tester or test manager the
background necessary to carry forward on two fronts: the competition for resources and
attention within the company and the competition to do the best possible test job with a given
technical configuration.

Business Needs At the technical level, most of the same technologies are available to
implement Web sites for Internet, extranet or intranet usage; the major difference may be only
whether the viewing audience is respectively public; a selected subset using public networking;
or users in the same enterprise, WAN, or domain. However, if QA and test practices are based on
business needs, there can be huge differences. An Intranet as a convenience among cooperating
friends can be a user-beware affair, casually produced and administered, where gross mistakes
are ignored or fixed by the local community according to their needs and time availability. But
poor service across a more formal interface can injure a relationship. A high standard for Web
quality and test practices can serve and enhance the business function. A customer on your Web
site is only a click away from gone. Any venture to use Web technology in business needs to
choose how to ensure and control the quality of their Web service. Today, most examples closely
fit one of the following: e-Commerce, such as a "dot-com" company just starting up with yet
another business model e-Business, such as a "bricks and mortar" company with an ongoing
business activity, adding a Web presence to communicate and conduct business with direct
customers or business partners and suppliers Web-enabled software, where a function
previously available only as an installed program or on a client-server network, is now being
served to users via Web technology For all of these cases, the rich and fast-changing set of
languages, development tools, browser and server software and techniques are subject to
failures just like any other software. Ensuring the appropriate level of service requires both
quality assurance and testing.

Why test? Quality assurance and testing of a business service that uses a Web site is driven by
the business needs of the sponsors and the technical practicalities of the implementation. The
business goals with respect to the Web site user are reliable, effective communication,
interaction, and satisfaction. A Web site is a valid Universal Resource Locator (URL) with the
means to provide via http or https protocols the HTML, XML, and supporting objects and
functions to serve a client-side interactive browser. Quality assurance concerns itself with the
business content and policies of the Web presence, off-line business practices, and development
and review practices in preparation of the Web site, as well as testing of the artifacts and
implementation of a given "issue" of the Web site. Any evaluation is easier to conduct and
explain if the reference criteria are explicit. It’s no surprise to software testers, however, that
much of the test criteria for the fast-changing Web arena are based on experience and common
sense – decide what is there, measure it against previous experience and the known desires of
the sponsor, and report aberrations. Business practices and policies are likely to be more fully
stated in older companies that have had a chance to mature; startups are developing those
policies on the fly. External standards are more rare. One approach is to choose some respected
site as a style reference. A second way is to choose from standards that already exist. An
example of that is the "Standard for Internet Commerce", which in draft form attempts to define
an opinion as to business functions that should be present for the customer in areas such as
disclosure of policies on payment and refunds, a customer’s information privacy, order
confirmation, product availability, and customer service. All are testable if adopted as standards.
A third approach is to develop a "style guide" as you go – questionable examples are collected as
they are resolved, and the borderline of your accepted practice builds day-by-day. The fourth
approach can start immediately, and continue along with any of the others. This approach tests
for mechanical errors as summarized below. Web site production practices that affect the quality
of the Web site service include publication schedules, staffing, roles and responsibilities, tools
qualification and usage, and various reviews from business practices to design to code inspection
to on-line testing. The goal is to prevent any rude or careless-seeming slips. We don’t want the
customer, about to place an order, to be able to click on a spot that takes them to the dreaded
HTTP Error 404 Not Found!

What should you test? Testing the mechanics of the Web site can find errors in usability,
functionality, compatibility and performance. Usability Conventionally, usability spans concerns
of usefulness, ease of use, and ease of learning. These qualities are somewhat abstract qualities.
Practical testing usually concentrates more on issues such as: Navigation: Is it clear how to get
around the various pages of the site? Are adequate means and markers provided? Consistency:
Input validation, standard controls, and styles of presentation should follow at least the local
standard when possible, to help the user learn and perform effectively. Functionality Each
control and possible action is a possible failure point, and should be covered by thorough testing.
This includes: Navigation: Does every navigation function work? Basic Operations: Check each
control and command and combine with navigation. Business Operations: Are the business
functions supported by the Web site? Do such functions as enrollment, sign-on, search and
transactions operate with normal and exceptional data? Error Conditions: Are input errors and
illicit values detected and deflected in a helpful manner? Compatibility Web sites for the general
public will be accessed by any of the last several releases of Netscape Navigator, Internet
Explorer, and America Online browsers running on one of several operating systems. Special-
purpose Web sites now and in the future might be browsed by a new generation of devices, from
WebTV devices to point of sale or warehouse terminals. The browsers differ in their capability to
support scripting, components and styles, for example. The main lesson of compatibility testing
is that the combinations of operating systems and browsers popular with your users, not only
your developers, need testing on your Web site. Performance Providing adequate performance of
Web sites is challenging from both the supply and demand sides. We can’t guarantee fast
response over a public Internet when the intermediate nodes and channels aren’t under our
control. The best we can do is supply good performance at our server sites. The demand for
service is typically uncontrolled, too; so, planning for the possibility of hugely popular usage
befits any site that doesn’t want to join the list of famous brownouts such as eTrade, eToys, and
eBay. Stages in performance testing include: Concurrency: Do the business operations work
properly when more than one user is operating at the same time? (For example, what if the
same customer tries to operate on their account from more than one browse session at a time?)
Response time Performance with increased load Capacity: Peak loads Stress: Overload failure
models

How much is enough? When do we stop testing? In an understaffed test group, the project
usually answers that for us – we stop when it’s time to "ship" – in the Web world, we stop for
deadline. And, sometimes, restart immediately for the next publication schedule. A responsible
answer is, we test to the quality goals of the project, and we release to the business criteria.
Priorities of testing are based first on efforts of risk reduction, then on coverage for quality goals.
We use evidence from testing as well as prior QA activities in estimating how "good" the
software is, and in deciding when to release. If your Web site production organization demands a
regular, short interval between new releases, then development of complex functions may need
to be staged over several release cycles. QA and test results will help determine what release
cycle for which it is ready.

Conclusion As this series of articles shows, testing Web software is much like other software
testing, but with special motives, techniques, and criteria. The objective of a quality program in
e-Business is to satisfy the customer efficiently. The perpetual software conflict between soon
and right continues. The ideal, of course, is "perfect, free and now!"

References Gary A. Anthes, "The Quest for e-Quality," Computerworld, December 13, 1999, 46-
47. Eric Kaufman, "Testing Your Web site," Testers’ Network, November 1999. See the archives
at http://www.data-dimensions.com/testersnet/archive.htm Gerry Ocampo, "Testing
Considerations for Web-enabled Applications," Testers’ Network, September, 1999. See the
archives at http://www.data-dimensions.com/testersnet/archive.htm "The Standard for
Internet Commerce," a draft standard by GII found under
http://www.gii.com/standard/index.html To benefit from Data Dimensions' expertise, please
call 877.DIAL.DDI (877.342.5334) or send e-mail to info@data-dimensions.com Copyright Home