Presenter: Byron Goodman

National Director of Quality

Assurance
Neudesic

 Some

thoughts on automation
Successful test automation
Components of a framework
Some coding standards
Standard routines

 Let

the machines do the tests

Easy to create alternate scenarios by
leveraging existing tests
Using tools can create more interest in
testing activities
Potential to reduce testing cycle time
Effective at catching regression bugs

 Tests

break too often!

Pass or fail results may not be reliable
Tools are expensive, although there are
some good open source solutions
Tools may oversimplify or overcomplicate
the degree of effort required to test
applications

 More

time spent working on automated

tests than time needed to do the manual
testing
Sporadic interest in implementing
automated testing
Statements by others that automated
testing is: a sham, ineffective, waste of
time
Shelving of test automation in the interest
of project success

 As

with all project related work, the time

spent creating automated testing must
offset this effort by the return of value. This
is demonstrated in various ways:
Less time expended during the testing cycle than

if manual testing is the only solution

More thorough and consistent regression
coverage
Reduce human error and boredom
Frees up people to test new functionality while
computers test existing functionality

 start simple and work towards a

more complete and complex solution
Timing start using tests as soon as they
are written, Do not wait for a completed
suite
Buy-in Demonstrate to others that each
test does exactly what it is intended to so
that there is confidence in results
Traceability Each automated test must
trace back to a specific automated test
Strategy

 Generally

speaking, some automated

testing is better than no automated testing
Select the most simple of tests to
automate
Begin by using record and playback to
minimize time investment but to show that
the tool does prove to be effective

 For

years, I have insisted that no tests will

be created using record and playback
I will admit that there is a place for R&P
Very light use to prove out the tool and to
minimize the reluctance people may have
to use complicated technology
However, R&P is does not have much gain
in efficiencies over manual testing

 Many

of us have experience with testing

frameworks
Other names DLLs, namespace, test
library
Its just a fancy name for code that is used
to automate tests
A testing framework is a set of routines
that are reused by tests that will speed up
test development while increasing test
reliability

 Needs

to be recognized to have the same

rigor as any development effort
Adds value to the overall test effort
Implements complex and redundant
routines for tests to leverage
Minimizes the impact of a changing
application to the tests, which reduces test
maintenance

 Each

layer is developed to promote

portability and reusability
The more generic a layer, the more
reusable it needs to be
i.e. the Generic Routines should be able to
work for everyone all the time regardless
of the environments and AUTs

Common Layer

Response Flow

Request Flow

Generic UI Layer
Technology Specific
Layer
Application Specific
Layer
Application Test Layer

 Common

routines that do not interact with

any application
Generic UI layer, non-technology-specific
manipulation routines
Technology-specific layer routines
Application-specific layer routines
Application-specific Scenario/transactional
layer routines
Tests layer

 Not

related to the application or an

interface
Build more capability and utility that the
test tool may not have
Functionality that a tester may need for
any reason
If written properly, these routines will work
with test tools that use the same
programming language

 Some

control types, like grids, that are

generic and rather independent from the
technology that they are written in
This is a good place to create additional or
alternate logging

handling specific technologies i.e.

Browsers, WinForms and WPF.
Custom controls will be wrapped with calls
that are identical to similar standard
controls
Create interfaces for new versions of
technology if the tool vendor does not yet
have the new technology versions
integrated
For

 Routines

that are specific to the AUT

Repository for UI mappings
Defines a handler for every individual
form/page
AUT service interfaces defined and
wrapped
Serves the calls needed for tests to
interact with the AUT

 Use

this as a distinguishable layer if the

routines for scenarios are complex and
extensive
May include multiple forms/pages
In cases where there will not be a lot of
routines, this can be combined into the
Application-specific manipulation routines

 This

layer will contain only tests

No test calls any other test
Tests call into the other layers
Tests should be very simple as all the
complexity is implemented in the other
layers

Common Layer

Response Flow

Request Flow

Generic UI Layer
Technology Specific
Layer
Application Specific
Layer
Application Test Layer

 Using

consistent coding standards

promoted reusability and maintainability
Standards need to be documented and
followed similar to what project
development would do
Use code reviews to enforce standards

 Naming

conventions for variables and

methods/functions
Use human readable names!

Limit

code complexity
Separation of layers
Comments

 Do

not use the control type in the name,

Address, not AddressEdit
No duplication of control names for any
specific window if the tools would normally
allow this
Consistent naming of similar controls form
to form, even if the forms label the controls
differently

anywhere tests need to written

so that they do not rely upon a very
specific environment configuration, unless
that is the purpose of the test
Unattended testing tests that need
human interaction to execute should be
segregated so that the other tests can all
be run unattended
Execute

hard coding use configuration files or

similar mechanisms to store information or
environment values
In tests, avoid using the automation tool
syntax
No

Tests should call into the other layers

This will make tests less likely to fail if the tool

vendor modifies their syntax

Can make it easier to port tests to new automation
tools

 Overuse

logging
Design for distributed testing and for
concurrent execution of tests
An automated test will test just a single
test
No test ever depends upon the successful
actions or completion of a previous test
Computers dont care if they do repetitive
actions

 Baseline

(Initialization)
Navigation
Editing
Verification
Tools

already have their own routines

defined for some of this however, you will
get more flexibility and maintainability
using these patterns

 Will

set the environment to the proper state

to start a test
This state is defined for each AUT and test
team
Can be simple to very complex
Determined by defining a known state that
all tests can reliably execute and get
consistent results

 Used

to determine if a page/form is active

and ready for interaction
Will navigate to the desired page/form if
the steps required are not complex, or the
steps will not alter the outcome of any test
Every page/form will have an unique
navigation routines created

 Enters

data into fields, pushes buttons,

sets checkboxes, selects values in lists,
etc.
The control types should not matter when
calling this routine it will figure out what
the control type is during test execution
and interact with the controls appropriately
Defined as an unique routine for every
individual page/form

 Reads

data from all controls types

Determines if the values read result in a
pass or fail
Responsible for logging results
Defined as an unique routine for every
individual page/form

 Solution

layout
Actual code examples
Dealing with particularly difficult controls
Configuration management
Skill sets
Specific tools capabilities

 Beginning

automated tests
Test framework goals
Layers of a framework
Coding standards
Initial routines that every page/form
requires

 Thanks

for your participation

Byron.goodman@neudesic.com