Quality assurance in software projects
State of the art
Pătru Marius-Aurelian
Computer Science
FILS
Bucharest, Romania
patrumarius2015@gmail.com
Abstract— Current program development methodologies have
adapted to meet customer requirements, in particular reducing
delivery times, minimizing maintenance costs, high customer
engagement, and high visibility of the project. In the application
development cycle, testing ensures that customer requirements
are met, provides suggestions for improvement for the
developed product, reports defects and potential risks, and
collaborates with the development team to increase product
quality.
Keywords-testing; test management; software projects; test
managers; test processes
I. THEME
Our life has become dependent on software-based
systems. Almost everything around us are completely or at
least partly controlled by software. Entities from all industries
and administrative agencies rely to a large extent on highly
complex IT systems: inventory control systems, biometric
characteristics in passports and ID cards and the electronic
health chip card.
Due to this strong dependency on software systems,
companies must invest considerably in quality assurance
activities to allow those systems to perform reliably. Software
testing is currently developing toward a specialized field of
study and professional discipline within computer sciences.
Test management is of particular importance within the
discipline of software testing. Test management comprises
classical methods of project and risk management and
knowledge of the well-defined test methods. With this
approach, the persons responsible with test management, test
managers, can select and implement specific measures to
ensure that a predefined product quality will be reached. This
way, test managers adopt an engineering approach.
Although today’s project management training is well
settled, and at the same time there are an enormous number of
study courses, training programs and professional literature to
choose from, there has, until not so long ago, been barely any
pursuit at defining the contents of training programs for the
software test manager. Considering the increasing
responsibility assumed by test managers in completion of their
job, this has been a disappointing situation.
With the proposed training for test managers is ISTQB
Certified Tester – Advanced Level – Test manager, we have,
for the first time, developed an internationally recognized
training that describes training contents and qualification
sections for the tasks of the test managers. This qualification
scheme consists of several levels, from the basic one to a more
advanced level of expertise.
II. A BRIEF HISTORY OF SOFTWARE TESTING
Modern testing tools are becoming more and more
advanced and user-friendly. Software testing is the activity of
running a series of dynamic executions software programs
after the software source code has been developed. [] It is
executed to discover and correct as many potential errors as
possible before delivery to the customer. As indicated earlier,
software testing is still an “art”. It can be recognized a risk
management technique; the quality assurance technique.
There has been many definitions and advances in software
testing throughout the history of software development. In
1950s, software testing was defined as “what programmers
did to find bugs in their programs”. 10 years later, the
definition of testing experienced a review. Exhaustive testing
was debated and expressed as the set of possible paths through
code that can be tested. It was noted that it was impossible to
comprehensively test an application because of dimension of
the program inputs and possible input paths and because
design and specification issues are difficult to test.
Between 1960 and 1970, software development matured,
and the activity of software was referred to as “computer
science”. Software was designated as “what is done to
demonstrate correctness of a program” or as “the process of
providing confidence that a system or program does was is
supposed to do”.
In the late 1970 it was stated that testing is a process of
executing a program with the intent of finding an error, not
demonstrating that it works. This new definition emphasized
that a well-constructed test case is one that has a high
probability of finding an undiscovered error. A successful test
is one that uncovers an undiscovered error.
 We can observe two objectives of software testing. The
first one is to prove that a program works and implies the use
of test data that have a low probability of causing a program
failure. The second objective is to uncover problems in the
software program by testing it with different input data.
There were proposed the following good testing practices:
- A test case must contain an expected output result
- The program should not be tested by the developers
who wrote it
- Each test result must be inspected comprehensively
- Corner cases should be written for invalid and
unexpected input conditions
- The probability of the existence of more errors in a
section of a program is proportional to the number of
errors already found in that section
The 1980s included defect prevention in the definition of
testing. Designing tests is one of the most effective bug
prevention technique known. It was addressed the need for a
testing methodology that must include reviews throughout the
entire software development life cycle. This should be
designed as managed process. It was also promoted the
importance of testing not just code but requirements, design
and tests themselves.
Up until the early 1980s, testing referred to the set of
activities done to a system after working code was delivered
and a system testing created. However, testing today implies
a larger spectrum of tasks for the tester. This means that the
tester should be involved in relatively every aspect of the
software development life cycle. After a deploy is made with
the current version of the code and the tester finds problems,
then the version under test must be investigated. It is
recommended to try to find bugs from earlier version of the
system, not wait until an actual working product is produced.
It is cheaper to fix a bug or issue if it is found in an early
life cycle. The cost grows by exponential amounts with the
maturity of the development phase. It is not enough to just test
the program and for bugs in it, we also must precisely review
the requirements and designs. There is the misconception,
even today, that software testing is a back-end activity.
In the mid-1980s, automated testing tools came up to
automate the manual testing effort to increase the productivity
and quality of the system under test. It was assumed that the
computer may achieve a greater quantity of tests executed in
a more solid way. At the beginning, the tools were not very
advanced and did not provide programming language
capabilities. In the early 1990s, the importance of early test
design was acknowledged. The definition of testing extended
to “planning, design, building, maintaining, and executing
tests and test environments”.
In the early 2000s it was introduced an even extensive
definition of testing when it was proposed the concept of
business technology optimization (BTO). This theory
coordinates the IT strategy and execution with business
objectives. It helps conduct the priorities, human resources,
and processes of IT. The essential approach is to assess and
maximize value over the IT service delivery life cycle to
assure applications and systems meet quality, performance,
stability, and availability goals.
III. MEASURES TO IMPROVE SOFTWARE QUALITY
There are many approaches available on how to enhance
software quality by using actions the prevent issues and by
using analytical mechanisms:
- Designate software development processes that
can grant structured and maintainable
development of software systems
- Improve the efficiency of test activities
execution by defining test processes and
controlled incident management
- Apply metrics and quality data standards to
evaluate software products and development
processes in an objective manner
- Standardize methods that grant correct
formulation of development document
Test managers should have knowledge about these
methods and techniques in order to be able to choose and
apply convenient methods during the software project. The
quality assurance measures are expressed according to the
predefined quality goals. Functionality, usability and
reliability are examples of quality attributes. There are
situations when a test result can be classified as faulty if the
behavior expected result is not met.
Test planning and control, test implementation and test
execution or test evaluation of the test exit criteria are
activities that define software testing. There are different test
levels depending on the level of abstraction or development
phase. We can distinguish component or unit test, integration
test, system test and acceptance test. Each test level presents
specific test goals, tools and methods.
In manual testing is base the entire test process, test plan,
test execution, test cases are done to be manually executed by
a person. Also, a test team member runs every step of each test
case, documents the results of each step, and helps developers
to reproduce the defects found. Also, testing maintenance is
much slower. Manual testing can be advantageous in small
projects without many regression test cases because the entire
automation process is not worth the effort when the test suite
is executed once or twice. Automated testing is the process of
using certain programs and software tools to control the
execution of test cases and compare the result with the
expected output. This whole process is fully automated and
requires no manual intervention from the test engineer.
This process should cover the entire testing process, from
the creation of test cases to reporting, and provide detailed
statistics on tested functionality, the creation of test data, and
the distribution of the test data to each scenario. It is very
important for the tests to be independent and not to require any
manual intervention. The manual intervention negates almost
all the benefits of automated testing because a well-automated
test should be executed completely unattended to bring a
benefit to the testing team that can handle the testing of other
functionalities in the meantime.
Also, a test must be robust and consistent, results should
not vary from one execution to another. A major difference
between automatic and manual testing is that in manual
testing, a person can execute a single test case, the tests being
sequential. In the case of automated testing, the same person
can execute in parallel as many test cases as are allowed by
the machine's computing and processing power on which
automated scenarios are running, which drastically reduces
execution time.
One of the misconceptions is that if automatic testing is
introduced into a project, manual testing will become
redundant, not necessarily necessary. Automated testing only
comes with manual testing because some test cases cannot be
automated or are not worth automating.
Moreover, the two teams must work together to help each
other, the automated testing team must know which areas are
at risk, the most frequently performed, most repetitive test
cases in order to ease the work manual test team and reduce
their load. These two teams must periodically synchronize for
the whole test process to run as few problems as possible, be
effective and bring value and quality to the project. The
general testing strategy exit criteria from the test cycle must
be set by both teams jointly to minimize misunderstandings.
Typically, in teams that implement agile development
methodologies, the same people who are responsible for
manual testing are also responsible for automated testing,
which facilitates the automation process because they know
both manual application testing and automated testing.
Often, members of the automated testing team do not
necessarily have to master the application, because in some
teams they are only responsible for identifying application
components and automating actions, and manual test team
members combine this to create test scenarios. This can be
done, for example, by reading from Excel files where
members of the manual test team only sequentially insert the
steps to be followed to perform the test scenario, and these
data are read by the automated test programs that transmit
these parameters in automatic tests that will run during the
night.
This also allows many test scenarios to be created because
the only effort, once the elements have been transposed into
code and the actions have been programmed, is to combine
them in desired ways, resulting in covering all possible test
combinations.
IV. QUALITY ASSURANCE AND TEST MANAGEMENT
Quality assurance (QA) is defined as “the essential
character of something, an inherent or distinguishing
character, degree, or grade of excellence.” Two general
concepts refer to quality assurance. The first represents
meeting requirements which means that the specifications
must be measurable in order to decide whether we have a
quality product or not. The second concept is expressed from
the customer’s point of view. He decides if the product or
system does what he needs. The product’s specification is the
most important document, and the quality relies the most on
it. It includes important quality attributes that the product must
have, e.g. usability, describes how easy the user can
communicate with the application, portability, the capability
of the system to be used across different hardware and
software architectures.
It is impossible to assess quality on an already completed
product. The objective is to prevent quality issues by
implementing quality assurance measures, including
structuring the development process with a software
development standard and supporting each life cycle of the
application with methods, techniques and tools. Another
important element in a quality management program is
process assessment. This can be achieved by documenting the
coding standards and using methods, tools, procedures for
data backup, test methodology and test management.
Test management decreases production costs because the
sooner a defect is found and fixed, the less costly it will be in
the long run. Using automated testing tools may initially imply
a substantial investment, but the long-term result will come
with a higher-quality products and reduced maintenance costs.
One of the tasks of the testing team that consumes the most
time is collecting quality metrics for a product. Statistics
should be collected about the number of defects occurring and
which functionality has produced the most defects, the time
spent for performing each test case, the result of each step in
each test scenario, whether or not we meet the acceptance
criteria, how many test cases has been executed and for what
functionality, what functionality is still untested and many
more.
By integrating a statistical utility and reporting into
automated testing, these processes can be deployed in a test
case where we can collect screen shots at each step, the status
of each test case, runtime, defects that are covered the test case
and any other information we want. We can also generate
automated graphs that can be understood by all team members
and customers, graphs that can be generated every day based
on the results from the previous day's execution, and so we
can have daily the evolution of application quality and defects.
The total cost of effective quality management is the sum
of four component costs: prevention, inspection, internal
failure, and external failure. Prevention costs consist of
actions taken to prevent defects from occurring in the first
place. Inspection costs consist of measuring, evaluating, and
auditing products or services for conformance to standards
and specifications. Internal failure costs are those incurred in
fixing defective products before they are delivered. External
failure costs consist of the costs of defects discovered after the
product has been released. The latter can be devastating
because they may damage the organization’s reputation or
result in the loss of future sales.
Software quality assurance is a planned effort to ensure
that a software product fulfills these criteria and has additional
attributes specific to the project, for example, portability,
efficiency, reusability, and flexibility. It is the collection of
activities and functions used to monitor and control a software
project so that specific objectives are achieved with the
desired level of confidence. It is not the sole responsibility of
the software quality assurance group but is determined by the
consensus of the project manager, project leader, project
personnel, and users.
The objectives of QA are achieved by making a plan
containing the methods used during the project to ensure that
the documents or products reviewed or produced are of high
quality at each checkpoint. A strategy for risk management is
software quality assurance. It was introduced because
software quality is a costly feature, especially in the late
phases of a development life cycle. Quality assurance consists
of several components, each with its own objectives.
1.Software testing is a risk management strategy used to
verify that functional requirements of a software product are
met. The drawback of this approach is that it is made in a late
phase of development, when the product has a stable version.
At that moment is too late to build quality into the product.
Nonetheless, not all defects are discovered doing this activity.
Verification and validation activities have the purpose to
ensure that software design, code, and documentation meet all
the requirements. During software design and
implementation, verification helps determine whether the pro
products of one phase of the software development life cycle
fulfill the requirements established during the previous phase.
A basic software configuration management activity is the
identification of the software components that make up a
deliverable at each point of its development. Software
configuration management provides guidelines to identify and
name software baselines, software components, and software
configurations.
3. Software quality assurance plan (SQA) plan is an
outline of quality measures to ensure quality levels within a
software development effort. The plan is used as a baseline to
compare the actual levels of quality during development with
the planned levels of quality. If the levels of quality are not
within the planned quality levels, management will respond
appropriately as documented within the plan. Steps to develop
and implement an SQA:
- document the plan
- obtain management acceptance
- obtain development acceptance
- plan for implementation of the SQA plan
- execute the SQA plan
V. CONCLUSIONS
Test management is an essential element in quality
assurance and software project development. It is
recommended that its principles be used from the early phases
of the development. From designing the specification until
software testing in the production environments, test
management follow the evolution of the code and of all
documents used by the contributors.
We seek for high quality of the product, and quality
depends on the strategies that we use. Quality assurance
manages the quality. If the processes are followed, the
management of a software project can be assured of product
quality, can have an overview of the current status and levels
of the product and can make decisions in order to minimize
the risks.

2. Software Configuration Management includes activity [1]
like labeling, tracking and controlling changes in software
elements of a system. The evolution of the system is managed
with versions of its software components. The purpose is to
identify the relationship between components of software and [2]
to have a big picture of their evolution throughout the
development life cycle phases. This approach can come with
[3]
considerable benefits for the software project. Software
configuration management identifies a system configuration
to systematically control changes, maintain integrity, and
[4]
enforce tractability of the configuration throughout its life
cycle.
Martin R. Woodware, Michae A. Hennell, Journal of Engineering and
Technology Management, vol 22, issues 1-2, March-June 2005, pages
113-140
Mario Winter, Thoma Rossner, Software Testing Practice: Test
management, Rocky Nook, 2007 ISBN: 978145711822
Black Rex (2009), Advanced Software Testing Vol. 1 ISBN-13: 978-
1-933952-19-2
Bruce R. , Kesseniti M., Software Quality Assurance, 2016,
http://www.sciencedirect.com/science/article/pii/B978012802301300
0028