Software Quality Engineering

BS(SE)-VI

Dr. Assad Abbas

Department of Computer Science

COMSATS University Islamabad, Islamabad Campus
assadabbas@comsats.edu.pk
Topics
n The elements of a quality assurance plan

n The contents of a quality assurance plan

n Quality plan for small and internal projects

n Preparing plan provides advantageous

5/13/20 2
Reality Check
n Is an SQA plan just busy-work, or does it really
pay off?
n Hughes Aircraft
5 moved from level 2 in 1987 to level 3 in 1990
5 cost = $500K
5 benefit = $2M annually

n Raytheon
5 moved from level 1 in 1988 to level 3 in 1993
5 productivity doubled
5 ROI = $7.70 per $1 invested

5/13/20 3
CMM Effectiveness Case Studies
n Hughes Aircraft
n Schlumberger
n Texas Instruments
n Tinker AFB

5/13/20 4
Boehm and Ross’s Top 10 Software Risk Items
1. Developing wrong software functions *
2. Unrealistic schedules and budgets *
3. Developing wrong user interface
4. Gold plating *
5. Continuing stream of requirement changes *
6. Shortfalls in externally furnished components
7. Shortfalls in externally performed tasks
8. Personnel shortfalls *
9. Real-time performance shortfalls *
10. Straining computer science capabilities

5/13/20 5
The Software Risk Management Process

5/13/20 6
The Elements of a Quality Plan
n List of Quality Goals
n Review Activities
n Software Tests
n Acceptance Tests for Software Externally
Developed
n Configuration Management Tools and Procedures

5/13/20 7
The Elements of a Quality Plan…
List of Quality Goals
n These refer to the quality requirements in the developed software system.
n Quantitative measures usually preferred to qualitative measures when
choosing goals because they are usually easier to assess objectively
during testing.
n Quality goals should reflect the major acceptance criteria found in the
requirement’s document correctness, reliability, robustness,
maintainability….

5/13/20 8
Example
n A software system to serve the help desk operations of an
electrical appliance manufacturer is to be developed. The
help desk system (HDS) is intended to operate for 100 hours
per week. The software quality assurance team was
requested to prepare a list of quantitative quality goals
appropriate to certain qualitative requirements.
n HDS qualitative requirement
5 The HDS should be user friendly.
n Related quantitative goals
5 A new help desk operator should be able to learn the details of
the HDS following a course lasting than 8 hours, and to
master operation of the HDS in less than 5 working days.

5/13/20 9
Example

05/13/2020

5/13/20 10
The Elements of a Quality Plan…
Planned Review Activities
n The planned reviews should include a complete listing of all planned
review activities.
5 Design reviews (DRs)
5 Technical reviews
5 Managerial reviews
5 Code inspections
5 Pros and Cons……………………
n All reviews need to include:
5 Scope – what does it cover
5 Type – emphasis – managerial, technical, super detailed…
5 Schedule – often based on previous reviews and outcomes
g As defined by its priority and the succeeding activities of the project process.
5 Procedures – action lists;
5 Who is to attend? Collateral interest?? *****
5 Responsibilities for review; documents needed, by when…

5/13/20 11
The Elements of a Quality Plan…
Planned Software Tests
n The quality plan Must provide a complete list of
planned tests.
n Each test must include the following:
5 Coverage of test: unit, integration, system, subsystem….
5 Type of test: may include computer-generated tests and their
application via test suites, and more
5 Planned test schedule – prioritized and follow up Exact procedures
(for different types of tests…)
5 Who is responsible for carrying out tests
notification, time, date, materials, facilities, etc.
Different people responsible at different times!! **

5/13/20 12
The Elements of a Quality Plan…
Planned Acceptance Tests for Externally Developed
Software
n A complete set of acceptance tests to be run for externally developed
software must be provided within the quality plan!
n Complete set must be run for our own developed software!
n Especially critical for purchased software, contracted software (i.e.
software developed by subcontractors), customer-supplied software.
n These tests can be run in parallel with internally-developed software tests
(tests that internally are developed to supplement other tests).

5/13/20 13
The Elements of a Quality Plan…
Configuration Management
n The quality plan MUST include configuration management tools and
procedures for managing the software configurations, versions, etc.
n Must be an intrinsic part of the entire project!
n The Quality Plan may be included within the Development Plan or as an
independent document.
n The document, however compiled, must be reviewed and approved by the
organization’s standard approval process.

5/13/20 14
Configuration Management

5/13/20 15
Sources of Changes
n Requirements
n Changes in funding
n Technology advancements 05/13/2020

n Solutions to problems
n Scheduling constraints
n Customer expectations
n Unexpected opportunities for an improved system

5/13/20 16
Contents of a Quality Assurance Plan
n Project Details
5 Project Manager, Quality Manager
5 Project objectives
n Project Standards and Methodologies
For IT based projects these might be such things as:
5 Development Methodology (ies) being used
5 Documentation standards
5 Coding standards e.g. ANSI C
5 Testing standards and methods
5 QA methods e.g. walkthroughs or inspections
5 Operating standards e.g. ISO 9000, ISO 17799
5 Project Management Methodology e.g. PRINCE
(PRojects IN Controlled Environments)

5/13/20 17
Plan Contents (deliverables)
n Project Deliverables
5 Design specifications
5 Test specifications
5 Documentation
5 Software and Hardware
n Quality Goals
5 Quantitative and Qualitative quality goals that are to
be met by the deliverables
g E.g., Quantitative : Tom Gilb’s attribute specifications
g E.g., Qualitative: User survey

5/13/20 18
Plan Contents (deliverables)…
n Quality Assurance Schedule
5 Methodologies to be used at each stage of development
(inspections, walkthroughs, tests etc.)
5 Test Plan reviews
5 Sampling methods to be used, e.g., on test results
5 Who will participate in reviews and inspections
n Risks and Exposures
5 List of key risks to project
5 Actions to be taken to address them
5 Personnel responsible
5 Checkpoints

5/13/20 19
Quality Control Points
Requirements Gathering

System Design
System Test Plans
Inspection
Inspection
Subsystem Design Integration Test Plans System Test Cases

Inspection Inspection Inspection

Test Cases
Module Design Module Test Plans
Inspection
Inspection Inspection

Write Source Code Test Cases

Inspection Inspection
Unit Testing Integration Testing System Testing
5/13/20 20
Development / Quality Plans for Small and Internal
Projects
n Natural for many to try to avoid hassle of preparing all these
plans.
n In fact, heavy-weight methodologies are often called plan-
centric; Agile methods try to avoid much planning and
documentation.
n The question is simply does a short, small project (likw 30-60
days; two or three individuals) deserve the time spent on
planning a development and quality plan?
n Answer: No, not exactly.

5/13/20 21
Development / Quality Plans for Small Projects
n Lots of issues here…
n Sometimes not done due to short duration / manpower
n Sometimes planning is left up to the project leader’s
discretion.
n Perhaps a critically-important and high risk but short duration
effort with high-penalty shouts for a plan…
n Sometimes, via contract, both development and quality plans
are simply required.

5/13/20 22
Development / Quality Plans for Small Projects…
n Several advantages to planned over unplanned projects:
1. A more comprehensive / thorough understanding of the task
is likely (gained when developing the plan).
2. Greater responsibility for meeting obligations can be
assigned, as they can be ‘seen’ more clearly since
articulated (who does what).
3. Easier to share control of the project and identify
unexpected delays (any plan better than no plan at all!)
4. Better understanding of requirements and timetable can be
reached between customer and developer.

5/13/20 23
Development / Quality Plans for Internal Projects
n Lots of projects done for the internal use of organization.
n Here, normally no external body is a customer
n Can be medium or large scale.
n Tendency to avoid adequate development / quality plans
5 avoiding plans is fraught with errors.
5 cost overruns, missed dates, internal friction among
cooperating shops, …

5/13/20 24
Preparing Plans provides Advantages
n Internal Customers ‘can’ enjoy advantages.
5 smaller deviations from planned completion dates.
5 smaller budget overruns.
5 better control over development process – problems
can be addressed locally.
n Organizationally,
5 reduced risk of market loss (done for internal use).
5 reduced risk of litigation (late arrival; non-
compliance).
5 reduced risk of impairing a firm’s reputation.
5 reduced risk of requesting a budget supplement.
5/13/20 25
References

1. Software Quality Assurance: from Theory to

Implementation. Daniel Galin, Chapter 6,
Pearson Education, 2004.

5/13/20 26