Sixsigma Spi

Carnegie Mellon
Software Engineering Institute

Pittsburgh, PA 15213-3890
Six Sigma &

Software/Systems Process
Improvement
Jeannine M. Siviy
Software Engineering Measurement & Analysis Initiative
Sponsored by the U.S. Department of Defense

2002 by Carnegie Mellon University
11th ICSQ, October 2001. - page 1

Carnegie Mellon
Outline / Objectives
Six Sigma Overview
Applications Survey
Initiative Synergy
Illustration
2002 by Carnegie Mellon University SEPG 2002 - page 2

Carnegie Mellon
Six Sigma Is
A Philosophy
A Metric
An Improvement Framework

Carnegie Mellon
Six Sigma Philosophy
Improve
customer satisfaction
by reducing and eliminating
defects
Greater Profits
Carnegie Mellon
What is a Defect?
Six Sigma:
Any product, service, or process variation which
prevents meeting the needs of the customer and/or
which adds cost, whether or not it is detected.
Personal Software ProcessSM:

Defects or faults are the result of errors or mistakes. At
a minimum, count a defect every time the program is
changed during compile or test, where the change
might be one character or multiple statements
[Humphrey 95]
SMPersonal Software Process and PSP are service marks of Carnegie Mellon University.
Carnegie Mellon
Six Sigma Metrics

3.4 ppm the most-cited metric
Other Measures
Defect Rate, parts per million (ppm)
Sigma Level
Defects per Unit (dpu)
Defects per Million Opportunities (dpmo)
Yield

Carnegie Mellon
3.4 and Sigma Metrics 1

New Car Buyers target: Lower Upper
Target
Spec Spec
35 miles per gallon (mpg)
29-41 mpg acceptable
29 35 41
Two Choices:
Car 1 Car 2
35 +/- 2 mpg 35 +/- 1 mpg
(35-29) / 2 = 3 (35-29) / 1 = 6
(41-35) / 2 = 3 (41-35) / 1 = 6
3 Sigma 6 Sigma
~3/1000 outside limits virtually 0 outside limits
Carnegie Mellon
3.4 and Sigma Metrics 2

Historical data:
1.5*standard deviation shift over time
Car 2:
Lower Upper
Target
Spec Spec Mean shifts to 33.5
Mean - 6*Std Dev now
29 35 41
extends below lower spec
27.5 33.5 39.5
Extension corresponds to 3.4
Shifted Range ppm if normal distribution

Carnegie Mellon
Six Sigma Process with Mean Shift

Assumptions:
normal distribution
process mean shift of
1.5 from nominal is
likely
process mean and
standard deviation are
known
defects are randomly
distributed throughout
s
-3
-3 s
+3
units
s

-6
-6 +6
+6 s

parts and process
steps are independent

Carnegie Mellon
Example Sigma Levels

IRS - Tax Advice (phone-in)
100,000
PPM per Part or Process Step
Restaurant Bills
Doctor Prescription Writing

10,000
Payroll Processing
Air Line Baggage Handling
1,000
Average
Company Orders Placed on Factory
100
US Navy Aircraft Accidents
Note: Sigma
10 Levels vary +/-
1s with source
Domestic Airline
publication date
Flight Fatality Rate
1
2 3 4 5 6 7
(308537) (66807) (6210) (233) (3.4)
Sigma Level
(ppm for shifted process)
[Harrold 98], [Harry 00]
Carnegie Mellon
Statistical Thinking
Everything is a process
All processes have inherent variability
Data is used to understand variation and to
drive decisions to improve the processes
Special Cause Variation
Data Spread due to

Original Mean
Common Cause Variation
New mean after improvement

(Spread due to common cause
variation will re-establish itself.)
[ASQ 00], [ASA 01]
Carnegie Mellon
Everything Is a Process
Example: Software Engineering
Require-
Design Code
ments
Compile Test Ship

Carnegie Mellon
Six Sigma Improvement Framework
Define Measure Analyze Improve Control

Carnegie Mellon
Six Sigma Toolkit

Define Measure Analyze Improve Control
Benchmark 7 Basic Cause & Effect Design of Statistical
Baseline Tools Diagrams Experiments Controls:
Contract/Charter Defect Failure Modes & Modeling Control
Kano Model Metrics (i.e., Effects Analysis Tolerancing Charts
Voice of the ppm) Decision & Risk Robust Time Series
Customer Data Analysis Design methods
Voice of the Collection Statistical
Business Forms, Plan, Inference Non-Statistical
Quality Function Logistics Control Charts Controls:
Deployment Sampling Capability Procedural
Process Flow Techniques Reliability adherence
Map Analysis Performance
Project Root Cause Mgmt
Management Analysis Preventive
Management -5 Whys activities
by Fact Systems
-4 Whats Thinking

Carnegie Mellon
Design for Six Sigma (DFSS)

New solutions
Rather than analysis of existing processes
Focus
Customer and business
Emphasis on critical-to-quality characteristics
Useful tools
Modeling, simulation, lean, systems thinking

Carnegie Mellon
Applications
Six Sigma applications for

Systems and Software Engineering
are emerging

Carnegie Mellon
Survey of Applications 1
Allied Signal
1997 air supply control system shutdowns
Black Belt project team commissioned to find solution
Motorola
Inspection data analysis & unit test optimization
Design of experiments methods & test cases
Complexity analysis & resource allocations
Quantitative risk management via uncertainty modeling
General Electric
DFSS
Six Sigma & Extreme Programming
[Harry 00], [Stoddard 00], [Kelliher 01]

Motorola & GE presentations available at http://seir.sei.cmu.edu
Carnegie Mellon
Survey of Applications 2
Honeywell
PSPSM/TSPSM & Six Sigma
- TSP provides the data needed to apply Six Sigma
JP Morgan
Capability Maturity Model (CMM) & Six Sigma
- Six Sigma methodology is beneficial on all levels of
maturity.
NCR
CMM & Six Sigma
- helps organizations working towards Level 4 & 5 deliver
the best business results.
[Pavlik 00], [A-M 99], [Demery 01]

Presentations available at http://seir.sei.cmu.edu
Carnegie Mellon
Initiative Synergy
CMM
Level 1-3
Level 4-5
CMM IntegrationSM (CMMISM)

Personal Software ProcessSM (PSPSM)
Team Software ProcessSM (TSPSM)
Capability Maturity Model and CMM are registered in the U.S. Patent and Trademark Office.
SMCMM Integration, CMMI, Personal Software Process, PSP, Team Software Process and TSP
are service marks of Carnegie Mellon University.

Carnegie Mellon
CMM and Six Sigma*

improvements and control Optimizing
5 Organization-wide 6 Process
methods improvement
Correlation between key process areas & 6
used within CMM efforts
6
Quantitative
4 Process measured
and controlled
Defined

3 Defined processes feed 6 Process characterized for the
organization and is proactive
philosophy & method focus

2 6 Repeatable
Process characterized for projects and
drilldown drives local
6 is often reactive
(but threaded) improvements Initial
may drive toward and
1 6 Process unpredictable and poorly controlled
accelerate CMM solution
Six Sigma is enterprise-wide.

Six Sigma focuses on critical to quality factors.
*Similar comments apply to CMMI
Carnegie Mellon
CMMI & Six Sigma*

Within CMMI
Quantitative Project Management (QPM)
Organizational Process Performance (OPP)
Organizational Innovation and Deployment (OID)
Measurement & Analysis (MA)
Capability Levels
Generic Practices
Around CMMI
SEPG process improvement rollout
Assessment methods
Prioritization of process areas
*Similar comments apply to CMM

Carnegie Mellon
Software Engineering Institute D M A I C
Illustration Define 1
Business Driver
Need 10% cost reduction in order to compete
in the marketplace and stay in business
Baseline data (PSP)

Productivity: 19 LOC/hr
33% of development time spent fixing defects
Approximately 250 defects/KLOC

Carnegie Mellon
Illustration Define 2
Goal:
Reduce or prevent defects to reduce cost
Quantitatively speaking:
Reduce cycle time by 22 minutes/program
Reduce fix time by 1.3 minutes/defect
Reduce defects by 6/program
Reduce defect density to 190 defects/LOC
or a combination that produces 21 LOC/hr

Carnegie Mellon
Illustration Define, Measure

PSP-based Process Map:
Design Code Compile Test
Requirements Resources Code Executable Code

Estimate Test Plan
Concept design
Detailed Design Code Executable Functional

Test cases Data: Code Code
Data: - Defect Quantity Data: Data:
- Design Review - Fix time - Defect Quantity - Defect Quantity
defects - Defect Injection - Fix time - Fix time
- Fix time Phase - Defect Injection - Defect Injection
- Phase duration - Phase duration Phase Phase
- Phase duration -Phase duration

Carnegie Mellon
Illustration Analyze
Opportunities to reduce repair time
Defects removed in test: 78% of repair time
Defects injected in design: 25% of repair time
Defects injected in code: 56% of repair time
Syntax defects in general: 63% of defects
Removed Defects Fix Time Injected Defects Fix Time,
Baseline programs 1-6 Baseline programs 1-6
Code: 2%
Test: 19% Design: 25%
Compile: 20%
Test: 78% Code: 56%

Carnegie Mellon
Illustration Improve
Improvement Plan at Program 6
Syntax checklist
Well-timed reviews
Subcategories within defect types
Defect Density Mean Comparison: Defect Density
450 450
400 400
350 350
Defects/KLOC
defects/KLOC N&C
300 300
250 250
200 200
150 150
100 100
50 50
0 0
first second All Pairs
1 2 3 4 5 6 7 8 9 10 Pre Post Tukey-Kramer
Program Improvement Improvement 0.05
Group

Carnegie Mellon
Illustration Control
Tracking performance
Quantitative goal statement
Hypothesized root causes
Countermeasures & contribution to impact
Key impact indicators
Productivity (LOC/hr) Direct causes (from countermeasures):

30
25 Fewer defects injected in code & test
Goal = 21
20 Defects removed earlier, faster (i.e., in
15
10
design & code)
5
0 Root cause (need new countermeasures):
1 2 3 4 5 6 7 8 9 10
Program Number
Re-learning curve
Carnegie Mellon
Illustration Analysis Summary

Tools used in full analysis included
Process Mapping
Descriptive statistics
Means comparisons & significance testing
Plots
- Pie Charts
- Trends
- Phase profiles
- Histograms
- Pareto charts
- Correlation plots
Cause & Effect Diagrams
Management by Fact
Focus was exploratory, investigative

Ready for stability & control monitoring
Carnegie Mellon
Illustration Scaling up
Illustration Real Life
Quickly drilled down from Drill down may be complex,
high level cost goal to may span wide breadth of
personal improvement organization
Defined process in place May need to select or define
process
Measures in place May need to develop measures
Continuous incremental Continuous incremental
improvements improvements
Event-based step- Event-based step-change
change improvements improvements
Re-learning curve Constantly changing skills,
technologies
Personal data Non-attributed data (e.g., team,
project)
Used productivity as one Excessive productivity focus
of impact measures may drive unwanted behaviors
Carnegie Mellon
Advancing the State of 6 & SW/SE

Repository of Examples
-http://seir.sei.cmu.edu
-concrete visualization
-relationship to models, initiatives
-variety of tools
-many views
project, process, product
software, systems
maturity/capability levels
Repository of Benefits

Carnegie Mellon
Summary
Customer satisfaction is key driver
All efforts should link

to business results

Carnegie Mellon
Contact Information
Jeannine Siviy
Measurement & Analysis Initiative
jmsiviy@sei.cmu.edu
412-268-7994

Carnegie Mellon
References 1
[A-M 99] Abdel-Malek, Nabil and Anthony Hutchings, Applying Six Sigma Methodology to CMM
for Performance Improvement, JP Morgan, European SEPG 1999, (slides available
to SEIR contributors at http://seir.sei.cmu.edu)
[ASA 01] American Statistical Association, Quality & Productivity Section, Enabling Broad
Application of Statistical Thinking, http://web.utk.edu/~asaqp/thinking.html, 2001
[ASQ 00] ASQ Statistics Division, Improving Performance Through Statistical Thinking,
Milwaukee: ASQ Quality Press, 2000. H1060
[Arnold 99] Arnold, Paul V., Pursuing the Holy Grail, MRO Today, June/July 1999,
www.progressivedistributor.com/mro/archives/editorials/editJJ1999.html
[Breyfogle 99] Breyfogle III, Forrest W., Implementing Six Sigma: Smarter Solutions Using Statistical
Methods, John Wiley & Sons, 1999
[Bylinsky 98] Bylinsky, Gene, How to Bring Out Better Products Faster, Fortune, 23 November
1998
[Demery 01] Demery, Chris and Michael Sturgeon, Six Sigma and CMM Implementation at a
Global Corporation, NCR, SEPG 2001, (slides available to SEIR contributors at
http://seir.sei.cmu.edu)
[Harrold 99] Harrold, Dave, Designing for Six Sigma Capability, Control Engineering Online,
January 1999, http://www.controleng.com/archives/1999/ctl0101.99/01a103.htm

Carnegie Mellon
References 2
[Harry 00] Harry, Mikel, Six Sigma: The Breakthrough Management Strategy
Revolutionizing the Worlds Top Corporations, Doubleday, 2000
[Kelliher 00] Kelliher, Timothy P., Daniel J. Blezek, William E. Lorensen, James V. Miller, Six
Sigma Meets Extreme Programming, General Electric Corporate R&D, (paper
available to SEIR contributors at http://seir.sei.cmu.edu)
[Pavlik 00] Pavlik, Rich, Cary Riall and Steve Janiszewski, Deploying PSPSM, TSPSM, and
Six Sigma Plus at Honeywell, Honeywell Air Transport, SEPG 2000, (slides
available to SEIR contributors at http://seir.sei.cmu.edu)
[Purcell 00] Purcell, Leitha, Experiences Using Six Sigma in a SW-CMM Based Process
Improvement Program, Northrop Grumman, SEPG 2000, (slides available to
SEIR contributors at http://seir.sei.cmu.edu)
[Pyzdek 01] Pyzdek, Thomas, The Six Sigma Handbook, McGraw-Hill Professional
Publishing, 2001
[Stoddard 00] Stoddard, Robert W., Implementing Six Sigma in Software, Motorola, Inc.,
Software Engineering Symposium 2000, (slides available to SEIR contributors
at http://seir.sei.cmu.edu)

Carnegie Mellon
Additional Reading 1
Books (General Six Sigma Topics, not software-specific):
Breyfogle III, Forrest W., Implementing Six Sigma: Smarter Solutions Using Statistical Methods,
John Wiley & Sons, 1999
Breyfogle, III, Forrest W., Cupello, James M., Meadows, Becki, Managing Six Sigma, John Wiley
& Sons
Harry, Mikel, Six Sigma: The Breakthrough Management Strategy Revolutionizing the Worlds Top
Corporations, Doubleday, 2000
Pyzdek, Thomas, The Six Sigma Handbook, McGraw-Hill Professional Publishing, 2001
Web pages & Web sites:

American Society for Quality Six Sigma Forum,
http://www.sixsigmaforum.com/concepts/var/index.shtml
International Quality Federation, www.iqfnet.org (Follow the black belt links)
Six Sigma Academy, www.6-sigma.com
Software Engineering Information Repository: http://seir.sei.cmu.edu (Follow links to
Measurement area then to Six Sigma)
SEI Software Technology Review: http://www.sei.cmu.edu/str/descriptions/sigma6.html

Carnegie Mellon
Additional Reading 2
Journals (URLs subject to change without notice)
6 Sigma Pro, Quality Digest, May 2000, http://www.qualitydigest.com/may00/html/sixsigmapro.html
6 Sigma Con, Quality Digest, May 2000, http://www.qualitydigest.com/may00/html/sixsigmacon.html
Arnold, Paul V., Pursuing the Holy Grail, MRO Today, June/July 1999,
http://www.mrotoday.com/mro/archives/Editorials/editJJ1999.htm
Card, David, Sorting out Six Sigma and the CMM, IEEE Software, May/June 2000
Dusharme, Dirk, Six Sigma Survey: Breaking Through the Six Sigma Hype, Quality Digest, November
2001, http://www.qualitydigest.com/nov01/html/sixsigmaarticle.html
Paul, Lauren Gibbons, "Practice Makes Perfect," CIO Enterprise Magazine, 15 January 1999,
http://www.cio.com/archive/enterprise/011599_process.html
Harrold, Dave, Designing for Six Sigma Capability, Control Engineering Online, January 1999,
http://www.controleng.com/archives/1999/ctl0101.99/01a103.html
Harrold, Dave, Optimize Existing Processes to Achieve Six Sigma Capability, Control Engineering
Online, March 1999, http://www.controleng.com/archives/1999/ctl0301.99/03a301.html
Lahiri, Jaideep, The Enigma of Six Sigma, The Net Business of the India Today Group,
http://www.india-today.com/btoday/19990922/cover.html
Pyzdek, Thomas, Why Six Sigma is Not TQM, Quality Digest, February 2001,
http://www.qualitydigest.com/feb01/html/sixsigma.html
Pyzdek, Thomas, Ignore Six Sigma at Your Peril, Quality Digest, April 2001,
http://www.qualitydigest.com/april01/html/sixsigma.html

Sixsigma Spi

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Sixsigma Spi

Caricato da

Copyright:

Formati disponibili

Carnegie Mellon

Software Engineering Institute

Six Sigma &

Sponsored by the U.S. Department of Defense

11th ICSQ, October 2001. - page 1

2002 by Carnegie Mellon University SEPG 2002 - page 2

2002 by Carnegie Mellon University SEPG 2002 - page 3

Six Sigma Philosophy

Personal Software ProcessSM:

Six Sigma Metrics

2002 by Carnegie Mellon University SEPG 2002 - page 6

3.4 and Sigma Metrics 1

3.4 and Sigma Metrics 2

2002 by Carnegie Mellon University SEPG 2002 - page 8

Six Sigma Process with Mean Shift

2002 by Carnegie Mellon University SEPG 2002 - page 9

Example Sigma Levels

Doctor Prescription Writing

Data Spread due to

New mean after improvement

Compile Test Ship

2002 by Carnegie Mellon University SEPG 2002 - page 12

Six Sigma Improvement Framework

Define Measure Analyze Improve Control

2002 by Carnegie Mellon University SEPG 2002 - page 13

Six Sigma Toolkit

2002 by Carnegie Mellon University SEPG 2002 - page 14

Design for Six Sigma (DFSS)

2002 by Carnegie Mellon University SEPG 2002 - page 15

Six Sigma applications for

2002 by Carnegie Mellon University SEPG 2002 - page 16

[Harry 00], [Stoddard 00], [Kelliher 01]

[Pavlik 00], [A-M 99], [Demery 01]

CMM IntegrationSM (CMMISM)

are service marks of Carnegie Mellon University.

CMM and Six Sigma*

philosophy & method focus

Six Sigma is enterprise-wide.

CMMI & Six Sigma*

*Similar comments apply to CMM

Baseline data (PSP)

2002 by Carnegie Mellon University SEPG 2002 - page 22

or a combination that produces 21 LOC/hr

2002 by Carnegie Mellon University SEPG 2002 - page 23

Illustration Define, Measure

Design Code Compile Test

Requirements Resources Code Executable Code

Detailed Design Code Executable Functional

2002 by Carnegie Mellon University SEPG 2002 - page 24

Test: 78% Code: 56%

2002 by Carnegie Mellon University SEPG 2002 - page 25

2002 by Carnegie Mellon University SEPG 2002 - page 26

Productivity (LOC/hr) Direct causes (from countermeasures):

Illustration Analysis Summary

Focus was exploratory, investigative

Advancing the State of 6 & SW/SE

2002 by Carnegie Mellon University SEPG 2002 - page 30

Customer satisfaction is key driver

All efforts should link

2002 by Carnegie Mellon University SEPG 2002 - page 31

2002 by Carnegie Mellon University SEPG 2002 - page 32

2002 by Carnegie Mellon University SEPG 2002 - page 33

2002 by Carnegie Mellon University SEPG 2002 - page 34

Web pages & Web sites: