Process
lmprovement
and the
Lorporate
Balance Sheet
RAYMOND Drohr , Ray theon
I) Since it started a for the success of a new software-devel-
opment process is the accurate evalua-
pm-improvement tion of how effective it is in reducing the
zizitiative in 1988, bottoni-line cost of getting the job
done.’ In this article, I describe
RaytbemkE+qnnmt Raytheon’s approach to measuring how
changes in its software process resulted
Dizirim bas hpoued in reduced development costs.
At Raytheon, we have been able to
its bottomline, zizcvased convince top-level management that
software-process improvements withln
pdzutivity, and its Equipment Division have yielded
+ a $7.70 return on every dollar in-
cbanged the cqorate vested;
+ a two-fold increase in productivity;
culture.Mwb oftbe and
saving camefi-om
+ an evolution from level 1 ( h t i a l )
to level 2 (Repeatable) to level 3 (De-
fined) process maturity, as measured by
reducing rework. the Software Engineering Institute’s
Capability hlaturity Model.
28 0 7 U 0 7 4 5 9 / 9 3 / 0700/’W28 SO? CG C IEEE
Raytheon, a diversified, internation-
al, technology-based company, is one of
the 100 largest corporations in the US.
The Equipment Division is one of eight
divisions and 11 major operating sub-
sidiaries within Raytheon, with annual
sales in electronic systems that comprise
about 13 percent of the corporation’s
$9.1 billion annual sales.
Most of these electronic systems are
software-driven and real-time, and most
of the software is developed by the divi-
sion’s Software Systems Laboratory,
which employs about 400 software engi-
neers.
There are seven &rectorates, or busi-
ness areas, w i b the Equipment Division,
each one servicing a different application
area and a unique set of customers. Thls
creates challenges for the SSL: Usu-
JULY 1993
ally, it is the application and sometimes the velopment and management process in analysis (first on pilot projects to confirm
customer that drives the selection of target accordance with the principles of W. Ed- the findmgs) and how to diffuse the new
processors for new systems. In turn,h s wards Demin$ andJosephJuran? Basically, methods among practitioners (technology
constrains the selection of their phlosophy teaches transition).

I
a development environ-
ment and tool set.
T h e software-devel-
opment process that was
in place in early 1988 was
not adequate to deal with
h s diversity. As a result,
many SSL brojects were
delivered late and over
that real proceSS-improve- Improvement is continuous; thus,
RAYTHEON’S ment must follow a se- completion of the h r d phase signals a be-
quence of steps, starting ginning of the first.
PARAD’‘’ with making the process
PUTS PROJECTS visible, then repeatable, FINANCING IMPROVEMENT
AT THE CENTER. a n ~ ~ ~ ~ ~ ; a t Ineattempting s , to apply h s process-im-
Raytheon’s process-im- provement paradigm to the development
m-ovement Daradigm is process over a broad spectrum of projects,
Y

.- That year, the SSL performed an based on a three-phase cycle of stabiliza- we found that the number, inherent com-
budget.
assessment of its own software-develop- tion, control, and change, which applies plexity, and predominantly intellectual
ment process using the SEI’s capability- the principles of Deming and Juran. nature of the labor-intensive steps in the
assessment questionnaire. The SEI’s five- In the procesr-stabilimtim phase, the process required that we proceed with
level maturity model and questionnaire emphasis is on distilling the elements of caution.
(described in the article onpp. 18-27) pro- the process actually being used (acheving
vide a simple analytical tool for bench- visibility) and progressively institutional- Funding sources. In the first place, it is
marking an organization’s status. In 1988, izing it across all projects (providmg re- expensive and time-consuming to iterate
the SSL rated itself at level 1 (In~tial). peatability). even once through the three-phase im-
In the procesr-control phase, the empha- provement cycle. Financing stabilization,
OBJECTIVES AND APPROACH sis shifts to instrumenting projects to control, and change with short-term proj-
gather significant data (measurement) and ect funds is likely to be unsuccessful.
T h e results of the self-assessment analyzing the data to understand how to Discretionary funding (overhead, in-
prompted the Equipment Division to ini- control the process. dependent research and development,
tiate, in mid 1988,a process-improvement In theprocesr-change phase, the empha- and reinvested profit) is the obvious solu-
program within the SSL. The program, sis is on determining how to adjust the tion, but tl-us approach requires two im-
known as the Software Engineering Ini- process as a result of measurement and portant ingredients if management is to
tiative, was strongly influenced by the
SEI’s process-assessment project.*
The Initiative has since been the focal I 1
point for improving the division’s soft-
ware-engineering process and a channel
for institutionalizing knowledge of soft-
ware-engineering methods and technol-
ogy as well as division policy, practices,
and procedures.
The Initiative’s objectives, goals, and
actions are determined by a Steering
Committee, whch includes the SSL line
managers responsible for software devel-
opment. The major activities withm the
initiative are directed by four worlung
groups: Policy and Procedures, Training,
Tools and Methods, and Process Database
(metria). These were the areas our self-as-
sessment had identified as needing im-
provement. In the box on pp. 30-3 1, I de-
scribe these entities and some of the
specific tasks they perform.
The Initiative’s fundamental objective Figure 1. Raythem’sthree-phase proresj-improvementparadigm applies the principlesof Demingand3uran,
is the continuous improvement of the de- andpkzcesprojectsat thefocalpoint.

IEEE SOFTWARE 29
 Perhapsbecausewebasedit uniqueCharafteristcs). One of ress in their own areaand aid well as development practices.
on the results of a self-asses- itsmostimportanttasksisto communication amonginter- Dismbutfdtoalldivision
ment, the organization of the prepare a n a n n d plan and actingpmjects.TheInitiative personnelinvolvedin anyway
Initiative has stood the test of present it to the division gen- Manager (which is my tide) insoftH.aredwelopment;tfiis
time. Since we started in 1988, eral manager. monitDrsthe day-to-day activi- documentis updated periodi-
the scope of the activities un- This group adjusts priorities ties, funding allocations,and cauy,as our standard process
derclken byeachentitywithin and reallocdtesbudgets, when the statusof individual asks. WOlYtS.
theInitiativehas broadened neasary. The amount of fund- ThegrOUphaSalSodefined
somewhat, but we h v e not had ing allocated to ea& of the bur Pakydi+fOdm%rn a set of detailedprodures for
to expgnd beyond a steering workinggroupsvarieseach working group is charged with use in keyprcmss areasand a
committeeand fourworlung y”,as priorities shift.In 1992, draftinga new kh-Engi- setof +lines that include
groups. the Initiative’s $9Sl,OOO budget neering Policy fbr the division recommended support soft-
was allocated as follows: Policy and keeping it ament. ware, fw example. These de-
ShingCa&tmThisen- and Procedures, 18percent; BYAuslst1~,~group tailed pllmxhlm cover things
tity, &ch meets at least once a T W , 23 percent;Tmls and hadddidapdicythatwasap- like unit testing, costing and
month, provides direction and Methods, 30 percenq and Pro- p m v e d b y b d r v i s i a p l ~ scheduling,design and code in-
oversight It reviewsthe status cess Database (metria), 29 per- m a r r a g e s a n d d t o a l l spections, andtimingandsiz-
and plans of each working cent SSLand dlremmentanagers, in- ing.The p u p continues to ex-
group,not only in the context The compositionof the ciudmghowthepolicywtobe pand the series.
of long-termimprovementin SteeringCommittee is very im- tr;msitiolledto** Policy and pnredures is
the division’ssranrtPdproces portant It is chaired by the By the third quarter of alsochargedwithmakingsure
(dehed by the SEI as a coher- SSL manager, who is the driv- 1990, it had put in place a rea- ongoingprojects are in compli-
ent collection of management ing force b d the Initia&e sonablyhigh level,yet practical, ance withthe Equipment
and software-engineering pro- and the communication chan- description of what is involved Division’s standard process.
cesses), but also in the context nel to top-level manage men^ in developing softwarein the This includesreviewing the de-
of the short-term+dpro- Other members ax the chairs Equipment Division. TI& pro- velopment plansfor all new
cessusedbyprojerrsandpro- and cochairs of the bur work- prietary document covers all as- proposalsand Programs
pasals (a version of the m d a r d inggroups, who contribute pens of the lifecyde and in- performingconlpliancech&
process tailored to a project’s first-hand knowledgeof prog- cludes managementissues as at project milestrwes. It also

persevere: First, there must be some short- provement approach places projects at the during the first two phases as well.
term benefit to ongoing projects; second, focal point. This focus on involving proj- During the stabilization phase, th
there must be a meaningful quantification ect personnel also helps to address tech- process is documented, disseminated to a]
ofwhat that benefit is. nology-transition issues - those who practitioners, and institutionalize,
Funding for the Initiative comes from contribute to changing the process essen- through training programs. To get fund
the SSL‘s discretionaly overhead budget. tially buy into the change and so adopt it ing, we knew the SSL had to get it
Over the period of the Initiative (nearly more readily. customers (the seven business areas), whi
five years), these funds have been used to of course would be affected by any proces
support a staff of only one or two full-time Measwing effect. In the second place, in changes, to buy into the process. We hell
personnel, plus many people working l a u n c h g the Initiativewe had to consider a series of briefings for our customer:
part-time - for an average equivalent of how individual small improvements im-
~ each briefing incorporating suggestion
about 15 people per year. At any one time, , plemented more or less in parallel would made at the previous one. In this way, w
about 100 people are involved in one pro- interact to produce a net loss or gain. For got project members to buy into what pro
cess-improvement task or another. example, introducing a new technique cedures were determined to be “best,
Obviously, the Initiative relies heavily ’ probably involves a detailed analysis of al- very early on. When the htiative the]
on the part-time efforts of many people. , ternatives, the development and docu- targeted those best procedures for trainin
Fully 2 5 percent of the SSL staff have mentation of procedures to support it, and courses, we saw an immediate improve
some formal involvement in the Initiative, new training materials. Instead o f w g to ment in project performance.
and 100 percent of the staff work on pro- ascertain the value ofeach change compo- During the control phase, the Initiativ
jects addressed by the Initiative. nent, we decided it would be easier to sees to it that projects are instrumented tl
Using the skills and experiences of so measure the overall effect of a change on obtain process-related measurements i
many project personnel to drive the Initia- the bottom line. preparation for root-cause analysis.Agair
tive guarantees that the resulting process is We also had to address the question of benefits accrued from measurement, eve:
a consensus that reflects the practical con- when we could begin to measure change. at the project level. Individual engineer:
siderations of diverse projects. It also pro- ’ Some believe that the rewards of process when made aware of error-data categori
vides the immediate, short-term benefits improvement are reaped onlywhen an en- zation, tend to be more careful in area
so important to management. tire three-phase cycle has been completed. with the highest frequency of errors.
As Figure 1 shows, our process-im- Our experience shows that benefits accrue Finally, in the change phase, adjust

30 JULY 199:
 helps projects review and tailor which are
the standard process and review
the lessons learned in applymg
the standard process at the
milestone checks.

Ris to grve engmeen a way

to undersand and implement a ’fn 1
uniform dewfopmentd o d .

process automation.
The Ada e&m inchtde ac-

tionofcorpor;tteAdan&t-
teis(theseefforts are indepen-
dent of Ada uaining effons,

ments are made first on pilot projects so as
to confirm the conclusions of the previous
analyses and then on all projects. Major
cost savings should be aclueved in this
phase. Of course, tlus is the point at which
careful attention should be paid to auto-
mating any new parts of the process, to
maximize gain.
return and continue to monitor it over
time.
We used Philip Crosby Associates’ ap-
proach’ to help us qmn@ the benefit of
improvements made to ongoing projects.
vention costs is what Crosby calls “the
cost of quality.” The total project cost is
simply the cost of quality plus the per-
formance costs.
In early 1990, we used the Crosby ap-

1 CALCULATINGSAVINGS
During the approximately five years
the Initiativehas been underway, the divi-
~

sion has invested about $1 million in dis-

cretionary funds in process improvement
each year. Tlus investment has moved the
organization to a solid SEI level 3 (De-
fined) maturity level - as determined by
two separate customer evaluations in late
1991 - and we are approaching level 4
(Managed).
However, from the standpoint of mak-
ing sound investment decisions, you must
be able to show that achieving level 3 or
level 4 provides a return that justifies such
a large expenditure. We have been able to
get management approval to continue this
investment because we have measured our
+ Apprairal.The costs
associated with testing the product to de-
termine if it is faulty.
+ Rewmk (nonc4mnce). The costs
associated with h g defects in the code
or design.
+ Prevention. The costs incurred in at-
tempting to prevent the fault from getting
into the product.
The sum of appraisal, rework, and pre-
communications, or air-
traffic-control systems using Raytheon
hardware; their development schedules
were aggressive but not unachievable;
they were staffed with a typical mix of soft-
ware-engineering SUS and experience in
the application area; and they were led by
progressive managers. However, two pro-
jects (A and B) were well underway before
the Initiative began, while the other four
-

IEEE SOFTWARE 31
 ! Cost of rework
I

project members were involved and data

collection took longer.
In preparation for instituting formal
inspections, the Policy and Procedures
group wrote detailed procedures, and the
Training group taught a course in inspec-
tions to developers just before the design
stage, using materials tailored to the proj-
ect. This course not only emphasized in-
spection techmques, but it also reinforced
the ideas ofMichael Fagan: The attention
you pay to uncovering errors up front is
guaranteed to save you money in the end.8
We had to convert many skeptics to tlus
doctrine.
Another factor in these increased costs
was that we paid a lot more attention to
getting the requirements solidly in place
before beginning major development. We
achieved tlus by imposing policy; reinforc-
(C, D, E, and E’) were just starting. percent of total project cost. It’s interest- ing the policy in trainingeveryone involved
The initial results of the analysis were ing that, in 1987, Barry Boehm estimated in software,not just front-line developers, on
promising, so we repeated the analysis that eliminating rework might reduce fundamentals;and integrating a computer-
nine months later, near the end of 1990, software costs by 30 to 50 percent.’ In- aided software-engineering tool method-
when projects A, B, C , and D were fin- deed, we have shrunk rework costs during ology into our way of doing business. T h ~ s
ished and projects E and F were about 90 thls period to about one fourth its original last policy did not come cheaply,as the box
and 70 percent complete, respectively. value (from 41 to 11 percent). on p. 3 5 describes.
The second analysis indicated that we had Rework has many components, so it is
eliminated about $9.2 rnillion in rework worthwhile to analyze the cost factors that Integmfion.The largest contributor to re-
C0StS.h contributed most to the overall reduction work costs is the cost associated with h n g
Since then, nine more projects (which, and to try to determine which process source-code problems found during inte-
on average, happen to be smaller in terrns changes might have brought about these gration. This cost decreased to about 20
of delivered source code) have been added reductions. percent of its original value. We believe
to the database,and a new analysis of all I 5 Also, rework savings are often achieved this savings can be credited to the design
projects, shown in Figure 2, indicates we at the expense of a small inn-easein the cost and code inspections, procedures, train-
have eliminated about $15.8 million in re- of other process stages. In our case, the ing, and requirements stability.
work costs through the end of 1992. cost of design and coding rose slightly The greater gain achieved by m a h g
Figure 2 also shows that appraisal costs because formal inspection replaced in-
have increased by about five percent. 4-formal reviews. However, it was this
though the increased rigor with which we very change that enabled us to achieve
conducted design and code inspections in rework savings in uncovering source-
the later stage accounts for some of this code problems before software integra-
the integration phase more efficient is no
doubt due to the fact that h n g problems
this late in the process costs so much more.
A less obvious, but perhaps as important I
factor, is that errors found during integra-
li
increase, most of it is due to a 30-percent tion and eliminating unnecessary retest- tion tend to have a compounding effect on ~

decrease in total project cost, which has ing. integration costs: Very often, discoveringa
pushed up appraisal cost proportionally, as
a percentage ofthat total.
1~
single error a t integration brings the entire
process to a halt until the problem can be
k i g n mdcding. The cost of fixing defects
Prevention costs, on the other hand, found during design rose from about 0.75 resolved.Lost time duringintegration is def-
have been increasingly borne 1-y the Initia- percent to about two percent of the total initelynot a linear function of the number of
tive and so are not subject to the same project cost, and the cost of fixing defects errors in the sobare being integrated.
proportional increases. found during coding rose from about 2.5
percent to about four percent of that total. Reteshg. The second largest contributor
Rework savings. As Figure 2 shows, the These increased costs were undoubt- to rework costs is the cost of retesting,
rework costs at the time the Initiative edly caused by conducting formal inspec- which decreased to about half its original
started (August 1988) averaged about 41 tions instead of informal reviews - more value. First-time testing remained reason

32 JULY 1993
 ably constant, at about nine to 10 percent
of the total effort, because all the software
is tested at least once.
The decrease in retesting indicates that
far fewer problems were found during the
first test, a h e c t effect of removing design
and coding errors found in inspections.
Detailed procedures and training helped
indirectly to decrease the amount of re-
testing, and having solid requirements
e h n a t e s the repetitive tests that a fluid
basehe creates.
The boffom he. The bottom h e is that
these reduced rework costs are real. Pro-
jects E and F completed s o b a r e develop-
ment during 1991, with software-only
costs at about $17 million per project.
Both hished ahead ofschedule and under
budget; project E was four percent under
budget and project F six percent under
budget. When project F was delivered to
the customer, Raytheon earned a $9.6 mil-
lion schedule-incentive payment (a bonus
for meeting or beating a deadhe), which
is not included in any of the return-on-in-
vestment calculations in h s article.
RETURN ON INVESTMENT
In calculatingthe annualized return on
investment, we made several simplifyrng
assumptions.
First, the savings in one year is due not
only to investments made that year, but
also to investments made in previous
years. We ignore h s factor, considering
only the full investment during one year
and the savings in the same year.6
Second, we prorated the process-im-
provement investment for the six projects
in the 1990 sample. These six projects em-
ployed about 58 percent of the total avail-
able SSL labor for 1990 and thus received
the benefits of that percentage of the $1
million invested. The total savings of the
six projects in reduced rework was $4.48
million during 1990; thus, the return on
investment was approximately 7.7 to I (a
$4.48 million return on a $0.58 million
investment).
The project savings in h s calculation
represents only the first-order effect of the
process improvement. You could argue
~
IEEE SOFTWARE
1- . toSm-D#emtM&,
standard prixess, and the SEI’SCapbili
tools, contents and uses of test
testers, developers,and software
-
managers, eight hours Ada strut-
ture and terminology, lessons leamed.
Fundmmm3 $A&, middle-leveltask managers, grmp and project managers,20 .
hours -Lecture course on how to read, but not write, Ada source code and how top”-
ticipate in code reviews.
Detcibd(OUE~LWe usually schedule courseson a just-in-time basis to help specific
projects. In some cases, the tmining materials are tailoredto a project. The goal of these
comes is to make the attendee more productive on the project that is fun- his atten-
dance so that the trainingis self-supporting.
Design and codehpectim,senior engineers, 14hours -%pres inspection team
leaders.
SRsmZs Generatirm zlsing s0fiwap.e Tb-oughM, senior engineers, 32 hours -
Structured analysis for requirements, using the CASE tool SoftwareThrough Pictures;
genemting associated DoD-Std-2167Adocuments, such as software-quirementsspeci-
ficationsand inte&ce-re+ments s~cations.
S0jiwu-e C+gwatim ikihqwmt, engineers, 12 hours -Standard policy and pro-
cedures, tools, utilities, and resources.
Unit T&g, engineers, 12 hours -When and how to performunit testing, tech-
niques and tools, unit-test plan and procedure preparation, conducting analysis of test-
ing, and =-exit criteria. Lab for test plan and procedure.
S @ m ppoJectit4z?zugeinent,development managers,40 hours-Skills and tech-
niques to develop softwarewithin budget and on schedule.
Sojiwm Sizing and Tm’ng,senior engineers, eight hours -Estimation techniques
used for proposals and in early development, emphasizingprototypes; trackmg tech-
niques, with examples.
S$wm Engmewig with A&, engineem, 64hours -Lecture-lab courseemphasizes
hands-on, interactive exercises in Ada coding. Allaspects ofAda in detail.
D m Sazrclzg-esUiingAdz,engineers, 32 hours -Lecture-lab course on fundamen-
tal data structuresand how to use Ada p ackages.
ulder This year the working group is developingmore courses to cover a
broader range of softwareagineerirtgtopics. Cuurses on DesiiTechniqUesUsing
SoftwareThrough Pictures, Software Testing Methods, Object-OrientedMethodology,
Costing and Scheduling,Measurementand Root-Cause Analysis, and SoftwareIntegra-
tion are now being developed.
d
33

I data, we found that average productivity
Year
that benefits are derived from improved
competitive position, higher morale, and
the lower absenteeism and amition rates
that typically result from successful pro-
cess-improvement programs. Vt'e did not
include any of these potential savings in
our calculations.
Also, most of the process-improve-
ment funding comes from overhead,
whereas the sakings comes from lower
project costs, which result in lower custo-
mer billings. Thus, process improvement
achieves a higher profit only if you con-
sider that getting the job done on time and
on budget will bring in more business and
that the business mill, in turn, yield an ad-
ditional profit. Of course, this applies only
to cost-plus contracts. For fixed-price
contracts, the savings flow directly to the
bottom line.
Finally, this calculation implies that
improved performance is a direct result of
process improvement and not simply a re-
sult of improving our development envi-
ronment, acquiring more powerful tools,
and increasing our MU'S per seat, all of
which we did do. I address this question in
the box on the facing page.
Red savings? The first challenge to the
analysis of real project savings is usually if
all costs are normalized, do you really have
34
l l
1 8 , . I 1
FMAMiJASOND
1992
saings a t all? Perhaps the costs expressed
as a percentage of total project cost are
being redistributed, but the absolute proj-
ect cost is, in fact, not changing at all or
even going up!
Because the characteristics of the deliv-
ered products are unique, it is impossible
simply to compare the bottom h e s of
later and earlier projects. So to answer thls
question, we have obtained productivity
data associated with all 15 projects in the
analysis, in terms of eqriicalent delizwed
soiwc'e i?mirL~ionsper. man-month. This is
our commonly accepted measure of the
engineering effort involved in developing
the product (whether it be all-new, modi-
fied, or reused sohare).
If? calculate source-code size with a
tool that counts lines in most common
lanpages (Ada, C , Fortran, and Pascal)
and several assembly languages. This tool
excludes blank and comment lines and in-
cludes executable, logical (as opposed to
physical) lines. Alodified and reused lines
of code are weighted according to the rel-
ative effort (as compared to new code) of
niodibing or reusing it (including its re-
lated documentation).
lye realize that this is not a scientifi-
cally accurate exercise because of varia-
tions among projects, but we feel that
there are far more similarities than differ-
ences in the typesof software applicationswe
develop. Because we have used approach
forsometime-notonlytocountcodeafter
development, but also to monitor code-
growth estimates during development and
estimate code size for bids - it is widely
accepted by managers and developers.
As Figure 3 shows, after computing the
weighted average of the productivity data
using the same technique we used for cost
had, in fact, increased substantially from
1988 to 1990. T h e increase was large
enough to convince the skeptics that the
reduction in project cost from the de-
creased cost of rework was real.
We continue to update weighted average
productivity as projects are added to our
database; we now have data on 18 projects,
not all of them complete. The latest results
are reflected in Figure 3, which shows that
we have obtained an average productivity
increase of about 130 percent over the pe-
riod of the Initiative. In other words, pro-
ductivity has increased by a factor of about
2.3 in about four and a halfyears.
Comparing these 18projects over time
is a valid way to evaluate productivity im-
provement because, although every proj-
ect is unique, they are all of the same type
- real-time, embedded - and withln a
reasonable size range - 70,000 to
300,000 lines of code. The method of
measuring productivity did not change,
either. Thus, if neither the nature of the
application nor the measurement method
changed in thls time, it is reasonable to
credit the improvement to what did
change, namely, the process.
Schedule and budget performance
gains are also impressive. Before the Ini-
tiative, most projects were completed be-
hind schedule and over budget. In many
cases, key personnel were being moved to
projects on the basis of crisis priorities. In
other words, we were operating in a fire-
fighting mode. Thls may have been the
reason our management was so receptive
to the Initiative proposal.
Now, most projects are finishing on or
ahead of schedule and below budget.
Less tangible rewords. In addition to the
project characteristics that we carefully
JULY 1993

track to document process-improvement
success (cost of rework, productivity, and
cost-performance index), we have ob-
served less tangible but equally important
results.
Software engineers are spending fewer
late nights and weekends on the job, and
there’s less turnover. There’s less pressure
being applied to get back on schedule, be-
cause schedule erosion is occurring much
less frequently.
T h s is certainlynot a phenomenon of
a reduced contract workload; on the con-
trary, in the last six months we have hired
100 SSL staff members to meet the de-
mands of new business.
Communication has improved. The
Initiativefunds a quarterly SSL staff news-
letter that includes a current list of ongo-
ing projects and proposal efforts.
Because the Initiative is now five years
old, we do not feel these benefits can be
attributed to the Hawthorne effect (which
says that anythmg under study wdl im-
prove simply because it is under study).
We continue to get enthusiasticp d c i p a -
tion from SSL staff in process-improve-
ment activities.
Of course, it’s difficult to point to direct
causal relationships for these h d of kctors,
but i n v o h g 2 5 percent of the personnel in
improving a process that they themselves
apply must make for higher levels of inter-
personal contact and job satidction.
We’ve experienced a definite culture
shift in the area of training. We no longer
expect practitioners to absorb the SUS
they need on the job (by associating with
other workers). Instead, we have a com-
prehensive, fully funded training pro-
gram. The courses are listed in the box on
p. 33. Project managers who once insisted
that all training be funded by overhead are
now accepting proposal bids that include
training costs, because we can demon-
strate a direct benefit to the project.
Naw that customers are using the SETS i s traditional position as the sobare-
software-capabjlityevaluations as a discrim- ngineering function has become more phyics from Providence College. He is a member of
inator in contract awards, process capability iature, disciplined, and consistent. In the SoftwareEngineering Institute’sCapabilityMatu-
has &en a seat much closer to the action. In
ha,we have recently been awarded at least 3r hardware engineering, spawning a
two contracts in which our sobare-process
capability was said LDhave made the Mer-
ence between winning and losing.
IEEE SOFTWARE
I
t program in open fonuns, Iamfie-
I report d t e d Erom the Initiative,
t, more powerful d s , and a mani-
orgmizatMnirnosntsplemaaer:Thereare
thathavefailedtopruduceimpmve-
Raytheon take full advantage oft&-
fCASEmls:Overtwotothreeyem,the
tire organlzatin, which has embraced the idea of wntinuously impraving the way we do
ourworkandiscommitted to
t h e p a s . The process was
mtiallyimproved.Why
ACKNOWLEDGMENTS
I thankBob Charette, Watts Humphrey, Blake Ireland, Stan &&in, and Jim Stubbe for their helpful re-
view comments. The comments and suggestions OfZEEESq’harc’r anonymous referees were very helpful.
REFERENCES
1. W.S. Humphrey, Managing the sofrure Pmces, Addison-Wesley, Reading,Mass., 1989.
2. W.S. Humphrey et al., “AMethod for Assessing the Software Engineering Capabilityof Contractors,”
Tech. Report CMU/SEI-87-TR-23, Software Engineering Institute, Pittsburgh, 1987.
3. WE. Deming, Out ofthe Crkis,MIT Center for Advanced Engineering Study, Cambridge, Mass., 1982.
4. J.M. Juran,Frun on Plamzmgfi Quulity,The Free Press, New York, 1988.
5. P.B. Crosby, Quahty Whmt Eun,McGraw-Hill, New York, 1984.
6. R. Dion, “Elements ofa Process-Improvement Program,”IEEE soffwure,July 1992,pp. 83-85.
7. B. Boehm, “Improving SoftwareProductivity,” Conputer, Sept. 1987,pp. 43-57,
8. M. Fagen, “Advanced in SoftwareInspections,”IEEE Puns. SofnuareEng.,July 1988, pp. 75 1-774.
k
nyone who has developed software
r the last 10 to 15 years in an o r e -
ization that is in the business of develop-
Raymond Dion is a prina-
pal engineer with Raytheon’s
Equipment Dvision, where
ig large, embedded systems can appreci- he has managed the software
te the traditional difference in the level of process-improvement pro-
espect accorded hardware and software gram since its inception. He
ngineers. has more than 30 years expe-
rience in software engineer.
We have seen a definite shiftaway from ing.
Dion received a BS in
rity Model Advisory Board and Software Capability
ict, our Initiative has become a model Evaluation Review Group and the IEEE.
Iardware Engineering Initiative that
Address questions about this article to Dion at Ray-
as profited from the lessons learned theon Equipment Division, 528 Boston Post Rd., Sud-
-om software. +
bury, MA 01776; Internet rdion@sud.ed.ray.com.
35