0 valutazioniIl 0% ha trovato utile questo documento (0 voti)
14 visualizzazioni40 pagine
The following are the most common perspectives on quality: 1. Transcendent 2. User-Based 3. Conformance to requirements 4. Product-Based 5. Value-Based Figure 2 shows a conceptual depiction of each one of these perspectives.
The following are the most common perspectives on quality: 1. Transcendent 2. User-Based 3. Conformance to requirements 4. Product-Based 5. Value-Based Figure 2 shows a conceptual depiction of each one of these perspectives.
The following are the most common perspectives on quality: 1. Transcendent 2. User-Based 3. Conformance to requirements 4. Product-Based 5. Value-Based Figure 2 shows a conceptual depiction of each one of these perspectives.
Resumen de lecturas basado en la siguiente bibliografa de base:
1) Garvin, D. A., "What does 'Product Quality' really mean?", Sloan Management Review, no. 1, pp. 25-43, 1984. 2) McCall, J. A., Richards, P. K., and Walters, G. F., "Factors in Software Quality", Nat'l Tech.Information Service, no. Vol. 1, 2 and 3, 1977. 3) Boehm, B. W., Brown, J. R., Kaspar, H., Lipow, M., McLeod, G., and Merritt, M., Characteristics of Software Quality, North Holland, 1978.
4) Boehm, Barry W., Brown, J. R, and Lipow, M.: Quantitative evaluation of software quality, International Conference on Software Engineering, Proceedings of the 2nd international conference on Software engineering, 1976. 5) ISO, International Organization for Standardization, "ISO 9126-1:2001, Software engineering - Product quality, Part 1: Quality model", 2001. 6) CMMI for Development, Version 1.3/ CMMI-DEV, V1.3 / CMU/SEI-2010-TR-033, ESC-TR-2010-033
2 Resumen de lecturas: Modelos de Calidad. Concepts and Principles What is Quality? Quality is an elusive concept. Most people can recognize quality and give examples of quality, but have trouble supplying a clear, unambiguous definition of the term. In everyday discourse this usually isn't a problem because language is very tolerant of ambiguity. However, before the quality goals of a project can be specified, managed, and assured, we need a good working definition of the term quality. This first section explores different perspectives on quality in general. These perspectives provide a foundation for the definition of software quality in particular presented in the next section. The following are the most common perspectives on quality [Garvin, Crosby]: 1. Transcendent 2. User-Based 3. Conformance to Requirements* 4. Product-Based 5. Value-Based Figure 2 shows a conceptual depiction of each one of these perspectives on quality. ______________________ *Garvin refers to conformance to requirements as the manufacturing-based view because it is the view preferred by those responsible for the production phase of a manufacturing process. The term conformance to requirements is used here because it is more descriptive of the software development environment where products are developed rather than manufactured. ______________________
Figure 2. Perspectives on quality. Transcendent. The transcendent view of quality equates quality with "innate excellence". According to this perspective, quality can't be described with words, it can only be understood through experience. The transcendent view of quality relies on perception and experience but isn't completely subjective. For example, wine tasting experts generally agree on the relative merits of different vintages of wine which suggests that the activity isn't completely subjective. Robert Pirsig takes on the topic of quality in Zen and the Art of Motorcycle Maintenance and concludes, "Quality is not objective... It doesn't reside in the material world....Quality is not subjective...It doesn't reside merely in the mind....Quality is neither a part of mind, nor is it a part of matter. It is a third entity which is independent of the two." [Pirsig] The transcendent view of quality is a poor basis from which to develop an operational definition of quality. The most obvious problem is measurability. There is no precise way of quantifying innate excellence. The transcendent view of quality is better suited to products that appeal to the senses. Software is practical and utilitarian and therefore not well suited to the transcendent view of quality. User-Based. The user-based view of quality is certainly more down-to-earth than the transcendent view. According to the user-based view of quality, quality is the satisfaction of user wants or needs. If product A satisfies more user wants and needs than product B, then product A has higher quality than product B. There are two potential problems with this definition of quality though. The first problem is that of designing a product that simultaneously meets the needs of a diverse group of users. For example, the popular image editing program PhotoShop is used by average consumers, professional photographers, and high-end animation studios. It would be difficult to design a product that simultaneously meets the needs of this diverse group of customers. What might be satisfying to one customer might be completely unacceptable to another.
Figure 3. Quality is only one component of user satisfaction The second problem with this perspective is more fundamental. User satisfaction is based on an aggregate of product factors--only one of which is quality. (See figure 3.) Just because a user is satisfied with a product, you can't conclude it is exclusively because of high quality. A user might be satisfied with a low quality product if the other factors that influence satisfaction are high enough to compensate for the dissatisfaction attributable to low quality. To say that user satisfaction is a good measure of product quality is to ignore the other factors. For example, there is a robust market for classic Jaguar automobiles from the early 90's. This suggests that these cars are providing user satisfaction. However, Jaguars of this vintage are notorious for electrical and mechanical problems. These cars might be meeting the overall needs of their buyers, but even their most enthusiastic admirers would freely admit that they are not high quality. More likely, it is a combination of product features including their visceral beauty and style that make them appealing in spite of their reputation for poor quality. (In fact, for some owners it's possible that the sense of adventure that comes from driving a low quality unpredictable automobile adds to their appeal.) There is certainly value in having a definition of quality that defines quality relative to user wants and needs. However, before this perspective can serve as a basis for quality management, there has to be some way of measuring the extent to which product quality is contributing to user satisfaction independent of other product features that might also be influencing overall user satisfaction. Conformance to requirements. Quality as conformance to requirements is a popular operational definition of quality. Philip Crosby, respected author and evangelist of quality, is probably the best-known advocate of this interpretation [Crosby, Quality is Free, p. 17]. It is also the definition used in the well-known ISO 9000 quality standard. When quality is defined as conformance to requirements, there is no ambiguity about what quality is. A quality product is one that conforms to specified requirements and design. According to this definition of quality, a meal at McDonalds is high quality as long as it conforms to the specifications of the franchise. By the same token a meal at the Four Seasons could be considered of lesser quality if, for example, the salad fork isn't chilled. If this definition of quality bothers you it's probably because you are equating quality with luxury or goodness [Crosby]. Viewed in this way, a Lexus has more quality than a Hyundai. A Rolex watch has more quality than a Timex watch. This is a common interpretation of quality but it's incompatible with the interpretation of quality as conformance to requirements. Quality as conformance to requirements is absolute. Quality can't be both conformance to requirements and luxury or goodness. The source of the confusion is in the use of the same term for two different concepts. A manufacture may plan a product with a certain level of quality (luxury or goodness). In order to deliver on this plan, the manufacture will need to maintain high standards of quality (conformance to requirements) during production. Quality as conformance to requirements is a good operational definition of quality because manufactures of all types of products can specify quality goals and control progress towards the accomplishment of these goals. Another potentially confusing aspect of defining quality as "conformance to requirements" is that it seems to disregard the needs of the user. Quality as conformance to requirements doesn't discount the importance of meeting the needs of the user, it just assumes that these needs will be completely expressed in the requirements. This puts the onus on the requirements process to capture the true requirements with accuracy and precision. The principle advantage of defining quality as conformance to requirements is that it simplifies the production or implementation phase of the product life cycle. It doesn't make it any easier to identify the right set of product features that will lead to customer satisfaction, but it does make it easier to manage and control the production or implementation phase of the product life cycle. When quality is defined as satisfaction of user needs the whole life cycle is working toward subjective hard-to-measure goals which makes the whole process hard to control. When quality is defined as conformance to requirements, the requirements phase isn't any easier to control, but the design and production phase are working toward an objective easier-to- measure goal--conformance to written requirements. The advantage of defining quality as conformance to requirements is that it makes it possible to define quality goals in specific terms and then measure and control progress towards these goals. Conformance to requirements is the definition of quality preferred by manufactures because it provides an objective easy-to-measure goal for quality during the production phase of the product life cycle. Product-Based. With the product-based approach to quality, differences in quality are measured by differences in the quantity of some ingredient or attribute of the product. For example, one way to measure the quality of fabric is by thread count. Fabrics with a higher thread count are usually considered more desirable. In practice, the quality of a product is often a function of multiple attributes, some more important than others. For example, three attributes that can be used to measure the quality of a digital camera are resolution, light sensitivity and frame rate. These aren't the only attributes by which the quality of a digital camera can be measured and how much each attribute contributes to overall product quality will be different for different buyers. Camera Quality = W Resolution * Resolution + W Sensitivity * Light Sensitivity + W Storage Capacity * Storage Capacity The challenge when defining product quality using this definition is determining the attributes and their weights. If the choice of attributes and weights are subjective, there may be no choice of attributes and weights that is optimal for all customers. The section on Software Quality Models in this chapter describes various models for defining software quality in terms of product attributes. Value-Based. The value-based approach to quality considers value to be an aspect of quality. Adding features to a product and controlling its defects, beyond a certain point, will likely increase its cost. Higher cost means less value. Tying value to quality means that as the cost of a product goes up, its perceived quality goes down. According to the value- based approach to quality, a quality product is one that has the features the customer desires, limited defects (if any), and is offered at a price that is acceptable to the customer. This approach to quality explains why, when shopping for flooring, linoleum starts to look better than tile after seeing the price of each. In practice, this view of quality confuses product quality with desire for a product. A customer may marvel at the quality of a pair of shoes, but upon seeing the price may loose interest in purchasing them. The quality of the shoes haven't changed but the customer's desire for owning a pair diminishes because they don't represent good value for the customer. The relationship between software quality and cost is explored in the section titled Cost of Quality.
When planning for software quality it's important to keep in mind the hierarchical nature of these factors [Humphrey 2005, p 136]. A relevant analogy is Abraham Maslow's theory of human motivation. Maslow's popular theory of motivation posits that humans are driven by a hierarchy of needs. Certain lower level needs including safety and security have to be met before higher level needs such as those for loving and belonging can be addressed. Once a need is satisfied, it no longer motivates and the next higher level need takes its place.
Figure 7. The primary factors that influence users' perception of software quality change as quality improves. There is an analogous hierarchy of factors that drive users' perceptions of software quality. Ease of use doesn't mean much if the frequency of defects prevents the software from being used at all. Lower level quality factors have to be addressed before higher level factors become relevant. Correctness. Quality starts with understanding the real problem and delivering a correct solution. If users ask for a mortgage calculator and they are given a bond trading system, the system wont be used and any other quality factors don't matter. System requirements are a mix of functional and non-functional requirements. The quality of a software system is measured by the degree to which functional as well as non- functional requirements are met. Correctness is the degree to which stated and implied functional requirements are met. The word implied makes it clear that quality isn't strictly conformance to written requirements. If a desired feature isn't implemented because it's not included in the requirements, it's a deficiency of the requirements process that also diminishes the overall quality of the product. Reliability and Availability. Beyond correctness, users look for reliability and availability. A mortgage calculator isn't effective unless it is available when needed and able to operate reliably for the duration it is needed. The frequency and severity of defects determines the reliability and availability of a software system. For most software systems, zero defects is an impossible goal. If there are any defects, they should interfere with the normal operation of the system. This usually means no severe defects and a limited number of minor ones. Usability. Usability refers to the ease of use or user friendliness of an application. Usability is the measure of effort required to learn and use the application. A mortgage calculator is easy to use if users understand the meaning of each field and can easily calculate the mortgage for different scenarios. Proficiency. Proficiency addresses the needs of power users. It is a measure of how well the software enables work to be done quickly and efficiently. A mortgage calculator that responds quickly (performance) and has short-cuts for frequently used operations is proficient. Enjoyment. Products at this level achieve an emotional appeal that goes beyond mere functionality. The best examples of companies with products at this level are: Apple, BMW and Hermes. Imagining an enjoyable mortgage calculator requires a stretch of the imagination, but an enjoyable mortgage calculator is one that you enjoy using because it makes you feel good. The feeling might come from a coherent design style, superior craftsmanship and/or attention to detail. In addition to being hierarchical, another subtle distinction among software quality factors is that some lead to satisfaction while others lead to dissatisfaction. Furthermore, these two sets of factors are mostly disjoint. Lower level quality factors such as correctness, availability and reliability have to be present in order to avoid negative feelings (dissatisfaction), where as high level factors such as usability, proficiency and enjoyment are needed to create a positive impression (satisfaction). Users expect correctness and reliability, they are delighted when they get proficiency and enjoyment.
Figure 8. Quality factors causing satisfaction are not the same as those causing dissatisfaction
Cost of Quality Everyone is for higher quality, but the reality is that expenditures on quality improvement initiatives are economically justified only when the expected savings from higher quality is greater than the initial upfront investment it takes to achieve higher quality. The cost of quality is both a metric and economic model for understanding the cost-benefit trade-offs of pursuing higher levels of quality. It can be used at the start of a project to justify an investment in quality improvement or at the conclusion of a project to measure the outcomes of such expenditures. Quality is one of the 4 variables of a project that has to be balanced against the other three which are cost, schedule and features. The following conceptual formula expresses the general relationship between these four variables: Cost * Schedule = Features * Quality The formula suggests that improving quality requires some combination of higher cost, longer schedule, or fewer features. Holding schedule and features constant, quality correlates with cost: Quality = a constant * Cost In practice the true relationship between quality and cost is not so simple. In some cases it's possible to increase quality without increasing costs. To understand why, consider that cost represents not only the money spent on quality improvement (mainly defect prevention and early reviews), but also the money not spent on late stage testing and rework when defects are avoided through additional quality improvement efforts. In other words: Cost-of-Quality = (cost of defect prevention and early reviews) + (cost of late stage testing and rework) Increasing the first category of costs (prevention) tends to reduce the second category (rework). When the amount saved by avoiding defects rises faster than the amount invested in order to avoid these defects, quality is free*. ________________________ *Philip Crosby was so convinced of this dynamic that he titled one of his books Quality is Free, lest anyone would miss the main point which was that the major costs of quality are the costs associated with poor quality. Crosby maintained that, as the voluntary costs of defect prevention are increased, the involuntary costs of poor quality (mainly rework) decrease by an even greater amount thus making quality free [Crosby]. ________________________ How much does quality cost? Is quality free? Is there a point of diminishing returns, beyond which an investment in quality is no longer cost-justified? The cost of quality model helps to answer these questions by modeling the relationship between quality and costs*. _____________________ *When discussing the cost of quality, quality is defined as freedom from defects. It explicitly doesn't mean luxury or features. More luxury and features almost always means higher cost. _____________________ Formally the cost of quality is the sum of conformance costs and non-conformance costs. Conformance costs further breakdown into prevention and appraisal costs. Conformance costs are the costs associated with attaining quality. These are the voluntary investment costs for things like defect prevention and early reviews. Non-conformance costs are the costs of dealing with defects. This includes things like debugging and rework resulting from exposed defects. These are involuntary costs and a source of potential savings if defects can be reduced. Non-conformance costs may be internal failure costs (cost of errors found before the product is released), and external failure costs (cost of errors found once the product has been released). Figure 16 shows the taxonomic breakdown of these costs and figure 17 shows when these costs generally occur during the software life cycle.
Figure 16. Taxonomy of Software Quality Costs
Figure 17. Relative Timing of Cost of Quality Expenses The following is a closer look at these 4 categories of costs. They are also summarized in table 2 below. Prevention costs - These are the costs associated with preventing defects from ever occurring. If you can prevent errors from occurring you can avoid the expense of finding and fixing them (not to mention the wasted effort spent making the error in the first place). Prevention costs include the costs of training, early reviews, quality planning, root cause analysis, baseline configuration management, and process improvement initiatives. Appraisal costs - These are the costs associated with analyzing and testing the product to make sure it conforms to requirements. The most common appraisal costs are the costs for testing, inspections and quality control. Internal failure costs - These are the costs associated with fixing defects found prior to product release. This includes the cost of rework, retesting, and the administrative overhead of pre-release defect management. It may also include the cost of updating documentation, additional reviews and inspections. External failure costs - These are the costs associated with fixing defects found after the product has been released. The process of diagnosing and fixing an external failure is much more complicated and involved than fixing an internal failure so costs are higher. There are the regular expenses that go along with internal failures, plus additional expenses that come with field defects. Less tangible external failure costs are the costs associated with possible damage to reputation and the loss of future sales. External failure costs may also include the unpredictable costs of litigation. Category Definition Example Prevention Costs associated with preventing defects. Training, early reviews, quality planning, tools, process improvement initiatives. Appraisal Costs associated with analyzing and testing the product to ensure it conforms to specifications. Inspections, testing, audits, quality control. Internal Failure Costs associated with fixing defects found prior to release. Repair, retesting, updating documentation. External Failure Costs associated with fixing defects found after release. Technical support, defect reporting and tracking, field updates, loss of future sales. Table 2. Software Quality Cost Categories The cost of quality is the sum of all conformance and non-conformance costs. The cost of quality is a real expense that shows up indirectly in the budget of every project or directly when the cost of quality is being tracked. It is also an expense that can be controlled by the project manager. In order to control the cost of quality the project manager must understand the complex relationship between conformance costs and non-conformance costs. There are three notable aspects of the relationship between conformance and non- conformance costs. First, the two categories of costs tend to be inversely related. Spending more on conformance should reduce non-conformance costs. Spending less on conformance will likely increase non-conformance costs. Second, conformance costs are voluntary while non-conformance costs are involuntary. The project manager decides how much to spend on conformance, and this in turn determines how much will be spent involuntarily on non-conformance. Third, conformance costs represent an investment that must be justified by even larger anticipated savings in non-conformance costs. It is an investment that usually has a point of diminishing returns beyond which spending more on conformance won't reduce failure costs enough to justify the initial expense and risk. The project manager is responsible for determining the optimal amount to spend on conformance activities such that the overall cost of quality (conformance and non- conformance costs) is minimal. Projects vary in size and budget so the amount won't be an absolute number but rather some portion of the total budget. The optimal cost of quality is usually expressed as the cost per good unit of product produced or as a percentage of total development costs. The problem of finding the optimal cost of quality is made more difficult by the fact that the optimal balance between conformance and non-conformance costs depends on the characteristics of the project and the maturity of the development process being used. There isn't one precise answer, but there are models for understanding in general the forces that affect the optimal balance in different environments. Figure 18 shows two such models. Figure 18.a shows the traditional cost of quality model and figure 18.b shows the contemporary one. The models depict the general relationship between costs and quality for different types of projects.
Figure 18.a. Traditional Cost of Quality Model
Figure 18.b. Contemporary Cost of Quality Model Both models show the costs of quality in terms of conformance and non-conformance as quality increases (the x axis). In both models the total cost of quality is the sum of conformance and non-conformance costs. Also in both models, non-conformance costs drop to 0 as quality reaches 100 % or zero defects. The models differ in their assumptions about the amount of investment needed in conformance to achieve perfect quality. In the traditional model, a sharp increase in conformance cost is needed to achieve perfect quality. The contemporary model shows a more modest increase in conformance costs as quality rises, thus making it possible to have both perfect quality and minimal cost. "Quality is free but perfection you have to pay for." Which model is correct? They are both correct, for the project environments they depict. The conventional cost of quality model in figure 18.a is appropriate for project environments where the cost of failure is low and the effort required to achieve zero defects increases rapidly as product quality approaches 100%. For example, this model accurately depicts the cost of removing all of the errors from a large daily newspaper. The cost of an error is minimal and the cost of removing every single error is prohibitively expensive. It could take as much time to find the last single error as it took to find all of the other errors combined. The economic impact of a reader finding an error isn't worth the extraordinary effort it would take to find all the errors. To paraphrase Crosby, the traditional cost of quality model seems to be saying that quality is free but perfection you have to pay for. The contemporary cost of quality model in figure 18.b is more appropriate in environments where: The cost of failure is high which justifies increased spending on conformance. There are more effective development and appraisal methods (usually automated) which reduces the cost of conformance. More effective development methods reduces the total number of errors and more effective appraisal methods reduces the cost of finding and removing the errors that are introduced. In some cases complete testing may be both feasible and economical. As an example, consider the development process for microprocessors. The cost of putting a defective microprocessors into production is very high thus justifying increased spending on conformance. Modern production processes limit the number and types of errors introduced during development, and automated testing tools in most cases make it economical to perform complete testing during appraisal. This reduces the cost of conformance. So, which model is most appropriate when the environment is software development? Again, it depends on the specific project, but for the average software development project the traditional cost of quality model is probably more appropriate. Not all of the trends that are making the contemporary cost of quality model more applicable in manufacturing apply when the product is software. The reason for this rather pessimistic assessment is that first, software development is a human-intensive activity. Humans are fallible, so it follows that software will always have defects as long as humans are in the loop. Second, all but the most trivial software systems are too complex to completely test. There will always be some inputs that haven't been tested. The difficulty of preventing all defects and the inability to test for all inputs, suggests that for most software products the goal of zero defects is either impossible or prohibitively expensive. This implies that for most software projects there is a point of diminishing returns beyond which improvements in quality are not cost-justified. The above conclusion is mainly a theoretical result--and not a very original one at that. Most software practitioners are well aware that for most software systems complete testing is impractical. What would be more useful is some way of predicting the point of diminishing returns for a software system. Looking at figure 18.a it appears that the rising cost of conformance (principally appraisal costs) is what causes the total cost of quality curve to reverse course and head upward. However, a rapid rise in conformance costs doesn't by itself determine the low point in the cost of quality curve. The cost of quality is conformance costs plus non-conformance costs. If the cost of a failure is high it can justify high, even exponentially increasing, conformance costs. For example, figure 19 shows the cost of achieving ultra high reliability with the primary flight control software on the Space Shuttle. As you might expect, developing near perfect code for the Space Shuttle is an order of magnitude more expensive than developing what might be considered very reliable software (1 defect per KSLOC). In spite of the high conformance costs necessary to achieve near-perfect code, the total cost of quality is still minimal because the cost of failure is even greater. The high cost of a defect in the Space Shuttle software in terms of human life, vehicle expense, and national pride can justify large expenditures on conformance. In general, a rapid rise in appraisal costs alone isn't enough to trigger a point of diminishing returns. Applications that have very high reliability requirements can justify very high, even exponentially increasing, conformance costs.
Figure 19. Cost of ultra-high reliability in space shuttle software [Image is from: Herb Krasner, Using the Cost of Quality Approach for Software, Crosstalk, Nov. 1998. Original data is from: Krasner, H., "A Case History of the NASA Space Shuttle Onboard Systems Project," SEMATECH Technology Transfer Report 94092551A-TR, Oct. 31, 1994] As the cost of failure rises, increased spending on conformance is justified and the point of diminishing returns in figure 18.a shifts to the right. In other words, a high cost of failure can justify large investments in quality improvement. When calculating the cost of failure the non-obvious costs are often overlooked. These include: 1. The intangible costs of failure, including loss of good will and future sales. 2. The user's cost of failure. Factoring in the user's cost of failure may justify even higher investments in conformance. The point of diminishing returns is also shifted right by improvements in the standard development process. With an improved development process fewer defects will be injected and conformance activities will be more efficient and effective. This makes higher quality more economical. Software Quality Models Models are common tools in software development. There are software life-cycle models (spiral, waterfall, etc.), process improvement models (CMMI, ISO 9000, etc.), and cost estimation models (COCOMO, SLIM, etc.). Models are used to manage complexity. They provide an abstraction of a system or phenomenon from a particular perspective for a particular purpose. This section introduces another type of software development model-- the software quality model. Software quality models provide a framework for expressing and measuring software quality as a collection of desirable attributes (the product-based perspective on quality mentioned above). Just as software life-cycle models help manage the software development process by suggesting activities and their timing, software quality models help manage the specification and measurement of quality in software systems by specifying quality characteristics, their relationships, and associated metrics. This section presents three well-know quality models: 1. McCall's quality model 2. Boehm's quality model 3. The ISO/IEC 9126 Product Quality Standard These models can be used as-is or as a starting point for creating project-specific or organization-specific software quality models. All three models have a common hierarchical structure which is illustrated in figure 8. High level quality factors represent external quality characteristics of interest to users, customers and managers. The models relate these high-level quality factors to lower-level subfactors that represent attributes of the software or production process. These subfactors are more meaningful to developers and technical staff. In order to quantify the presence of each factor in a system, metrics are defined for all subfactors. It may also be possible to define direct metrics for some higher level quality factors.
Figure 9. Framework for quality models McCall's Quality Model McCall's quality model is shown in figure 10. It groups software quality characteristics according to the main phases in the software life cycle. The high-level categories of quality characteristics are: 1. Product operation - quality characteristics related to using the product. 2. Product revision - quality characteristics related to maintaining the product in its current environment. 3. Product transition - quality characteristics related to porting the product to a new environment.
Figure 10. McCall's Quality Model Like most software quality models the high level quality factors in McCall's model generally represent external emergent system properties. Emergent system properties are system characteristics not attributable to specific elements of the system but rather those that emerge from the structure and interaction of individual system elements. Low level quality factors in the model refer to more tangible internal properties of the system. McCall's model links high-level quality characteristics of interest to end users to low-level system properties that developers understand. More generally, it provides a vocabulary of quality factors that users and managers can use when setting the quality goals of a project. For example, end users may decide that usability is an important quality requirement. When quality requirements are stated in terms of high-level quality factors, it's not always clear to technical staff what specific product properties are needed in order to accomplish the desired system qualities. McCall's quality model breaks down high-level quality factors into their constituent components or quality criteria. These low level quality criteria are more familiar to technical staff and represent product properties that can be built directly into work products. Besides suggesting quality characteristics and their relationships, McCall's model--and the others that will be examined--also suggest metrics for measuring the degree to which quality characteristics are present. McCall's model breaks down high-level quality factors that are difficult to measure into low-level detailed quality criteria which are more tangible and measurable. For each quality criteria, McCall suggests one or more metrics for measuring the degree to which the system possesses or exhibits the associated quality criteria. For example, storage efficiency may have the associated metrics: (1) total disk space required, and (2) average internal memory required while the program is running. Every quality criteria has at least one metric, but metrics may also be associated with some high level quality factors. For example, mean time to failure (MTTF) might be used to measure reliability. One of the key features of McCall's model is that it provides a quantitative measure of quality. Metric values are expressed in quantitative terms and the relationship between quality criteria and quality factors is formally expressed with a linear equation of the form: QF a = c 1 *m 1 + c 2 *m 2 + ... + c n *m n
Where: QF a is the degree to which quality factor a is present. c i is a constant which denotes the relative importance of the measurement m i to quality factor a. This constant may be determined by using regression analysis on historical data from similar projects. m i is a metric value associated with quality factor QF a . As an example, the quality criteria "self-descriptiveness" which is associated with quality factor maintainability might have the the metric "number of modules with standard headers". If 9 out of 10 modules have a standard header, and historical data suggests that this metric has a 30% influence on the quality factor maintainability, the formula becomes: QF maintainability = ... + 30% * 9/10 Note, that the the number of modules with standard headers is divided by the total number of modules. Metric values should be normalized to remove any bias related to the size of the product. The formal relationship between metrics and quality factors is captured in the equation's coefficients (c i in the equation above). McCall showed that for a limited domain significant correlation could be established between predicted quality and actual quality. However, its not clear that a quality model calibrated for one domain is generalizable to other domains. Like other software models calibrated from empirical data, the accuracy of McCall's model depends heavily on tuning and using the model on projects that are similar in type. One of the ways McCall's software quality model is distinguished from the others that are presented here is that it defines software broadly to include not only code but also requirements and design. A few of the quality characteristics in McCall's model are limited to code (i.e. efficiency and instrumentation), but most are general enough to apply to code and other pre-code artifacts such as requirements and design. For example, traceability and completeness are two desirable characteristics of requirements and design documents. Boehm's Quality Model Boehm's quality model is shown in figure 11. It has the same basic categories of quality characteristics found in McCall's model. In Boehm's model, overall system quality (called general utility in the model) is the aggregation of portability, usability (called "as is utility" in the model), and maintainability.
Figure 11. Boehm's Quality Model Like McCall's software quality model, Boehm's quality model is hierarchical with metrics defined for lower-level quality attributes. Boehm's model is similar to McCall's but does have a few notable differences. One obvious difference is its factoring of quality characteristics. As an example, in McCall's model maintainability and testability are at the same level which means in McCall's model a product can be maintainable without being testable and vise versa. In Boehm's model, testability is a subfactor of maintainability, which means an increase in testability implies an increase in maintainability. By itself this isn't necessary an important distinction but it does point out the difficulty of having a definitive quality model. Another way Boehm's model is different from McCall's model is that Boehm's model applies only to code. Boehm's quality model includes 151 metrics which provide measurements of 12 primitive quality characteristics. Each metric is expressed in the form of a question. Stated in this way the metrics form a checklist of good programming practice. Table 1 shows a partial checklist for the quality characteristic self-descriptiveness. Self-descriptiveness is the extent to which a program or module contains enough information for the reader to understand the function of the module and its dependencies. Self-descriptiveness is a measure of a product's testability and understandability. 1. Does each program module contain a header block of commentary which describes program name, purpose, modification history, and assumptions? 2. Are the functions of the modules as well as inputs/outputs adequately defined to allow module testing? 3. Where there is module dependence, is it clearly specified by commentary, program documentation, or inherent program structure. 4. Are variable names descriptive of the physical or functional property represented? 5. Do functions contain adequate descriptive information (i.e., comments) so that the purpose of each is clear? Table 1. Partial checklist for the quality self-descriptiveness in Boehm's quality model By expressing quality metrics in the form of checklists, the metrics used for quality measurement can also be used for code reviews and general process documentation and improvement. The metrics developed in Boehm's original work are for Fortran programs, making them mostly outdated by today's standards. However, the quality characteristics and their factorings as shown in figure 11 above are just as applicable today as when they were proposed in 1978, demonstrating that the principles of quality are timeless. The ISO/IEC 9126 Product Quality Standard ISO/IEC 9126 is an international standard for defining and measuring software product quality. It comes from the International Organization for Standards (ISO) which is also the publisher of the popular ISO 9000 series of process quality standards (ISO 9000, ISO 9001, ISO 9000-3) ISO/IEC 9126 is a software product quality standard designed to complement the well-known process quality standards offered by the ISO. ISO 9126 isn't remarkably different from the other software quality models discussed here, but it is distinguished in some important ways. First, it is an internationally recognized standard and a safe choice for any organization looking for a standard way to define and measure product quality. In the 80's there was a popular saying that nobody got fired for buying an IBM computer. Today, it can be said that no one gets fired for choosing to follow ISO standards. A universal standard provides a common language and framework for specifying and measuring software product quality. Having a common definition of software quality makes it easier to compare the quality of one product with another. Just as some organizations rely on internationally recognized ISO process quality standards to ensure a certain level of process quality, ISO 9126 can be used to ensure a certain level of product quality. Second, the ISO has shown a willingness to revise the standard and keep it up-to-date. Quality characteristics are generally timeless, but the metrics associated with quality characteristics depend on current technology and product environments. Since being introduced in 1999, the ISO 9126 standard has gone through 4 significant revisions or updates. The original version of ISO 9126 introduced in 1991 was rather sparse. The original title of the standard, "Information technology -- Software product evaluation -- Quality characteristics and guidelines for their use", included almost as much information as the actual standard. As the title suggests the original standard offered quality characteristics and a process for applying them. The standard was essentially 6 high level quality characteristics and their definition. The standard claimed that the 6 quality characteristics were sufficient to represent any aspect of software quality. The model was limited because it didn't offer a standard factoring or taxonomy of quality characteristics or associated metrics. Sub-characteristics and metrics were suggested in an annex to the standard but they weren't part of the official standard. Overall the standard identified the elements of a software quality model for specifying and measuring software product quality, but fell short of offering a complete and comprehensive model. Figure 12 shows the evolution of the ISO 9126 standard since its introduction in 1991. The original standard is now represented in two separate sets of standards. The ISO 9126-x series represents the expanded quality model, and the ISO 14598-x series details a process model for evaluating quality. The new standard has evolved significantly from its roots. The two main changes are (1) suggested quality sub-characteristics that were informative in the 1991 standard are now normative, and (2) an expended set of suggested metrics have been added for measuring quality characteristics from multiple perspectives.
Figure 12. Evolution of ISO/IEC 9126 ISO 9126-1 contains the original quality model and its extensions. The quality sub- characteristics that were specified in an annex in the original standard have been modified slightly and are now part of the official standard. Figure 13 shows the full taxonomy of quality characteristics in the ISO 9126-1 standard. It also illustrates the connection to the other two related standards for external and internal metrics which are defined in ISO 9126- 2 and ISO 9126-3, respectively.
Figure 13. ISO 9126 Quality Model The quality model in ISO 9126-1 is the foundation for the other three parts of the ISO 9126 standard which define metrics and techniques for measuring quality from three different perspectives: internal quality, external quality, and quality in use. Figure 14 shows the relationship between these three perspectives.
Figure 14. Quality Perspectives ISO 9126-2 defines external metrics for measuring sub-characteristics in the base quality model. These are measures of quality from the perspective of the external behavior of the product at its interface. The difference between external quality metrics and quality in use metrics is that external quality metrics measure the external behavior of the product while it is running in a simulated environment. Quality in use metrics measure effects on actual end users while the product is running in its production environment. The difference is analogous to the difference between system testing a product with fabricated data and acceptance testing with real user data in the presence of actual users. ISO 9126-3 defines internal metrics for measuring sub-characteristics in the base quality model. Some quality characteristics can only be measured with internal product metrics. For example, the maintainability of a solution isn't visible from a product's external interface. Internal quality metrics are typically measurements of static structural properties of the implementation or intermediate forms of the implementation. ISO 9126-4 defines quality in use metrics that measure the results of using the system in a specific environment. Quality in use is a measure of quality from the user's perspective while using the product in its environment. Quality in use measures the ease and extent to which users can accomplish their goals. As figure 15 shows, quality in use metrics are based on the smaller set of user-oriented quality characteristics: effectiveness, productivity, safety, and satisfaction.
Figure 15. ISO 9126-4 Quality Model The same basic quality characteristic from the ISO 9126-1 model can be measured in three separate ways depending on the perspective taken. For example, consider the quality characteristic efficiency. Internal efficiency can be measured by analyzing the asymptotic complexity of the algorithms used in the implementation. In terms of efficiency, an O(n) algorithm is more efficient or has higher internal quality than an O(n 2 ) algorithm. External efficiency can be measured by simulation. If a product performs well in a simulated environment it is of high quality. From the quality in use perspective, program efficiency indirectly influences productivity and satisfaction. The product has sufficient quality if performance doesn't negatively impact the productivity or satisfaction of end users. The ISO 9126 standard defines sub-characteristics and associated metrics but allows for application-specific quality characteristics and metrics. It would be difficult to mandate a specific closed set of quality characteristics or metrics because they are heavily dependent on implementation technology and the application domain. The ISO 9126 standard also doesn't define an absolute measure of quality because, again, the relative importance of each characteristic is application-dependent. For example, reliability is more important for mission critical applications; efficiency for time-critical applications; and usability for interactive applications. Recent updates make the ISO 9126 standard one of the most current. At the very least it offers another taxonomy of quality characteristics and metrics to consider. The standard will be of special interest to organizations that are already using ISO process standards. 3 Descripcin del modelo ISO 9126 3.1.1 Funcionalidad: 1. Adecuacin (Suitability): Capacidad del producto software para proporcionar un conjunto apropiado de funciones para tareas y objetivos de usuario especificados. 2. Exactitud (Accuracy): Capacidad del producto software para proporcionar los resultados o efectos correctos o acordados, con el grado necesario de precisin. 3. Interoperabilidad (Interoperability): Capacidad del producto software para interactuar con uno o ms sistemas especificados. 4. Seguridad de acceso (Security): Capacidad del producto software para proteger informacin y datos de manera que las personas o sistemas no autorizados no puedan leerlos o modificarlos, al tiempo que no se deniega el acceso a las personas o sistemas autorizados 5. Cumplimiento funcional (Functionality Compliance): Capacidad del producto software para adherirse a normas, convenciones o regulaciones en leyes y prescripciones similares relacionadas con funcionalidad. 3.1.2 Confiabilidad (Reliability) 1. Madurez (Maturity): Capacidad del producto software para evitar fallar como resultado de fallos en el software. 2. Tolerancia a fallos (Fault Tolerance): Capacidad del software para mantener un nivel especificado de prestaciones en caso de fallos software o de infringir sus interfaces especificados. 3. Capacidad de recuperacin (Recoverability): Capacidad del producto software para reestablecer un nivel de prestaciones especificado y de recuperar los datos directamente afectados en caso de fallo. 4. Cumplimiento de la fiabilidad (Reliability Compliance): Capacidad del producto software para adherirse a normas, convenciones o regulaciones relacionadas con al fiabilidad 3.1.3 Usabilidad (Usability) 1. Capacidad para ser entendido (Understandability): Capacidad del producto software que permite al usuario entender si el software es adecuado y cmo puede ser usado para unas tareas o condiciones de uso particulares. 2. Capacidad para ser aprendido (Learnability): Capacidad del producto software que permite al usuario aprender sobre su aplicacin. 3. Capacidad para ser operado (Operability): Capacidad del producto software que permite al usuario operarlo y controlarlo. 4. Capacidad de atraccin (Attractiveness): Capacidad del producto software para ser atractivo al usuario. 5. Cumplimiento de la usabilidad (Usability Compliance): Capacidad del producto software para adherirse a normas, convenciones, guas de estilo o regulaciones relacionadas con la usabilidad. 3.1.4 Eficiencia: 1. Comportamiento temporal (Time Behavior): Capacidad del producto software para proporcionar tiempos de respuesta, tiempos de proceso y potencia apropiados, bajo condiciones determinadas. 2. Utilizacin de recursos (Resource utilization): Capacidad del producto software para usar las cantidades y tipos de recursos adecuados cuando el software lleva a cabo su funcin bajo condiciones determinadas. 3. Cumplimiento de la eficiencia (Efficiency Compliance): Capacidad del producto software para adherirse a normas o convenciones relacionadas con la eficiencia. 3.1.5 Mantenibilidad: 1. Capacidad para ser analizado (Analyzability): Es la capacidad del producto software para serle diagnosticadas deficiencias o causas de los fallos en el software, o para identificar las partes que han de ser modificadas. 2. Capacidad para ser cambiado (Changeability): Capacidad del producto software que permite que una determinada modificacin sea implementada. 3. Estabilidad (Stability): Capacidad del producto software para evitar efectos inesperados debidos a modificaciones del software. 4. Capacidad para ser probado (Testability): Capacidad del producto software que permite que el software modificado sea validado. 5. Cumplimiento de la mantenibilidad (Maintainability Compliance): Capacidad del producto software para adherirse a normas o convenciones relacionadas con la mantenibilidad 3.1.6 Portabilidad: 1. Adaptabilidad (Adaptability): Capacidad del producto software para ser adaptado a diferentes entornos especificados, sin aplicar acciones o mecanismos distintos de aquellos proporcionados para este propsito por el propio software considerado. 2. Instalabilidad (Installability): Capacidad del producto software para ser instalado en un entorno especificado. 3. Coexistencia(Coexistence): Capacidad del producto software para coexistir con otro software independiente, en un entorno comn, compartiendo recursos comunes. 4. Capacidad para reemplazar (Rreplaceability): Capacidad del producto software para ser usado en lugar de otro producto software, para el mismo propsito, en el mismo entorno. 5. Cumplimiento de la portabilidad (Portability Compliance): Capacidad del producto software para adherirse a normas o convenciones relacionadas con la portabilidad.
4 CMMI- Capability Maturity Model Integration (CMMI) Tying It All Together Now that you have been introduced to the components of CMMI models, you need to understand how they fit together to meet your process improvement needs. This chapter introduces the concept of levels and shows how the process areas are organized and used. CMMI-DEV does not specify that a project or organization must follow a particular process flow or that a certain number of products be developed per day or specific performance targets be achieved. The model does specify that a project or organization should have processes that address development related practices. To determine whether these processes are in place, a project or organization maps its processes to the process areas in this model. The mapping of processes to process areas enables the organization to track its progress against the CMMI-DEV model as it updates or creates processes. Do not expect that every CMMI-DEV process area will map one to one with your organizations or projects processes. Understanding Levels Levels are used in CMMI-DEV to describe an evolutionary path recommended for an organization that wants to improve the processes it uses to develop products or services. Levels can also be the outcome of the rating activity in appraisals. 1 Appraisals can apply to entire organizations or to smaller groups such as a group of projects or a division.
1 For more information about appraisals, refer to Appraisal Requirements for CMMI and the Standard CMMI Appraisal Method for Process Improvement Method Definition Document [SEI 2011a, SEI 2011b]. CMMI supports two improvement paths using levels. One path enables organizations to incrementally improve processes corresponding to an individual process area (or group of process areas) selected by the organization. The other path enables organizations to improve a set of related processes by incrementally addressing successive sets of process areas. These two improvement paths are associated with the two types of levels: capability levels and maturity levels. These levels correspond to two approaches to process improvement called representations. The two representations are called continuous and staged. Using the continuous representation enables you to achieve capability levels. Using the staged representation enables you to achieve maturity levels. To reach a particular level, an organization must satisfy all of the goals of the process area or set of process areas that are targeted for improvement, regardless of whether it is a capability or a maturity level. Both representations provide ways to improve your processes to achieve business objectives, and both provide the same essential content and use the same model components. Structures of the Continuous and Staged Representations Figure 3.1 illustrates the structures of the continuous and staged representations. The differences between the structures are subtle but significant. The staged representation uses maturity levels to characterize the overall state of the organizations processes relative to the model as a whole, whereas the continuous representation uses capability levels to characterize the state of the organizations processes relative to an individual process area. Process Areas Capability Levels Continuous Representation Generic Practices Generic Goals Specific Goals Specific Practices Maturity Levels Staged Representation Generic Practices Generic Goals Specific Goals Specific Practices Process Areas
Figure 3.1: Structure of the Continuous and Staged Representations What may strike you as you compare these two representations is their similarity. Both have many of the same components (e.g., process areas, specific goals, specific practices), and these components have the same hierarchy and configuration. What is not readily apparent from the high-level view in Figure 3.1 is that the continuous representation focuses on process area capability as measured by capability levels and the staged representation focuses on overall maturity as measured by maturity levels. This dimension (the capability/maturity dimension) of CMMI is used for benchmarking and appraisal activities, as well as guiding an organizations improvement efforts. Capability levels apply to an organizations process improvement achievement in individual process areas. These levels are a means for incrementally improving the processes corresponding to a given process area. The four capability levels are numbered 0 through 3. Maturity levels apply to an organizations process improvement achievement across multiple process areas. These levels are a means of improving the processes corresponding to a given set of process areas (i.e., maturity level). The five maturity levels are numbered 1 through 5. Table 3.1 compares the four capability levels to the five maturity levels. Notice that the names of two of the levels are the same in both representations (i.e., Managed and Defined). The differences are that there is no maturity level 0; there are no capability levels 4 and 5; and at level 1, the names used for capability level 1 and maturity level 1 are different. Table 3.1 Comparison of Capability and Maturity Levels Level Continuous Representation Capability Levels Staged Representation Maturity Levels Level 0 Incomplete Level 1 Performed Initial Level 2 Managed Managed Level 3 Defined Defined Level 4 Quantitatively Managed Level 5 Optimizing The continuous representation is concerned with selecting both a particular process area to improve and the desired capability level for that process area. In this context, whether a process is performed or incomplete is important. Therefore, the name Incomplete is given to the continuous representation starting point. The staged representation is concerned with selecting multiple process areas to improve within a maturity level; whether individual processes are performed or incomplete is not the primary focus. Therefore, the name Initial is given to the staged representation starting point. Both capability levels and maturity levels provide a way to improve the processes of an organization and measure how well organizations can and do improve their processes. However, the associated approach to process improvement is different. Understanding Capability Levels To support those who use the continuous representation, all CMMI models reflect capability levels in their design and content. The four capability levels, each a layer in the foundation for ongoing process improvement, are designated by the numbers 0 through 3: 0. Incomplete 1. Performed 2. Managed 3. Defined A capability level for a process area is achieved when all of the generic goals are satisfied up to that level. The fact that capability levels 2 and 3 use the same terms as generic goals 2 and 3 is intentional because each of these generic goals and practices reflects the meaning of the capability levels of the goals and practices. (See the Generic Goals and Generic Practices section in Part Two for more information about generic goals and practices.) A short description of each capability level follows. Capability Level 0: Incomplete An incomplete process is a process that either is not performed or is partially performed. One or more of the specific goals of the process area are not satisfied and no generic goals exist for this level since there is no reason to institutionalize a partially performed process. Capability Level 1: Performed A capability level 1 process is characterized as a performed process. A performed process is a process that accomplishes the needed work to produce work products; the specific goals of the process area are satisfied. Although capability level 1 results in important improvements, those improvements can be lost over time if they are not institutionalized. The application of institutionalization (the CMMI generic practices at capability levels 2 and 3) helps to ensure that improvements are maintained. Capability Level 2: Managed A capability level 2 process is characterized as a managed process. A managed process is a performed process that is planned and executed in accordance with policy; employs skilled people having adequate resources to produce controlled outputs; involves relevant stakeholders; is monitored, controlled, and reviewed; and is evaluated for adherence to its process description. The process discipline reflected by capability level 2 helps to ensure that existing practices are retained during times of stress. Capability Level 3: Defined A capability level 3 process is characterized as a defined process. A defined process is a managed process that is tailored from the organizations set of standard processes according to the organizations tailoring guidelines; has a maintained process description; and contributes process related experiences to the organizational process assets. A critical distinction between capability levels 2 and 3 is the scope of standards, process descriptions, and procedures. At capability level 2, the standards, process descriptions, and procedures can be quite different in each specific instance of the process (e.g., on a particular project). At capability level 3, the standards, process descriptions, and procedures for a project are tailored from the organizations set of standard processes to suit a particular project or organizational unit and therefore are more consistent, except for the differences allowed by the tailoring guidelines. Another critical distinction is that at capability level 3 processes are typically described more rigorously than at capability level 2. A defined process clearly states the purpose, inputs, entry criteria, activities, roles, measures, verification steps, outputs, and exit criteria. At capability level 3, processes are managed more proactively using an understanding of the interrelationships of the process activities and detailed measures of the process and its work products. Understanding Maturity Levels To support those who use the staged representation, all CMMI models reflect maturity levels in their design and content. A maturity level consists of related specific and generic practices for a predefined set of process areas that improve the organizations overall performance. The maturity level of an organization provides a way to characterize its performance. Experience has shown that organizations do their best when they focus their process improvement efforts on a manageable number of process areas at a time and that those areas require increasing sophistication as the organization improves. A maturity level is a defined evolutionary plateau for organizational process improvement. Each maturity level matures an important subset of the organizations processes, preparing it to move to the next maturity level. The maturity levels are measured by the achievement of the specific and generic goals associated with each predefined set of process areas. The five maturity levels, each a layer in the foundation for ongoing process improvement, are designated by the numbers 1 through 5: 1. Initial 2. Managed 3. Defined 4. Quantitatively Managed 5. Optimizing Remember that maturity levels 2 and 3 use the same terms as capability levels 2 and 3. This consistency of terminology was intentional because the concepts of maturity levels and capability levels are complementary. Maturity levels are used to characterize organizational improvement relative to a set of process areas, and capability levels characterize organizational improvement relative to an individual process area. Maturity Level 1: Initial At maturity level 1, processes are usually ad hoc and chaotic. The organization usually does not provide a stable environment to support processes. Success in these organizations depends on the competence and heroics of the people in the organization and not on the use of proven processes. In spite of this chaos, maturity level 1 organizations often produce products and services that work, but they frequently exceed the budget and schedule documented in their plans. Maturity level 1 organizations are characterized by a tendency to overcommit, abandon their processes in a time of crisis, and be unable to repeat their successes. Maturity Level 2: Managed At maturity level 2, the projects have ensured that processes are planned and executed in accordance with policy; the projects employ skilled people who have adequate resources to produce controlled outputs; involve relevant stakeholders; are monitored, controlled, and reviewed; and are evaluated for adherence to their process descriptions. The process discipline reflected by maturity level 2 helps to ensure that existing practices are retained during times of stress. When these practices are in place, projects are performed and managed according to their documented plans. Also at maturity level 2, the status of the work products are visible to management at defined points (e.g., at major milestones, at the completion of major tasks). Commitments are established among relevant stakeholders and are revised as needed. Work products are appropriately controlled. The work products and services satisfy their specified process descriptions, standards, and procedures. Maturity Level 3: Defined At maturity level 3, processes are well characterized and understood, and are described in standards, procedures, tools, and methods. The organizations set of standard processes, which is the basis for maturity level 3, is established and improved over time. These standard processes are used to establish consistency across the organization. Projects establish their defined processes by tailoring the organizations set of standard processes according to tailoring guidelines. (See the definition of organizations set of standard processes in the glossary.) A critical distinction between maturity levels 2 and 3 is the scope of standards, process descriptions, and procedures. At maturity level 2, the standards, process descriptions, and procedures can be quite different in each specific instance of the process (e.g., on a particular project). At maturity level 3, the standards, process descriptions, and procedures for a project are tailored from the organizations set of standard processes to suit a particular project or organizational unit and therefore are more consistent except for the differences allowed by the tailoring guidelines. Another critical distinction is that at maturity level 3, processes are typically described more rigorously than at maturity level 2. A defined process clearly states the purpose, inputs, entry criteria, activities, roles, measures, verification steps, outputs, and exit criteria. At maturity level 3, processes are managed more proactively using an understanding of the interrelationships of process activities and detailed measures of the process, its work products, and its services. At maturity level 3, the organization further improves its processes that are related to the maturity level 2 process areas. Generic practices associated with generic goal 3 that were not addressed at maturity level 2 are applied to achieve maturity level 3. Maturity Level 4: Quantitatively Managed At maturity level 4, the organization and projects establish quantitative objectives for quality and process performance and use them as criteria in managing projects. Quantitative objectives are based on the needs of the customer, end users, organization, and process implementers. Quality and process performance is understood in statistical terms and is managed throughout the life of projects. For selected subprocesses, specific measures of process performance are collected and statistically analyzed. When selecting subprocesses for analyses, it is critical to understand the relationships between different subprocesses and their impact on achieving the objectives for quality and process performance. Such an approach helps to ensure that subprocess monitoring using statistical and other quantitative techniques is applied to where it has the most overall value to the business. Process performance baselines and models can be used to help set quality and process performance objectives that help achieve business objectives. A critical distinction between maturity levels 3 and 4 is the predictability of process performance. At maturity level 4, the performance of projects and selected subprocesses is controlled using statistical and other quantitative techniques, and predictions are based, in part, on a statistical analysis of fine-grained process data. Maturity Level 5: Optimizing At maturity level 5, an organization continually improves its processes based on a quantitative understanding of its business objectives and performance needs. The organization uses a quantitative approach to understand the variation inherent in the process and the causes of process outcomes. Maturity level 5 focuses on continually improving process performance through incremental and innovative process and technological improvements. The organizations quality and process performance objectives are established, continually revised to reflect changing business objectives and organizational performance, and used as criteria in managing process improvement. The effects of deployed process improvements are measured using statistical and other quantitative techniques and compared to quality and process performance objectives. The projects defined processes, the organizations set of standard processes, and supporting technology are targets of measurable improvement activities. A critical distinction between maturity levels 4 and 5 is the focus on managing and improving organizational performance. At maturity level 4, the organization and projects focus on understanding and controlling performance at the subprocess level and using the results to manage projects. At maturity level 5, the organization is concerned with overall organizational performance using data collected from multiple projects. Analysis of the data identifies shortfalls or gaps in performance. These gaps are used to drive organizational process improvement that generates measureable improvement in performance. Advancing Through Maturity Levels Organizations can achieve progressive improvements in their maturity by achieving control first at the project level and continuing to the most advanced levelorganization-wide performance management and continuous process improvementusing both qualitative and quantitative data to make decisions. Since improved organizational maturity is associated with improvement in the range of expected results that can be achieved by an organization, maturity is one way of predicting general outcomes of the organizations next project. For instance, at maturity level 2, the organization has been elevated from ad hoc to disciplined by establishing sound project management. As the organization achieves generic and specific goals for the set of process areas in a maturity level, it increases its organizational maturity and reaps the benefits of process improvement. Because each maturity level forms a necessary foundation for the next level, trying to skip maturity levels is usually counterproductive. At the same time, recognize that process improvement efforts should focus on the needs of the organization in the context of its business environment and that process areas at higher maturity levels can address the current and future needs of an organization or project. For example, organizations seeking to move from maturity level 1 to maturity level 2 are frequently encouraged to establish a process group, which is addressed by the Organizational Process Focus process area at maturity level 3. Although a process group is not a necessary characteristic of a maturity level 2 organization, it can be a useful part of the organizations approach to achieving maturity level 2. This situation is sometimes characterized as establishing a maturity level 1 process group to bootstrap the maturity level 1 organization to maturity level 2. Maturity level 1 process improvement activities may depend primarily on the insight and competence of the process group until an infrastructure to support more disciplined and widespread improvement is in place. Organizations can institute process improvements anytime they choose, even before they are prepared to advance to the maturity level at which the specific practice is recommended. In such situations, however, organizations should understand that the success of these improvements is at risk because the foundation for their successful institutionalization has not been completed. Processes without the proper foundation can fail at the point they are needed mostunder stress. A defined process that is characteristic of a maturity level 3 organization can be placed at great risk if maturity level 2 management practices are deficient. For example, management may commit to a poorly planned schedule or fail to control changes to baselined requirements. Similarly, many organizations prematurely collect the detailed data characteristic of maturity level 4 only to find the data uninterpretable because of inconsistencies in processes and measurement definitions. Another example of using processes associated with higher maturity level process areas is in the building of products. Certainly, we would expect maturity level 1 organizations to perform requirements analysis, design, product integration, and verification. However, these activities are not described until maturity level 3, where they are defined as coherent, well-integrated engineering processes. The maturity level 3 engineering process complements a maturing project management capability put in place so that the engineering improvements are not lost by an ad hoc management process.
Table 3.2 provides a list of CMMI-DEV process areas and their associated categories and maturity levels. Table 3.2 Process Areas, Categories, and Maturity Levels Process Area Category Maturity Level Causal Analysis and Resolution (CAR)
Support
5 Configuration Management (CM)
Support
2 Decision Analysis and Resolution (DAR)
Support 3 Integrated Project Management (IPM)
Project Management 3 Measurement and Analysis (MA)
Support
2 Organizational Process Definition (OPD)
Process Management 3 Organizational Process Focus (OPF)
Process Management 3 Organizational Performance Management (OPM)
Process Management 5 Organizational Process Performance (OPP)
Process Management 4 Organizational Training (OT)
Process Management 3 Product Integration (PI) Engineering
3 Project Monitoring and Control (PMC)
Project Management 2 Project Planning (PP)
Project Management 2 Process and Product Quality Assurance (PPQA) Support 2 Quantitative Project Management (QPM) Project Management 4 Requirements Development (RD) Engineering