Tmmi Impl Guideline 110608061949 Phpapp01

Path to reach Level 3 test process maturity in TMMi framework
M C Nagaraj nagaraj@stagsoftware.com
in.linkedin.com/in/mcnagaraj
Agenda
TMMi Level 1 to 5
Requirement interpretation Linking current process to TMMi Specific practices
All TMMi related information are taken from TMMi framework document V 3.1 created and Copyrights owned by TMMi foundation. www.tmmifoundation.org
Copyright STAG Software Pvt. Ltd., 2011
TMMi Framework
Level 5 : Optimization
Defect Prevention Total Process Optimization Quality Control
Level 4 : Measured
Test Measurement Software Quality Evaluation Advanced Peer Reviews
Level 3 : Defined
Test Organization, Test Training Program, Test Lifecycle and Integration, Non-functional Testing, Peer Reviews
Level 2 : Managed
Test Policy and Strategy, Test Planning, Test Monitoring and Control, Test Design and Execution, Test Environment
Level 1 : Initial
Five levels of Testing Process Maturity
TMMi Framework
Level 4 : Measured
Level 3 : Defined
Level 2 : Managed
Level 1 : Initial
Level-1
Testing is a chaotic process Not distinguished from debugging Documented set of specification not available Tests are developed in ad-hoc way after coding is completed The objective of testing is to show that software works
TMMi Level 2
Level 4 : Measured
Level 3 : Defined
Level 2 : Managed
Level 1 : Initial
Process areas at TMMi Level 2

2.1 Test Policy and strategy 2.2 Test Planning 2.3 Test Monitoring and Control 2.4 Test Design and Execution 2.5 Test Environment
2.1 Test Policy and Strategy

Specific Goals SG 1 - Establish a Test Policy Specific Practices SP 1.1 Define test goals SP 1.2 Define test policy SP 1.3 Distribute the test policy to stakeholders Guidelines to implement -Look at organization business goal and role of testing in achieving the same - Policy should address test cycle activities are planned, done by trained engineers, process improve with management support/focus -Use organization standard way of identifying risks in a project -Define quality levels and objectives -Define how you measure success of test process -Set some target for key metrics and measure what you achieved
SG 2 - Establish a Test Strategy
SP 2.1 Perform generic product risk assessment SP 2.2 Define test strategy SP 2.3 Distribute the test strategy to stakeholders SP 3.1 Define test performance indicators SP 3.2 Deploy test performance indicators
SG 3 - Establish Test Performance Indicators

2.2 Test Planning

Specific Goals
SG 1- Perform Product Risk Assessment
Specific Practices
SP 1.1 Define product risk categories SP 1.2 Identify product risks SP 1.3 Analyze product risks SP 2.1 Identify items and features to be tested SP 2.2 Define the test approach SP 2.3 Define entry criteria SP 2.4 Define exit criteria SP 2.5 Define suspension and resumption criteria SP 3.1 Establish a top-level work breakdown structure SP 3.2 Define test life cycle SP 3.3 Determine estimates for test effort and cost
Guidelines to implement
-Look at from end users perspective value & prioritize requirements -Can you predict what can upset end users? - Extract list of features -Extract list of user types and business use cases they use -Extract potential defect types -Quality levels and types of defects to be uncovered at each levels -Entry, exit, suspension and resumption criteria for all levels -Have organization standard test life cycle activity list -Customize this to project specific -Estimate effort, review update and baseline -Track number of cycles of reestimation done
SG 2 - Establish a Test Approach
SG 3 - Establish Test Estimates
2.2 Test Planning

Specific Goals
SG 4 Develop a Test Plan
Specific Practices
SP 4.1 Establish the test schedule SP 4.2 Plan for test staffing SP 4.3 Plan for stakeholders involvement SP 4.4 Identify test project risks SP 4.5 Establish the test plan SP 5.1 Review test plan SP 5.2 Reconcile work and resource levels SP 5.3 Obtain test plan commitments
-Milestone definition -Detail schedule -Risk assessment and prioritization -Plan document with important sections in place
SG 5 Obtain Commitment to the Test Plan
-Review plan with all concerned -Update feedback and ensure confidence is high to meet the plan -Team commitments for all dependency will help

2.3 Test Monitoring and Control

Specific Goals
SG 1 Monitor Test Progress against Plan
Specific Practices
SP 1.1 Monitor test planning parameters SP 1.2 Monitor test environment resources provided and used SP 1.3 Monitor test commitments SP 1.4 Monitor test project risks SP 1.5 Monitor stakeholder involvement SP 1.6 Conduct test progress reviews SP 1.7 Conduct test progress milestone reviews SP 2.1 Check against entry criteria SP 2.2 Monitor defects SP 2.3 Monitor product risks SP 2.4 Monitor exit criteria SP 2.5 Monitor suspension and resumption criteria SP 2.6 Conduct product quality reviews SP 2.7 Conduct product quality milestone reviews
-Monitor effort/cost, schedule variance against plan -Ensure test environment is same as stated in expectation in document -Milestone review, weekly review etc. -Stakeholders update on key milestone and seek feedback -Risk tracking -Quality levels exit criteria defined and actual - Monitor suspension and resumptions cycles - Test summary report at key quality levels achievement
SG 2 Monitor Product Quality against Plan and Expectations
2.3 Test Monitoring and Control

Specific Goals
SG 3 Manage Corrective Action to Closure
Specific Practices
SP 3.1 Analyze issues SP 3.2 Take corrective action SP 3.3 Manage corrective action
Guideline to implement
-Track project issues formally -Track action items and ensure closure

2.4 Test Design and Execution

Specific Goals SG 1 Perform Test Analysis and Design using Test Design Techniques Specific Practices SP 1.1 Identify and prioritize test conditions SP 1.2 Identify and prioritize test cases SP 1.3 Identify necessary specific test data SP 1.4 Maintain horizontal traceability with requirements SP 2.1 Develop and prioritize test procedures SP 2.2 Create specific test data SP 2.3 Specify intake test procedure SP 2.4 Develop test execution schedule Guidelines to implement -Have a clear test case architecture to be followed across organization -Organize test cases at quality level -Maintain requirement traceability to test scenarios/test cases, fault type, types of test applicable -Test cycle schedule -Build health check criteria -Create re-usable test data - Quality level entry/exit criteria
SG 2 Perform Test Implementation
2.4 Test Design and Execution

Specific Goals SG 3 Perform Test Execution Specific Practices SP 3.1 SP 3.2 SP 3.3 SP 3.4 Perform intake test Execute test cases Report test incidents Write test log Guidelines to implement -Verify entry criteria before starting any levels of testing - Execute test as per plan and record outcome -Capture defects to track and close -Be part of change control board to resolve all issues reported during testing - Ensure defects fixed are verified and closed in each release - Defect tracking report in various forms so that helps to take right decision
SG 4 Manage test incident to closure
SP 4.1 Decide deposition of test incidents in configuration control board SP 4.2 Perform appropriate action to close the test incident SP 4.3 Track the status of test incidents

2.5 Test Environment

Specific Goals
SG 1 Develop Test Environment Requirements
Specific Practices
SP 1.1 Elicit test environments needs SP 1.2 Develop the test environment requirements SP 1.3 Analyze the test environment requirements SP 2.1 Implement the test environment SP 2.2 Create generic test data SP 2.3 Specify test environment intake test procedure SP 2.4 Perform test environment intake test
-Capture required environment clearly at one place - Ensure environment available and required are identified -Set up test environment as needed - Create generic test data and archive for re-use - Verify whether needs of environment required for testing is met formally
SG 2 Perform Test Environment Implementation
2.5 Test Environment

Specific Goals Specific Practices Guidelines to implement -Plan and track system requirements -Plan development of test data and ensure it is version controlled - Ensure required coordination process in place when environment are shared - If environment issues found capture and track till it is closed SG 3 Manage and Control SP 3.1 Perform systems Test Environments management SP 3.2 Perform test data management SP 3.3 Co-ordinate the availability and usage of the test environments SP 3.4 Report and manage test environment incidents
What you can see in L2 organization?

Organization test goals and policy published There is a clear separation between debugging and testing phase It is a planned activity in project plan Plan starts after coding is complete (only execution based testing visible) Basic testing techniques in place Testing is multi-leveled Test environment managed as required
TMMi Framework
Level 4 : Measured
Level 3 : Defined
Level 2 : Managed
Level 1 : Initial

3.1 Test Organization 3.2 Test Training Program 3.3 Test Lifecycle and Integration 3.4 Non-functional testing 3.5 Peer Reviews
3.1 Test Organization

Specific Goals
SG 1 Establish a Test Organization
Specific Practices
SP 1.1 Define the test organization SP 1.2 Obtain commitment for the test organization SP 1.3 Implement the test organization SP 2.1 Identify test functions SP 2.2 Develop job descriptions SP 2.3 Assign staff members to test functions SP 3.1 Establish test career path SP 3.2 Develop personal test career development plans
-Clear definition of test team organization - Management commitment to support the test organization - Create team to fill all roles -Define key test lifecycle activities -Map role and responsibilities to key test life cycle activities -Ensure there is clear definition of career path for test engineers - Ensure individual career plans prepared and tracked by managers
SG 2 Establish Test Functions for Test Specialists
SG 3 Establish Test Career Paths
3.1 Test Organization

Specific Goals SG 4 Determine, Plan and Implement Test Process Improvement Specific Practices SP 4.1 Assess the organizations test process SP 4.2 Identify the organization test process improvements SP 4.3 Plan test process improvements SP 4.4 Implement test process improvements SP 5.1 Deploy standard test process and test process assets SP 5.2 Monitor implementation SP 5.3 Incorporate lessons learnt into the organizational test process Guidelines to implement -Assess organization test process formally - Identify improvement areas - Plan test process improvements by having a team - Ensure team work as per plan and achieve improvements planned -Define standard process - Capture test process assets - Monitor adherence to process by planned audits - Analyze completed projects and recommend any improvement to process needed
SG 5 Deploy the Organization Test Process and Incorporate Lessons Learned
3.2 Test Training Program

Specific Goals SG 1 Establish an Organizational Test Training Capability Specific Practices SP 1.1 Establish the strategic test training needs SP 1.2 Align the organizational and project test training needs SP 1.3 Establish an organizational test training plan SP 1.4 Establish test training capability Guidelines to implement -Define role based competency and standard training needs - Identify project specific competency required and plan training - Have clear training plan for team and project specific training on need basis -Ensure trainers internal/external identified and ready for all topics -Conduct training as per plan - Maintain test training records - Verify the effectiveness of training conducted on job
SG 2 Provide Necessary Test Training
SP 2.1 Deliver test training SP 2.2 Establish test training records SP 2.3 Assess test training effectiveness
3.3 Test Lifecycle and Integration

Specific Goals
SG 1 Establish Organizational Test Process and Assets
Specific Practices
SP 1.1 Establish standard test process SP 1.2 Establish test lifecycle model descriptions addressing all test levels SP 1.3 Establish tailoring criteria and guidelines SP 1.4 Establish the organizations test process database SP 1.5 Establish the organization's test process asset library SP 1.6 Establish work environment standards SP 2.1 Establish integrated test lifecycle models SP 2.2 Review integrated lifecycle models SP 2.3 Obtain commitments on the role of testing within the integrated lifecycle models
-Have organization standard test process approved - Define standard test lifecycle activities -Establish tailoring guidelines and criteria -Capture test process followed at project level as data - Capture process assets including tailoring done - Ensure work environment is defined and adhered to as per plan -Ensure multiple test cycles are planned with focus on all levels of testing - Review with all concerned on depends across each team and agree on entry/exit criteria
SG 2 Integrate Test Lifecycle Models with Development Models
3.3 Test Lifecycle and Integration

Specific Goals SG 3 Establish a Master Test Plan Specific Practices SP 3.1 Perform product risk assessment SP 3.2 Establish the test approach SP 3.3 Establish test estimates SP 3.4 Define the organization for testing SP 3.5 Develop the master test plan SP 3.6 Obtain commitment to the master test plan Guidelines to implement -Prioritize customer expectation - Define test strategy - Estimate effort required - Define team required -Have master test plan with milestone - Have detail plan for all milestone - Have commitment from all concerned in team on schedule
3.4 Non-functional Testing

Specific Goals SG 1 Perform a NonFunctional Product Risk Assessment SG 2 Establish a NonFunctional Test Approach Specific Practices SP 1.1 Identify non-functional product risks SP 1.2 Analyze non-functional product risks SP 2.1 Identify features to be tested SP 2.2 Define the nonfunctional test approach SP 2.3 Define non-functional exit criteria Guidelines to implement -Identify non-functional product requirements - Analyze what can go wrong here -Identify scope for NF testing - Define test approach for NF testing - Define exit criteria for all planned NF tests
3.4 Non-functional Testing

Specific Goals
SG 3 Perform Non-Functional Test Analysis and Design
Specific Practices
SP 3.1 Identify and prioritize nonfunctional test conditions SP 3.2 Identify and prioritize nonfunctional test cases SP 3.3 Identify necessary specific test data SP 3.4 Maintain horizontal traceability with non-functional requirements SP 4.1 Develop and prioritize nonfunctional test requirements SP 4.2 Create specific test data SP 5.1 Execute non-functional test cases SP 5.2 Report non-functional test incidents SP 5.3 Write test log
-Identify use cases where NF requirements may not be fully met - Have scenarios and tests cases to replicate the identified use case - Ensure NF requirement to test assets traceability is established
SG 4 Perform Non-Functional Test Implementation SG 5 Perform Non-Functional Test Execution
-Scope and plan for NF testing -Create required scripts to produce the targeted use case scenario -Execute planned NF tests -Report NF test issues and track till closure -Capture test outcome
3.5 Peer Reviews

Specific Goals SG 1 Establish a Peer Review Approach Specific Practices SP 1.1 Identify work products to be reviewed SP 1.2 Define Peer Review Criteria SP 2.1 Conduct peer reviews SP 2.2 Testers review test basis documents SP 2.3 Analyze peer review data Guidelines to implement -Agree upon work products which will go through peer review - Expectation defined for peer review -Peer review happens as per plan - Test engineers are part work product reviews which impacts their deliverables -Analyze peer review outcome and improve organization checklist
SG 2 Perform Peer Reviews
What you can see in L3 organizations?

There is an established test organization Testing is integrated to SDLC Test plan is developed, tracked and controlled (Integrated with project plan) Key work products go through peer reviews Organization review checklist for all documents evolve Test engineers drive test process improvement Users/clients attend milestone meeting User/clients support in developing usability test plans
Thank You
Email ID: nagaraj@stagsoftware.com

Tmmi Impl Guideline 110608061949 Phpapp01

Caricato da

Informazioni sul documento

Descrizione originale:

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Tmmi Impl Guideline 110608061949 Phpapp01

Caricato da

Copyright:

Formati disponibili

Path to reach Level 3 test process maturity in TMMi framework

Copyright STAG Software Pvt. Ltd., 2011

Five levels of Testing Process Maturity

Five levels of Testing Process Maturity

Copyright STAG Software Pvt. Ltd., 2011

Five levels of Testing Process Maturity

Process areas at TMMi Level 2

Copyright STAG Software Pvt. Ltd., 2011

2.1 Test Policy and Strategy

SG 2 - Establish a Test Strategy

SG 3 - Establish Test Performance Indicators

Copyright STAG Software Pvt. Ltd., 2011

Process areas at TMMi Level 2

Copyright STAG Software Pvt. Ltd., 2011

2.2 Test Planning

SG 2 - Establish a Test Approach

SG 3 - Establish Test Estimates

Copyright STAG Software Pvt. Ltd., 2011

2.2 Test Planning

SG 5 Obtain Commitment to the Test Plan

Copyright STAG Software Pvt. Ltd., 2011

Process areas at TMMi Level 2

Copyright STAG Software Pvt. Ltd., 2011

2.3 Test Monitoring and Control

SG 2 Monitor Product Quality against Plan and Expectations

Copyright STAG Software Pvt. Ltd., 2011

2.3 Test Monitoring and Control

Copyright STAG Software Pvt. Ltd., 2011

Process areas at TMMi Level 2

Copyright STAG Software Pvt. Ltd., 2011

2.4 Test Design and Execution

SG 2 Perform Test Implementation

Copyright STAG Software Pvt. Ltd., 2011

2.4 Test Design and Execution

SG 4 Manage test incident to closure

Copyright STAG Software Pvt. Ltd., 2011

Process areas at TMMi Level 2

Copyright STAG Software Pvt. Ltd., 2011

2.5 Test Environment

SG 2 Perform Test Environment Implementation

Copyright STAG Software Pvt. Ltd., 2011

2.5 Test Environment

Copyright STAG Software Pvt. Ltd., 2011

What you can see in L2 organization?

Five levels of Testing Process Maturity

Process areas at TMMi Level 3

Copyright STAG Software Pvt. Ltd., 2011

3.1 Test Organization

SG 2 Establish Test Functions for Test Specialists

SG 3 Establish Test Career Paths

Copyright STAG Software Pvt. Ltd., 2011

3.1 Test Organization

SG 5 Deploy the Organization Test Process and Incorporate Lessons Learned

Copyright STAG Software Pvt. Ltd., 2011

3.2 Test Training Program

SG 2 Provide Necessary Test Training

Copyright STAG Software Pvt. Ltd., 2011

3.3 Test Lifecycle and Integration

SG 2 Integrate Test Lifecycle Models with Development Models

Copyright STAG Software Pvt. Ltd., 2011

3.3 Test Lifecycle and Integration

Copyright STAG Software Pvt. Ltd., 2011

3.4 Non-functional Testing