Sei sulla pagina 1di 36

Quality Control Strategies Westgard Rules and beyond

Sten Westgard, MS Westgard QC, Inc. Madison, WI www.westgard.com

Objectives of this presentation


Identify different approaches to Quality Control Describe the Westgard Rules and their use Evaluate the strengths and weaknesses of different QC strategies List the different options CLIA gives you for QC Assess the suitability of EQC and Risk Management for your laboratory QC

Outline
5 Possible Quality Control Strategies for non-waived test systems
1. 2. 3. 4. 5. Real World Standard Deviations from Good QC Westgard Rules: a multirule approach Ideal QC Design: Optimizing QC for tests CLIA Compliance QC Future of QC: Risk Management

Strategy #1. QC in the Real World: Standard Deviations from GLP QC


The Anorexic The Gambler The Blind Man

Other Common Deviations


Misuse of control range from manufacturers package insert
Assigned values usually represent performance from several or many different instruments and laboratories Ranges will be too wide for a single method in a single laboratory Causes a loss in error detection

Other Common Deviations (2)


Miscalculation of control limits using out-ofcontrol data points
CAP recommends this practice to avoid narrowing of control limits But that is due to common use of 2SD limits When QC properly designed to avoid 2SD rule, then any control results from out-of-control runs should NOT be included

Why dont single rules suffice?


Analogy to fire alarm
False alarms (false rejection) True alarms (error detection)

False rejection problems with LJ charts that use 2 SD control limits


N=1, 5% N=2, 9% N=3, 14% N=4, 18%

Error detection problem with 3 SD limits Reason for use of multirule QC

Strategy #2: Westgard Rules What are they and why us?
Maximize error detection from few measurements Attempt to balance work with practicality Classic laboratory workaround Concept described 1977 Example posted 1981
official name: multirule Westgard Rules

Download from westgard.com

Which rules tell about which kinds of errors?


Random error
Rules that look for points that are in the tails or the distribution or measure the width of the distribution 13s, R4s

Systematic error
Rules that count consecutive measurements beyond a certain limit 22s, 41s, 10x

Is a 2s warning rule still needed?


Recommended for manual operations
Minimizes the time and effort to inspect and review control data

Not necessary for computerized applications


But often used anyway!

Important to carefully define rules to be used and conditions for use across runs, across materials, etc.

8x
8

8
4

Download from westgard.com

Download from westgard.com

Summary of Westgard Rules


A lot of detection for a little measurement Wont work if you set the wrong limits, feed in wrong data, or ignore flags
Not one size fits all
Some tests may not even need them There are different ways to construct, implement, and interpret the rules

Strategy #3. QC by Optimal Design


Select the QC rules needed to assure quality

General guidance provided by ISO 15189


verify the attainment of the intended quality of test results

Specific guidance by CLSI C24-A3


QC Planning Process / QC Design

Practical tools (free)


Sigma-metrics QC selection tool Charts of Operating Specifications

QC Design by Sigma Metrics: Six Sigma Approach


Universal Benchmark Scale of 0 to 6 6 is world class


(3.4 DPM, or defects per million)

3 is minimum for any business or manufacturing process (66,807 DPM)

Assessment Tool For analytical performance: Sigma-metric Equation

Sigma-metric = (TEa Bias)/CV


- TEa Bias True Value + TEa

CV

defects

-6s -5s -4s -3s -2s -1s 0s 1s 2s 3s 4s 5s 6s

Where can you find quality requirements?

Analytical benchmarks

PT and EQA standards CLIA PT criteria RiliBK (Germany) RCPA (Royal College of Pathologists of Australasia)

Clinical Benchmarks (better)

Biologic Variation database (Ricos et al.)


ISO 15189, ISO 15197

Simulation studies with patient data (Klee and Karon, 2010) Clinical Decision Intervals:
Evidence-Based Medicine & Clinical Guidelines How do your Doctors use the Test? (BEST)

Example Sigma-metric Calculation


Cholesterol
CLIA PT requirement for acceptability = 10% NCEP CV recommendation: 3% NCEP Bias recommendation: 3%

Sigma = (10 3) / 3 =7/3 = 2.33 Normalized Calculations: 3/10 = 30%


19

QC Design tool for analytical processes: Operating Specifications (OPSpecs) chart:

Free download at http://www.westgard.com/normcharts.html

OPSpecs display of 2008 POC Device:


8 methods, 5 cannot be controlled by QC

Strategy #4: QC compliance options from CLIA Whats possible under CLIA?
1. Do the right Statistical QC
Design the right rules and # of controls

2. Default QC
Just run 2 controls per day

3. Equivalent QC (EQC)
Maybe only 2 controls per week or month

4. Maybe Alternative QC

CLIA Final Rule


493.1256 Standard: Control procedures.
(a) Laboratory is responsible for having control procedures that monitor the accuracy and precision of the complete

analytical process.

(b) must establish the number, type, and frequency of testing control materials

(c 1) The control procedures must detect immediate errors that occur due to test system failure, adverse environmental conditions, and operator performance;
(c 2) Monitor over time the accuracy and precision of test performance

The origin of default/minimum/minimal QC


Final CLIA Rule 1992
Incomplete, delayed implementation of QC clearance which would allow laboratories to just follow manufacturers QC instructions Meanwhile, laboratories allowed to get by with running a minimum of 2 levels of controls per 24 hour period

Final, Final, Final, Final, Final CLIA Rule 2003


QC clearance eliminated, no longer needed???? New concept of EQC introduced by CMS in State Operations Manual (SOM)

Is min. 2/day controls evidence-based? Not really

EQC /Equivalent QC Options


Option 1. Device monitors complete analytical process Evaluation: 2 levels/day for 10 days then Run QC once/month Option 2. Device monitors some of analytical process. Evaluation: 2 levels/day for 30 days then Run QC once/week

Option 3. Device monitors none of analytical process. Evaluation: 2 levels/day for 60 days then Run QC once/week ( Is this based on Science? No. ) CMS admits in 2005: We blew it.

Revisiting Equivalent Quality Control. Clinical Lab News, June 2005

EQC Options: Its all on you


D5445, Interpretative Guidelines of SOM:
NOTE: Since the purpose of control testing is to detect immediate errors and monitor performance over time, increasing the interval between control testing (i.e. weekly, or monthly) will require a more extensive evaluation of patient test results when control failure occurs (see 493.1282). The director must consider the laboratory's clinical and legal responsibility for providing accurate and reliable patient test results versus the cost implications of reducing the quality control testing frequency.

Strategy #5: QC of the future


If EQC Options 1-3 arent an option, what then?
ADVAMED proposes EQC Option 4 2004
CLSI unable to provide scientific basis

CMS sponsors development of Alternative QC (AQC) through CLSI: EP18, EP22, EP23 CMS Roles Out New CLIA Policy Changes CLN 2008
Legalizes Equivalent Quality Control (EQC)

CLSI EP22 & EP23 published Jan 2010


Risk management approach adopted

CLSIs effort to support EQC


EP18: Risk Management Techniques to Identify and Control Laboratory Error Sources (already a guideline)
EP22: Presentation of Manufacturers Risk Information EP23: Laboratory QC Based on Risk Management (due mid-2011)

Choose the process to study

Risk Management Process: Practical Steps


Risk Analysis and Assessment FMEA: Failure Modes Effects Analysis Hazard Analysis Risk Estimation
Estimate Occurrence Estimate Severity Estimate Detectability

Assemble Team
multidisciplinary

Risk Evaluation
Acceptability Matrix Criticality Prioritization

Assemble Info
Process map

Identify hazards Failure-modes Assess harm

Risk Monitoring
Identify failures Estimate frequency Assess risk

Risk Control
Identify residual risks Develop control plan Implement controls

Risk Reduction
Analyze root cause Prevent Error Mitigate Error

Risk Management: Something you already do every day

Risk Acceptability Matrix: EP23


Risk Matrix Frequent. Probable ????? Unacceptable Risk Negligible Minor Serious Critical Catastrophic

Occasional
Remote Improbable Acceptable Risk ?????

CLSI EP23 Risk Acceptability Matrix. Note the ????? Identifies cells where ISO 14971 classifies risk as unacceptable, but EP23 classifies risk as acceptable. Note also that Detectability has been eliminated in this matrix. Using Sigma-metrics for more quantitative matrices is allowed.
31

How much effort does it take to conduct Risk Management for one process?
2009 review of 13 HFMEA projects in Dutch healthcare settings: (Lab HFMEA) Avg # team members = 7.2 (8) Avg # meetings = 6.3 (6) Avg # person-hours = 69.1 (72) Avg # failure modes = 51.8 (27) Avg # remedial actions = 16.2 (11)

Habraken, M. M. P., Van der Schaaf, T. W., Leistikow, I. P. and Reijnders-Thijssen, P. M. J.(2009)'Prospective risk analysis of health care processes: A systematic evaluation of the use of HFMEA in Dutch health care', Ergonomics,52:7,809 819

Risk Analysis and FMEA example


Studies On The Improvement Of Critical Laboratory Value Notification Using A Failure Mode And Effect Analysis S. Yenice, C. Maden, T. Esin. Gayrettepe Florence Nightingale Hospital, Istanbul, Turkey Poster A-90, 2010 AACC conference
6 processes, 31 sub-processes 66 failure modes, 97 failure causes and effects FMEA is a potent and invaluable tool to trap the potential failures. Yet, process is complex, time-consuming, and requires an intensive labor input.a good team effort and detailed planning should be reserved.

Conclusion: When it comes to QC, you have options


5 Possibilities
1. 2. 3. 4. 5. Anorexia, Gambling, Blindness Westgard Rules Optimized, Customized QC by Design Tools Compliance options from CLIA Possible Future? Risk Management

Not all options are created equal

Self-Assessment Questions
1. Which of the following is a CLIA-acceptable QC strategy?
a) b) c) d) Anorexic QC Gambler QC Blind Man QC EQC options 1-3

3.

Westgard Rules combine to detect what kinds of errors?


a) b) c) d) Random & systematic Single-rule and multirule CLIA and ISO Waived and non-waived

2.

Which CLSI Risk Management standard was proposed but then disbanded?
a) b) c) EP18 EP22 EP23

4.

What design tool would you use to select appropriate QC?


a) b) c) d) Risk Acceptability Matrix EQC Options 1-3 Westgard Rules OPSpecs chart

Questions? westgard@westgard.com

36

Potrebbero piacerti anche