Risk-Based QC

for Clinical Laboratories

Learning Objectives
• Describe the concept of risk management
in laboratory medicine
• Summarize the basic process for
developing a risk-based QC plan
• List resources that can be utilized as an
aid to the process
• State the five components evaluated in a
risk assessment
What is risk?
What is Risk?
• Combination of the probability of occurrence of
harm and the severity of that harm
(ISO 15190; ISO/IEC Guide 51)

• Harm - physical injury or damage to the health of

people, or damage to property or the
environment [ISO/IEC Guide 51:1999, definition 3.3]

• Severity - measure of the possible

consequences of a hazard
[ISO 14971- 2007 definition 2.25]

• Hazard - potential source of harm

[ISO/IEC Guide 51:1999, definition 3.5]
Why now?
Risk assessment for laboratories?

• Consolidation and decreasing budgets

• One size doesn’t fit all

• What is appropriate QC frequency?

Other Factors Favoring
Risk Approach

• Protect the patient’s well being

• Guard the laboratory’s reputation

• ISO 15189 requires risk to be addressed

ISO 15189: 2012

4.14.6 Risk Management

“The laboratory shall evaluate the impact of work

processes and potential failures on examination results as
they affect patient safety, and shall modify processes to
reduce or eliminate the identified risks and document
decisions and actions taken.”

7
ISO 15189: 2012

5.6.2.2 Quality Control Materials

“Quality control materials shall be periodically examined

with a frequency that is based on the stability of the
procedure and the risk of harm to the patient from an
erroneous result.”

8
Creating a Risk-Based Plan

ENLIGHTENMENT

PLANNING

VERIFY
EFFECTIVENESS

RISK ANALYSIS

IMPLEMENT THE
DEVELOP THE QC PLAN
PLAN
Enlightenment
• ISO 14971: 2012
Medical devices – Application of risk management
to medical devices
• ISO/TS 22367: 2009
Medical laboratories – Reduction of error through
risk management and continual improvement
• CLSI EP23-A
Laboratory QC Based on Risk Management
• Westgard: Six Sigma Risk Analysis
• Testing Documentation…
Gathering information
• Internal audits
• Laboratory complaint logs/ user surveys
• Instructions for use (IFU)
• Historical laboratory QC, calibration and PT records
• Instrument manual
• Manufacturer blogs or webpages for users
• FDA Medical Device report database search
– https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfMDR/Search.cfm
• PT summary reports
• Inter-laboratory reports provided by control
manufacturers
Specific Information
from Manufacturers
• Reliability scores
• Mean time between failure (MTBF)
• Black box processes when necessary
• Information about on-board (embedded) controls
• Information about electronic controls
Planning

13
Before you start
Put together a small quality team
– 5-7 members
– Stakeholders
– Desirable characteristics

14
Team Members:
Key Characteristics
• Inquisitive
• Team player
• Knowledgeable
• Open minded
• Creative thinker
• Good communicator
• Pragmatic
• Able to finish a task
Team Dynamics

• Have a facilitator to keep team on track

• Have objectives for each team meeting
• Brainstorm – open, non-judgmental
discussion
• All members have equal standing
Risk Analysis

17
Some Sources of Risk
• Communication processes
• Environmental
• Knowledge of test operators
• Management commitment
• Outside influences
• Resources
• Technical components
• Process/Procedure
• Financial – decisions based on cost not quality
18
Process Map: Example

19
5 Components Evaluated
– Specimen

– Environment

– Reagent

– Test system

– Personnel

20
 Test System
Pre-Analytical Analytical Post-Analytical
Calibration Consumables (quality) Results: review / approve
Calibration Verification Reagent dispense Result Transmission
Maintenance Sample dispense Retrospective Review
daily, w / m / semi-annual
Reaction Chamber Trend Analysis
Electrical Temperature
Monitoring, Surge Protection Sigma Metrics
Measurement
Dedicated Circuit Frequency of recalibration
Filter wheel
Water Supply (if required) Light source integrity Freq. of Device Failures
Water quality Clot detection
Water integrity (air) Interfering substances Verification of Test
Results
Humidity (mfr Quality Determination
requirement) QC approach used
Temp. (mfr requirement) QC materials used
QC frequency
PT Performance QC Rules
Calibration of ancillary Patient risk
equip (# patients between QCs)
 Information Gathering / Analysis
Test System for POCT
Pre-Analytical Analytical Post-Analytical
Calibration Consumables (quality) Results: review / approve
Calibration Verification Reagent dispense Result Transmission
Maintenance Sample dispense Test Report
daily, w / m / semi-annual
Reaction Chamber Retrospective Review
Electrical Temperature, Black Box
Monitoring, Surge Prot Trend Analysis
Measurement
Dedicated Circuit, Battery Sigma Metrics
Filter wheel
Water Supply (if required) Light source integrity Frequency of recalibration
Water quality Clot detection
Water integrity (air) Interfering substances Freq. of Device Failures
Humidity (mfr Quality Determination Verification of Test
requirement) QC approach used Results
Temp. (mfr requirement) QC materials used
QC frequency
PT Performance QC Rules
Calibration of ancillary Patient risk
equip (# patients between QCs)
 Analysis
Brainstorming a POCT Test
Possible Hazard Points
Analytical
Reaction Chamber
PROCESS
Black Box
MAPPING
Measurement OR
Interfering Substances
FISHBONE
Quality Determination DIAGRAM What Can Go
QC approach used Wrong?
QC materials used
QC frequency
Patient risk
(# patients between QCs)
 Analysis
Brainstorming a POCT Test
Possible Hazard Points
Analytical
• Does the the device manual or the
Reaction Chamber product insert describe in detail the
PROCESS
Black Box analytical sequence?
MAPPING
Measurement • Does the product insert, the device
OR
Interfering Substances manual or the manufacturer describe in
FISHBONE
Quality Determination sufficient detail how the function checks
DIAGRAM
QC approach used work and are themselves
QC materials used • Has the manufacturer provided a product
QC frequency reliability score or the mean time
Patient risk between failure?
(# patients between QCs)
 Analysis
Brainstorming a POCT Test
Analytical Possible Hazard Points
Reaction Chamber
PROCESS
Black Box • Does the product insert….
MAPPING
Measurement OR
Interfering Substances • How is the patient cleared for interfering
FISHBONE substances?
Quality Determination DIAGRAM
QC approach used
QC materials used
QC frequency
Patient risk
(# patients between QCs)
 Analysis
Brainstorming a POCT Test
Possible Hazard Points
Analytical
• Does the product insert….
Reaction Chamber
PROCESS • How is the patient cleared for …
Black Box
MAPPING • What QC modality is used?
Measurement OR Function checks only?
Interfering Substances
FISHBONE Electronic QC only? Traditional QC?
Quality Determination DIAGRAM • First party, second party or third party
QC approach used controls used?
QC materials used
• Embedded control used?
QC frequency
• Liquid or solid phase QC? Shortcomings?
Patient risk
• How frequently is QC frun?
(# patients between QCs)
• Can errors/mistakes/failures/hazards be
detected immediately?
• How are QC limits established?
• What QC rules are used?
• How many patient test results are
reported between QC testing events?
 Analysis
Brainstorming a POCT Test
Possible Hazard Points
Analytical
• Does the product insert….
Reaction Chamber
PROCESS • How is the patient cleared for …
Black Box
MAPPING • What QC modality is used?
Measurement OR Function checks only?
Interfering Substances
FISHBONE Electronic QC only? Traditional QC?
Quality Determination DIAGRAM • First party, second party or third party
QC approach used controls used?
QC materials used • Embedded control used?
QC frequency • Liquid or solid phase QC? Shortcomings?
Patient risk
• How frequently is QC frun?
(# patients between QCs)
• Can errors/mistakes/failures/hazards be
detected immediately?
• How are QC limits established?
• What QC rules are used?
• How many patient test results are
reported between QC testing events?
Risk Exposure:
# Patients Between QC Events
• Consider cost and feasibility of retesting versus
cost of increased control testing
• How many patient samples are affected by an error or
malfunction and is undetected until the next QC event?

Assume a lab testing 500 samples/day.

• 1 / month: exposure = 15,000 patients
• 1 / week: exposure = 3500 patients
• 1 / day: exposure = 500 patients
• 1 / 8-hour shift: exposure = 63 patients
• 2 / 8-hour shift: exposure = 31 patients
• 1 / patient sample: exposure = 1patient

– Bracketed QC
• 1 / 20 patient samples exposure = 20
Grading and Ranking Risk
• Can use FMEA approach
– Grade (score 1-5 or 1-10) for occurrence,
severity, detection
– Multiply scores to get Risk Priority Number
(RPN)
– Rank for importance by RPN and Acceptance
criteria
• How much risk is acceptable? Set by team
 Alternative grading system
Adapted from ISO 14971

Severity of harm
Negligible Minor Serious Critical Catastrophic
Frequent not ok not ok not ok not ok not ok
Probability

Probable ok not ok not ok not ok not ok

Occasional ok ok ok not ok not ok

Remote ok ok ok ok not ok

Inconceivable ok ok ok ok ok
Develop and Implement
Risk Mitigation Plan

31
Response to Risk Findings

• Get more information

• Accept the risk – do nothing
• Reallocate resources
• Eliminate the risk entirely
• Transfer the risk
• Develop and implement a Risk Mitigation
Plan

32
Risk Mitigation Plan
Must Be Based On:
• Sources of potential device failures
• Potential for errors/mistakes and device
failures
• Impact of failures and errors/mistakes
• Ability to detect failures and
errors/mistakes
• Residual risk post mitigation
• Unique conditions
33
Unique Risk Conditions Affecting
QC Frequency in the Plan
• Immediate application of test result
• Result leads to significant medical response
• Result affects diagnosis, prognosis or treatment
• Low volume, infrequently performed test
• Technique sensitive tests
• Low sigma (<3.0) tests
• Analyte stability
• Use of first or second party control materials as primary
controls
• Use of electronic or imbedded controls as primary
controls
Consider Increasing QC Frequency

When:

• Risk of reporting an erroneous result is

moderate to high
• Risk that an erroneous result can harm a
patient
• Actionable residual risk is present
Verify Effectiveness

36
Quality Assessment
• On-going review for effectiveness
– No periodicity required
• Resource documents (a few)
– Historical QC data
– PT records
– Patient result review
– Specimen rejection logs
– Preventive action/corrective action records
– Competency assessment records

37
The Risk-Based Plan

ENLIGHTENMENT

PLANNING

VERIFY
EFFECTIVENESS

RISK ANALYSIS

IMPLEMENT THE
DEVELOP THE QC PLAN
PLAN
Evaluation of residual risk
• Re-estimate risk potential after mitigation
to evaluate residual risk
– Is it acceptable?

39
Records
No…not these
Records
This might be a bit much…..
ISO 15189: 2012

4.13 Control of Records

• “The laboratory shall have a documented procedure for

identification, collection, indexing, access, storage,
maintenance, amendment and safe disposal of quality
and technical records…

Records shall include the following…

n) Risk management records”

42
As with all ISO quality systems
–Document, document, document
To summarize:
• Form a team
• Set a timeline and objectives
• Set the scope
• Gather relevant information
• Perform the analysis
– Identify hazards and prioritize the importance
• Evaluate the risk, decide on and implement
mitigations
• Verify effectiveness
To summarize

• Keep it simple

• Stay on point

• Do not rely solely on manufacturer

templates
Most important

When doing the risk assessment

and making decisions about risk
and what is acceptable quality,
ALWAYS be guided by what is best
for your patients.
References
• ISO 14971:2012
Medical devices – Application of risk
management to medical devices
• ISO 15189:2012
Medical laboratories – Requirements for
quality and competence
• ISO/TS 22367
Medical laboratories –Reduction of error
through risk management and continual
improvement