Use of EBM in Developing Clinical Practice Guidelines

Kuntjoro Harimurti
Department of Internal Medicine Center for Clinical Epidemiology and Evidence-Based Medicine Faculty of Medicine UI / Cipto Mangunkusumo Hospital Jakarta

What is EBM?
Evidence-based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. - David Sackett -

Evidence-based Medicine
Evidence-based Medicine

Gathering medical information

Evaluating quality of medical information

Making medical decisions using best evidence

The EBM Paradigm

Patient with problem

Apply the evidence

Formulate in answerable question

Critically appraise the evidence

Search the evidence

Elements of EBM - identify relevant best evidence

Define clinical question to be answered (for a specific clinical situation)
Find best scientific evidence relevant to the specific clinical question (using systematic methods) Rely on best evidence to guide clinical decisionmaking

Elements of EBM - identify relevant best evidence

Original evidence clinical epidemiology (e.g. clinical trials) descriptive epidemiology outcomes research / economic studies Literature synthesis / analysis systematic literature reviews / meta-analyses

clinical practice guidelines

cost-effectiveness studies

Elements of EBM - focus on clinical outcomes

Clinical outcomes (things that matter to patients and families) survival impairment / disability / disfigurement symptom severity quality of life (QOL) cost / convenience

Elements of EBM - use standard measures of effect

Evaluate therapies using standard measures: How does it affect rate, degree or timing of:
Physical impairment, disfigurement or death Return to normal functioning Symptom resolution Potential harms (from therapy) Total costs (for patient, employer and society)

Allows comparisons of benefits and harms of different clinical methods

Elements of EBM - assess likely benefits / harms

Evaluate clinical methods on benefits / harms
Beneficial Likely to be beneficial Trade off between benefits and harms Unknown effectiveness Unlikely to be beneficial Likely to be ineffective or harmful

Promotes informed decision-making by clinicians and patients

Elements of EBM - base decisions on best evidence

Find best scientific evidence that is applicable to the specific clinical situation
individual patients clinical situation program or policy decision

Use evidence to guide decision-making

does the likelihood of benefit outweigh likelihood of harm enough to justify the cost? same question applies to individual and society

EBM is important because provides a science-based method for

Improving physician practice
increase effectiveness / decrease harms (better clinical outcomes / cost-effectiveness)

Increasing consumer knowledge

understand potential benefits / harms

Building quality into healthcare systems

using practice guidelines, quality indicators

Guiding government / employer policies

guide programs / policies on healthcare

What are clinical practice guidelines?

Clinical practice guidelines are recommendations for clinicians and consumers about optimal and

appropriate care for specific clinical situations

Formal or informal guidelines are the basis for all clinical practice
Basis for most clinical decisions Foundation of clinical teaching

Mental short-cuts and memory aids for common or complex problems

Primary method to evaluate care patterns and monitor standards of care

When are guidelines needed ?

Institute of Medicine (IOM) criteria clinical practice guidelines are useful when: the problem is common or expensive there is great variation in practice patterns there is enough scientific evidence to determine appropriate and optimal care

(IOM , 1992)

Types of guideline development approaches

Single author - expert opinion
Single author - systematic literature review Consensus panel using expert opinion only Consensus panel using evidenced-based approach

Steps in developing guideline recommendations

define clinical questions of interest develop summary of evidence on:
clinical efficacy (potential benefits) potential harms / projected costs

weigh likelihood of benefit versus likelihood of harms, and consider costs develop finding / recommendation statements document all aspects of the process

Evidence-based guidelines - need to explicitly document

methodology and assumptions evidence reviewed summary of findings decision-making rules for recommendations rationale for each conclusion and recommendation statement

Types of guideline statements

Based on AHCPR guidelines, the types of guideline statements are: Recommendation for use Option for use Recommendation against use

Strength of evidence ratings for guideline recommendations

A = Strong research based evidence
multiple relevant, high quality studies

B = Moderate research based evidence

one relevant, high quality study

C = Limited research based evidence

one adequate study, somewhat relevant

D = Panel opinion
based on information not meeting criteria for A-C

Steps in EBM practice

Formulate clinical problems in answerable questions 2. Search the best evidence: use internet or other onVIA line database for current evidence 3. Critically appraise the evidence for Validity (was the study valid?) Importance (were the results clinically important?) Applicability (could we apply to our patient?) 4. Apply the evidence to patient 5. Evaluate our performance
1.

Main area
Diagnosis (Determination of disease or problem) Treatment (Intervention necessary to help the patient)
Prognosis (Prediction of the outcome of the disease)

Others:
Meta-analysis Clinical guidelines Economic analysis Clinical decision making Cost-effectiveness analysis Qualitative research

(I) Formulating clinical questions

Four elements of good clinical question: PICO

The Patient or Problem The Intervention Comparative intervention (if relevant) The Outcome

Four elements of a well constructed clinical question: PICO

P I C O

The Outcome Description The main of patient intervention alternative expected or problem considered to compare from this with the intervention? intervention
B e b r i e f a n d s p e c i f i c

Relevance: Type of Evidence

POE: Patient-oriented evidence
mortality, morbidity, quality of life DOE: Disease-oriented evidence pathophysiology, pharmacology, etiology

Comparing DOES and POEMs

Example DOE
Drug A PVC On ECG

POEM
Drug A > mortality Drug X mortality

Comment
DOE & POEM contradicts

Antiarrhythmic Therapy
Antihypertens. Therapy Prostate screening

Drug X BP

POEM agrees With DOE

PSA screening detects prostate Ca. early

? whether PSA screening mortality

DOE exists, but POEM unknown

II Searching the evidence

Online Medical Information Resource

Resource Google Scholar MedlinePubmed emedicine Medscape Medical Matrix Clinical practice guidelines MD consult UpToDate Best Evidence
18:24

Internet address http://scholar.google.com.br/ www.ncbi.nlm.nih.gov/PUBMED www.emedicine.com www.medscape.com www.medmatrix.org/index.asp www.guidelines.gov www.cma.caa/cpgs www.mdconsult.com

Annual Cost Free Free Free Free Free Free Free $200

www.Uptodate.com $495 www.acponline.org/catalog/electronic/best_evidence.htm $85

EBM Searching

Step Searching in PubMed

1. 2. 3. Formulate a focused PICO question. Define keywords. Identify the type of article that is most likely to answer your question: therapy, diagnosis, harm, etc. 4. Identify the design that is most likely to answer your question: RCT, cohort, cross sectional, case control, etc. 5. Always try to find systematic reviews. 6. Use MeSH or Thesaurus or text words for searching 7. Combine concepts with Boolean operators (AND/OR/NOT). 8. Limiting to age, sex, language, year of publication, etc. 9. Evaluate search results. 10. Use "Related Articles" link option.

Types of study
Question
Intervention Etiology and risk factor

Best study design

Randomised controlled trial (RCT) RCT Cohort study Case-control study Cohort study Cross-sectional study Cross-sectional study with random or consecutive sample Cohort/survival study

Frequency and rate Diagnosis Prognosis and prediction

Boolean operators
AND

OR

NOT

Strategy of using Boolean operators: first increase sensitivity, then specificity

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. One keyword concerning the patient Another keyword concerning the patient #1 OR #2 One keyword concerning the intervention Another keyword concerning the intervention #4 OR #5 One keyword concerning the outcome Another keyword concerning the outcome #7 OR #8 #3 AND #6 AND #9

Answerable question - PICO

Patient or Problem
Description of the patient or the target disorder of interest questions

Intervention

Comparison Intervention

Outcome

Could include: Relevant most often Clinical outcome of Exposure when looking at interest to you and Diagnostic test therapy your patient Prognostic factor Therapy Patient perception etc.

Centre for Evidence-Based Medicine; http://www.cebm.utoronto.ca

Answerable question - PICO

P Population/patients
In elderly patients with hypertension,

Intervention/indicator
does the ACE-inhibitors use

C Comparator
compared to other hypertension treatment

O Outcome
will reduce cardiovascular-related mortality/morbidity?

Effective searchinguse PI(C)O!

AND AND

Patients Elderly OR OR OR Older people Geriatric

Intervention ACE-inhibitors ACE-i Angiotensinconverting enzymes

Outcome Death Stroke Myocardial infarction

Senior

Clinical problem Define important, searchable question Select most likely resource Select second most likely resource

General search strategy

Design search strategy

Summarized the evidence Apply the evidence Poor yield

Design search strategy

Summarized the evidence

III Appraising the evidence: VIA

VIA
Validity: In Methods section: design, sample, sample size, eligibility criteria (inclusion, exclusion), sampling method, randomization method, intervention, measurements, methods of analysis, etc Importance: In Results section characteristics of subjects, drop out, analysis, p value, confidence intervals, etc Applicability: In Discussion section + our patients characteristics, local setting

Example: Critical appraisal Diagnostic Test

Was there an independent, blinded comparison with a gold standard? Was the test evaluated in an appropriate spectrum of patients? Was the reference standard applied regardless of the test result? Was the test validated in a second, independent group of patients? Calculate sensitivity, specificity, positive and negative predictive value, likelihood ratio

Example: Critical appraisal for therapy

Were the subjects randomized? Were all subjects received similar treatment? Were all relevant outcomes considered? Were all subjects randomized included in the analysis? Calculate CER, EER, RRR, ARR, and NNT Were study subjects similar to our patients in terms of prognostic factors?

Translating evidence into a clinical practice guideline

Using and gathering opinion
Will be used to interpret evidence and to derive recommendation in the absence of evidence To assess issue such as generalisability of evidence

Resource implications and feasibility

Different view of implication among clinician and healthcare providers Feasibility issues: time, skills, staffs, equipment

Grading recommendation
Provide user a confidence that following guideline will produce the desired health outcome Strength of recommendation classification: range from simple to complex

Weight of Scientific Scrutiny

Hierarchy of evidence
Meta-analysis of RCT
Level 1

Rec A

Large RCT
Small RCT
Level 2

Non-Randomized trials
Observational studies Case series / reports Anecdotes, expert, consensus
Level 4 Level 3

Suggestions in developing guidelines

Making the goals of each guideline explicitfor example, effectiveness, cost effectiveness, equity Providing information on the reasons for disagreements within a guideline development groupsuch as, differences in interpretation of the research literature, differences in personal experience, different perceptions of or responses to costs Publishing information on the closeness of agreement about a recommendation as well as the strength of support for a recommendation.

Updating clinical guideline

Situations might require clinical guidelines to be updated:
Changes in evidence on the existing benefits and harms of interventions Changes in outcomes considered important Changes in available interventions Changes in values placed on outcomes Changes in resources available for health care

Thank you