Appraising an article on therapy

Madeleine Grace M. Sosa, MD FPPS,FPNA,FCNSP,MSCE Full professor DLSHSI College of Medicine

General objective
To apply the various rules of evidence in deciding if the results of an article on treatment is VALID

Specific Objectives
To discuss the rationale of each of the users guides pertaining to the validity of claims on effectiveness To recognize the strengths and weaknesses of RCTs as compared to other non-experimental study designs To define and differentiate the concepts of validity and precision, as pertain to the results of clinical trials To differentiate between dichotomous , continuos and other scales of treatment

Specific Objectives
To recognize the differences between various measures of treatment effect including absolute risk reduction, relative risk , relative risk reduction and number needed to treat To differentiate the concepts of noise and bias, as they pertain to clinical trials To differentiate between a point estimate and interval estimate of a treatment effect

APPRAISAL TIPS
Evaluating Directness Appraising Validity Interpreting Results Assessing Applicability Individualizing the Results

Evaluating directness
Does the study provide a direct enough answer to your clinical question in terms of
P: patient population with a certain disease E/I: the exposure( or treatment) to be administered C: Comparison group O: the outcome( or condition ) that the treatment are intended to prevent or promote M: Methodology

Architecture of a focused question: a 4-part review question

P - Who is the patient or what problem is

being addressed?

I - What is the intervention or exposure? C What is the comparison group? O - What is the outcome or endpoint?
+ study design
Richardson et al. The well-built clinical question: a key to evidence-based decisions. ACP Journal Club 1995;A-12 Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med 1997;127:380-7.

Formulation of a therapy question

Intervention Outcome

Is Zinc effective in treating cold?

Patient/problem Intervention

In children with common cold, is oral Zinc effective in reducing the duration of symptoms, as compared to placebo?

Outcome

Comparison

+ RCTs

Compare the clinical Question with the Research question

Clinical Question
P E/I

Research Question

C
O

When will there be a mismatch in P? What will happen if there is a mismatch in E? When will there be a discrepancy in O?

Appraising validity
Were patients randomly assigned to treatment groups? Was allocation concealed? Were baseline characteristics similar at the start of the trial? Were patients blinded to the treatment assignment?

Appraising validity
Were caregivers blinded to treatment assignment? Were study personnel blinded to the treatment assignment? Were all patients analyzed under the groups to which they were originally randomized? Was follow-up rate adequate?

BLINDING
Issues in the assessment of outcome status Use the same methods of ascertainment for treatment and control groups
Single Double Triple

Subject Observer
Analyst

X X X

Appraising validity

Randomization

Allocation Concealment

Blinding

Appraising validity.... Methods

Interpreting results
How large was the effect of the treatment? How precise was the estimate of the treatment effect?

How large is the effect of treatment?

Compare the outcomes in the treatment and control groups Outcomes: Continuous variable e.g. change in weight, change in quality of life scale, change in BP Dichotomous variable e.g. dead or alive, hospitalized or not

Measurement of estimates
Dichotomous variable RR, RRR, ARR Continuous variable Mean difference

Absolute risk reduction (ARR)

ARR= risk change
ARR: usually in percent >0%: Treatment is beneficial =0%:Treatment has no effect <0%: Treatment is harmful

ARR= Rc-Rt
Rc: risk in the control Rt: risk in the treatment

Example: express your change in weight, if it went down from 80kg ( Wc) to 60kg(Wt)

Absolute weight reduction: weight change= 20kg

I lost 20kg ( 2% of my risk)

RR: new weight/original weight= 60/80=0.75

My risk is now 0.75( I now weigh 75% of what I used to weigh)

Relative weight reduction: 20/80= 25%

I lost 25% of my risk( I lost 25% of my weight)

Relative risk (RR)

RR= new risk/ original risk RR= Rt/Rc
RR: usually in decimal <1.0: treatment is beneficial =1.0: treatment has no effect >1.0: treatment is harmful

Example: express your change in weight, if it went down from 80kg ( Wc) to 60kg(Wt)

Absolute weight reduction: weight change= 20kg

I lost 20kg ( 2% of my risk)

RR: new weight/original weight= 60/80=0.75

My risk is now 0.75( I now weigh 75% of what I used to weigh)

Relative weight reduction: 20/80= 25%

I lost 25% of my risk( I lost 25% of my weight)

Relative risk reduction (RRR)

RRR= risk change/original risk RRR= Rc-Rt / Rc
RRR: usually in percent >0%: Treatment is beneficial =0%:Treatment has no effect <0%: Treatment is harmful

Example: express your change in weight, if it went down from 80kg ( Wc) to 60kg(Wt)

Absolute weight reduction: weight change= 20kg

I lost 20kg ( 2% of my risk)

RR: new weight/original weight= 60/80=0.75

My risk is now 0.75( I now weigh 75% of what I used to weigh)

Relative weight reduction: 20/80= 25%

I lost 25% of my risk( I lost 25% of my weight)

 Old drug: Ethoxusimide

Treated = 53% Failure= 47% (Rc)

ARR= .47-.71= - .24 RR= .71/.47= 1.5

RR > 1.0

New drug: Lamotrigine

Treated= 29% Failure= 71%(Rt)

RRR= -.24/.47= -.51% 51%

RRR< 0

****Freedom from Failure : Treated

 Old drug: Ethoxusimide

Treated= 53% Failure= 47% (Rc)

New Drug : Valproic Acid

Treated: 58% Failure= 42% (Rt)

ARR= .47 - .42= .05 RR= .42/.47 = 0.89 **RR< 1.0 RRR= .05/.47= .106= 10.6% **RRR >0%

****Freedom from Failure : Treated

The primary outcome was freedom from treatment failure after 16 weeks of therapy

Absolute risk reduction: Rc- Rt

risk change(____ of my risk)

RR: Rt/Rc
My risk is now _____( The risk is now ____ of the previous risk )

Relative weight reduction :Rc-Rt/RC

I lost ____of my risk

HOW PRECISE WAS THE ESTIMATE OF THE TREATMENT EFFECT?

Interval estimates Estimated at 95% CI- 95% sure that the true effect of the treatment lies within this range RRR 75% ( 95%CI: 52%, 90%)

CI: confidence interval

When both ends of the CI are on the side of the benefit, the treatment is definitely beneficial When both ends of the CI are on the side of the harm , the treatment is definitely harmful

CI: confidence interval

When one end reflects important benefit and the other end reflects harm, then the study is inconclusive When one end reflects a small unimportant benefit and the other end reflects a small unimportant harm, then for all intents and purposes the two treatments being compared are equal.

Confidence Interval = 95%

RR= Rt/Rc
RR: usually in decimal <1.0: treatment is beneficial =1.0: treatment has no effect >1.0: treatment is harmful Ex. RR= 0.89 CI= .05- .91 CI = .05-1.23

beneficial

1.0

Harmful

Which is the best CI???

RR=1.0

Relative risk (rr)

RR= new risk/ original risk RR= Rt/Rc A. RR=2.3( 95%CI:1.5, 3.1) B. RR=0.98(95%CI:0.95, 1.02) RR: usually in decimal C. RR=0.63 (95%CI: <1.0: treatment is 0.53,0.73) beneficial D. RR=0.98( =1.0: treatment has no 95%CI:0.50,1.50)
effect >1.0: treatment is harmful

Assessing applicability
Are there biologic issues that may affect applicability of treatment? Are there socio-economic issues affecting applicability of treatment?

Individualizing the results

Are the likely treatment benefits worth the potential harm and costs? Compute for NNT ( number needed to treat) NNT=100/ARR ARR= .05 NNT = 100/.05= 2000
you need to treat 2000 patients with seizures to treat 1 absence epilepsy NNT x cost of treatment

 Even if a treatment is beneficial, it may have adverse effects or it may be too expensive Estimate NNT X cost of treatment (and duration if relevant)>> overall cost to prevent the event

EBM Steps

Learning Opportunities

Formulating Questions

Searching for Evidence

Critical Appraisal

Integrating Evidence with Clinical scenario

Bringing Change to Practice

Ward Round

+++

(+)

+++

Journal Club

++

+++

Clinical Teaching And Assessment

++

(+)

++

+++

Outpatients Clinic

+++

(+)

++

+++

Formal Clinical Meeting

+++

(+)

++

Audit

++

++

++

+++

Clinical tools to teach various steps of EBM in different clinical settings

Clinical problem Decision making

Experience and expertise

Authoritative practice

Clinical problem

Experience and expertise

Authoritative practice

making

Clinical tools to teach various steps of EBM in different clinical settings

Clinical problem Apply evidence in decision making

Ask answerable questions

Ward rounds, Clinics

Acquire (and appraise) evidence Journal club Appraise evidence Managing bringing change in practice

Morbidity/mortality meeting Audit Integrate evidence into practice

EBM practice requires:

Asking Acquiring Appraising Applying Assessing