Dreaded p Value

IN MEDICINE
S P RAO
 WHY p?

Q: Why do so many medical colleges teach p = .05?

A: Because that's still what the scientific community and
journal editors use.

Q: Why do so many people still use p = 0.05?

A: Because that's what they were taught in college
 NARAYANA MEDICAL COLLEGE, NELLORE
POST GRADUATE THESIS, 2017
Type of
Topic Title Department Statistics Used
Study
Comparison of DEXMEDETOMIDINE-PROPOFOL AND Student t test,
CLONIDINE-PROPOFOL for insertion of Laryngeal Mask Anaesthesiology RCT Chi-square/ Fisher
Airway- A Clinical Study Exact test
Prospective Randomized Comparative Study of Clinical
student’s ’t’ test &
Performance of Two Supraglotic Airways, Proseal lma and Anesthesiology RCT
Chi-Square test
I-Gel in Elective Surgeries
A Comparative Study Between Dexmedetomidine 1μg/kg
and Magnesium Sulphate 30 mg/kg in Attenuating Independent sample
Anesthesiology RCT
Cardiovascular Stress Response During Direct ‘t’ test
Laryngoscopy and intubation
Comparative study between ROPIVACAINE with
DEXMEDETOMIDINE and BUPIVACAINE with Student’s ’t’ test &
Anesthesiology RCT
DEXMEDETOMIDINE in Brachial Plexus Blocks in Upper Chi-Square test
Limb Surgeries
A Comparative study between Oral PREGABALIN150mg
and GABAPENTIN 300mg for Postoperative Analgesia Anesthesiology RCT Student-t test
 LAUGHTER SIDE

• Statisticians are from Mars: Clinicians are from Venus

• 3 kinds of lies: lies, damned lies, & statistics
• One out of every four people is suffering from some form of
mental illness. Check three friends. If they're OK, then it's
you!
 Hey! You are Here

Medical Inference Statistical Inference

Hypothesized Underlying Disease Hypothesized True Treatment Differences
Hypothesis 1 Hypothesis 2 Hypothesis 3
Disease A Disease B Disease C ∆=0% ∆=5% ∆=10%

Deduction Deduction

Induction Induction
Cough Fever Rash Angina Splenomegaly - 5% 0% 5% 10% 15%

Possible Observed Signs & Symptoms Possible Observed Treatment Differences

WAH! THAT’S GAUSSIAN!!!
 OH! NO!!!

The ‘scientific method’ of testing hypotheses by statistical

analysis (p values) stands on a flimsy foundation

ASA,2010
SEDUCTIVE CERTAINTY OF SIGNIFICANCE

S tatistical
H ypothesis
I nference
T esting
 EXAMPLE

A recent randomized, controlled trial of hydrocortisone

treatment for the chronic fatigue syndrome showed a
treatment effect that neared the threshold for statistical
significance, P < 0.06.

The discussion section began, “. . . hydrocortisone

treatment was associated with an improvement in
symptoms... This is the first such study . . . to demonstrate
improvement with a drug treatment of [the chronic fatigue
syndrome]”
 WHY p Value?

Probability that the difference will not vanish if

repeated samples from the same population/
nature are studied
 PHILOSOPHICAL BASIS
STATISTICAL TESTS

• Evidence is sought against NULL Hypothesis and

not for the NULL
eg. Peptic ulcer caused by increased acidity
Assumption of Innocence

• Deals with uncertainties in sample fluctuations

• Do not consider the Effect Size

 ERRORS IN JUDGEMENT

COURT SETTING STATISTICAL SETTING

Assumption of NULL Hypothesis
Judgement Innocence Study Results
True False True False

Serious H0 Reject Type I

Guilty √ Error √
Error

√ H0 Not √ Type II
Not Guilty Error Error
Rejected
STATISTICAL SIGNIFICANCE p VALUE

• Uses Gaussian Properties

• Chance of obtaining the observed values if NULL is

TRUE

• If Chance is small, possibly NULL is Not Accepted:

Statistical Significance
 p VALUE
• Probability of Type I Error* is P-Value

• P value could be between 0 and 1

• NULL hypothesis of No difference is NOT ACCEPTED

• Inter-individual variation

• Table value: Probability that the data would be at least

as extreme those observed if the H o is true
 Level of Significance
• Type I error seriousness requires threshold fixed

• Threshold = Level of Significance

• 0.05 but not 0.00

 STOP

A classic statistical puzzle

involves two treatments, A
and B, whose effects are
contrasted in each of six
patients. Treatment A is better
in the first five patients
and treatment B is superior in
the sixth patient
 PROBLEM

Suppose you have to go to UHTC,

There are two routes; one through highway and the other
shortest way through the dusty streets of Nellore. You
calculate the time required to be travelled by these
routes. There is not much difference. P value <0.001. The
gain in time is 57 sec.
Which route do you prefer?
 PROBLEM

After many hours of packing of bags in the car, we are about

to leave for the airport. I sat in the drivers seat and my hand is
on the ignition key. At that point my wife asked me” Where is
the Camera?”
I had no clue. My course of action is……………………..
I decided to search inside the house. A few minutes later, I told
my wife that I have looked in all the normal places inside and
couldn’t find the camera.
We conclude that it must be in the car
 Statistical Vs Clinical Significance
• P value for a difference in samples is less than the predetermined

ά (0.05), the difference is Statistically Significant

• Statistical Significance is only one aspect of results

• p Value hides: Quality of thought; Planning; Execution; biologic/

clinical meaning; and cause of difference

 POWER
• Type II error : failure to detect real difference
 ESTIMATES
• How big or small something is and how big or small
something is compared to something else?

• The whole point of statistics is to produce p Values?

 TRYING MULTIPLE THINGS UNTIL YOU GET
THE DESIRED RESULT

• Data-dredging
• Snooping
• Fishing
• Significance-chasing
• Double-dipping
• P-hacking
 INTERPRETATION OF p >0.05

a) The chances are greater than 1 in 20 that a difference would be

found again if the study were repeated

b) The probability is less than 1 in 20 that a difference this large could

occur by chance alone

c) The probability is greater than 1 in 20 that a difference this large

could occur by chance alone

d) The chance is 95% that the study is correct.

INTERPRETATION p VALUE
 INTERPRETATION p VALUE

We have a hypothesis: Kohli rocks as usual. If the

hypothesis had been true, the probability of him
losing would have been so small, say, < 5%. But he
lost the game. So the unlikelihood was considered as
evidence against his fair play.
 ANOTHER EXAMPLE

Player Goals Total

Positive Negative

Myself 3 4 7

Expert 7 0 7
P value= o.07
 BAYES THEOREM

Prior ODDS of Hypothesis being Subjective Component

TRUE
(Before seeing the Data)
X
Bayes Factor
Data Component (“evidence”)

Final (“Posterior”) ODDS of Null

Hypothesis being True Probability of TRUTH
DO YOU KNOW THIS

R. NUZZO; SOURCE: T. SELLKE ET AL. AM. STAT. 55, 62–71 (2001)

 If Not, then
P-certified, not P-hacked

• Effect size

• Confidence interval

• Bayesian Statistics: Likelihood Ratio

 BELIEVE IT OR NOT?

Believe