Slide 10-1

Chapter 10
Hypothesis Testing: Deciding between Reality and Coincidence

2/10/2012

Slide 10-2

Hypothesis Testing
e.g., Is it real? Or is it just coincidence?

Deciding between two possibilities based on data Hypothesis: a statement about the population
e.g., More than 30% of customers recognize our product e.g., You will win the election e.g., Strategy Z will make you rich in the stock market

Note: a hypothesis is either TRUE or FALSE

Even with data, you may never know for sure, because of randomness
2/10/2012

Slide 10-3

Example: Dishwasher Detergent

(italics added)

From a box of Cascade: A hypothesis

Individual packages of Cascade may weigh slightly more or less than the marked weight due to normal variations incurred with high speed packaging machines, but each days production of Cascade will average slightly above the marked weight

This hypothesis is either true or false

We do not know which The package claims that it is true We could test it, e.g., by weighing a sample of boxes
2/10/2012

Slide 10-4

Example: Pure Randomness

2 tosses: Probability 1/4 for each of HH, HT, TH, TT 3 tosses: Probability 1/8 for each of
HHH, HHT, HTH, HTT, THH, THT, TTH, TTT HHH is no less likely than any other particular sequence

Coin tossing: Probability 1/2 for Heads or Tails

10 tosses: probability 1/1,024 of HH HHH HH HHH

If you toss 1,000 times, it is not surprising to find ten Heads in a row somewhere in the sequence But if you toss only 10 times and find ten Heads
You will rightly be suspicious!!!!! Reject the hypothesis that this is an ordinary coin?
2/10/2012

Slide 10-5

Null and Research Hypothesis

Research Hypothesis H1
Has burden of proof. Requires convincing evidence Often general Often your theory A, B are dependent (related) Ad works

Null Hypothesis H0
The Default. Accept unless disproven x Often specific Often randomness A, B are independent (no connection) Ad has no effect on purchase

Q = Q0
Population mean equals reference value Long-run mean oven temperature, Q, equals the desired setting Q0=3250
2/10/2012

Q { Q0
Population mean does not equal reference value Long-run mean oven temperature, Q, does not equal the desired setting

Q0=3250

Slide 10-6

The General Approach

Build a model. Identify hypotheses H0 and H1

Think Look at data and select a method to use

Confidence interval Test statistic

Decide
H0
UNLESS it is unlikely that the data could have come from the null hypothesis H0, so choose

H1
2/10/2012

Slide 10-7

Testing the Population Mean

A known, fixed number (not from the data) that we test the population mean Q against

The Reference Value Q0

Does Q = Q0?
Test by comparing X to Q0

Population (mean Q) Sample Data X1, , Xn

Does Q = Q0?

The hypotheses
H0 : Q = Q 0 H1 : Q { Q 0
2/10/2012

Is X close to Q0?

Slide 10-8

Two Ways to Decide

Is Q0 in the interval? Is Q0 a reasonably possible value for Q?

Confidence Interval X s ttableS X

t Statistic tstat ! X  Q0 / S X
Is tstat ttable ? Is X only randomly different from Q0? If YES If NO
ACCEPT H0 : Q = Q0 Accept H1 : Q { Q0 Reject H0 : Q = Q0

X IS NOT significantly different

from Q0
2/10/2012

X IS significantly different
from Q0

Slide 10-9

Example: Oven Temperature

Mean oven temperature (Q) should be Q0 = 3250 over the long run Adjust as needed, and only if needed
n = 20 X = 323.0 S = 6.0 S X = 1.34 measurements average temperature variability (of individual measurements) variability (of the sample average)

We are 95% sure that the unknown mean temperature Q is between 320.2 and 325.8

Should we turn up the temperature?

2/10/2012

Slide 10-10

Example (continued)

We are 95% sure that the unknown mean temperature Q is between 320.2 and 325.8

Should we turn up the temperature?

NO!
Reference value Q0 = 325 is in the confidence interval The mean oven temperature could reasonably be 325 Adjusting now could add to the variability of the system
320.2 is lower 95% limit 320 X ! 323 .0
2/10/2012

Q0 = 325

325.8 is upper 95% limit 330

Slide 10-11

Example (hypotheses and results)

H0: Q = 325 Null Hypothesis H1: Q { 325 Research Hypothesis

325 is in the 95% confidence interval

(from 320.2 to 325.8)

So accept the null hypothesis H0: Q = 325 The sample average (323.0) is not significantly different from the reference value (Q0 = 325)

Using the t statistic

tstatistic ! X  Q 0 / S X ! (323 .0  325 ) / 1.34 ! 1.49
tstatistic !  1.49 ! 1.49 2.093 ! ttable So accept H0
2/10/2012

Slide 10-12

Example: Candy Bars

n = 300 X = 1.466 S = 0.132 S X = 0.00762 and measurements (candy bars weighed) average weight in ounces variability (of individual measurements) variability (of the sample average) H1: Q { 1.5
1.451 is lower 95% limit 1.481 is upper 95% limit

Package claims 1.5 ounces

Test whether population mean weight Q is 1.5 ounces

H0: Q = 1.5

95% confidence interval: from 1.451 to 1.481

Reject H0 Accept H1
Because 1.5 is not in the interval
2/10/2012

Q0 = 1.5

1.4 X ! 1.466

1.5

Slide 10-13

Example (candy bars continued)

Population mean weight Q could not reasonably be 1.5 ounces Sample average (1.466) is significantly different from claimed weight (Q0 = 1.5)
More than just randomly different. Less than a 5% chance of seeing such a large difference if Q were 1.5

Average weight (1.466) is significantly less than the claimed weight (1.5)
This is a one-sided conclusion
for a significant two-sided test
2/10/2012

Slide 10-14

Example: Binomial
T0 = 20% is the reference value T = ?, the population percent, is unknown X = 83 out of n = 268 interviewed said they would buy p = X/n = 31.0%, the sample percent

Goal: that 20% would buy the product

Has goal has been reached? Or is it just random?

H0: T = 0.20 and H1: T { 0.20

95% confidence interval for T is from 25.4% to 36.5%

Since T0 is not in the confidence interval

Reject H0 and accept H1
The difference (from 20% to 31.0% is significant)
2/10/2012

Slide 10-15

Example: Binomial (continued)

The population percentage (p = 31.0%) could not reasonably be T0 = 20% The sample percentage (p = 31.0%) is significantly different from the goal (T0 = 20%) The observed percentage (p = 31.0%) is significantly larger than the goal
[This is a one-sided conclusion for a significant two-sided test]

Conclusions

We have significantly exceeded the goal!

2/10/2012

Slide 10-16

Interpretation
H0 could not reasonably have produced the data Either
H1 is true, or H0 is true, but you made a TYPE I ERROR
Happens 5% of the time when H0 is true

If you reject H0 and accept H1

A strong conclusion A significant result

You have earned a license to explain the observed difference

2/10/2012

Slide 10-17

Interpretation (continued)
H0 could reasonably have produced the data Either
H0 is true, or H1 is true, but you made a TYPE II ERROR
Difficult to control This error is possible, and is very likely if Q0 is close to Q

If you accept H0

A weak conclusion Not a significant result

Little or nothing to explain The observed difference might just be random

2/10/2012

Slide 10-18

Errors in Hypothesis Testing

Your Decision
Accept Null Hypothesis H0 Null Hypothesis H0 Research Hypothesis H1 Yay! Correct Decision Whoops! Type II Error [not easily controlled] Accept Research Hypothesis H1 Whoops! Type I Error [level 0.05] Yay! Correct Decision

2/10/2012

The Truth

Slide 10-19

Assumptions

Assumptions needed for validity of Hypothesis Testing

1. Data are a RANDOM SAMPLE from the population of interest
(So that the sample can tell you about the population)

2. The sample average X is approximately NORMAL

Either the data are normal (check the histogram) Or the central limit theorem applies:
Large enough sample size n, distribution not too skewed

(So that the t table is technically appropriate)

(Same assumptions as for confidence interval)

2/10/2012

Slide 10-20

Test Levels: 5% and others

t table columns correspond to different test levels

Set the probability of wrongly rejecting the null hypothesis H0 when it is true (Type I error rate)
The usual standard is the 5% test level
Corresponds to 95% confidence interval If H0 is rejected, the result is significant (p<0.05)

Can also test at the 1% test level (99% confidence interval)

If H0 is rejected, the result is highly significant (p<0.01)
Because the evidence against the null hypothesis is stronger

And at the 0.1% test level (99.9% confidence interval)

If H0 is rejected, the result is very highly significant (p<0.001) And, sometimes, at the 10% test level (90% confidence interval)
2/10/2012

Slide 10-21

p-Values
Often provided by computer analysis
e.g., p = 0.0297

The smallest test level that is significant

Tells the strength of the evidence against H0

Small p value says data unlikely to come from H0 Reject H0 if p is small enough
Not significant (p > 0.05) Significant (p < 0.05) Highly significant (p < 0.01) Very highly significant (p < 0.001)

What if p = 0.374? Not significant because p > 0.05

2/10/2012

Slide 10-22

Example: Medical Production

n = 10 production runs X = 68.3 grams produced S X = 4.617 grams

Production of a new drug for heart attack treatment

Test: is yield 55 grams (on average, long-term)?

H0: Q=55 and H1: Q{55
The reference value, 55, is the budget break-even point

95% confidence interval: from 57.86 to 78.74

Significant at 5% level because 55 is not in the interval

99% confidence interval: from 53.29 to 83.31

Not significant at 1% level because 55 is in the interval

Result: significant (p < 0.05)

2/10/2012

Slide 10-23

Example (continued)

The yield is significantly above the budget breakeven point Significant at the 5% level but not at the 1% level
If the long-term yield Q were 55, we would see such a large difference (Q !55 to X ! 68.3) less than 5% of the time but not less than 1% of the time

Actual p-value from computer calculation

p = 0.0181 It is indeed less than 5% and greater than 1%
2/10/2012