Griffiths D. Head First Statistics

Подождите немного. Документ загружается.

this is a new chapter 521

using hypothesis tests

Look At The Evidence

I thought you

said these were

real horses.

Not everything you’re told is absolutely certain.

The trouble is, how do you know when what you’re being told isn’t right? Hypothesis

tests give you a way of using samples to test whether or not statistical claims are likely

to be true. They give you a way of weighing the evidence and testing whether extreme

results can be explained by mere coincidence, or whether there are darker forces at

work. Come with us on a ride through this chapter, and we’ll show you how you can use

hypothesis tests to confirm or allay your deepest suspicions.

522 Chapter 13

Statsville’s new miracle drug

Statsville’s leading drug company has produced a new remedy for

curing snoring. Frustrated snorers are flocking to their doctors in

hopes of finding nightly relief.

The drug company claims that their miracle drug cures 90% of

people within two weeks, which is great news for the people with

snoring difficulties. The trouble is, not everyone’s convinced.

IS your SnorIng geTTIng you down?

Then you need new SnoreCull,

The ulTImaTe remedy for SnorIng.

SnoreCull CureS 90%

of SnorerS wIThIn 2 weekS.

Cull ThoSe SnoreS wITh new SnoreCull

nore

ull

nore

ull

48 T

ableTS

the ultimate snoring remedy?

you are here 4 523

using hypothesis tests

The doctor at the Statsville Surgery has been prescribing SnoreCull

to her patients, but she’s disappointed by the results. She decides to

conduct her own trial of the drug.

She takes a random sample of 15 snorers and puts them on a course

of SnoreCull for two weeks. After two weeks, she calls them back in

to see whether their snoring has stopped.

Here are the results:

If the drug cures 90% of people, how many people in the sample

of 15 snorers would you expect to have been cured? What sort of

distribution do you think this follows?

I’m not sure these claims are

true. If they were, more of

my patients would be cured.

Cured? Yes No

Frequency

11 4

All the doctor records is

whether or not the patients

snoring has been cured.

Doctor

524 Chapter 13

If the drug cures 90% of people, how many people in the sample

of 15 snorers would you expect to have been cured? How does

this compare with the doctors results? What sort of distribution

do you think this follows?

11 12 13 1410 15

This is how many people should

have been cured, according to

the drug company.

90% of 15 is 13.5, so you’d expect 14 people to be cured. Only 11 people in the doctors sample were cured, which is

much lower than the result you’d expect

There are a specific number of trials and the doctor is interested in the number of successes, so the number of

successes follows a binomial distribution. If X is the number of successes then X ~ B(15, 0.9).

So what’s the problem?

Here’s the probability distribution for how many people the drug

company says should have been cured by the snoring remedy.

The number of people cured by SnoreCull in the doctor’s sample

is actually much lower than you’d expect it to be. Given the claims

made by the drug company, you’d expect 14 people to be cured,

but instead, only 11 people have been.

So why the discrepancy?

P(X = x)

This is how many people were

actually cured by SnoreCull.

sharpen solution

you are here 4 525

using hypothesis tests

But can we really be certain

that the drug company is at fault?

Maybe the doctor was unlucky.

The drug company’s claims might actually be accurate.

Rather than the drug company being at fault, it’s always possible that the

patients in the doctor’s sample may not have been representative of the

snoring population as a whole. It’s always possible that the snoring remedy does

cure 90% of snorers, but the doctor just happens to have a higher proportion

of people in her sample whom it doesn’t cure. In other words, her sample might

be biased in some way, or it could just come down to there being a small

number of patients in the sample.

Does that mean that the drug

company is telling lies about their

product? Shouldn’t the drug have

cured more of the doctor’s patients?

The drug company might not be deliberately telling lies,

but their claims might be misleading.

It’s possible that the tests of the drug company were flawed, and this might

have resulted in misleading claims being made about SnoreCull. They may

have inadvertent conducted flawed or biased tests on SnoreCull, which

resulted in them making inaccurate predictions about the population.

If the success rate of SnoreCull is actually lower than 90%, this would explain

why only 11 people in the sample were cured.

How do you think we can resolve this? How can we

determine whether to trust the claims of the drug

company, or accept the doctor’s doubts instead?

526 Chapter 13

Resolving the conflict from 50,000 feet

So how do we resolve the conflict between the doctor and the drug

company? Let’s take a very high level view of what we need to do.

We can resolve the conflict between the drug company and the doctor by

putting the claims of the drug company on trial. In other words, we’ll accept

the word of the drug company by default, but if there’s strong evidence

against it, we’ll side with the doctor instead.

Here’s what we’ll do:

Take the claim of

the drug company.

See how much evidence we need to

reject the drug company’s claim, and

check this against the evidence we

have. We do this by looking at how

rare the doctors results would be if

the drug company is correct.

Depending on the evidence,

accept or reject the claims of

the drug company.

In general, this process is called hypothesis testing, as you take a

hypothesis or claim and then test it against the evidence. Let’s look at

the general process for this.

Examine the claim

Examine the evidence

Make a decision

nore

ull

nore

ull

48 T

ableTS

the hypothesis testing process

you are here 4 527

using hypothesis tests

The six steps for hypothesis testing

Here are the broad steps that are involved in hypothesis testing. We’ll go

through each one in detail in the following pages.

Why all the

formality? It’s

obvious there’s

something going on.

We need to make sure we properly test the drug

claim before we reject it.

That way we’ll know we’re making an impartial decision either way,

and we’ll be giving the claim a fair trial. What we don’t want to to do

is reject the claim if there’s insufficient evidence against it, and this

means that we need some way of deciding what constitutes sufficient

evidence.

Decide on the hypothesis you’re going to test

Choose your test statistic

Find the p-value of the test statistic

Determine the critical region for your decision

See whether the sample result is within the

critical region

Make your decision

This is the

claim that we’re

putting on trial

We need to pick the statistic that

best tests the claim.

We need a

certain level of

certainty.

We need to see how rare our results

are, assuming the claims are true.

We then see if it’s within

our bounds of certainty.

528 Chapter 13

So what’s the null hypothesis for SnoreCull?

The null hypothesis for SnoreCull is the claim of the drug company: that it

cures 90% of patients. This is the claim that we’re going to go along with,

unless we find strong evidence against it.

We need to test whether at least 90% of patients are cured by the drug, so this

means that the null hypothesis is that p = 90%.

The drug company’s claim

According to the drug company, SnoreCull cures 90% of patients within

2 weeks. We need to accept this position unless there is sufficiently strong

evidence to the contrary.

The claim that we’re testing is called the null hypothesis. It’s represented

by H

, and it’s the claim that we’ll accept unless there is strong evidence

against it.

I’m the null

hypothesis. I’m the

default position. If you

think I’m wrong, gimme

the evidence.

Step 1: Decide on the hypothesis

Let’s start with step one of the hypothesis test, and look at the key claim we

want to test. This claim is called a hypothesis.

The null hypothesis is the claim you’re

going to test. It’s the claim you’ll accept

unless there’s strong evidence against it.

: p = 0.9

This is the null hypothesis

for the SnoreCull trial.

Decide on the

hypothesis you’re going

to test

Choose your test

statistic

Determine the critical

region for your decision

Find the p-value of the

test statistic

See whether the

sample result is within

the critical region

Make your decision

You

are

Here

nore

ull

nore

ull

48 T

ableTS

You have to assume I cure

90% of people unless you

can come up with good

evidence that I don’t.

null and alternate hypotheses

you are here 4 529

using hypothesis tests

The doctor’s perspective

The doctor’s view is that the claims of the drug company are too good to be

true. She doesn’t think that as many as 90% of patients are cured. She thinks

it’s far more likely that the cure rate is actually less than 90%.

The counterclaim to the null hypothesis is called the alternate hypothesis.

It’s represented by H

, and it’s the claim that we’ll accept if there’s strong

enough evidence to reject H

I’m the alternate

hypothesis. If H

let’s

you down, then you’ll

have to accept that

you’re better off with

Now that we have the null and alternate hypotheses for the SnoreCull

hypothesis test, we can move onto step 2.

So what’s the alternative?

We’ve looked at what the claim is we’re going to test, the null hypothesis, but

what if it’s not true? What’s the alternative?

The alternate hypothesis is the

claim you’ll accept if you reject H

The alternate hypothesis for SnoreCull

The alternate hypothesis for SnoreCull is the claim you’ll accept if the drug

company’s claim turns out to be false. If there’s sufficiently strong evidence

against the drug company, then it’s likely that the doctor is right.

The doctor believes that SnoreCull cures less than 90% of people, so this

means that the alternate hypothesis is that p < 90%.

: p < 0.9

This is the alternate

hypothesis for the

SnoreCull trial

530 Chapter 13

Why are we assuming the null hypothesis is

true and then looking for evidence that it’s false?

A: When you conduct a hypothesis test, you, in effect,

put the claims of the null hypothesis on trial. You give

the null hypothesis the benefit of the doubt, but then you

reject it if there is sufficient evidence against it. It’s a bit

like putting a prisoner on trial in front of a jury. You only

sentence the prisoner if there is strong enough evidence

against him.

Do the null hypothesis and alternate

hypothesis have to be exhaustive? Should they

cover all possible outcomes?

A: No, they don’t. As an example, our null hypothesis

is that p = 0.9, and our alternate hypothesis is that

p < 0.9. Neither hypothesis allows for p being greater

than 0.9.

Isn’t the sample size too small to do this

hypothesis test?

A: Even though the sample size is small, we can still

perform hypothesis tests. It all comes down to what test

statistic you use — and we’ll come to that on the next

page.

So are hypothesis tests used to prove whether

or not claims are true?

A: Hypothesis tests don’t give absolute proof. They

allow you to see how rare your observed results actually

are, under the assumption that your null hypothesis

is true. If your results are extremely unlikely to have

happened, then that counts as evidence that the null

hypothesis is false.

no dumb questions

When hypothesis testing, you assume the null hypothesis

is true. If there’s sufficient evidence against it, you reject

it and accept the alternate hypothesis.