Kallen A. Understanding Biostatistics

Подождите немного. Документ загружается.

140 LEARNING ABOUT PARAMETERS, AND SOME NOTES ON PLANNING

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

β(θ) = Probability for success

Population parameter θ

2θ

Figure 5.8 The geometric derivation of a power curve when the null distribution is symmetric

around zero.

the critical value x

crit

to be x

. This gives us the equation 1 − F

, 0) = α, which ﬁxes n

(recall that x

= θ

). It means that only observations x of the test statistic such that x>x

will be ‘statistically signiﬁcant’. For a shift model this can also be written β(θ

) = 0.5, so that

the size of the experiment is such that the power for the critical value θ

is 50% (so if θ

the true value, only 50% of experiments will be successful). However, the conventional way

to determine the sample size uses a different type of argument. We ﬁrst decide on a value θ

which is such that we ‘do not want to miss’ it. The precise meaning of this is that we specify

the probability of getting a successful outcome of the experiment, if θ

is the true value. In

Figure 5.8 this has been set to 80% and is shown by the dashed lines. It means that we select n

so that we obtain the power function that starts at α and satisﬁes β(θ

) = 0.8. The choice of θ

deﬁnes how steep the power curve will be, since it is anchored at level α at zero. The smaller

the θ

, the steeper the curve, which means a larger study. Usually this computation involves a

Gaussian approximation to the CDF of the test statistic, and the analytic computations needed

to derive n are outlined in Box 5.4.

We should also note the following. For the shift model we have that the p-value

(for the null hypothesis θ = 0) based on the observation x of our test statistic is

p = 1 − F (x) = 1 − β(x + θ

), which shows that the p-value will be α if x = θ

, but also

that it is considerably smaller than α if x = θ

(the effect size we did not want to miss).

It is important to understand that the power β(θ) gives the probability that the study will

produce a p-value less than α if θ is the true value. Since we do not know this true value, this

is a very crucial assumption, on which the result depends. To get an estimate of the probability

that the study as such turns out to be successful requires information on how likely it is for

different θ to be the true parameter value. If we have such information, which amounts to an

a priori distribution for θ, we can use Bayesian ideas to compute an overall probability for

a successful study outcome. To emphasize the Bayesian nature of this probability, it is often

called assurance instead.

There is a geometrical relationship between the power function β(θ) and the conﬁdence

function C(θ), in particular in the case where we have a shift model (for the location model

THE POWER CURVE OF AN EXPERIMENT 141

Box 5.4 Sample size computations

In order to determine the sample size n of an experiment, we usually start with a test

statistic

θ which estimates a parameter θ about which we want to make claims, together

with a model which describes how the distribution of

θ depends on n. This is usually

of the form

θ ∈ N(θ, σ

(θ)/n).

For a one-sided test θ ≤ 0 this means that the power function is

β(θ) = 



√

n(θ − θ

)

σ(θ)



which we solve for n under a criterion of the form β(θ

) = q. Here q is usually 0.8or

0.95, corresponding to power 80% or 95%, respectively.

To solve for n in this equation, write q = 1 − β and note that

1 − β = 



√

− θ

σ(θ

)



= 



√

nθ

σ(θ

)

− z



,R=

σ(θ

)

σ(θ

)

Here z

= 

−1

(1 − α) and we have used the fact that

√

nθ

/σ(θ

) = z

. Solving for n,

we get

n = (z

R + z

)



σ(θ

)



When σ does not depend on θ, so that R = 1, we see that this is the product of two terms,

the ﬁrst entirely concerned with conﬁdence (choice of signiﬁcance level and power) the

second the inverse of the square of θ

/σ. To use this in practice, we can ﬁrst compute

an n for the commonly used two-sided test at signiﬁcance level 2α = 0.05 and power

80%, using the formula

n =

7.85σ

We then multiply by the factor in the following table to obtain n for other levels of error

control:

Power (%)

2α

50 60 70 80 90 95

0.01 0.85 1.02 1.22 1.49 1.90 2.27

0.05 0.49 0.62 0.79 1.00 1.34 1.66

0.10 0.34 0.46 0.60 0.79 1.09 1.38

0.20

0.21 0.30 0.42 0.57 0.84 1.09

To illustrate the use of this, assume we want to compare the mean for two groups of

sizes rn and (1 − r)n, respectively, where 0 <r<1. The standard deviation σ above is

then given by σ

√

r(1 − r), where σ

is the standard deviation of the observations in

one group. With equal numbers in the two groups we have σ = 2σ

. Note that we get

best power if the groups are equal in size.

142 LEARNING ABOUT PARAMETERS, AND SOME NOTES ON PLANNING

and a one-sided test, the power curve that has 50% power at θ

∗

is identical to the conﬁdence

curve corresponding to the observation x

∗

= θ

∗

); it is important to note that the power function

is only of interest before the experiment is performed, whereas the conﬁdence function is

deﬁned by the outcome of the experiment.

For the rest of this section we will apply this general discussion to the particular problem

of planning a study which will compare two independent binomial proportions. The key part

of this planning is to decide on how large we need the study to be. Let us assume that we

want to draw samples of equal size from two groups, and we propose to use a sample of size

n = 100 for each group. We want to understand what we can expect from such a study in terms

of what the true difference in proportions needs, to be for the test to ‘provide a statistically

signiﬁcant result’ (our success criteria).

First, we need to decide on a signiﬁcance level for the test; we use the conventional level

α = 0.05 for a one-sided test,  ≤ 0. For ﬁxed p, the critical value 

is obtained as the

solution to the equation 1 − (/σ(0,p

)) = α, where σ

(, p) is deﬁned in equation (5.2)

with both n

equal to n = 100. If the observed difference 

∗

is greater than 

, we will reject

the null hypothesis. This 

will depend on p

, so we write it as 

). For ﬁxed p

this

gives us the power function

β() = 



 − 

)

σ(, p

)



Figure 5.9 gives a graphical illustration of the situation for two different values of p

In the upper left part we have plotted, with thinner lines, (/σ(0,p

)) as a function of .

They both start at the level 0.5. On the -axis we have marked where these curves intersect

the horizontal line at level 1 − α = 0.95; these intersections deﬁne the two 

)s. We have

also plotted, with thick lines, the power functions β() for the two choices of p

; note that

they both have 50% power at the respective points 

). They are, however, not simple

translations of the thinner curves because the scale factor σ(, p

) depends on .

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Power β (Δ)

0.10 0.2

Risk difference Δ = p

– p

p = 0.5

p = 0.2

Figure 5.9 Power curves for a two-group binomial test with 100 patients in each group.

SOME CONFUSING ASPECTS OF POWER CALCULATIONS 143

We see that the proposed size of the trial would have 80% chance of success (meaning

a statistically signiﬁcant result) if the true value of  is 0.15, provided the proportion in the

control group is p

= 0.2, whereas the corresponding number is 0.17 when p

= 0.5. The

latter number is larger, because the variance σ(0,p

) varies with p

in such a way that it has

its maximum when p

= 0.5, and a larger variance will require a larger sample size for the

same effect and power.

5.7 Some confusing aspects of power calculations

As mentioned above, power considerations are not fully understood by everyone, in particular

not by those who do not take the trouble to try. Some of this confusion stems from the extremely

common problem of mistaking an estimate of a parameter for knowledge about it. For this

discussion we will assume that the model is a shift model, so that the test statistic is an estimate

of θ, and that this measures an effect. In the discussion on power in Section 5.6 we introduced

two effect sizes of particular interest:

the critical level θ

, which is such that if the estimate is greater than this, we have sufﬁcient

evidence to claim an effect;

the effect we do not want to miss θ

, which is a number, chosen by us, such that if this is

the true effect, there should be a high probability of obtaining sufﬁcient evidence that

there is an effect.

We also had the true, but unknown, effect size, denoted θ

true

. In addition to these, medical

science often introduces one further number:

the minimal clinically important difference which we denote θ

MID

and call MID

for short.

The (medical) meaning of MID is that unless θ

true

>θ

MID

, the effect is not sufﬁciently large

to merit clinical interest. (Actually it is slightly more complicated: the parameter θ is usually a

mean difference, and in order to transform this into a test statistic we need to divide the observed

mean by a standard deviation. In order to carry calculations through we therefore will need to

assume a value for this standard deviation, which allows us to translate means to means over

standard deviations. However, this distinction is of no importance for the present discussion.)

In terms of planning it may be wise to take θ

= θ

MID

; we do not want to miss the minimal

clinically important effect. Once the experiment is done, we may ﬁnd an estimate θ

∗

of θ,

which is such that θ

<θ

∗

<θ

MID

. To some people this is confusing, because the study was

a success (θ

<θ

∗

) but the effect was not clinically important (θ

∗

<θ

MID

), despite the fact

that we planned the study to ﬁnd clinically important effects.

The underlying reason is the confusion of estimates with knowledge. The estimate in

itself does not help us understand what we actually know about θ; in an extreme situation the

estimate may be obtained from a single observation. The knowledge about θ is contained in the

conﬁdence functions, and the conﬁdence intervals deﬁned by it. If the appropriate conﬁdence

interval actually misses the MID, we have evidence that θ<θ

MID

, but most likely it will

contain the MID and we cannot say, with conﬁdence, that the true effect is not as good as that

deﬁned by the clinically important difference.

144 LEARNING ABOUT PARAMETERS, AND SOME NOTES ON PLANNING

Box 5.5 The (ir)relevance of the MID

The concept of a minimal important difference in clinical research has much in common

with the signiﬁcance level in statistics. Sometimes this author wonders if the MID

concept was introduced because the Type I error concept was too difﬁcult for those

making decisions on biostatistical data. This box, which is my personal opinion, explains

this viewpoint.

The MID should, like the signiﬁcance level, be context-dependent. Here it is not

about what level of evidence one wants, but about a cost–beneﬁt assessment. Suppose

there is a drug that prolongs life by one month. Is that clinically relevant? If it is free

of cost and totally devoid of side-effects, who would say no? If, on the other hand, it

costs half of what you earn and gives you constant diarrhoea, chances are you would

not consider it worth the cost. In this sense what is clinically important must depend on

other factors, including various types of costs. It may be meaningful for some outcomes

to deﬁne an MID for a speciﬁc purpose, such as the degree of improvement at which the

community is prepared to pay for a new drug. But that again must depend on the prize.

The real usefulness of the MID concept is for planning studies, as input to how to de-

cide on the sample size, and not for deciding the clinical importance of the drug. But that

leads to a catch-22 situation, since the estimate of the treatment effect should not be com-

pared to the MID for decision making. Which may be difﬁcult to avoid, since it was used

at the planning stage. But the study is done for a purpose, and in that particular context

a MID may be deﬁned. It is the universality of the concept that should be cast in doubt.

If showing that θ>θ

MID

is what we need to do, we must test the null hypothesis θ = θ

MID

instead, which is equivalent to carrying out the analysis above in the new parameter θ − θ

MID

instead. But the result of this samples size calculation too often produces numbers far too

large for studies to be affordable.

Unfortunately, estimates are regularly used as evidence by various types of reviewers

of clinical trials, including health authorities when they decide whether they can approve

drugs. Scientiﬁc arguments will often not convince the ignorant, and we may need to consider

actually powering our study so that its success criteria are that we obtain both a statistical

signiﬁcant result, and a point estimate larger than MID. This does not really provide any new

problems in itself from a mathematical point of view; we deﬁne the power function as

β(θ) = F (θ − θ

MID

)

instead. This is the power function for a test of the null hypothesis θ = 0 at the smaller

signiﬁcance level α = F (−θ

MID

), so the power decreases compared to the case without this

extra requirement. From a sample sizing perspective, we need larger studies to maintain power.

Since the requirement that the estimate should come out above the MID is essentially

equivalent to a reduction in the signiﬁcance level for the test, we might ask if we could not

retain the signiﬁcance level and instead decrease the size of the study and pick the size so that

= θ

MID

. If we do that, and the true value of θ is equal to θ

MID

, we only have a 50% chance

of success. In other words, the drug needs to do substantially better than MID for this trial to

have a reasonable power to justify the investment in it. From that observation alone it should

be clear that this way of reasoning is not fruitful, and the reason should also be obvious: do

not take an estimate alone as reliable evidence.

REFERENCES 145

5.8 Comments and further reading

The way we have introduced conﬁdence intervals and p-values, using the conﬁdence function,

is not standard in textbooks on statistics. Nor is it a new idea, since it was suggested in 1958 by

David Cox (Bender et al., 2005). It must be emphasized that it is only a graphical description

of the mathematics that is usually presented.

When we compared two binomial proportions, we did not expand further on the inaccuracy

that is a consequence of the fact that the underlying distributions are discrete distributions. It

is present, as it was for a single proportion, and is discussed by Agresti (2003). As was the

case for a single binomial parameter, there are a number of different methods to compute the

conﬁdence intervals (Newcombe, 1998). See Eide and Heuch (2001) for a discussion about

the epidemiologically important concept of the attributable fraction.

The Mantel–Haenszel pooled odds ratio for a series of 2 × 2 tables was ﬁrst proposed

by Mantel and Haenszel (1959), together with the test based on the quadratic form Q

(1).

The same test had been suggested ﬁve years earlier by Cochran, except that he used a binomial

model instead of a hypergeometric model. The test is therefore often called the Cochran–

Mantel–Haenszel chi-squared test. Since then various extensions have led to a whole family

of ‘Mantel–Haenszel methods’, some indicated in Box 5.3. As a test of association it is easily

extended to different situations with more than two levels of one or both of the two factors;

see Kuritz et al. (1988) which also gives a review of different ways to estimate the variance

of the pooled Mantel–Haenszel odds ratio. As we will discuss in a later chapter, the Mantel–

Haenszel technique is for all practical purposes equivalent to the popular logistic regression

approach, as long as we analyze the same model (i.e., use the stratum variable as a factor

in the case of logistic regression), but the latter method allows us to design more complex

models. We will encounter another application of the Mantel–Haenszel methodology when

we discuss survival analysis in Chapter 12.

References

Agresti, A. (2003) Dealing with discreteness: making ‘exact’ conﬁdence intervals for proportions,

difference of proportions and odds ratios more exact. Statistical Methods in Medical Research, 12,

3–21.

Bender, R., Berg, G. and Zeeb, H. (2005) Tutorial: Using conﬁdence curves in medical research. Bio-

metrical Journal, 47(2), 237–247.

Eide, G.E. and Heuch, I. (2001) Attributable fractions: fundamental concepts and their visualization.

Statistical Methods in Medical Research, 10(3), 159–193.

Kuritz, S.J., Landis, J.R. and Koch, G.G. (1988) A general overview of Mantel-Haenszel methods:

applications and recent developments. Annual Review of Public Health, 9, 355–367.

Mantel, N. and Haenszel, W. (1959) Statistical aspects of the analysis of data from retrospective studies

of disease. Journal of the National Cancer Institute, 22, 719–748.

Newcombe, R.G. (1998) Interval estimation for the difference between independent proportions: com-

parison of eleven methods. Statistics in Medicine, 17(8), 873–890.

Phillips, A. and Holland, P.W. (1987) Estimators of the variance of the Mantel-Haenszel log-odds-ratio

estimate. Biometrics, 43, 425–431.

146 LEARNING ABOUT PARAMETERS, AND SOME NOTES ON PLANNING

5.A Appendix: Some technical comments

5.A.1 The non-central hypergeometric distribution and 2 × 2 tables

Fisher’s hypergeometric distribution is often introduced through an urn model: you have n

red balls and n

white balls and randomly draw r of these without replacement, in which case

the number X of red balls drawn follows a hypergeometric distribution. Fisher’s non-central

distribution occurs when picking red balls and picking white balls are not exchangeable; one

is θ times more likely to pick a red ball than a white one. (From a sampling perspective it

is important that it just happened to be r balls drawn; r is observed after the experiment. If

we instead draw balls one by one until r are drawn, we get a slightly different non-central

distribution, often referred to as Wallenius’ non-central hypergeometric distribution. In the

central case when θ = 1 these are the same.) Fisher’s non-central hypergeometric distribution

Hyp(n

,r,θ) is deﬁned using the hypergeometric function (which is not a CDF)

F (θ) =

min(n

,r)



j=0





r − j



In fact, the probability for the outcome j is obtained by taking the corresponding term in this

sum and divide it by F (θ). To compute the mean and variance of this distribution there is a

trick which uses the function κ(β) = ln F(e

). In fact, if β = ln θ, then

(X) = F(e

)

−1







r − j



βj

= κ



(β)

and, similarly, V

(X) = κ



(β). These formulas are no accident, they hold because the non-

central hypergeometric distribution belongs to the exponential family, discussed in Sec-

tion 9.7. For the central hypergeometric distribution we can compute the mean and variance

explicitly as

(x) =

+ n

(x) =

r(n

+ n

− r)

+ n

)

+ n

− 1)

Fisher’s hypergeometric distribution appears, as was explained in Box 4.4, in a 2 × 2 table

with the three constraints x

+ x

= n

+ x

= n

and x

+ x

= r. In such a case

the stochastic variable X

has the Hyp(n

,r,θ) distribution, where θ is the odds ratio

in the table. Using the function F (θ) and properties of the binomial coefﬁcients, it can be

shown that

(a)

(b)

(c)

(d)

) = θ

(a−e)

(b+e)

(c+e)

(d−e)

), (5.4)

where x

(k)

= x(x − 1) ...(x − k + 1). The ﬁrst consequence of this is that

− θX

) = 0, or θ =

)

If we replace the expectations here with the observations, this means that we estimate θ with

the empirical odds ratio. Another way to express this is to say that we estimate θ using the

APPENDIX: SOME TECHNICAL COMMENTS 147

estimating equation

U(θ) = x

− θx

= 0,

and knowledge about θ can then be obtained from the study of the conﬁdence function C(θ) =

(U(θ)/

√

(U(θ))). To do this we need to be able to compute the variance of U(θ), which

can be done using equation (5.4) and some algebra (Phillips and Holland, 1987), as

(U(θ)) =

((X

+ θX

)(X

+ X

+ θ(X

+ X

))),

where n is the table total. This is estimated by

(U(θ)) =

+ θx

)(x

+ x

+ θ(x

+ x

)).

5.A.2 The gamma and χ

distributions

If X is a stochastic variable with CDF F (x), the CDF for X

is given by

G(x) = P(X

≤ x) = F(

√

x) − F (−

√

x),x>0,

which equals 2F (

√

x) − 1 if the distribution F(x) is symmetric around zero, as for the Gaussian

distribution. For the standard Gaussian distribution this means that the density is given by



(x) =

√

ϕ(

√

x) =

√

2πx

−x/2

and the distribution belongs to the general family of gamma distributions, in which (p, a)is

deﬁned by the probability density function

(p)

p−1

−ax

This family has the property that the sum of two independent such variables with a common

a is also of the same type:

(p

,a) + (p

,a) = (p

+ p

,a).

The most important cases are when p = n/2 and a = 1/2, which deﬁnes the χ

(n) distribution.

This means that if X ∈ N(0, 1), then X

∈ χ

(1). The addition formula then shows that the

sum of n squares of independent standardized Gaussian variables has a χ

(n) distribution.

The observation that the distribution of a sum of squares of Gaussian variables is a χ

distribution holds true in more generality. A multidimensional Gaussian distribution in p

variables is one for which the probability density is proportional to e

−Q(x)/2

, where Q(x)isa

quadratic form in x = (x

,...,x

) such that Q(x) ≥ 0 for all x.IfX has such a distribution,

Q(X) has a χ

distribution. To see why, we note that, by integration over slices {y; Q(y) = r},

P(Q(X) ≤ x) = C



Q(y)≤x

−Q(y)/2

dy = C



p−1

−r/2

dr.

The factor r

p−1

comes from the (p − 1)-dimensional ‘areas’ of the slices. The formula is

actually only true if the quadratic form is positive deﬁnite (i.e., never zero except at the

148 LEARNING ABOUT PARAMETERS, AND SOME NOTES ON PLANNING

origin); if not p should be the dimension of the largest linear space in which it is (this

allows for singular Gaussian distribution). The result is that Q(X) ∈ χ

(p), where p is the

dimension of the maximal space on which Q(x) is positive deﬁnite. Notation-wise we have

that X ∈ N

(m, ) if the quadratic form can be written in matrix language as

Q(x) = (x − m)



−1

(x − m)

for some matrix p × p matrix , which may be non-singular (in which case the matrix inverse

is used) or singular (in which case a generalized inverse is used).

Empirical distribution functions

6.1 Introduction

So far we have discussed rather general aspects of the role and purpose of statistics in clinical

research. We have done so against the background of 2 × 2 tables which, although simple, con-

tain most of the statistical issues that are not directly connected with speciﬁc types of data. In

this chapter we will begin the rest of this book, which is about different statistical methods. The

ﬁrst step in this is to understand how to describe non-dichotomized data in an appropriate way.

The underlying philosophy is that the values of what we measure as the outcome variable

follow a distribution in the population we study. If we take a representative sample from this

population we can use these data to compute an empirical distribution, which is an estimate

of this (true) distribution. In this chapter we will discuss the relation between the unobserved

true distribution and the observed one, and how we can deﬁne and estimate parameters in

the population distribution. The way we do this gives us a general method to obtain sample

counterparts of population parameters. However, statistics is not primarily about estimated

parameters, but rather about describing the knowledge we have acquired about the population

parameters. For this we also need to understand how we obtain knowledge about the true

distribution from the empirical one.

In addition to understanding how to obtain knowledge about a distribution function the

mathematical way (deriving formulas), we will also discuss some related areas. One is how

to obtain conﬁdence the non-mathematical way, with a technique called bootstrapping which

replaces mathematics (theory) with in silico experiments (computer simulations). The other

is a short discussion about meta-analysis and heterogeneity. The story below will explain

the connection.

6.2 How to describe the distribution of a sample

So far we have analyzed proportions. The data consist of a set {x

,...,x

}of data points, such

that each x

is either 0 or 1, and the estimate p

∗

of the proportion p is simply the average of

Understanding Biostatistics, First Edition. Anders K¨all´en.