Kallen A. Understanding Biostatistics

Подождите немного. Документ загружается.

170 EMPIRICAL DISTRIBUTION FUNCTIONS

were using a certain sedative in two different isomer forms. The full story of both these data

and the test is described by Senn and Richardson (1994).

Bootstrapping is one, but not the only, computer-intensive method that can help us make

reasonable inference in situations where we cannot obtain analytical expressions for the statis-

tics involved. It was pre-dated by a similar method, called the jackknife (named after the Swiss

army knife) which from a data set of n observations formed n new data sets of size n − 1

by leaving out one observation at a time. As far as bootstrapping is concerned, we have only

presented the general idea and ignored the bells and whistles, of which there are many. In par-

ticular, we do not discuss how to obtain a more efﬁcient algorithm if we require a conﬁdence

interval with good precision in the tails, but not necessarily in the center of the distribution.

Both the book by Efron and Tibshirani (1993) and the shorter tutorial by Carpenter and Bithell

(2000) discuss such problems.

The discussion in Section 6.8 is about random effects and heterogeneity. This is more or

less the normal state of affairs in biology, and we took the opportunity to illustrate how we

can approach it model-wise and analysis-wise. More examples will come later, in particular

when we discuss modeling in general and dose response in particular, where further important

aspects of this will be discussed. The data we used as background material for the discussion

here is from Yusuf et al. (1985).

References

Carpenter, J. and Bithell, J. (2000) Bootstrap conﬁdence intervals: when, which, what? A practical guide

for medical statisticians. Statistics in Medicine, 19, 1141–1164.

aki, E. (1984) Handbook of Statistics vol. 4 New York: Elsevier Science chapter Empirical distribution

function, pp. 405–430.

Efron, B. and Tibshirani, R.J. (1993) An Introduction to the Bootstrap vol. 57 of Monographs on Statistics

and Applied Probability. New York: Chapman & Hall.

Senn, S. and Richardson, W. (1994) The ﬁrst t-test. Statistics in Medicine, 13, 785–803.

Yusuf, S., Peto, R., Lewis, J., Collins, R. and Sleight, P. (1985) Beta blockade during and after myocardial

infarction: an overview of the randomized trials. Progress in Cardiovascular Diseases, 27, 335–371.

APPENDIX: SOME TECHNICAL COMMENTS 171

6.A Appendix: Some technical comments

6.A.1 The extended family of the univariate Gaussian distributions

In this appendix we will outline the basic and classical statistical theory for data with a

common Gaussian distribution, and mathematically derive both Student’s t distribution and

Fisher’s F distribution.

The standardized Gaussian distribution N(0, 1) was deﬁned in Box 4.5, where it was

also noted that a stochastic variable has the N(m, σ

) distribution if it can be written

m + σZ, where Z ∈ N(0, 1). We then have that Z

∈ (1/2, 1/2), where the (p, a) distri-

bution was deﬁned in Section 5.A.2. We therefore learn a lot about the Gaussian distribution

if we understand the gamma distribution. One simple observation is that if X ∈ (p, a) then

E(X

) = a

−k

(p + k)/(p) for all k>−p, but the most important relation for the gamma

distribution is probably that

(X, Y ) ∈ (p, a) ⊗ (q, a) ⇒



X + Y,



∈ (p + q, a) ⊗ β



(p, q). (6.3)

Here β



(p, q) denotes the β distribution of the second kind and ⊗ means that the components

are independent. This is derived in Box 6.5, where the density of the β



(p, q) distribution is

given and its relation to the β distribution of the ﬁrst kind, introduced in Section 4.6.2, is

clariﬁed. Note, in particular, that X/(X + Y ) ∈ β(p, q).

To ‘take the square-root’ of the β(p, q) distributions when p = 1/2 is of traditional

importance, since this is how the t distribution appears. To take the square root of a dis-

tribution means to ﬁnd the distribution which is symmetric around zero, and for which the

squared variable has the given distribution. That F(x) is the CDF for the square root of a



(1/2,q) distribution means that

2F (x) − 1 = P(X

≤ x

) = B(1/2,q)



−1/2

(1 + t)

−q−1/2

dt,

from which we deduce that the density for X is given by

(q +

)

√

π(q)

(1 + x

)

−q−1/2

Similarly, we can take the square root of the β(1/2,q) distribution, which will give us a

distribution with the density function B(1/2,q)(1 − x

)

q−1

, |x|≤1.

When this is applied to statistics, at least to the part of statistics that is about Gaussian

data, we consider the special cases where a = 1/2 and take p = f/2,q = f



/2 for integers

f and f



. The gamma distributions are then χ

distributions and equation (6.3) means that if

(X, Y ) ∈ χ

(f ) ⊗ χ



) then X + Y is distributed as χ

(f + f



) and is independent of the

ratio X/Y . The distribution for the latter is more commonly referenced as

X/f

Y/f



∈ F(f, f



where F (f, f



) = (f



/f )β



(f/2,f



/2) is called the F distribution. Similarly, if X has the

distribution of the square root of the β



(1/2,f/2) distribution, we consider

√

fX instead,

whose distribution is the t(f ) distribution. Its density is obtained by replacing x

by x

172 EMPIRICAL DISTRIBUTION FUNCTIONS

Box 6.5 Deriving formula (6.3)

It sufﬁces to show the claim when a = 1. If X ∈ (p, 1) and Y ∈ (q, 1) are indepen-

dent, then



X + Y ≤ z,

≤ u



(p)(q)



x+y≤z,x/y≤u

p−1

q−1

−x−y

dxdy.

To evaluate the integral, introduce the new variables s = x + y, t = x/y. The functional

determinant is then given by −s/(1 + t)

and the integrand in the integral in these new

coordinates becomes

p+q−2

p−1

(1 + t)

p+q−2

(1 + t)

= s

p+q−1

−s

p−1

(1 + t)

−p−q

,s,t>0.

This is a product of a function in s and one in t, which means that X + Y and X/ Y are

independent. Furthermore, we see that the function in s is proportional to the density

for the (p + q, 1) distribution, which shows that this is the distribution for X + Y .It

now follows that the distribution for X/Y must have density B(p, q)t

p−1

(1 + t)

−p−q

where B(p, q) = (p + q)/[(p)(q)]. The distribution with this density is called the

beta distribution of the second kind, denoted by β



(p, q).

The relationship to the beta distribution β(p, q) of the ﬁrst kind is that if

X ∈ β



(p, q), then Y = X/(1 + X) ∈ β(p, q). To see this, note that with the substitu-

tion u = t/(1 + t) the density becomes B(p, q)u

p−1

(1 − u)

q−1

(including the factor

|du/dt|=1/(1 + t)

in the expression for the density for X, together with the appropriate modiﬁcation of

the coefﬁcient.

Example 6.5 Assume that X

,...,X

are independent observations of N(0, 1). Then

∈ χ

(1) and S =



−

∈ χ

(n − 1), and these are independent. A direct conse-

quence of the discussion above is the well-known fact that

√

n − 1



∈ t(n − 1)

(the equality means that they have the same distribution), but we also see that

√





√



S + n

has the density function

(

)

√

nπ(

n−1

)



1 −



(n−3)/2

, |x|≤

√

APPENDIX: SOME TECHNICAL COMMENTS 173

6.A.2 The Wiener process and its bridge

One of the key properties of the Gaussian distribution is that the sum of Gaussian variables

itself has a Gaussian distribution. A consequence of this is that if X ∈ N(0, 1), then we can

write X = S

= X

+ ...+ X

, where the X

are independent with the same distribution

∈ N(0, 1/n). If we consider all the cumulative sums {S

}

k=1

simultaneously, we can plot

the different S

versus k/n and connect these points with straight lines to obtain a polygon

starting at the origin and ending up at (1,X). If we view this the opposite way, we have for

each t a stochastic variable x(t) such that when t = k/n,itisgivenbyS

, whereas for other

values of t it is obtained by linear interpolation of such points. For this stochastic process x(t)

we have the following:

•

all its paths are continuous and start at zero;

•

the increments over disjoint time intervals are all independent (because they are derived

from independent X

);

•

x(t) − x(s) ∈ N(0, |t − s|) when t = k/n and s = j/n.

There is a stochastic process which has precisely these properties, including the last one

for all s, t ∈ [0 , 1]. It is called the Wiener process, which we denote by {w(t); t ∈ [0, 1]}.It

plays much the same role for stochastic processes as the standardized Gaussian plays for

stochastic variables.

The Wiener process is named after Norbert Wiener who deﬁned it mathematically in the

late 1920s. Intuitively it represents the cumulative sum of inﬁnitely many, inﬁnitely small and

independent errors. Its physical realization is, however, much older and is named after Robert

Brown who, in 1828, described the apparently random movement of particles suspended in a

medium such as a ﬂuid, something we today call Brownian motion in his honor. Although it

was Wiener who put this process in the context of stochastic processes, it was Albert Einstein

who was the ﬁrst to describe it mathematically. When he described it in the famous Einstein

year 1905 (the same year he published the theory of special relativity and discovered the

photon) he did not know about Brown’s writings. Einstein’s mission was different. He felt

that if there are molecules (which was only a theory at the time) there should be macroscopic

manifestations of their motion, and if these could be observed, this would serve as conﬁrmation

of their existence. This was a problem in mathematical physics, and Einstein showed that under

certain assumptions, reﬂecting what should happen on average in a world full of colliding

particles, the probability density ρ(x, t) of particles should be governed by the diffusion

equation ∂

ρ = D

ρ, where 

is the Laplacian (sum of second-order derivatives) and D

a diffusion coefﬁcient. From this it follows that if the particle starts at the origin, the density

ρ at time t is distributed according to a Gaussian distribution with mean zero and variance

Dt. Einstein then derived an expression for the diffusion coefﬁcient and laid out a program

for ‘proving’ the existence of the atom. This program was later carried out by Jean Baptiste

Perrin, winning him the Nobel Prize in 1926.

One problem with Brownian motion as described was that the paths, as can easily

be guessed from the description, are very irregular. In fact, they are continuous but non-

differentiable at every point. This mathematical puzzle was laid to rest by Wiener (and

others). (Actually non-differentiable functions had been known to mathematicians for quite

some time, since the work of Weierstrass in the late nineteenth century.)

174 EMPIRICAL DISTRIBUTION FUNCTIONS

As a stochastic process, the Wiener process has some important properties, including the

reﬂection principle and the strong Markov property. Without being explicit about what these

are, one important consequence of them is that



max

[0,t]

|w(s)| <x,w(t) ∈ I





h(t, y)dy,

where

h(t, y) =

∞



−∞

(−1)

√

2πt

−(y−2nx)

/2t

A close relative of the Wiener process is the Brownian bridge {w

(t); t ∈ [0, 1]}, which is

essentially a process deﬁned for 0 ≤ t ≤ 1, consisting of those paths of the Wiener process

which end at zero when t = 1. Mathematically it can be obtained from the Wiener process

by applying either of the functions h(x)(t) = x(t) − tx(1) or h(x)(t) = (1 − t)x(t/(1 − t)) on

it, each of which produces another Gaussian process with mean zero, but now the covariance

between times s and t is given by s ∧ t − st. Among its most important properties is that



max

[0,1]

(t)|≤x



= 1 + 2

∞



(−1)

−2n

, (6.4)

which is derived from the formula above by considering a sequence of intervals I decreasing

to zero. Together with a large-sample approximation this observation is the Kolmogorov test

in statistics, as is described in the next section.

6.A.3 Conﬁdence regions for the CDF and the Kolmogorov–Smirnov

test

Let X be a stochastic variable with CDF F (x). We have seen that, for a given x, the CLT

implies that

√

n(F

(x) − F (x)) ∈ AsN(0,F(x)(1 − F(x))).

In order to extend this to pairs of such values, ﬁx x and y such that x<y. Since we must

have that F

(x) ≤ F

(y), these two variables cannot be independent. To compute their co-

variance, note that the bivariate CDF of the pair U = (I(X ≤ x),I(X ≤ y)) is given by

P(X ≤ x, X ≤ y) = F (x ∧ y) (where x ∧ y denotes the smaller of x and y), and since the

marginal distributions are F (x) and F(y), respectively, it follows that the covariance of U is

given by F (x ∧ y) − F (x)F (y), and the covariance of F

(x) and F

(y) is therefore this divided

by n. When x = y this reduces to the variance formula for F

(x). It is now a consequence of

the multivariate CLT that the bivariate distribution of

(

√

n(F

(x) − F (x)),

√

n(F

(y) − F(y)))

APPENDIX: SOME TECHNICAL COMMENTS 175

is asymptotically a Gaussian bivariate distribution with zero mean and a covariance matrix

given by

 =



F (x)(1 − F (x)) F(x ∧ y) − F (x)F (y)

F (x ∧ y) − F(x)F (y) F(y)(1 − F (y))



. (6.5)

We can extend this to any vector of values of the e-CDF at different points. But this means

that the stochastic process {

√

n(F

(x) − F (x))} has the same ﬁnite-dimensional distributions

as the process {w

(F (x))}, where {w

(t); t ∈ [0, 1]} is the Brownian bridge deﬁned in the

previous section. Using this, it is not a far-fetched guess that we have that

{

√

n(F

(x) − F (x))}→{w

(F (x))} in distribution,

a fact that can be veriﬁed in different ways. One way uses the CLT for martingales in continuous

time and is described in Appendix 4.A. The observation has many useful consequences,

derived from the general fact that we then also have that

h({

√

n(F

(x) − F (x))}) → h({w

(F (x))}) in distribution,

for any function h which is a continuous function on continuous functions over the interval

[0, 1]. An example of such a function is h(x) = max

[0,1]

|x(t)|, so we have that

max

√

n|F

(x) − F (x)|→max

(F (x))| in distribution.

If F (x) is continuous, the right-hand side is max

[0,1]

(t)|, the distribution of which was

given in equation (6.4). It follows that



√

n max

(y) − F(y)|≤x



≈ 1 + 2

∞



k=1

(−1)

−2k

,x>0, (6.6)

for large n. The immediate effect of this expression is that it provides us with a statistical

test, called the Kolmogorov one-sample test, for the hypothesis that the distribution we have

sampled from has CDF F (x). It also provides us with simultaneous conﬁdence intervals for

the function F (x), based on the function F

(x). But these intervals are of ﬁxed width and do

therefore not generalize the conﬁdence intervals for F (x) we discussed Section 6.5, since the

width of those was proportional to

√

F (x)(1 − F (x)). To obtain a corresponding simultaneous

region, just note that



√

n(F

(x) − F (x))

√

F (x)(1 − F (x))



→



(F (x))

√

F (x)(1 − F (x))



in distribution.

For a continuous distribution F(x) we can therefore compute the distribution of

max

(t)/

√

t(1 − t)| and determine its percentiles in order to get a simultaneous conﬁ-

dence region proportional to the pointwise intervals. We do not pursue this any further.

176 EMPIRICAL DISTRIBUTION FUNCTIONS

We can also derive the two-sample Kolmogorov–Smirnov test from this. One way uses

the observation that



n + m

(x) − F (x) − (G

(x) − G(x))) =



n + m

(x) −



n + m

(x),

where Y

(x) and Z

(x) are independent processes with Brownian bridge limits. From this we

get a Gaussian process that has the same mean and covariance structure as the Brownian bridge

when F (x) = G(x). The full Kolmogorov–Smirnov test for the equality of two distributions

therefore follows from the corresponding Kolmogorov one-sample test.

Correlation and regression

in bivariate distributions

7.1 Introduction

In the previous chapter we discussed the CDF of a single stochastic variable and how we

estimate it from data – this as a generalization of the problem of estimating a single binomial

parameter. The next step is to generalize the analysis of a 2 × 2 table to general data. As

we have seen, there are two closely related but (philosophically) different ways to view a

2 ×2 table, both of which describe the relation between an exposure and a response. We can

either see it as a problem of association, where we consider both the exposure and response

as characteristics of members of the population and wish to understand how they are related,

or we can see it as an estimation problem, where we wish to assess how much the response

changes because of the exposure. We will generalize both these viewpoints to more general

situations, starting with association in this chapter. The description in terms of effect sizes is

postponed to the next chapter.

The association aspect is a two-dimensional problem, whereas the effect size aspect is

about comparing two one-dimensional CDFs. Describing association is about describing the

pair (exposure, response), while the effect size is about investigating the response conditional

on the exposure status. The former therefore requires the mathematics of functions of two

variables, whereas the latter is concerned with functions of one variable only. So association is

more technically involved, but we should still start with it, because this discussion will allow

us to take into account baseline information when we compare groups in the next chapter.

There are essentially three parts to what we wish to do in this chapter. The ﬁrst is to discuss

how we describe distributions of two variables (i.e., bivariate distributions), with emphasis

on the dependence of the two components on each other, expressed as correlation. We also

discuss how we can use correlation to our advantage in the design of experimental studies.

This part introduces the important bivariate Gaussian distribution, which is uniquely deﬁned

by adding a correlation coefﬁcient to the list of parameters describing Gaussian distributions.

Understanding Biostatistics, First Edition. Anders K¨all´en.

178 CORRELATION AND REGRESSION IN BIVARIATE DISTRIBUTIONS

Having introduced this distribution we turn to the all-important concept of regression to

the mean, its meaning and consequences. The term refers to a mathematical property of the

bivariate Gaussian distribution, which is of great importance in understanding the relationship

between two consecutive measurements. Finally, we address the problem of how we use data

to obtain knowledge about the two means in a bivariate distribution, which is an extension of

the univariate t-test to bivariate data, and we will see how these ideas will allow us to analyze

functions of binomial parameters in a way that takes all uncertainty into account.

7.2 Bivariate distributions and correlation

A bivariate stochastic variable (X, Y ) is a pair of stochastic variables (possibly two test statis-

tics) and is described by a two-dimensional CDF

F (x, y) = P (X ≤ x, Y ≤ y).

Similar to the univariate CDF, this is a function that starts at zero far in the lower left ‘corner’

of the xy-plane and increases to one as we move upwards toward the upper right ‘corner’. If

we denote the CDFs for the components X and Y by F

(x) and F

(y), respectively, these can

be obtained as the marginal CDFs

(x) = F (x, ∞),F

(y) = F (∞,y).

For obvious reasons it may sometimes be convenient to denote these marginal distributions

by F

(x) and F

(y), respectively.

(x) −F(x, y) F

(x, y)

F (x, y) F

(y) − F (x, y)

(x, y) = 1 −F

(x) −F

(y) + F (x, y)

There is a relationship between bivariate distribu-

tions and 2 × 2 tables in that if we dichotomize the two

variables X and Y according to cut-off points x and y,

respectively, the proportions in each cell are as shown

in the table to the right. (To dichotomize perfectly valid

measurement data in this way is common in medical

statistics.) Marginal probabilities can be obtained by

choosing cut-off points for the two marginal distributions separately, but in order to get a ﬁt

to the complete table we need to determine the association between the variables.

If X and Y are independent stochastic variables we have that F (x, y) = F

(x)F

(y), so

when this is not the case, the two variables X and Y are dependent in some way. We wish to

describe such dependence in a simple way, in a single parameter, in the same way as location

is described by the mean and dispersion is described by the variance. The simplest parameter

that describes dependence is called the covariance, and is deﬁned as

C(X, Y ) =



∞

−∞



∞

−∞

(x − m

)(y − m

)dF (x, y),

where m

and m

are the means of X and Y, respectively. It is estimated by the

sample covariance

n − 1



− x )(y

− y ).

BIVARIATE DISTRIBUTIONS AND CORRELATION 179

Box 7.1 Spearman correlation, concordant pairs and Kendall’s tau

There are alternatives to the Pearson correlation coefﬁcient to describe the association

between two stochastic variables X and Y . One is the Pearson correlation coefﬁcient for

the pair (F

(X),F

(Y)) instead of (X, Y ). The marginal distributions for this variable are

uniformly distributed on (0, 1), so this coefﬁcient, known as the Spearman correlation,

is obtained as

= 3





uv dU(u, v) − 1



where U(u, v) is the CDF of (F

(X),F

(Y)). To estimate it we use e-CDFs, which is

equivalent to computing the Pearson sample correlation of the rank-transformed data.

Another way to describe association is obtained if we rewrite the covariance param-

eter as

C(X, Y ) =



− x

)(y

− y

)dF (x

Two pairs (x

) and (x

) are said to be concordant if (x

− x

)(y

− y

) > 0 and

discordant if the reverse inequality holds true. We can measure correlation by using any

function of (x

− x

)(y

− y

). A particular case is the sign function, which leads to

the parameter

τ = P((X

− X

)(Y

− Y

) > 0) − P((X

− X

)(Y

− Y

) < 0),

which is called Kendall’s tau. For a continuous variable this is τ = 4p

− 1, where

= E





I(X>x,Y>y)dF (x, y)





(x, y)dF (x, y).

The ratio of the two probabilities above, concordant to discordant pairs, deﬁnes a quan-

tity that reduces to the odds ratio in a 2 × 2 table, and is therefore a kind of generalized

odds ratio.

−1

−0.5

0.5

−1 10

Spearman ρs

Kendall’s τ

The analytical expressions for these corre-

lation coefﬁcients can be found for a bivariate

Gaussian distribution:

arcsin(ρ/2),τ=

arcsin(ρ).

The graph to the right shows these functions as

functions of ρ in gray and black, respectively.

We have that ρ

≈ ρ to a very high precision,

which means that if estimates of Pearson’s and

Spearman’s correlation coefﬁcients differ sub-

stantially, the data are probably not from a bi-

variate Gaussian distribution. The magnitude of Kendall’s τ must, however, be inter-

preted in a different way than how ρ is interpreted.