Devore J.L., Berk K.N. Modern Mathematical Statistics with Applications

Подождите немного. Документ загружается.

variance that is nearly as small as can be achieved by any unbiased estimator.

Stated another way, the mle

y is approximately the MVUE of y.

Because of this result and the fact that calculus-based techniques can usually be

used to derive the mle’s (although often numerical methods, such as Newton’s

method, are necessary), maximum likelihood estimation is the most widely used

estimation technique among statisticians. Many of the estimators used in the

remainder of the book are mle’s. Obtaining an mle, however, does require that

the underlying distribution be specified.

Note that there is no similar result for method of moments estimators. In general,

if there is a choice between maximum likelihood and moment estimators, the mle is

preferable. For example, the maximum likelihood method applied to estimating

gamma distribution parameters tends to give better estimates (closer to the parameter

values) than does the method of moments, so the extra computation is worth the price.

Some Complications

Sometimes calculus cannot be used to obtain mle’s.

Example 7.23 Suppose the waiting time for a bus is uniformly distributed on [0, y] and the results

,...,x

of a random sample from this distribution have been observed. Since

f(x; y) ¼ 1/y for 0  x  y and 0 otherwise,

f ðx

; ...; x

; yÞ¼

1=y

0  x

 y; ...; 0  x

 y

0 otherwise



As long as max(x

)  y , the likelihood is 1/y

, which is positive, but as soon as

y < max(x

), the likelihood drops to 0. This is illustrated in Figure 7.7.Calculuswill

not work because the maximum of the likelihood occurs at a point of discontinuity,

but the figure shows that

y ¼ max x

ðÞ. Thus if my waiting times are 2.3, 3.7, 1.5, .4,

and 3.2, then the mle is

y ¼ 3:7. Note that the mle is biased (see Example 7.5).

Example 7.24 A method that is often used to estimate the size of a wildlife population involves

performing a capture/recapture experiment. In this exper iment, an initial sample of

M animals is captured, each of these animals is tagged, and the animals are then

returned to the population. After allowing enough time for the tagged individuals to

mix into the population, another sample of size n is captured. With X ¼ the number

of tagged animals in the second sample, the objective is to use the observed x to

estimate the population size N.

Likelihood

max(x

)

Figure 7.7 The likelihood function for Example 7.23 ■

358

CHAPTER 7 Point Estimation

The parameter of interest is y ¼ N, which can assume only integer values, so

even after determining the likelihood function (pmf of X here), using calculus to

obtain N would present difficulties. If we think of a success as a previously tagged

animal being recaptured, then sampling is without replacement from a population

containing M successes and N – M failures, so that X is a hypergeometric rv and the

likelihood function is

pðx; NÞ¼hðx; n; M; NÞ¼





N  M

n  x



The integer-valued natu re of N notwithstanding, it would be difficult to take the

derivative of p(x; N). However, let’s consider the ratio of p(x; N)top(x; N – 1):

pðx; NÞ

pðx; N  1Þ

ðN  MÞðN  nÞ

NðN  M  n þ xÞ

This ratio is larger than 1 if and only if (iff) N < Mn/x. The value of N for which

p(x; N) is maximized is therefore the largest integer less than Mn/x. If we use

standard mathematical notation [r] for the largest integer less than or equal to r,

the mle of N is

N ¼½Mn=x. As an illustration, if M ¼ 200 fish are taken from a

lake and tagged, subsequently n ¼ 100 fish are recaptured, and among the 100

there are x ¼ 11 tagged fish, then

N ¼½ð200Þð100Þ=11¼½1818:18¼1818. The

estimate is actually rather intuitive; x/n is the proportion of the recaptured sample

that is tagged, whereas M/N is the proportion of the entire popul ation that is tagged.

The estimate is obtained by equating these two propor tions (estimating a population

proportion by a sample proportion).

■

Suppose X

, X

,...,X

is a random sample from a pdf f(x; y) that is

symmetric about y, but the investigator is unsure of the form of the f function. It

is then desirable to use an estimator

y that is robust, that is, one that performs well

for a wide variety of underlying pdf’s. One such estimator is a trimmed mean. In

recent years, statisticians have proposed another type of estimator, called an M-

estimator, based on a generalization of maximum likelihood estimation. Instead of

maximizing the log likelihood Sln[f(x; y)] for a specified f, one seeks to max imize

Sr(x

; y). The “objective function” r is selected to yield an estimator with good

robustness properties. The book by David Hoaglin et al. (see the bibliography)

contains a good exposition on this subject.

Exercises Section 7.2 (21–31)

21. A random sample of n bike helmets manufactured

by a company is selected. Let X ¼ the number

among the n that are flawed, and let p ¼ P

(flawed). Assume that only X is observed, rather

than the sequence of S’s and F’s.

a. Derive the maximum likelihood estimator of p.

If n ¼ 20 and x ¼ 3, what is the estimate?

b. Is the estimator of part (a) unbiased?

c. If n ¼ 20 and x ¼ 3, what is the mle of the

probability (1 –p)

that none of the next five

helmets examined is flawed?

22. Let X have a Weibull distribution with parameters

a and b,so

EðXÞ¼b  Gð1 þ 1=aÞ

VðXÞ¼b

fGð1 þ 2=aÞ½Gð1 þ 1=aÞ

7.2 Methods of Point Estimation 359

a. Based on a random sample X

,...,X

, write

equations for the method of moments estimators

of b and a. Show that, once the estimate of a has

been obtained, the estimate of b can be found

from a table of the gamma function and that the

estimate of a is the solution to a complicated

equation involving the gamma function.

b. If n ¼ 20,



x ¼ 28:0, and

¼ 16; 500;

compute the estimates. [Hint:[G(1.2)]

/G(1.4)

¼ .95.]

23. Let X denote the proportion of allotted time that a

randomly selected student spends working on a

certain aptitude test. Suppose the pdf of X is

f ðx; yÞ¼

ðy þ 1Þx

0  x  1

0 otherwise



where 1 < y. A random sample of ten students

yields data x

¼ .92, x

¼ .79, x

¼ .90,

¼ .65, x

¼ .86, x

¼ .47, x

¼ .73, x

¼ .97,

¼ .94, x

¼ .77.

a. Use the method of moments to obtain an esti-

mator of y, and then compute the estimate for

this data.

b. Obtain the maximum likelihood estimator of y,

and then compute the estimate for the given

data.

24. Two different computer systems are monitored for

atotalofn weeks. Let X

denote the number of

breakdowns of the first system during the ith week,

and suppose the X

’s are independent and drawn

from a Poisson distribution with parameter l

. Sim-

ilarly, let Y

denote the number of breakdowns of

thesecondsystemduringtheith week, and assume

independence with each Y

Poisson with parameter

. Derive the mle’s of l

, l

,andl

– l

.[Hint:

Using independence, write the joint pmf (likeli-

hood)oftheX

’s and Y

’s together.]

25. Refer to Exercise 21. Instead of selecting n ¼ 20

helmets to examine, suppose we examine helmets

in succession until we have found r ¼ 3 flawed

ones. If the 20th helmet is the third flawed one (so

that the number of helmets examined that were not

flawed is x ¼ 17), what is the mle of p? Is this the

same as the estimate in Exercise 21? Why or why

not? Is it the same as the estimate computed from

the unbiased estimator of Exercise 17?

26. Six Pepperidge Farm bagels were weighed, yield-

ing the following data (grams):

117.6 109.5 111.6 109.2 119.1 110.8

(Note:4oz¼ 113.4 g)

a. Assuming that the six bagels are a random

sample and the weight is normally distributed,

estimate the true average weight and standard

deviation of the weight using maximum likeli-

hood.

b. Again assuming a normal distribution, estimate

the weight below which 95% of all bagels will

have their weights. [Hint: What is the 95th

percentile in terms of m and s? Now use the

invariance principle.]

c. Suppose we choose another bagel and weigh it.

Let X ¼ weight of the bagel. Use the given

data to obtain the mle of P(X  113.4). (Hint:

P(X  113.4) ¼ F[(113.4 – m)/s)].)

27. Suppose a measurement is made on some physical

characteristic whose value is known, and let X

denote the resulting measurement error. For an

unbiased measuring instrument or technique, the

mean value of X is 0. Assume that any particular

measurement error is normally distributed with

variance s

. Let X

,...X

be a random sample

of measurement errors.

a. Obtain the method of moments estimator of s

b. Obtain the maximum likelihood estimator

of s

28. Let X

,...,X

be a random sample from a gamma

distribution with parameters a and b.

a. Derive the equations whose solution yields the

maximum likelihood estimators of a and b.Do

you think they can be solved explicitly?

b. Show that the mle of m ¼ ab is

m ¼

29. Let X

, X

,...,X

represent a random sample from

the Rayleigh distribution with density function

given in Exercise 15. Determine

a. The maximum likelihood estimator of y and

then calculate the estimate for the vibratory

stress data given in that exercise. Is this estima-

tor the same as the unbiased estimator sug-

gested in Exercise 15?

b. The mle of the median of the vibratory stress

distribution. [Hint: First express the median in

terms of y.]

30. Consider a random sample X

, X

,...,X

from the

shifted exponential pdf

f ðx; l; yÞ¼

lðxyÞ

x  y

0 otherwise



Taking y ¼ 0 gives the pdf of the exponential

distribution considered previously (with positive

density to the right of zero). An example of the

360

CHAPTER 7 Point Estimation

shifted exponential distribution appeared in

Example 4.5, in which the variable of interest

was time headway in traffic flow and y ¼ .5 was

the minimum possible time headway.

a. Obtain the maximum likelihood estimators of

y and l.

b. If n ¼ 10 time headway observations are

made, resulting in the values 3.11, .64, 2.55,

2.20, 5.44, 3.42, 10.39, 8.93, 17.82, and 1.30,

calculate the estimates of y and l.

31. At time t ¼ 0, 20 identical components are

put on test. The lifetime distribution of each is

exponential with parameter l. The experimenter

then leaves the test facility unmonitored. On

his return 24 h later, the experimenter immedi-

ately terminates the test after noticing that

y ¼ 15 of the 20 components are still in operation

(so 5 have failed). Derive the mle of l.

[Hint: Let Y ¼ the number that survive 24 h.

Then Y~Bin(n, p). What is the mle of p?

Now notice that p ¼ P(X

 24), where X

exponentially distributed. This relates l to p,so

the former can be estimated once the latter

has been.]

7.3

Sufficiency

An investigator who wishes to make an inference about some parameter y will base

conclusions on the value of one or more statistics – the sample mean

X, the sample

variance S

, the sample range Y

 Y

, and so on. Intuitively, some statistics will

contain more information about y than will others. Sufficiency, the topic of this

section, will help us decide which functions of the data are most informative for

making inferences.

As a first point, we note that a statistic T ¼ t(X

,...,X

) will not be useful for

drawing conclusions about y unless the distribution of T depends on y. Consider, for

example, a random sample of size n ¼ 2 from a normal distribution with mean m

and variance s

, and let T ¼ X

 X

.ThenT has a normal distribution with mean

0 and variance 2s

, which does not depend on m. Thus this statistic cannot be used

as a basis for drawing any conclusions about m, although it certainly does carry

information about the variance s

The relevance of this obser vation to sufficiency is as follows. Sup pose an

investigator is given the value of some statistic T, and then examines the condi-

tional distribution of the sample X

, X

,...,X

given the value of the statistic – for

example, the conditional distribution given that

X ¼ 28:7. If this conditional

distribution does not depend upon y, then it can be concluded that there is no

additional information about y in the data over and above what is provided by T.

In this sense, for purposes of making inferences about y,itissufficient to know

the value of T, which contains all the information in the data relevant to y.

Example 7.25 An investigation of major defects on new vehicles of a certain type involved

selecting a random sample of n ¼ 3 vehicles and determining for each one the

value of X ¼ the number of major defects. This resulted in observations x

¼ 1,

¼ 0, and x

¼ 3. You, as a consulting statistician, have been provided with a

description of the experiment, from which it is reasonable to assume that X has a

Poisson distribution, and told only that the total number of defects for the three

sampled vehi cles was four.

Knowing that T ¼ ∑X

¼ 4, would there be any additional advantage in

having the observed values of the individual X

’s when making an inference

about the Poisson parameter l? Or rather is it the case that the statistic T contains

all relevant information about l in the data? To address this issue, consider the

conditional distribution of X

, X

given that ∑ X

¼ 4. First of all, there are only

7.3 Sufﬁciency 361

a few possibl e (x

, x

) triples for which x

+ x

¼ 4. For example, (0, 4, 0)

is a possibility, as are (2, 2, 0) and (1, 0, 3), but not (1, 2, 3) or (5, 0, 2). That is,

PðX

¼ x

; X

¼ x

; X

¼ x

i¼1

¼ 4Þ¼0 unless x

þ x

¼ 4

Now consider the triple (2, 1, 1), which is consistent with ∑X

¼ 4. If we let

A denote the event that X

¼ 2, X

¼ 1, and X

¼ 1 and B denote the event that

∑X

¼ 4, then the event A implies the event B (i.e., A is contained in B), so the

intersection of the two events is just the smaller event A.Thus

PðX

¼ 2; X

¼ 1; X

¼ 1j

i¼1

¼ 4Þ¼PðAjBÞ¼

PðA \ BÞ

PðBÞ

PðX

¼ 2; X

¼ 1; X

¼ 1Þ

P SX

¼ 4ðÞ

A moment generating function argument shows that ∑X

has a Poisson distribution

with parameter 3l. Thus the desired conditional probability is

l

 l



l

 l



l

 l

3l

 3lðÞ

 2!

Similarly,

PðX

¼ 1; X

¼ 0; X

¼ 3j

i¼1

¼ 4Þ¼

 3!

The complete conditional distribution is as follows:

PðX

¼ x

; X

¼ x

; X

¼ x

i¼1

¼ 4Þ

ðx

; x

Þ¼ð2; 2; 0Þ; ð2; 0; 2 Þ ; ð0; 2; 2Þ

ðx

; x

Þ¼ð2; 1; 1Þ; ð1; 2; 1 Þ ; ð1; 1; 2Þ

ðx

; x

Þ¼ð4; 0; 0Þ; ð0; 4; 0 Þ ; ð0; 0; 4Þ

ðx

; x

Þ¼ð3; 1; 0Þ; ð1; 3; 0 Þ ; ð3; 0; 1Þ; ð1; 0; 3Þ; ð0; 1; 3Þ; ð0; 3; 1Þ

This condition al distribution does not involve l. Thus once the value of the statistic

∑X

has been provided, there is no additional information about l in the individual

observations.

To put this another way, think of obtaining the data from the experiment in

two stages:

1. Observe the value of T ¼ X

+ X

from a Poisson distribution with

parameter 3l.

2. Having observed T ¼ 4, now obtain the individual x

’s from the conditional

distribution

362 CHAPTER 7 Point Estimation

PðX

¼ x

; X

¼ x

; X

¼ x

i¼1

¼ 4Þ

Since the conditional distribution in step 2 does not involve l, there is no addi tional

information about l resulting from the secon d sta ge of the data generation process.

This argument holds more generally for any sample size n and any value t other

than 4 (e.g., the total number of defects among ten randomly selected vehicles

might be ∑X

¼ 16). Once the value of ∑X

is known, there is no further informa-

tion in the data about the Poisson parameter.

■

DEFINITION

A statistic T ¼ t(X

,...,X

) is said to be sufficient for making inferences

about a parameter y if the joint distribution of X

, X

,...,X

given that T ¼ t

does not depend upon y for every possible value t of the statistic T.

The notion of sufficiency formalizes the idea that a statistic T contains all relevant

information about y. Onc e the value of T for the given data is available, it is of no

benefit to know anything else about the sample.

The Factorization Theorem

How can a sufficient statistic be identified? It may seem as though one would have

to select a statistic, determine the conditional distribution of the X

’s given any

particular value of the statistic, and keep doing this until hitting paydirt by finding

one that satisfies the defining condition. This would be terribly time-consuming,

and when the X

’s are continuous there are additional technical difficulties in

obtaining the relevant conditional distribution. Fortunately, the next result provides

a relatively straightforward way of proceeding.

THE NEYMA N

FACTORI-

ZATION

THEOREM

Let f(x

, x

,...,x

; y) denote the joint pmf or pdf of X

, X

,...,X

. Then

T ¼ t(X

,...,X

) is a sufficient statistic for y if and only if the joint pmf or

pdf can be represented as a product of two factors in which the first factor

involves y and the data only thr ough t(x

,...,x

) whereas the second factor

involves x

,...,x

but does not depend on y:

; x

; :::; x

; yðÞ¼gtx

; :::; x

ðÞ; yðÞhx

; :::; x

ðÞ

Before sketching a proof of this theorem, we consider several examp les.

Example 7.26 Let’s generalize the previous example by considering a random sample X

, X

,...,

from a Poisson distribution with parameter l, for example, the numbers of

blemishes on n independently selected DVD’s or the numbers of errors in n batches

of invoices where each batch consists of 200 invoices. The joint pmf of these

variables is

7.3 Sufﬁciency 363

f ð x

; ...; x

; lÞ¼

l



l



l

nl

 l

þx

þþx

!  x

! x

¼ e

nl

 l



!  x

! x



The factor inside the first set of parentheses involves the parameter l and the

data only through ∑x

, whereas the factor inside the second set of parentheses

involves the data but not l. So we have the desired factorization, and the sufficient

statistic is T ¼ ∑X

as we previously ascertained directly from the defini tion of

sufficiency.

■

A sufficient statistic is not unique; any one-to-one function of a sufficient

statistic is itself sufficient. In the Poisson example, the sample mean

X ¼ð1=nÞ

is a one-to-one function of ∑X

(knowing the value of the sum

of the n observations is equivalent to knowing their mean), so the sample mean is

also a sufficient statistic.

Example 7.27 Suppose that the waiting time for a bus on a weekday morning is uniformly

distributed on the interval from 0 to y, and consider a random sample X

,...,X

of waiting times (i.e., times on n independe ntly selected mornings). The joint pdf of

these times is

f ð x

; ...; x

; yÞ¼





0  x

 y; ...; 0  x

 y

0 otherwise

To obtain the desired factorization, we introduce notation for an indicator function

of an event A: I(A) ¼ 1if(x

, x

,...,x

) lies in A and I(A) ¼ 0 otherwise. Now let

A ¼ x

; x

; :::; x

ðÞ: 0  x

 y; 0  x

 y; :::; 0  x

 y

That is, A is the indicator for the event that all x

’s are between 0 and y. But all n of

the x

’s will be between 0 and y if and only if the smallest of the x

’s is at least 0 and

the largest is at most y. Thus

IðAÞ¼I 0  minfx

; :::; x

gðÞI max x

; :::; x

 ygðÞ

We can now use this indicator function notation to write a one-line expression for

the joint pdf:

f ð x

; x

; ...; x

; yÞ¼

 Iðmaxfx

; ...; x

gyÞ



 Ið0  minfx

; ...; x

gÞ

The factor inside the square brackets involves y and the x

’s only through the

function t(x

,...,x

) ¼ max{x

,...,x

}. Voila, we have our desired factorization,

and the sufficient statistic for the uniform parameter y is T ¼ max{X

,...,X

}, the

largest order statistic. All the information about y in this uniform random sample is

contained in the largest of the n observations. This result is much more difficult to

obtain directly from the definition of sufficiency.

■

Proof of the Factorization Theorem A general proof when the X

’s

constitute a random sample from a continuous distribution is fraught with technical

details that are beyond the level of our text. So we content ourselves with a proof in

364 CHAPTER 7 Point Estimation

the discrete case. For the sake of concise notation, denote X

, X

,...,X

by X and

, x

,...,x

by x.

Suppose first that T ¼ t(x) is sufficient, so that P(X ¼ x | T ¼ t) does not

depend upon y. Focus on a value t for which t(x) ¼ t (e.g., x ¼ 3, 0, 1, t(x) ¼ ∑x

so t ¼ 4). The event that X ¼ x is then identical to the event that both X ¼ x and

T ¼ t because the former equality implies the latter one. Thus

f x; yðÞ¼P X; yðÞ¼P X ¼ x; T ¼ t; yðÞ

¼ P X ¼ x T ¼ t; y

ðÞPT¼ t; yðÞ¼P X ¼ x T ¼ t

ðÞPT¼ t; yðÞ

Since the first factor in this latter product does not involve y and the second one

involves the data only through t, we have our desired factorization.

Now let’s go the other way: assume a factorization, and show that T is

sufficient, i.e., that the conditional probability that X ¼ x given that T ¼ t does

not involve y.

PðX ¼xjT ¼t

;yÞ¼

PðX ¼x;T ¼t;yÞ

PðT ¼t;yÞ

PðX ¼x;yÞ

PðT ¼t;yÞ

gðt;yÞhðxÞ

u:tðuÞ¼t

PðX ¼u;yÞ

gðt;yÞhðxÞ

u:tðuÞ¼t

g½tðu;yÞhðuÞ

hðxÞ

u:tðuÞ¼t

hðuÞ

Sure enough, this latter ratio does not involve y. ■

Jointly Sufﬁcient Statistics

When the joint pmf or pdf of the data involves a single unknown parameter y, there is

frequently a single statistic (single function of the data) that is sufficient. However,

when there are several unknown parameters—for example, the mean m and standard

deviation s of a normal distribution, or the shape parameter a and scale parameter b

of a gamma distribution—we must expand our notion of sufficiency.

DEFINITION

Suppose the joint pmf or pdf of the data involves k unknown parameters y

,...,y

. The m statistics T

¼ t

,...,X

), T

¼ t

,...,X

),...,

¼ t

,...,X

) are said to be jointly sufficient for the parameters if the

conditional distribution of the X

’s given that T

¼ t

, T

¼ t

,...,T

¼ t

does not depend on any of the unknown parameters, and this is true for all

possible values t

, t

,...,t

of the statistics.

Example 7.28 Consider a random sample of size n ¼ 3 from a continuous distribution, and let T

, T

and T

be the three order statistics – that is, T

¼ the smallest of the three X

’s, T

¼ the

second smallest X

,andT

¼ the largest X

(these order statistics were previously

denoted by Y

, Y

,andY

.). Then for any values t

, t

,andt

satisfying t

< t

7.3 Sufﬁciency 365

¼ x

; X

¼ x

; X

¼ x

¼ t

; T

¼ t

; T

¼ t

ðÞ

; x

¼ t

; t

0 otherwise

For example, if the three ordered values are 21.4, 23.8, and 26.0, then the condi-

tional probability distribution of the three X

’s places probability

on each of the 6

permutations of these three numbers (23.8, 21.4, 26.0, and so on). This conditional

distribution clearly does not involve any unknown parameters.

Generalizing this argument to a sample of size n, we see that for a random

sample from a continuous distribution, the order statistics are jointly sufficient for

, y

,...,y

regardless of whether k ¼ 1 (e.g., the exponential distributio n has a

single parameter) or 2 (the normal distribution) or even k > 2.

■

The factorization theorem extends to the case of jointly sufficient statistics:

, T

,...,T

are jointly sufficient for y

, y

,...,y

if and only if the joint pmf or

pdf of the X

’s can be represented as a product of two factors, where the first

involves the y

’s and the data only through t

, t

,...,t

and the second does not

involve the y

’s.

Example 7.29 Let X

,...,X

be a random sample from a normal distribution with mean m and

variance s

. The joint pdf is

f ð x

; ...; x

; m; s

Þ¼

i¼1

ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

2ps

ðx

mÞ

=ð2s

 e

 Sx

2mSx

þnm

ðÞ

=ð2s







n=2

This factor ization shows that the two statistics SX

and SX

are jointly sufficient for

the two parameters m and s

. Since SðX

 XÞ

¼ SX

 n XðÞ

there is a one-to-

one correspondence between the two sufficient statistics and the statistics

and SðX

 XÞ

; that is, values of the two original sufficient statist ics uniquely

determine values of the latter two statistics, and vice-versa. This implies that the

latter two statistics are also jointly sufficient, which in turn implies that the sample

mean and sample variance (or sample standard deviation) are jointly sufficient

statistics. The sample mean and sample variance encapsulate all the information

about m and s

that is contained in the sample data. ■

Minimal Sufﬁciency

When X

,...,X

constitute a random sample from a normal distribution, the n order

statistics Y

,...,Y

are jointly sufficient for m and s

, and the sample mean and

sample variance are also jointly sufficient. Both the order statistics and the pair

X; S

Þ reduce the data without any information loss, but the sample mean and

variance represent a greater reduction. In general, we would like the greatest

possible reduction without information loss. A minimal (possibly jointly) suffi-

cient statistic is a function of every other sufficient statistic. That is, given the

value(s) of any other sufficient statistic(s), the value(s) of the minimal sufficient

statistic(s) can be calculated. The minimal suffic ient statistic is the sufficient

366 CHAPTER 7 Point Estimation

statistic having the smallest dimensionality, and thus represents the greatest possi-

ble reduction of the data without any information loss.

A general discussion of minimal sufficiency is beyond the scope of our text.

In the case of a normal distribution with values of both m and s

unknown, it can be

shown that the sample mean and sample variance are jointly minimal sufficient (so

the same is true of

and

). It is intuitively reasonable that because there

are two unknown parame ters, there should be a pair of sufficient statistics. It is

indeed often the case that the number of the (jointly) sufficient statistic(s) matches

the number of unknown parameters. But this is not always true. Consider a random

sample X

,...,X

from the pdf f(x ;y) ¼ 1/{p[1 + (x  y)]

} for 1< x < 1,

i.e., from a Cauchy distribution with location parameter y. The graph of this pdf

is bell shaped and centered at y, but its tails decrease much more slowly than those

of a normal density curve. Because the Cauchy distribution is continuous, the order

statistics are jointly sufficient for y. It would seem, though, that a single sufficient

statistic (one-dimensional) could be found for the single parameter. Unfortunat ely

this is not the case; it can be show n that the order statistics are minimal sufficient!

So going beyond the order statistics to any single function of the X

’s as a point

estimator of y entails a loss of information from the original data.

Improving an Estimator

Because a sufficient statistic cont ains all the information the data has to offer

about the value of y, it is reasonable that an estim ator of y or any function of y

should depend on the data only through the sufficient statistic. A general result

due to Rao and Blackwell shows how to start with an unbiased statistic that is

not a function of sufficient statistics and create an improved estimator that is

sufficient.

THEOREM

Suppose that the joint distribution of X

,...,X

depends on some

unknown parameter y and that T is sufficient for y. Consider estimating

h(y), a specified function of y.IfU is an unbiased statistic for estimating

h(y) that does not involve T, then the estimator U* ¼ E(U | T) is also

unbiased for h(y) and has variance no greater than the original unbiased

estimator U.

Proof First of all, we must show that U* is indeed an estimator—that it is a

function of the X

’s which does not depend on y. This follows because, given that T

is sufficient, the distribution of U conditional on T does not involve y, so the

expected value calculated from the conditional distribution will of course not

involve y. The fact that U* has smaller variance than U is a consequence of a

conditional expectation-conditional variance formula for V(U) introduced in Sec-

tion 5.3:

VðUÞ¼VEUjTðÞ½þEV UjTðÞ½¼VU



ðÞþEV UjTðÞ½

Because V(U | T), being a variance, is positive, it follows that V(U)  V(U*) as

desired. ■

7.3 Sufﬁciency 367