Wooldridge J. Introductory Econometrics: A Modern Approach (Basic Text

Подождите немного. Документ загружается.

APPENDIX 3A

3A.1 Derivation of the First Order Conditions in Equation (3.13)

The analysis is very similar to the simple regression case. We must characterize the solu-

tions to the problem



i1

 b



…

 b

)

Taking the partial derivatives with respect to each of the b

(see Appendix A), evaluating

them at the solutions, and setting them equal to zero gives

2



i1











…





)  0

2



i1











…





)  0, for all j  1

,…,

Canceling the 2 gives the first order conditions in (3.13).

3A.2 Derivation of Equation (3.22)

To derive (3.22), write x

in terms of its fitted value and its residual from the regression

of x

on x

,…,x

: x

 xˆ

 rˆ

,for all i  1, …, n. Now, plug this into the second equa-

tion in (3.13):



i1

(

xˆ

 rˆ

)(y









 … 



)  0.

(3.60)

By the definition of the OLS residual uˆ

, since xˆ

is just a linear function of the explana-

tory variables x

,…,x

, it follows that



i1

xˆ

uˆ

 0. Therefore, equation (3.60) can be

expressed as



i1

rˆ









 … 



)  0.

(3.61)

Since the rˆ

are the residuals from regressing x

on x

,…,x



i1

rˆ

 0, for all j 

2, …, k. Therefore, (3.61) is equivalent to



i1

rˆ





)  0. Finally, we use the

fact that



i1

xˆ

rˆ

 0, which means that



solves



i1

rˆ





rˆ

)  0.

Now, straightforward algebra gives (3.22), provided, of course, that



i1

rˆ

 0; this is

ensured by Assumption MLR.3.

min

, b

,…,b

Chapter 3 Multiple Regression Analysis: Estimation 119

3A.3 Proof of Theorem 3.1

We prove Theorem 3.1 for



; the proof for the other slope parameters is virtually identi-

cal. (See Appendix E for a more succinct proof using matrices.) Under Assumption

MLR.3, the OLS estimators exist, and we can write



as in (3.22). Under Assumption

MLR.1, we can write y

as in (3.32); substitute this for y

in (3.22). Then, using



i1

rˆ

 0,



i1

rˆ

 0, for all j  2, …, k, and



i1

rˆ





i1

rˆ

, we have













i1

rˆ







i1

rˆ



(3.62)

Now, under Assumptions MLR.2 and MLR.4, the expected value of each u

,given all inde-

pendent variables in the sample, is zero. Since the rˆ

are just functions of the sample inde-

pendent variables, it follows that



X) 









i1

rˆ

E(u

X)





i1

rˆ













i1

rˆ

0





i1

rˆ







where X denotes the data on all independent variables and E(



X) is the expected value



,given x

,…,x

,for all i  1, …, n. This completes the proof.

3A.4 General Omitted Variable Bias

We can derive the omitted variable bias in the general model in equation (3.31) under the

first four Gauss-Markov assumptions. In particular, let the



, j  0, 1, ..., k be the OLS esti-

mators from the regression using the full set of explanatory variables. Let the



, j  0,1,

..., k  1 be the OLS estimators from the regression that leaves out x

. Let



, j  1, ...,

k  1 be the slope coefficient on x

in the auxiliary regression of x

on x

, x

, ... x

i,k1

, i 

1, ..., n. A useful fact is that













(3.63)

This shows explicitly that, when we do not control for x

in the regression, the estimated

partial effect of x

equals the partial effect when we include x

plus the partial effect of x

on yˆ times the partial relationship between the omitted variable, x

, and x

, j  k. Condi-

tional on the entire set of explanatory variables, X,we know that the



are all unbiased

for the corresponding



, j  1, ..., k. Further, since



is just a function of X, we have



|X)  E(



|X)  E(



|X)













(3.64)

Equation (3.64) shows that



is biased for



unless



 0—in which case x

has no

partial effect in the population—or



equals zero, which means that x

and x

are

120 Part 1 Regression Analysis with Cross-Sectional Data

partially uncorrelated in the sample. The key to obtaining equation (3.64) is equation

(3.63). To show equation (3.63), we can use equation (3.22) a couple of times. For sim-

plicity, we look at j  1. Now,



is the slope coefficient in the simple regression of y

on ˜r

, i  1 ..., n,where the ˜r

are the OLS residuals from the regression of x

on x

, ..., x

i,k1

. Consider the numerator of the expression for



i1

˜r

. But for each i,we

can write y









 ... 



 uˆ

and plug in for y

. Now, by properties of the

OLS residuals, the ˜r

have zero sample average and are uncorrelated with x

, x

, ..., x

i,k1

in the sample. Similarly, the uˆ

have zero sample average and zero sample correlation with

, x

, ..., x

. It follows that the ˜r

and uˆ

are uncorrelated in the sample (since the ˜r

are

just linear combinations of x

, x

, ..., x

i,k1

). So



i1

˜r









i1

˜r











i1

˜r



(3.65)

Now,



i1

˜r





i1

˜r

,which is also the denominator of



. Therefore, we have

shown that















i1

˜r









i1

˜r









This is the relationship we wanted to show.

3A.5 Proof of Theorem 3.2

Again, we prove this for j  1. Write



as in equation (3.62). Now, under MLR.5,

Var(u

X) 



,for all i  1, …, n. Under random sampling, the u

are independent, even

conditional on X, and the rˆ

are nonrandom conditional on X. Therefore,

Var(



X) 





i1

rˆ

Var(u

X)





i1

rˆ









i1

rˆ







i1

rˆ











i1

rˆ



Now, since



i1

rˆ

is the sum of squared residuals from regressing x

on x

,…, x



i1

rˆ

 SST

(1  R

). This completes the proof.

3A.6 Proof of Theorem 3.4

We show that, for any other linear unbiased estimator



,Var(



)  Var(



), where



is the OLS estimator. The focus on j  1 is without loss of generality.

For



as in equation (3.59), we can plug in for y

to obtain







i1





i1





i1

 … 



i1





i1

Chapter 3 Multiple Regression Analysis: Estimation 121

Now, since the w

are functions of the x



X) 



i1





i1





i1

 … 



i1





i1

E(u

X)





i1





i1





i1

 … 



i1

because E(u

X)  0, for all i  1, …, n under MLR.2 and MLR.4. Therefore, for E(



X)

to equal



for any values of the parameters, we must have



i1

 0,



i1

 1,



i1

 0, j  2, …, k.

(3.66)

Now, let rˆ

be the residuals from the regression of x

on x

,…,x

. Then, from (3.66), it

follows that



i1

rˆ

 1

(3.67)

because x



 rˆ

and



i1

 0. Now, consider the difference between

Var(



X) and Var(



X) under MLR.1 through MLR.5:





i1









i1

rˆ



(3.68)

Because of (3.67), we can write the difference in (3.68), without



,as



i1







i1

rˆ







i1

rˆ



(3.69)

But (3.69) is simply



i1





rˆ

)

(3.70)

where









i1

rˆ





i1

rˆ



, as can be seen by squaring each term in (3.70),

summing, and then canceling terms. Because (3.70) is just the sum of squared residu-

als from the simple regression of w

on rˆ

—remember that the sample average of rˆ

zero—(3.70) must be nonnegative. This completes the proof.

122 Part 1 Regression Analysis with Cross-Sectional Data

Multiple Regression Analysis: Inference

his chapter continues our treatment of multiple regression analysis. We now turn

to the problem of testing hypotheses about the parameters in the population regres-

sion model. We begin by finding the distributions of the OLS estimators under the added

assumption that the population error is normally distributed. Sections 4.2 and 4.3 cover

hypothesis testing about individual parameters, while Section 4.4 discusses how to test a

single hypothesis involving more than one parameter. We focus on testing multiple restric-

tions in Section 4.5 and pay particular attention to determining whether a group of inde-

pendent variables can be omitted from a model.

4.1 Sampling Distributions

of the OLS Estimators

Up to this point, we have formed a set of assumptions under which OLS is unbiased;

we have also derived and discussed the bias caused by omitted variables. In Section

3.4, we obtained the variances of the OLS estimators under the Gauss-Markov assump-

tions. In Section 3.5, we showed that this variance is smallest among linear unbiased

estimators.

Knowing the expected value and variance of the OLS estimators is useful for describ-

ing the precision of the OLS estimators. However, in order to perform statistical inference,

we need to know more than just the first two moments of



; we need to know the full

sampling distribution of the



. Even under the Gauss-Markov assumptions, the distribu-

tion of



can have virtually any shape.

When we condition on the values of the independent variables in our sample, it is clear

that the sampling distributions of the OLS estimators depend on the underlying distribu-

tion of the errors. To make the sampling distributions of the



tractable, we now assume

that the unobserved error is normally distributed in the population. We call this the nor-

mality assumption.

124 Part 1 Regression Analysis with Cross-Sectional Data

Assumption MLR.6 (Normality)

The population error u is independent of the explanatory variables x

, x

, …, x

and is normally

distributed with zero mean and variance



: u ~ Normal(0,



Assumption MLR.6 is much stronger than any of our previous assumptions. In fact,

since u is independent of the x

under MLR.6, E(ux

,…,x

)  E(u)  0 and Var(ux

,…,

)  Var(u) 



. Thus, if we make Assumption MLR.6, then we are necessarily assum-

ing MLR.4 and MLR.5. To emphasize that we are assuming more than before, we will

refer to the full set of Assumptions MLR.1 through MLR.6.

For cross-sectional regression applications, Assumptions MLR.1 through MLR.6 are

called the classical linear model (CLM) assumptions. Thus, we will refer to the model

under these six assumptions as the classical linear model. It is best to think of the CLM

assumptions as containing all of the Gauss-Markov assumptions plus the assumption of a

normally distributed error term.

Under the CLM assumptions, the OLS estimators



,…,



have a stronger effi-

ciency property than they would under the Gauss-Markov assumptions. It can be shown

that the OLS estimators are the minimum variance unbiased estimators,which means

that OLS has the smallest variance among unbiased estimators; we no longer have to

restrict our comparison to estimators that are linear in the y

. This property of OLS under

the CLM assumptions is discussed further in Appendix E.

A succinct way to summarize the population assumptions of the CLM is

yx ~ Normal(











 … 





where x is again shorthand for (x

,…,x

). Thus, conditional on x, y has a normal distri-

bution with mean linear in x

,…,x

and a constant variance. For a single independent vari-

able x, this situation is shown in Figure 4.1.

The argument justifying the normal distribution for the errors usually runs something

like this: Because u is the sum of many different unobserved factors affecting y, we can

invoke the central limit theorem (see Appendix C) to conclude that u has an approximate

normal distribution. This argument has some merit, but it is not without weaknesses. First,

the factors in u can have very different distributions in the population (for example, abil-

ity and quality of schooling in the error in a wage equation). Although the central limit

theorem (CLT) can still hold in such cases, the normal approximation can be poor depend-

ing on how many factors appear in u and how different are their distributions.

A more serious problem with the CLT argument is that it assumes that all unobserved

factors affect y in a separate, additive fashion. Nothing guarantees that this is so. If u

is a complicated function of the unobserved factors, then the CLT argument does not

really apply.

In any application, whether normality of u can be assumed is really an empirical mat-

ter. For example, there is no theorem that says wage conditional on educ, exper, and

tenure is normally distributed. If anything, simple reasoning suggests that the opposite

is true: since wage can never be less than zero, it cannot, strictly speaking, have a nor-

mal distribution. Further, because there are minimum wage laws, some fraction of the

population earns exactly the minimum wage, which also violates the normality assump-

tion. Nevertheless, as a practical matter, we can ask whether the conditional wage dis-

tribution is “close” to being normal. Past empirical evidence suggests that normality is

not a good assumption for wages.

Often, using a transformation, especially taking the log, yields a distribution that is

closer to normal. For example, something like log(price) tends to have a distribution that

looks more normal than the distribution of price. Again, this is an empirical issue. We will

discuss the consequences of nonnormality for statistical inference in Chapter 5.

There are some examples where MLR.6 is clearly false. Whenever y takes on just a

few values it cannot have anything close to a normal distribution. The dependent vari-

able in Example 3.5 provides a good example. The variable narr86, the number of times

a young man was arrested in 1986, takes on a small range of integer values and is zero

for most men. Thus, narr86 is far from being normally distributed. What can be done

in these cases? As we will see in Chapter 5—and this is important—nonnormality of

the errors is not a serious problem with large sample sizes. For now, we just make the

normality assumption.

Chapter 4 Multiple Regression Analysis: Inference 125

f(ylx)

E(yx)  b

 b

normal distributions

FIGURE 4.1

The homoskedastic normal distribution with a single explanatory variable.

126 Part 1 Regression Analysis with Cross-Sectional Data

Normality of the error term translates into normal sampling distributions of the OLS

estimators:

Theorem 4.1 (Normal Sampling Distributions)

Under the CLM assumptions MLR.1 through MLR.6, conditional on the sample values of the

independent variables,



~ Normal[



,Var(



)], (4.1)

where Var(



) was given in Chapter 3 [equation (3.51)]. Therefore,

(







)/sd(



) ~ Normal(0,1).

The proof of (4.1) is not that difficult, given the properties of normally distributed ran-

dom variables in Appendix B. Each



can be written as











i1

,where w



rˆ

/SSR

, rˆ

is the i

residual from the regression of the x

on all the other independent vari-

ables, and SSR

is the sum of squared residuals from this regression [see equation (3.62)].

Since the w

depend only on the independent variables, they can be treated as nonrandom.

Thus,



is just a linear combination of the errors in the sample, {u

: i  1, 2, …, n}. Under

Assumption MLR.6 (and the random sam-

pling Assumption MLR.2), the errors are

independent, identically distributed Nor-

mal(0,



) random variables. An important

fact about independent normal random

variables is that a linear combination of

such random variables is normally distrib-

uted (see Appendix B). This basically completes the proof. In Section 3.3, we showed that



) 



, and we derived Var(



) in Section 3.4; there is no need to re-derive these facts.

The second part of this theorem follows immediately from the fact that when we stan-

dardize a normal random variable by subtracting off its mean and dividing by its standard

deviation, we end up with a standard normal random variable.

The conclusions of Theorem 4.1 can be strengthened. In addition to (4.1), any linear

combination of the



,…,



is also normally distributed, and any subset of the



has

a joint normal distribution. These facts underlie the testing results in the remainder of this

chapter. In Chapter 5, we will show that the normality of the OLS estimators is still

approximately true in large samples even without normality of the errors.

4.2 Testing Hypotheses about a Single

Population Parameter: The t Test

This section covers the very important topic of testing hypotheses about any single param-

eter in the population regression function. The population model can be written as

Suppose that u is independent of the explanatory variables, and it

takes on the values 2, 1, 0, 1, and 2 with equal probability of

1/5. Does this violate the Gauss-Markov assumptions? Does this

violate the CLM assumptions?

QUESTION 4.1

y 







 … 



 u,

(4.2)

and we assume that it satisfies the CLM assumptions. We know that OLS produces unbi-

ased estimators of the



. In this section, we study how to test hypotheses about a par-

ticular



. For a full understanding of hypothesis testing, one must remember that the



are unknown features of the population, and we will never know them with certainty.

Nevertheless, we can hypothesize about the value of



and then use statistical inference

to test our hypothesis.

In order to construct hypotheses tests, we need the following result:

Theorem 4.2 (t Distribution for the Standardized Estimators)

Under the CLM assumptions MLR.1 through MLR.6,

(







)/se(



) ~ t

nk1

(4.3)

where k  1 is the number of unknown parameters in the population model y 







 … 



 u (k slope parameters and the intercept



This result differs from Theorem 4.1 in some notable respects. Theorem 4.1 showed

that, under the CLM assumptions, (







)/sd(



) ~ Normal(0,1). The t distribution in

(4.3) comes from the fact that the constant



in sd(



) has been replaced with the ran-

dom variable



ˆ. The proof that this leads to a t distribution with n  k  1 degrees

of freedom is not especially insightful. Essentially, the proof shows that (4.3) can be

written as the ratio of the standard normal random variable (







)/sd(



) over the

square root of



. These random variables can be shown to be independent, and (n

 k  1)



 

nk1

. The result then follows from the definition of a t random

variable (see Section B.5).

Theorem 4.2 is important in that it allows us to test hypotheses involving the



. In

most applications, our primary interest lies in testing the null hypothesis



 0,

(4.4)

where j corresponds to any of the k independent variables. It is important to understand

what (4.4) means and to be able to describe this hypothesis in simple language for a par-

ticular application. Since



measures the partial effect of x

on (the expected value of) y,

after controlling for all other independent variables, (4.4) means that, once x

, x

,…,x

j1

j1

,…,x

have been accounted for, x

has no effect on the expected value of y. We can-

not state the null hypothesis as “x

does have a partial effect on y” because this is true for

any value of



other than zero. Classical testing is suited for testing simple hypotheses

like (4.4).

As an example, consider the wage equation

log(wage) 







educ 



exper 



tenure  u.

Chapter 4 Multiple Regression Analysis: Inference 127

The null hypothesis H



 0 means that, once education and tenure have been accounted

for, the number of years in the workforce (exper) has no effect on hourly wage. This is an

economically interesting hypothesis. If it is true, it implies that a person’s work history

prior to the current employment does not affect wage. If



 0, then prior work experi-

ence contributes to productivity, and hence to wage.

You probably remember from your statistics course the rudiments of hypothesis test-

ing for the mean from a normal population. (This is reviewed in Appendix C.) The

mechanics of testing (4.4) in the multiple regression context are very similar. The hard

part is obtaining the coefficient estimates, the standard errors, and the critical values, but

most of this work is done automatically by econometrics software. Our job is to learn how

regression output can be used to test hypotheses of interest.

The statistic we use to test (4.4) (against any alternative) is called “the” t statistic or

“the” t ratio of



and is defined as







/se(



(4.5)

We have put “the” in quotation marks because, as we will see shortly, a more general form

of the t statistic is needed for testing other hypotheses about



. For now, it is important to

know that (4.5) is suitable only for testing (4.4). For particular applications, it is helpful to

index t statistics using the name of the independent variable; for example, t

educ

would be

the t statistic for



educ

The t statistic for



is simple to compute given



and its standard error. In fact, most

regression packages do the division for you and report the t statistic along with each coef-

ficient and its standard error.

Before discussing how to use (4.5) formally to test H



 0, it is useful to see why



has features that make it reasonable as a test statistic to detect



 0. First, since se(



)

is always positive, t



has the same sign as



: if



is positive, then so is t



, and if



is neg-

ative, so is t



. Second, for a given value of se(



), a larger value of



leads to larger val-

ues of t



. If



becomes more negative, so does t



Since we are testing H



 0, it is only natural to look at our unbiased estimator of



,for guidance. In any interesting application, the point estimate



will never exactly

be zero, whether or not H

is true. The question is: How far is



from zero? A sample

value of



very far from zero provides evidence against H



 0. However, we must

recognize that there is a sampling error in our estimate



, so the size of



must be weighed

against its sampling error. Since the standard error of



is an estimate of the standard devi-

ation of



, t



measures how many estimated standard deviations



is away from zero.

This is precisely what we do in testing whether the mean of a population is zero, using

the standard t statistic from introductory statistics. Values of t



sufficiently far from zero

will result in a rejection of H

. The precise rejection rule depends on the alternative hypoth-

esis and the chosen significance level of the test.

Determining a rule for rejecting (4.4) at a given significance level—that is, the prob-

ability of rejecting H

when it is true—requires knowing the sampling distribution of



when H

is true. From Theorem 4.2, we know this to be t

nk1

. This is the key the-

oretical result needed for testing (4.4).

128 Part 1 Regression Analysis with Cross-Sectional Data

Wooldridge J. Introductory Econometrics: A Modern Approach (Basic Text - 3d ed.)

Подождите немного. Документ загружается.