Devore J.L., Berk K.N. Modern Mathematical Statistics with Applications

Подождите немного. Документ загружается.

hypothesis H

: b

¼ 0. Since we do not reject the hypothesis of no interaction, let’s

look at the results for the difference b

in the mode l with two parallel lines. The

variable Priv1_St0 is x

, the dummy variable with value 1 for private and 0 for state

universities. The P-value for its coefficient is .0231, so we can reject the hypothesis

that it is 0 at the .05 level. The value of the coefficient is 13.17, which means that a

private university is estimated to have a graduation rate about 13 percentage points

higher than a state university with the same freshman SAT. This is pretty large,

especially in comparison with the coefficient for SAT, which is .06869. Dividing

.06869 into

¼ 13:17 gives 192, which means that it takes 192 points in SAT to

make up the difference between private and public universities. To put it another

way, a private university with freshman SAT of 1000 is estimated to have the same

graduation rate as a state unive rsity with SAT of 1192.

■

You might think that the way to handle a three-category situation is to define

a single numerical variable with coded values such as 0, 1, and 2 corresponding to

the three categories. This is incorrect, because it imposes an ordering on the

categories that is not necessarily implied by the problem context. The correct

way to incorporate three cat egories is to define two differ ent dummy variables.

Suppose, for example, that y is a score on a posttest taken after instruction, x

is the

score on an ability pretest taken before instruction, and that there are three methods

of instruction in a mathematics unit (1) with symbols, (2) without symbols, and (3)

a mixture with and without symbols. Then let

1 instruction method 1

0 otherwise



1 instruction method 2

0 otherwise



For an individual taught with method 1, x

¼ 1 and x

¼ 0, whereas for an

individual taught with method 2, x

¼ 0 and x

¼ 1. For an individual taught

with method 3, x

¼ x

¼ 0, and it is not possible that x

¼ x

¼ 1 because

an individual cannot be taught simultaneously by both methods 1 and 2. The

no-interaction model would have only the predictors x

, x

, and x

. The following

interaction model allows the mean change in lifetime associated with a 1-unit

increase in pretest to depend on the method of instruction:

Y ¼ b

þ b

þ e

Construction of a picture like Figure 12.34 with a graph for each of the three

possible (x

, x

) pairs gives three no nparallel lines (unless b

¼ b

¼ 0). How

would we interpret statistically significant interaction? Suppose that it occurs to

the extent that the lines for methods 1 and 2 cross. In particular, if the line for

method 1 is higher on the right and lower on the left, it means that symbols work

well for high ability students but not as well for low ability students.

More generally, incorporating a categor ical variable with c possible cate-

gories into a multiple regression model requires the use of c  1 indicator variables

(e.g., five methods of instruction would necessitate using four indicator variabl es).

Thus even one categor ical variable can add many predicto rs to a model.

Indicator variables can be used for categorical variables without any other

variables in the model. For example, consider Example 11.3, which compared three

different compounds in their ability to prevent fabric soiling. Using a regression

698 CHAPTER 12 Regression and Correlation

with two dummy variables gives the following regression ANOVA table, just like

the one in Example 11.2:

Source DF SS MS F P

Regression 2 0.06085 0.03043 0.99 0.401

Residual error 12 0.37008 0.03084

Total 14 0.43093

Analysis that involves both quantitative and categorical predictors, as in

Example 12.31, is calle d analysis of covariance, and the quantitative variable is

called a covariate. So metimes more than one covariate is used.

Other Models The logistic regression model introduced in Section 12.1 can be

extended to incorporate more than one predictor. Various nonlinear models are also

used frequently in applied work. An example is the multiple exponential model

Y ¼ e

þb

þþb

 e

Taking logs on both sides shows that ln(Y) ¼ b

+ b

+  + b

+ e

, where

¼ ln(e). This is the usual multiple regression model with ln(Y) as the response

variable.

Exercises Section 12.7 (78–90)

78. Cardiorespiratory fitness is widely recognized as a

major component of overall physical well-being.

Direct measurement of maximal oxygen uptake

(VO

max) is the single best measure of such fit-

ness, but direct measurement is time-consuming

and expensive. It is therefore desirable to have a

prediction equation for VO

max in terms of easily

obtained quantities. Consider the variables

y ¼ VO

max L=minðÞx

¼ weight kgðÞ

¼ age yrðÞ

¼ time necessary to walk 1 mile minðÞ

¼ heart rate at the end of the walk beats=minðÞ

Here is one possible model, for male students,

consistent with the information given in the article

“Validation of the Rockport Fitness Walking Test

in College Males and Females” (Res. Q. Exercise

Sport, 1994: 152–158):

Y ¼ 5:0 þ :01x

 :05x

 :13x

 :01x

þ e

s ¼ :4

a. Interpret b

and b

b. What is the expected value of VO

max when

weight is 76 kg, age is 20 year, walk time is

12 min, and heart rate is 140 beats/min?

c. What is the probability that VO

max will be

between 1.00 and 2.60 for a single observation

made when the values of the predictors are as

stated in part (b)?

79. Let y ¼ sales at a fast-food outlet ($1000’s), x

number of competing outlets within a 1-mile

radius, x

¼ population within a 1-mile radius

(1000’s of people), and x

be an indicator variable

that equals 1 if the outlet has a drive-up window

and 0 otherwise. Suppose that the true regression

model is

Y ¼ 10:0  1:2x

þ 6:8x

þ 15:3x

þ e

a. What is the mean value of sales when the

number of competing outlets is 2, there are

8000 people within a 1-mile radius, and the

outlet has a drive-up window?

b. What is the mean value of sales for an outlet

without a drive-up window that has three com-

peting outlets and 5000 people within a 1-mile

radius?

c. Interpret b

80. The article “Analysis of the Modeling Methodol-

ogies for Predicting the Strength of Air-Jet Spun

Yarns” (Textile Res. J., 1997: 39–44) reported on a

12.7 Multiple Regression Analysis 699

study carried out to relate yarn tenacity (y,ing/

tex) to yarn count (x

, in tex), percentage polyester

), first nozzle pressure (x

, in kg/cm

), and

second nozzle pressure (x

, in kg/cm

). The esti-

mate of the constant term in the corresponding

multiple regression equation was 6.121. The esti-

mated coefficients for the four predictors were

.082, .113, .256, and .219, respectively, and

the coefficient of multiple determination was .946.

Assume that n ¼ 25.

a. State and test the appropriate hypotheses to

decide whether the fitted model specifies a

useful linear relationship between the depen-

dent variable and at least one of the four model

predictors.

b. Calculate the value of adjusted R

and

comment.

c. Calculate a 99% confidence interval for true

mean yarn tenacity when yarn count is 16.5,

yarn contains 50% polyester, first nozzle pres-

sure is 3, and second nozzle pressure is 5 if the

estimated standard deviation of predicted

tenacity under these circumstances is .350.

81. The article “Selling Prices/Sq. Ft. of Office Build-

ings in Downtown Chicago – How Much Is

It Worth to Be an Old But Class A Building?”

(J. Real Estate Res., 2010: 1–22) considered a

regression model to relate y ¼ ln($/ft

) to 16 pre-

dictors, including age, age squared, number of

stories, occupancy rate, and indicator variables

for whether a building has a restaurant and

whether it has conference rooms. The model was

fit to data resulting from 203 sales.

a. The coefficient of multiple determination was

.711. What is the value of the adjusted coeffi-

cient of multiple determination? Does it sug-

gest that the relatively high R

value was the

result of including too many predictors in the

model relative to the amount of data available?

b. Using the R

value from (a), carry out a test of

hypotheses to see whether there is a useful

linear relationship between the dependent var-

iable and at least one of the predictors.

c. The estimated coefficient of the indicator vari-

able for whether or not a building was class A

was .364. Interpret this estimated coefficient,

first in terms of y and then in terms of $/ft

d. The t ratio for the estimated coefficient of (c)

was 5.49. What does this tell you?

82. An investigation of a die casting process resulted

in the accompanying data on x

¼ furnace tem-

perature, x

¼ die close time, and y ¼ tempera-

ture difference on the die surface (“A Multiple-

Objective Decision-Making Approach for Asses-

sing Simultaneous Improvement in Die Life and

Casting Quality in a Die Casting Process,” Qual.

Engrg., 1994: 371–383).

1250 1300 1350 1250 1300

67676

y 80 95 101 85 92

1250 1300 1350 1350

8878

y 87 96 106 108

MINITAB output from fitting the multiple regres-

sion model with predictors x

and x

is given here.

The regression equation is

tempdiff ¼200 + 0.210 furntemp

+3.00 clostime

Predictor Coef Stdev t-ratio p

Constant 199.56 11.64 17.14 0.000

furntemp 0.210000 0.008642 24.30 0.000

clostime 3.0000 0.4321 6.94 0.000

s ¼ 1.058 R-sq ¼ 99.1% R-sq(adj) ¼ 98.8%

Analysis of Variance

Source DF SS MS F p

Regression 2 715.50 357.75 319.31 0.000

Error 6 6.72 1.12

Total 8 722.22

a. Carry out the model utility test.

b. Calculate and interpret a 95% confidence inter-

val for b

, the population regression coefficient

of x

c. When x

¼ 1300 and x

¼ 7, the estimated

standard deviation of

Y is s

¼ :353. Calculate

a 95% confidence interval for true average

temperature difference when furnace tempera-

ture is 1300 and die close time is 7.

d. Calculate a 95% prediction interval for the

temperature difference resulting from a single

experimental run with a furnace temperature of

1300 and a die close time of 7.

e. Use appropriate diagnostic plots to see if there

is any reason to question the regression model

assumptions.

83. An experiment carried out to study the effect of

the mole contents of cobalt (x

) and the calcination

700 CHAPTER 12 Regression and Correlation

temperature (x

) on the surface area of an iron–

cobalt hydroxide catalyst (y) resulted in the

accompanying data (“Structural Changes and Sur-

face Properties of Co

3x

Spinels,” J. Chem.

Tech. Biotech., 1994: 161–170).

.6 .6 .6 .6 .6 1.0 1.0

200 250 400 500 600 200 250

y 90.6 82.7 58.7 43.2 25.0 127.1 112.3

1.0 1.0 1.0 2.6 2.6 2.6 2.6

400 500 600 200 250 400 500

y 19.6 17.8 9.1 53.1 52.0 43.4 42.4

2.6 2.8 2.8 2.8 2.8 2.8

600 200 250 400 500 600

y 31.6 40.9 37.9 27.5 27.3 19.0

A request to the SAS package to fit the regression

function b

+ b

, where x

(an interaction predictor) yielded the accom-

panying output.

a. Predict the value of surface area when cobalt

content is 2.6 and temperature is 250, and

calculate the value of the corresponding resid-

ual.

b. Since

¼46:0, is it legitimate to conclude

that if cobalt content increases by 1 unit while

the values of the other predictors remain fixed,

surface area can be expected to decrease by

roughly 46 units? Explain your reasoning.

c. Does there appear to be a useful relationship

between y and the predictors?

d. Given that mole contents and calcination tem-

perature remain in the model, does the interac-

tion predictor x

provide useful information

about y? State and test the appropriate hypoth-

eses using a significance level of .01.

e. The estimated standard deviation of

Y when

mole contents is 2.0 and calcination tempera-

ture is 500 is s

¼ 4:69. Calculate a 95% con-

fidence interval for the mean value of surface

area under these circumstances.

f. Based on appropriate diagnostic plots, is there

any reason to question the regression model

assumptions?

84. A regression analysis carried out to relate y ¼

repair time for a water filtration system (hr) to

¼ elapsed time since the previous service

(months) and x

¼ type of repair (1 if electri-

cal and 0 if mechanical) yielded the following

model based on n ¼ 12 observations: y

¼ .950 + .400x

+ 1.250x

. In addition, SST

¼ 12.72, SSE ¼ 2.09, and s

¼ :312.

SAS output for Exercise 83

Dependent Variable: SURFAREA

Analysis of Variance

Source DF Sum of Squares Mean Square F Value Prob > F

Model 3 15223.52829 5074.50943 18.924 0.0001

Error 16 4290.53971 268.15873

C Total 19 19514.06800

Root MSE 16.37555 R-square 0.7801

Dep Mean 48.06000 Adj R-sq 0.7389

C.V. 34.07314

Parameter Estimates

Variable DF

Parameter

Estimate

Standard

Error

T for H0:

Parameter ¼ 0

Prob

> |T|

INTERCEP 1 185.485740 21.19747682 8.750 0.0001

COBCON 1 45.969466 10.61201173 4.332 0.0005

TEMP 1 0.301503 0.05074421 5.942 0.0001

CONTEMP 1 0.088801 0.02540388 3.496 0.0030

12.7 Multiple Regression Analysis 701

a. Does there appear to be a useful linear

relationship between repair time and the

two model predictors? Carry out a test of

the appropriate hypotheses using a signifi-

cance level of .05.

b. Given that elapsed time since the last ser-

vice remains in the model, does type of

repair provide useful information about

repair time? State and test the appropriate

hypotheses using a significance level of

.01.

c. Calculate and interpret a 95% CI for b

d. The estimated standard deviation of a pre-

diction for repair time when elapsed time

is 6 months and the repair is electrical is

.192. Predict repair time under these cir-

cumstances by calculating a 99% predic-

tion interval. Does the interval suggest that

the estimated model will give an accurate

prediction? Why or why not?

85. The article “The Undrained Strength of Some

Thawed Permafrost Soils” (Canad. Geotech.

J., 1979: 420–427) contains the following

data on undrained shear strength of sandy

soil (y, in kPa), depth (x

, in m), and water

content (x

, in %).

Obs yx

yy

ye*

1 14.7 8.9 31.5 23.35 8.65 1.50

2 48.0 36.6 27.0 46.38 1.62 .54

3 25.6 36.8 25.9 27.13 1.53 .53

4 10.0 6.1 39.1 10.99 .99 .17

5 16.0 6.9 39.2 14.10 1.90 .33

6 16.8 6.9 38.3 16.54 .26 .04

7 20.7 7.3 33.9 23.34 2.64 .42

8 38.8 8.4 33.8 25.43 13.37 2.17

9 16.9 6.5 27.9 15.63 1.27 .23

10 27.0 8.0 33.1 24.29 2.71 .44

11 16.0 4.5 26.3 15.36 .64 .20

12 24.9 9.9 37.8 29.61 4.71 .91

13 7.3 2.9 34.6 15.38 8.08 1.53

14 12.8 2.0 36.4 7.96 4.84 1.02

The predicted values and residuals were com-

puted by fitting a full quadratic model, which

resulted in the estimated regression function

y ¼151:36  16:22x

þ 13:48x

þ :094x

 :253x

þ :492x

a. Do plots of e * versus x

, e* versus x

, and

e* versus

y suggest that the full quadratic

model should be modified? Explain your

answer.

b. The value of R

for the full quadratic

model is .759. Test at level .05 the null

hypothesis stating that there is no linear

relationship between the dependent vari-

able and any of the five predictors.

c. Each of the null hypotheses H

: b

¼ 0

versus H

: b

6¼ 0, i ¼ 1, 2, 3, 4, 5, is not

rejected at the 5% level. Does this make

sense in view of the result in (b)? Explain.

d. It is shown in Section 12.8 that

VðYÞ¼s

¼ Vð

YÞþVðY 

YÞ. The esti-

mate of s is

s ¼ s ¼ 6:99 (from the full

quadratic model). First obtain the esti-

mated standard deviation of Y 

Y, and

then estimate the standard deviation of

(i.e.,

when x

¼ 8.0 and x

¼ 33.1.

Finally, compute a 95% CI for mean

strength. [Hint: What is ðy 

yÞ=e



e. Sometimes an investigator wishes to

decide whether a group of m predictors

(m > 1) can simultaneously be eliminated

from the model. The null hypothesis says

that all b’s associated with these m predic-

tors are 0, which is interpreted to mean that

as long as the other k  m predictors are

retained in the model, the m predictors

under consideration collectively provide

no useful information about y. The test is

carried out by first fitting the “full” model

with all k predictors to obtain SSE(full)

and then fitting the “reduced” model con-

sisting just of the k  m predictors not

being considered for deletion to obtain

SSE(red). The test statistic is

F ¼

½SSEðredÞSSEðfullÞ=m

SSE(full)/[n ðk þ 1Þ

The test is upper-tailed and based on m

numerator df and n  (k + 1) denominator

df. Fitting the first-order model with just the

predictors x

and x

results in SSE ¼ 894.95.

State and test at significance level .05 the null

hypothesis that none of the three second-order

predictors (one interaction and two quadratic

predictors) provides useful information about

y provided that the two first-order predictors

are retained in the model.

702

CHAPTER 12 Regression and Correlation

86. The following data on y ¼ glucose concen-

tration (g/L) and x ¼ fermentation time

(days) for a particular blend of malt liquor

was read from a scatter plot in the article

“Improving Fermentation Productivity with

Reverse Osmosis” (Food Tech., 1984:

92–96):

x 12345678

y 74 54 52 51 52 53 58 71

a. Verify that a scatter plot of the data is

consistent with the choice of a quadratic

regression model.

b. The estimated quadratic regression equa-

tion is y ¼ 84.482  15.875x+1.7679x

Predict the value of glucose concentration

for a fermentation time of 6 days, and

compute the corresponding residual.

c. Using SSE ¼ 61.77, what proportion of

observed variation can be attributed to the

quadratic regression relationship?

d. The n ¼ 8 standardized residuals based on

the quadratic model are 1.91, 1.95, .25,

.58, .90, .04, .66, and .20. Construct a

plot of the standardized residuals versus x

and a normal probability plot. Do the plots

exhibit any troublesome features?

e. The estimated standard deviation of

Y6

—that is,

ð6Þþ

ð36Þ—is

1.69. Compute a 95% CI for m

Y·6

f. Compute a 95% PI for a glucose concen-

tration observation made after 6 days of

fermentation time.

87. Utilization of sucrose as a carbon source for

the production of chemicals is uneconomical.

Beet molasses is a readily available and low-

priced substitute. The article “Optimization of

the Production of b-Carotene from Molasses

by Blakeslea trispora”(J. Chem. Tech. Bio-

tech., 2002: 933–943) carried out a multiple

regression analysis to relate the dependent

variable y ¼ amount of b-carotene (g/dm

)

to the three predictors: amount of linoleic

acid, amount of kerosene, and amount of anti-

oxidant (all g/dm

a. Fitting the complete second-order model in

the three predictors resulted in R

¼ .987

and adjusted R

¼ .974, whereas fitting

the first-order model gave R

¼ .016.

What would you conclude about the two

models?

b. For x

¼ x

¼ 30, x

¼ 10, a statistical

software package reported that

y ¼ :66573, s

¼ :01785 based on the

complete second-order model. Predict

the amount of b-carotene that would

result from a single experimental run

with the designated values of the indepen-

dent variables, and do so in a way that

conveys information about precision and

reliability.

Obs Linoleic Kerosene Antiox Betacaro

1 30.00 30.00 10.00 0.7000

2 30.00 30.00 10.00 0.6300

3 30.00 30.00 18.41 0.0130

4 40.00 40.00 5.00 0.0490

5 30.00 30.00 10.00 0.7000

6 13.18 30.00 10.00 0.1000

7 20.00 40.00 5.00 0.0400

8 20.00 40.00 15.00 0.0065

9 40.00 20.00 5.00 0.2020

10 30.00 30.00 10.00 0.6300

11 30.00 30.00 1.59 0.0400

12 40.00 20.00 15.00 0.1320

13 40.00 40.00 15.00 0.1500

14 30.00 30.00 10.00 0.7000

15 30.00 46.82 10.00 0.3460

16 30.00 30.00 10.00 0.6300

17 30.00 13.18 10.00 0.3970

18 20.00 20.00 5.00 0.2690

19 20.00 20.00 15.00 0.0054

20 46.82 30.00 10.00 0.0640

88. Snowpacks contain a wide spectrum of pollu-

tants that may represent environmental

hazards. The article “Atmospheric PAH

Deposition: Deposition Velocities and Wash-

out Ratios”(J. Environ. Engrg., 2002:

186–195) focused on the deposition of poly-

aromatic hydrocarbons. The authors proposed

a multiple regression model for relating

deposition over a specified time period (y,in

mg/m

) to two rather complicated predictors

(mg-s/m

) and x

(mg/m

) defined in terms

of PAH air concentrations for various species,

total time, and total amount of precipitation.

Here is data on the species fluoranthene and

corresponding MINITAB output:

Obs x1 x2 flth dep

1 92017 .0026900 278.78

2 51830 .0030000 124.53

3 17236 .0000196 22.65

12.7 Multiple Regression Analysis 703

4 15776 .0000360 28.68

5 33462 .0004960 32.66

6 243500 .0038900 604.70

7 67793 .0011200 27.69

8 23471 .0006400 14.18

9 13948 .0004850 20.64

10 8824 .0003660 20.60

11 7699 .0002290 16.61

12 15791 .0014100 15.08

13 10239 .0004100 18.05

14 43835 .0000960 99.71

15 49793 .0000896 58.97

16 40656 .0026000 172.58

17 50774 .0009530 44.25

The regression equation is

flth dep ¼33.5 + 0.00205 x1 + 29836 x2

Predictor Coef SE Coef T P

Constant

33.46 14.90 2.25 0.041

x1 0.0020548 0.0002945 6.98 0.000

x2 29836 13654 2.19 0.046

S ¼ 44.28 R-Sq ¼ 92.3% R-Sq(adj) ¼ 91.2%

Analysis of Variance

Source DF SS MS F P

Regression 2 330989 165495 84.39 0.000

Residual

error

14 27454 1961

Total 16 358443

Formulate questions and perform appropriate

analyses.

Construct the appropriate residual plots,

including plots against the predictors. Based

on these plots, justify adding a quadratic term,

and fit the model with this additional term. Is

this term statistically significant, and does it

help the appearance of the diagnostic plots?

89. The following data set has ratings from rateb-

eer.com along with values of IBU (interna-

tional bittering units, a measure of bitterness)

and ABV (alcohol by volume) for 25 beers.

Notice which beers have the lowest ratings

and which are highest.

a. Find the correlations (and the

corresponding P-values) among Rating,

IBU, and ABV.

b. Regress Rating on IBU and ABV. Notice

that although both predictors have strongly

significant correlations with Rating, they

do not both have significant regression

coefficients. How do you explain this?

c. Plot the residuals from the regression of (b)

to check the assumptions. Also plot rating

against each of the two predictors. Which

of the assumptions is clearly not satisfied?

d. Repeat the multiple regression in (b) with

the square of IBU as a third predictor.

Again check assumptions.

e. How effective is the regression in (d)?

Interpret the coefficients with regard to

statistical significance and sign. In particu-

lar, discuss the relationship to IBU.

f. Summarize your conclusions.

Beer IBU ABV Rating

Amstel Light 18 3.5 1.93

Anchor Liberty Ale 54 5.9 3.60

Anchor Steam 33 4.9 3.31

Bud Light 7 4.2 1.15

Budweiser 11 5 1.38

Coors 14 5 1.63

DAB Dark 32 5 2.73

Dogfish 60 Minute IPA 60 6 3.76

Great Divide Titan IPA 65 6.8 3.81

Great Divide Hercules Double

IPA

85 9.1 4.05

Guinness Extra Stout 60 5 3.38

Harp Lager 21 4.3 2.85

Heineken 23 5 2.13

Heineken Premium Light 11 3.2 1.62

Michelob Ultra 4 4.2 1.01

Newcastle Brown Ale 18 4.7 3.05

Pilsner Urquell 35 4.4 3.28

Redhook ESB 29 5.77 3.06

Rogue Imperial Stout 88 11.6 3.98

Samuel Adams Boston Lager 31 4.9 3.19

Shiner Light 13 4.03 2.57

Sierra Nevada Pale Ale 37 5.6 3.61

Sierra Nevada Porter 40 5.6 3.60

Terrapin All-American Imperial

Pilsner

75 7.5 3.46

Three Floyds Alpha King 66 6 4.04

90. The article “Promoting Healthy Choices:

Information versus Convenience” (Amer.

Econ. J.: Applied Econ., 2010: 164 – 178)

reported on a field experiment at a fast-food

704

CHAPTER 12 Regression and Correlation

sandwich chain to see whether calorie infor-

mation provided to patrons would affect calo-

rie intake. One aspect of the study involved

fitting a multiple regression model with 7 pre-

dictors to data consisting of 342 observations.

Predictors in the model included age and

dummy variables for gender, whether or not a

daily calorie recommendation was provided,

and whether or not calorie information about

choices was provided. The reported value of

the F ratio for testing model utility was 3.64.

a. At significance level .01, does the model

appear to specify a useful linear relation-

ship between calorie intake and at least

one of the predictors?

b. What can be said about the P-value for the

model utility F test?

c. What proportion of the observed variation in

calorie intake can be attributed to the model

relationship? Does this seem very impres-

sive? Why is the P-value as small as it is?

d. The estimated coefficient for the indicator

variable calorie information provided was

71.73, with an estimated standard error

of 25.29. Interpret the coefficient. After

adjusting for the effects of other predic-

tors, does it appear that true average calo-

rie intake depends on whether or not

calorie information is provided? Carry

out a test of appropriate hypotheses.

12.8

Regression with Matrices

In Section 12.7 we used an additive model equation to relate a dependent variable y

to independent variables x

, ..., x

. That is, we used the model

Y ¼ b

þ b

þþb

þ e;

where e is a random deviation or error term that is normally distributed with mean

0, variance s

, and the variou s e’s are independent of one another. Simple linear

regression is the special case in which k ¼ 1.

The Normal Equations

Suppose that we have n observations, each consisting of a y value and values of the

k predictors (so each observation consists of k + 1 numbers). We have then

þ b

þþb

þ e

þ b

þþb

þ e

For example, if there are n ¼ 6 cars, where y is horsepower, x

is engine size

(liters), and x

indicates fuel type (regular or premium), then we are trying to

predict horsepower as a linear function of the k ¼ 2 predictors engine size and fuel

type. The equations can be written much more compactly using vectors and

matrices. To do this, form a column vector of observations on y, a column vector

of regression coefficients, and a vector of random deviations:

y ¼

b ¼

e ¼

12.8 Regression with Matrices 705

Also form an n  (k + 1) matrix in which the first column consists of 1’s

(corresponding to the constant term in the model), the second column consists of

the values of the first predictor x

(i.e., of x

, x

, ..., x

), the third column has the

values of x

, and so on.

X ¼

1 x

... x

1 x

... x

The X matrix has a row for each observation, consisting of 1 and then the values of

the k predictors. The equations relating the observed y’s to the x

’s can then be

written very concisely as

¼ y ¼ Xb þ e ¼

1 x

... x

1 x

... x

We now estimate b

, b

, ..., b

using the principle of least squares: Find b

, b

, ..., b

to minimize

i¼1

½y

ðb

þ b

þþb

Þ

¼ðy  XbÞ

ðy  XbÞ¼ y  Xb

where b is the column vector with entries b

, b

, ..., b

, and ||u|| is the length of u.

If we equate to zero the partial derivative with respect to each of the

coefficients, then it leads to the normal equations:

i¼1

1 þ b

i¼1

þþb

i¼1

þ b

i¼1

þþb

i¼1

þ b

i¼1

þþb

i¼1

In matrix form this is

i¼1

...

i¼1

...

i¼1

...

i¼1

706 CHAPTER 12 Regression and Correlation

The matrix on the left is just X

X and the matrix on the right is X

y, where X

indicates X-transpose, so the normal equations become X

Xb ¼ X

y. We will

assume throughout this section that X

X has an inverse, so the vector of estimated

coefficients is

b ¼ b ¼½X

X

1

Example 12.32 Ba sed on six cars, we try to predict horsepower (hp) using engine size (liters) and

fuel type. Here is the data set:

Make hp Eng Size Fuel

Ford 132 2.0 Regular

Mazda 167 2.0 Premium

Subaru 170 2.5 Regular

Lexus 204 2.5 Premium

Mitsubishi 230 3.0 Regular

BMW 260 3.0 Premium

The hp column will be used for y, and engine size values are placed in the second

column of X, but numbers must be used instead of words in the third column. We

use 0 for “regular” and 1 for “premium.” Any two numb ers could be used instead of

0 and 1, but this choice is convenient in terms of the interpretation of the coeffi-

cients. This gives

X ¼

12:00

12:01

12:50

12:51

13:00

13:01

y ¼

132

167

170

204

230

260

X ¼

615 3

15 38:57:5

37:53

y ¼

1163

3003

631

Therefore,

b ¼½X

X

1

y ¼

79=12 5=2 1=3

5=21 0

1=30 2=3

1163

3003

631

61:417

95:5

The coefficient 95.5 for engine size means that, if the fuel type is held constant, then

we estimate that horse power will increase on average by 95.5 when the engine size

increases by one liter. Similarly, the coefficient 33 for fuel means that, if the engine

size is held constant, then we estimate that horsepower will increase on average by

33 whe n the fuel type increases by 1. However, increasing fuel type by 1 unit means

switching from regular fuel to premium fuel, so the difference in horsepower

corresponding to the difference in fuels is 33. Notice that this is the difference

between the average for the three premium-fuel cars and the average for the three

regular-fuel cars.

■

Residuals, ANOVA, F, and R-Squared

The estimated regression coefficients can be used to obtain the predicted values.

Recall that

þþ

. The expression for

is the product

of the ith row of X and the

b vector. The vector of predicted values is then

12.8 Regression with Matrices 707