Soong T.T. Fundamentals of Probability and Statistics for Engineers
Подождите немного. Документ загружается.
Since
0
. That is, we conclude that the data do not
indicate a linear relationship between E Y and x; the probability that we are
wrong in accepting H
0
is 0.05.
In closing, let us remark that we are often called on to perform tests of
simultaneous hypotheses. For example, one may wish to test H
0
:0and
1 against H
1
: 0 or 1 or both. Such tests involve both estimators
multiple linear regression, to be discussed in the next section. Such tests
customarily involve F-distributed test statistics, and we will not pursue them
here. A general treatment of simultaneous hypotheses testing can be found in
Rao (1965), for example.
11.2 MULTIPLE LINEAR REGRESSION
The vector–matrix approach proposed in the preceding section provides a smooth
transition from simple linear regression to linear regression involving more than
one independent variable. In multiple linear regression, the model takes the form
Again, we assume that the variance of Y is
2
and is independent of x
1
,x
2
,...,and
x
m
. As in simple linear regression, we are interested in estimating (m 1) regres-
sion coefficients
0
,
1
,...,and
m
, obtaining certain interval estimates, and testing
hypotheses about these parameters on the basis of a sample of Y values with their
associated values of (x
1
,x
2
,...,x
m
). Let us note that our sample of size n in this
case takes the form of arrays (x
11
,x
21
,...,x
m1
,Y
1
), (x
12
,x
22
,...,x
m2
,Y
2
),...,
(x
1n
,x
2n
,...,x
mn
,Y
n
). For each set of values x
ki
, k 1, 2, . . . , m, of x
i
,Y
i
is an
independent observation from population Y defined by
As before, E is the random error, with mean 0 and variance
2
.
11.2.1 LEAST SQUARES METHOD OF ESTIMATION
To estimate the regression coefficients, the method of least squares will again be
employed. Given observed sample-value sets (x
1i
,x
2i
,...,x
mi
,y
i
),i 1,2,...,n,
the system of observed regression equations in this case takes the form
354
Fundamentals of Probability and Statistics for Engineers
0 31 < 0 61, we accept H
and and hence require their joint distribution. This is also often the case in
^
f g
1
: 6 0o 6
^
A
EfY g
0
1
x
1
2
x
2
m
x
m
: 11:45
Y
0
1
x
1
m
x
m
E:11:46
y
i
0
1
x
1i
m
x
mi
e
i
; i 1; 2; ...; n: 11:47
:
:
^
B
TLFeBOOK
If we let
and
Equation (11.47) can be represented by vector–matrix equation:
Comparing Equation (11.48) with Equation (11.12) in simple linear regression,
we see that the observed regression equations in both cases are identical except
that the C matrix is now an n (m 1) matrix and is an (m 1)-dimensional
vector. Keeping this dimension difference in mind, the results obtained in the
case of simple linear regression based on Equation (11.12) again hold in the
multiple linear regression case. Thus, without further derivation, we have for
the solution of least-square estimates of [see Equation (11.15)]
The existence of matrix inverse (C
T
C)
1
requires that there are at least (m 1)
distinct sets of values of (x
1i
,x
2i
,...,x
mi
) represented in the sample. It is noted
that C
T
C is a (m 1) (m 1) symmetric matrix.
Ex ample 11. 6. Problem: the average monthly electric power consumption (Y )
at a certain manufacturing plant is considered to be linearly dependent on the
average ambient temperature (x
1
) and the number of working days in a month
(x
2
). Consider the one-year monthly data given in Table 11.4. Determine the
least-square estimates of the associated linear regression coefficients.
Table 11. 4 Average monthly power consumption y (in thousands of kwh), with
number of working days in the month, x
2
, and average ambient temperature, x
1
,(in F)
for Example 11.6
x
1
20 26 41 55 60 67 75 79 70 55 45 33
x
2
23 21 24 25 24 26 25 25 24 25 25 23
y 210 206 260 244 271 285 270 265 234 241 258 230
Linear Models and Linear Regression
355
C
1 x
11
x
21
x
m1
1 x
12
x
22
x
m2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1 x
1n
x
2n
x
mn
2
6
6
6
4
3
7
7
7
5
; y
y
1
y
2
.
.
.
y
n
2
6
6
6
4
3
7
7
7
5
; e
e
1
e
2
.
.
.
e
n
2
6
6
6
4
3
7
7
7
5
;
q
0
1
.
.
.
m
2
6
6
6
4
3
7
7
7
5
;
y Cq e: 11:48
q
^
q q
^
q
C
T
C
1
C
T
y:11:49
TLFeBOOK
Answer: in this case, C is a 12 3 matrix and
We thus have, upon finding the inverse of C
T
C by using either matrix inversion
formulae or readily available matrix inversion computer programs,
or
The estimated regression equation based on the data is thus
Since Equation (11.48) is identical to its counterpart in the case of simple linear
regression, much of the results obtained therein concerning properties of least-
square estimators, confidence intervals, and hypotheses testing can be dupli-
cated here with, of course, due regard to the new definitions for matrix C and
vector .
Let us write estimator for in the form
We see immediately that
Hence, least-square estimator is again unbiased. It also follows from Equa-
356
Fundamentals of Probability and Statistics for Engineers
tion (11.21) that the covariance matrix for isgivenby
C
T
C
12 626 290
626 36; 776 15; 336
290 15; 336 7; 028
2
6
4
3
7
5
;
C
T
y
2; 974
159; 011
72; 166
2
6
4
3
7
5
:
^
q C
T
C
1
C
T
y
33:84
0:39
10:80
;
^
0
33:84;
^
1
0:39;
^
3
10:80:
d
Efyg
^
0
^
1
x
1
^
2
x
2
33:84 0:39x
1
10:80x
2
:
q
^
Q q
^
Q C
T
C
1
C
T
Y: 11:50
Ef
^
QgC
T
C
1
C
T
EfYgq: 11:51
^
Q
^
Q
covf
^
Qg
2
C
T
C
1
: 11:52
2
4
3
5
;
TLFeBOOK
Confidence intervals for the regression parameters in this case can also be
established following similar procedures employed in the case of simple linear
regression. Concerning hypotheses testing, it was mentioned in Section 11.1.5
that testing of simultaneous hypotheses is more appropriate in multiple linear
regression, and that we will not pursue it here.
11.3 OTHER REGRESSION MODELS
In science and engineering, one often finds it necessary to consider regression
models that are nonlinear in the independent variables. Common examples of
this class of models include
Polynomial models such as Equation (11.53) or Equation (11.55) are still
linear regression models in that they are linear in the unknown parameters
0
,
1
,
2
,..., [etc. Hence, they can be estimated by using multiple linear
regression techniques. Indeed, let x
1
x, and x
2
x
2
in Equation (11.53), it
takes the form of a multiple linear regression model with two independent
variables and can thus be analyzed as such. Similar equivalence can be estab-
lished between Equation (11.55) and a multiple linear regression model with
five independent variables.
Consider the exponential model given by Equation (11.54). Taking logar-
ithms of both sides, we have
In terms of random variable ln Y , Equation (11.57) represents a linear regres-
sion equation with regression coefficients ln
0
and
1
. Linear regression tech-
niques again apply in this case. Equation (11.56), however, cannot be conveniently
put into a linear regression form.
Ex ample 11. 7. Problem: on average, the rate of population increase (Y ) asso-
ciated with a given city varies with x, the number of years after 1970. Assuming that
compute the least-square estimates for
0
,
1
,and
2
based on the data pre-
sented in Table 11.5.
Linear Models and Linear Regression 357
Y
0
1
x
2
x
2
E; 11:53
Y
0
exp
1
x E; 11:54
Y
0
1
x
1
2
x
2
11
x
2
1
22
x
2
2
12
x
1
x
2
E; 11:55
Y
0
x
1
E: 11:56
ln Y ln
0
1
x E: 11:57
EfY g
0
1
x
2
x
2
;
TLFeBOOK
Answer: let x
1
x, x
2
x
2
,andlet
The least-square estimate for is given by Equation (11.49), with
and
Thus
or
Table 11. 5 Population increase, y, with
number of years after 1970, x, for Example 11.7
x 012345
y
(%) 1.03 1.32 1.57 1.75 1.83 2.33
358 Fundamentals of Probability and Statistics for Engineers
q
0
1
2
2
4
3
5
:
q,
^
q,
C
10 0
11 1
12 4
13 9
1416
1525
2
6
6
6
6
6
6
4
3
7
7
7
7
7
7
5
y
1:03
1:32
1:57
1:75
1:83
2:33
2
6
6
6
6
6
6
4
3
7
7
7
7
7
7
5
^
q C
T
C
1
C
T
y
61555
15 55 225
55 225 979
2
6
4
3
7
5
1
9:83
28:68
110:88
2
6
4
3
7
5
1:07
0:20
0:01
2
6
4
3
7
5
;
^
0
1:07;
^
1
0:20; and
^
2
0:01:
TLFeBOOK
Let us note in this example that, since x
2
x
2
1
, matrix C is constrained in that
its elements in the third column are the squared values of their corresponding
elements in the second column. It needs to be cautioned that, for high-order
polynomial regression models, constraints of this type may render matrix C
T
C
ill-conditioned and lead to matrix-inversion difficulties.
REFERENCE
Rao, C.R., 1965, Linear Statistical Inference and Its
Applications
, John Wiley & Sons
Inc., New York.
FURTHER READING
Some additional useful references on regression analysis are given below.
Anderson, R.L., and Bancroft, T.A., 1952, Statistical Theory in
Research,
McGraw-Hill,
New York.
Bendat, J.S., and Piersol, A.G., 1966, Measurement and Analysis of Random Data, John
Wiley & Sons Inc., New York.
Draper, N., and Smith, H., 1966, Applied Regression Analysis, John Wiley & Sons Inc.,
New York.
Graybill, F.A., 1961, An Introduction to Linear Statistical Models, Volume 1 . McGraw-
Hill, New York.
PROBLEMS
11.1 A special case of simple linear regression is given by
Determine:
(a) The least-square estimator ;
(b) The mean and variance of
(c) An unbiased estimator for
2
, the variance of Y .
11.2 In simple linear regression, show that the maximum likelihood estimators for and
are identical to their least-square estimators when Y is normally distributed.
11.3 Determine the maximum likelihood estimator for variance
2
of Y in simple linear
regression assuming that Y is normally distributed. Is it a biased estimator?
11.4 Since data quality is generally not uniform among data points, it is sometimes
desirable to estimate the regression coefficients by minimizing the sum of weighted
squared residuals; that is, and in simple linear regression are found by minimizing
Linear Models and Linear Regression
359
Y x E:
^
B
^
B
^
^
X
n
i1
w
i
e
2
i
;
for
;
TLFeBOOK
where w
i
are assigned weights. In vector–matrix notation, show that estimates
and now take the form
where
11.5 (a) In simple linear regression [Equation (11.4)]. use vector–matrix notation and
show that the unbiased estimator for
2
given by Equation (11.33) can be
written in the form
(b) In multiple linear regression [Equation (11.46)], show that an unbiased esti-
mator for
2
is given by
11.6 Given the data in Table 11.6:
(a) Determine the least-square estimates of and in the linear regression
equation
(b) Determine an unbiased estimate of
2
, the variance of Y .
(c) Estimate E Y at x 5.
(d) Determine a 95% confidence interval for .
(e) Determine a 95% confidence band for x.
11.7 In transportation studies, it is assumed that, on average, peak vehicle noise level
(Y ) is linearly related to the logarithm of vehicle speed (v). Some measurements
taken for a class of light vehicles are given in Table 11.7. Assuming that
Table 11.6 Data for Problem 11.6
x0123456789
y 3.2 3.1 3.9 4.7 4.3 4.4 4.8 5.3 5.9 6.0
360 Fundamentals of Probability and Statistics for Engineers
^
^
^
q
^
^
C
T
WC
1
C
T
Wy;
W
w
1
0
w
2
.
.
.
0 w
n
2
6
6
6
4
3
7
7
7
5
:
c
2
1
n 2
Y C
^
Q
T
Y C
^
Q:
c
2
1
n m 1
Y C
^
Q
T
Y C
^
Q:
Y x E:
f g
Y log
10
v E;
TLFeBOOK
determine the estimated regression line for Y as a function of log
10
v.
11.8 An experimental study of nasal deposition of particles was carried out and
showed a linear relationship between E Y and ln d
2
f , where Y is the fraction
of particles of aerodynamic diameter, d (in mm), that is deposited in the nose
during an inhalation of f(l min
1
). Consider the data given in Table 11.8 (four
readings are taken at each value of ln d
2
f). Estimate the regression parameters in
the linear regression equation
and estimate
2
, the variance of Y .
11.9 For a study of the stress–strain history of soft biological tissues, experimental
results relating dynamic moduli of aorta (D) to stress frequency ( ) are given in
Table 11.9.
(a) Assuming that E D , and
, estimate regression coefficients
and .
(b) Determine a one-sided 95% confidence interval for the variance of D.
(c) Test if the slope estimate is significantly different from zero at the 5%
significance level.
11.10 Given the data in Table 11.10
(a) Determine the least-square estimates of
,and
2
assuming that
Table 11. 7 Noise level, y (in dB) with vehicle
speed, v (in km h
1
), for Problem 11.7
v2030405060708090100
y5563687072787476 79
Table 11. 8 Fraction of particles inhaled of diameter d (in mm), with ln d
2
f
(f is inhalation, in min
1
), for Problem 11.8
ln d
2
f 1.6 1.7 2.0 2.8 3.0 3.0 3.6
y 0.39 0.41 0.42 0.61 0.83 0.79 0.98
0.30 0.28 0.34 0.51 0.79 0.69 0.88
0.21 0.20 0.22 0.47 0.70 0.63 0.87
0.12 0.10 0.18 0.39 0.61 0.59 0.83
Table 11.9 The dynamic modulus of aorta, d (normalized) with frequency,
(in Hz), for Problem 11.9
12345678910
1.60 1.51 1.40 1.57 1.60 1.59 1.80 1.59 1.82 1.59
Linear Models and Linear Regression
361
f g
EfYg ln d
2
f ;
!
f g !
2
D
2
,
0
,
1
,
EfYg
0
1
x
1
2
x
2
:
l
!
!
d
TLFeBOOK
(b) Estimate
11.11 In Problem 11.7, when vehicle weight is taken into account, we have the multiple
linear regression equation
where w is vehicle unladen weight in Mg. Use the data given in Table 11.11 and
estimate the regression parameters in this case.
11.12 Given the data in Table 11.12:
(a) Determine the least-square estimates of
0
,
1
,and
2
assuming that
(b) Estimate
11.13 A large number of socioeconomic variables are important to account for mortal-
ity rate. Assuming a multiple linear regression model, one version of the model for
mortality rate (Y ) is expressed by
where
x
1
mean annual precipitation in inches,
x
2
education in terms of median school years completed for those over 25 years
old
x
3
percentage of area population that is nonwhite,
x
4
relative pollution potential of SO
2
(sulfur dioxide).
Table 11.10 Data for Problem 11.10
x
1
1 1112233
x
2
1 2345678
y 2.0 3.1 4.8 4.9 5.4 6.8 6.9 7.5
Table 11.11 Noise level, y (in dB), with vehicle weight (unladen,
in Mg) and vehicle speed (in km h
1
), for Problem 11.11
v20406080100120
w 1.0 1.0 1.7 3.0 1.0 0.7
y54597891 78 67
Table 11.12 Data for Problem 11.12
x01234567
y 3.2 2.8 5.1 7.3 7.6 5.9 4.1 1.8
362 Fundamentals of Probability and Statistics for Engineers
EfYg at x
1
x
2
2.
Y
0
1
log
10
v
2
log
10
w E;
EfYg
0
1
x
2
x
2
:
EfYg at x 3.
Y
0
1
x
1
2
x
2
3
x
3
4
x
4
E;
TLFeBOOK
Some available data are presented in Table 11.13. Determine the least-square
estimate of the regression parameters.
Table 11.13 Data for Problem 11.13
x
1
13 11 21 30 35 27 27 40
x
2
9 10.5 11 10 9 12.3 9 9
x
3
1.5 7 21 27 30 6 27 33
x
4
4216467172882101
y 795 841 820 1050 1010 970 980 1090
Linear Models and Linear Regression
363
TLFeBOOK