Keyfitz N., Caswell H. Applied Mathematical Demography
Подождите немного. Документ загружается.
12.6. The Components of Forecasting Error Ex Ante 295
pattern of a series of model tables. If each time such a decision is made
one or more other reasonable possibilities are also followed through, a large
number of combinations of these alternatives will be available. Extrapola-
tion from short versus long experience of deaths, and from separate ages
versus all ages together following a series of model tables, and similarly for
births, alone gives 16 combinations and therefore 16 forecasts. The variation
among these provides an ex ante estimate of error.
Alternatively, if the calculation is more intuitive than these mechanical
extrapolations, and a 2 percent per year decline in fertility is proposed,
the estimator can ask himself whether 1 percent and 3 percent are also
reasonable, or whether he would bet 19 to 1 odds that the range is limited
to 1
1
2
to 2
1
2
percent. Similarly in regard to mortality and migration. The
aggregation of these sources of error provides upper and lower bounds for
the future population. The aggregation may be based on the assumption
that the several errors act independently, that is to say, their squares are
added, and at the end the square root taken, or more conservatively the
errors may themselves be added.
12.6.1 The Length of the Experience Base
Every forecast works from some period in the past. As a minimum it takes
the population of an initial year as its jumping-off point and assumes that
the age-specific rates of birth, death, and migration of the initial year re-
main constant into the future. To derive a trend in such age-specific rates
requires at least 2 past years; better estimates of trend can be obtained
from 5, 10, or more years, using least squares or other fitting. The longer
base will be of value also in providing an estimate of error of the fit. If the
past were homogeneous, we could never have too much of it, and would
incorporate the whole available statistical record in the forecast.
Since we always suspect heterogeneity in the past, under what circum-
stances is it desirable to use a long experience base and when is a short
base to be preferred (Anderson 1971)? If sudden or sharp changes occur
from time to time, or a gradual drift in relationships is taking place, the
experience base should be confined to the most recent time and be rela-
tively short, in the hope that the part of the past used is homogeneous with
the future. A prediction for the United States today would not be greatly
aided by what happened before World War I, even if we had good data for
that long past time.
The forecaster who would provide a distribution needs data even more
badly than the one content to produce a single figure. He is even more
tempted to go back in time to increase his stock, and also more concerned
about the danger that his results will be distorted by changes in the system
the further back he goes.
To ascertain what part of the past record is relevant to the future being
predicted is a subsidiary but not unimportant subject of investigation.
296 12. Projection and Forecasting
12.7 Ex Post Evaluation of Point Estimates
To check on the ex ante estimate of error of a particular forecasting ap-
proach, one can apply the approach to the past, noting, say, the differences
between the population in 1970 as predicted in 1935 and the observed 1970
population now available. Experiments on the experience base should lead
to improved estimates of future population and their variance.
After all justifiable use of the past has been made, concentrated on a
point estimate for a future moment, and after the moment arrives and a
census is taken, one would think that then, at least, we would be in a
position to judge the estimate unambiguously. But further problems arise;
surprisingly, even the possession of the actual population count, with which
the forecast made earlier can be compared to provide an ex post assessment
of its accuracy, leaves some residual difficulties. Statement of the accuracy
of prediction attained is not unambiguous even after the event, at least if
we want to compare two predictions made at different times for the same
future date, and more generally to compare estimates over different time
intervals and in different countries.
For all such purposes a benchmark of some kind is needed, an extension
of the “persistence” (forecasting that tomorrow’s weather will be the same
as today’s) that meteorologists use to see how well their elaborate methods
have served, and that Mosteller et al. (1949, p. 297) have applied to election
polls. The benchmark may be a naive estimate of the kind discussed above,
a standard form of projection by age and sex with fixed rates, or some
other, as will be suggested below.
The Scripps medium projection made in 1935 for the 1970 population of
the conterminous United States was 155 million. The actuality as counted
in the 1970 census was 203 million for the same area. Was 48 million a large
error? The estimate was 76 percent of the true figure, or 24 percent short,
but was this good or bad? If one could predict the national income 35 years
in advance, within 24 percent, he would be doing well; on the other hand,
24 percent error in predicting next year’s federal tax collections would be
decidedly poor. The only way to assess the quality of a point forecast is to
compare it with some standard way of forecasting.
If the quality of a prediction has plainly to be assessed in relation to
some other prediction produced by a standard method that can be taken
as a base or benchmark, it is much less obvious how this base should be
selected. One benchmark corresponds to persistence in weather forecasting:
assume that the population will not change from the 1935 level. The 1935
level was 127 million, so the true increase over the whole period from 1935
to 1970 was the difference between this and the realization: 203 −127 = 76
million. This was the error to which the use of the 1935 figure for 1970
would have been subject.
12.7. Ex Post Evaluation of Point Estimates 297
Somewhat more formally, the general expression for the quality of a
forecast as measured ex post is
Quality of forecast =
Forecast − Benchmark
Realization − Benchmark
. (12.7.1)
In effect, our first effort was to use a benchmark of zero, giving for 1970
Quality =
155 − 0
203 − 0
=0.76, (12.7.2)
the complement of which, 24 percent, is one way of describing the error.
The second effort described above, is to use the 1935 observed population
of 127 million as a benchmark, and we find, again for 1970,
Quality =
155 − 127
203 − 127
=
28
76
=0.37. (12.7.3)
The more precise benchmark provides a more rigorous evaluation, as though
we estimate not the 1970 population but the increase from 1935.
We now proceed to a yet tighter measure of quality. The forecast will
not be credited with the fraction of the increase from the jumping-off point
that it includes, but only the part of it not covered by a projection at 1935
rates of birth and death. Such a projection from 1935 gives 148 million for
1970, so we have
Quality =
Forecast − Projection at rates of 1935
Realization − Projection at rates of 1935
=
155 − 148
203 − 148
=
7
55
=0.13. (12.7.4)
This last measure of quality in effect recognizes that whoever made the
forecasts in 1935 knew the 1935 population, birth rates, and death rates.
He could have projected with these rates, but used his knowledge and
intuition to improve on them. We will credit him with good judgment only
to the degree that he bettered that fixed-rate forecast.
The same expression (12.7.1) can be applied to assess 1950 and 1960 as
well as 1970, and Table 12.6 shows the results. Note that (12.7.2) seems
to give worse results as one proceeds in time, and so does (12.7.3). Only
(12.7.4) constitutes a measure of quality that is virtually the same for 1950,
1960, and 1970.
Population 10 to 30 years ahead is better predicted than income, demand
for housing, interest rates, unemployment, prices, technology, weather,
earthquakes, or any other common subject of forecasting. But we can-
not take much satisfaction in this superiority. Our sophisticated techniques
are effective only in the degree to which they produce better results than
naive methods. Thus one direction of evaluation of population projections
is how they compare with simple straight-line extrapolation and other naive
methods (Agnew, 1972).
298 12. Projection and Forecasting
Table 12.6. Three retrospective or ex post evaluations of the 1935 Scripps Medium
Projection to 1950, 1960, and 1970
Departure from
Estimated increase fixed-rate
Forecast from 1935 as benchmark as
as percent percent of percent of
of census count census increase census departure
Year (8.7.2) (8.7.3) (8.7.4)
1950 94 61 12
1960 84 43 12
1970 76 37 13
Table 12.7. Annual rate of future increase implied by official United States
projections (percents)
Period CDE F
1972–1980 1.26 1.13 0.88 0.76
1972–1990 1.35 1.19 0.92 0.75
1972–2000 1.30 1.12 0.84 0.65
Source: Statistical Abstract of the
United States (1973, p. 6.)
The benchmark is a simple-minded, conventional, stylized standard. It
is intended to judge how good refined methods are, and to compare one
refined method with another. The refined methods can use intuitive ele-
ments that do not have to be justified; the benchmark to judge them must
be entirely objective, so that it can be constructed after the event, the
predictand, is known. If the benchmark is made too good, the forecasts be-
ing judged by it look too bad. Some correspondents have argued that the
benchmark ought to be nothing but the actual population at the jumping-
off point, but this seems to let the forecaster off too lightly, since anyone
can improve on such a benchmark. In principle the benchmark should in-
corporate the minimum knowledge that the forecaster should have taken
into account, and so it records how much more than the minimum skill he
possessed.
12.7.1 Future Percentage Increase
To regard projections in terms of the percentage increase they imply ac-
cords with the wider range of uncertainty in prospective population as we
go forward in time. Table 12.7 shows the angle of climb of the curve ex-
pressed in terms of the average percentage increase from 1972, implied by
the four official projections, C, D, E,andF , for the United States.
12.8. A Division of Labor 299
The differences between 1972–80, 1972–90, and 1972–2000 are small com-
pared with the differences between projections C, D, E,andF , and one
can summarize the whole table by its bottom line, which says that the pro-
jected rate of increase is between 0.65 and 1.30 percent. For the increase of
the whole period the C estimate comes out just double the F estimate.
12.8 A Division of Labor
Who ought to estimate future population? Is it true that in a free-enterprise
society the risks of decision are taken by the entrepreneur, that these risks
include the danger of being wrong regarding the future population, and that
if we leave things alone there will be a natural selection of entrepreneurs
favoring those who have, among other talents, a superior ability to forecast
population (or who can hire better demographers)? In this scheme there is
no room for official forecasting of population.
Yet such an austere view has never been held in regard to knowledge
in general, whose officially sponsored diffusion is, rather, part of the in-
frastructure on which a free-enterprise system functions. If there is some
information regarding the future that can serve everyone and improve many
decisions, then let it be assembled, published, and paid for by the public
authorities. This is so notwithstanding the fact that mistakes are made in
forecasting the future; the test is whether forecasts add more to the quality
of decisions than is spent in making them.
Among other difficulties, all users do not want the same forecast. One
user needs a conservative estimate of the future—a figure that she is rea-
sonably sure will be exceeded. Another needs a high figure; he wants to
guard especially against any possibility that the estimate he uses will be
exceeded by the realization. How can we separate the task of forecasting
from the adaptation of the forecast to a particular use? The loss function
does this in a precise way.
12.8.1 The Loss Function Permits a Three-Way Division of
Labor
Only the user can know how much he stands to lose through a projection
being wrong in one direction or the other, or by a certain amount. His loss
function may be strongly asymmetric. Suppose that the future population
of a town is wanted for deciding the capacity of a new water reservoir.
If the reservoir turns out to be too small, another will soon have to be
built, and this will be expensive. If it turns out to be too large, it will be
underutilized, and the loss is only the marginal cost of the excess capacity,
which may be small.
300 12. Projection and Forecasting
The logic for arriving at exactly the right prediction ˆx for a particular
use is based on developments in statistics going back to the 1930s, and in
its application to demography is set forth by Muhsam (1956). Suppose that
the user has studied his own application enough to be able to judge how
much loss he would suffer with a particular departure of the estimate ˆx from
the true population x (not knowable until the future date has arrived), and
suppose that he can express this in the form of a loss function L(ˆx, x)or,
less generally but more simply, L(ˆx − x).
He is given the population forecast in the form of a probability distribu-
tion P (x) by the official agency mentioned above. Not being able to control
x, the user integrates it out to provide his total expected loss:
R(ˆx)=
+∞
−∞
L(ˆx − x)P (x) dx. (12.8.1)
It remains only to choose ˆx so as to minimize the total expected loss R(ˆx);
the value of ˆx required is the solution of the equation dR(ˆx)/dˆx =0,
provided that the functions P (x)andL(ˆx − x) are well behaved.
For a highly simplified numerical example, suppose the official estimate
to be discrete and to have the following form:
Forecast
x Probability
persons P (x)
50,000 0.25
60,000 0.50
70,000 0.25
Now suppose that the user’s loss function is as follows:
Forecast error Loss
ˆx − xL(ˆx − x)
−20,000 $2,000,000
−10,000 500,000
00
10,000 100,000
20,000 200,000
Thus, if he uses ˆx =50, 000 and the performance turns out to be x =
70, 000, he loses $2,000,000, and so on.
We can make a two-way table of loss, L(x − ˆx):
12.9. Interval Estimates as Currently Provided 301
x
ˆx
50,000 60,000 70,000
50,000 0 500,000 2,000,000
60,000
100,000 0 500,000
70,000
200,000 100,000 0
The table can be summarized by the sum
P (x)L(x−ˆx) for each possible
forecast ˆx:
Forecast Expected loss
ˆx
P (x)L(x − ˆx)
50,000 $750,000
60,000 150,000
70,000 100,000
The right forecast for the particular purpose is 70,000.
12.9 Interval Estimates as Currently Provided
Ex post we know in what fraction of cases the upper and lower projections
straddle the subsequent performance. If a collection of projections is in
any sense homogeneous, and the method currently in use is similar to the
techniques employed in the collection, the fraction of instances in which
the subsequent census was straddled constitutes a level of confidence for
the present method. A large collection of early projections is available to
be compared with subsequent actuality.
Such material may be used to test various hypotheses about the process
of projection as it has been practiced in the past. One of these is that,
after a steady movement of births and population in a consistent direction
(such as occurred from the 1870s to the 1930s), confidence increases, and
the range of admitted future possibilities narrows—the projection fan is
made smaller. When a sudden change (especially in the birth series) creates
uncertainty the fan is made very large. A second hypothesis is that series
that are abandoned may ultimately turn out unexpectedly well, perhaps
better than those that are continued. Third, the level of births anticipated
may be too sensitive to current birth rates; one could test the hypothesis
that waves in future population estimates correspond to waves in current
births.
302 12. Projection and Forecasting
12.9.1 Official Agencies Have Backed into Confidence
Intervals
What has happened as a result of the insistence on neutral projections
rather than forecasts, and on making several so that the user can choose
among them, is that the agencies involved have backed into confidence
intervals. It seems unlikely that any user does what the theory of projections
says she should do—examine the assumptions of the several sets, given more
or less completely in fine print in the introduction, use personal judgment
as to which is right, and only then look at the corresponding projection and
adopt it for use. A user frequently picks the series that seems to give the
right answer, and pays only secondary attention to the fine print describing
the assumptions. Insofar as she does this, the whole theory of projections is
subverted, for she could just as well choose among a set of random numbers.
But users do want the government estimates; they may pick a middle fig-
ure from the assortment presented, or a high or low figure according to their
need, and suppose that the assumptions made by the official demographers
represent the range of reasonable possibilities. This is to say that they are
using the published high and low series as defining a confidence interval.
Like the producer of the series, they do not formally attach a probability
to the range, but even brief experience shows them that there is a finite
chance that the performance can fall outside the range given.
In the 1970s, an informal estimate based on 30 cases suggested that the
confidence attachable to responsible official estimates was about two-thirds.
It seems likely that some such number has implicitly been playing the role
in projection that 0.95 plays in tests of statistical significance. Because the
instruments are much cruder, no one is interested in 95 percent limits for
forecasts; they are apparently too wide for planning purposes. But much
more research is needed, especially ex post studies that look back on earlier
projections and note what fraction turns out to straddle the population
performance.
The need for projections steadily increases, and the demand bears no
relation to accuracy or inaccuracy as demonstrated ex post. The Bureau of
the Census will always be required to make projections, independently of
how good these projections are in an absolute sense: the least bad forecasts
available are what people seek. It can be criticized, not for failing to predict
what subsequently occurs, but only for failure to use sound methods and to
take account of all relevant data (cf. Lee and Tuljapurkar 1994, Bongaarts
and Bulatao 2000).
13
Perturbation Analysis of Matrix
Models
The results of a demographic analysis are functions of the vital rates, and
through them of biological and environmental variables. For many reasons,
the vital rates might have been otherwise, or might be otherwise in the
future, and the results of the analysis would be different. This chapter
introduces methods for perturbation analysis—asking what would happen
to some dependent variable if one or more independent variables were to
change. Perturbation analysis is used in several contexts.
• Predicting the results of future changes in the vital rates. These could
result from natural changes in the environment, be imposed as a
management plan to protect an endangered species or control a pest,
or result from family planning or changes in healthcare.
• Accounting for past changes. Given data from several populations
that differ because of their environments, one may want to know how
much of the resulting change in, say, λ can be attributed to changes
in survival, how much to changes in reproduction, and so forth.
• Predicting the action of natural selection. Selection deals with per-
turbations; it favors changes in the phenotype that increase fitness.
Fitness is a demographic concept (the rate at which a genotype prop-
agates itself), so an essential part of the theory of natural selection
is understanding how changes in the phenotype affect λ.
• Designing sampling schemes. Errors in estimates of the vital rates
produce errors in the estimate of anything calculated from those rates,
for example, λ. The most important errors will be those in the rates
304 13. Perturbation Analysis of Matrix Models
to which λ is most sensitive; this information can be used to design
sampling procedures that focus on estimating the vital rates where
accuracy matters most.
A Note on Examples.
In this chapter we will use examples to show how calculations work and to
explore patterns that arise from different kinds of life cycles. Most of the
examples are based on the following matrices, which include age-classified,
stage-classified, and size-classified life cycles.
• The age-classified matrix for the human population of the United
States in 1965, given in Example 7.2.
• The stage-classified model for teasel (Dipsacus sylvestris). The life
cycle is shown in Figure 11.2 and the matrix is given in Example 11.3.
This is an example of a model in which, potentially, transitions are
possible between any two stages.
• A size-classified matrix for the desert tortoise (Gopherus agassizii), a
threatened species living in the southwestern U.S. (Doak et al. 1994).
The matrix follows the standard size-classified model (3.8.2), with
seven size classes:
A =
⎛
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎝
000001.30 1.98 2.57
0.716 0.567000000
00.149 0.56700000
000.149 0.6040000
0000.235 0.560 0 0 0
00000.225 0.678 0 0
000000.249 0.851 0
0000000.016 0.860
⎞
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎠
.
(13.0.1)
This is an example of a model in which the stages have a natural
ordering and individuals may stay in their stage or move to the next
stage; no other transitions are allowed.
The entries in this matrix are expressed in terms of lower-level vital
rates. Let P
i
= a
ii
be the probability of surviving and staying in
thesamesizeclass(stasis), and G
i
= a
i+1,i
be the probability of
surviving and growing to the next size class (growth). Then
P
i
= σ
i
(1 − γ
i
) (13.0.2)
G
i
= σ
i
γ
i
, (13.0.3)
where σ
i
is the survival probability and γ
i
the growth probability of
size class i (Caswell 1985).