Издательство John Wiley, 2006, -518 pp.
The concepts of uncertainty and information studied in this book are tightly interconnected. Uncertainty is viewed as a manifestation of some information deficiency, while information is viewed as the capacity to reduce uncertainty. Whenever these restricted notions of uncertainty and information may be confused with their other connotations, it is useful to refer to them as information- based uncertainty and uncertainty-based information, respectively. The restricted notion of uncertainty-based information does not cover the full scope of the concept of information. For example, it does not fully capture our common-sense conception of information in human communication and cognition or the algorithmic conception of information. However, it does play an important role in dealing with the various problems associated with systems, as I already recognized in the late 1970s. It is this role of uncertainty-based information that motivated me to study it.
One of the insights emerging from systems science is the recognition that scientific knowledge is organized, by and large, in terms of systems of various types. In general, systems are viewed as relations among states of some variables. In each system, the relation is utilized, in a given purposeful way, for determining unknown states of some variables on the basis of known states of other variables. Systems may be constructed for various purposes, such as prediction, retrodiction, diagnosis, prescription, planning, and control. Unless the predictions, retrodictions, diagnoses, and so forth made by the system are unique, which is a rather rare case, we need to deal with predictive uncertainty, retrodictive uncertainty, diagnostic uncertainty, and the like. This respective uncertainty must be properly incorporated into the mathematical formalization of the system.
In the early 1990s, I introduced a research program under the name generalized information theory (GIT), whose objective is to study information-based uncertainty and uncertainty-based information in all their manifestations. This research program, motivated primarily by some fundamental issues emerging from the study of complex systems, was intended to expand classical information theory based on probability. As is well known, the latter emerged in 1948, when Claude Shannon established his measure of probabilistic uncertainty and information.
GIT expands classical information theory in two dimensions. In one dimension, additive probability measures, which are inherent in classical information theory, are expanded to various types of nonadditive measures. In the other dimension, the formalized language of classical set theory, within which probability measures are formalized, is expanded to more expressive formalized languages that are based on fuzzy sets of various types. As in classical information theory, uncertainty is the primary concept in GIT, and information is defined in terms of uncertainty reduction.
Each uncertainty theory that is recognizable within the expanded framework is characterized by: (a) a particular formalized language (classical or fuzzy); and (b) a generalized measure of some particular type (additive or nonadditive). The number of possible uncertainty theories that are subsumed under the research program of GIT is thus equal to the product of the number of recognized formalized languages and the number of recognized types of generalized measures. This number has been growing quite rapidly. The full development of any of these uncertainty theories requires that issues at each of the following four levels be adequately addressed: (1) the theory must be formalized in terms of appropriate axioms; (2) a calculus of the theory must be developed by which this type of uncertainty can be properly manipulated; (3) a justifiable way of measuring the amount of uncertainty (predictive, diagnostic, etc.) in any situation formalizable in the theory must be found; and (4) various methodological aspects of the theory must be developed.
GIT, as an ongoing research program, offers us a steadily growing inventory of distinct uncertainty theories, some of which are covered in this book. Two complementary features of these theories are significant. One is their great and steadily growing diversity. The other is their unity, which is manifested by properties that are invariant across the whole spectrum of uncertainty theories or, at least, within some broad classes of these theories. The growing diversity of uncertainty theories makes it increasingly more realistic to find a theory whose assumptions are in harmony with each given application. Their unity allows us to work with all available theories as a whole, and to move from one theory to another as needed.
The principal aim of this book is to provide the reader with a comprehensive and in-depth overview of the two-dimensional framework by which the research in GIT has been guided, and to present the main results that have been obtained by this research. Also covered are the main features of two classical information theories. One of them covered in Chapter 3, is based on the concept of probability. This classical theory is well known and is extensively covered in the literature. The other one, covered in Chapter 2, is based on the dual concepts of possibility and necessity. This classical theory is older and more fundamental, but it is considerably less visible and has often been incorrectly dismissed in the literature as a special case of the probability-based information theory. These two classical information theories, which are formally incomparable, are the roots from which distinct generalizations are obtained.
Introduction
Classical Possibility-Based Uncertainty Theory
Classical Probability-Based Uncertainty Theory
Generalized Measures and Imprecise Probabilities
Special Theories of Imprecise Probabilities
Measures of Uncertainty and Information
Fuzzy Set Theory
Fuzzification of Uncertainty Theories
Methodological Issues
Conclusions
A Uniqueness of the U-Uncertainty
B Uniqueness of Generalized Hartley Measure in the Dempster–Shafer Theory
C Correctness of Algorithm 6.1
D Proper Range of Generalized Shannon Entropy
E Maximum of GSa in Section 6.9
F Glossary of Key Concepts
G Glossary of Symbols
The concepts of uncertainty and information studied in this book are tightly interconnected. Uncertainty is viewed as a manifestation of some information deficiency, while information is viewed as the capacity to reduce uncertainty. Whenever these restricted notions of uncertainty and information may be confused with their other connotations, it is useful to refer to them as information- based uncertainty and uncertainty-based information, respectively. The restricted notion of uncertainty-based information does not cover the full scope of the concept of information. For example, it does not fully capture our common-sense conception of information in human communication and cognition or the algorithmic conception of information. However, it does play an important role in dealing with the various problems associated with systems, as I already recognized in the late 1970s. It is this role of uncertainty-based information that motivated me to study it.
One of the insights emerging from systems science is the recognition that scientific knowledge is organized, by and large, in terms of systems of various types. In general, systems are viewed as relations among states of some variables. In each system, the relation is utilized, in a given purposeful way, for determining unknown states of some variables on the basis of known states of other variables. Systems may be constructed for various purposes, such as prediction, retrodiction, diagnosis, prescription, planning, and control. Unless the predictions, retrodictions, diagnoses, and so forth made by the system are unique, which is a rather rare case, we need to deal with predictive uncertainty, retrodictive uncertainty, diagnostic uncertainty, and the like. This respective uncertainty must be properly incorporated into the mathematical formalization of the system.
In the early 1990s, I introduced a research program under the name generalized information theory (GIT), whose objective is to study information-based uncertainty and uncertainty-based information in all their manifestations. This research program, motivated primarily by some fundamental issues emerging from the study of complex systems, was intended to expand classical information theory based on probability. As is well known, the latter emerged in 1948, when Claude Shannon established his measure of probabilistic uncertainty and information.
GIT expands classical information theory in two dimensions. In one dimension, additive probability measures, which are inherent in classical information theory, are expanded to various types of nonadditive measures. In the other dimension, the formalized language of classical set theory, within which probability measures are formalized, is expanded to more expressive formalized languages that are based on fuzzy sets of various types. As in classical information theory, uncertainty is the primary concept in GIT, and information is defined in terms of uncertainty reduction.
Each uncertainty theory that is recognizable within the expanded framework is characterized by: (a) a particular formalized language (classical or fuzzy); and (b) a generalized measure of some particular type (additive or nonadditive). The number of possible uncertainty theories that are subsumed under the research program of GIT is thus equal to the product of the number of recognized formalized languages and the number of recognized types of generalized measures. This number has been growing quite rapidly. The full development of any of these uncertainty theories requires that issues at each of the following four levels be adequately addressed: (1) the theory must be formalized in terms of appropriate axioms; (2) a calculus of the theory must be developed by which this type of uncertainty can be properly manipulated; (3) a justifiable way of measuring the amount of uncertainty (predictive, diagnostic, etc.) in any situation formalizable in the theory must be found; and (4) various methodological aspects of the theory must be developed.
GIT, as an ongoing research program, offers us a steadily growing inventory of distinct uncertainty theories, some of which are covered in this book. Two complementary features of these theories are significant. One is their great and steadily growing diversity. The other is their unity, which is manifested by properties that are invariant across the whole spectrum of uncertainty theories or, at least, within some broad classes of these theories. The growing diversity of uncertainty theories makes it increasingly more realistic to find a theory whose assumptions are in harmony with each given application. Their unity allows us to work with all available theories as a whole, and to move from one theory to another as needed.
The principal aim of this book is to provide the reader with a comprehensive and in-depth overview of the two-dimensional framework by which the research in GIT has been guided, and to present the main results that have been obtained by this research. Also covered are the main features of two classical information theories. One of them covered in Chapter 3, is based on the concept of probability. This classical theory is well known and is extensively covered in the literature. The other one, covered in Chapter 2, is based on the dual concepts of possibility and necessity. This classical theory is older and more fundamental, but it is considerably less visible and has often been incorrectly dismissed in the literature as a special case of the probability-based information theory. These two classical information theories, which are formally incomparable, are the roots from which distinct generalizations are obtained.
Introduction
Classical Possibility-Based Uncertainty Theory
Classical Probability-Based Uncertainty Theory
Generalized Measures and Imprecise Probabilities
Special Theories of Imprecise Probabilities
Measures of Uncertainty and Information
Fuzzy Set Theory
Fuzzification of Uncertainty Theories
Methodological Issues
Conclusions
A Uniqueness of the U-Uncertainty
B Uniqueness of Generalized Hartley Measure in the Dempster–Shafer Theory
C Correctness of Algorithm 6.1
D Proper Range of Generalized Shannon Entropy
E Maximum of GSa in Section 6.9
F Glossary of Key Concepts
G Glossary of Symbols