INDEX 737
high probability set, 62
histogram, 174
historical notes, xv
HMM, see hidden Markov model (HMM)
Hochwald, B.M., 693
Hocquenghem, P.A., 214, 703
Holsinger, J.L., 704
Honig, M.L., 704
Hopcroft, J.E., 704
Horibe, Y., 704
horse race, 5, 6, 11, 159–182, 622, 626
Huffman code, 103, 118–127, 129–131,
137, 142, 145, 146, 149, 151, 155,
157, 357, 427, 436, 460, 491, 492
competitive optimality, 158
dyadic distribution, 151
Huffman, D.A., 158, 704
Hui, J.Y., 704
Humblet, P.A., 704
hypothesis testing, 1, 4, 11, 355, 375, 380,
384, 389
Bayesian, 384
optimal, see Neyman-Pearson lemma
i.i.d. (independent and identically
distributed) source, 307, 318, 344, 357
identification capacity, 610
Ihara, S., 704
image, 305
distortion measure, 305
entropy rate, 171
Kolmogorov complexity, 499, 505, 506
Immink, K.A.S., 704
incompressible sequence, 477, 479
independence bound on entropy, 31
India, 441
indicator function, 194, 219, 486, 497, 503
induction, 95, 123, 127, 674
inequalities, xviii–xx, 53, 207, 418,
657–687
inequality,
arithmetic mean geometric mean, 669
Brunn-Minkowski,
see Brunn-Minkowski inequality
Cauchy-Schwarz, 393
Chebyshev’s, 64
data processing, see data processing
inequality
determinant, see determinant inequalities
entropy power, see entropy power
inequality
Fano’s, see Fano’s inequality
Hadamard’s, see Hadamard’s inequality
information, 29, 410, 659
Jensen’s, see Jensen’s inequality
Kraft, see Kraft inequality
log sum, see log sum inequality
Markov’s, see Markov’s inequality
McMillan’s, see McMillan’s inequality
subset, see subset inequalities
Young’s, 676
Ziv’s, 450
inference, 1, 3, 4, 463, 484
infinite bandwidth, 273
infinitely often, 621
information, see also Fisher information,
mutual information, self information
information capacity, 207, 263, 274, 277
information channel capacity, 184
information divergence, 55, see
also relative entropy
information for discrimination, 55, see
also relative entropy
information rate distortion function, 306,
307, 329
innovations, 282
input alphabet, 183, 209, 268
input distribution, 188, 227, 228, 278, 335,
430, 431, 532, 544, 546, 591
instantaneous code, see code, instantaneous
integer,
binary representation, 469
descriptive complexity, 469
integrability, 248
interference, xix, 3, 11, 273, 509, 511, 515,
518, 519, 527, 547, 588, 610
interference channel, 510, 518, 519, 610
degraded, 610
Gaussian, 518, 519, 610
high interference, 518
strong interference, 610
interleaving, 611
internet, 218
intersymbol interference, 94
intrinsic complexity, 464
investment, 4, 9, 11, 159, 614, 619, 623,
636, 655, 656
investor, 619, 623, 627, 629, 633, 635