I(X; Y ) = H(X)−H(X|Y ) = H(Y )−H(Y |X) = H(X)+H(Y )−H(XY ).
I(X; Y ) ≤ min {H(X), H(Y )}.
I(X; Y ) ≤ min {log |X|, log |Y |}.
I(X; Y ) ∩
p(x)
I(X; Y ) ∪
p(y|x)
(x, y) ∈ XY
H(X) ≤ log | X|
¤
XY Z = {(x, y, z), p(x, y, z)}. z ∈ Z
p(x, y|z) =
p(x, y, z)
p(z)
.