σ
2
f(x)
h(X) ≤ log
√
2πeσ
2
,
[0, a]
f(x) f
0
(x)
[0, a]
log a − h(X) =
Z
a
0
f(x) log adx +
Z
a
0
f(x) log f(x)dx =
=
Z
a
0
f(x) log
f(x)
1/a
dx =
= L(f||f
0
).
¤
X
Y
I(X; Y ) =
Z
X
Z
Y
f(x, y) log
f(y|x)
f(y)
dxdy.
I(X; Y ) = I(Y ; X) = h(Y ) − h(Y |X) = h(X) − h(X|Y );
I(X; Y ) ≥ 0
X Y