Of Notes
Surprisal Self Information
Table of Contents
This is a numerical measure of how surprising some observation is, introduced by Claude Shannon as self-information.
Naturally, the less probable some event is, the more surprising it is.
We measure this information in bits and define:
Origin of the Logarithm
The log property of
From which we can apply the log rule and say:
We introduce the negative since