Of Notes
Large Deviations Theory & Rate Functions
Table of Contents
This is part of the topic of Probability Theory > concentration of measure > large deviations > rate functions.
We study how collecting more data (numbers), leads to changes in the average measure. In large deviation theory specifically we ask, if the
Rate Functions
This is how we measure this oddity of that outcome. Say we have some average of a random process
To get to how this equation came to be we start with a coin
which for a fair coin
But imagine outcomes such as:
The farther we deviate from the expected natural outcome the more cost we generate.
which tells us that
Now if we know that
With a non-zero rate function and a growing
- NOTE: a rate function can also have two minima or more.
KL Divergence
We will often measure