Of Notes
Bayesian Rule & Inference
Table of Contents
Everything is governed by Thomas Bayes' Rule:
In inference we think about
This can be thought about simplified as:
Prior -
this is what you believe about A before you see B, this could be that "A person has a disease" which occurs naturally with
Likelihood -
Now our observation
which means that 99% of the people who have the disease and take the test, test positive.
Evidence -
Now we need to understand all the cases in which we would observe the given evidence or outcomes on which we are conditioning - when B would be Positive
In case of disease we need to account for:
| Sick |
Healthy |
|
| Positive |
0.99 | 0.05 |
this gives us that:
To compute
Posterior
The fundamental idea of a posterior is that it is a combination of the prior and likelihood.
Now we can calculate the posterior probability of
Derivation of Bayes Rule
We start with simple conditional probability:
You can rearrange either formulation to obtain the joint probability and plug that into the other.
And boom.
Inference
When we want to use this to infer something we write it as
where
After flipping 10 times we get
Once we have observed
since the evidence which we do not include in the above formulation must encompass all possible variations of some continuous parameter
A much more in depth explanation and development can be found @ Bayesian Inference.
Decision Theory
We can extend this to deciding what to do next given some data
where
and with this expected calculation we can wrap that into a minimization step:
This translates well into Losses.