Of Notes
Full Bayesian Inference
In order to prevent overfitting we want to each out statistical model to be a little skeptical aswell. We started with MLE and then fixed some over confidence wtih MAP which still creates a point estimate and discards the rest out outcomes.
In this full bayesian estimation we skip the step of reducing some belief distribution into a single number and instead keep it fully.
When we actually want to use our posterior distribtion on some new
which if fully expanded
And we call this marginalization.
We call it marginalization because we are removing some variable from a probability distribution by summing or integrating over all the possible values it has. The name comes from probability tables, where these sums were traditionally written in the margins.