Of Notes
Maximum A Posteriori
Table of Contents
To fix the overfitting problem of MLE we introduce the Bayes Theorem and conjugate prior. Once we have observed some data likelihood we define some prior.
Taking the product of these two yields the posterior which we can then take the log of, find derivative
For a binomial case:
Binomial Derivation
The likelihood is binomial and the prior is a Beta distribution, so multiplying the two together gives:
Collecting the powers of
Taking the log turns the product into a sum, with everything constant in
To maximise
Putting the right-hand side over the common denominator
A fraction is zero only when its numerator is, so:
Expanding both products:
The
Factoring
which rearranges to:
Leaving the MAP estimate: