Of Notes

Maximum A Posteriori

1 min read · revised 17 October 2026 · referred to by 2 notes · ·

Table of Contents

To fix the overfitting problem of MLE we introduce the Bayes Theorem and conjugate prior. Once we have observed some data likelihood we define some prior.

Taking the product of these two yields the posterior which we can then take the log of, find derivative in order to maximize .

For a binomial case:

Binomial Derivation

The likelihood is binomial and the prior is a Beta distribution, so multiplying the two together gives:

Collecting the powers of and shows the posterior is another Beta:

Taking the log turns the product into a sum, with everything constant in absorbed into :

To maximise we differentiate and set the derivative to zero:

Putting the right-hand side over the common denominator :

A fraction is zero only when its numerator is, so:

Expanding both products:

The and cancel, and the two lone terms combine:

Factoring out of the remaining terms:

which rearranges to:

Leaving the MAP estimate:

/ or Ctrl K
search; empty, it shows notes to revise and lately read
m
mark this note to revise, or unmark it
c
cover maths, proofs and solutions to test yourself
n
uncover the next covered item
y
copy this note’s org-roam link
e
copy this note as markdown, LaTeX kept as TeX
p
the next problem to work; there h hints, s shows the solution, x passes it for today
→ ← j k space
turn the page, when reading in pages
l
on the map, trace the greatest distance, then the longest path
double-click
a formula, to copy its TeX
rest on maths
bind a part of a formula and open the wheel of notes it comes from; scroll or ↑ ↓ widens it, 1–6 goes
?
this card