Of Notes

Full Bayesian Inference

1 min read · revised 28 October 2026 · referred to by 2 notes · ·

In order to prevent overfitting we want to each out statistical model to be a little skeptical aswell. We started with MLE and then fixed some over confidence wtih MAP which still creates a point estimate and discards the rest out outcomes.

In this full bayesian estimation we skip the step of reducing some belief distribution into a single number and instead keep it fully.

When we actually want to use our posterior distribtion on some new we construct a posterior predictive distribution in the shape of:

which if fully expanded

And we call this marginalization.

We call it marginalization because we are removing some variable from a probability distribution by summing or integrating over all the possible values it has. The name comes from probability tables, where these sums were traditionally written in the margins.

/ or Ctrl K
search; empty, it shows notes to revise and lately read
m
mark this note to revise, or unmark it
c
cover maths, proofs and solutions to test yourself
n
uncover the next covered item
y
copy this note’s org-roam link
e
copy this note as markdown, LaTeX kept as TeX
p
the next problem to work; there h hints, s shows the solution, x passes it for today
→ ← j k space
turn the page, when reading in pages
l
on the map, trace the greatest distance, then the longest path
double-click
a formula, to copy its TeX
rest on maths
bind a part of a formula and open the wheel of notes it comes from; scroll or ↑ ↓ widens it, 1–6 goes
?
this card