Of Notes

Maximum Likelihood

1 min read · revised 15 October 2026 · referred to by 4 notes · ·

Table of Contents

This is a form of Frequentist Statistical Inference of which the result might also be a Biased Estimator in certain cases.

The general idea is to take some observations (and finding parameters of a distribution to align the two) and constructing a joint distribution in shape of:

Which we take the log of:

And then optimize to find the maximum by doing:

Binary Distributions Case

We observe some dataset which is made up of observations from a Bernoulli Distribution for example.

We then define a likelihood of having observed this dataset which is given by the probability of each observation given some :

Now we have a function which can learn from our data and we can optimize over one parameter :

And if the polynomial inside is continuous we can differentiate naively and set to obtain the parameter which would optimize our problem. This is called the Maximum Likelihood Estimation:

This is however very naive and "over-fitted" since this model would predict that the only plausible future is heads.

/ or Ctrl K
search; empty, it shows notes to revise and lately read
m
mark this note to revise, or unmark it
c
cover maths, proofs and solutions to test yourself
n
uncover the next covered item
y
copy this note’s org-roam link
e
copy this note as markdown, LaTeX kept as TeX
p
the next problem to work; there h hints, s shows the solution, x passes it for today
→ ← j k space
turn the page, when reading in pages
l
on the map, trace the greatest distance, then the longest path
double-click
a formula, to copy its TeX
rest on maths
bind a part of a formula and open the wheel of notes it comes from; scroll or ↑ ↓ widens it, 1–6 goes
?
this card