Of Notes

Richard Leibler

1 min read · revised 16 October 2026 · referred to by 2 notes · ·

American mathematician and cryptanalyst of the NSA generation, less famous than his name is. In 1951, with Solomon Kullback, he wrote "On Information and Sufficiency", the paper that defined the divergence the whole field now calls the KL Divergence — a measure of how badly the wrong distribution explains data drawn from the true .

  • The work note builds it directly from the Shannon Entropy machinery: expand the log, subtract, and the cross-terms collapse.
/ or Ctrl K
search; empty, it shows notes to revise and lately read
m
mark this note to revise, or unmark it
c
cover maths, proofs and solutions to test yourself
n
uncover the next covered item
y
copy this note’s org-roam link
e
copy this note as markdown, LaTeX kept as TeX
p
the next problem to work; there h hints, s shows the solution, x passes it for today
→ ← j k space
turn the page, when reading in pages
l
on the map, trace the greatest distance, then the longest path
double-click
a formula, to copy its TeX
rest on maths
bind a part of a formula and open the wheel of notes it comes from; scroll or ↑ ↓ widens it, 1–6 goes
?
this card